WASHINGTON--The Senate rejected President Reagan's nomination of Robert H. Bork to the Supreme Court by a 58-42 vote yesterday, ending a long and contentious debate over a judge alternately portrayed as a brilliant jurist and a dangerous extremist.

Bork, who had been nominated July 1 to replace retired Justice Lewis F. Powell, became the 26th man in history to fail to win Senate confirmation to the nation's highest court, only the sixth this century.

He became the 11th man defeated by a vote of the Senate. Others who failed to win confirmation had their names withdrawn, or the Senate chose not to act on their nominations.

The vote against Bork represents the largest negative total ever recorded for a Supreme Court nominee.

Clement F. Haynsworth's nomination in 1969 was defeated by a 55-45 vote. G. Harrold Carswell, the only other nominee defeated by a vote in a 100-member Senate, went down 51-45 in 1970.

Bork's defeat had been anticipated for weeks. A majority of members had announced their intention to vote against him even before the Senate this week staged more than 25 hours of debate over his qualifications.

His critics accused him of embracing too narrow views on constitutional protections for civil rights and individual liberties, calling him anti-civil rights, anti-labor, anti-consumer and an opponent of equal rights for women.

Senate leaders quickly called the president to inform him of the vote. Hours earlier, White House spokesman Marlin Fitzwater said, "We don't expect to have a [new] nominee today."

But White House Chief of Staff Howard H. Baker told reporters a new nomination could come by Monday.

"I think soon the president will focus on recommendations for a nomination," Baker said just before the Senate vote. "I would estimate that may be as early as Monday. But I also think the president will not make a decision ... until after there has been an opportunity to consult with the leadership and the relevant jurisdictional Republicans."