Four Republicans broke ranks and joined all Democrats in the chamber to vote 216-214 against the measure.
Opponents of President Joe Biden blame Mr. Mayorkas for a surge in illegal immigration at the US frontier.
Border security is becoming a top political issue in the 2024 election.
Three Republican defectors, Ken Buck of Colorado, Tom McClintock of California and Mike Gallagher of Wisconsin, voted no on Tuesday evening.
A fourth, Blake Moore of Utah, switched his vote from yes to no, as a procedural maneuver.
Even if the House had passed the vote, Mr. Mayorkas was unlikely to have been found guilty by the Democratic-controlled Senate.
The impeachment proceedings were initiated by Marjorie Taylor Greene, a Georgia Republican, and after the vote she said she would try again.
“My colleagues who voted no, I think they’ll be hearing from their constituents,” she told reporters outside the Capitol.
House Speaker Mike Johnson’s spokesman, Raj Shah, posted on X that Republicans would try again to impeach Mr. Mayorkas “when we have the votes for passage”.
Several congressional Republicans said the setback would have no bearing on their separate impeachment investigation into President Biden.
Signs of dissent over the Mayorkas vote had surfaced among the party rank-and-file earlier in the day.
Mr. McClintock said on Tuesday morning he would vote against impeachment as the articles “fail to identify an impeachable crime that Mayorkas has committed” and “stretch and distort the Constitution”.
Impeachment, a process set out in the US Constitution, is the first step in removing a federal official for a high crime or misdemeanor.
It requires a simple majority in the House and a two-thirds majority in the Senate to succeed. Democrats currently have a slight majority in the upper chamber.
The last cabinet secretary to be impeached was Secretary of War William Belknap in 1876, though he resigned shortly before the vote.
A Department of Homeland Security spokeswoman said in a statement: “This baseless impeachment should never have moved forward; it faces bipartisan opposition and legal experts resoundingly say it is unconstitutional.
“Whether House Republicans are serious about border security, they should abandon these political games,” she added.
House Republicans held two hearings in January and charged Mr. Mayorkas with failing to enforce immigration policies and lying to lawmakers about whether the southern border was secure.
Before Tuesday’s vote, Democrat Jim McGovern of Massachusetts said Republicans wanted to “create chaos, they want to create confusion, and they want to create a campaign issue for Donald Trump going into the next election”.
But Republican Chip Roy of Texas argued that Mr. Mayorkas had failed to enforce US immigration law “in a way that has led directly to the death of American citizens” as well as migrants.
A January poll conducted by CBS – the BBC’s US partner – suggests that 63% of Americans want “tougher” border policies.
Over 6.3 million migrants are known to have entered the US illegally since Mr. Biden took office in 2021.
About 2.4 million were allowed into the US, where the majority wait for immigration court dates in which they can make a case for asylum. The system is so overwhelmed that this can take years.
This week, a bipartisan group of US senators announced a bill that would step up border enforcement efforts as well as provide additional aid for Ukraine and Israel.
House Republicans, however, have rejected the bill outright.
Immediately after Tuesday’s impeachment vote, the House considered a separate Republican bill that would have provided $17.6bn (£14bn) to Israel, but it, too, failed to pass.
pll/rgh/dfm
