Schulz faced anger within the SPD for his willingness to take the foreign ministry job, despite vowing never to serve in a Merkel-led cabinet - it would also have been a move that would have been both unpopular in the party and left it with a significant credibility problem.
But failure to secure votes for the new coalition would significantly undercut her political authority in the party and plunge the SDP deeper into crisis.
Many young and leftist members are against the renewed so-called Grand Coalition between the SPD and the Union.
Ziemiak said Merkel needed to make clear by the time of a CDU congress on February 26 how staffing in the government and party would look in future.
There was no sign of an imminent end to talks between Merkel's Union bloc and the Social Democrats early Wednesday, some 21 hours into what party leaders have said should be the final round of negotiations.
The SPD's traditional left-wing youth organization, Jusos, have opposed a coalition government with the Christian Democrats, and called for reform within the party. He reversed course, however, after Merkel's coalition talks with two smaller parties collapsed in November. Opponents of the political alliance argue that the party should head into the opposition to sharpen its profile and reconnect with voters.More news: Up to 6 inches of snowfall possible by Monday night
More news: PH gov't to seek other sellers if Canada refuses chopper deal
More news: George W. Bush Says There's 'Clear Evidence' Russia Meddled With the Election
The rejection of Mr Schulz, 62, completes his fall from grace.
"The only thing left, really, is remorse over how disrespectful we've become with one another in our dealings and how little someone's word still counts", Gabriel said. Gabriel himself surrendered the party chairmanship to Schulz in January 2017.
For the SPD, the backbiting went public when Gabriel, Germany's acting foreign minister, attacked the leadership for planning to remove him from his post - and implying that he'd been promised that he could continue as the country's top diplomat. He was officially elected by a party congress with 100 percent of the vote. However, three crushing state electoral defeats followed, damaging Schulz's campaign last summer irreparably.
The shock announcement comes just days after Merkel's conservatives and the SPD agreed to form a coalition government.
If a deal emerges, it won't bring an immediate end to the political limbo following Germany's September 24 election.