The poll found that 53% of respondents do not think the CDU/CSU would gain more support under a different chancellor.
About 41% said they believe the alliance would benefit from a change in leadership.
Even among CDU/CSU supporters, 42% said the conservatives would have better prospects with another chancellor from their ranks, while 56% disagreed.
The survey was conducted among 1,008 people.
Earlier, Bild and Stern reported that members of the Christian Democratic Union, or CDU, had discussed a possible replacement for Merz before the end of his term.
People close to Merz denied those reports.
Speculation about the chancellor’s possible resignation emerged amid reports that his popularity had fallen to a historic low.