WASHINGTON (CNN) — House Speaker Paul Ryan is moving aggressively to end speculation that he may be a late entry into the Republican presidential race, planning to pull himself out of consideration once and for all.

In remarks to be delivered later Tuesday at the Republican National Committee headquarters in Washington, Ryan plans to say he won’t be the GOP nominee under any circumstance.

“He’s going to rule himself out and put this to rest once and for all,” a Ryan aide told CNN.

The move comes as the chatter over a possible Ryan bid at a deadlocked convention is becoming a major distraction for the speaker. But Ryan aides believe even a late entry would not be feasible and would divide the party even further.

Ryan continues to stress that his main priority is developing an agenda for the eventual nominee to adopt, to show what Republicans would do to replace Obamacare, reform the tax code, address poverty and boost job creation.

Even before Ryan planned to address the issue on Tuesday, conservatives were warning that any move to install him as the party’s standard-bearer at the Republican convention in Cleveland in July would backfire.

“The one thing that would completely destroy the party is coming out (of Cleveland) with Paul Ryan as the nominee,” Republican South Carolina Rep. Mick Mulvaney told CNN on Monday.

Another conservative told CNN he opposed Ryan — or any other individual who hasn’t been campaigning — from getting the party’s nomination.

“To parachute someone in who hasn’t run for the office, who hasn’t done the grueling work of campaigning — if you want to do something that heightens cynicism to all-time high, that would do it,” Virginia Rep. Dave Brat told CNN on Monday.

Several House Republicans told CNN that Ryan has been clear with them that he has been trying to tamp down the speculation and he doesn’t want to run for president. They say he made a conscious decision when he decided to run for speaker last fall that he was removing himself from a White House bid.

Mulvaney said his concerns had nothing to do with Ryan personally, but because Republican activists would feel “something was taken from them unfairly and they would stay home.” He predicted 60% of the voters in his district wouldn’t vote in November if the convention handed the nomination to the House speaker.

House Republicans weren’t the only ones dismissing a potential Ryan bid.

“I think it’s simply nonsense that someone outside of the three people running for this nomination are going to be nominated at this convention,” Colorado Sen. Cory Gardner told CNN’s John Berman on “At This Hour.” “It’s just nonsense. One of the three will be our nominee, and I know there are some people who think there is going to be this through-the-looking-glass-moment for Paul Ryan or perhaps others, but it’s simply not going to happen.”

Democratic National Committee Chairwoman Debbie Wasserman Schultz also commented on Ryan’s announcement, which she said is no surprise given the contentious GOP nomination fight.

“I’m not at all surprised he doesn’t want to step in the way of this trainwreck,” Wasserman Schultz said.

The-CNN-Wire ™ & © 2016 Cable News Network, Inc., a Time Warner Company. All rights reserved. (Photo: CNN)