DES MOINES — (CNN) Ben Carson’s campaign is in crisis — and the finger-pointing is beginning.

Top players in Carson’s orbit are bickering in public and his unorthodox campaign structure is proving to be more of a liability than an asset to the political neophyte. As the retired neurosurgeon tries to revive his flailing presidential bid, he might need to turn his campaign slogan — heal, inspire, revive — inward.

At the center of the fighting is Armstrong Williams, Carson’s media-savvy business manager and longtime friend, who doesn’t have an official role in the campaign, and Carson’s official campaign leadership.

Williams has hit Carson’s campaign team for putting the candidate in situations that have embarrassed him, such as with the recent Republican Jewish Coalition speech where Carson repeatedly mispronounced the name of Palestinian group Hamas.

“His campaign has to do a better job at preparing him,” Williams said in an interview with CNN. “Don’t give him speeches at the last minute. It’s not only on him, it’s on them.”

Williams said he never wants to be sycophantic and he has often told Carson, “You have to be better than this,” but his criticism often finds its way into the news cycle.

“I don’t think he is always helpful,” Barry Bennett, Carson’s campaign manager, said of Williams’ public criticism of Carson. “You always defend your candidate.”

Doug Watts, Carson’s communications director, said Williams’ actions have rubbed the candidate the wrong way.

“He does present us with some problems being on the outside of the campaign, Watts said. “Dr. Carson has expressed a little frustration here and there.”

The infighting comes as Carson has fallen in the polls. He stood at 22% in mid-October, according to a CNN/ORC survey, dropping to a tie for third place at 14% in the same poll released last Friday.

A top fundraiser left earlier this month, citing a problem with Carson’s leadership team. Poor communication led to an embarrassing New York Times story that raised questions about Carson’s grasp of foreign policy. And a recent speech suggested that Carson, who as a young doctor obsessively prepared and studied, lacked fluency in international affairs.

Carson has dismissed the dip in polls, framing his decline as simply the nature of campaigns.

“Polls go up and down, when people were asking me when I was polling at the top spot was I excited, I said no, because polls go up and down, and I will say that now, I think you will see them continue to rise,” he said on CNN in an interview with Jake Tapper. “At this stage of the game, very few people can sustain the kinds of attacks I have and still be at this level, so that tells you that there is a strong base of support there.”

Carson is working to reverse the downward trend. He just released a plan for health care reform. He’s traveling to Africa for a three-country tour. And his campaign just announced the addition of another foreign policy adviser.

For Carson, professionalizing his campaign will mean becoming a better candidate and learning how to assert himself more as the head of a sprawling operation that often misfires.

“He is a pretty easygoing guy and he does leave the campaign up to us and he is really very compliant and trusting of us,” Watts said. “Sometimes we don’t meet up to the standard.”

Williams frames it differently.

“Dr. Carson is the leader, the campaign follows his lead, they are looking to him,” he said. “Whatever happens, it will reflect on him.”

Williams emerged this summer as the most visible spokesman for Carson, whom he has known for over two decades. As Carson climbed the top of the polls, Williams explained his appeal. And when Carson struggled after saying he would not back a Muslim-American president, Williams was there to double-down.

He has served as a kind of “Carson whisperer,” helping translate Carson’s worldview and vouch for his personal story. But that closeness — they speak several times a day — has also posed problems.

Dogged by questions about his Detroit childhood, Carson lashed out at the media, yet Williams essentially told him to stop whining and learn to deal with the scrutiny that comes with a White House bid.

The not-ready-for-the-big-leagues problem with the campaign was most recently on display at the Republican Jewish Coalition speech.

Carson, who normally speaks extemporaneously, spoke from a prepared text, rarely looking up at his audience. When he seemed to pronounce Hamas, as “hummus” at times, it launched a barrage of criticism and outright mockery.

Bennett brushed aside the impact of that appearance, saying that “less than 1% of America watched anything that had to do with the RJC speech.”

But those watching were Republican insiders who were looking for reasons to either dismiss Carson or give him another look.

“Poor Ben Carson. Someone should have told him how to pronounce Hamas,” tweeted Ari Fleischer, former press secretary to President George W. Bush. “He sounds like he’s not familiar with the group.”

Watts acknowledged it wasn’t the campaign’s best moment.

“He was well aware of the speech and the content and the development of the speech,” Watts said. “But clearly he could have used more time reading it over several times, given that he rarely delivers a prepared text. I will take responsibility for that.”

Williams has also fumbled in high-profile ways, botching a recent New York Times piece about Carson’s foreign policy by giving a reporter the name of Duane R. Clarridge, an infrequent adviser who then trashed Carson, implying the candidate doesn’t grasp the intricacies of world politics.

“I had no idea that Armstrong was talking to the Times,” Bennett said. “I had never heard of that guy.”

The campaign was completely blindsided by the story, later going on cleanup duty, which entailed Carson brushing back Williams.

“He has nothing to do with the campaign. Nothing,” Carson told reporters in Mobile, Alabama, last month, shortly after the story ran. “Armstrong can comment on his own behalf. He does not speak for the campaign at all. He does not speak for me. He speaks for himself.”

Careful to note how difficult it is to run a campaign at the presidential level when the candidate is a novice, Williams said he gets along with everyone in the campaign. He said Watts recently sent him an e-mail saying he was thankful Williams often had his back.

“I prefer to give people who run his operation as much leeway as possible and get out of their way,” Williams said. “I respect them and I want them to do their jobs without hanging over their heads.”

But he also suggested he could help right the ship.

“If I travel with him, there will be no gaffes if there is a press conference,” Williams said, adding that Carson’s press secretary Deana Bass would likely take cues from how he ran a recent press event in South Carolina.

Yet Williams acknowledges he hasn’t always been helpful as he tries to balance his role as a surrogate and tough-love friend.

“Chris Wallace, that hurt him, and then The New York Times and myself saying he was on a learning curve, that hurt him,” Williams said. “If I could do it over again, I would.”

Bennett said they “are operating a single campaign at this point.” A heavy focus on Iowa, where a top-three finish is a must, will bring Carson to a state he hasn’t had a steady presence in. The campaign boasts a heavy social media footprint, with Carson bypassing the traditional press to connect directly with his base; Carson has 5 million fans on Facebook. Bennett is focused on a pool of 103,000 people.

“If we can turn out 35,000 to go to caucus, we can win,” he said.

But turning out voters in Iowa takes discipline and organization, things the campaign and candidate have struggled with.

“It’s a growth process for all of us,” Williams said. “We are learning, we have been reflective to see what we need to do. You will see the difference.”

The-CNN-Wire ™ & © 2015 Cable News Network, Inc., a Time Warner Company. All rights reserved. (PHOTO: CNN)