OTTAWA - Former MP Rahim Jaffer got tangled in a dispute over money with his Tory caucus colleagues last year, senior party sources told The Canadian Press.

And that falling-out worsened a relationship with the party that was already on the rocks after the 2008 election, making it highly implausible that Jaffer had special access to the halls of power, the insiders say.

The Toronto Star has reported that a business acquaintance of Jaffer bragged the former politician had access to Prime Minister Stephen Harper's office.

Conservative sources said Jaffer, former national caucus chair, had been arguing with some MPs over his management of a small fund that paid for snacks and social events for caucus members.

When Jaffer lost his Edmonton seat in the fall 2008 election, Ontario MP Guy Lauzon took over the fund -- and that's when the friction ensued.

Insiders said only a small sum of money was in dispute, perhaps a few hundred dollars, but the confrontation created bad feelings between Jaffer and some MPs.

The matter was kept confidential within a select group inside the caucus, as it did not involve public or party funds. And there is no suggestion of any impropriety by Jaffer, only that there was an active dispute over accounting.

One caucus member, who asked not to be identified, said it created more tension with the former MP, who was already in Harper's bad books following the 2008 election.

"Rahim offers no special influence over the PMO (Prime Minister's Office) whatsoever," said the MP.

"Anyone who knows the PM knows he is a completely different person from Rahim."

A spokesman for the Prime Minister's Office, Andrew McDougall, said any suggestion Jaffer holds special sway with Harper is "false" and "absurd."

A Conservative logo that had been featured on Jaffer's personal website disappeared following a request by the party Thursday.

Calls and messages to Jaffer and his lawyer were not answered Thursday. But Jaffer's business partner at the company Green Power Generation, Patrick Glemaud, issued a news release saying the company planned to "seek legal action against the Toronto Star."

"Mr. Jaffer is a valued partner of GPG. The allegations in the Toronto Star are inaccurate and a complete mischaracterization of the contact between Mr. Jaffer, a principal of GPG, and M. Nazim Gillani, of International Strategic Investments (ISI)," wrote Glemaud.

Lauzon declined to provide details about the disputed Tory caucus fund.

"Anything that is mentioned or discussed, we just don't discuss it," Lauzon said. "I believe in caucus solidarity and confidentiality, and I respect it 100 per cent."

Jaffer was previously a well-liked figure in the Conservative caucus and around Parliament Hill, a perpetually upbeat bon-vivant with little partisan edge.

But after he lost his seat to an NDP candidate in Edmonton-Strathcona, Jaffer's stock dropped dramatically within Harper's office. Conservatives say Harper was upset the party did not repeat its clean sweep of Alberta, and blamed Jaffer for not working hard enough in the riding.

Jaffer got the cold shoulder from the party when it came time for the next nomination battle in the riding.

Party officials scheduled the vote at a time Jaffer said he would not be available to compete. The nomination went to young businessman and former Parliament Hill staffer, Ryan Hastman.

A few months later, Jaffer was arrested north of Toronto after police stopped his speeding car. He was initially charged with speeding, drunk driving and cocaine possession, but pleaded guilty to the lesser charge of careless driving after the Crown said it was unlikely to win a conviction on the other charges.

Those who know Jaffer say he has become more isolated over the last year, and has little contact with his old friends.

The Toronto Star story described Jaffer's alleged activities on the evening he was arrested, saying they included a dinner meeting with some business associates and three prostitutes.

One of the men allegedly at the dinner, Nazim Gillani, purportedly sent an email to the others the following day saying Jaffer "has opened up the Prime Minister's Office to us."

The story also says Jaffer handed out parliamentary business cards even though he had lost his seat, and had bragged that he could access environmental funds from the government.

Jaffer says in his online biography that he helps the energy company he co-founded secure contracts with government, using the "countless relationships" he forged as a parliamentarian.

Liberal Leader Michael Ignatieff demanded answers from Harper on Thursday.

"These are allegations about Mr. Jaffer and they go to the heart of the integrity of the Conservative government. So the question is, who did Mr. Jaffer see in the Conservative government, what promises did Mr. Jaffer make, what did he propose to deliver to his clients?" Ignatieff said in Mirabel, Que.

"These are questions not just for Mr. Jaffer. They're questions for the Conservative government and Canada needs answers to the questions right away."