With chants of “Rob Ford has got to go” and signs pointing to allegations of drug use, hundreds of Torontonians gathered in front of city hall Saturday afternoon to call on the city’s embattled mayor to step down.

Citing various reasons, Ford critics argued the mayor is unfit to lead the city until his four-year term expires in 2014, and they demanded his resignation as they rallied in Nathan Phillips Square.

Ryan North said he attended the rally because he wanted to let people know “that there’s more to Toronto than Rob Ford and there’s more to Toronto than crack cocaine allegations.”

North said he’s concerned the allegations and the worldwide media attention are overshadowing city hall’s priorities.

“It’s what most of the American and world media knows about Toronto and I worry about that because (former mayor Mel) Lastman called out the Army 10 years ago and we still get the Army jokes about Toronto,” North said, referring to the time Lastman brought in the military in January 1999 to dig the city out of a massive snowstorm.

“This crack cocaine story has been going for two weeks now with no end in sight and I worry that it’s overshadowing not just the business of the city but the city itself,” North added, as people around him chanted slogans against Ford.

Outspoken critic John Veillette, carrying a sign reading “Crack in Rob’s Armour,” accused Ford of lacking leadership since he was elected in 2010.

“The main problem really for me, personally, is Transit City,” Veillette, referring to a public transit plan developed under former mayor David Miller, told CP24 reporter Jackie Crandles. “The way (Ford) just advocated slashing it was just ridiculous.”

The rally was organized on Facebook by Chris Wright, who urged people to make it a “positive, respectful and upbeat event.”

“We want to join voices to inject hope and joy into Toronto’s future,” Wright, the Facebook group’s creator, wrote on the social-networking website.

Wright also asked people to write messages in chalk on the concrete in front of city hall. He urged people to write positive messages and avoid personal attacks on the mayor.

Wright declined an interview request.

According to the Facebook group, more than 4,000 people pledged to attend the event, but the actual attendance figure was much lower. More than 25,000 Facebook users received an invite.

Calls for the mayor's resignation have increased after the U.S. gossip website Gawker and the Toronto Star reported drug allegations against him last month.

According to the reports, an alleged cellphone video shows Ford smoking from a glass pipe.

The alleged video has not surfaced, and its authenticity has not been substantiated.

On his radio show last Sunday, Ford told a caller “there’s no video, so that’s all I can say. I can’t comment on something that doesn’t exist.”

With files from CP24 reporter Jackie Crandles.

@ChrisKitching is on Twitter. For instant breaking news, follow @CP24 on Twitter.