Activists with the Occupy Toronto movement spent the night outside city hall in an ongoing protest against fiscal inequality, but Mayor Rob Ford says they won't be there for long.

The city will not stand by idly as activists turn a stretch of pavement near city hall into another St. James Park, Ford said in an interview with CP24 Wednesday.

"They are not supposed to be there and they will be asked to leave just like they were at St. James Park," Ford said. "Look, it's the same group of people; it is the same group of 400 or 500 that have been protesting from day one."

The protesters, known for their two-month occupation of St. James Park, had originally planned to pitch their tents on the grounds of Nathan Phillips Square but security promptly asked them to move off the property.

They subsequently moved to a narrow stretch of land adjacent to city hall, where they remain. The land is jointly owned by Osgoode Hall and the province, which raises issues around jurisdiction.

Some protesters who spoke to CP24 Wednesday said they would stay at the site for three days in hopes of setting a legal precedent while others suggested they would leave once they had written notice to do so.

As of 7 p.m. on Wednesday night, only about half a dozen protesters remained at the site.

"They are informing us that it is privately run and privately owned by the private ministry and that we cannot be here, but we want it in writing because we are still in search of what is public land," John Erb told CP24. "If buildings that our tax dollars have paid for and built are not a public space where we can stand and demonstrate and gather, what in this city is available for us?"

Hundreds of demonstrators gathered for a peaceful protest against Ford's 2012 budget around 5:30 p.m. Tuesday. But after 90 minutes of music and speeches, things turned violent as demonstrators tried to break a police blockade protecting the entrance of city hall.

CP24 had exclusive video of protesters being arrested (Check out video attached). At the height of the unrest, as many as 10 people were detained. In the end, four men in their 20's were charged with obstructing police, unlawful assembly, and mischief. Many of those in the crowd claimed police used excessive force, something the mayor rebuked Wednesday.

"I want to thank (the police) for doing a fantastic job," he said. "They showed a lot of restraint."

Despite the opposition, the budget passed -- but not without a few amendments.

A number of motions to restore cuts were passed by council. Including one by rookie Coun. Josh Colle that rescued $15 million worth of services. The budget is now a very different looking document than the one Ford originally presented last fall.

Money has been restored to a number of programs including child care, pools, ice rinks, shelters as well as money to stop service reductions to the TTC. Ford says that even though he didn't get all the cuts he wanted, he still views this as a victory.

"We got about 99.9 per cent of what we wanted and we are saving the taxpayers millions upon millions upon millions of dollars with this budget," he said. "The property tax increase was the most important thing. (Some councillors) wanted it to go to 3.5 per cent and we held it to 2.5 per cent, which is a huge achievement."

@chrisbfox and @SandieBenitah are on Twitter.