OTTAWA -- Confessions extracted through so-called Mr. Big police sting operations tend to produce unreliable confessions, are open to abuses and must be regulated more carefully in order to be admissible in court, the Supreme Court of Canada ruled Thursday.

The decision by the country's highest court calls into question the convictions of Canadians behind bars across the country.

In a majority decision, the court ruled that stings like the one used to convict a Newfoundland man of drowning his three-year-old twin daughters can be risky.

The justices ruled that Nelson Hart's confession during a sting operation more than a decade ago cannot be used against him should he face another trial. It will now be up to the Crown to determine whether Hart can be retried.

Hart was initially convicted of first-degree murder in 2002 in the drowning deaths of his daughter, Krista and Karen. At trial, court heard that Hart showed undercover officers, posing as members of the mafia, how he drowned the girls by shoving them into the waters of Gander Lake, Newfoundland.

But the conviction was overturned in 2007 by the province's appeal court by a 2-1 margin. The ruling questioned the reliability of his confession.

The Supreme Court ruled that Hart's Charter rights may have been violated and that Canada's legal system does not adequately protect the rights of people who are subject to Mr. Big sting operations.

"I have concluded that the April 1 confession must also be excluded," Justice Michael Moldaver wrote on behalf of the majority.

"As such, it is doubtful whether any admissible evidence remains upon which a jury, properly instructed and acting reasonably, could convict," he added. "However, the final decision on how to proceed rests with the Crown."

The Mr. Big investigative technique involves undercover police officers who recruit a suspect to a ficticious criminal organization while posing as gangsters. The aim is to obtain a confession to a crime.

But there are "significant problems" with such stings, the court said.

"To be sure, Mr. Big operations can become abusive, and they can produce confessions that are unreliable and prejudicial," said the ruling.

The court said police and lawmakers must figure out a better way to extract solid confessions from suspects in criminal cases.

"We must seek a legal framework that protects accused persons, and the justice system as a whole, against these dangers," it said.

"Mr. Big operations are a creative and sometimes useful law enforcement technique, but the courts must carefully police their boundaries lest they stray from being useful strategies into ploys that allow the state to manipulate and destroy the lives of individuals who are presumed to be innocent."