Gary Kaplan, founder of the online sports book BetOnSports, has lost his appeal to avoid paying $36 million in taxes and penalties to the IRS in conjunction with his indictment.

A panel of federal judges from the Eighth US Circuit Court of Appeals struck down Kaplan’s challenge on Wednesday, ruling that his 2008 plea deal with US authorities did not preclude the government from going after back taxes.

Kaplan, a high school dropout and former New York bookie, helped create what was once one of the biggest online betting websites in the world. After being arrested on sports booking charges in 1993, Kaplan decided to transfer the operation offshore to Aruba, later moving again to Antigua and then finally settling on Costa Rica.

Business boomed in the early 2000s and the website employed nearly 2,000 people. In 2004, BetOnSports went public on the London Stock Exchange, but issued investors the following warning: “In general, persons and organizations engaged in the business of sports betting in the US via the telephone or internet are in violation of existing US federal law.”

Kaplan’s stake in the company made him fabulously wealthy and he stashed his ill-gotten gains in banks all over the world. Prior to going public, Kaplan created a trust “off the coast of France” into which he transferred $98 million. As the sole grantor of the trust, Kaplan was required to pay federal income and capital gains taxes, but he filed neither.

In 2006, Kaplan, along with other BetOnSports execs, were indicted in St. Louis and accused of running an illegal betting operation. He was later arrested in the US during a layover for a trip back to Costa Rica. In 2009, Kaplan struck a deal with federal authorities and pleaded guilty to bookmaking, racketeering, and mail fraud. He was sentenced to a 51-month prison term and ordered to pay the US Government $43 million.

This was the only the start of Kaplan’s financial woes. The taxman soon came calling for its share of the bounty, demanding $36 million from the former BetOnSports CEO in back taxes and fines.

But Kaplan believed that he was immune to such tax liability and pointed to a clause in his plea deal that stated the US Government “shall not seek civil forfeiture in connection with this case or any asset constituting or derived from the receipt of income from the BetOnSports organization.”

In 2009, however, a judge asked Kaplan if he understood that there “was a difference between a criminal tax proceeding and a civil tax proceeding,” to which he responded that he did. He further clarified, telling Kaplan the deal didn’t “preclude the initiation of any civil tax proceeding or administrative action” against him.

The BetOnSports chief argued that according to the deal, any further action by the IRS against him would essentially be double jeopardy. He also claimed that the six-year statute of limitations had expired for the tax returns in question. But as the court noted, Kaplan had never filed tax returns for 2004 and 2005; therefore, the statute of limitations had not yet even begun.

The court also reminded Kaplan that it was his own lawyers who crafted a presentence report which maintained the governments’ right to impose further tax liability. “It was Kaplan, if anyone, who took the position of no tax liability being present,” the panel ruled.

With those factors in mind, the Eighth Circuit panel ruled that the plea deal language was “unambiguous” on the government’s right to collect the $36 million sum.

Want the latest poker headlines and interviews? Follow PocketFives on Twitterand Like PocketFives on Facebook.