Carlos Rafael was made on the waterfront. For decades, the balding seafood magnate haunted the docks and early-morning fish auctions in New Bedford, Massachusetts, where he had gone from gutting fish as a high school dropout to controlling one of the largest fishing fleets in the United States. Though he estimated his net worth at $10 million to $25 million, he still walked the creaky, bait-scented wharves in flannel shirts and worn jeans every day, barking out commands and alternating between foul-mouthed English and rapid-fire Portuguese as he chain-smoked Winston cigarettes and monitored the day’s catch.

That all changed in 2016, when federal authorities revealed that Rafael was at the center of a sprawling criminal investigation involving fake Russian mobsters, fraudulent haddock and duffel bags of cash. Now 67, Rafael will never fish again, according to the terms of a settlement with the National Oceanic and Atmospheric Administration that was announced Monday.

It’s the latest chapter in the downfall of the man known as the “Codfather,” who is serving nearly four years in federal prison, and, under the new settlement, owes the government more than $3 million in fines.

Under the circumstances, getting out of the fishing business was the right choice, Rafael’s attorney, John Markey, told The Washington Post. But it also amounts to a significance sacrifice for the seafood tycoon, who wasn’t yet ready to retire. Up until the day Rafael reported to prison, Markey said, he still went to work on the docks each day at 6 a.m., driving a 10-year-old pickup truck.

“This is what he enjoyed doing,” Markey said. “It was a part of him.”

Until the law finally caught up with him, Rafael looked like an all-American success story. Born to a farming family in the Azores, a Portuguese archipelago in the Atlantic, he was sent to live at a monastery at 12 because his parents feared he would be drafted into the colonial government’s wars in Angola and Mozambique. As a teenager, Rafael desperately wanted to move to the U.S., he recalled in a 2004 oral history. But his father was hesitant, so Rafael forced the issue by getting himself kicked out of the monastery. Knowing that their son would almost certainly be conscripted if they didn’t leave, Rafael’s parents agreed to move to New Bedford, a historic whaling port where nearly a third of inhabitants claim Portuguese ancestry.

Arriving in Massachusetts at 15, Rafael dropped out of school after a week, finding the lessons too basic – “they’re telling me dog, cat, fork, knife” – and got a job making linguica, a smoked Portuguese sausage. That, too, didn’t last long: He quit after four days when he was told that he couldn’t take frequent smoke breaks. In his 2004 oral history, Rafael recalled that he told his boss, “Look, the American Dream that I wanted wasn’t this, to come and make linguica and I cannot even go for a cigarette after an hour’s work […] You keep your linguica and I’m leaving.”

New Bedford consistently ranks as the most lucrative commercial fishing port in America, so it was perhaps inevitable that Rafael would end up finding work on the waterfront. He started out as a fish cutter – gutting, cleaning and deboning fish as soon as they arrived on the wharves – and worked his way up through the ranks to become a foreman. By the early 1980s, he had saved enough money to buy his first boat and start his own business, Carlos Seafood.

In the years that followed, one boat turned into several dozen, and before long, Rafael was sitting at the head of a massive fishing empire and controlling about a fifth of the New England cod market. His success was puzzling to some, given that the industry was hurting badly: Overfishing had depleted the supply of groundfish like cod and flounder, and the federal government responded by imposing strict regulations on commercial fishermen. Meanwhile, many of Rafael’s business dealings raised eyebrows – he was sentenced to six months in prison for tax evasion in 1984, indicted but ultimately acquitted for price-fixing a decade later, and later pleaded guilty to forging sales receipts in 2001.

“I think they love to knock you down,” he complained in his 2004 oral history. “It looks like it’s deliberate when they do things like that in the system. If you do well, you not supposed to do well. I guess it’s like it’s against the law if you’re successful.”

Rafael didn’t like having to follow rules about where his boats could go and what they were allowed to catch any better than he had liked being told when he could take a cigarette break at the linguica factory. He once compared federal fisheries regulators to the Gestapo, and in 1994 predicted that new, conservation-minded laws would either force business owners like him to go bankrupt, or turn them into outlaws, the New Bedford Standard-Times reported. He chose the latter.

“I am a pirate,” he told a group of federal regulators. “It’s your job to catch me.”

Eventually, they did. In January 2015, Rafael told the Standard-Times that he was looking to sell his business and move to Cape Verde, a former Portuguese colony off the African coast. Months later, he received a visit from undercover agents who were posing as Russian immigrants “involved in organized crime” and their broker, according to federal court records.

Unconcerned about whatever shady dealings the Russians might be involved in, Rafael told them his seafood company was the perfect place to launder money. “You can become a laundromat,” he offered. “You’ll never find a better laundromat than this.” He volunteered to sell them the business for $175 million – a staggering sum considering that he had been telling the IRS that the company made only $3 million to $4 million a year, and that its total assets were worth around $21 million. To prove that it was worth that much, he pulled out a second set of ledgers, showing that he had made more than $600,000 in off-the-books cash in only six months.

Over the course of several meetings with the Russians, Rafael explained how he was pulling in millions of dollars while other fishermen struggled to make a living. Since haddock was abundant in the northern Atlantic and there was no risk that his boats would catch enough to meet the generous annual quota, any other fish they netted would also get labeled as haddock on official forms when dockside inspectors weren’t looking. That way, Rafael could get around the tighter quotas for more lucrative fish such as sole and flounder, which were in short supply.

Conveniently, Rafael also owned the seafood distributor that would buy the mislabeled fish and falsify a second set of reports, paying a lowball price for “haddock” that was then labeled correctly and sold at its much higher market price to a New York broker, who paid Rafael with duffel bags full of cash. Much of that money, the seafood magnate said, was smuggled to Portugal – at one point with the help of a sheriff’s deputy who was later convicted of his role in the scheme.

“You could be the IRS in here,” Rafael mused at one point, unaware that the conversation was being recorded. But he reasoned the agency wouldn’t have Russians working undercover. “That would be some bad luck!”

In fact, the agents were there as part of a joint operation between the IRS and NOAA, and in February 2016, Rafael was arrested during a raid at his New Bedford warehouse. He pleaded guilty in March 2017 to falsifying fish quotas, tax evasion, and conspiracy, saying that he wanted “to get this over with,” and was sentenced to 46 months in prison.

“I did it because I wanted to make sure my people kept getting a paycheck,” he wrote in a statement that was read out loud by his attorney at the sentencing hearing. “The waterfront is a hard world we work in.”

Once behind bars, Rafael still faced a lawsuit from NOAA. The pending litigation meant that his boats were grounded and their crews went without work, which had a devastating ripple effect on New Bedford’s economy. Under the terms of Monday’s settlement, Rafael will be allowed to sell those boats and keep the proceeds. The resolution, New Bedford Mayor Jon Mitchell told The Washington Post in an emailed statement, “enables the Port of New Bedford to turn the page on the Carlos Rafael saga.”

Rafael is slated to get out of prison in 2021, and plans to spend more time with his children and grandchildren. He’ll split his time between New Bedford and Corvo, his native island in the Azores, where he can “enjoy the benefits of his hard work without having to fight the government for another five or 10 years,” Markey, his attorney, said.

“But it’s difficult,” he added, “when you give up a big part of who you are.”