When I read last week about Justice Clarence Thomas accepting but not disclosing numerous lavish vacations paid for by a billionaire businessman, I thought about my time as a federal government employee. And especially about the brick I received and had to return as a “thing of value” I could not lawfully accept.

Let me explain. As a prosecutor at the U.S. Securities and Exchange Commission in the 1980s, I and my staff sued a company that packed bricks in boxes and shipped them claiming they were computer hard drives. When the case was completed, the attorney representing the company’s board sent me a brick encased in Lucite as a memento. After consulting with federal ethics officials, I was told to return it because it was considered “a thing of value” I could not accept. At the time, I thought this was excessively bureaucratic, but I never questioned my obligation as a federal employee paid by American tax dollars to avoid even the appearance of showing favor to anyone with whom I dealt.

Which brings me to the latest revelations about Justice Thomas. In assessing Justice Thomas’ conduct, we should focus on the fundamental principle that judges should, at all times, conduct themselves in a manner that does not create the appearance of impropriety. In plain English, this means they should not create the appearance either of favoritism or of using their judicial position for personal enrichment.

Earlier this year, Chief Justice John Roberts assured us that the federal judicial code of ethics need not apply to the court because the justices “do in fact consult the code of conduct in assessing their ethical obligations.” But even a cursory review of the code makes it perfectly clear that accepting luxury travel and vacations is a no no. The code requires that: “A judge should neither lend the prestige of the judicial office to advance the private interests of the judge or others nor convey or permit others to convey the impression that they are in a special position to influence the judge.”

Thomas’ acceptance of these gifts violates both aspects of this provision. These expensive, exclusive gifts clearly create the impression the billionaire donor is in a special position to influence Justice Thomas. This impression is reinforced by the oil painting shown in new reports of Justice Thomas sharing a cigar with the donor and others, including Northeast Harbor property owner Leonard Leo, whose career is devoted to influencing court decisions. Moreover, Thomas’ acceptance of lavish benefits he could not personally have paid for on his federal salary advances his private interests, a perception that was not dispelled by his statement last week. Thomas noted that the billionaire gift giver and his wife have been “among our dearest friends” for 25 years. But Thomas has been on the court for more than 30 years. Does he sincerely believe, or expect us to believe, that one of his “dearest friends” invited him on these luxury jaunts because Thomas is a great guy and not because he sits on our nation’s highest court?

In this instance, Thomas clearly acted improperly and, in doing so, did further damage to the court’s already-tattered reputation for impartiality and integrity. Just as I had to return the brick years ago, Thomas should have declined these gifts and, like the rest of us mere mortals, paid for his own vacations.