In all the discussions around the appropriateness of building an Islamic cultural center two blocks from the desolate hole where the World Trade Center’s twin towers once stood, we should be clear about one thing from the outset:

The reason more than two- thirds of Americans say they disapprove of what has come to be called the “Ground Zero Mosque” is not because Muslims do not have a right to build a house of worship even where most people may oppose it.

We are, at least ideally, a “government of laws and not of men.” So, where zoning rules allow such structures and the necessary permits have been acquired, no one should tell peaceful believers they cannot have the structure they want in the place where they want it.

But that is not the issue here, despite the efforts some have made to libel the mosque’s opponents as callous bigots.

Indeed, one of the major differences between this nation and many Islamic states is that we permit a nearly infinite variety of structures to be built for religious purposes, while many Islamic countries do not.

There are literally no churches or synagogues permitted in Saudi Arabia, for example, and many other Islamic states restrict or ban them, too.

The conclusion that unites people as disparate as Sen. Joseph Lieberman, Senate Majority Leader Harry Reid, columnist Charles Krauthammer and Deborah Burlingame, head of 9/11 Families for a Safe and Strong America, is their clear vision that religious liberty is not the issue here at all.

There are many mosques in New York City, and the only requirements their builders had to meet were the same ones Christians, Jews or members of any other faith had to meet to create their worship spaces.

But, no matter what its supporters say, this mosque is different. It is located, as Krauthammer pointed out last Friday, next to “the site of the greatest mass murder in human history — perpetrated by Muslims of a particular Islamist orthodoxy in whose cause they died and in whose name they killed.”

And, while it is certainly true that most Muslims are not followers of that orthodoxy, many hold it around the world. There is no doubt that they would view the construction of this worship center as a stellar victory for their cause, no matter what the intent of its builders might be.

As Burlingame says, the 13-story building “will further energize those who regard it as a ratification of their violent and divinely ordered mission: the spread of shariah law and its subjugation of all free people, including secular Muslims who come to this country fleeing that medieval ideology, which destroys lives and crushes the human spirit.”

That the sponsors have every right to build this mosque where they want it is beyond dispute. Still, what would please most Americans beyond all measure would be the sponsors’ own free decision to forgo that right.

In that voluntary act of sacrifice, they would deny the violent extremists who say their version of Islam is the only proper one the chance to claim a phony victory that they did not earn and never could deserve.