Most people think of the border as a distant line tracing the outside edge of the United States. That is not how the U.S. government sees it. The border is officially defined as a zone that extends 100 miles into the interior of the United States from all external boundaries.

In that huge area – it encompasses 10 states, such cities as Los Angeles, New York and Chicago, and almost two-thirds of the nation’s population – the Border Patrol has expansive, nearly unchecked powers.

A recent Supreme Court decision, Egbert v. Boule, underlined those powers. A Border Patrol agent approached a Turkish guest at an inn in Washington state. When the inn’s owner, an American citizen, asked the agent to leave, the agent shoved him against a car and then threw him on the ground. The man filed suit but the court ruled that even when the constitutional rights of an American citizen are violated by a federal agent, there is no right to sue if the violation occurred in the course of the agent’s official duties.

With that decision, the Supreme Court added to decades of precedents that have allowed the Border Patrol to barely abide by the Constitution, especially its 4th Amendment protections against unreasonable searches and seizures.

Honduran asylum seekers are taken into custody by U.S. Border Patrol agents after the group crossed the U.S. border wall into San Diego, in this view from Tijuana, Mexico. Moises Castillo/Associated Press

The Border Patrol isn’t stationed at ports of entry and airports, which are manned by Customs and Border Protection’s field operations officers. Instead, the Border Patrol polices immigration between crossing points. In the 100-mile border zone, its agents stop cars if they have a “reasonable suspicion” about their occupants. When they set up interior checkpoints, everyone must stop for questioning and inspection. And they use racial profiling to boot.

This is not what Congress envisioned when it established the Border Patrol in 1924. The original authorization said an agent could “arrest any alien who in his presence or view is entering or attempting to enter the United States in violation of any law.”

Sen. David A. Reed, a Pennsylvania Republican, explained in the 1925 Senate debates about the legislation that Border Patrol agents could only operate close to the border itself: “They have no right to go into the interior city and pick up aliens on the street and arrest them.” He continued, “We are all on alert against granting too much power to these officials to act without a warrant.”

Nevertheless, a second clause in the Border Patrol’s authorization did allow agents to extend their authority away from the border line. They could “board and search” any “conveyance” in which they believed “aliens” were being brought into the United States.

The agents’ wide jurisdiction was unpopular with border residents and in Congress. In 1946, the Border Patrol’s authorization was revised: It could operate inside the United States, but only “within a reasonable distance from any external boundary.”

Unfortunately, the next year, a routine policy interpretation in the Federal Register set “a reasonable distance” at 100 miles from borders and coastlines. Border Patrol agents were free to “board and search” any vehicle deep into the interior of the country.

At the time, there were so few Border Patrol agents, the agency’s seemingly unconstitutional authority went unscrutinized for decades. Finally, in the 1970s the conflict between the 4th Amendment and the Border Patrol’s congressional authorization reached the Supreme Court.

In Almeida-Sanchez v. United States (1973), the court reined in the agents, at least when it came to formal searches. In a 5-4 decision, the justices said the Border Patrol needed consent, a warrant or probable cause to conduct a search, just like other police officers.

Two years later, in United States v. Brignoni-Ponce, the justices considered Border Patrol stops. Could they be based only on the race of the occupants?

In a unanimous decision, the justices said no, but they didn’t rule out race as a criterion. To meet a standard of “reasonable suspicion,” the court required two factors, and one could be race. The list they provided of other acceptable factors was so broad it in effect has allowed agents to stop almost any vehicle under any circumstances in the border zone.

In 1976, United States v. Martinez-Fuerte tested the constitutionality of the Border Patrol’s fixed or temporary checkpoints, set up 25 to 100 miles from the border. The ruling? At a checkpoint, agents can stop every “conveyance” to question people about their immigration status and inspect the exterior of vehicles. On the basis of a single factor of suspicion they can pull someone over for more thorough questioning. Race alone meets that standard.

U.S. Border Patrol officials detain migrants who came ashore on the intracoastal waterway in Pompano Beach, Fla. Joe Cavaretta/South Florida Sun-Sentinel via AP

Today there are more than 19,000 Border Patrol agents, and terrorism prevention and drug enforcement, in addition to immigration, have become top priorities. In the 100-mile border zone, a Latino driver may be considered reasonably suspicious because he stared at a passing agent or because he wouldn’t make eye contact. Roving stops don’t even have to be based on anything related to the vehicle in question, just the characteristics of the area.

At the 113 interior Border Patrol checkpoints, what an agent sees in your back seat may get you pulled over for further investigation that could well end in a search – most people consent when asked; they don’t know they can refuse or that agents need probable cause to open their trunk otherwise. At many of the checkpoints, the agents use dogs to sniff for drugs, radiation sensors to look for dirty bombs and collect license numbers that are stored in a database for 15 years. And now agents are immune from prosecution for even flagrant violations of the constitutional rights of immigrants and citizens alike.

The Border Patrol’s lack of limits should concern all Americans. As the Supreme Court has pointed out, Congress created the agency’s authorization and Congress can curtail or end it.

Legislation has been introduced in the House to cut back the border zone to 25 miles from external boundaries. An even smaller zone might be appropriate given that 58 percent of apprehensions happen within one mile of the border.

Congress should also draft a new statute ending lawsuit immunity for federal agents who violate constitutional rights, and it should ban the use of interior checkpoints, which the Border Patrol’s data show primarily ensnare American citizens traveling domestically and carrying small amounts of drugs.

Even if Congress does not act, closing interior checkpoints and substantially shrinking the size of the border zone could be carried out by the secretary of the Department of Homeland Security within its existing congressional authorization.

The Border Patrol has not yet begun to deploy its authority on a widespread basis in major cities in the 100-mile border zone. But it could. Before a new checkpoint goes up in L.A. or San Francisco or New York, we should reconsider its power.