PORTLAND (AP) — The U.S. has come far in the journey toward racial equality, but change must continue, former defense secretary William Cohen said on Monday.

Unemployment remains twice as high among blacks, and they make up a disproportionate part of prison populations, said Cohen, secretary of defense from 1997 to 2000, in his keynote address at the Martin Luther King Jr. Day breakfast hosted by the NAACP Portland Branch. Meanwhile, drug laws discriminate against some races, and black communities have limited access to health care and healthy foods, he said.

If King were alive, he would tell President Barack Obama that “things are not well,” Cohen said.

“We have got to change, Mr. President, and you have to take the leadership,” Cohen said at the event, which drew roughly 750 people.

But Cohen, who represented Maine in the U.S. House and Senate before serving as defense secretary under President Bill Clinton, said it’s challenging for Obama to discuss and take action on discrimination issues because it remains a contentious topic. He called on the audience to continue working toward equality, like not traveling to states that have discriminatory laws.

The Portland Press Herald reported that Cohen recalled his experience with prejudice growing up in Maine. Cohen said he was called names and didn’t receive a summer job because he was the son of a Russian Jewish immigrant who owned a bakery in Bangor.

“It taught me something about discrimination,” Cohen said. “It taught me about being the other.”

Cohen, a Bowdoin College and Boston University School of Law graduate, attended the event with his wife, Janet Langhart, a black author and TV personality. Cohen is now chairman and CEO of a lobbying and consulting firm in Washington, D.C.

The event also honored the 50th anniversary of the NAACP Portland Branch and King’s visit to Maine in May 1964.