Powell, 68, who recently visited storm survivors at Reunion Arena in Dallas, said he was "deeply moved" by the families displaced by the devastating storm and was critical of the preparations for Hurricane Katrina. "I think there have been a lot of failures at a lot of levels — local, state and federal. There was more than enough warning over time about the dangers to New Orleans. Not enough was done. I don’t think advantage was taken of the time that was available to us, and I just don’t know why," Powell told ABC News’ Barbara Walters in an exclusive interview for "20/20."

Powell doesn’t think race was a factor in the slow delivery of relief to the hurricane victims as some have suggested. "I don’t think it’s racism, I think it’s economic," he told Walters.

"When you look at those who weren’t able to get out, it should have been a blinding flash of the obvious to everybody that when you order a mandatory evacuation, you can’t expect everybody to evacuate on their own. These are people who don’t have credit cards; only one in 10 families at that economic level in New Orleans have a car. So it wasn’t a racial thing — but poverty disproportionately affects African-Americans in this country. And it happened because they were poor," he said.

Making False Case for Iraq War a ’Blot’ on Record

When Powell left the Bush administration in January 2005, he was widely seen as having been at odds with Defense Secretary Donald Rumsfeld and Vice President Dick Cheney over foreign policy choices.

It was Powell who told the United Nations and the world that Saddam Hussein had weapons of mass destruction and posed an imminent threat. He told Walters that he feels "terrible" about the claims he made in that now-infamous address — assertions that later proved to be false.

When asked if he feels it has tarnished his reputation, he said, "Of course it will. It’s a blot. I’m the one who presented it on behalf of the United States to the world, and [it] will always be a part of my record. It was painful. It’s painful now."

He doesn’t blame former CIA Director George Tenet for the misleading information he says he pored over for days before delivering his speech; he faults the intelligence system.

"George Tenet did not sit there for five days with me misleading me. He believed what he was giving to me was accurate. ... The intelligence system did not work well," he said.

Nonetheless, Powell said, some lower-level personnel in the intelligence community failed him and the country. "There were some people in the intelligence community who knew at that time that some of these sources were not good, and shouldn’t be relied upon, and they didn’t speak up. That devastated me," he said.

While Powell ultimately supported the president’s decision to invade Iraq, he acknowledges that he was hesitant about waging war. "I’m always a reluctant warrior. And I don’t resent the term, I admire the term, but when the president decided that it was not tolerable for this regime to remain in violation of all these U.N. resolutions, I’m right there with him with the use of force," he said.

Powell told Walters he is unfazed by criticism that he put loyalty to the president over leadership. "Loyalty is a trait that I value, and yes, I am loyal. And there are some who say, ’Well, you shouldn’t have supported it. You should have resigned.’ But I’m glad that Saddam Hussein is gone. I’m glad that that regime is gone," he said.

When Walters pressed Powell about that support, given the "mess" that the invasion has yielded, Powell said, "Who knew what the whole mess was going to be like?"

While he said he is glad that Saddam’s regime was toppled, Powell acknowledged that he has seen no evidence of a link between Saddam Hussein and the 9/11 terrorist attack. "I have never seen a connection. ... I can’t think otherwise because I’d never seen evidence to suggest there was one," he told Walters.

Despite his differences with the administration, Powell said he never considered resigning in protest. "I’m not a quitter. And it wasn’t a moral issue, or an act of a failure of an active leadership. It was knowing what we were heading into, and when the going got rough, you don’t walk out," he told Walters.

Stay the Course in Iraq

When asked what steps he would take in Iraq, Powell said, "I think there is little choice but to keep investing in the Iraqi armed forces, and to do everything we can to increase their size and their capability and their strength," he said.

Still, he questions some of the administration’s post-invasion planning. "What we didn’t do in the immediate aftermath of the war was to impose our will on the whole country, with enough troops of our own, with enough troops from coalition forces, or, by re-creating the Iraqi forces, armed forces, more quickly than we are doing now. And it may not have turned out to be such a mess if we had done some things differently. But it is now a difficult situation, but difficult situations are there to be worked on and solved, not walked away from, not cutting and running from."

Powell said he is sensitive to Cindy Sheehan and other mothers and family members whose loved ones have been wounded or killed in Iraq, but stressed that soldiers are risking their lives for a worthy purpose. When asked what he would say to Sheehan, who has grabbed media attention with her daily anti-war protests near Bush’s Crawford, Texas, ranch, he told Walters he’d tell her what he’d tell any mother who suffered such a loss: "We regret the loss, but your loved one died in service to the nation and in service to the cause."