The suspect has been charged with murder while three other people have been held in custody. Police said they were trying to determine how they might be involved.
Prime Minister Jacinda Ardern said some of the victims may have been new immigrants and refugees.
In addition to the dead, health officials said 48 people were being treated at Christchurch Hospital for gunshot wounds. Injuries range from critical to minor.
Three other people - two men and a woman - were also arrested. At least one, though, is not believed to have had anything to do with the shooting. Dunedin is a city near the southern tip of New Zealand, around 225 miles from Christchurch.
Police said the investigation had extended 360 kilometers (240 miles) to the south, where homes in Dunedin were evacuated around a "location of interest". They gave no details.
At the White House, Trump called the bloodshed "a bad thing" but rejected any suggestion the white nationalist movement is a rising threat around the world, saying it is "a small group of people that have very, very serious problems".
A man who claimed responsibility for the shootings, 28-year-old Brenton Tarrant, an Australian citizen, left a 74-page manifesto in which he explained who he was and his reasoning for the attack, although police have not confirmed he is the shooter.
Police patrols are being stepped up around mosques in Peterborough following the terrorist attack in New Zealand.
British Prime Minister Theresa May offered deepest condolences "after the horrifying terrorist attack in Christchurch".
"Many of those affected may be migrants to New Zealand".
Referring to the four people detained by police, Ardern said, "These are people who I would describe as having extremist views that have absolutely no place in New Zealand".
Bush said police had found two improvised explosive devices in one auto, a clarification from an earlier statement that there were devices in multiple vehicles.
"We have one person in custody but we are unsure if there are other people".