After seizing state TV, an army spokesman announced it was targeting people close to Mr Mugabe who had caused "social and economic suffering".
The move came after Mr Mugabe sacked his deputy, Emmerson Mnangagwa, in favour of his wife, Grace.
Heavy gun and artillery fire could be heard in northern parts of the capital Harare early on Wednesday.
A statement read out by a general on air denied it was a coup. There was no immediate word from the president himself.
Mr Mugabe, 93, has dominated the impoverished country's political scene since independence from the UK.
The UK Foreign Office advised Britons "currently in Harare to remain safely at home or in their accommodation until the situation becomes clearer", while the US embassy in Harare advised US citizens in Zimbabwe to "shelter in place" until further notice.
China, Zimbabwe's biggest trading partner, says it is closely watching the situation and hopes that the relevant parties can properly handle their internal affairs.
How did the military justify its move?
Troops in armoured vehicles have been out in the streets of the capital Harare since Tuesday.
After soldiers overran the headquarters of the ZBC broadcaster, Maj Gen Sibusiso Moyo went on air to say the military wished to "assure the nation that his Excellency the president... and his family are safe and sound and their security is guaranteed".
"We are only targeting criminals around him who are committing crimes... that are causing social and economic suffering in the country," he said.
"As soon as we have accomplished our mission, we expect that the situation will return to normalcy."
Other key points of the statement included:
Citizens should remain calm and limit unnecessary movement
The military assures the Zimbabwean judiciary that its independence is guaranteed
Security services should "co-operate for the good of our country" and any provocation would "be met with an appropriate response"
All leave for the defence forces is cancelled and personnel should return to barracks immediately
It is not clear who is leading the military action. Army chief Gen Constantino Chiwenga had said the army was prepared to act to end purges within the ruling Zanu-PF party.
What do we know of the shooting?
The firing was coming from northern suburbs where Mr Mugabe and a number of government officials live, the BBC's Shingai Nyoka reports from Harare.
Has anyone been detained?
A government source told Reuters news agency that Finance Minister Ignatius Chombo had been detained.
He is a leading member of a faction of Zanu-PF led by Grace Mugabe.
Is this a coup?
Alex Magaisa, former adviser to Zimbabwean opposition leader Morgan Tsvangirai, told the BBC: "They have decided not to call it a coup because they know that a coup does not sell, it will be condemned.
"But as far as authority is concerned it seems very clear that President Mugabe is now just a president in name and authority is now residing in the military."
Zanu-PF had accused Gen Chiwenga of "treasonable conduct" after he issued his warning that the army might intervene.
What was the political situation before the army acted?
Mr Mugabe sacked Mr Mnangagwa last week, amid a row over succession.
Mr Mnangagwa had previously been seen as a potential heir to the president, but First Lady Grace Mugabe had since become the clear front-runner.
Last month, Mrs Mugabe accused allies of Mr Mnangagwa of planning a coup.
The rivalry between Mrs Mugabe and Mr Mnangagwa split Zanu-PF.
Gen Chiwenga is a close ally of Mr Mnangagwa and both are veterans of the 1970s war which ended white minority rule.
The leader of the war veterans, Chris Mutsvangwa, welcomed the military move, telling Reuters: "This is a correction of a state that was careening off the cliff.
"It's the end of a very painful and sad chapter in the history of a young nation, in which a dictator, as he became old, surrendered his court to a gang of thieves around his wife."
Source: BBC