A spokesman for the Syrian Democratic Forces (SDF) told the AP news agency "the most experienced" jihadists are defending their last stronghold.
The statement gave no indication of how long it could take to capture Baghouz, but President Donald Trump said last week he had been told that the full territorial conquest of the Islamic State could be completed in the coming week.
British, French, US and other forces are actively looking for wanted IS operatives among those fleeing the combat zone with civilians.
"Some of them dress as women because we don't ask the women to raise their hijab", he said.
SDF-held areas make up a third of the whole country, and Damascus has repeatedly said it would eventually see them revert to government control. "There were heavy clashes this morning, with landmines going off", said SOHR head Rami Abdel Rahman.
At its peak in 2014, IS established a "caliphate" stretching across Syria and Iraq that was similar in size to the United Kingdom and ruled over more than 7.7 million people.
But a series of separate military operations, including by the SDF, have left its proto-state in tatters.
Major General Christopher Ghika, the coalition's deputy commander, described the size of the last IS pocket as "now less than one percent of the original caliphate".
At the group's height, 7.7 million people were estimated to live under ISIS' rule, according to Operation Inherent Resolve (CJTF-OIR), the official name for the coalition fighting ISIS.
The withdrawal would also include troops deployed at the U.S. military base at Tanf, near the Syrian border with Iraq and Jordan, they said.
The Kurdish-Arab militia - backed by USA air strikes - have now resumed the fight to take the last four-square-kilometre (one-square-mile) patch of territory in eastern Syria's Deir az-Zour from IS.
Washington has been trying to reach agreement with Turkey, which considers the US-backed Syrian Kurdish YPG militia a terrorist organization, for the safety of the YPG fighters after it pulls out.
IS does still hold another scrap of territory in Syria - and it continues to carry out dozens of attacks - many targeting the SDF. It said 10 attackers were killed, while two managed to flee.
Across the border in Syria, US special forces remain to "advise and assist" the SDF, and are involved in screening people fleeing from the fighting.
Back in December 2018, Trump announced a plan to withdraw the USA military in Syria, . a decision that unsettled its allies at the time. At the time, Trump announced the withdrawal of 2,000 USA troops from the conflict-ridden Middle Eastern country.