Retaking Beiji, 250 kilometers (155 miles) north of Baghdad, could allow Iraqi forces a base to attack neighboring Tikrit, taken by the extremists in their lightning advance this summer.
But troops backed by Shiite popular volunteer militias faced pockets of stiff resistance around Beiji, hindering their advance.
State television quoted the top army commander in Beiji, Gen. Abdul-Wahab al-Saadi, as saying troops recaptured the city's local government and police headquarters at the center of the town.
Raed Ibrahim, the governor of Salahuddin province, where Beiji and Tikrit are located, said the military had secured about 75 percent of the town as of Tuesday, retaking the center of the town and outlying districts.
He said government forces continued to meet fierce resistance from the terrorists, whom he said were using suicide bombers to stall the military's advance.
Ibrahim also told The Associated Press that many terrorists booby-trapped buildings in Beiji, posing an added threat.
A senior military official earlier told the AP that troops had recaptured of about 75 percent of Beiji.
Al-Saadi said Saturday that his forces had recaptured most of the city and that it would soon be entirely rid of ISIS group terrorists.
There was no word on the fate of the refinery, which lies on the outskirts of the town and has been besieged by ISIS fighters since June. The small army unit inside the refinery, resupplied and reinforced by air for months, successfully resisted wave after wave of extremist assaults.
In recent weeks, Iraqi allied forces recaptured a string of small towns and villages, but taking Beiji would be strategically significant in what is shaping up to be a drawn-out campaign against the terrorists.
Recapturing Beiji also would be a major boost for Iraq's government and could pave the way for a fresh offensive to drive ISIS terrorists from the nearby city of Tikrit, Saddam Hussein's hometown and the capital of Salahuddin province.
The Beiji campaign has been carried out by a contingent of troops and security forces drawn from a nearby military base and airlifted from government-controlled areas elsewhere.
Airstrikes by a U.S.-led coalition have aided Iraqi forces, militias and Kurdish peshmerga fighters battling ISIS militants. Hundreds of U.S. advisers and trainers also have been working with the Iraqis.
U.S. Central Command said Monday that coalition aircraft conducted seven airstrikes near Beiji since Friday, destroying three small militant units, a sniper position and two militant vehicles, including one used for construction.