(afp) -- The announcement comes a month after the capture by Iraqi forces of second city Mosul further east in a major blow to the terrorists.
In a televised speech, Abadi, dressed in military uniform and standing in front of an Iraqi flag and map of the country, announced "the start of an operation to free Tal Afar".
"I am saying to Daesh [ISIS] that there's no choice other than to leave or be killed," he said.
"We have won all our battles, and Daesh have always lost," he said, telling the country's troops that "the entire world is with you."
Tal Afar is located 70 kilometres (43 miles) west of Mosul, where government forces ended the group rule in July after a months-long battle.
In June 2014, ISIS fighters overran Tal Afar, a Shiite enclave in the predominantly Sunni province of Nineveh, on the road between Mosul and Syria.
At the time it had a population of around 200,000, but local officials said it was now impossible to know the exact number still living inside the city as most are cut off from the outside world.
However, authorities have accused the approximately 1,000 ISIS members in the city of using civilians as human shields during Iraqi and coalition air strikes earlier this week in preparation for the ground assault.
Abadi said that Iraq's paramilitary Hashed al-Shaabi forces would help various army, police and counter-terrorism units in Tal Afar.
The umbrella organisation, which is dominated by Iran-backed [forces], has already been fighting to retake a number of other Iraqi cities from the Islamic State.
- 'Victory is near' -
"In the early hours, the guns and flags turned towards their targets," said Ahmed al-Assadi, a spokesman for Hashad.
"Victory is near" in Tal Afar, an "Iraqi city taken hostage and humiliated for years by attacks from these barbarians," he said.
ISIS overran large areas north and west of Baghdad in 2014, but Iraqi forces have since regained much of the territory.
Once Tal Afar is retaken, Iraqi authorities intend to launch a fight to retake ISIS-held Hawija, in the province of Kirkuk, 300 kilometres north of Baghdad.
ISIS still hold areas in Anbar, a western province that faces major security challenges.