Haider al-Abbadi confirmed the decision in a statement on his website on Sunday, Al Jazeera reported.
Al Jazeera's Mohammed Jamjoom, reporting from Baghdad, said the statement says that all military officials with the rank of officer or above, who left their Ramadi posts without permission, will be court-martialled.
"What is not clear is when those court martials will happen, or if any of those members of the military have already (faced proceedings)," he said.
ISIL seized Anbar province's capital Ramadi in May, extending its control over the Euphrates valley west of Baghdad and dealing a major setback to Abbadi and the US-backed army he entrusted with its defence.
The fall of Ramadi was widely considered as the Iraqi army's worst defeat since ISIL swept through north Iraq last June.
Iraqi forces have launched a number of offensives to retake ground in Anbar province since ISIL took over Ramadi, but have so far been largely unsuccessul in recapturing the provincial capital.
Thousands of people have been displaced by the fighting and more than 1,500 schools have been damaged or destroyed as a result of the conflict in Anbar province alone, according to an Iraqi official.
Abbadi launched a sweeping reform campaign last weekaimed at combating corruption and mismanagement in the biggest shake-up in the governing system since the United States' military occupation.