Akhmetov told a cabinet meeting that his resignation had been granted by President Nursultan Nazarbayev, the government of the oil-exporting country said on Wednesday.
No reason was given for Akhmetov's abrupt dismissal, but President Nazarbayev has broad powers to change prime ministers at his choosing.
His press-service said on Twitter that the Kazakh leader has already proposed that former premier Karim Masimov takes up the role again.
"According to the Constitution, the dismissal of the prime minister is followed by the resignation of the entire government," the government press service said in a statement on its website.
Akhmetov, 55, worked in the metal industry before becoming deputy mayor of Astana, the capital of Kazakhstan.
He served in various government posts and was appointed to the premiership in September 2012, succeeding Masimov, who served five years before him.
HH/HH