Thani quits less than a week after parliament tasked him with forming a new cabinet and a month after his predecessor was ousted for failing to rein in the lawlessness gripping the North African country.
He said in a statement that he would not accept the premiership after a "traitorous attack" on himself and his family, but he would stay on in a caretaker capacity until a new prime minister is appointed.
Amid controversy over his appointment, Thani, who was defense minister under ousted premier Ali Zeidan, was named on Tuesday.
"I will not accept Libyans killing each other over this post," Thani said in his statement addressed to the General National Congress (GNC).
Thani said the attack on Saturday had terrorized inhabitants of a residential district and "put the lives of some of them at risk", without giving specific details.
A source close to Thani told AFP that the incident took place on the road from the capital to its airport and caused no casualties.
As premier, Thani was faced with the daunting task of bringing former rebel brigades to heel following the 2011 NATO-backed uprising that ended Moamer Kadhafi's four-decade rule.
Libya has seen near daily attacks, particularly in the restive east, as well as a challenge from rebels who blockaded vital oil terminals for nine months, and a growing political crisis stemming from the interim parliament's decision to extend its mandate.
Omar Hmidan, a spokesman for the GNC, the country's highest political authority, said Thani was chosen after legislators failed to reach a consensus on other candidates.
He was given one week to form a new government.
SHI/SHI