Khalid Batarfi, a senior al-Qaeda figure who had been held for more than four years, was among more than 300 prisoners who escaped from the jail in Hadramawt province, the official told AFP.
Two prison guards and five inmates were killed in clashes, the official said.
Batarfi is one of al-Qaeda in the Arabian Peninsula's (AQAP) top regional commanders, known for his leading role in a 2011-2012 battle with Yemeni government troops during which extremists seized large parts of the south and east.
Al-Qaeda militants also clashed Thursday with troops guarding the local adminstration complex in the provincial capital Mukalla, a branch of the central bank and the police headquarters, the official said.
Fighting also broke out at the harbour and around a presidential palace in Mukalla, security officials said.
The militants met no resistance as they seized the local radio headquarters, the officials said, adding that broadcasting was interrupted.
Yemen has descended further into chaos since a Saudi-led coalition launched air strikes in the country a week ago.
Observers have warned that Yemen-based AQAP, classified by the United States as the network's deadliest franchise, could exploit the unrest to strengthen its presence in the country