A vehicle carrying the men was hit overnight as it travelled through Wadi Dikha in Abyan province, the official said, after it had set off from the southern city of Mukalla, a bastion of Al-Qaeda.
Five people, including a local Al-Qaeda chief named as Abu Ahmad al-Kazimi, died in the strike, according to the official.
The US is the only country known to operate armed drones over Yemen, and attacks by the unmanned aircraft have continued during months of clashes between pro- and anti-government forces and air strikes from a Saudi-led coalition.
Al-Qaeda in the Arabian Peninsula (AQAP), considered by Washington to be the extremist network's most dangerous franchise, announced the death in June of its Yemen commander Nasir al-Wuhayshi in a US drone attack.
AQAP was behind several plots against Western targets and claimed the January massacre at French satirical magazine Charlie Hebdo.
The group has taken advantage of the chaos gripping Yemen to make territorial gains in the south, including taking control of Mukalla, the capital of the vast desert Hadramawt province.