The attack took place near an army checkpoint close to the Belhaf gas export terminal, which loads liquefied natural gas (LNG) to South Korea, Yemen's biggest customer, and to European companies.
Attacks on pipelines feeding the port have been regular and interrupted exports earlier this year.
The militants, who were in two cars, surrounded the policemen's vehicle and opened fire, killing all of them, the residents said.
The gunmen were believed to be members of al-Qaeda in the Arabian Peninsula, they said.
Shabwa Province, a lawless and rugged part of Yemen, has seen much fighting in recent years between militants and the security forces.
RA/NJF