Human Rights Watch said in a Tuesday report that top former officials -- including Saleh's relatives, such as his nephew -- should be held responsible for the slayings.
The New York-based group was referring to the March 18, 2011 incident, when the government unleashed security forces on the demonstrators in the capital, Sana’a.
Saleh stepped down last year, following yearlong protests against his rule.
HRW's senior researcher Letta Tayler warned that "if Yemen doesn't fairly investigate and prosecute those responsible for this deadly attack, it risks perpetuating the culture of impunity at the heart of Yemen's uprising."
"Human Rights Watch found that several senior former and current government officials appear to have played a role in the massacre but have not been charged," the report said.
HRW said its report was based on more than 60 interviews with witnesses, victims of the shooting, lawyers, government officials, human rights defenders, and journalists.
President Abed Rabbo Mansour Hadi, who promised sweeping reforms and accountability for the abuses of the past removed Yahya Saleh from his post in December, but without disciplinary proceedings due to a sweeping immunity law that the Yemeni parliament passed last year in exchange for Saleh's resignation. The law granted Saleh and all those who served with him, immunity from prosecution for all crimes except acts of terrorism.
The report said 43 of the 78 suspects whom prosecutors indicted in June 2011 were listed as fugitives from justice, but 31 of them were never apprehended and the other 12 disappeared after they were provisionally released pending the outcome of the trial. 27 defendants were released on bail.
In October, the victims' lawyers filed a motion in court demanding the indictment of at least 11 additional government officials for the shootings, including Saleh, his nephew and former interior minister. The trial judge in the case however sent the motion to the Supreme Court to investigate its validity in light of the immunity law. The trial has been suspended waiting for the Supreme Court decision.
HRW also called on the UN Security Council, Persian Gulf Cooperation Council, European Union, United States and other concerned countries to impose an asset freeze and travel ban on current and former officials implicated in the attack and other serious human rights violations.