The protesters marched from different spots towards downtown Kuala Lumpur amid tightened security measures on Saturday, chanting "Save Democracy" and "Bersih, Bersih," which means “clean” in Malaysian.
Najib is implicated in a multi-billion dollar misappropriation scandal, including about $700 million transferred to his private accounts by the Saudi royal family.
The protest came after the head of pro-democracy group Bersih was arrested on Friday, along with several opposition leaders and student activists.
"Our country is being governed by clowns and crooks.
So I'm here to protest against our prime minister," said artist Fahmi Reza, carrying a poster of a clown-faced Najib.
"We want to see Malaysia more developed and not robbed of billions of ringgit," another protester, Wan Aisyah Wan Ariffin, said.
This is the second time in 15 months that huge crowds have turned out to vent anger over allegations that billions of dollars were looted from Najib's brainchild investment fund, 1MDB.
The scandal has gripped the country for more than a year and sparked investigations in several countries.
Lawsuits filed by the US Justice Department in July said over $3.5 billion was stolen from 1MDB, which was founded by Najib, and that some of those funds flowed into the accounts of an unnamed top Malaysian official.
A Malaysian Cabinet official has since admitted that individual was Najib.
Najib last year abruptly shut down Malaysian investigations, fired the attorney general and shored up his position by purging ruling-party critics.
Judiciary authorities later cleared him of any criminal wrongdoing in the case, saying the money was from donations from the Saudi royal family.
Critics have dismissed the ruling, arguing that the transfer of personal donations did not rule out corruption.
SA - 12