Demonstrators demanded federal charges to be brought against Zimmerman, 29, over the February 2012 incident.
A Florida jury agreed that the neighborhood watch volunteer killed the 17 year-old in self-defense.
The protests against the court's decision were led by the National Action Network, headed by civil rights activist the Reverend Al Sharpton.
"We are not coming out with violence, we are coming to denounce violence. The violence that was perpetrated on an unarmed, innocent man named Trayvon Martin," Mr Sharpton told at a rally in New York.
Sharpton told supporters on Saturday morning that he wanted to see the scrapping of "stand your ground" self-defense laws, such as that in force in Florida.
"We are trying to change laws so that this never, ever happens again," he said.
The teenager's mother, Sybrina Fulton, told the crowd: "Today it was my son. Tomorrow it might be yours."
Thousands also gathered for "Justice for Trayvon" protests in at least 100 cities across America, including Atlanta, Chicago, Los Angeles and Miami.
In Miami, Trayvon Martin's father, Tracy Martin, was among those who gathered to call for fair trial of murderer of his black boy.
What happened to Trayvon
The deadly encounter took place on February 26, 2012, as Martin walked back to his father's fiancee's house through the rain from a Sanford convenience store. The 17-year-old was carrying Skittles and a drink.
Zimmerman, a neighborhood watch volunteer, spotted him and called police.
A 911 dispatcher told Zimmerman that officers were on the way and not to follow the allegedly suspicious person.
Nonetheless, Zimmerman got out of his car, later telling police he just wanted to get a definitive address to relay to authorities.
Sometime after that, Zimmerman and Martin got into a physical altercation. Some neighbors took notice: On one 911 call, anguished cries for help can be heard.
Who was yelling? Martin's mother testified she's "absolutely" sure it was her son; Zimmerman's parents said, with as much conviction, that it was their own child.
There are also disputes about who was the aggressor, and some also accuse Zimmerman -- who identifies himself as Hispanic -- of racially profiling the African-American teenager.
The prosecution argue that he wrongly and spitefully prejudged Martin as one of those "f***ing punks," as he's heard saying under his breath in his call to police, who had pulled off crimes in his neighborhood.
"The defendant didn't shoot Trayvon Martin because he had to," Assistant State Attorney John Guy said. "He shot him because he wanted to. That's the bottom line."
The defense contended that Martin jumped out of some bushes and pounced on Zimmerman as he was walking back toward his car that night, then punched him and slammed his head repeatedly into the concrete sidewalk.
NTJ/SHI