Identification of gene expression traits unique to the human brain sheds light on the mechanisms of human cognition. Here we searched for gene expression traits separating humans from other primates by analyzing 88,047 cell nuclei and 422 tissue samples representing 33 brain regions of humans, chimpanzees, bonobos, and macaques. We show that gene expression evolves rapidly within cell types, with more than two-thirds of cell type-specific differences not detected using conventional RNA sequencing of tissue samples. Neurons tend to evolve faster in all hominids, but non-neuronal cell types, such as astrocytes and oligodendrocyte progenitors, show more differences on the human lineage, including alterations of spatial distribution across neocortical layers.