Phylogenetic comparative methods have become standard for investigating evolutionary hypotheses, including in studies of human evolution. While these methods account for the non-independence of trait data due to phylogeny, they often fail to consider intraspecific variation, which may lead to biased or erroneous results. We assessed the degree to which intraspecific variation impacts the results of comparative analyses by investigating the "social brain" hypothesis, which has provided a framework for explaining complex cognition and large brains in humans. This hypothesis suggests that group life imposes a cognitive challenge, with species living in larger social groups having comparably larger neocortex ratios than those living in smaller groups. Primates, however, vary considerably in group size within species, a fact that has been ignored in previous analyses. When within-species variation in group size is high, the common practice of using a mean value to represent the species may be inappropriate. We conducted regression and resampling analyses to ascertain whether the relationship between neocortex ratio and group size across primate species persists after controlling for within-species variation in group size. We found that in a sample of 23 primates, 70% of the variation in group size was due to between-species variation. Controlling for within-species variation in group size did not affect the results of phylogenetic analyses, which continued to show a positive relationship between neocortex ratio and group size. Analyses restricted to non-monogamous primates revealed considerable intraspecific variation in group size, but the positive association between neocortex ratio and group size remained even after controlling for within-species variation in group size. Our findings suggest that the relationship between neocortex size and group size in primates is robust. In addition, our methods and associated computer code provide a way to assess and account for intraspecific variation in other comparative analyses of primate evolution.