Many so-called pathogenic bacteria such as Neisseria meningitidis, Haemophilus influenzae, Staphylococcus aureus and Streptococcus pneumoniae are far more likely to colonize and maintain populations in healthy individuals asymptomatically than to cause disease. Disease is a dead-end for these bacteria: virulence shortens the window of time during which transmission to new hosts can occur and the subpopulations of bacteria actually responsible for disease, like those in the blood or cerebral spinal fluid, are rarely transmitted to new hosts. Hence, the virulence factors underlying their occasional pathogenicity must evolve in response to selection for something other than making their hosts sick. What are those selective pressures? We address this general question of the evolution of virulence in the context of phase shifting in N. meningitidis, a mutational process that turns specific genes on and off, and, in particular, contingency loci that code for virulence determinants such as pili, lipopolysaccharides, capsular polysaccharides and outer membrane proteins. We use mathematical models of the epidemiology and the within-host infection dynamics of N. meningitidis to make the case that rapid phase shifting evolves as an adaptation for colonization of diverse hosts and that the virulence of this bacterium is an inadvertent consequence of short-sighted within-host evolution, which is exasperated by the increased mutation rates associated with phase shifting. We present evidence for and suggest experimental and retrospective tests of these hypotheses.