Epidemiologic studies suggest that maternal ambient air pollution exposure during critical periods of pregnancy is associated with adverse effects on fetal development. In this work, we introduce new methodology for identifying critical periods of development during post-conception gestational weeks 2-8 where elevated exposure to particulate matter less than 2.5 µm (PM2.5 ) adversely impacts development of the heart. Past studies have focused on highly aggregated temporal levels of exposure during the pregnancy and have failed to account for anatomical similarities between the considered congenital heart defects. We introduce a multinomial probit model in the Bayesian setting that allows for joint identification of susceptible daily periods during pregnancy for 12 types of congenital heart defects with respect to maternal PM2.5 exposure. We apply the model to a dataset of mothers from the National Birth Defect Prevention Study where daily PM2.5 exposures from post-conception gestational weeks 2-8 are assigned using predictions from the downscaler pollution model. This approach is compared with two aggregated exposure models that define exposure as the average value over post-conception gestational weeks 2-8 and the average over individual weeks, respectively. Results suggest an association between increased PM2.5 exposure on post-conception gestational day 53 with the development of pulmonary valve stenosis and exposures during days 50 and 51 with tetralogy of Fallot. Significant associations are masked when using the aggregated exposure models. Simulation study results suggest that the findings are robust to multiple sources of error. The general form of the model allows for different exposures and health outcomes to be considered in future applications. Copyright © 2016 John Wiley & Sons, Ltd.