Contact tracing is an important control strategy for containing Ebola epidemics. From a modeling perspective, explicitly incorporating contact tracing with disease dynamics presents challenges, and population level effects of contact tracing are difficult to determine. In this work, we formulate and analyze a mechanistic SEIR type outbreak model which considers the key features of contact tracing, and we characterize the impact of contact tracing on the effective reproduction number, Re, of Ebola. In particular, we determine how relevant epidemiological properties such as incubation period, infectious period and case reporting, along with varying monitoring protocols, affect the efficacy of contact tracing. In the special cases of either perfect monitoring of traced cases or perfect reporting of all cases, we derive simple formulae for the critical proportion of contacts that need to be traced in order to bring the effective reproduction number Re below one. Also, in either case, we show that Re can be expressed completely in terms of observable reported case/tracing quantities, namely Re = k((1-q)/q)+km where k is the number of secondary traced infected contacts per primary untraced reported case, km is the number of secondary traced infected contacts per primary traced reported case and (1-q)/q is the odds that a reported case is not a traced contact. These formulae quantify contact tracing as both an intervention strategy that impacts disease spread and a probe into the current epidemic status at the population level. Data from the West Africa Ebola outbreak is utilized to form real-time estimates of Re, and inform our projections of the impact of contact tracing, and other control measures, on the epidemic trajectory.