Given how well Professor Complexicus does in explaining the time patients needed to recover in the past, it seems intuitive that his estimations should also fare better than those of Doctor Heuristicus when it comes to predicting future patients’ time to recover. Yet this is not necessarily the case. Goodness-of fit measures alone cannot Inhibitors,research,lifescience,medical disentangle the variation in the observations due to the relevant variables from the variation

due to random error, or noise. In fitting past observations, models can end up taking into account such noise, thus mistakenly attributing meaning to mere chance. As a result, a model can end up overfitting these observations. (Figure 5). illustrates a corresponding situation in which one model, Model A (thin line) overfits already existing, past observations (filled see more circles; eg, old patients) by chasing after noise in those observations. As can Inhibitors,research,lifescience,medical be seen, this model fits the past observations perfectly but does a relatively poor job of predicting new observations (filled triangles; eg, new patients). Model B (thick line), while not fitting the past observations as well as Model A, captures the main trends in the data

and ignores the noise. This makes it better equipped Inhibitors,research,lifescience,medical to predict new observations, as can be seen from the deviations between the model’s predictions and the new observations, which are indeed smaller than the deviations for Model A. Figure 5. Illustration of how two models fit past observations (filled circles) and how they predict new obsen/ations (triangles). The complex Model A (thin line) overfits

the past observations and is not as accurate in predicting the new observations as the simple … Importantly, the degree to which a model Inhibitors,research,lifescience,medical is susceptible to overfitting is related to the model’s complexity. One factor that contributes to a model’s complexity is its number of free parameters. As is illustrated Inhibitors,research,lifescience,medical in Figure 5, the complex, information-greedy Model A overfits past observations; Model B, in turn, which has fewer free parameters and which takes into account less information, captures only the main trends in the past observations, but better predicts the new observations. The same is likely to hold others true with respect to Professor Complexicus’ and Doctor Heuristicus’ strategies: Professor Complexicus’ complex strategy is likely to be more prone to overfitting past observations than Doctor Heuristicus’ simple one. As a result, Dr. Heuristicus’ strategy is likely to be better able to predict new observations than Professor Complexicus’ strategy. In short, when data are not completely free of noise, increased complexity (eg, integrating as much information as possible) makes a model more likely to end up overfitting past observations, while its ability to predict new ones decreases (although see Box 4).