Overfitting

Given a hypothesis space H, a hypothesis h element of H is said to overfit the training data if there exists some alternative hypothesis h' element of H, such that h has smaller error than h' over the training examples, but h' has a smaller error than h over the entire distribution of instance.