In general, exhaustive search is the only technique guaranteed to find the predictor variable subset with the best evaluation criterion. It is often the ideal technique when the number of possible predictor variables is less than 20; note that this number depends to some degree on the computational complexity of evaluating a predictor variable subset.

The problem with exhaustive search is that it is often a computationally intractable technique for more than 20 possible predictor variables. As Figure 3 shows, every possible subset of N predictor variables has a unique bit representation that uses N bits. It is easy to see that there are 2N – 1 possible subsets, excluding the empty set, and exhaustive search must check them all. For regression models with 25 predictor variables, exhaustive search must check 33,554,431 subsets, and this number doubles for each additional predictor variable considered. Clearly, exhaustive search is not always a practical technique, and other selection techniques may have to be considered.