2 Quotes by Christopher d. manning

With limited training data, a more constrained model tends to perform better.

In principle, more analytic power can be achieved by varying multiple things at once in an uncorrelated (random) way, and doing standard analysis, such as multiple linear regression. In practice, though, A/B testing is widely used, because A/B tests are easy to deploy, easy to understand, and easy to explain to management.