Occam’s razor and the law of diminishing data returns

· 3AG blog,analytics

Most people have heard of the law of diminishing returns, but many don’t know what it means. Wikipedia defines diminishing returns as:

the decrease in the marginal (incremental) output of a production process as the amount of a single factor of production is incrementally increased, while the amounts of all other factors of production stay constant.

In layman’s terms, nothing lasts forever and/or it is possible to have too much of a good thing. In the business world, the law of diminishing returns looks something like this: As you put more effort into an initiative, build a bigger factory, or work longer hours, instead of being proportionately more successful initiatives will begin to fail and you’ll eventually hit a wall.

cat hitting the wall

What does this mean for data in your operations? The same pattern applies: There is a limit to how much additional insight you can get by adding more data to your analyses or reports. This is counterintuitive, so worth repeating: Adding more data might not help improve your organization’s data analytics. In fact, it might actively undermine analytics usefulness and accuracy.

The question then is, how do you know when you have enough data to perform useful, robust analyses, but not too much?

Born and raised in a world of data, we at 3AG have some ideas on how to deal with this tricky balancing act. And we will spend the rest of this article discussing how you can be sure you hav