The *Left Table* shows that, if the denominator (View) is stable, the ratio metric moves proportionally as the numerator (Click) moves, and the uncertainty of the data is easy to estimate, and the scale of uncertainty doesn’t change much.

In the *Right Table*, when the denominator (View) is large enough as shown on the first few rows, ratio (CTR) is very stable with only 1–2% uncertainty. However, if you look at the bottom rows, the ratio can be very sensitive to changes and unstable when the *denominator* is small! When this is the case, it’s better to monitor the Views and Clicks and expect a wide range of scenarios when you make decisions.

## What should we do?

- Set a threshold for minimal acceptable value for
. As the ratio can have great variance when denominator is small, we only trust the ratio when denominator is large enough. If you have to use the ratio metric to make decisions when denominator is small, make sure you report a range that covers the fluctuation.*denominator* - Monitor the actual values (numerator, denominator) that we use for the ratio calculation. Understand the range of the ratio by simulating different scenarios of the numerator and denominator.

Data analytics is not just calculation, it’s also the measurement of uncertainty

While summary statistics like mean or some ratio metrics help us ‘** Zoom Out**’ and see a big picture of data and our business, we also need to ‘

**’ for the range and shape of data, to make sure we understand the uncertainty associated with the metrics.**

*Zoom In*- A data point is NOT enough! Create the
**range**around it, and use variance to estimate uncertainty or different segmentations of the data. - If your metric is ratio like Click-Through Rate, analyze different scenarios to see how the metric change as the denominator and numerator change. Be careful if your denominator is small, which means the ratio can be more sensitive to the change of data, and might not be reliable!
- Visualize it to make sure we don’t miss any pattern in the data or outliers.

Follow me and give me a few claps if you find this helpful 🙂