Calculator
Practical-Effect Calculator
By Jason Hreha
A “7% improvement” sounds useful. But a percentage alone does not tell you how many people were affected or what happened to them.
This calculator turns the difference between two rates into a count for a group of any size. If you know the added cost, it also works out the cost of each extra completion or avoided problem. Before you start, be clear about which result you are counting: a completed application, an attended appointment and a person still active a month later are not the same thing.
The calculator works with results that can be counted as yes or no: did someone complete the application, for example? It puts a reported difference in practical terms. It cannot tell you whether a study is credible, whether the change caused the difference, or whether the change is worth making.
What would this difference mean?
Suppose the share of people who complete an application rises from 20% to 21.4%. You could report a 7% relative increase, or 1.4 extra completions per 100 applicants. Both are correct. The first compares the increase with the original 20% rate; the second shows what it means among 100 applicants.
Enter the rate for a comparison group (the baseline) and the rate for the group that received the program or change being studied (the intervention). Then enter the size of the group you care about. You will see how many more or fewer results those rates would mean for that group. You still need sound evidence to judge whether the program caused the difference.
Your entries stay in this browser tab. This tool does not save or send them. Reloading the page clears them.
Name something that either happened or did not happen, such as “application completed.”
Are the percentages about people, applications, sessions, or something else? Ten appointments may involve fewer than ten people.
Use the same rule for both rates and for your group size. For example, if the study counted first-time applicants, count only first-time applicants here.
Within 30 days of receiving the form, for example. Both rates must cover the same period.
Enter 20 for 20%, without the % sign. Use the rate for the comparison group, such as people who received the usual email instead of the new one. The baseline does not have to come from an earlier date.
Enter the rate for the group that received the program or change, such as 21.4, without the % sign. Count the same result in the same way as you did for the comparison group.
Enter a whole number without commas, such as 10000. It does not have to match the number in the study.
How much more would the intervention cost for this group and period? Enter digits only, such as 7000, with no commas or currency symbol. Leave blank if unknown. Enter 0 only if there is no additional cost.
For example: USD, GBP, or EUR. Required if you enter a cost.
The calculator cannot check this. Ten sessions are not ten different people. If the measures differ, correct them before comparing.
Your scenario
A worked example: 20% to 21.4%
In this made-up example, eligible applicants receive either the usual form or a new one. With the usual form, 20% complete an application within 30 days. With the new form, 21.4% do. The difference is 1.4 percentage points (21.4 − 20). Relative to the original 20% rate, that is a 7% increase (1.4 ÷ 20 × 100).
If those rates applied to 10,000 eligible applicants, you would expect 140 more completed applications (10,000 × 0.014). Suppose the new form costs USD 7,000 more for that group and period. That works out to USD 50 per additional completion. To try this example in the calculator, enter 20, 21.4, 10000 and 7000, and type USD in the currency field.
Would you pay USD 50 for one more completed application? Could you get more completions for the same money another way? Neither the 7% figure nor the 1.4-point figure answers those questions. Each is a way to state the effect size: how large the difference is. Whether a difference that size is worth its cost is a question of practical significance. You would also need to know whether the people who completed an application later used the service.
How the calculation works
- Difference in percentage points = intervention rate − baseline rate.
- Relative change = difference in percentage points ÷ baseline rate × 100. If the baseline is zero, you cannot divide by it, so a relative change cannot be calculated.
- Expected change in the number of results = group size × difference in percentage points ÷ 100. A negative answer means fewer results.
- If you want fewer missed appointments, a lower rate is better. In that case, a drop of 2 percentage points counts as an improvement, even though the difference between the rates is −2.
- Cost per added or avoided result = additional cost ÷ expected improvement. An improvement can mean one more wanted event, such as a completed application, or one fewer unwanted event, such as a missed appointment. This cost cannot be calculated when there is no improvement, the group size is zero, or no cost is entered.
If the calculation gives 2.5 extra completions, that is an expected count based on the rates you entered; it does not mean that half a person completed something. Results are rounded. Very small or very large numbers use a compact form called scientific notation. For example, 1.00000e-6 means 0.000001. This lets the calculator display a tiny change without rounding it to zero.
The calculator uses the numbers you enter; it cannot tell you how reliable they are. To make a decision, you need to know how the study compared the groups, how much the estimate could vary (a confidence interval shows this), and whether the result lasted long enough to matter. A small difference can help many people when a program reaches a large group. But multiplying the numbers cannot fix a misleading comparison, and more sign-ups still do not tell you how many people kept taking part. The behavioral economics examples apply the same arithmetic to real studies. Try the three evidence exercises to practice keeping those claims separate.