← Read

The collector who wins every segment and still loses

Collector A beats Collector B on small accounts and on large accounts, and still ranks 10 points lower. Simpson's paradox is hiding in most collections leaderboards. Here is the maths, and how to rank people fairly.

Collections Craft · Ep 227 Oct 20266 min read

Every collections team has a leaderboard. Most of them rank people by one number: the share of past-due dollars each collector brought in this month. It looks fair. It is the same formula for everyone.

Here is a leaderboard where the formula is the same for everyone, and the ranking is wrong.

The leaderboard

Two collectors, the same month, the same size of book: $600k past due at the start of the month each. Illustrative numbers, as always.

Past due ($k) Collected ($k) Collection rate
Collector A 600 390 65%
Collector B 600 450 75%

B leads by 10 points. B gets the recognition, and probably the next promotion conversation.

Now split it by segment

Both books hold two kinds of account: small accounts, which mostly pay once someone asks, and large accounts, where payment runs through approval chains, disputes and procurement.

Small accounts Large accounts
Collector A 90 of 100 = 90% 300 of 500 = 60%
Collector B 340 of 400 = 85% 110 of 200 = 55%

A beats B on small accounts by 5 points. A beats B on large accounts by 5 points. B still wins overall by 10.

Nothing is miscalculated. This is Simpson’s paradox.

What Simpson’s paradox is

Simpson’s paradox is what happens when a pattern holds inside every group of your data, and then reverses when you add the groups together. It is named after the statistician Edward Simpson, who described it in 1951, though Karl Pearson and Udny Yule had noticed the effect around 1900. It is not a quirk of bad data. It follows from how averages work.

The most famous case comes from medicine. A 1986 study in the British Medical Journal compared two treatments for kidney stones:

Small stones Large stones All patients
Treatment A 93% (81 of 87) 73% (192 of 263) 78% (273 of 350)
Treatment B 87% (234 of 270) 69% (55 of 80) 83% (289 of 350)

Treatment A worked better on small stones and better on large stones, yet looked worse overall. The reason: doctors sent the large stones, the hard cases, mostly to Treatment A. A’s overall number was dragged down by the difficulty of its patients, not by the treatment.

Another well-known case is the University of California, Berkeley, in 1973. Overall, 44% of male applicants to its graduate programmes were admitted against 35% of female applicants, which looked like bias against women. Broken down by department, the picture changed: women had mostly applied to the more competitive departments, and once that was accounted for, the data showed a small bias in their favour.

The pattern is always the same. Two groups are compared on one overall rate, but they took on different mixes of easy and hard cases. The overall rate then measures the mix as much as the performance.

Now look at the collectors again. Collector A is Treatment A: better on every kind of account, and handed most of the hard ones.

Why both can be true

An overall rate is a weighted average of the segment rates, and the weights are the shape of the book, not the skill of the collector.

Collector A:  1/6 × 90%  +  5/6 × 60%  =  65%
Collector B:  2/3 × 85%  +  1/3 × 55%  =  75%

Five-sixths of A’s book is large accounts, the hard ones. Two-thirds of B’s book is small accounts, the easy ones. B’s 10-point lead is the book, not the collector. On equal work, A is better.

The paradox needs two ingredients, and collections has both in abundance:

  1. Segments with very different base rates. Small accounts at 85–90%, large accounts at 55–60%.
  2. Uneven mix across the people or periods you compare. Books are rarely assigned at random. They follow regions, industries, languages, or whoever had capacity when the account arrived.

Account size is only one way to split. The most common version in practice is aging mix: a collector who inherits a book of 90+ day invoices will lose to one holding mostly 1–30 day invoices, whatever their skill. Risk grade, region and customer industry all work the same way.

Compare people on the same book

There are two simple fixes, and both use numbers you already have.

1. Mix-adjusted rate. Apply each collector’s segment rates to one common mix, the team’s. The team’s book is $500k small and $700k large, so 5/12 small and 7/12 large:

Collector A:  5/12 × 90%  +  7/12 × 60%  =  72.5%
Collector B:  5/12 × 85%  +  7/12 × 55%  =  67.5%

On the same book, A leads by 5 points, exactly the gap in every segment.

2. Actual against expected. Ask what an average collector would have brought in from this collector’s book, using the team’s segment rates (86% on small, 58.57% on large), and compare.

Expected ($k) Actual ($k) Against expected
Collector A 100 × 86% + 500 × 58.57% = 378.9 390 +11.1 (103%)
Collector B 400 × 86% + 200 × 58.57% = 461.1 450 −11.1 (98%)

A beat the book by $11.1k. B fell short of it by the same amount. That is the leaderboard you want: it rewards what the collector did, not what they were handed.

It happens to whole teams too

The same arithmetic catches teams from one month to the next.

Small accounts Large accounts Overall
September 430 of 500 = 86.0% 410 of 700 = 58.6% 840 of 1,200 = 70.0%
October 264 of 300 = 88.0% 540 of 900 = 60.0% 804 of 1,200 = 67.0%

Both segments improved. The team’s overall rate fell 3 points, because October’s past due shifted towards large accounts. A quarter-end push from a few big customers is enough to do it.

A manager looking only at the headline would start a performance conversation with a team that got better at both kinds of account. The same trap sits inside CEI and inside DSO, which is why The DSO Bridge splits a move into causes before anyone is blamed.

Before you rank anyone

  1. Rank within segments first. Account size, aging bucket, risk grade, whichever drives the biggest difference in base rates on your book.
  2. Report a mix-adjusted rate alongside the raw one, using the team’s mix as the common weight.
  3. Score collectors against expected, not against each other. “Collected 103% of what an average collector would have on this book” is a fair sentence. “Collected 65%” is not.
  4. Flag disagreement. If the overall ranking and the segment rankings point in different directions, you are looking at the paradox. Do not publish the leaderboard until you have looked.
  5. Fix the cause, not just the report. If books are badly unbalanced, rotate or rebalance them, or every comparison will keep needing a correction.

There is a second-order reason to care. If the raw leaderboard drives recognition, collectors learn quickly which accounts make the number look good. Large, slow, complicated accounts become the ones nobody wants, and those are usually the ones that matter most to cash.

The fix costs one extra column on the leaderboard. The question is whether yours would survive a segment split.

About the author

Abhishek Sahu

Order to Cash Product Manager

Senior product manager with more than six years in order-to-cash, across HighRadius, Deloitte and EXL. He has built products for collections, cash application, deductions and O2C analytics, and holds three US patents in AI-driven financial automation.

Collections Craft
  1. Collections Craft: five levers that move your days past due
  2. The collector who wins every segment and still loses