My own tool was wrong. So I overruled it.
A tool is only as good as the benchmark you point it at. This is the story of a tool I built telling me to spend a whole design cycle on the wrong product — and how questioning one number sent that budget to the product that actually needed it.
The verdict
A tool I built to rank products by creative-rebuild priority flagged one product as the clear #1 — its conversion looked low, so "fix the images" seemed obvious.
The catch
It was scoring conversion against a blended average across the whole catalog. But a supplement and a phone case don't share a conversion bar. The right question isn't "how does it convert versus everything?" — it's "how does it convert versus its own market?"
The flip
Re-scored against real market conversion, the verdict reversed: the product actually out-converts its category. It didn't have a creative problem — it had a visibility problem. Rebuilding its images would have burned a full cycle fixing something that wasn't broken.
What I did
The design budget went to the product that genuinely under-converted its market. The flagged product got what it actually needed — more traffic — not a redesign.
Why this matters to you
Tools hand you outputs. The money is in the judgment to overrule them. I build the systems and the operator sense to know when they're pointed at the wrong number — so your budget lands on the real constraint, not the obvious one.
Anonymized on purpose — the method is the point, not a client's private data.