Check the data work of humans and AI alike for the pitfalls that mislead.
Get back what is wrong, why it matters, and how to fix it.
Pro subscription required
The Pitfall Detector is available to Pro subscribers.
Pro includes
- • Every course in the library
- • AI Coach for personalized guidance
- • AI Tutor on every lesson
- • Pitfall Detector in the Lab
- • Badges and shareable verification
$29 / month or $290 / year. 7-day money-back guarantee.
Already a Pro member? Sign in
What you get back
A verdict, not a score. Your report reads like notes from a careful reviewer, in this order:
- 1
Verdict and summary. One of four tiers, a sentence or two, and a count of what was found.
- 2
Start here. The single most important finding, in full.
- 3
Detected pitfalls. Problems evident from the work itself.
- 4
Potential pitfalls. Risky patterns that are only a problem if something is true of data the tool can’t see.
- 5
Pitfalls avoided. Good practices the work already shows.
The four verdicts
- No pitfalls detected
- Conditions to verify
- Needs attention
- Serious pitfalls found
Each finding says how much it matters
The chart's message is clear, but the axis and the figure sizes exaggerate the height differences between countries. The most important fix is to use a zero-based axis with evenly spaced ticks, or to use plain bars or dots instead of scaled pictograms.
4 detected, 0 potential
A real report on the average female height sample, saved so it costs nothing to show.
Start here
Truncated Y-Axis
Readers will see India as a fraction of Latvia's height, when the real gap is only a few inches. The baseline hides the fact that the heights are all fairly close.
- Weakens supportDisproportionate Encoding
- Weakens supportUnevenly Spaced Axis
- PolishInconsistent Color Encoding
See it on real charts
Real published charts. Open one to read what the detector said about it. These are saved reports, so viewing them doesn’t use a scan.
Five ways to check
Chart image
Up to 4 images, scanned together to catch pitfalls across charts.
Written analysis
Pasted prose, or a PDF, Word, text or Markdown file.
Slide deck
A .pptx file, checking each slide’s text and charts.
Code
Python, SQL, R and more, pasted or uploaded.
Full analysis
Up to 10 pieces of one analysis, checked together for pitfalls that only show up across steps.
78 rules in eight domains
Every finding is tied to a named rule in the taxonomy from Avoiding Data Pitfalls.
01
Epistemic Errors
Mistaking the data for reality.
02
Technical Trespasses
Mishandling the data itself: dropped rows, broken joins.
03
Mathematical Miscues
Calculation slip-ups: mixed units, unweighted averages of rates.
04
Statistical Slip-Ups
Comparison and inference: an average taken as typical, small samples.
05
Analytical Aberrations
Flawed analysis: extrapolating too far, metrics that flatter the story.
06
Graphical Gaffes
Charts that mislead: truncated axes, wrong chart type, cherry-picked windows.
07
Design Dangers
Design choices that trip up readers: colorblind-unsafe palettes, inconsistent colors.
08
Biased Baseline
Who or what the data leaves out.
Built on datapitfalls
The Pitfall Detector runs on datapitfalls, an open-source engine based on Ben Jones’s Avoiding Data Pitfalls (Wiley). Claude does the analysis, and every finding is tied to a named rule in the book’s taxonomy.
Available on Pro, and to teams with AI features turned on. 25 scans per person per day.