How we know what we know.
A classification is only as honest as the evidence behind it. This page documents what Robovations counts as evidence, how we weigh it, and what each robot’s evidence depth label means.
Four depth labels.
Every product in our database carries one of four evidence-depth labels. The label describes how much evidence we’ve reviewed — not how good the robot is.
Pre-release
The product is announced but not yet shipping. No owner data, teardowns, or independent reviews can exist yet, so classification is forward-looking.
- Manufacturer claims only
- No shipping product to verify
- Pre-Release Assessment in place of Score
Pending
The product is shipping, but evidence collection has not yet begun in earnest. Listed in our database while the review queue catches up.
- Classification is placeholder
- Awaiting full evidence pass
Provisional
Enough evidence to publish a working classification, with gaps acknowledged in the product's Evidence Ledger.
- Classification published with caveats
- May change as more sources surface
Verified
Sufficient public evidence across the major source categories to publish a non-provisional classification.
- Cross-checked against owner reports
- Re-evaluated quarterly
Six channels we draw from.
A Verified classification requires evidence across at least five of these six categories. The Evidence Ledger on each product page itemizes which channels have been reviewed.
Manufacturer documentation
Product specifications, user manuals, dock placement guides, and firmware release notes published by the manufacturer.
Authoritative for hardware specs; skeptical of performance claims.
Independent teardown
iFixit, RepairCraft, and similar disassembly reports.
Reveals component choices, repairability, and confirms or refutes manufacturer claims about build.
Press & video reviews
Reviews from established outlets (The Verge, Wirecutter, GearLab, Tom's Guide) and credentialed video reviewers.
We exclude sponsored placements without explicit disclosure.
Owner reports
Aggregated reports from Reddit communities, manufacturer forums, Amazon verified reviews, and Trustpilot.
Useful for real-world reliability, edge cases, and warranty experiences.
Firmware release notes
Vendor changelogs documenting capability changes over time.
Important for tracking what the robot's autonomy actually is at a given firmware version.
Longitudinal signal
Robovations' dated log of how a product's documented behavior has changed over months: the same public channels above, tracked over time rather than at a single review moment.
Owner-reported battery degradation, dock-finding reliability, regression after firmware updates. We aggregate and date-stamp public evidence; we do not operate the robots ourselves.
The line between signal and noise.
- Manufacturer specifications and product manuals (treated as factual for hardware)
- Third-party teardowns with documented methodology
- Press reviews from outlets with editorial independence policies
- Owner reports aggregated across at least 50 unique reporters
- Firmware release notes published by the manufacturer
- Longitudinal aggregation of owner reports spanning ≥ 3 months of documented use
- Sponsored reviews without a "paid placement" disclosure
- Single-data-point claims without context
- Vendor-affiliated blogs without independent editorial oversight
- Press releases used as standalone evidence
- Pre-release demos at trade shows without owner verification
- Marketing materials labeled as reviews
How and when we revisit.
We re-run the evidence review on every Verified product quarterly. The cycle picks up newly-published reviews, firmware drops, recall notices, and shifts in owner-report sentiment.
Three triggers pull a product into early re-evaluation:
- A firmware update that changes a documented capability
- A safety recall or warranty change
- A material shift in owner-report sentiment (tracked via moving averages)
Each early re-evaluation that changes the classification creates a tracker entry, which becomes a new evidence source for the next quarterly cycle.
What we can’t see.
Our evidence has known limits. We document them per-product when they apply.
- English-language bias
- We don’t yet cover non-English owner forums systematically. Robots popular in non-English markets get thinner Owner Reports coverage than they deserve.
- Enthusiast skew
- Owner reports come disproportionately from technically engaged buyers. Casual owners’ experiences are underrepresented.
- Manufacturer reliance for specs
- We treat published hardware specs as factual unless contradicted. A manufacturer that publishes inaccurate specs can mislead our classification until owner reports or teardowns surface the gap.
- Lag time
- Pre-release products spend time in the Pending tier before enough independent evidence accumulates for Provisional or Verified.