How we know what we know.
A classification is only as honest as the evidence behind it. This page documents what Robovations counts as evidence, how we weigh it, and what each robot’s evidence depth label means.
On this page5 sections
Four depth labels.
Every product in our database carries one of four evidence-depth labels. The label describes how much evidence we’ve reviewed, not how good the robot is.
Pre-release
The product is announced but not yet shipping. Manufacturer material and dated announcement coverage can exist and are logged; owner reports, teardowns, firmware history, and longitudinal data cannot exist yet, so the classification is forward-looking.
- Manufacturer claims and announcement coverage only
- No shipping product to verify
- Pre-Release Assessment in place of Score
Pending
The product is shipping, but evidence collection has not yet begun in earnest. Listed in our database while the review queue catches up.
- Classification is provisional: the rung is assigned from maker documentation and revised when independent coverage arrives
- Awaiting full evidence pass
Provisional
Enough evidence to publish a working classification, with gaps acknowledged in the product's Evidence Ledger.
- Classification published with caveats
- May change as more sources surface
Verified
Enough public evidence to publish a non-provisional classification: five or more reviewed sources that neither this site nor the maker hosts. The Ledger on each product page shows exactly which channels contribute, so you can judge the breadth yourself rather than take the tier for it.
- At least five sources, none of them ours and none hosted by the maker
- Breadth across channels is itemized, not required
- Re-evaluated quarterly
Depth is not lifecycle. A robot also carries a lifecycle state (shipping, discontinued, or not sold to consumers), which you will see in the database Status filter. Lifecycle describes whether you can buy the robot; depth describes how much evidence stands behind its classification. A discontinued robot can be Verified; a shipping robot can be Pending. The two are recorded separately and neither implies the other.
Methodology note, September 14, 2026. The count now excludes sources the maker hosts. Product pages, support portals, firmware release notes and specification sources stay in the Ledger but no longer count toward the tier. 182 records moved down a tier that day: Verified went from 174 to 59, Provisional from 138 to 170, and Pending from 29 to 112.
Eight channels we draw from.
A Verified classification requires at least five reviewed sources that neither this site nor the maker hosts, drawn from these eight channels. It is a count of sources, not a requirement to fill every channel, because some channels do not exist for every machine. Three channels are the maker’s own (manufacturer documentation, firmware release notes, specification sources); their links are listed and not counted. The Evidence Ledger on each product page itemizes exactly which sources were reviewed and in which category, so the figure is checkable by hand.
Manufacturer documentation
Product specifications, user manuals, dock placement guides, and firmware release notes published by the manufacturer.
Authoritative for hardware specs; skeptical of performance claims.
Independent teardown
iFixit, RepairCraft, and similar disassembly reports.
Reveals component choices, repairability, and confirms or refutes manufacturer claims about build.
Press & video reviews
Reviews from established outlets (The Verge, Wirecutter, GearLab, Tom's Guide) and credentialed video reviewers.
We exclude sponsored placements without explicit disclosure.
Owner reports
Aggregated reports from Reddit communities, manufacturer forums, Amazon verified reviews, and Trustpilot.
Useful for real-world reliability, edge cases, and warranty experiences.
Firmware release notes
Vendor changelogs documenting capability changes over time.
Important for tracking what the robot's autonomy actually is at a given firmware version.
Longitudinal signal
Robovations' dated log of how a product's documented behavior has changed over months: the same public channels above, tracked over time rather than at a single review moment.
Owner-reported battery degradation, dock-finding reliability, regression after firmware updates. We aggregate and date-stamp public evidence; we do not operate the robots ourselves.
Specification sources
The page each published specification was read from, one URL per figure, written by the specs resolver.
Usually the maker's own page, so it is listed but does not count toward the tier.
Monitoring record
The sources cited by the Tracker entries that follow this robot: recalls, firmware changes, demos, and assessment changes.
Counts toward the tier when the publisher is independent of the maker.
The line between signal and noise.
- Manufacturer specifications and product manuals (treated as factual for hardware)
- Third-party teardowns with documented methodology
- Press reviews from outlets with editorial independence policies
- Owner reports aggregated across at least 50 unique reporters
- Firmware release notes published by the manufacturer
- Longitudinal aggregation of owner reports spanning ≥ 3 months of documented use
- Sponsored reviews without a "paid placement" disclosure
- Single-data-point claims without context
- Vendor-affiliated blogs without independent editorial oversight
- Press releases used as standalone evidence
- Pre-release demos at trade shows without owner verification
- Marketing materials labeled as reviews
How and when we revisit.
We re-run the evidence review on every Verified product quarterly. The cycle picks up newly-published reviews, firmware drops, recall notices, and shifts in owner-report sentiment.
Three triggers pull a product into early re-evaluation:
- A firmware update that changes a documented capability
- A safety recall or warranty change
- A material shift in owner-report sentiment (tracked via moving averages)
Each early re-evaluation that changes the classification creates a tracker entry, which becomes a new evidence source for the next quarterly cycle.
What we can’t see.
Our evidence has known limits. We document them per-product when they apply.
- English-language bias
- We don’t yet cover non-English owner forums systematically. Robots popular in non-English markets get thinner Owner Reports coverage than they deserve.
- Enthusiast skew
- Owner reports come disproportionately from technically engaged buyers. Casual owners’ experiences are underrepresented.
- Manufacturer reliance for specs
- We treat published hardware specs as factual unless contradicted. A manufacturer that publishes inaccurate specs can mislead our classification until owner reports or teardowns surface the gap.
- Lag time
- Pre-release products spend time in the Pending tier before enough independent evidence accumulates for Provisional or Verified.