How our price data is collected and verified

Most vendors describe their data collection as a black box. We publish ours, because the only way to judge a price monitoring tool before you buy it is to understand how it fails.

1. Which pages we fetch

We fetch publicly accessible product pages and marketplace listings. We do not create accounts, log in, bypass access controls, or solve challenges designed to keep automated clients out. We identify our crawler, respect rate limits, and back off when a site signals that it is under load.

Which URLs get fetched comes from three sources: what you import from your store or CSV, what we discover by looking up product identifiers, and what you add by hand. Every tracked URL is attached to a product in your catalogue, so there is no crawling that is not doing work for a specific customer.

2. How a price is extracted

Extraction runs in a fixed order, cheapest and most robust first. This ordering is the single biggest driver of both cost and reliability.

1. 1

   Structured product data

   Most modern storefronts publish schema.org Product markup as JSON-LD, which contains the offer price, currency and availability in a machine-readable form. Where it exists and passes validation we use it, because it does not depend on the page's layout and therefore does not break when the design changes.

2. 2

   Per-domain extraction recipe

   Where structured data is absent, incomplete or contradicts what the page displays, we use a recipe specific to that domain: a set of rules describing where the displayed price, was-price, availability and seller sit. The recipe is generated once per domain and then executed deterministically, so the ongoing cost is per-domain rather than per-page.

3. 3

   Rendered-page extraction

   For sites that assemble price on the client, we render the page before extracting. This is the slowest and most expensive path and we use it only where the first two do not work.

3. Validation before anything is stored

An extracted number is not a price until it passes checks. A value that fails is not written, and the previous observation is not overwritten with a guess.

• Currency and format — the value parses as a monetary amount in the expected currency for that locale.
• Magnitude against history — a price that moves more than a configured threshold from the last observation is held for a second confirmation rather than published immediately. This catches a strikethrough price being read as the sale price.
• Coherence with availability — a price on a listing marked out of stock is stored but excluded from position and repricing calculations.
• Cross-field agreement — where both structured data and the rendered page are available, they must agree. Disagreement is an incident, not a coin flip.

Repeated validation failure on a domain triggers recipe regeneration. If the regenerated recipe also fails validation, the source is marked degraded, a human review task is created, and you are alerted. We would rather tell you a source is broken than show you a stale number that looks fine.

4. How listings are matched to your products

Matching is where competitor price data most often becomes quietly wrong, so it runs in order of decreasing certainty and every match carries a score.

When a product URL changes or a listing is delisted, we do not treat the match as lost. Matches are keyed to the product identity rather than to the URL, so the listing is re-discovered and the match repointed without a gap in your price history. The full reasoning is in the product matching guide.

5. How often data refreshes

Refresh frequency is a plan setting: daily on Starter, twice daily on Growth, four times daily on Scale, and hourly on Enterprise. Within a plan, refresh is prioritised — products you have flagged, products under an active repricing rule, and products with recent competitor movement are checked first, so the freshest data is on the SKUs where it matters.

Every observation is timestamped and stored append-only. We never overwrite history, which means a price you saw last month is still there next year and an audit of a repricing decision is always possible.

6. What we are explicit about not knowing

• Cart-only and coupon pricing. Where a site shows one price on the product page and a lower one at checkout, the advertised price is what we capture. We flag known coupon patterns but we do not claim to catch every one.
• Personalised pricing. If a retailer shows different prices to different visitors, we observe one of them, not all of them.
• Volume in the market. We measure prices, not sales. Any elasticity figure derived from our data uses your volumes, not a competitor's, which we cannot see.
• Intent. We can tell you a seller is below MAP. We cannot tell you whether it was deliberate.

7. How we source facts on this website

Separately from product data, we publish research and vendor comparisons. The rules we hold ourselves to there: every competitor fact comes from that vendor's own public pages and is dated; third-party estimates are labelled as estimates with their source; we do not publish market statistics we cannot attribute; and we do not use customer testimonials, review scores or logos we have not earned. Where a page contains a worked example rather than measured data, it says so.