Empty runs and correct billing for pay-per-result scrapers

The situation

In November 2025 we built eight small scrapers ("actors", in the platform's terms) in three days and left them alone. Nine months later a diagnosis pass found that two of them had never worked. One read a feed that the site only renders in the browser; the other used an unauthenticated API the platform behind it had closed. Both exited SUCCEEDED on every run, with a green tick and an empty dataset.

That is the core risk of a product billed per result. A filter that is too narrow, a blocked request and a changed page layout all produce the same thing: zero rows and a success status. To the person who ran it, that looks like a broken product. The opposite error matters too: if a job bills for rows it should not have produced, the buyer pays for our bug.

From July to September 2026 we rebuilt the fleet with a shared runtime kit and a release pipeline. This page covers the two parts that decide whether a run can be trusted: what happens when it delivers nothing, and whether it charges for what it delivered.

What we did

A reason for every empty run

"Zero rows" has several causes, and one of them is a correct answer. In monitoring mode a buyer asks only for postings they have not seen before, so "nothing new" is the feature working. A plain if (rows === 0) throw would fail a scheduled buyer's run every quiet hour. The order of the checks also matters: two orderings we shipped were wrong, and one reported a quiet monitoring run as "your filters removed everything", with an empty list of filters.

So we extracted a tiny kit with no runtime dependencies. A pure function takes the run's counters and returns a reason, a message the buyer can act on, and whether this is a failure. Only one reason is not a failure, and a test asserts that. A thin platform layer is the only place a run is failed, and it saves the run statistics before it throws, because a failed run is when someone reads them. An illustration of the shape:

function diagnoseEmpty(c: Counters): Diagnosis {
  if (c.blocked > 0 && c.parsed === 0)
    return fail('blocked', `Blocked on all ${c.blocked} requests. Enable a proxy and retry.`);
  const filtered = Object.entries(c.filtered).filter(([, n]) => n > 0);
  if (filtered.length > 0)                       // check the breakdown, not `parsed`
    return fail('filtered_out', `Read ${c.parsed} items; your filters removed all of them ` +
                `(${filtered.map(([k, n]) => `${k}=${n}`).join(', ')}).`);
  if (c.seenBefore > 0)                          // the one legitimate empty run
    return { reason: 'all_seen', message: 'Nothing new since the last run.', isFailure: false };
  if (c.fetched > 0 && c.parsed === 0)           // items in, nothing usable out
    return fail('none_usable', `The source returned ${c.fetched} records with no usable fields.`);
  return fail('source_empty', 'The source returned nothing for this input.');
}

// The only place a run fails. Evidence first, then the throw.
await Actor.setValue('RUN_STATS', { ...stats, zeroRowReason: d.reason });
await Actor.setStatusMessage(d.message);
if (d.isFailure && input.failOnZeroResults) throw new ZeroResultsError(d.reason, d.message);

The kit is copied into each actor by a sync script with a drift check, so every actor runs the same rules without a package to publish. The ordering regression tests each name the incident they prevent. The "items in, nothing usable out" branch came from our own nightly pipeline, where the same gap had produced four empty runs that looked healthy.

What it found

Making each actor explain its empty runs found a defect in every one that adopted the kit. None of them had raised an error before.

• One actor's default configuration returned 0 rows. Its input schema declared a default of [], which shadowed the code's default, because in JavaScript [] ?? x is []. Fixed, the same input returned 25 rows.
• Another discarded 245 fetched rows whenever one of its nine feeds answered 403, because one failure threw away the whole loop. It now counts blocked feeds and keeps the rest.
• A third had an "equity only" filter that rejected 20 of 20 postings.

Filters that failed open

On one job-board actor, the category filter read "if the buyer set categories and this row has a category, check it". A row with no detectable category skipped the filter. With the category values our own listing advertised, which were not real labels on that board, the filter kept the 14 rows it knew nothing about and rejected all 85 it could classify. The fix is to fail closed: a row with no category cannot satisfy a category filter, and "could not tell" gets its own rejection counter, so the empty-run message can separate a wrong value from an unlabelled source. The same filterActive && fieldPresent shape turned up in two more actors.

The guard then caught a problem we had made in the listing itself. The platform's input form prefilled one category and one region, filters combine with AND, and on the live board the intersection was empty. We isolated it by running each case: the category alone kept 23 rows, the region alone 6, both together 0, no filters 99. Because failing on zero is the default, those runs went red with a status line naming the filter counts. We removed the region prefill and put the measured yields into each filter's description, so a buyer can predict an empty intersection before running it. We did not switch failing on zero off by default, which would have hidden the symptom.

Billing for what we deliver

One actor could return and bill more items than were asked for. Two request handlers ran concurrently and shared a counter of items pushed. The counter was incremented after an await, which is a yield point, so both handlers read zero and both pushed a full batch:

// Reserve slots before the first await. JavaScript runs this block without
// interruption, so the read-modify-write below is atomic.
const remaining = Math.max(0, maxItems - itemsPushed);
const batch = rows.slice(0, remaining);
itemsPushed += batch.length;                // previously done after the await
if (batch.length > 0) await Dataset.pushData(batch);

Another actor's spend cap read a set that was only filled later in the run, so a capped request queued every link it found. Both bugs showed up only when we ran the public copies end to end for the first time. Four fixes in this period aligned what we charged with what we delivered.

Results

Related reading

• Silent zeros, four times a pipeline said success and was not