How it works

A webset build is a durable agent run on eve. Starting a run creates an eve task session whose message names the webset id; the agent then works through four deterministic tools — get_webset, save_plan, run_searches, verify_results — while the app streams its progress to your browser. All judgment beyond planning lives in tool code, so runs are reproducible and resumable.

1. Plan

On a fresh webset the agent designs a plan and persists it with save_plan:

• Queries — 3–6 diverse web search queries, phrased to surface pages dedicated to individual entities (official sites, product pages, repositories, profiles) rather than listicles or comparison roundups.
• Criteria — up to 5 verification criteria. Each is an objectively checkable yes/no statement about a single candidate ("The company is headquartered in Europe").
• Enrichments — up to 6 data columns to extract per entity, each with a format: text, url, number, or date. The first column is always Name.

The plan is saved exactly once. Resumed or extended runs skip planning — you can also add criteria and enrichment columns to an existing webset, and a later run fills in only the missing cells.

2. Search

run_searches executes every pending query (two at a time). Each query fans out concurrently to every configured provider — the always-on AI Gateway search-grounded model plus any of Exa, Parallel, and Firecrawl with keys set (see Search providers).

The per-provider ranked lists are merged with reciprocal rank fusion: each URL scores Σ 1/(60 + rank) across the providers that returned it, so a URL surfaced by several providers outranks any single provider's top hit. URLs are normalized before merging (hash stripped, www. dropped) and deduplicated per webset — a URL already found by an earlier query isn't stored twice, and its title and page content are backfilled only when the stored row was missing them. Per-provider provenance (provider id + rank) is stored with every result.

Failed queries are recorded per query; a run only fails outright if every query fails.

3. Verify

verify_results processes candidates in batches of 8 (4 concurrent) until none are incomplete:

1. Page content — if no provider returned page text inline, the page is fetched directly (10 s timeout, HTML only, stripped to plain text, capped at 15,000 characters). A failed fetch never crashes the run.
2. One structured LLM call per candidate checks all missing criteria and extracts all missing enrichment values against the page content. Each criterion gets a verdict — match, partial, no_match, or needs_review — plus a short verbatim evidence quote from the page. Unavailable content or a missing model answer degrades to needs_review, never to no_match. List/comparison pages are rejected: a candidate must be a page about one specific entity.
3. Overall verdict per row: no_match if any criterion failed, match if all matched, needs_review if all need review, otherwise partial.

Because verification only targets missing cells, adding a criterion or column later re-verifies only the new cells — existing verdicts are untouched.

4. Stream

Results land in Postgres as they complete, and the browser follows along live over an NDJSON event stream proxied from the eve session (see API). The agent narrates each step in plain sentences ("12 verified so far; checking the remaining 8."), and each event doorbells the client to refetch the session — the stream itself carries no webset data.

Resume semantics

Runs hold a 60-second lease: a webset counts as "running" while its status is processing and the agent's heartbeat (refreshed on every search and verify step) is under 60 seconds old. Starting a run while the lease is alive is rejected; once the lease lapses (crash, timeout, deploy), starting again resumes from the database state — the existing plan is kept, completed queries are skipped, and only incomplete results are verified.

Caps

Query and result caps are tunable via environment variables when self-hosting.