Docs
How Sift works, and every endpoint it exposes.
How Sift works
The layer around retrieval, not the retrieval.
Sift is a search API built for agents rather than for people. It does not run its own crawler or index — it sits on top of an EU-hosted search engine and owns everything around the call: the parts that decide whether an agent finishes its run inside its context window and inside its budget, which is where agents actually fall over.
One request to POST /v1/search runs the whole retrieval loop an agent would otherwise orchestrate by hand: it searches, fetches the promising sources in full, splits them into overlapping passages, ranks those passages against your query, and packs the best ones until your token budget is full. What comes back is passages, not links — text your model can use directly, with character offsets pointing back into a document you can re-read for free.
An API key is a policy, not just a secret. Each one carries its own scopes, depth ceiling, default context budget, monthly budget and audit-snapshot setting, so the key an autonomous agent holds can be strictly less powerful and less expensive than the one you test with. Spending is gated by a prepaid balance in euros; there are no plans and no tiers.
The wire format is snake_case throughout. Every failure is the same structured envelope at every status, so an agent can branch on error.code instead of parsing a message, and request_id is the one thing to quote in a support ticket.
{
"error": {
"type": "budget",
"code": "budget_exhausted",
"message": "Per-run budget of €0.75 reached.",
"details": { "spent_eur": 0.75, "cap_eur": 0.75 },
"request_id": "req_2f81c04b"
}
}Quickstart
Six calls, no SDK.
- 1Create a key
Create one on the API keys page and copy it — only a hash is stored, so this is the one and only time the plaintext is shown. Give it the
sessionsscope as well assearchif you want to follow the whole walkthrough.export SIFT_KEY="sk_live_…"
- 2Open a session
A session is one agent run. Passing its id on every search is what lets Sift recognise a question the run has already asked and answer it for free.
curl https://api.siftsearch.si/v1/sessions \ -H "Authorization: Bearer $SIFT_KEY" \ -H 'Content-Type: application/json' \ -d '{ "agent_tag": "policy-analyst" }' - 3Run a search
Add the
session_idfrom the previous step to make this call session-aware.max_tokensis the knob that matters — it bounds the response, andmax_resultsonly bounds how many candidates are considered.curl https://api.siftsearch.si/v1/search \ -H "Authorization: Bearer $SIFT_KEY" \ -H 'Content-Type: application/json' \ -d '{ "query": "changes to EU AI Act GPAI obligations", "max_tokens": 4000, "depth": "auto" }' - 4Read what came back
passagesis the payload;withheldexplains every source that was dropped and why;depthreports what actually ran;sessionandusagereport what it saved and what it cost.{ "request_id": "req_2f81c04b", "passages": [ { "doc_id": "doc_7c21f0", "title": "Commission guidelines on GPAI obligations", "url": "https://example.eu/gpai-guidelines", "domain": "example.eu", "text": "…", "support": 0.94, "as_of": "2026-08-14", "char_start": 8412, "char_end": 8689, "tokens": 612 } ], "withheld": [{ "doc_id": "doc_44b1e9", "reason": "already_in_session" }], "depth": { "requested": "auto", "chosen": "standard", "source": "model", "clamped": false }, "session": { "id": "sess_9f3ac2e1", "cache_hit": false, "urls_already_read": 9, "tokens_not_spent": 7900, "eur_saved": 0.0184 }, "usage": { "tokens_returned": 3812, "eur_charged": 0.0061, "latency_ms": 1900, "upstream_calls": 6 } } - 5Re-read a source for free
Take any
doc_idfrom the response and swap it into the path below. This returns the whole document and is never charged — it reads a cache your workspace has already paid for.curl https://api.siftsearch.si/v1/docs/doc_7c21f0 \ -H "Authorization: Bearer $SIFT_KEY"
- 6Replay the request
Swap in the
request_idfrom your search response. Replay reproduces exactly the passages that request returned, re-cut from stored snapshots with no upstream call, which is what makes evaluations deterministic. It needs the original key to have had audit snapshots switched on.curl https://api.siftsearch.si/v1/replay/req_2f81c04b \ -X POST \ -H "Authorization: Bearer $SIFT_KEY"
All samples target https://api.siftsearch.si, the host you are on right now.
Context budget
Ask for tokens, not for results.
The single biggest failure mode in agentic search is an agent destroying its own context window with search output. max_results cannot prevent that, because ten results might be four hundred tokens or forty thousand — you find out after you have already paid for them and pasted them into the model.
So the knob is max_tokens: between 500 and 32,000, defaulting to whatever the calling key is configured for. Passages are packed greedily against that budget and the total never exceeds it. A passage is never truncated to make it fit — a shortened passage would carry offsets that no longer slice back into its source — so a passage that does not fit is withheld whole.
Every passage carries char_start and char_end into its document, plus a doc_id. That is what makes the budget safe to set low: you are not throwing information away, you are deferring it. If the model decides it needs more of a source, it fetches that source on demand instead of you having paid the context cost up front for every candidate.
At most 3 passages come from any one document, so a single verbose page cannot consume the whole budget and crowd out corroborating sources.
- over_budget
- It ranked below the cut once the token budget was full.
- diversity_cap
- Its document had already contributed the maximum number of passages.
- duplicate_span
- It overlapped text already returned from the same document.
- already_in_session
- This run has already been given this source’s text, and you asked for deltas only.
- stale
- The content dates itself outside the freshness window you asked for.
Withheld entries are still returned as bare doc_ids rather than being hidden, so an agent can decide for itself whether to spend context on one. Rehydrating a document with GET /v1/docs/{id} is free and always will be: charging for it would push agents straight back to hoarding page text, which is the behaviour this API exists to remove.
Reference:searchgetDocument
Sessions & deltas
Stop paying twice for the same page.
Left alone, agents re-search near-identical queries in a loop and re-read the same handful of URLs on every pass. Each of those is a full-price upstream call and a fresh dent in the context window, and neither the agent nor you can see it happening.
Open a session with POST /v1/sessions and pass its id on every search in the run. A question the run has already asked — matched both exactly and semantically, so a rephrasing still counts — is answered from what that earlier answer returned: no upstream call, and nothing charged. The response says so plainly with session.cache_hit.
deltas_only goes further and withholds sources whose text the run is already holding, spending the fetch budget on pages it has not read instead. The distinction is deliberate: a source handed over as a bare id has been seen but not read, and stays eligible to be returned in full later. Only text the run actually received is suppressed.
The session block reports the work that did not happen — urls_already_read, tokens_not_spent, eur_saved. It is there so an agent can reason about its own efficiency, and so the saving is auditable rather than a claim.
Sessions expire 24 hours after their last use; every call slides the expiry forward.
Reference:createSessiongetSessionsearch
Depth, cost & credits
Guardrails an agent can reason about.
Depth decides how hard Sift looks, and with it how many sources are fetched in full. Deeper means better passages, more upstream calls, more latency and more money — deep costs roughly ten times what standard does.
- fast
- No fetches — ranks the search snippets alone
- standard
- Up to 5 sources fetched in full
- deep
- Up to 8 sources fetched in full
Nothing stops an autonomous agent from picking deep for "the capital of Portugal". So depth: "auto" hands the choice to a small classifier that picks the cheapest depth able to answer the query, and the response’s depth block reports what ran, which route chose it and why. The routing runs after the budget check, never before, and can only choose something cheaper than what you were cleared to spend.
A key can carry a depth ceiling. An explicitly requested depth above it is refused with depth_ceiling_exceeded rather than quietly downgraded — silently giving an agent a worse answer than it asked for is how you get a wrong conclusion with no trace of why. auto is the exception: it is clamped to the ceiling and reports clamped: true.
For a messy goal rather than a query, POST /v1/answer fans it out into 4 sub-queries (up to 8), runs them, merges and dedupes the results, and synthesises one grounded answer with citations. The sub-queries share a single fetch fan-out and a single embedding batch, so a four-way decomposition costs about four searches rather than four whole runs.
Spending is gated by one thing: a prepaid balance in euros, topped up whenever you like. There are no plans, no tiers and no monthly commitment. Every request is priced from a versioned price book, and the version it was priced under is stored with it, so historical figures never drift when prices change.
On top of the balance you can impose your own caps: a per-run cap and a per-key monthly budget, both hard — they refuse; and a workspace monthly budget, which is soft and only warns. These are guardrails you choose, not a plan you are on.
Every refusal is structured, machine-readable and final. Budget refusals are 402 rather than 429 on purpose: a 429 invites a retry loop, and an agent that has genuinely run out of money should stop cleanly, not hammer the endpoint until its wrapper gives up.
- depth_ceiling_exceeded 403 · policy
- The key is not allowed to search this expensively. Retry at a lower depth, or with auto.
- budget_exhausted 402 · budget
- A cap you configured was reached. The details block carries what was spent and the cap. Do not retry.
- insufficient_credits 402 · budget
- The workspace balance cannot cover this call. Top up; retrying will not help.
- rate_limited 429 · rate_limit
- The key’s per-minute limit was exceeded. Retry-After, and details.retry_after_ms, say how long to wait.
Freshness
Dates read off the page, not guessed.
Date filtering on web search is unreliable, and not because of a bug: search indexes carry a publication date that frequently disagrees with when the page was last meaningfully updated, and plenty of pages carry no usable date at all. A filter built on that metadata quietly lets stale pages through and quietly drops fresh ones.
So Sift reads the date off the content itself, after fetching it, and reports it as as_of on every passage. Sources the content dates outside the window you asked for are withheld with the reason stale, rather than being handed to your model as current.
A date that could not be read is null, and the source is kept. This is the same rule the whole API follows: an unknown value is never a guessed one. Reporting the fetch time as the publication date would be a fabrication, and one your model would have no way to detect.
Be clear about what this buys you: it corrects precision, not recall. It removes stale pages the upstream filter let through. It cannot recover fresh pages the upstream filter wrongly excluded, because those never reached us to be read.
freshness and from_date/to_date are mutually exclusive — sending both is invalid_request. Silently preferring one would be exactly the kind of guess this field exists to remove. And set dates in these fields, never inside the query text.
- any
- No freshness constraint. Nothing is withheld as stale.
- last_24_hours
- Rolling one-day window.
- last_7_days
- Rolling one-week window.
- last_30_days
- Rolling one-month window.
- last_90_days
- Rolling one-quarter window.
- last_year
- Rolling one-year window.
Reference:search
Provenance & replay
Every fact your agent used, provable.
Set verify: true and every returned passage comes back with a support score: how strongly that passage actually supports the query. It costs one extra model call, which is why it is opt-in rather than always on, and it appears in upstream_calls like any other cost.
support is a measurement, from a judge model reading the passage. It is deliberately not the retrieval similarity score — reporting how close a passage sits to your query in embedding space and calling it evidence would be a category error, and a convincing one. A passage whose support could not be measured returns null, never a number.
When a key has audit snapshots enabled, the exact bytes behind every passage are copied into immutable, workspace-owned storage. Not a link to the live page — the page as it read at that moment, which is the only version that can still justify what your agent concluded once the site has moved on.
POST /v1/replay/{request_id} then reproduces exactly the passages that request returned, re-cut from those snapshots at the same offsets. No upstream call, no charge, and no dependence on the web still looking the way it did. This is what makes evaluations deterministic and audits possible.
Replay depends on the original request having been made with a key that had snapshots on — you cannot switch it on afterwards and replay the past. When the id is valid but no evidence was kept, the error is snapshot_unavailable, kept distinct from not_found precisely so you can tell "wrong id" from "right id, wrong policy".
Snapshots are kept 90 days from last use; every request that pins the same content slides the expiry forward.
Reference:replayRequest
Research
Long jobs, without the polling misery.
Some questions need minutes of multi-source investigation rather than a second of search. That work cannot happen inside an HTTP request, and agents handle long polls badly — they either block a whole run on one call or forget the job exists.
POST /v1/research therefore returns a ticket immediately, with 202. The ticket is cheap to read, and it always tells you what to do next rather than making you guess a polling interval.
{
"id": "rsch_4b81f0",
"status": "running",
"stage": "searching",
"output_available": false,
"poll_after_ms": 30000,
"usage": { "eur_reserved": 0.58, "eur_charged": 0 }
}poll_after_ms is how long to wait before checking again, and the same value is sent as a Retry-After header. Reading a ticket is free and does not count against the job, so a well-behaved agent can check as often as it is told to without paying for its own impatience. stage and progress let it report something useful meanwhile.
Better still, supply a webhook_url and skip polling altogether. Deliveries are signed with an HMAC over a timestamped payload using the same scheme Stripe uses, so if you have verified a Stripe webhook before you already have the code. Retries are re-signed rather than replayed.
A research job that fails comes back as a 200 with status: "failed" and an error block — not an HTTP error. The request to read the ticket succeeded; it is the work that did not, and conflating the two is how a polling loop ends up retrying the read instead of surfacing the failure.
Because a job can run for minutes and cost real money, credits are reserved when it starts and settled when it finishes. A job that fails releases its reservation in full. Your available balance is the balance minus everything currently reserved, so two expensive jobs cannot both be admitted against the same euros.
Send an Idempotency-Key and a retried submission returns the original ticket instead of starting — and paying for — a second investigation.
Reference:createResearchgetResearchlistResearch
Watches
A standing query that reports only what changed.
A watch is a search that runs on a schedule and tells you what is different since last time. Each run returns a delta — sources that are new, changed, unchanged or gone — rather than the same result set over and over, so an agent reading it can act on the difference instead of diffing full payloads itself.
- hourly
- Every hour.
- every_6_hours
- Four times a day.
- daily
- Once a day.
- weekly
- Once a week.
Change is a similarity, not a hash. Live pages carry navigation rails, rotating promos and relative timestamps, so byte equality would report every source as changed on every single run — an alerting product that cries wolf constantly, which is indistinguishable from a broken one. A source counts as unchanged when its content still overlaps the previous run by at least 90%.
A watch’s query and search parameters are fixed for its life; you can rename it, re-schedule it, pause it or resume it, but not re-word it. A delta across a changed query would be a diff between two different questions, which is a number that looks meaningful and is not.
Every run is billed as the search it is, against the key that created the watch — so that key’s scopes, depth ceiling and budget govern the schedule for as long as it exists. Deleting a watch removes its runs and leaves the billing history intact.
Reference:createWatchgetWatchRun
Prefer to talk to Sift over MCP instead of HTTP? The same search, answer and document tools are exposed at POST /v1/mcp, authenticated with the same key. MCP setup has a copy-pasteable client config.