Who this is for: whoever is looking at a pass that failed, a corpus that
looks wrong, or an engine that refuses to run at all. You need: a shell on
the instance (or the update endpoint) and revetly status — nothing here bills
anything. Not here: what a pass costs before it runs
(adopting-history.md prices the historical kind).
Two questions the engine used to answer badly, and the two forks each story left open on purpose. Both are settled here, and both were measured rather than theorised — the dates below are real failures on this instance.
One retry authority (revetly/llm_client.py), one budget
(REVETLY_LLM_MAX_ATTEMPTS, default 5), and one classification with two sides.
Everything is read from the provider's structured fields — type, code,
status_code, x-should-retry — never from message text, so a provider
rewording a sentence cannot move a failure from one side to the other.
| Side | What it is | What happens |
|---|---|---|
| terminal | The account cannot buy this call: insufficient_quota, credit_balance_exhausted, credit_balance_too_low, billing_hard_limit_reached, in either type or code |
Fails on first contact, carrying the provider's own sentence ("You have no credits remaining") and the fields it was read from. No attempts, no waiting. |
| transient | The statuses the openai SDK's own loop covered before max_retries=0 turned it off: 408, 409, 429, anything ≥ 500, plus APITimeoutError and APIConnectionError |
Retried inside the shared budget with jittered exponential backoff, honouring Retry-After up to a 60s cap. |
| neither | A 400, a 401, a 404, anything that is not a provider error at all | Raised on first contact, exactly as before either story. |
x-should-retry outranks the status rule, because the SDK's loop obeyed it and
this loop replaced that one. A quota body outranks everything: a provider
asking for a retry on an account with no balance is asking for a wall to be
walked into five times.
The classification walks the exception chain, because graphiti_core converts a
429 into its own message-only RateLimitError and everything structured ends up
in __cause__. It also follows an explicit cause attribute, which is how
IngestFailure carries what a pass died on.
Measured. 2026-08-02: five attempts with exponential backoff spent against
an account with no credits, reported as Rate limit exceeded — congestion
wording on a billing problem. 2026-08-04: InternalServerError ... server_error
killed a pass that succeeded by hand four minutes later, because turning the
SDK's loop off left nothing covering 5xx.
It does not let the remaining artifacts try. Every remaining episode needs the same account, so continuing means N more refusals for one human fix, and the work that landed before it is already in the manifest — the next pass resumes from there and pays for none of it again. The alternative (let the rest try) buys nothing but a longer failure report.
graphiti_core re-raises openai.AuthenticationError
without converting it, so it never reaches the retry loop. Catching a bad key
before a pass bills is revetly doctor's job, not this loop's.embedder — no longer. Graphiti's embedder used to hold its own SDK
client with its own max_retries, outside this authority, and an embedding
429 failed a pass. The declared provider now builds the embedder on
a client whose own loop is at zero, and retrying_embedder installs the same
budget on its two calls — so it is inside, and this list is one item shorter.A pass that dies leaves the graph holding what it held before that pass. The files and the commit are durable either way, by design — so the question is only who finds out.
The record. Every pass writes its outcome to .revetly/ingest-health.json:
when it ran, whether it succeeded, the failure detail, whether that failure was
terminal, how far it got (episodes sent, links minted — read off the failure
itself, so the fields and the sentence cannot disagree), and which capture
handles and artifact ids it covered. Bounded to the
last 25 passes, written atomically, gitignored — one machine's pass outcome is
operational state, not a fact about the corpus other clones should inherit.
It is a record, never a queue. The capture ledger is deliberately not durable, and nothing here reverses that: no work is replayed out of this file. What is read back out of it is what to say, and whether to try again.
Recovery. A failed pass arms its own retry — recover_attempts (default 2),
the first after recover_delay (default 60s) and each after that at double the
last. A terminal failure is never retried. A timer lives in the process that
armed it, so a restart during the wait drops it; the next server start reads the
record and schedules the attempt the dead process owed, which is what makes
recovery independent of the next human action rather than merely independent of
the next write.
The MCP server is one process per client connection, and the previous one does not necessarily go away — two were observed up at once on 2026-08-10 (154 and 92 minutes old). Each had read the same failure record and armed its own ladder, and the record shows what that cost: passes overlapping (18:47:47–19:12:35 against 19:07:01–19:19:20), two corpus-wide passes contending on one manifest, doubling the concurrent load against a provider that was already failing, and spending two budgets where the policy declares one. The manifest gate meant nothing was double-written; what doubled was the spend and the concurrency.
So recovery is held, for the whole ladder, by one server at a time:
flock, held for the whole ladder. An owner recorded in a file has to be
expired by somebody, and every expiry is a guess about how long a pass takes:
guess short and a second server steals the lease mid-pass, guess long and a
crash costs the instance its recovery until the clock runs out. The kernel
releases an flock when the holder exits however it exits, including a
SIGKILL that runs no cleanup, so nothing has to decide whether the holder is
still alive. The file's contents — pid, since when, what is armed — are for
reading; a reader asks the kernel whether anyone still holds the lock, so the
line left by a process that died reads as free. The file is never unlinked:
two processes holding fds to an unlinked inode would each hold a lock on a
different file and both believe they owned recovery..revetly/, and that is measured. The instance directory is
a bind mount, and Docker Desktop for macOS does not carry locks across it —
two processes in the one container each took flock on the same file under
/project, and fcntl record locks behaved the same way; only exclusive
create is atomic there, and a lock whose release depends on the holder
tidying up is the expiry guess again. So it lives in the runtime temp
directory, where the same container measured exclusive, named
revetly-recovery-<digest of the instance root>.lock — shared by exactly the
processes that can race, because a server per connection is a server per
docker compose exec into one container, and keyed so two instances under one
container do not share an authority. context_health and revetly doctor say
where it is, since it is not where the rest of this instance's state is. The
declared limit: two engines in two containers against one instance would not
exclude each other — not a shape this deployment produces, and the filesystem
between them cannot carry a lock in any case.context_health with the capture handle), on the record, and in the server log at startup. Nothing is
lost by deferring — the pass the owner has scheduled is corpus-wide and covers
the decliner's records too.recovery_attempt, so recover_attempts = 2 is two automatic
passes for one failure however many servers come and go, rather than two per
connection. A reconnect into an exhausted ladder is told it is exhausted and
pointed at revetly ingest.context_health names the holder in recovery_owner when there is one, which
is what tells "the retry is armed, just not by this server" from "nobody armed
a retry".A host command does not open a ledger the container owns. The same bind
mount is why. SQLite's locks do not cross it either, and on 2026-10-09 a
revetly doctor and a revetly ingest --dry-run typed on the host failed the
container's running pass with disk I/O error. So where the instance's compose
file declares the engine service and docker compose ps names a running engine
container, a command on the host that would open a project's manifest.db
refuses and prints the same command as docker compose exec -T revetly revetly … (with -f where the compose file is not docker-compose.yml). revetly doctor, snapshot, restore, reset and migrate refuse before they touch
anything. revetly --on-host <command> runs it on the host regardless, for an
operator who knows the loop is stopped. Docker is asked at most once every ten
seconds per process and given five seconds to answer; a Docker that does not
answer counts as no engine running, which is how a host command ran before the
guard. Inside the container nothing changes. revetly status, revetly ingest --dry-run and doctor's ledger reads open the ledger read-only wherever they
run, and a dry run on an instance that has never ingested creates no
manifest.db.
Out of scope by declaration: how the client launches the server (a stdio server per connection is the protocol's normal shape), reaping the old processes, and the concurrency ceiling itself, which a separate mechanism already bounds inside one process.
A pass a human runs (revetly ingest) is recorded the same way, because it
is the fix the warning names — so it has to be what clears it. A pass started
from the configuration page files that same record, and clears the warning the
same way.
search_context, get_artifact, list_backlog and get_authoring_context carry a
context_health key only when the last recorded pass failed. A dict gains
the key; a list is wrapped as {"context_health": ..., "results": [...]}.
They say nothing when:
The reason for the narrowness is the whole point: a warning attached to every read is one a consumer learns to skip, and it would be skipped on the read where it mattered. On a healthy instance every read is byte-for-byte the read it was before this story.
context_health is the tool for asking outright, and it always answers —
corpus is current, stale, or unknown, with the last pass, the declared
recovery policy, and what is scheduled. revetly doctor reports the same
as a warning, not a failure: nothing is misconfigured, but it is the one line
that explains why every other check reads fine and a search comes back empty.
The capture handle read of context_health reads the record too, so a handle from a process that is gone
comes back with its pass's outcome and from_a_previous_process: true.
And a handle whose pass is still running one process over comes back pending,
with pending_elsewhere: true and who holds the pass. That is the
third case, and it is now the ordinary one: a handle lives in the ledger of the
server that filed it, the durable record is written when a pass finishes, so a
pass in flight in a sibling server has no trace in either place. Two servers
against one instance is this deployment's normal shape — a stdio server per
terminal connection beside the long-lived HTTP one — so
a capture filed through one and polled through the other is routine rather than
an accident. Asked before this, it answered unknown, offering two explanations
that were both false while the artifact was minutes from landing.
The authority is the same ingest lock, widened to mean one pass at a time per instance. Nothing new is recorded and nothing is written on the path a capture takes: the kernel is asked whether anybody still holds the lock, so a line left behind by a process that died reads as free. A lock this process holds is not an answer either — inside one process a handle absent from the ledger is not in flight here — so a single server against an instance answers exactly as it always did.
unknown therefore means all three at once: no ledger holds the handle, no
finished pass covers it, and none is in flight. It is the answer that tells a
caller to stop waiting, and one word was being given to three situations.
Everything above is about a pass that ran and died. Watch mode adds the case where no pass runs: a source somebody else edits all day, in a system Revetly is not part of, with nothing in this instance creating the event that would read it.
Freshness is per source, and "the last pass succeeded" is not it. With one
tick reading the two sources that moved and leaving the others alone, an
instance-wide answer cannot say whether what CONTEXT holds about one source is
an hour or a fortnight old. .revetly/sources.json carries a row per connector:
last read successfully, last change seen, whether that view was complete, why the
last attempt failed, and when the next attempt is due. revetly status prints
it, context_health returns it as sources, and every pass writes it —
including a manual revetly ingest, so an instance that has never run watch can
still say when anyone last read a source.
A quiet tick is not a pass. A tick where every change check said "nothing
moved" writes to the watch record and deliberately not to the health record: a
pass that ingested nothing must not file itself as a success, or a failure nobody
has fixed would be papered over by the next quiet minute. context_health keeps
reporting stale until a pass actually lands.
Watch keeps a corpus current; it does not create one. A tick over a project
nothing has been read into runs no pass. It asks no source and opens no graph,
records awaiting-first-pass in the watch record with the command that prices
the pass, and keeps the plain interval. "Read into" is decided from what the
engine already writes: a pass in ingest-health.json, landed or failed, or a
row in the project's ledger. revetly ingest and the page's first-pass action
both record a pass, and a dry run writes neither. context_health adds
watch.waiting_for_first_pass while it holds, and a project with no recorded
pass and an empty ledger reads as never ingested, where an instance with rows
and no record (an older engine's corpus) keeps the older "no pass has recorded
an outcome" reading.
Two reasons a pass runs, and the report tells them apart. A change check
asks the source whether it moved, and the source is only half of what "changed"
means. A pass that stopped at a bound — comment_threads_per_pass on Todoist or
Azure DevOps — left a slice of the board unread, which is a difference between
CONTEXT and the source exactly as much as an edit is, and one no question to the
source can find: nobody touched those threads. So a landed pass records what it
left behind in the connector's own state slot, under the same land-first rule as
every cursor, and the next check reads it there before it asks anybody anything.
That costs no request, so a source that is unreachable does not stop an instance
knowing it owes itself a read.
It decides only whether. The pass it triggers is the same full-state
extract(), and which threads that pass opens is the connector's rotation as it
always was. What it changes is the clock: N threads at a bound of B come
round in ceil(N/B) ordinary ticks rather than ceil(N/B) full-pass intervals.
The signal says which reason it was, and which instrument answered — "the sync
token: changed — 2 change(s) in scope" against "this instance's own unread work:
changed — this instance owes itself a read of 'comment' — 3 comment thread(s)
were not read" —
in the tick line, in revetly status and in context_health, because a board
being kept up with and one only being cycled through cost the same pass and mean
the opposite about how current it is. A source under a bound never goes quiet:
that is the trade a bound buys, bounded per tick rather than unbounded per pass.
A signal is dated, and only watch mode takes one. Every pass writes the
freshness row, but only a tick runs a change check: a manual revetly ingest
and every capture pass a write tool triggers read the source and let the
manifest gate decide, which is what makes them idempotent and what makes them
ask nothing. So the sentence on a row and the row's own last_checked_at can
come from different moments, and a row that served the first inside the second
reports the last tick's finding as this pass's. Two keys, so the key
changes with the claim: signal is what the check behind this row's own last
look said, and last_signal — with last_signal_at beside it — is a sentence
no check since has re-taken. revetly status says the same in words:
change check not run since 5d01h ago, when it said: …, rather than appending a
bare sentence to a line that opens with read 9s ago. An empty last_signal_at
is a row written before signals were dated: unknown, and never recent. An
instance running watch on its interval sees no difference — every tick re-takes
the signal, so it stays signal and reads exactly as it always did — and one
that has never run watch has neither key.
This is why the row's other carried-forward fields are not the same case. A revision is a fact about the source that stays true until the source moves, so a pass that did not reach a repo-backed source keeps the one it had. A signal is the outcome of one check at one moment and stops being true the instant anything moves.
Backoff is per source, and it is not the recovery ladder. A rejected
credential, a rate limit or an outage backs that source off — doubling from
interval to backoff_max — while every other source keeps ingesting, and the
source is left out of the pass's scope entirely while it waits, so nothing of its
is tombstoned for being absent from a run it was not part of. A failure that is
nobody source's fault (the provider, the graph) backs off the whole tick and
lands in the health record above, with the next tick written into its recovery
field so the record says what happens next rather than reading like a retry
nobody armed.
Watch mode does not take the recovery authority. It has its own clock and
retries corpus-wide already; arming a second recovery ladder on top of that would be
the doubling this lock exists to prevent. What it does take is the
ingest lock: every entry point into the pipeline — a tick, a capture, a
human's revetly ingest — holds it for the pass, so one instance runs one pass
at a time across processes. revetly migrate --apply holds it too, across its
steps, and waits for a pass in flight rather than landing under it. A tick that finds it held folds into the running
pass and consumes no change signal; a capture waits, because the pass in flight
may have extracted before its files were committed and must not be allowed to
report them as ingested.
A connector that cannot run does not end the loop. At a terminal, a source that cannot run as declared is a hard stop naming the fix and stays one. Unattended, it is a source that failed: recorded, backed off, reported per source, and never read as deletion — because an instance whose Todoist token was revoked at 3am must not stop reading its four healthy sources until somebody notices.
Nor does anything else a tick can do. Whatever happens inside a tick, the loop it runs in survives it, files it as a failed tick with the reason on the health record, and keeps its declared interval — the guarantee is stated where it can be kept rather than inferred from every path inside a pass. An unattended engine holding a corpus somebody is reading does not get to exit.
A cursor that could not be written is the cheap half of a trade, and it is
never silent. A connector hands its next state back to the engine, which
writes it after the pass it belongs to has landed — so a pass that died leaves
the cursor where it was and the next pass re-reads that slice, rather than
skipping records nothing ever ingested. What the slot will not hold, it refuses
by name: a mapping of string keys to json values, plus a datetime written as
ISO 8601, and nothing else. A refusal costs one repeated read and never the pass
that earned it, which had already sent its episodes and committed its files.
Where it happens, three places say so — the ingest report, the source's own row
in .revetly/sources.json (cursor NOT PERSISTED), and the pass's entry in the
health record, which stays recorded as the success it was: the corpus is
current, and only the bookkeeping about the next pass is not.
Everything above is about a pass that ran and died, or a source nobody read. This is the case where a pass ran, succeeded, read every file under its roots, and answered from a repository three weeks out of date — because until this story nothing in the engine fetched, pulled or asked a remote anything.
Three tiers, and each one earns a different authority. The records
repository holds what this instance itself writes: the backlog, conversation
notes and delivery records. Where [git] records_clone names a path the engine
owns that clone, is its only automated writer, and is the one repository it
pushes to. A doc root is prose a project already had, and it commonly sits in
the product's own repository; the engine fetches it, fast-forwards a clean
checkout and writes nothing to it — no push, no branch, no tag, no commit. A
root outside git, or in a repository with no remote, is read exactly as it was
before any of this existed and says so by name.
A records store declared in place — inside a repository the engine does not own, which is how every install before this story declares it — is read under the doc-root rules. Upgrading changes nothing for an instance that has not moved its store.
Fast-forward only, and what is in the way is named rather than resolved. A diverged checkout is not merged; there is no merge policy in this engine. A records repository that is dirty or diverged is refused and reported — a state that should not arise on a clone the engine owns, and is not assumed away. A doc root that is dirty or diverged is read at the revision it holds, because it is a repository developers are working in and a pass that refused on that ground would refuse most of the day.
A stale read is an incomplete view, and that is the safety property. A checkout that is behind returns fewer artifacts than the ledger holds, and every one of those absences is indistinguishable from a source that withdrew them. A partial extract is never tombstoned; a stale read reports itself through exactly that machinery, so the deletion pass is suppressed for that connector and nothing new decides what to tombstone. The consequence is worth stating plainly: an instance that cannot reach its remotes stops detecting deletions rather than inventing them.
Reading less of a source is not the source withdrawing what you stopped
reading. A stale checkout is a partial view and suppresses the
pass, as above. A floor that moved later is not partial at all — the pass reads
its source completely, from a later date, and every artifact under that date is
an id it did not return from a connector that vouched for what it did. That is
the shape of a deletion and it used to be treated as one: raising
[connectors.commits] since tombstoned every commit below the new floor, with
nothing in the record able to say a repository had withdrawn them.
So a pass says what it looked at, and that claim now bounds it as well as arming
it: an artifact whose own moment falls outside every span the connector claimed
for its kind and unit is held — kept in CONTEXT exactly as it was, not
offered by this pass, and never tombstoned. The pass report names it as held
rather than removed; revetly doctor names the span a raise put out of view and
how many artifacts are in it, before the pass that would act on it; and the
history key on status and the read tools says "read and now out of view,
held in CONTEXT, not withdrawn" beside the floor itself.
Three silences leave the old rules exactly where they were: a kind nobody
claimed a view for, an artifact with no moment of its own, and a unit nobody
claimed a view for — the last being a repository that left repos, which is a
source this project no longer declares rather than one it reads less of. And a
floor coming back down over ground this instance has already read re-covers
it for free: the artifacts return to view out of the ledger, with no episode
sent, while ground below anything ever read stays a priced backfill.
One case this cannot repair: a tombstone written before the ledger recorded why. A narrowed view and a real withdrawal produced identical rows, so an instance carrying them is told plainly that those cannot be told apart — everything tombstoned from now on says which it was.
And a whole source leaving the declaration is the largest version of the same
failure. A floor loses a span of one unit; taking a name out of
[connectors] enabled loses everything that connector ever contributed, in one
ordinary pass — a watch tick, a capture, the pass after an edit — reported as a
removed count nobody connects to a one-word edit in revetly.toml. Measured on
this engine's own instance: eighteen rows across three retired connectors,
tombstoned when the fixtures were switched off, every one of them carrying no
record of who withdrew it, because nobody had.
Those same rows carry no date either. The column recording when a deletion was found arrived later than the first tombstones did, and an empty one is a real answer — "deleted, time unknown", not "not deleted" — so a row tombstoned before that column existed reads as still held to anything that asks whether the date is missing. A live row is the one the ledger marks not tombstoned, which every row carries and no migration can leave unset. That is the only question about this table that is total, and it is the one the ledger answers.
So a row whose connector this instance no longer declares is unconfigured:
held exactly as it is, never tombstoned, and named per connector in the pass
report, in revetly status, on the freshness row context_health serves, and by
revetly doctor before any pass acts. It is the strongest of the four
exemptions in the deletion pass and the only one that is not about a source at
all — the other three are a source that was read and could not vouch for
something, and this is a source that was never asked. Putting the name back
reads it again with no re-extraction, because nothing was lost.
And a source routed into a shared namespace is retired by the same edit.
That answer did not reach the shared namespaces, because the question asked
before a pass runs there is an older one: pending_rehomes asks whether each
row's home is still where the declaration says, and it read a retirement as a
move. Retiring a routed source produced a pending move to project '' — a
project that does not exist — and a refusal is instance-wide, so every pass
over every project died on it, watch mode included, including projects with no
connection to that namespace or that source. The way out the message named is
revetly rehome --apply, which retires the old namespace's episode, nodes and
ledger rows on the promise that the pass after it re-ingests them somewhere.
Here nothing would, because nothing reads that source any more: following the
tool's own advice erased the corpus, with no tombstone and no record it had
existed.
A move and a retirement are different questions and only one of them has a
destination. A row is a move where its owner — the project whose
source_instance the row carries, asked of every project this instance
declares rather than only the ones routing here today — still declares that
connector; then it goes to wherever that project now routes it, named rather
than left empty. Everything else is a retirement: the connector left its
project's declaration, or no owner can be identified at all, and a move whose
destination cannot be named is not one anybody can act on. Those are held by the
ordinary pass under the rule above, and named — revetly status --shared for
the namespace, and revetly doctor beside the roster, which is where a shared
namespace's own ledger belongs because it belongs to no project's chain.
The declaration the pass judges against is what the configuration names plus
every connector the pass actually asked: a connector that ran has been asked
whatever the file says, and where the two disagree the answer has to be the one
that goes on judging that source rather than the one that silently stops.
revetly ingest --connector is unaffected and stays silent about the sources it
did not ask — a scope is one run's narrowing, and this is a standing
fact about the configuration, which is why it is said on every pass until
somebody puts the name back or means it.
And a unit leaving a declaration is the same edit, one level down. A
repository dropped from [connectors.commits] repos, a root dropped from
[docs] roots: the connector still runs, still vouches for a complete view,
and simply never opens that unit. Measured on this instance before the change,
dropping one of three repositories: removed: 5, withdrawn: 0. One
repository in the same array holds 648 commits, so the same one-word edit
against a different line of the same list costs two orders of magnitude more.
So a connector states the units it currently declares — which is not the
claim history_view makes, and the difference is the whole rule: a unit that is
declared and unreachable is an incomplete view and suppresses the pass, and
a unit that is not declared was never asked. A row belonging to
none of its connector's units is unconfigured too, held under the same
sentence, and the report names which unit it came from.
Stated by the connector, never derived here: a commit's source_path is its
repository and a doc's is a file under a root.
And a unit that is not a path at all is stated the same way. A
tracker's project, area path, board or saved query narrows a pass exactly as a
repos line does and appears in no source_path — for an API record the path is
the request that produced it, identical in shape on every row. The prior unit
rule left those alone and said so, which meant the two tracker connectors went
on tombstoning on an edit: a project dropped from [connectors.todoist] projects,
an area_path pushed one level deeper, a line removed from work_item_types, a
saved query swapped, an organization or project changed. Each reported
removed: N, withdrawn: 0 — the same signature, two levels down.
So a connector may also stamp each artifact with the unit it came from, and the ledger keeps it. A row that states a unit is matched against the declaration exactly; a row that states none is matched by the path rule, which is what keeps every file-backed connector and every ledger written before that column exactly where they were. The stamp is the declared unit the artifact matched rather than the deepest true unit it sits in — a work item three areas below the declared one carries the declared area — so nothing here has to understand a hierarchy it was never told about, and narrowing a scope re-stamps every row still being read onto the new declaration, leaving only the rows nothing asks for behind. Which is what makes the hold self-correcting rather than permanent.
A source that has narrowed nothing still declares nothing, and that stays the honest answer rather than an omission: an instance reading its whole Todoist account asks every project on every pass, so a project's absence really is that project's absence and claiming units there would hold a board somebody deleted. Azure DevOps always declares one, because an organization and a project are required there and there is no unnarrowed instance to be.
Three rules keep this from being a way to disarm deletion detection by accident.
A declaration whose own returned artifacts fall outside it is not believed —
a prefix or a key spelled other than the way the connector records it would
otherwise put every row of that connector outside its declaration and switch the
pass off in silence, so the connector and the ledger have to agree before the
claim counts. A connector that stamps some of its artifacts and not others is
not believed either, for the sharper version of the same reason: the unstamped
half would be judged by a path rule against a declaration that is not paths, and
held for ever without a word. And the unit check is asked last, after every
claim the connector actually made: a row held by history_only, by a kind it
could not vouch for, or by a span outside what it looked at keeps that narrower
and truer explanation. Its parent — a whole connector that left
[connectors] enabled — is asked first, because there nothing ran at all and no
other explanation can apply.
The ledger fills that column on the free path as well as on ingest, which is the half without which the rule would disarm itself for one pass: a row an ordinary pass merely saw is a row the declaration still asks for, and filling it only on re-extraction would leave a whole corpus looking, once, like one sitting outside its own declaration. Nothing is back-filled — a row written before the column was produced by a connector that stated no unit, and NULL says exactly that.
An unreachable remote is that fact. No credential, no network, an
authentication refused — reported as itself, never as an unchanged source and
never as an empty one, and the pass that proceeded on the stale checkout is not
recorded as a clean read. [git] fetch = false is how an instance that is
genuinely local-only says so and keeps exactly the behaviour it had, including
reading a checkout nobody moved without calling that stale.
Every read names the revision it answered from. "Read successfully at 14:02"
says nothing about what state was read. .revetly/sources.json carries the
revision per repository beside when that source was last read, revetly status
prints it and context_health returns it — so a read is one somebody can
reproduce rather than an answer about an unnamed past. A project whose roots span
several repositories carries several: one current and another behind is two
answers, never one freshness number for the instance.
Watch mode asks the remote. A teammate's push changes no file here, so the
tree fingerprint cannot see it. The cheap check for a repo-backed source is git ls-remote — one line, no object transfer — which is what makes a teammate's work
visible within one declared interval rather than never.
"Moved" means a head this engine has not already seen. Not "differs from
HEAD": a checkout with unpushed commits differs from its remote all day, and
reading it on every tick bought a whole pass that sent nothing. A remote head
is news unless HEAD already contains it — equal, or ahead — or it is the head
the last landed read already observed and its commit is here. So a behind
checkout is read once and fast-forwarded; a diverged or dirty one is read once,
reported incomplete by name, and not again until the remote moves on or
something here moves; and a sha whose object is not here is always news,
because that is what a teammate's push looks like before the fetch. What was
observed is kept once a read lands whole, or partial for a reason the next read
would repeat exactly — diverged, dirty, held, a credential fault, an unreachable
remote — because a checkout like that is read incomplete on every pass, and a
check that forgot what it saw would buy the same read on every tick. A read that
failed keeps nothing, and neither does one that was partial for a reason worth
retrying — a root that could not be walked, a fast-forward that met another
process's index.lock or an I/O fault — so the next tick tries again. A
fast-forward git refused because it would overwrite local changes is not one of
those: work in progress anywhere in the repository, outside the doc root
included, is settled for as long as the paths git named stay changed (the whole
worktree, where it named none), and read at once when they are clean.
The full cadence does not bring the churn back. Its clock is a source's last
complete read, which a diverged or offline checkout never has, so after
[watch] full_interval it would be due on every tick. It yields when the check
says the last read already saw exactly this state, or when an unreachable
remote's next read is already scheduled — and only within one full_interval
of the last read that landed at all, so even a checkout that is always read
partial is read in full once per interval. "Already saw" is re-checked, not
remembered: the last read keeps why it was partial (unreachable, held,
diverged, dirty), and the moment that stops being true — the remote answers,
the checkout contains the remote's head, the uncommitted work is gone — the
next tick reads, and a head that read had observed is news again. A source
whose reads are whole keeps its periodic full read.
An unreachable remote is read once, then on the backoff schedule. A check
that cannot answer still falls back to the read it was trying to save — once.
While the remote stays unreachable and nothing here has moved since that read,
the next one is due on the schedule a failing source is retried on: doubling
from [watch] interval, capped at backoff_max. Anything that moves here is
read at once, backoff or not — including a push to another repository behind
the same roots, which is read now and named beside the remote that would not
answer — and the run ends the first time the remote answers again.
A signal's sentence is the finding of the half that fired. A repo-backed
check asks two questions — the local tree or git rev-parse HEAD, and the remote
— and only the halves that found something describe the row: changed — web: the checkout moved to 0a34386e2f0f, or the repository and ref a remote moved to, or
both, in that order. A credential a remote refused is reported behind whatever
moved rather than as the reason for it. What a positive verdict never
carries is the other half's negative finding: changed — no repository behind its roots has moved on its remote was a true sentence about the half that saw
nothing, and it reads as a broken check rather than as the commit that had just
landed. The unchanged row is a single sentence about both halves, and says so:
unchanged — no declared repository has moved, here or on its remote. A remote
that moved is named under the instrument that saw it — the file tree + git ls-remote, never the file tree alone, which had seen nothing.
Unchanged and unreachable is a third row, and it says both. Inside the
backoff window the answer is not changed and is still not an answer: could not answer (web: could not reach origin) — no file under its roots has moved since the last read; reading in full again at 2026-09-28T12:10:00+00:00 if it still cannot. Never a bare unchanged, which would be a remote nobody could reach
reported as one that agreed, and never reading in full when no read is coming.
What the engine writes, and as whom. A commit into the records repository
carries the human who approved it as git's author, taken from the same
review_log.by the record already keeps; the engine stays the committer.
Without that, an engine that is the sole committer flattens every author to the
container it runs in, and git_times reads %an <%ae> — so a commit's account
of who made a change would be a claim by nobody.
Sharing a repository with product source is advice, not a fault. revetly doctor warns where the records store sits in a repository that also holds code,
and neither it nor revetly init ever fails on that ground. This engine's own
docs/ cannot move — the manual pages ship with the engine and are checked
against the checkout — and this repository holds engine source,
records store and docs together.
No connector reads product source code. git_history is the history of the
artifact files, not of the code beside them. Stated here rather than left as an
absence, because a scope nobody wrote down is one somebody assumes.
Everything above is about a corpus that is behind its sources. A backfill is the case where it is behind on purpose and nobody has finished catching it up.
An unfinished backfill is not a failure and does not make the corpus stale.
The last pass succeeded; what CONTEXT holds of that source's history is the
windows already covered and no more. Those are two different facts and reporting
one as the other would be wrong in both directions — a stale warning nobody can
act on, or a corpus silently missing 2019 with nothing saying so. So it is its
own key: context_health carries backfill only while some historical pass has
windows left, and revetly status prints the same lines. Windows done, windows
remaining, where it stopped, and the command that resumes it.
Where it stopped is one of four places, and the checkpoint under
.revetly/backfill.json says which:
| outcome | what happened | what to do |
|---|---|---|
stopped-at-ceiling |
the declared budget would have been exceeded by the next window | raise [backfill] ceiling_episodes / ceiling_spend, or pass --max-episodes, then resume |
interrupted |
somebody stopped it, or the process went away | resume; the checkpoint is at the last completed window boundary |
failed |
the provider refused | a terminal refusal needs credit or a key, not a retry; a transient one already exhausted the declared recovery policy |
running |
a pass is in flight right now | nothing — it is checkpointing as it goes |
Resuming re-sends nothing. A covered window is not even read, so a completed backfill re-run costs no provider call and no API call at the source. A window that was read but never covered — the one the ceiling stopped before, or the one a failure died in — is read again, which is free, and the manifest gate means the artifacts inside it that already landed are not paid for twice: a history artifact declares itself immutable and is gated on identity rather than on bytes.
How far back a source was read outlives the pass that read it.
The backfill key above is about work left to do, so it correctly goes quiet
the moment a backfill finishes. What does not go quiet is the floor: an
instance that read Todoist back to 2025 answers "we have nothing from 2019" for
the life of that corpus, and "there was nothing there" and "nobody paid to look"
are the same silence. So context_health carries a history key naming that
floor per source — with the same sentence on each sources row, beside how
recently that source was read — and the read tools carry it too, once any
backfill has run. A source nothing has backfilled says so rather than saying
nothing. It is a separate key from the staleness warning on purpose: a bounded
backfill is not a failure and reporting it as one earns a warning nobody can act
on. adopting-history.md is the whole of that workflow.
A source with no current state has a floor whether or not anybody backfilled
it. Almost every source has a snapshot an ordinary pass reads
whole; a git repository does not, because its history is its snapshot. A
connector of that shape declares it, and the engine then floors each of its units
— each repository — at the moment that unit was connected to this instance,
recorded in .revetly/connections.json and never re-derived from the current
pass. So the same history key answers per repository: read from 2026-08-21T09:14:03+00:00 (connected); nothing older has been read. That is not
a complaint and not a partial backfill; it is the standing answer to how far back
this corpus goes for that repository, and it is why a query about older commits
is answered as an unread past rather than an empty one.
Three gestures change that floor and none of them is an unattended pass. A floor
declared for a repository is honoured, at the moment of connection, as the
explicit statement it is. A declared floor moved later narrows what is read
and is honoured immediately. A declared floor moved earlier is not read
down to: the span between it and the floor in force is reported as history that
is now available and unread, because a free text edit that arms a spend which
watch, init or an MCP write tool then pays for is exactly what this rule
exists to prevent. Reading it takes revetly backfill, which prices per window
before it sends anything. A repository declared once, removed and declared again
keeps its earlier connection in the record and is floored at the later one,
with the gap between them named as unread.
A repository whose every commit is already at or above its floor has no unread
past, and that is neither an error nor an empty answer: revetly backfill
against it says so by name and sends nothing.
A read that asks about time answers about it twice. get_scope(question='deltas')
takes a window and both records above become part of the answer rather than a
footnote beside it. A source whose last_read_at predates the window has read
nothing that window holds, so the read says the window is not covered for that
source rather than contributing an empty result somebody would read as "nothing
changed" — the per-source freshness record already knows the difference, and this
is the read where the difference is the answer. A window reaching below a
source's history floor carries that floor for the same reason, narrowed to the
sources the window is actually below rather than the general note every read
carries. And the third silence is neither: an artifact whose source declares no
time, or a change stated with no time at all — an undated deletion, which
reads as unknown rather than recent — is reported as unplaceable, in
neither the window nor out of it. All three would otherwise arrive as the same
empty list.
A carried change list says where it stops. Where a source can
enumerate its own change dates, the deltas read reports one revised per edit
inside the window rather than one per artifact — and the dates it carries are
bounded, because an artifact with a thousand commits must not put a thousand
entries into every read of it. So an entry whose list was truncated carries
changes_bounded: how many are carried, how many there were, and the oldest one
that is. A window below that date is a stretch the corpus holds nothing for,
which is the same distinction the floors above draw and not the same as a
stretch in which nothing happened. The dates are metadata about when: what an
artifact said at one of those moments is a version only a backfill reads, and a
revised marked edit_date is not claiming to hold one.
A backfill never makes the corpus look fresher than it is. A window over
2019 is not a read of the source's current view, so it writes no freshness row —
revetly status still says how long ago the snapshot of that source was read,
and watch still takes its own tick on the declared interval. It also holds the
pass lease per window rather than for the whole run, so captures and ticks keep
landing while a decade loads.
Everything above is about a corpus that is behind, or an engine that cannot read it. This is the third case: CONTEXT read something, the source then said something else, and both readings are now in the graph.
Extraction adds; it does not retract. Where an artifact's body was replaced by a different one, its node keeps the old body's facts and gains the new body's beside them, because nothing in the new text contradicts the old text in a way a model was asked to resolve. Re-ingesting adds a third layer rather than removing the first, and the manifest gate means an unchanged artifact is not even re-read.
revetly forget <artifact-id>… is the retraction. It takes back what a provider
extracted for exactly those ids and marks them to be read again on the next
pass; the rest of the corpus is untouched and nothing else is re-extracted.
The corpus gate runs first: the whole gate, revision and
namespace, before the confirmation and before any graph read, so a corpus this
engine cannot read refuses the act rather than the confirmation.
Sources are never touched — this is CONTEXT forgetting a reading, not the
project forgetting a story, and the file the id names is still on disk
afterwards. The re-read costs what those artifacts cost, once, on the next
ordinary revetly ingest.
Reach for it when a read returns something no current source says, and for the
withdrawal half of a source that overwrote rather than edited. It is not the
answer to a stale corpus (that is a pass that has not run) or to a refused one
(that is revetly migrate).
Everything above is about a corpus that is behind its sources. This is the other question: whether the engine in front of the corpus is one that can read it at all.
The engine declares a corpus revision — one ordered integer, CORPUS_REVISION
in corpus.py, owned by the engine and independent of the release version. A
release that only changes behaviour leaves it alone; a release that changes what
an already-written manifest, graph or durable record means bumps it and
declares the Migration that carries an older corpus across. READS_FROM is
the oldest revision the engine still reads unaided.
The revision is written where the data is, twice: in manifest.db's meta
table, so a ledger copied to another machine carries its own answer, and in
.revetly/ingest-health.json beside the passes, so a ledger that was deleted or
reset does not take the answer with it. They are read together and the higher
wins — a manifest of one shape beside a record of another holds data of both,
and refusing to write is recoverable where writing the older shape over the
newer is not.
A corpus carrying no revision is the baseline, not an unknown. Every release cut before this story wrote one shape and recorded no revision, so absence reads as revision 1 — nothing is guessed, re-ingested or rewritten to learn it. A ledger this engine creates carries its revision from the start; a ledger that already holds artifacts and carries no stamp is left exactly as it is.
An instance that has never ingested is not an old corpus. Where there is no ledger, or a ledger with no artifacts in it, there is nothing that could be in an older shape and the reading is this engine's own revision. The two were the same answer while nothing had ever bumped the revision, and they stop being the same answer the first time something does — a fresh install must not be told to migrate data it does not have.
Revision 2 is the first bump. Under the earlier rule a work item whose
type an instance had declared creatable ingested carrying its store's own
word as its kind — Bug, not pbi — with
a level property on the node standing in for the umbrella at the five places
that ask whether an artifact is a work item. This engine asks the kind alone
again, so on revision-1 data such a work item is silently outside the backlog,
outside the supersession stamp and ineligible for a delivery. The step costs
nothing at a provider: it restamps those ledger rows in place and the next
ordinary pass re-asserts the kind in CONTEXT and moves the node onto the story
shelf, because a label is written when a node is created and does not move on its
own.
Revision 3 is why READS_FROM is 3 rather than 2. A
backlog artifact stands on the ProductBacklogItem shelf where revision 2 wrote
Story, and a MATCH against a label nothing carries is not an error — it is an
empty backlog, returned confidently, to every read. That is the silence that
earns a stop rather than a best effort. The step is one statement, renaming the
label in place: no episode is sent, nothing is re-extracted and no provider is
called. Both older revisions are unreadable here, so a corpus written by any
release up to and including 0.2.1 — revision 1, recorded or not — walks
1 → 2 → 3, and revetly migrate runs both steps in order. A gap is a hard stop
rather than a shorter walk: an engine that cannot name the step from 2 to 3 does
not get to run the one from 3 to 4 and call the corpus migrated.
Revision 4 is the first step that reads rather than
rewrites. Prose carries three shelves it did not before — Decision,
Assumption and Constraint, read out of conversations and design docs, which
no source has a field for. Nothing already stored is wrong after it: a note
ingested last month carries none of them because no pass ever looked, so the
node is missing rather than misfiled and no Cypher can mint it. That is why this
step names artifacts for a re-read rather than rewriting what a provider
produced; the only row it writes is the tier it judged.
It is a migration rather than something the next ingest quietly fixes because
the manifest gate is content-addressed. A note nobody edited is unchanged, so an
ordinary pass would never re-send it, and get_scope(question='binding') would answer out
of whichever half of the corpus happened to have been touched since the upgrade
— confidently, while reporting that it was reading the three. An empty answer
from a half-read corpus reads exactly like "nothing binds this", which is the one
answer that read must never give by accident.
So revetly migrate names the artifacts, prices them, and leaves the spend to
an ordinary revetly ingest somebody chooses to run. It starts from the ledger's
live rows, so a document its source has withdrawn is never priced for a
re-read it will never get. Live means the row is not tombstoned, and that is a
correction: until the ledger answered this question behind one name, the pricing
step asked whether the deletion carried a date, which is not "live" but "live,
or deleted with the date unknown", so a document tombstoned by an engine older
than that column was priced for a re-extraction it can never receive. The
corrected price is lower or equal, never higher. Then it asks the sources which
artifacts those are rather than reading kind alone, because the rule is finer:
a decision note is document-kind and is read for none of the three — it already
is the record of everything stated in it — so re-reading one buys a corpus
what it already has. On this project's own corpus, that was 193 of 272 document
artifacts, and naming them would have tripled the bill for nothing.
The step also stamps the tier it decided under. It asks a
document's tier before naming it — a document on index is read by no model
either way, so naming it for a statement re-read would quote a read the pass
that follows would never make — and on a lite instance that is every document.
A document written before tiers existed carries no tier at all, so the step
stamps index on each live pre-tier document row it judged indexed, with no
episode sent and no provider called. The row then reads as what it is, a
document no model has read under this policy, and the promotion rule fires for
it when a provider is declared later. The stamp is idempotent and is part of
what the step does, so it runs on every revetly migrate, including one over a
corpus already at this engine's revision: a row an earlier engine left behind
is reached without the corpus revision moving.
Revisions 5 and 6 are additive, so READS_FROM stayed 3 through them.
Revision 5 stamps the ledger row onto every episode and revision 6 stamps
search_text onto every episode and anchor, and both are derived from what the
graph and the ledger already hold. A revision-3 corpus carries neither stamp and
every read behaves correctly without them: the binding read serves gathered_from saying in words that
no prose-only statement has been extracted here. Disclosed is the opposite of
silent, so this did not earn a stop: an instance is not halted over an
enrichment it has not asked for and may have declined with [extraction] statements = false.
Revision 7 is the first bump that moves READS_FROM off 3. It changes the
shape of what the engine itself wrote, not merely what it knows: a statement
becomes a :Statement node joined by STATED_IN rather than an :Entity node
under a RELATES_TO link, and a declared link becomes a DECLARED
relationship. An engine from revision 7 reads the new shapes and writes none of
the old ones, so a corpus below 7 is one it cannot write to without downgrading
it, and the comparison stops it: revetly doctor and context_health name the
stop and revetly migrate carries the corpus across. The step is free — one
Cypher pass that keeps every uuid and calls no provider, building the indices
over the new shapes before the corpus calls itself revision 7 — and a revision-6 corpus is
refused, not read, until it runs. That is the stop this release's recipients
meet, and it is why READS_FROM is 7 rather than 3.
What revision 7 makes impossible, and what it leaves. After it, Graphiti
cannot damage an engine-owned row, because it never meets one: its candidate
searches match :Entity and RELATES_TO and a statement or declared link is
neither. The damage a revision-6 corpus already carries moves with the row —
the migration is a shape move and restores nothing — so the heal that re-asserts
a declared link's tags and clears a Graphiti invalidation, and
repair_shared_edges, still run over DECLARED as legacy cleanup, and a
statement node whose stamps a merge wiped was read as retired where its link
was invalidated until revision 8 wrote that verdict onto the node. The artifact anchors
are still Graphiti's to resolve mentions onto; what keeps them honest is that
a pass re-asserts their source-of-record properties wherever they may have
moved (the artifact's facts changed, or an episode of that pass touched its
node), every pass puts back a tombstone or a withdrawn id a merge lost from the
ledger's own record, and a full re-stamp of everything read runs once per
[watch] full_interval, per connector, to reconcile anything no rule
foresaw. The identity
repairs a pass once opened with (unsourced, fused and misplaced ids, merged
edges) are revetly repair anchors, run when revetly audit-links reports a
finding.
The damage a revision-6 corpus already carries is revetly repair statements's to undo. The migration restores nothing it moved: a statement
whose stamps a merge wiped still has no stated_in, and a declared edge
Graphiti invalidated keeps its invalidation. The command surveys that damage
for free, prices the one provider call it needs — the re-read of the notes
whose statements were wiped, through the statements pass alone — and repairs it
in one order: re-read first, so a wiped statement's fate is decided (restored
if the note still states it, retired if it does not) before anything clears an
invalid link; then clear invalid_at/expired_at on a STATED_IN edge only
where the node still carries stated_in and no retired_at; then put a
deleted anchor's tombstone, deleted time and clock, and a withdrawn node's
unsourced_id, back from the manifest's own record through the same writer an
ordinary pass runs; then type a retracted edge's invalid_at as a datetime.
It is dry run by default — it surveys, prints the quote and changes nothing —
and --apply takes the pass lock, surveys again, prints the same quote and
asks before the re-read is sent. Every count is printed before and after, a
run over a corpus that needs none of it says so, and a second run changes
nothing and spends nothing. It calls no add_episode, no search and no
embedding, and it deletes nothing: a statement the re-read no longer finds is
retired, never erased. A re-read that comes back stating nothing at all retires
nothing — the note's bytes did not change, so an empty answer says more about
the model than about the note — and its wiped statements stay undecided and
counted. Because the re-read goes through the same retire a pass runs, a
healthy statement of the same note that the new reading reworded is
retired too and its new wording written; the report counts those apart from
the wiped ones. A note the command just read at its source is never
re-tombstoned from a stale ledger row. The revision-7 leftovers it cannot place — a
statement-typed :Entity with no STATED_IN, an inferred RELATES_TO on a
:Statement — are counted and named, not changed.
Revision 8 moves READS_FROM to 8. Revision-7 reads kept a second
retirement rule for statements a graphiti merge wiped: an invalidated
STATED_IN on a node with no stated_in read as retired on the link's time.
Revision 8 writes that verdict onto the node (retired_at from the link,
marked retired_from = invalidated-link, the same write revetly repair statements makes), moves an invalidated link off a wiped node that also has
a live one onto a retired copy, and moves any declared or statement link still
on RELATES_TO. The reads then ask the node's retired_at alone, so a
revision-7 corpus would read its wiped statements as live and is refused until
revetly migrate --apply runs. The step is free and a second run changes
nothing.
A ledger row with no PBI type is a separate one-shot, not a migration step.
revetly backfill-types teaches the ledger the type of every backlog row that
has none — a row whose type CONTEXT does not know is one every parent rule
downstream has nothing to hold a draft to, and the id-prefix parse that used to
paper over it (epic read off EPIC-000) is exactly the guess revetly backfill-types exists to replace. --dry-run counts and names what has none
and writes nothing; --all walks every project this instance declares. It
reads no source and calls no provider. It is deliberately not a step of an
ordinary pass: a corpus should not change shape because somebody ingested.
The comparison runs before anything reads or writes the corpus — ingest, capture,
a write tool, watch — and before any provider call, including on --dry-run.
The commands that take the whole gate are revetly ingest (and --dry-run,
which prices nothing over a corpus that cannot run), revetly backfill,
revetly forget and revetly watch, and the MCP read and write tools. The
namespace half alone is taken by the terminal commands that read the corpus
rather than write it — inspect, branch, binding, drift, deltas,
search, audit-links and eval check — because a read keeps answering from a
corpus that is no longer being fed but must not answer from a namespace the
corpus was not written under. revetly eval clear and revetly rehome take the
namespace half too: the first reaches this instance's graph to clear an eval
namespace, the second prices and retires a routing move, and neither should run
while the corpus in front of this engine has been refused. revetly eval run
takes the namespace half: it writes an isolated namespace of its own and reads
none of the calling corpus, and the pass it runs builds the keyword leg's index
before its first provider call, so the revision of the corpus beside
it is not a fact about the data it scores — the full account is in
scoring-extraction.md:
| stored vs engine | what happens |
|---|---|
| equal | runs, silently |
behind, at or above READS_FROM |
runs, silently |
behind READS_FROM |
stop, naming revetly migrate, corpus untouched |
| ahead of the engine | stop — the data was written by a later engine and continuing would downgrade it |
The ahead case is the deliberate rollback the update path makes easy, which is exactly why it is refused: an older engine does not know about a column, an edge kind or a node property the newer one relied on, so it reads what it understands, ignores what it does not, and rewrites rows in its own shape.
A refused corpus reaches a reader in the same place a failed pass does — the
context_health key on the read tools, which leads with the refusal because it
is the cause and a stale pass may be its symptom — and revetly doctor reports
it as a failure, not a warning: unlike a stale corpus, the wrong engine is
installed for this data and every write path is stopped until that is resolved.
A write tool answers with the refusal and writes no file at all, rather than
committing something and handing back a capture handle for extraction that
cannot happen.
Migration is never implicit. No start, update, ingest or MCP call performs
one. revetly migrate states the steps and, when a step needs artifacts read
again, prices that with the same instrument that prices a pass — the spend is
the ordinary revetly ingest that follows, gated by the ordinary manifest, so a
migration that is half-run costs nothing twice. revetly migrate --apply is the
go-ahead; on a corpus already at this engine's revision it says so and changes
nothing.
A migration takes the pass lock, and a landing never lowers the stamp. A
pass reads the revision at its gate and stamps it when it lands, minutes later.
A migration that lands in between has already written the next revision, and a
pass that wrote its gate's number over it would undo the migration without a
word: revetly doctor would read the old revision again and revetly migrate
would offer the same step twice. Two rules stop that. revetly migrate --apply
takes the pass lock from its first step to the stamp, the lock every pass,
snapshot and restore holds, so it can be run while watch is running. A pass
that is running when it starts is waited for, up to 900 s, the same wait
revetly ingest gives somebody else's pass. It prints whose pass it is, and
past the wait it refuses and changes nothing rather than running beside it.
While the migration holds the lock, a watch tick folds into it. A capture or an
ingest waits for it up to the same 900 s and then runs beside it, as either
does beside a pass. revetly migrate without --apply writes nothing and
takes no lock. The lock is what keeps a migration and a pass apart; the
second rule is what keeps the migration where the lock did not, on a filesystem
that cannot hold one or after a wait that ran out. A pass that lands reads
the stamp again and writes the higher of the two, its gate's revision or the
corpus's now, to both places. So a pass still never claims a revision above
what is stored, and a migration it did not see is kept. When that happens,
.revetly/ingest-health.json records the race with both revisions, and the
doctor's corpus-revision line says the migration landed while a pass was in
flight and that the pass kept it. That is information, not a fault, and the
line goes once a later migration moves the revision on.
The update path asks before it swaps. Each release's statement carries
{"corpus": {"writes": N, "reads_from": M}}, rendered by the engine that built
it, and a statement with no such block is the baseline. ./revetly-update reads
it, compares it against the revision this instance's data is in, and stops
before installing: a release that needs a migration is reported and installed
only if the recipient says --migrate-ok, and a release whose corpus revision is
behind this instance's is refused outright. --check reports the same verdict
and installs nothing. That is asked while the engine that can still read the data
is the one running. The same statement names where the release came from — the
commit it was built from, the date it was cut, a CHANGELOG.md beside the
tarballs that says what changed and which corpus revision each release writes,
and the lanes that passed on that commit — and --check prints them, so "which
engine wrote my graph, proven by what" is answered by the folder.
Revision 5 is a free step. From this revision every
episode the engine records carries the ledger row that sent it — the artifact
id, connector, kind, source path and times, content hash and spend — as
properties on the Episodic node under a revetly_ prefix. That is what makes
manifest.db reconstructible from the graph without a provider (revetly rebuild-ledger). The step stamps every episode already in the graph from the
ledger beside it: a batched property write, no episode sent, nothing
re-extracted. READS_FROM does not move — a revision-4 corpus is read as it
stands, and the start of every pass heals any episode still carrying no stamp —
so nothing refuses while the migration waits to be run; running it is how the
corpus gets there on purpose rather than pass by pass.
Revision 6 is the keyword leg's own. Every episode and its anchor node
gain a search_text property — the title, every id spelling, and the body
already sitting on ep.content and n.body — indexed under Neo4j's own
english analyzer, which is what lets a keyword search stem at all
(revetly manual show extraction-policy has what that buys a reader). The
step, named search-text-backfill, writes the property from what the ledger
and the graph already hold, so it calls no provider and prices at zero —
revetly migrate says so outright, in words, rather than printing nothing for
a step with nothing to reread. It builds the index that property is read
through before it writes a single value, so revetly search answers through
the keyword leg on the very next command rather than waiting for a pass: an
index that is not there is not an empty result, it is a refusal, and a read
that met one used to print Cypher above its answer. A read on a corpus still
missing it says so under keyword_leg, with the reason index-missing and
both gestures that build it. READS_FROM does not move: a revision-5
corpus is read exactly as it stands and simply has no keyword leg until this
step runs. context_health.corpus_revision names the step as pending until
then, and so does revetly doctor's corpus line — how many steps are pending,
to which revision, and the two commands that price and land them. A corpus this
engine reads as it stands is a corpus nothing refuses, so the doctor is where
somebody learns a step is owed at all.
The revision above says what a corpus means. One field decides where it lives,
and until recently it was not in the comparison at all: [project] group_id is
the graph namespace, every node and every edge carries it, and every read filters
on it. Change that line and the engine addresses a namespace that is not the one
holding its data — and three surfaces said it was fine. revetly status
reported a full corpus, because it reads the ledger. context_health reported
the last pass as ok, because it was. search_context returned nothing,
because it reads the graph. Two of the three were confidently wrong.
So the namespace is recorded beside the revision, in the ledger's meta
(corpus_group_id) and in .revetly/ingest-health.json, and compared at the
same gate. It is an optional key rather than a corpus-revision move: the three
revision rows are read by name, an engine that has never heard of this one
passes over it, and nothing about what an already-written manifest, graph or
record means has changed.
Nothing is back-filled. A ledger written before this states no namespace and is judged exactly as it is today — no stop, no guess. The next pass stamps what it writes under, and from then on the comparison has two sides.
A moved namespace is a stop, not a warning. A pass, a read or a write
against a corpus written under a different namespace refuses before anything is
read, sent or billed, and names both namespaces rather than returning an empty
result. Refusing is a judgement rather than a derivation: a typo in group_id
takes the instance down until somebody reads the message, where today it merely
answers nothing — and a stop is visible where a silent empty answer is not.
context_health answers rather than refusing, and names it. It is the read
a client asks outright once every other one has refused, so a refusal there
would leave them nowhere to ask — the same call the corpus revision already
makes, which that tool reports whether or not the corpus is refused. Its
corpus_namespace key carries both namespaces and the three ways out, and it
appears only where the declaration has moved: an instance whose declaration has
not moved grows no key.
The stop prices three ways out and decides none of them.
revetly migrate --carry-namespace prices
relabelling every node, every relationship and every episode carrying the old
namespace; --apply runs it, under the pass lock. Zero provider calls, no
episode sent, nothing re-extracted — the nodes are identical and only the
label differs, so this is revetly migrate's shape rather than revetly rehome's. Like every other migration it is never implicit, priced first and
confirmed, and it is refused into a namespace that already holds a corpus:
merging two populated namespaces is not this and has no undo.revetly reset clears what
the newly declared namespace holds and unlinks the ledger; the next pass
re-extracts the whole corpus at your provider's expense, and what is under
the old namespace stays in the graph with nothing left that reads it. Reset
says that in as many words before it asks, because it is the obvious recovery
from reads that went empty and it is the one that makes the orphaning
permanent. It also forgets the records of passes over the erased corpus:
the passes and namespace in ingest-health.json (the revision stamp
stays), watch.json, sources.json, the backfill checkpoint
backfill.json and a stale watch.pid. The project then reads as never
ingested, and the watch loop waits for a first pass run by hand.A relabel does not rewrite ids that carry the namespace.
derive_artifact_id elides the instance's own namespace, so an artifact this
project owns keeps a bare id and the relabel finishes the job for it. A
connector that defaults its own instance half to the group — Todoist's
account, undeclared — mints todoist:<group>:<native>, and those ids move
when the declaration does: every one of them reads as new on the next pass and
is re-extracted at your provider's expense. The stop names those connectors and
how many rows each has, read off the ledger rather than guessed, and says the
one line that decouples them: declaring [project] source_instance pins the id
half to the name it already has.
revetly doctor and revetly status say it before anything acts. Both
name the namespace the corpus was written under and whether it is the declared
one — on every run, not only when something is wrong: "this corpus lives where
you say it does" is the sentence whose absence let a changed group_id read as
an empty corpus, and a line that appears only on failure cannot be checked
beforehand. A corpus that states no namespace is reported as stating none.
Everything above is about a corpus behind its sources or ahead of its engine.
There is a third state: the ledger says an artifact was sent and the graph no
longer holds the episode — a volume reset or replaced beside a surviving
manifest.db. Nothing above catches it, because the hash gate reads every body
as unchanged and the pass reports zero episodes with a clear conscience.
Every pass now asks the graph for the ledger's live episodes by uuid and reports
the rows it cannot find as graph_missing, by connector; context_health
answers corpus: divergent with the count and the way out, revetly status
prints the same line, and revetly doctor carries a ledger and graph check
that asks live. A divergent corpus is stronger than a stale one: stale is behind
and a pass fixes it, divergent answers from less than it says it was sent and
no pass fixes that by itself. The two ways out — restoring a snapshot for
nothing, or pricing a re-extraction of exactly those rows — and the snapshots
themselves are snapshot-and-restore.md.