GAINSEC / REFALLSYSTEMS & OPERATING CODEX
READY ARMOR / REFALL CONTROL · CAPABILITY · EVIDENCE

REFALL AB

Persistent research loops. Bounded work. Reviewable evidence.

REFALL AB makes a research campaign persistent across worker conversations and manager sessions. Shared coverage, evidence, obligations, scoped launch decisions, and checked closeout connect bounded work to reviewable conclusions.

01Understand the approachPurpose, authority, and useful outcomes
02Trace the mechanismsComponents, decisions, and durable state
03Operate the productInstallation, workflows, and references
01 / IN PRACTICE

Use cases and outcomes.

Source security research

Combine measured source examination with attack-surface review, candidate findings, verification, and remediation work.

A reviewable evidence trail, explicit unresolved work, and findings linked to source and reproduction material.

Reverse engineering and architecture mapping

Accumulate component determinations, interface observations, flow relationships, saved analysis, and a technical narrative.

A persistent codex that later workers can extend without repeating the entire examination.

Compiled-binary investigations

Use the standalone binary family to seed function questions, examine decompilation, challenge claims, and collect runtime proof.

Function-anchored research, explicit validation blockers, and reproducible finding packages when proof is available.

Long research campaigns

Coordinate bounded specialist work across interrupted execution, provider changes, unresolved decisions, and repeated report revisions.

Resumable subject state with archived evidence, current analyst notes, and an inspectable closeout decision.
READY ARMOR / CONTROL IN DEPTH

Build control from the boundary up.

REFALL controls campaign resources, brokered launch decisions, scheduled work, and evidence acceptance. Runtime mounts and tool permissions supply the execution boundary. Recorded receipts keep scope decisions and completion reviewable.

Lowest implemented boundary first. Select a layer to inspect its checks, activation, and decision scope.

LAYER 01operation gate

Source and workspace projection

Boundary
Host source directory to the configured Agent Canvas container.
Implemented mechanism
Resolve the source directory, validate the subject name, inspect the container, and reuse its matching mount or copy source files.
Decision or effect
Invalid source directories, unsafe subject names, and failed projection cannot complete this synchronization route.
Activation and coverage
Backend launch supplies an available source path. Existing Docker mounts retain their configured access mode.

Code-path example

INPUT

Project a missing source directory or an unsafe subject name.

RESULT

Synchronization raises AgentWorkspaceError. A copied directory remains an ordinary container directory.

LAYER 02observation

Content identity and source observations

Boundary
Configured source root to the campaign inventory.
Implemented mechanism
Hash file content and record relative paths, file facts, unreadable files, and symlink observations.
Decision or effect
No worker operation is blocked by inventory collection. Membership supports later finding checks.
Activation and coverage
Inventory enumeration runs for source intake or an explicit inventory refresh.

Code-path example

INPUT

A file symlink resolves outside the source root.

RESULT

Record an off-root observation. File facts can still read the resolved file.

LAYER 03operator authority

Authenticated capability changes

Boundary
Capability grant and revoke HTTP routes.
Implemented mechanism
Compare X-REFALL-Operator-Token with the configured token file. Replace the supplied grantor with operator after authentication.
Decision or effect
Unauthenticated grant or revoke requests are refused when token protection is configured.
Activation and coverage
Set REFALL_CAPABILITY_OPERATOR_TOKEN_PATH to a readable token file. The default unset value disables token authentication and retains caller-supplied grantor identity.

Code-path example

INPUT

Submit a grant without the configured operator token.

RESULT

HTTP 403; no grant is stored. An unreadable credential file gives HTTP 503.

LAYER 04policy gate

Scoped authorization decisions

Boundary
An action request presented to the capability broker.
Implemented mechanism
Require a class and declared effects. Match an active grant against applicable target and effect conditions.
Decision or effect
Requests with no matching unexpired grant receive a refusal decision.
Activation and coverage
Call /execution/authorize or use a backend caller that consults the store. Scope checks depend on populated request and grant fields.

Code-path example

INPUT

Request a host, method, or path outside the configured grant.

RESULT

Return accepted:false with refusal reasons and record the API decision. This decision-only route launches no command; callers must obey refusal.

LAYER 05operation gate

Backend process release

Boundary
Named Start or Update command to the standing-loop process.
Implemented mechanism
Apply operation switches and the command allowlist. Authorize the subject-path run_local_command request before launch.
Decision or effect
A refused capability decision prevents this backend route from starting the loop process.
Activation and coverage
Enable REFALL_ENABLE_OPS and REFALL_ENABLE_OPS_COMMANDS; both default false. Loop launch defaults true. Disabled launch skips authorization and starts no process.

Code-path example

INPUT

Start a subject without a matching process grant.

RESULT

HTTP 403 before process launch. Start without stored context returns HTTP 409.

LAYER 06task gate

Work selection and claim allocation

Boundary
Stored work to a scheduled subject, unit, or finding assignment.
Implemented mechanism
Prioritize obligations, subject setup, fresh breadth, finding work, and later unit passes. Use claims and per-target progress.
Decision or effect
Eligible selection branches skip modules with unmet dependencies, completed target work, duplicate active target work, matching exclusions, or exhausted module slots.
Activation and coverage
Standing-loop selection applies branch-specific conditions. Fresh breadth bypasses module eligibility; later unit selection omits unit_state. Global concurrency still applies.

Code-path example

INPUT

A later unit pass does not match its path filter or has an unmet dependency.

RESULT

Skip that pass. Fresh characterization uses direct coverage claims and the global conversation limit.

LAYER 07task gate

Conversation and recovery bounds

Boundary
A scheduled assignment to the Agent Canvas conversation API.
Implemented mechanism
Send the task workspace, provider settings, objective, and max_iterations. Classify failures before bounded retry.
Decision or effect
Recognized scope, capability, and operator-stop outcomes do not receive ordinary automatic worker retries.
Activation and coverage
The normal dispatcher sends conversation limits to Agent Canvas. Fresh characterization requests 30 iterations; recoverable worker retries default to two attempts.

Code-path example

INPUT

A worker reports scope_blocked or capability_missing.

RESULT

Preserve available output and create an operator or capability obligation instead of repeating the action.

LAYER 08model guidance

Scope and workspace instructions

Boundary
Campaign context to specialist reasoning and tool choice.
Implemented mechanism
Build an objective from stored context, scope, target, module, routing, output requirements, source location, and scratch location.
Decision or effect
These instructions request in-scope behavior. They do not mechanically prevent external tool actions.
Activation and coverage
The dispatcher constructs the specialist initial message.

Code-path example

INPUT

An assignment identifies source as read-only and provides a scratch directory.

RESULT

The worker receives explicit writing instructions. Actual write permissions come from mounts and process access.

LAYER 09task gate

Continuation input comparison

Boundary
A checkpointed campaign to a resumed standing loop.
Implemented mechanism
Compare recorded input hashes with current source and configuration hashes through resume preflight.
Decision or effect
A returned drift refusal stops the supported resume path and exposes a decision obligation.
Activation and coverage
Prior checkpoints and the preflight API are available. Missing optional support and some service errors bypass refusal.

Code-path example

INPUT

Resume with changed checkpoint inputs.

RESULT

Record resume_preflight_refused and stop that continuation path.

LAYER 10observation

Continuous result capture

Boundary
Worker events and workspace results to persistent campaign evidence.
Implemented mechanism
Read conversation events and submission files during execution. Terminal handling classifies, parses, checks identity, and preserves observations before handling the classified outcome.
Decision or effect
Observation itself does not stop the worker action that produced a result.
Activation and coverage
Standing-loop polling and completion run ingestion. Structured terminal submission remains optional unless REFALL_REQUIRE_STRUCTURED_SUBMIT is enabled.

Code-path example

INPUT

A worker fails after it produced useful facts or artifacts.

RESULT

Preserve available output, classify the failure, and expose repair or operator work.

LAYER 11record gate

Finding admission and declared scope

Boundary
A proposed finding to canonical finding storage.
Implemented mechanism
Require evidence, confidence, and source or artifact references. Check available inventory membership and explicit external-target scope.
Decision or effect
Unsupported or out-of-scope candidates are refused admission to the finding store.
Activation and coverage
add_finding evaluates a proposed record after worker activity. Inventory membership is advisory when inventory is unavailable; external URL hosts require explicit scope.

Code-path example

INPUT

A candidate cites https://outside.example.test without that host in explicit external_targets.

RESULT

Refuse promotion. The API records the refusal and creates review work.

LAYER 12record gate

Typed finding transitions

Boundary
Existing finding state to a new status or quality tier.
Implemented mechanism
Check current status, allowed states, receipt types, evidence references, pending contradictions, and supplied source hashes.
Decision or effect
A refused transition preserves the existing finding state and records the refused attempt.
Activation and coverage
The transition path receives a proposed decision. Hash freshness checks activate when the receipt supplies source or configuration hashes.

Code-path example

INPUT

Mark a finding FALSE_POSITIVE without an accepted receipt type and evidence.

RESULT

Refuse the transition. Live validation, duplicate, downgrade, and chain deferral have separate receipt requirements.

LAYER 13record gate

Persistent resolution obligations

Boundary
Open or claimed pending work to a terminal resolution.
Implemented mechanism
Keep missing access, blocked scope, incomplete evidence, and refused decisions in pending records. Validate required resolution receipts.
Decision or effect
Required receipt failures prevent pending-action resolution; open obligations can block closeout.
Activation and coverage
Resolve, defer, or cancel a pending action. Required types vary by action; an untyped done action does not universally require evidence.

Code-path example

INPUT

Defer an obligation without evidence references.

RESULT

Reject the resolution. Required receipt types must match and include evidence.

LAYER 14record gate

Report replacement protection

Boundary
Enriched report content to the current report object.
Implemented mechanism
Compare populated report sections before replacing the current formal report. Retain a refused candidate as a diagnostic object.
Decision or effect
An unapproved structural downgrade cannot replace the current report through this assembly path.
Activation and coverage
assemble_report defaults allow_structural_downgrade to false. Explicit override bypasses this guard. Generic object writes use a separate API path.

Code-path example

INPUT

Replace a rich report with a candidate that loses protected sections.

RESULT

Return downgrade_refused and keep the current report.

LAYER 15record gate

Campaign closeout acceptance

Boundary
Current campaign records to a closed lifecycle result.
Implemented mechanism
Check pending actions, receipts, refused transitions, finding proof, expired claims, context, and report freshness.
Decision or effect
Blocking failures prevent a successful closed result from the closeout operation.
Activation and coverage
Named Closeout or the loop manager evaluates the gate. Draft/current reports can already exist; this gate controls the closed lifecycle result.

Code-path example

INPUT

Close a campaign with open pending work or a stale report.

RESULT

Return blocked and record loop.closeout.blocked. Coverage below 95 percent is advisory.

LAYER 16operator authority

Recorded closeout exceptions

Boundary
A declared operator exception to selected closeout checks.
Implemented mechanism
Require check IDs and a nonempty receipt. Append an accepted override entry and make matching failed checks nonblocking warnings.
Decision or effect
Missing check IDs or receipt prevent this override route from accepting the exception.
Activation and coverage
The override route receives an explicit decision record. It records the supplied operator name without capability-token authentication; deployment access control establishes caller authority.

Code-path example

INPUT

Accept a selected closeout exception with a receipt and reason.

RESULT

Keep the failed condition visible as an overridden warning and retain the ledger reference.

02 / SYSTEM ARCHITECTURE

Follow the control and data.

Read the layers, then select a component to inspect its responsibility and implementation.

CONTROLCONTROLEXECUTIONEVIDENCEPUBLICATIONdirectscommandsrequests authorizationpermits launchassignsusesproducespersistsinforms next worksupports reviewrecords receiptssupplies evidencepreserves run statesupports continuationsupplies current reportsupplies obligationsauthorizes exceptions01OperatorDefines objective, scope,resource authority, and02Workbench consolePresents context, coverage,evidence, decisions, and03Backend operationsGenerates context and executesauthorized lifecycle commands.04Capability brokerMatches declared actions toscoped grants.05Finding gatesCheck candidate evidence and receipt-backed state changes.06Module schedulerSelects bounded work from subject andtarget state.07Specialist conversationsRun the assigned research objective inconfigured execution workspaces.08Analysis resourcesSupply source tools, decompilation,harnesses, and optional MCP services.09Evidence captureReads event messages, result files, andsaved artifacts.10Research stateStores inventory, coverage, facts, leads,findings, and obligations.11Runtime continuityMaintains active work, notes, manifests,checkpoints, and supervision.12Reports and notesCompile persistent evidence while preserving technical detail.13Closeout gateDistinguishes completion barriers, warnings, and overrides.
Select a component to examine its role and source entry points.
03 / CAPABILITY ATLAS

What it does. How it works.

Each capability connects an operating purpose to an implementation mechanism and its source entry points.

28 capabilities
05 / CAPABILITY

Modules with local dependencies

Combine breadth, decomposition, review, verification, research, and development within one subject.

Subject, finding, and later unit selection apply dependencies and module limits. Later unit passes also apply recorded-path filters. Fresh characterization uses coverage claims and global concurrency.
07 / CAPABILITY

Breadth before repeated depth

Keep an initial component subset from consuming the whole campaign.

Pending actions and subject tasks run first. Fresh units precede finding tasks and later unit passes.
14 / CAPABILITY

Finding state and proof strength stay separate

Distinguish a workflow decision from the quality of its evidence.

Six lifecycle states coexist with hypothesis, code-confirmed, and live-confirmed tiers. Valid promotion requires a live reproduction receipt.
25 / CAPABILITY

Optional analysis resources

Use semantic tools, symbol indexes, graphs, metrics, and focused property-graph analysis.

The installable toolchain supplies source analysis and managed MCP services. Subject tasks select the resources they need.
26 / CAPABILITY

Research ledger and Output view

Facts, leads, findings, validation, and operator decisions remain with the subject after individual conversations end.

An append-only JSONL ledger supplies filtered queries, statistics, the Ops Output projection, and report inputs. Finding and authority APIs manage their separate state transitions.
27 / CAPABILITY

Wishlist fulfillment and delivered-resource briefing

Missing tools and other research resources become explicit requests with recorded delivery.

The watchdog checks host executables, optionally runs allowlisted installers, and appends provided rows and ledger receipts. Head cycles and research-driver startup snapshots carry delivered items into their own worker objectives.
28 / CAPABILITY

Campaign cycles and target-specific passes

A coordinator cycle, a module pass, a worker retry, and report or supervision review have distinct state and completion conditions.

Pending actions preempt setup, fresh breadth, finding work, and later unit work. Target-local dependencies and completion records govern module eligibility. Coordinator limits preserve active work for continuation.
04 / END-TO-END TRACES

A complete path through the system.

Follow each operation from its entry point through decisions, state changes, evidence, and review.

The campaign cycle

One coordinator pass selects work, polls conversations, retains evidence, refreshes records, and saves continuation state. A dashed return represents another eligible coordinator cycle.

SELECTIONEXECUTIONEVIDENCE AND DECISIONSCONTINUATION01Select recorded workClaim worker-eligible pending actions first. Then consider threat-model or subject setup, fresh coverage, finding work, and later unitmodules.02Dispatch bounded workPrepare the task workspace and provider lane. Send the taskobjective with its conversation iteration limit.03Poll active conversationsRead active worker status. Ingest available events and structuredfiles while a conversation remains active.04Retain evidence and decideClassify terminal results, parse output, check submissionidentity, and retain useful observations before outcome handling.05Refresh shared recordsUpdate determinations and applicable report state. Report andvisualization workers have their own cadence and slot rules.06Save and continue or returnPersist active conversations and coordinator state. Module completion has a separate progress store. Continue when configured andeligible; idle, stalls, or limits can return control.
Select a node to inspect its behavior and source. Dashed arrows return to a later cycle.
  1. 01

    Select recorded work

    Claim worker-eligible pending actions first. Then consider threat-model or subject setup, fresh coverage, finding work, and later unit modules.

  2. 02

    Dispatch bounded work

    Prepare the task workspace and provider lane. Send the task objective with its conversation iteration limit.

  3. 03

    Poll active conversations

    Read active worker status. Ingest available events and structured files while a conversation remains active.

  4. 04

    Retain evidence and decide

    Classify terminal results, parse output, check submission identity, and retain useful observations before outcome handling. Completion and retry use their own checks.

  5. 05

    Refresh shared records

    Update determinations and applicable report state. Report and visualization workers have their own cadence and slot rules.

  6. 06

    Save and continue or return

    Persist active conversations and coordinator state. Module completion has a separate progress store. Continue when configured and eligible; idle, stalls, or limits can return control.

Passes on one target

Example security-pack path. Unit dependencies are local to that unit. Finding verification targets finding records; review does not have to produce a finding.

COVERAGE TARGETFINDING TARGETDECISIONsame-unit dependencysame-unit dependencycandidate finding availableverification result01Characterize a unitDefault security assessment pulls incharacterization through its dependencies.02Map its attack surfaceThe security reconnaissance module followscharacterization on the same target.03Review concrete risksThe security review module followsreconnaissance for that unit. Retain source04Select finding verificationWhen an eligible finding exists and finding work is selected, the security verification module receives that finding as its target.05Record an evidence-backed decisionPromotion and disposition use their own evidence and receipt checks. Accepted and refused decisions remain available to later work andcloseout.
Select a node to inspect its behavior and source.
  1. 01

    Characterize a unit

    Default security assessment pulls in characterization through its dependencies. Explicit baseline selection supports characterization-only work.

  2. 02

    Map its attack surface

    The security reconnaissance module follows characterization on the same target. Identify inputs, parsers, privileges, and trust boundaries.

  3. 03

    Review concrete risks

    The security review module follows reconnaissance for that unit. Retain source evidence and candidate findings through the corresponding persistence paths.

  4. 04

    Select finding verification

    When an eligible finding exists and finding work is selected, the security verification module receives that finding as its target.

  5. 05

    Record an evidence-backed decision

    Promotion and disposition use their own evidence and receipt checks. Accepted and refused decisions remain available to later work and closeout.

The resource feedback cycle

Tool delivery is a recorded resource workflow. Head cycles reread delivered items; compatibility research drivers load them at startup. The standing scheduler selects its own work queues.

REQUESTFULFILLMENTPERSISTENT RECORDSREUSERESEARCHprovided or installation succeeds01Record a resource needCreate an open wishlist item with its kind, item, reason, and requester. Access and data needs can remain for operator handling.02Check host availabilityThe watchdog reads open and provided records. It skips matchingprovided tools and checks the host executable path for supported03Provide a supported toolAn already available executable can be recorded as provided. Whenenabled, a fixed allowlisted installer can supply a missing tool;04Record delivery and its receiptAppend a new provided row and an ops-event ledger receipt linking the source request and delivery. The original request remains inhistory.05Brief later workersEach head cycle reads provided items. The research driver and three binary drivers include their startup snapshot in worker objectives,with instructions to avoid duplicate requests.06Continue authorized researchUse resources in the appropriate host or worker environment. Resolve related obligations and obtain any required action grants throughtheir separate APIs. A later need can create another wish.
Select a node to inspect its behavior and source. Dashed arrows return to a later cycle.
  1. 01

    Record a resource need

    Create an open wishlist item with its kind, item, reason, and requester. Access and data needs can remain for operator handling.

  2. 02

    Check host availability

    The watchdog reads open and provided records. It skips matching provided tools and checks the host executable path for supported request kinds.

  3. 03

    Provide a supported tool

    An already available executable can be recorded as provided. When enabled, a fixed allowlisted installer can supply a missing tool; success must pass an executable check. Unhandled requests remain visible.

  4. 04

    Record delivery and its receipt

    Append a new provided row and an ops-event ledger receipt linking the source request and delivery. The original request remains in history.

  5. 05

    Brief later workers

    Each head cycle reads provided items. The research driver and three binary drivers include their startup snapshot in worker objectives, with instructions to avoid duplicate requests.

  6. 06

    Continue authorized research

    Use resources in the appropriate host or worker environment. Resolve related obligations and obtain any required action grants through their separate APIs. A later need can create another wish.

From operator intent to bounded work

The normal workbench lifecycle reaches one authorized backend launch path and a persistent scheduler.

  1. 01

    Create the subject

    Attach the source path to a subject workbench with its own research state.

  2. 02

    Compile shared context

    Generate a briefing from selected source material, metadata, workbench text, and operator direction.

  3. 03

    Select work and scope

    Choose packs, custom modules, path rules, scope constraints, and optional threat-model work.

  4. 04

    Check readiness

    Inspect providers, services, paths, capabilities, and remaining work through Doctor and Estimate.

  5. 05

    Authorize launch

    Compare the process request and subject path with a grant. Refusal blocks backend launch. Grant authentication activates with the configured operator token.

  6. 06

    Prepare the execution workspace

    Reuse the source mount or copy its snapshot. Create separate writable task scratch.

  7. 07

    Inventory the material

    Record file identity and establish coverage units; request a plan for binary-only intake.

  8. 08

    Choose the next target

    Prioritize obligations and subject setup. Claim fresh breadth directly before eligible finding work and later unit passes.

  9. 09

    Build the task contract

    Combine context, scope, shared notes, the selected target, module intent, and required outputs.

  10. 10

    Dispatch a specialist

    Start the configured conversation with its lane, workspace, objective, and iteration limit.

From worker output to reviewable evidence

Observations are captured independently of whether the worker can complete its assignment.

  1. 01
  2. 02
  3. 03

    Normalize the result

    Parse typed envelopes and supported facts, leads, findings, flows, and determination fields.

  4. 04

    Prevent repeated ingestion

    Use stable observation keys and persistent submission identities across live and terminal paths.

  5. 05

    Store canonical evidence

    Append ledger records and preserve finding locations, artifacts, and scheduler inputs.

  6. 06

    Evaluate completion separately

    Classify the invocation, parse output, and check submission identity. Preserve observations before handling success, scheduling repair, or creating obligations.

  7. 07
  8. 08

    Validate the candidate

    Check evidence, confidence, source membership, scope, and claimed verification material.

  9. 09

    Record the disposition

    Require the appropriate receipt for promotion, suppression, duplicates, deferral, and quality changes.

  10. 10

Adapt one campaign to its subject

Work declarations connect broad examination with focused research without a global phase counter.

  1. 01

    Select the target family

    Use the whole subject, coverage units, or findings as the work source.

  2. 02

    Compose packs

    Combine characterization, reverse engineering, security, research, development, or analysis packs.

  3. 03

    Add local questions

    Declare a custom module with its task class and bounded objective.

  4. 04

    Constrain expensive work

    Apply path globs to later unit passes and module concurrency limits to subject, finding, and later unit selection.

  5. 05

    Resolve prerequisites

    Add built-in dependencies and reject unknown or cyclic module relationships.

  6. 06

    Establish breadth

    Claim fresh coverage units directly and dispatch characterization under the global concurrency limit.

  7. 07

    Reuse earlier results

    Supply prior summaries, findings, shared notes, and artifacts to follow-on passes.

  8. 08

    Challenge findings

    Dispatch finding-specific verification or patch work when eligible.

  9. 09

    Select required tools

    Install and start supporting analysis resources only for the subject work that needs them.

  10. 10

    Persist target progress

    Save completed module identifiers and known targets so later execution continues from the same state.

From long-running research to checked publication

Persistent execution state and report state preserve both the work and the conditions for accepting it.

  1. 01

    Record run identity

    Store active conversations, source identities, lane configuration, stage state, and checkpoints.

  2. 02

    Handle temporary failure

    Apply transport retry or classified worker repair within the configured limits.

  3. 03

    Escalate unresolved conditions

    Preserve scope, capability, exhausted retry, or recovery decisions as visible obligations.

  4. 04
  5. 05
  6. 06

    Enrich from evidence

    Group facts, retain related locations, and include runtime proof and decision records.

  7. 07

    Protect the current report

    Compare populated sections before replacing the selected report object.

  8. 08

    Refresh the authoritative draft

    Include evidence written during enrichment before evaluating publication freshness.

  9. 09

    Evaluate completion

    Inspect blocking obligations, validation, receipts, claims, context, and report freshness.

  10. 10

    Record the accepted outcome

    Resolve barriers or record explicit override receipts, then preserve the closeout decision.

05 / FIELD GUIDES

The complete technical reading.

Product concepts, architectural decisions, source-linked mechanisms, and operating instructions.

FIELD GUIDE / 0116 MIN READ

REFALL AB

Persistent research loops for source code, binaries, and evidence.

REFALL AB coordinates specialist agents around a shared research workbench. It turns a subject into bounded work, records what each worker examined, and preserves the evidence behind each conclusion. A subject can continue across many conversations, interrupted workers, report revisions, and manager sessions.

The operator defines the objective and scope. The manager selects work and execution resources. Specialists inspect source, examine binaries, build harnesses, challenge claims, or compile reports. The workbench stores context, coverage, facts, leads, findings, artifacts, and outstanding decisions.

Why the design is distinctive

A long investigation must retain more than its latest answer. It must preserve which material was examined, why a conclusion was accepted, and which questions remain open. REFALL makes those records part of the research system.

The subject owns the state. Workers receive bounded assignments, contribute observations, and return results through defined record paths. Later work can reuse the inventory, determinations, artifacts, finding history, and unresolved obligations after a conversation ends or a manager restarts.

The distinctive design connects four decisions:

DecisionMechanismWhy it matters
What should run next?Coverage claims, target-local module progress, and pending-action priority.Broad examination and focused follow-up share the same campaign.
What authority was granted?Scoped capability records and a backend launch caller that obeys broker refusal.An assignment and permission to perform an action remain separate decisions.
What conclusion is supported?Captured evidence, finding admission, and typed disposition receipts.A worker's narrative does not silently become an accepted finding state.
What counts as finished?Separate worker, module, report, and closeout conditions.Captured output, an idle loop, and campaign completion have different meanings.

These mechanisms reinforce each other through shared records. An interrupted worker can leave useful evidence without completing its assignment. A rejected or deferred finding can retain its proof and next obligation. A later worker or configured provider can extend that work without replacing the subject's history.

REFALL applies the Ready Armor approach at the campaign and evidence boundaries. Control in Depth separates the decisions; persistent records keep their results available to later workers and operators. These campaign controls run in backend services on the deployment host. Their enforcement comes from API admission, database transactions, authorization decisions, and checked record transitions. They do not require a virtual machine or container as their security foundation.

For example, backend Start and Update refuse an unauthorized launch before the new loop process starts. Finding admission and disposition routes also evaluate stored evidence and receipt requirements independently of the worker's prompt. The host supplies filesystem, network, and process permissions for tools that run directly. REFALL controls campaign decisions and evidence acceptance; direct specialist commands use the execution environment's permissions. Read Ready Armor in REFALL for the composition, practical value, and implementation paths.

What REFALL does

A research campaign needs more than a sequence of agent replies. It needs a measured examination surface, clear ownership, reusable observations, and an explicit account of unfinished work. REFALL supplies those records and connects them to execution.

CapabilityResult for the operator
Subject workbenchesContext, evidence, decisions, and reports remain together for each target.
Evidence ledgerFacts, leads, findings, validation, artifacts, and operator decisions remain queryable after a worker ends.
Resource wishlistMissing tools or access become recorded requests, with provided records and delivery receipts.
Source inventoryFile identities and coverage units define the recorded examination surface.
Work modulesResearch, decomposition, review, harness creation, and verification can share one subject.
Specialist lanesReasoning, implementation, and verification work can use different providers.
Continuous evidence captureResults appear while workers run and survive incomplete conversations.
Finding gatesPromotion, rejection, duplicates, and deferral retain their supporting evidence.
Scoped capabilitiesGrants control requests presented to the broker; backend loop launch obeys its decision.
Persistent obligationsMissing access, failed output, and unanswered scope decisions remain visible.
Run continuityActive work, module progress, checkpoints, and shared notes survive a manager restart.
Evidence graphRecorded observation nodes and supplied relationship data form an inspectable evidence map.
Campaign supervisionLiveness checks, progress review, recovery limits, and configured usage ceilings support long campaigns.
Living reportsFindings, architecture, runtime proof, and analyst notes develop with the research.
Mechanical closeoutThe operator can inspect exactly which conditions prevent completion.

Research runs in connected cycles

REFALL separates the campaign, the target-specific work, and each specialist conversation. A campaign can continue while individual workers finish, retry, or wait for a decision. The next assignment uses recorded state and the enabled module definitions.

STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 2STAGE 201Objective, scope, andsubject context02cope03ope04pe05e06elect07lect08ect09ct10t11ispatch12spatch13patch14atch15tch16ch17h18apture19pture20ture21ure22re23ecide24cide25ide26de27tate28ate29te30eport31port32ort33rt34Select eligible work35Dispatch boundedspecialist work36Capture observations andartifacts37Evaluate result andretain evidence38Update coverage, obligations, andmodule progress39Refresh the living report40Check closeout conditions
Control and data flow

The campaign cycle

The standing loop fills available worker slots, polls active conversations, captures new evidence, updates reports, and saves run state. A polling cycle can collect partial output while a worker remains active. Worker completion has a separate outcome decision.

The scheduler selects the next task in this order:

  1. Claim an eligible pending action.
  2. Handle threat-model work and eligible subject-level setup modules.
  3. Claim a fresh coverage unit when breadth work is enabled.
  4. Select eligible finding work, such as verification or a patch proposal.
  5. Select later module work on previously examined units.

Dependencies apply to each target. A component can enter review while another component still needs its first examination. The default registry enables security assessment and adds its required characterization dependency. Select the baseline pack explicitly for characterization-only work.

The target-specific cycle

The security assessment pack connects characterization, reconnaissance, and review for each coverage unit. Finding-specific verification uses the findings produced by the research.

STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 2STAGE 201One coverage unit02nit03it04t05haracterize06aracterize07racterize08acterize09cterize10terize11erize12rize13ize14ze15e16econ17con18on19n20eview21view22iew23ew24w25inding26nding27ding28ing29ng30g31erify32rify33ify34fy35y36Characterize37Security reconnaissance38Security review39Candidate finding40Finding verification41Evidence and disposition decision
Control and data flow

Other packs add reverse engineering, general research, development, or toolchain work. A subject can combine packs and add custom modules with their own objectives and dependencies. The development pack includes a finding-scoped patch proposal. Patch verification requires the corresponding workflow configuration or assigned follow-up work.

Built-in modules run once per target. A custom module can set once_per_target: false to permit repeated eligible work. A pending action can request a focused investigation when a result needs more proof or an operator decision.

The worker and supervision cycles

Each specialist receives a bounded task and an output contract. While it runs, REFALL can ingest facts, leads, findings, flows, and workspace artifacts. At termination, the executor retains useful evidence before it handles failure, retry, or successful completion. A failed conversation can contribute observations without completing its coverage unit.

The watchdog reviews services, progress, dependencies, and configured runtime, token, and cost limits. The supervisor combines frequent liveness checks with broader review and bounded recovery. A coordinator cycle limit controls polling work; active worker state can remain available for a later continuation. Repeated recovery can end in an operator hold.

Read Research cycles for the exact scheduling, retry, completion, and closeout paths. Open the interactive cycle maps to inspect the nodes and their source references.

The ledger is the campaign's research journal

The subject ledger preserves evidence and decisions outside the current conversation. Entries can record facts, leads, findings, validation, artifacts, runtime events, capability decisions, pending actions, and closeout results. Each appended record receives an identifier and timestamps when those fields are absent.

The workbench stores the ledger as append-only JSONL. Queries can filter entries by kind, specialist lane, severity, and time. Statistics summarize the recorded kinds, lanes, and severities. This gives a replacement manager a durable account of what was examined, what changed, and why.

The Ops panel exposes a Ledger view and an Output view. Output compiles selected research records from the ledger. The report compiler also reads ledger entries and statistics alongside coverage and runtime state. A later report can therefore reuse earlier observations, validation, and artifact references.

Finding acceptance, coverage completion, and ledger capture have separate responsibilities. The finding gates validate accepted conclusions. Coverage tracks examined units and their determinations. The ledger retains the evidence and decision history behind those records.

Sources: ledger storage, ledger API, Ops panel.

The wishlist turns missing resources into visible requests

A specialist can record a missing tool, dependency, access requirement, or other research need in the subject wishlist. The request carries its kind, item, reason, requester, and status. This preserves the need for the next operator or worker.

STAGE 2STAGE 2STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1command succeeds and executable is foundcommand succeeds and executable is foundcommand succeeds and executable is foundcommand succeeds and executable is foundcommand succeeds and executable is foundcommand succeeds and executable is foundcommand succeeds and executable is found01Research identifies amissing resource02Record an open wishlistitem03Watchdog checksrequested tools04Tool already available05Configured allowlistedinstallation06Append a provided recordand ledger receipt07Head cycle and researchdrivers read delivered08Continue with theavailable resource09eed10ed11d12pen13en14n15heck16eck17ck18k19vailable20ailable21ilable22lable23able24ble25le26e27nstall28stall29tall30all31ll32l33rovided34ovided35vided36ided37ded38rief39ief40ef41f42esearch43search44earch45arch46rch47ch48h
Control and data flow

For supported tool requests, the watchdog checks whether the executable is already available on its host. When installation is enabled, it can run a fixed installer command for an allowlisted tool. The current installer table covers Binwalk, ExifTool, radare2, Rizin, and YARA, including the configured command aliases. Successful installation must also pass an executable availability check.

Delivery appends a separate provided wishlist row. An ops-event ledger entry links the original request, the provided record, and the delivery receipt. The original open request remains in history. The watchdog skips matching tools that already have a provided record.

The head orchestrator reads provided items for each head cycle. The research driver and three binary drivers load delivered items at startup and include that snapshot in their worker objectives. Their instructions tell workers to use delivered resources before requesting duplicates. Other resource needs remain available for operator handling. Host installation and worker-container availability are distinct deployment steps.

Wishes, obligations, and authority

RecordWhat it controls
Wishlist itemA requested or provided research resource.
Pending actionAn unfinished obligation, with ownership, status, and any required resolution receipt.
Capability requestA request for permission to perform an action class.
Capability grantThe scope and conditions checked by the authorization broker.
Coverage claimA worker's lease on an examination unit.

The standing loop claims eligible pending actions before ordinary research work. An operator decision, fulfilled dependency, and authorized action each retain their own records. Use the corresponding resolution and grant paths to make blocked work eligible again.

Read Ledger and wishlist for request examples, API paths, delivery behavior, and the two dashboard interfaces.

Persistent memory connects the cycles

REFALL maintains several records because each answers a different operational question.

RecordQuestion it answers
Subject context and scopeWhat are we examining, and under which boundaries?
Source inventory and coverageWhich exact files and units have been examined?
LedgerWhat evidence and decisions have accumulated?
BlackboardWhich compact shared notes should later tasks receive?
Module progressWhich passes have completed for each target?
WishlistWhich resources were requested or provided?
Pending actionsWhich obligations still need work or a decision?
Runtime observations and evidence graphWhich observations and supplied relationships were recorded?
Run manifest and checkpointsWhich providers, resources, inputs, and active work belong to this run?
Living report and analyst notesWhat does the accumulated research explain so far?

A manager restart can recover active conversations and module progress from stored state. Resume preflight compares recorded input hashes before it continues active work. Artifacts and archived conversations preserve details beyond a compact summary. The evidence graph exposes recorded nodes and supplied relationships. Call graphs and other code-analysis graphs remain separate tool outputs.

What the architecture combines

Coverage and investigation serve different purposes. Coverage records which units were examined. Research follows concrete leads across callers, parsers, boundaries, and observed behavior. The two records let an operator distinguish broad examination from deeper proof.

Work belongs to the subject. The loop builds each task from stored context, scope, shared notes, its target, and an output contract. A replacement worker can read earlier determinations and artifacts before it extends the work. Expensive decompilation or source analysis can become reusable input for later passes.

Evidence collection continues during execution. REFALL reads agent-authored event messages and structured files in the worker workspace. It captures facts, leads, findings, and flow relationships while the conversation remains active. At termination, it also preserves useful results from interrupted or malformed output. Successful completion requires a separate decision.

Negative conclusions also require evidence. A finding cannot disappear through an unsupported false-positive or duplicate decision. Typed receipts record why the transition was accepted or refused. Source hashes can bind that proof to the files that were examined.

Publication has its own state. A current report has an explicit workbench object identifier. Report replacement checks preserve populated technical sections. Closeout examines unresolved obligations, proof, finding decisions, expired claims, and report freshness.

Who uses it

AudienceTypical use
Security researchersTrace candidate vulnerabilities from source or binary anchors to review and runtime proof.
Reverse engineersBuild an accumulating map of components, interfaces, functions, and data flow.
Engineering reviewersCombine source examination, harness proposals, verification, and remediation work.
Research operatorsCoordinate long campaigns with bounded workers, reusable artifacts, and visible decisions.

For a firmware subject, the useful output can include a component inventory, function observations, validation requests, and a reproducible finding package. For a source repository, it can include per-file determinations, architecture facts, review results, and a prioritized research frontier. For a mapping task, facts, leads, and flow records can build a technical codex without vulnerability findings.

Ready Armor design approach

The Ready Armor approach separates agent capability from operational authority. It also treats the provenance of a claim as a separate control concern. REFALL applies these ideas to research coordination: bounded assignments, recorded authority, durable evidence, explicit decisions, and reportable completion.

BRA provides a governed assessment architecture. ARA provides an agent runtime and policy substrate. REFALL provides a research campaign loop and evidence workbench. Their public architectures describe related control principles; REFALL's code supplies its own scheduler, result contracts, and decision gates. The control stack starts with configured source resources and execution permissions. Source projection prepares the container location; inventory records content identity. The broker then checks presented action requests against applicable grants. Backend Start and Update refuse process launch when that authorization fails.

Above execution, module selection bounds assignments and conversation requests carry iteration limits. Stored context guides the worker's scope and tool choices. Continuous capture preserves observations, while finding gates and typed receipts check accepted conclusions. Closeout checks whether unresolved work and report state permit a closed campaign result.

Each layer controls a different transition. The broker is a decision service, not a universal worker tool interceptor. Read-only source permissions come from configured mounts rather than the source-copy fallback. Grant authentication requires the configured operator token; an unset token leaves identity fields caller supplied. These boundaries let operators combine campaign controls with the execution environment's actual protections.

Read Ready Armor in REFALL for the design rationale and its distinctive research lifecycle. Read Control in depth for the layers, activation conditions, refusal sequence, and operator decisions. Read Research architecture for the wider responsibility model.

Source and binary research

The main workbench flow runs the module scheduler through backend operations. It supports configurable subject, unit, and finding work. The repository also includes standalone source and binary research drivers. Those drivers use the compatibility coverage service and an investigation frontier.

Binary research starts from imported functions and their callers. Workers decompile selected functions, trace input, and enqueue concrete follow-up questions. Independent review challenges reachability and mitigations. Validation records reproduction attempts, observed effects, proof artifacts, and remaining blockers. Read Source and binary workflows for the execution families.

Start

  1. Complete Installation.
  2. Configure the backend, Agent Canvas, and provider commands.
  3. Create a subject workbench.
  4. Generate and inspect its context.
  5. Select the work modules and execution scope.
  6. Start the loop through Operating procedures.
bash
python3 -m ref_all_executor subject create example --source-path /absolute/path/to/example
python3 -m ref_all_executor ops command context example
python3 -m ref_all_executor ops command doctor example
python3 -m ref_all_executor ops command estimate example
python3 -m ref_all_executor loop start example

The backend checks launch authorization before it starts the loop. The operator interface also exposes context, scope, threat model, coverage, pending actions, capabilities, runtime evidence, and closeout state.

Read further

GuideContents
Ready Armor in REFALLWhy persistent state, bounded work, authority, evidence, and completion form one research system.
Feature catalogWhat each subsystem does and why it is useful.
Ledger and wishlistResearch history, requested resources, delivery receipts, APIs, and operator actions.
Research cyclesCampaign polling, per-target passes, finding work, retries, supervision, and closeout.
End-to-end workflowsOperator actions, scheduling, evidence, review, reports, and closeout.
Technical codexExecution paths, state ownership, and implementation details.
Research architectureDesign principles, Ready Armor relationships, and evidence boundaries.
Work modulesPacks, dependencies, subject configuration, and adaptation.
Architecture mapComponents, stores, and service relationships.
Documentation indexInstallation, configuration, operations, APIs, and runtime contracts.

The manager startup instructions are in starthere/ and docs/codex-startup/.

Interactive technical atlas

Open the self-contained technical atlas for the capability catalog, architecture diagrams, operating traces, searchable guides, and source excerpts. The complete site works offline as one HTML file. Read Documentation site for rebuilding and GitHub Pages publication.

License

Personal, noncommercial use by individuals is permitted under the Ready Armor Personal Use License. Every other use requires a separate written license from GainSec LLC. This includes employer, client, business, institutional, hosted-service, and compensated use. Contact licensing@gainsecmail.com to request a license.

Third-party and separately licensed components retain their own terms. The bundled ArcticBase component retains its MIT license.

FIELD GUIDE / 0211 MIN READ

Ready Armor in REFALL AB

REFALL applies the Ready Armor approach to sustained technical research. The campaign owns its state, evidence, and decisions. Worker conversations supply bounded examination and analysis.

The distinctive design connects work allocation, evidence capture, decision receipts, unresolved obligations, and the next research cycle. These mechanisms make earlier work usable after a conversation ends, a manager restarts, or another configured provider takes an assignment.

The research problem

A long investigation contains several kinds of unfinished work. Some code remains unexamined. Some observations need interpretation. Some findings need reproduction. Some actions need authority or an execution resource. A report can be readable while these obligations remain open.

REFALL gives each condition a persistent record and a corresponding decision path. The next worker receives an assignment built from that shared state. The operator can inspect progress without treating the latest conversation as the entire campaign.

This design is useful when a subject needs repeated source examination, binary analysis, harness work, independent review, and report revisions. It also supports architecture mapping, where observations and relationships can be useful without vulnerability findings.

Capability, authority, observation, and acceptance

Ready Armor separates useful capability from the authority to exercise it. It also separates an operation from the claim made about its result. REFALL expresses those distinctions through campaign records and checked transitions.

Control in Depth assigns each check to a defined operation or record transition. Persistent records keep the decision available to later workers and operators. Together, these principles connect immediate control with continuity across changes in the configured execution resources.

QuestionREFALL record or mechanismPractical effect
What can the worker contribute?Provider lane, task class, execution resources, and module objective.Assign reasoning, implementation, review, or reporting work deliberately.
Which work should run next?Claims, pending actions, module dependencies, and per-target progress.Coordinate fresh examination and later passes across workers.
Which presented action is authorized?Capability request, scoped grant, broker decision, and backend launch caller.Preserve the authority decision and enforce refusal on that launch path.
What was observed?Ledger entries, determinations, artifacts, and runtime observations.Retain material beyond the worker's final response.
What change is accepted?Finding admission, typed disposition receipts, and pending-action resolution.Check the record transition independently of the worker's narrative.
Is the campaign ready to close?Closeout checks over obligations, proof, claims, context, and report freshness.Make completion a visible decision about stored state.

The composition matters because each question can have a different answer. A permitted launch does not establish a validated finding. An examined unit does not establish that every behavior is safe. An idle coordinator does not establish campaign closeout.

Sources: scheduler, capability decisions, finding transitions, closeout.

Host deployment and concrete enforcement

REFALL's campaign controls can operate as host services. A virtual machine or container is not the security foundation for these backend decisions. The enforcing code runs at API admission, work allocation, process release, record acceptance, and closeout. These checks evaluate requests and stored state independently of the worker's prompt content.

BoundaryEnforcing mechanismResult
Assignment ownershipA database transaction selects and leases an open or expired coverage unit.Concurrent claim requests receive coordinated ownership.
Enabled backend loop launchStart and Update call the capability broker before process creation.Broker refusal returns HTTP 403 before the new process starts.
Candidate findingAdmission checks evidence, confidence, references, available inventory, and explicit external-target scope.An unacceptable candidate receives refusal reasons.
Finding dispositionTyped receipts and applicable stored-state checks govern the requested transition.Narrative alone cannot perform the transition.
Campaign closeoutStored obligations, receipts, proof state, claims, context, and report freshness feed the closeout gate.Blocking conditions prevent an ordinary closed result.

The capability operator token protects grant and revoke routes when configured. It gives the backend an authenticated operator identity for those mutations. Without that setting, identity fields remain caller supplied. Protecting that API access is part of the deployment's authority boundary.

The same design separates campaign control from direct execution permissions. The host supplies filesystem, network, and process permissions. Agent Canvas and other execution resources apply their configured environment controls. A specialist's direct shell command does not automatically pass through REFALL's authorization broker. Brokered callers enforce its decision before they release their own supported action.

This division makes host deployment useful without assigning every security responsibility to one sandbox or one model instruction. The backend controls the campaign transitions it owns. The execution host controls the resources available to the running tools. The operator can inspect the retained authority decision, evidence, refusal, and next obligation.

Sources: claim transaction, launch authorization, operator authentication, finding admission, typed dispositions, closeout gate.

1. The subject retains the research

A workbench groups context, source inventory, coverage, evidence, findings, obligations, and the current report pointer. These records have different lifecycles. Their separation lets the operator distinguish an observation, a hypothesis, an accepted conclusion, and a completion decision.

Source inventory identifies available content with paths, hashes, sizes, line counts, and file kinds. Coverage units turn part of that surface into claimable work. The ledger preserves appended evidence and decision records. Runtime storage retains events, observations, artifacts, manifests, and checkpoints.

A replacement manager can inspect those records before selecting further work. Later unit objectives explicitly ask workers to reuse prior summaries, findings, shared notes, and saved analysis. That guidance connects retained state with the next assignment. It does not depend on one model retaining an unlimited conversation history.

Sources: subject layout, inventory, ledger, reuse instructions.

2. Breadth and depth share one campaign

Coverage establishes an examination surface. Module passes can then deepen the understanding of individual targets. The scheduler stores completed passes separately for each subject, unit, or finding target. A prerequisite completed on one unit does not satisfy another unit's prerequisite.

The standing loop prioritizes worker-claimable pending actions and subject setup before fresh breadth. Fresh coverage then precedes ordinary finding modules and later unit passes. The built-in security path connects characterization, reconnaissance, and review. Finding modules can request reproduction or a patch proposal.

This combination supports broad examination without losing focused questions. Pending actions provide a route for urgent validation. Local dependencies let different components advance at different rates. A custom module can add a concrete research objective without replacing the entire coordinator.

Claims coordinate ownership with leases. The backend selects open or expired units within a database transaction. Completed module identifiers persist separately from the current conversation. These controls reduce repeated allocation while keeping expired or unfinished work available for later examination.

Sources: coverage claims, module declarations, per-target progress.

3. Useful output survives an incomplete invocation

A worker can produce useful analysis before it produces a final response. REFALL reads available conversation events and workspace results during execution. It can retain facts, leads, findings, flows, and saved analysis artifacts while the conversation remains active.

Terminal processing first classifies the invocation, parses its output, and checks submission identity. It then preserves available observations before handling the classified outcome or scheduling a replacement. A duplicate applied submission returns before terminal salvage.

Successful assignment completion has separate conditions. Characterization coverage submission requires its own completion branch. A module task without a unit record must produce durable canonical observations before its progress can advance. The campaign can therefore retain partial research while still showing an unfinished assignment.

This separation is useful when a provider stops, output is truncated, or a worker saves files without a complete final narrative. It preserves the available material for repair and review. It also keeps an execution failure from being reported as completed examination merely because some evidence was captured.

Sources: incremental ingestion, terminal order, empty-output check.

4. Conclusions have their own acceptance path

The finding store checks proposed evidence, confidence, source references, and explicit external-target scope. Inventory membership is checked when inventory is available. A canonical finding can retain related evidence locations without representing each location as a different root cause.

Later finding decisions use typed receipts. False-positive, duplicate, downgrade, deferred-chain, and live-validation decisions have distinct requirements. Unresolved contradictory obligations can refuse a suppressive change. A receipt that supplies source hashes also activates applicable source-root and current-content checks.

These decisions preserve negative conclusions as well as positive findings. A later reviewer can inspect why a claim was rejected, deferred, reduced, or accepted. A supported research primitive can remain visible while its wider exploitability still needs examination.

The gates check specified record, reference, and receipt conditions. The operator reviews the interpretation and the meaning of the proof. A well-written explanation alone does not perform the corresponding finding transition.

Sources: candidate admission, canonical finding storage, disposition receipts.

5. Missing resources and missing authority remain distinct

A wishlist records a requested tool, environment, access requirement, or other research resource. The watchdog can recognize host availability and process supported tool requests through its allowlisted installation path. Delivery appends a provided record and a linked ledger receipt. Supported head and research-driver prompts can include delivered resources in later objectives.

A pending action records work or a decision that remains unfinished. A capability grant records authority for a presented action class and scope. Providing a tool does not grant permission to use it against every target. Recording a capability request does not resolve the related obligation automatically.

Backend Start and Update provide a concrete release boundary. They consult the broker before an enabled loop process launch. A refused decision gives HTTP 403 before that process starts. Grant and revoke authentication requires the configured operator token file. Without that token setting, the API retains caller-supplied identities.

REFALL supplies those campaign and broker controls. The chosen execution environment supplies filesystem permissions, network boundaries, and any direct tool interception. The generic authorization endpoint records a decision without executing an arbitrary command. Callers need an integration that obeys the decision before action release.

Sources: wishlist lifecycle, backend launch, operator authentication.

6. Reports and closeout compile the campaign state

Reports read stored evidence, findings, coverage, runtime material, capabilities, and unresolved work. A current report has an explicit workbench object identifier. The report assembly path can refuse a structural replacement that loses populated technical sections. This preserves accumulated explanation while the underlying research continues.

Closeout examines the campaign through a separate gate. Open obligations, missing required receipts, unproved finding states, expired claims, missing context, and stale reports can block the closed result. Coverage below the default 95-percent threshold is advisory. Draft and current report objects can exist before closeout succeeds.

The distinction makes a useful report and a completed campaign separately visible. The operator can resolve the blocking work or record an explicit exception for identified checks. The exception remains linked to its decision record.

Sources: report inputs, replacement protection, closeout checks.

A continuation example

Consider a source campaign that has characterized a parser and recorded a concrete reproduction lead. The current worker stops after saving an analysis artifact.

  1. The loop preserves available observations and records the failed invocation.
  2. A recoverable failure can receive a bounded replacement attempt.
  3. An unresolved proof or authority question remains a pending action.
  4. The manager reads the stored evidence and selects the next eligible work.
  5. The replacement worker receives the subject context, target, objective, and prior-work instructions.
  6. It extends the evidence and proposes any necessary finding transition.
  7. The backend evaluates the transition receipt independently of the new narrative.
  8. Reports reuse the accumulated state, and closeout checks the remaining obligations.

Changing a configured worker provider does not replace the subject's evidence identifiers, finding history, or module progress. A resumed coordinator can recover saved active conversations and progress. Supported resume checks compare recorded inputs before continuing active work. The detailed continuation behavior and its activation conditions are in Control in depth.

The distinctive contribution

REFALL connects retained technical knowledge with the machinery that selects the next research task. Coverage, claims, evidence, decisions, and obligations participate in the same continuing campaign. This gives the operator a practical explanation of what was examined, what supports a conclusion, and what should happen next.

ARA, BRA, and REFALL apply related Ready Armor principles through separate implementations. ARA addresses agent runtime and capability composition. BRA addresses governed assessment operations and materialization. REFALL addresses persistent research coordination, evidence acceptance, and campaign completion.

Its contribution is that connected research lifecycle. The model supplies analysis, tools supply observations, and the campaign retains the records needed to continue and review the work.

Read Research cycles for scheduling and stopping behavior. Read Ledger and wishlist for the persistent resource and evidence paths. Read Control in depth for each actual decision boundary.

FIELD GUIDE / 0313 MIN READ

REFALL AB research cycles

REFALL coordinates a campaign through repeated scheduling cycles and bounded worker conversations. The campaign owns the context, evidence, progress, findings, and unresolved decisions. A worker owns one assignment for one conversation.

These loops have different purposes. A scheduling cycle examines available work and running conversations. A module pass examines a particular subject, unit, or finding. A retry attempts to repair a failed invocation. Report and supervision cycles examine the accumulated campaign state.

Campaign, cycle, and conversation

TermMeaning
CampaignPersistent research state for a subject workbench.
Standing loopThe coordinator that selects, dispatches, polls, and records work.
Scheduling cycleOne complete pass through the coordinator's main loop.
Worker conversationA bounded request sent to Agent Canvas/OpenHands.
Module passA declared research activity on one target.
RetryA replacement conversation for the same assignment after a recoverable failure.
Pending actionA stored obligation with a status, owner, and applicable receipt requirements.
CloseoutA separate decision about whether campaign records support completion.

The default coordinator has no fixed cycle limit: max_cycles=0. It still returns when its idle conditions are met. It does not wait indefinitely for an operator to add future work.

A finite cycle limit bounds coordinator passes. It does not cancel every conversation when the limit is reached. The coordinator can return running with active conversations saved for continuation. The backend Update operation requests one coordinator cycle. Its Start operation requests an unrestricted cycle count.

Source: standing-loop execution, backend Start and Update.

One scheduling cycle

Startup loads saved active conversations, the iteration count, and campaign progress. The coordinator loads source identity and performs applicable resume checks. A returned preflight refusal for saved active work produces resume_blocked before new scheduling. When coverage is empty, source enumeration can establish the initial units. The loop then records its running manifest.

Each cycle performs this sequence:

  1. Read coverage and gate state.
  2. Fill available worker slots according to scheduling priority.
  3. Poll every active conversation, including newly dispatched workers.
  4. Process terminal outcomes or ingest available output from running workers.
  5. Update determinations and applicable reports.
  6. Consider report and visualization workers.
  7. Save active conversations, cycle state, and progress.
  8. Evaluate the cycle limit, stall condition, and idle conditions.
  9. Wait for the configured poll interval when another cycle is needed.

Global parallelism defaults to four conversations. The poll interval defaults to five seconds. These are coordinator settings rather than a guarantee about every tool process that a worker can create.

STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 2STAGE 2STAGE 2NoNoNoNoNoYesYesYesYesYesYesYesYesYesYesYesYesYesNoNoNoNoNoNoNoNo01Load campaign and activeconversations02estore03store04tore05ore06re07e08ill09ll10l11oll12tate13ate14te15inish16nish17ish18sh19h20ngest21gest22est23st24t25andle26ndle27dle28le29efresh30fresh31resh32esh33ave34ve35ontinue36ntinue37tinue38inue39nue40ue41Select work and fillslots42Poll activeconversations43Terminal outcome?44Ingest available eventsand files45Classify, parse, checkidentity, preserve46Handle completion,retry, or obligation47Update determinationsand reports48Save active work andprogress49Another coordinator cycle?50Return idle or running state
Control and data flow

A terminal conversation increments the loop's completed counter. That counter includes failed conversations. Use coverage, module progress, finding state, and pending actions to assess research completion.

Source: cycle body and return conditions.

What receives the next available slot

The scheduler uses this priority order:

PriorityWorkSelection rule
1Worker-claimable pending actionsClaim eligible open actions or expired leases.
2Requested threat-model workRequire the necessary context and inventory inputs.
3Subject modulesSelect eligible identification, research, toolchain, or synthesis work.
4Fresh coverageClaim an open or expired coverage unit for characterization.
5Finding modulesSelect verification or patch work for registered finding targets.
6Later unit modulesDeepen units already recorded as characterized.

Priority applies whenever the coordinator selects another task. A worker already running a lower-priority assignment is not interrupted merely because higher-priority work appears.

Subject modules can run before fresh coverage. Their own local priority determines which eligible subject module is selected. A subject synthesis module therefore does not automatically wait for all unit review work. Declare the intended prerequisites rather than treating pack names as global phases.

Fresh coverage precedes finding modules and later unit passes. This spreads initial examination across available source units before repeated depth on an early subset. Pending actions still provide a route to urgent validation before ordinary breadth.

Source: dispatch priority, module selection.

Pending work is a queue with ownership

Pending selection accepts open actions and claimed actions whose leases have expired. It orders candidates by priority, action kind, and recorded time. At equal priority, live validation precedes an ordinary operator-decision action.

Some obligations remain manager work. The worker selector excludes watchdog-stop reviews. It also excludes specified finding-promotion reviews that require manager examination or identify an inventory mismatch. The operator can still see these obligations even when workers have nothing eligible to claim.

A pending-action worker receives the obligation and its required result. Successful processing attempts to resolve it through the pending-action API. Required receipt types and evidence still apply at that API boundary. An unresolved action can remain a closeout blocker after the coordinator becomes idle.

Coverage claims have a separate lease. The backend selects open units or expired claims and updates ownership within a database transaction. The default coverage lease is 900 seconds. An expired claim makes the unit eligible for another claim; it is not successful completion.

Sources: pending selection, worker exclusions, coverage claims.

Depth develops per target

Module dependencies apply to the same target. A completed pass on parser.py does not satisfy a dependency for writer.py. The scheduler stores completed module identifiers separately for each target.

With no explicit module configuration, the registry selects security_assessment. Its dependency resolution also loads baseline characterization. The built-in source sequence is:

STAGE 1STAGE 2STAGE 3STAGE 4STAGE 5STAGE 601characterize: describe the unit02sec_recon: identify attack surface03sec_review: examine candidate weaknesses04Proposed findings and evidence05Canonical finding admission and review06Register output that carries a finding ID07sec_verify: reproduction attempt
Control and data flow

Characterization records the unit in scheduler progress and unlocks dependent passes. Coverage submission has its own acceptance branch. Module progress and recorded coverage therefore answer different questions.

Other packs add different paths:

PackBuilt-in target path
reverse_engineeringSubject identification; unit characterization → decomposition → review.
security_assessmentCharacterization → reconnaissance → review; finding verification.
dynamic_testingCharacterization → harness drafting → runtime verification.
developmentCharacterization → harness drafting; finding patch proposal.
static_analysisSubject toolchain work; characterization → unit review.
researchSubject research and subject synthesis, selected by their local priorities.

Eligible later passes apply completed-work checks, dependencies, active-target checks, applicable path filters, and module concurrency. Fresh characterization uses direct coverage claims instead of that eligibility function. It attaches the first active breadth module and uses the fixed characterize task class. Later unit selection supplies the recorded path but does not supply unit_state. Read Work modules for the configuration fields and their actual selection branches.

Sources: built-in packs, default registry, per-target progress, eligibility.

Findings create another research frontier

Finding modules operate on registered finding identifiers. The scheduler retains a compact identifier, path, and title for each such target. Completion processing registers finding outputs that contain identifiers. A finding also has canonical evidence, validation, and disposition records outside this scheduling index. Registration in that compact index does not establish that the canonical finding admission gate accepted the candidate.

sec_verify asks a worker to attempt reproduction. Its requested result includes exact steps, expected behavior, observed behavior, and proof status. dev_patch asks for a minimal, verifiable proposed change. Those objectives define assignments; they do not prove reproduction or apply a patch automatically.

When both built-in finding modules are eligible, verification has priority 20 and patch work has priority 70. The built-ins do not declare a patch-to-verification dependency. A patch proposal therefore does not automatically reopen the completed verification module. Use an explicit pending action or a distinct configured verification pass for another examination.

A proposed finding status or quality change passes through typed transitions. Live validation, false-positive decisions, duplicates, downgrades, and deferred chains require their respective receipts. A refused transition preserves the existing finding state and records the refusal. Research can continue while the missing proof or decision remains visible.

Sources: finding registration and selection, finding assignment, transition application, disposition gate.

Evidence arrives before an assignment is complete

Running conversations can contribute facts, leads, findings, flows, and saved artifacts. The loop reads conversation events and workspace submission files during polling. Stable identities prevent repeated ingestion from producing the same stored observation repeatedly. A failed backend write remains eligible for a later persistence attempt.

Terminal handling uses a specific order:

  1. Classify the invocation status and terminal submission contract.
  2. Parse the output and assignment metadata.
  3. Check whether the structured submission was already applied.
  4. Preserve available research observations.
  5. Handle the classified completion, failure, retry, or operator obligation.

A duplicate applied submission returns before terminal salvage. For other terminal results, salvage precedes failure handling and retry dispatch. A failed conversation can therefore retain useful research without receiving successful assignment completion.

The successful branch applies proposed transitions and persists control requests. Coverage submission additionally requires the exact finished status, a characterization task, and an eligible parsed coverage result. Later modules do not resubmit breadth coverage.

The empty-output guard applies to module tasks that carry no unit record. It requires canonical facts, leads, findings, or flows and successful persistence before module completion. The current subject and finding dispatch shapes use that guard. A completion message alone does not satisfy it.

Sources: incremental ingestion, terminal processing, empty-output guard.

Repetition, retry, and exhaustion

Built-in modules normally use once_per_target=true. A completed module is then excluded for that target. The active-target check also avoids another instance of that same once-per-target pass while it runs.

Setting once_per_target=false removes those particular exclusions in eligible module selection. Such subject, finding, or later unit passes can become eligible repeatedly. Dependencies, applicable filters, and concurrency settings still affect selection. This is an explicit repeated-work configuration rather than an automatic new research hypothesis.

Invocation retry is a separate mechanism. The classifier defaults to two total attempts for recoverable worker failures. Strict terminal-contract refusal requires REFALL_REQUIRE_STRUCTURED_SUBMIT=true; that setting defaults to false. Claude lane A can fail over to Codex lane B; lane B remains on B. The replacement retains the assignment, module target, and accumulated campaign evidence.

Recognized capability, scope, operator-stop, and tool-execution failures do not use ordinary automatic retry. Capability failures create an access request and decision obligation. Scope failures create an obligation requiring a scope decision receipt. Other exhausted or nonretryable outcomes produce manager-visible review work.

A failed module does not receive successful progress merely because its retry budget ended. It can remain eligible for later scheduling. An expired coverage lease can also expose the original unit again. The invocation retry limit therefore bounds that replacement chain, not every future campaign assignment for the same target.

Sources: eligibility and repetition, failure classification, replacement dispatch, decision obligations.

Reports and visuals have their own cadence

The default draft refresh interval is ten coordinator cycles. Ordinary assessment refresh requires at least one finding ledger entry. Mapping-mode refresh can instead use recorded facts, leads, flows, or artifacts.

A report-writing conversation uses a different trigger. By default, it needs an available slot, the source-coverage completion predicate, and 30 new facts, leads, or findings. That predicate compares completed and total file counts when both are available. If those counts are absent, it returns true. A nonpositive report threshold disables the default report-worker trigger.

The optional continuous report policy reserves one slot and replaces completed report workers without the default volume trigger. This policy can keep reporting activity available after ordinary analysis work is exhausted.

Visualization is considered every 20 cycles. It requires the source-coverage predicate, an available slot, and the configured resolved-coverage threshold. The default threshold is 25 percent. Ordinary assessment also requires three finding entries; mapping mode uses its separate objective without that finding-count condition.

Worker requestDefault iteration limit
Characterization, subject modules, finding modules, and later unit passes30
Report worker25
Visualization worker20
Replacement retry30

These limits are sent to Agent Canvas with the conversation request. They do not establish campaign closeout or a host-wide resource ceiling.

Sources: draft cadence, report-worker trigger, visualization trigger, conversation request.

Idle, stalled, supervised, and closed

An idle coordinator has no active conversations under its return conditions. The loop also examines remaining module work and worker-claimable pending actions before an idle exit after dispatch activity. Idle does not mean that every obligation is resolved. Manager-only actions, refused transitions, or missing report proof can still block closeout.

The stall check measures cycles since the most recent terminal conversation. Its default threshold is 36 cycles, with a strict greater-than comparison. A terminal failure updates that completion clock too. The threshold is therefore not a measure of scientific progress or successful proof. A stall return can retain active conversations and report running.

The watchdog supplies another observation and response cycle. It samples services, active conversations, progress signatures, reported usage, and failure signals. With enforcement active, selected conditions can delete a managed conversation and create stop-review records. Its sampled thresholds are separate from per-conversation iteration limits.

The watchdog also examines resource requests in the wishlist. It can recognize an already installed tool or use a configured allowlisted installer when installation is enabled. Unsupported or failed requests remain reported as unhandled work. A wishlist record does not itself grant live execution authority.

The supervisor has fast liveness and full watchdog/status modes. Its default cron entries request a fast pass each minute and a full pass every three minutes. Service recovery can restart the configured backend or Agent Canvas. An absent loop with otherwise healthy services is reported as idle; that branch does not automatically launch a new research loop. Repeated service recoveries can trigger a hold and stop the coordinator process until an operator clears the fault.

Closeout remains a separate operation. It checks pending actions, finding proof, transition receipts, expired claims, stored context, and report freshness. Coverage below the default 95-percent threshold is advisory rather than blocking. The manager can resolve the underlying work or record an explicit, receipted exception for identified checks. Read Control in depth for the gate and authority boundaries.

Sources: return and stall conditions, watchdog pass, wishlist handling, supervisor tick, closeout gate.

STAGE 2STAGE 2STAGE 2STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1Recoverable failureRecoverable failureRecoverable failureRecoverable failureRecoverable failureRecoverable failureRecoverable failureRecoverable failureMissing proof or authorityMissing proof or authorityMissing proof or authorityMissing proof or authorityMissing proof or authorityMissing proof or authorityMissing proof or authorityMissing proof or authorityAccepted module completionAccepted module completionAccepted module completionAccepted module completionAccepted module completionAccepted module completionAccepted module completionAccepted module completionBlocking recordsBlocking recordsBlocking recordsBlocking recordsAccepted campaign stateAccepted campaign stateAccepted campaign stateAccepted campaign state01Research assignment02Stored observations andartifacts03Result and requireddecision04Bounded replacementinvocation05Persistent pendingaction06Worker or manager review07Per-target progress08Next eligible module ortarget09Separate closeout evaluation10Closed lifecycle result11ork12rk13k14bserve15serve16erve17rve18ve19e20ecision21cision22ision23sion24ion25on26n27etry28try29ry30y31ending32nding33ding34ing35ng36g37eview38view39iew40ew41w42rogress43ogress44gress45ress46ess47ss48s49ext50xt51t52ate53te
Control and data flow

Examine a campaign's next step

  1. Read the stored context, scope, and source inventory.
  2. Inspect active conversations and the last recorded cycle.
  3. Examine worker-claimable obligations before ordinary coverage work.
  4. Check remaining fresh units and expired claims.
  5. Compare each target's completed modules with its dependencies.
  6. Inspect finding verification, patch proposals, and required receipts separately.
  7. Review failed invocations and distinguish their retry chain from later rescheduling.
  8. Inspect the ledger, artifacts, determinations, and report freshness.
  9. Evaluate closeout independently of the coordinator's idle or running status.

Use End-to-end workflows for the operating sequence. Use Technical codex for the broader implementation path.

FIELD GUIDE / 0412 MIN READ

Evidence ledger and resource wishlist

REFALL AB keeps research results and resource needs with the subject workbench. The evidence ledger preserves findings, facts, leads, decisions, and operational history. The wishlist records missing tools, environments, dependencies, access, and data. Together, these records help the next worker continue useful work with the resources already available.

A ledger entry records what happened or what the research supports. A wishlist item records what a researcher needs. A pending action records an obligation for a worker or manager. A capability request asks for permission to perform a scoped action. These records have separate APIs and lifecycles.

Select the correct record

NeedRecordResult
Preserve a source fact or a possible research path.Ledger fact or lead.Later workers and reports can reuse the evidence.
Preserve a proposed vulnerability.Ledger finding and the coverage finding API.The evidence history and finding lifecycle remain available.
Request a missing executable or analysis environment.Wishlist item.The watchdog or operator can examine the need.
Track work that must receive a decision or receipt.Pending action.Eligible actions enter the standing loop before ordinary coverage work.
Request a live action outside current permissions.Capability request.An operator can grant the necessary scoped capability.
Record a grant, refusal, transition, or exception.Corresponding decision API.The API updates its own state and appends a ledger record.

The ledger accepts rich JSON records. Its append route validates the entry kind, rather than the complete evidence contract. Use the finding, disposition, capability, and pending-action APIs for their state changes. Appending a ledger entry alone does not perform those changes. See evidence procedures and Control in Depth.

Source: ledger append, pending action creation, and finding persistence.

Evidence ledger

Storage and entry fields

Each workbench has a refall_ledger.jsonl file beneath its storage directory. Each nonempty line contains one JSON entry. The append API writes a new line and records a storage audit event.

FieldBehavior
kindDefaults to fact. The value must be a supported kind.
idDefaults to a new identifier with the led_ prefix.
workbench_slugDefaults to the slug from the route.
created_atDefaults to the current UTC timestamp.
updated_atReceives the current UTC timestamp on append.
Other fieldsRemain in the JSON entry.

Useful evidence fields include title, summary, evidence, confidence, limitations, source_anchors, and artifact_refs. Use lane, severity, subject, agent, and unit_id when they identify the work. Preserve exact source references and artifact identifiers. Record secret presence without the secret value.

The supported kinds are:

  • fact, lead, finding, observation, and hypothesis.
  • remediation, attack-chain, ask, coverage, validation, and artifact.
  • ops-event, disposition-transition, and pending-action.
  • capability-grant, capability-request, and capability-decision.
  • runtime-event and closeout-gate.

Source: entry kinds and storage.

Append, correction, and order

The ledger API exposes append, list, lookup, and statistics operations. It has no entry update or delete route. To correct a result, append a new entry with a new identifier. Include the earlier identifier and explain the correction. Use the relevant state API when the correction changes a finding or decision.

List responses preserve file order. The implementation generates no numeric sequence, previous-entry hash, or ledger hash chain. created_at can come from the submitted record. Use the entry identifier to cite evidence, rather than treating its timestamp as a trusted sequence number.

A repeated caller-supplied identifier can produce multiple lines. Lookup by identifier returns the first matching entry. A repeated append is therefore not an update operation.

Source hashes serve a different purpose. Receipts can carry source_hashes, and the run manifest records source and configuration hashes. Those hashes identify input bytes; they do not authenticate the JSONL ledger.

Source: append and lookup, evidence receipts, and manifest hashes.

Other ledger files

Subject bootstrap also creates .refall/evidence_ledger.json with an empty entries array. The workbench ledger API uses refall_ledger.jsonl instead. Runtime events and checkpoints have their own storage and APIs. Use the workbench ledger routes for shared evidence and decisions.

Source: subject bootstrap and ledger path.

Read before work

  1. Read the stored subject context and scope.
  2. Read the coverage summary for the assigned unit.
  3. Read prior facts, leads, and findings for that unit.
  4. Examine referenced artifacts and saved analysis.
  5. Extend the evidence and record the new result.

The standing loop puts these reuse instructions in objectives for later unit modules. They tell workers to use prior coverage, ledger records, and the shared blackboard. This is worker guidance; the ledger API does not enforce a mandatory read before a write.

The loop also ingests evidence during active conversations and at completion. It reads tagged output and saved workspace results. Its in-memory identity set suppresses repeated facts, leads, findings, and flows during that run. A backend failure removes staged identities so a later attempt can retry persistence. This process does not add a general deduplication rule to the ledger API.

Flow evidence uses kind: "fact" with fact_kind: "flow" and a nested flow object. The loop separately submits findings to the coverage finding store. Finding promotion can refuse a candidate even when its ledger evidence remains available.

Source: reuse instructions, incremental ingestion, ingestion identities, and entry conversion.

Read and append through the API

Set REFALL_PUBLIC_BASE_URL to the backend address. Set REFALL_WORKBENCH to an existing workbench slug. The examples use a fictional source file and artifact identifier.

bash
export REFALL_PUBLIC_BASE_URL="http://127.0.0.1:7673"
export REFALL_WORKBENCH="example"

curl -fsS "$REFALL_PUBLIC_BASE_URL/api/workbenches/$REFALL_WORKBENCH/ledger/stats"

curl -fsS --get \
  "$REFALL_PUBLIC_BASE_URL/api/workbenches/$REFALL_WORKBENCH/ledger/entries" \
  --data-urlencode 'kind=fact' \
  --data-urlencode 'lane=source-review'

curl -fsS -X POST \
  "$REFALL_PUBLIC_BASE_URL/api/workbenches/$REFALL_WORKBENCH/ledger/entries" \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "kind": "fact",
    "subject": "example",
    "lane": "source-review",
    "unit_id": "unit-example",
    "title": "Configuration reaches the parser",
    "summary": "The loader passes the configuration to parse_config.",
    "evidence": "The call passes the loaded byte buffer.",
    "confidence": "high",
    "limitations": ["Runtime behavior was not examined."],
    "source_anchors": ["src/config.py:42"],
    "artifact_refs": ["artifact-example"]
  }'

The append response contains the entry identifier. Use it with GET /api/workbenches/{slug}/ledger/entries/{entry_id}.

RouteResult
POST /api/workbenches/{slug}/ledger/entriesAppends an entry; returns HTTP 201.
GET /api/workbenches/{slug}/ledger/entriesLists entries in file order.
GET /api/workbenches/{slug}/ledger/entries/{entry_id}Returns the first matching entry, or HTTP 404.
GET /api/workbenches/{slug}/ledger/statsReturns totals by kind, lane, and severity.

List filters are kind, lane, severity, and since. The first three use exact matches. since compares the stored created_at strings with the supplied value. Use consistent UTC timestamp formats for time queries. An unsupported kind returns HTTP 422.

Source: ledger routes and list filters.

Interface and report use

Open the workbench Ops panel and select Ledger. The panel shows the latest 80 appended entries in reverse file order. Each row shows its title, kind, timestamp, status, provider or lane, and summary when available. Select Output to examine facts, leads, and findings with source or artifact references. Use the API for the complete ledger.

Report drafting reads ledger entries, ledger statistics, coverage, runtime evidence, pending actions, and capability state. Report enrichment also reads the stored ledger. The report combines recorded evidence; a narrative alone does not perform finding verification.

Source: Ledger view, Output filters, report drafting, and report enrichment.

Resource wishlist

Workbench records

The current coverage client defaults to workbench coverage. Its wishlist records reside in the workbench refall_coverage.db database.

FieldPurpose
idRequest identifier. The backend generates a wish_ identifier when absent.
subjectSubject name. The route slug supplies the default.
kindResource category, such as tool, env, dep, access, or data.
itemNecessary resource. This field must be nonempty.
reasonWork that the resource will support.
requested_byResearcher or operator identifier.
statusDefaults to open; provided identifies delivery records.
created_at, updated_atDatabase timestamps.

kind and status are string fields rather than fixed enums. The automation specifically interprets open and provided. A custom status does not introduce an automatic workflow.

The workbench API creates rows and lists them. It exposes no wishlist update or resolve route. A delivery record is a new row with status: "provided". Keep the original request identifier in its reason or in a related ledger entry.

Source: wishlist request fields, database operations, and client mode.

Record and examine a request

  1. Check the provided list before requesting a resource.
  2. Name one resource and the work it will support.
  3. Create an open wishlist item.
  4. Examine watchdog results or arrange operator provision.
  5. Record delivery and verify availability in the worker environment.
  6. Obtain any necessary capability grant before live use.
bash
curl -fsS --get \
  "$REFALL_PUBLIC_BASE_URL/api/workbenches/$REFALL_WORKBENCH/coverage/wishlist" \
  --data-urlencode 'status=provided'

curl -fsS -X POST \
  "$REFALL_PUBLIC_BASE_URL/api/workbenches/$REFALL_WORKBENCH/coverage/wishlist" \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "kind": "tool",
    "item": "exiftool",
    "reason": "Read metadata from the assigned sample files.",
    "requested_by": "research-worker",
    "status": "open"
  }'

curl -fsS --get \
  "$REFALL_PUBLIC_BASE_URL/api/workbenches/$REFALL_WORKBENCH/coverage/wishlist" \
  --data-urlencode 'status=open'

Both list requests can also use subject. The backend lists matching rows by updated_at, newest first. There is no separate automatic approval state in this wishlist API.

For manual provision, create another row through the same POST route. Set status to provided and identify the original request in reason. For example, a delivered test environment can use this body:

json
{
  "kind": "env",
  "item": "isolated parser environment",
  "reason": "Provided for request wish-example; use the assigned worker environment.",
  "requested_by": "operator",
  "status": "provided"
}

This row records delivery. Configure the worker environment separately before its next task.

Source: wishlist routes.

Watchdog provision

When the watchdog runs, it collects the workbench's open and provided wishlist rows. It normalizes the first token of each resource name to an executable name. It skips names already represented in the provided list.

For an eligible request, it first checks shutil.which on the watchdog host. If the executable exists, the watchdog records delivery. Otherwise, an enabled installation path can use the fixed Homebrew commands below.

Requested executableHomebrew package
binwalkbinwalk
exiftoolexiftool
radare2 or r2radare2
rizin or rz-binrizin
yarayara

The eligible categories are tool, cli, binary, package, dependency, or an empty kind. The installation setting defaults to enabled. Without an explicit watchdog configuration, REFALL_WATCHDOG_INSTALL_WISHLIST controls that setting. REFALL_WATCHDOG_WISHLIST_INSTALL_TIMEOUT_S defaults to 900 seconds. An explicit WatchdogConfig supplies its own installation values.

A successful command must also make the executable discoverable through shutil.which. The watchdog then creates a new provided row. It appends an ops-event with action watchdog.wishlist.provided. The event links the request and delivery identifiers and includes an availability or installation receipt. The original open row remains unchanged. Thus, an open count can include a request that already has a separate delivery record.

Dry-run mode skips installation and returns proposed delivery records without writing them. Unsupported categories, disabled installation, missing installers, and failed installation appear in the result's unhandled list. A host executable check establishes availability on the watchdog host. Check its path or mount separately before use inside a worker container or remote environment.

Source: installer table, configuration, provision decisions, and delivery receipt.

Delivered resources reach agents

The head orchestrator reads provided items during each cycle. It inserts them into the specialist objective under DELIVERED WISHLIST ITEMS. The research, binary research, binary review, and binary validation drivers also insert this information into their prompts. They tell workers to check delivered resources before requesting duplicates. A failed provided-list read returns an empty list in the shared helper.

The standing loop uses a separate scheduler. It does not directly turn wishlist rows into tasks or insert the shared delivered-item block. Its pending-action path selects claimable obligations before ordinary research modules. Some review actions remain manager work and are excluded from worker dispatch.

Source: head cycle, prompt formatter, research driver, binary drivers, scheduler order, and manager-only exclusions.

Compatibility dashboard

The separate coverage service retains its own wishlist API and SQLite store. Its subject dashboard shows wishlist items and a provided button for open items. That button calls POST /wishlist_resolve and updates the existing row. This differs from the workbench API's separate delivery row.

Compatibility routeBehavior
POST /wishlistCreates an open item with subject, item, optional kind, reason, and requested_by.
GET /wishlist?subject=example&status=providedLists matching items; the store returns at most 500.
POST /wishlist_resolveUpdates an existing row from a body with id and status.

Use these routes only with the standalone coverage service. They operate on that service's database, rather than automatically mirroring workbench rows. The native workbench Ops panel provides ledger, pending-action, and capability controls. It does not provide this compatibility wishlist button.

Source: compatibility routes, update behavior, and dashboard action.

From a resource need to completed work

Consider a worker that needs metadata extraction and later needs a live network check.

  1. The worker reads earlier evidence and the provided wishlist.
  2. It records an open exiftool request with the research reason.
  3. The watchdog finds the host executable or uses its configured installation path.
  4. The watchdog writes a delivery row and a ledger receipt.
  5. A supported head or driver prompt includes the delivered tool on its next dispatch.
  6. The worker checks tool availability in its execution environment and records source or artifact evidence.
  7. The worker requests a scoped capability before the live network action.
  8. The operator examines the capability request and supplies an appropriate grant.
  9. The supported execution path requests authorization for the intended action.
  10. The worker submits the result and resolves related pending work with the necessary receipt.

A provided wishlist item records resource availability, rather than permission to use that resource against any target. A capability grant carries the class, scope, expiry, and execution conditions used by the authorization decision. Operator-token authentication applies to capability mutations when the deployment configures that credential. The decision API records its result; it does not execute an arbitrary submitted command. See execution boundaries.

A direct capability_missing failure can create a capability request and an operator-decision pending action. A direct scope_blocked failure can create a pending action with a scope_decision receipt type. Neither path creates a wishlist item automatically. The operator can track a related resource need in the wishlist and keep the decision in its own queue. A capability grant creates a separate grant record; it does not update the request row or a wishlist item.

Source: failure routing, capability API, and pending-action receipt routes.

FIELD GUIDE / 058 MIN READ

REFALL AB feature catalog

REFALL organizes research into persistent subjects, bounded execution, evidence, decisions, and deliverables. These features work together around one subject workbench.

Subject workbench

Each subject has metadata, a source path, a storage path, context, and a theme. Coverage, ledger entries, pending actions, capabilities, and reports belong to the same workbench. The operator can inspect the research without reconstructing it from terminal history.

The Ops panel presents Context, scope, threat model, live agent state, coverage, runtime records, and closeout information. It can generate or save context, request a threat model, run diagnosis, estimate work, and issue lifecycle commands. Pending actions can be resolved, deferred, or cancelled with receipts. Capability requests and grants remain visible beside the research.

Source: WorkbenchOpsPanel.

Research ledger and Output projection

The subject ledger records facts, leads, findings, validation, artifacts, decisions, and operational events. Its API appends JSONL records and supplies filtered lists, individual entries, and summary statistics. The Ops Ledger view exposes the history. The Output view filters research records and expands their evidence, next steps, and worker identifiers. Reports read the ledger together with coverage and runtime state.

The ledger preserves evidence history. Finding, capability, and pending-action APIs perform their corresponding state transitions. Read Ledger and wishlist for exact entry fields and route examples.

Resource requests and delivery

The wishlist records missing tools, dependencies, access, and other research needs. The watchdog reads open and provided records. For supported executable requests, it checks host availability and can use its configured fixed installer list. Delivery appends a provided row and a linked ledger receipt. The original request stays in history.

Head cycles read the delivered resources before preparing objectives. The research driver and three binary drivers include a provided-item snapshot loaded at startup. Their prompts tell workers to use delivered items before requesting duplicates. The standing loop selects pending actions, coverage units, and work modules through separate queues. Read the delivery lifecycle.

Connected research cycles

Campaign polling, module passes, worker retries, reporting, and supervision have separate triggers and state. A coordinator cycle can ingest observations from a still-active worker. A terminal result preserves useful evidence before handling the classified outcome or scheduling retry. Per-target dependencies permit different components to advance at different rates. Read Research cycles for the priorities, limits, and feedback paths.

Shared context and readiness

Context generation collects subject metadata, source material, workbench text, and operator direction. The selected provider produces a Markdown briefing. REFALL stores the briefing in a workbench object, subject file, and coverage context record. The backend refuses a start request without stored context.

Doctor examines services, provider commands, paths, context, capabilities, secrets presence, and relevant tooling. Estimate combines inventory size, remaining coverage, pending actions, and readiness. Its task count helps the operator plan the run.

Source: Context and readiness operations.

File identity and coverage

The inventory records relative paths, SHA-256 hashes, file sizes, line counts, and detected kinds. It streams file content without a fixed file-size ceiling. It records unreadable files and relevant symlink observations. Scanner exclusions keep dependencies and generated runtime state outside ordinary inventory.

Coverage units divide files into symbols or line ranges. Claims use leases so interrupted workers do not retain permanent ownership. A successful characterization stores a summary, sensitivity, data touched, incoming flows, outgoing flows, and leads. The coverage view can then answer both what was examined and what the examiner learned.

A resolved percentage measures recorded characterization and dead-code dispositions. It is distinct from vulnerability proof and behavioral completeness. Binary-only intake requests an explicit analysis plan instead of inventing line coverage.

Sources: Inventory, coverage store.

Composable work modules

A module declares its task class, target source, dependencies, selection rules, priority, and execution limits. Its target can be the entire subject, an examined coverage unit, or a finding. A dependency applies to the same target rather than imposing a global phase on every worker.

Seven packs supply characterization, reverse engineering, security assessment, research, development, dynamic examination, and static analysis. Custom declarations can add subject-specific work without a new core driver. Path filters focus later unit passes on recorded component paths. Per-module limits control subject, finding, and later unit selection. Fresh characterization uses coverage claims and the global loop concurrency limit. That breadth path attaches the first active breadth module without applying module path filters.

The default selection enables security assessment and its characterization prerequisite. An explicit baseline selection supports a characterization-focused campaign. Read Work modules for configuration examples.

Source: Module registry and scheduler.

Specialist routing

The router maps task classes to specialist lanes and verification expectations. Claude normally handles research, interpretation, review, and synthesis. Codex normally handles enumeration, tooling, harness creation, patching, and verification. A subject lane pattern can distribute configured providers across active slots.

Optional Grok work uses a host-managed sandbox configuration. The manager transfers permitted subject material and records accepted results through REFALL. It has a separate execution role from the normal Claude and Codex Agent Canvas lanes.

Sources: Router, lane configuration.

Continuous evidence capture

Agent output can occur in messages, action text, structured submission files, or saved analysis files. REFALL reads these channels during the run. A specialist can therefore contribute useful observations before its final response.

Incremental ingestion and terminal processing share stable evidence identities. This prevents the same fact from appearing again on each poll. If the backend cannot persist an item, its identity remains eligible for a later attempt.

At termination, the loop classifies the invocation and parses its output. It checks submission identity and preserves observations before handling the classified outcome. A stopped worker can leave valuable evidence while its assigned task remains unfinished. A subject module with no durable observations creates pending work instead of a false success.

Source: Evidence capture.

Structured results

The refall.submit.v1 schema separates task results, findings, evidence changes, control requests, and threat models. Evidence-bearing results carry confidence, limitations, and source anchors or artifact references. Nested findings require concrete locations. Stable submission identifiers prevent repeated application of terminal results.

The parser also accepts legacy facts, leads, flows, coverage markers, and finding blocks. Strict mode makes a missing or malformed structured result a pending obligation. The loop can fill assignment metadata that it already knows. It does not create missing source proof.

Sources: Submission parser, typed contracts.

Findings and decisions

A candidate finding must supply evidence, confidence, and a source or artifact location. The promotion gate checks source references against the inventory when that inventory is available. It checks external hosts against explicit scope. A claimed validation result requires verification material.

Finding identity uses the subject, vulnerability class, normalized title, and affected component. Repeated locations stay attached to a canonical finding.

Lifecycle state and evidence quality remain separate. An accepted VALID transition requires a live_repro receipt. False-positive, duplicate, deferred, and downgrade decisions also require appropriate receipts. A stale source hash or conflicting pending obligation can refuse the transition. Accepted and refused attempts remain inspectable.

Sources: Promotion gate, finding transitions.

Scoped action authority

A capability grant identifies an action class, scope, grantor, expiry, and additional conditions. Authorization requests identify the actor, command, target, and declared effects. The broker compares these records before it returns an authorization decision.

Its checks include applicable host, port, HTTP method, filesystem path, expiry, and declared-identity conditions. Grant and revoke authentication requires a configured REFALL_CAPABILITY_OPERATOR_TOKEN_PATH. With the default unset token, the API retains caller-supplied identity fields. It distinguishes network, Docker, Codebadger, secret use, and destructive actions. A destructive grant can require a dry run and an approval token. The production launch path uses this broker before starting the standing loop.

The broker governs requests submitted to its authorization route. That route records a decision without executing the command. Direct worker tools do not automatically pass through it. The execution environment supplies the operating-system and container boundaries for agent processes. Existing mounts supply their configured permissions; the source-copy fallback does not make files read-only. Read Control in depth for the complete boundary model.

Source: Capability matching.

Pending obligations

Missing capability, blocked scope, incomplete proof, exhausted retries, and unanswered operator decisions become pending actions. An action identifies its source, priority, ownership, status, and required receipt. Workers claim eligible pending work before ordinary coverage units. An operator can resolve, defer, or cancel the action through the workbench.

This makes a blocked result visible work. It also gives later manager sessions an explicit account of what remains unresolved.

Source: Pending action store.

Continuity and supervision

Loop state preserves active conversations and the current iteration. Module progress preserves completed work for each target. The blackboard supplies compact shared notes. Runtime manifests and checkpoints connect a run to source identities, provider configuration, and stage results. Resume preflight checks recorded input hashes before resumed execution.

Transport retries handle temporary service failures. Worker retries distinguish repairable output from capability, scope, and operator-stop conditions. A retry can switch from Claude to Codex within the configured attempt limit. Successful conversations are archived before cleanup.

The watchdog examines progress, failures, budgets, and service state. The supervisor separates fast liveness checks from broader review. Repeated recovery can produce an operator hold rather than endless restart activity.

Sources: Standing loop, watchdog, supervisor.

Living reports

The draft combines findings, architecture facts, flows, coverage, runtime proof, pending actions, and capability decisions. Enrichment groups evidence and retains related locations. A report worker can maintain analyst notes alongside the formal deliverable. Report scheduling can use new evidence volume or reserve a continuous report slot.

The current report is an explicitly selected workbench object. Replacement checks protect populated technical sections from a thinner generated report. The operator can inspect a refused replacement candidate separately.

Visual evidence can include graph images, screenshots, captures, CodeCharta data, or focused property-graph slices. The report standards connect these artifacts to their claims and provenance.

Sources: Draft, enrichment, assembly.

Checked closeout

The closeout gate evaluates unfinished actions, disposition receipts, proof, expired claims, context, and report freshness. It separates blocking failures from advisory warnings. Coverage threshold, high-impact visuals, toolchain use, and disclosure output can produce warnings. Open obligations and a stale report can block completion. An explicit override receipt records an operator exception.

Source: Closeout gate.

FIELD GUIDE / 066 MIN READ

REFALL AB end-to-end workflows

From subject to research campaign

1. Attach the subject

Create a workbench for the selected source tree or material directory. The subject record separates the examined source from the workspace that stores run state and outputs. The workbench receives its own context, coverage, ledger, and report pointer.

bash
python3 -m ref_all_executor subject create example --source-path /absolute/path/to/example

2. Establish shared context

Generate the briefing through the Context view or named context operation. The context writer receives source material, workbench text, metadata, and operator direction. REFALL stores the resulting Markdown in the backend and subject workspace.

Read the briefing before execution. Use it to define the target, available material, research priorities, constraints, and done conditions.

bash
python3 -m ref_all_executor ops command context example

3. Select the work

Choose module packs and any subject-specific declarations. Set include or exclude globs for later passes over characterized units. Set the scope overlay and request a threat model when useful. A mapping campaign can select characterization and decomposition without security assessment modules.

4. Check readiness and authority

Run Doctor to inspect services, provider commands, paths, context, capabilities, and supporting tools. Run Estimate to inspect remaining work and prerequisites. Provide the subject-scoped launch grant through the capability interface. A configured operator token authenticates grant changes; the unset default does not. The browser client does not attach that token automatically. Use an authenticated host-side client when token protection is active.

bash
python3 -m ref_all_executor ops command doctor example
python3 -m ref_all_executor ops command estimate example

5. Start through backend operations

The web interface and production CLI use the same backend launch path. The backend checks context and authorizes the process request. It then checks Agent Canvas and projects the source into its configured workspace. A matching bind mount can supply the source directly. Otherwise, the source-sync helper copies a snapshot into the container.

bash
python3 -m ref_all_executor loop start example

6. Build the examination surface

If coverage is empty, the standing loop builds a source inventory. Each included file has a path, content hash, size, kind, and line count. Coverage units form the denominator for recorded examination. A binary-only root opens an analysis-plan action instead of becoming synthetic line coverage.

7. Assign a bounded task

The scheduler first considers pending actions and requested threat-model work. It then considers subject modules, fresh coverage units, finding modules, and later unit passes. Subject, finding, and later unit selection apply the relevant module eligibility checks. Fresh characterization uses coverage claims and the global loop concurrency limit. It attaches the first active breadth module and its objective.

The task objective contains context, scope, shared notes, target details, routing rationale, and an output contract. The dispatcher adds source, writable workspace, and approved resource information. Agent Canvas starts the selected specialist conversation.

8. Compile observations during execution

The loop polls execution state and reads agent-authored event messages. It also reads structured submission files and registers saved analysis files. Facts, leads, findings, and flow relationships become durable research records during the run. The workbench can show those results before the conversation ends.

9. Separate evidence from completion

Terminal handling first classifies the invocation, parses its output, and checks submission identity. It preserves useful observations before handling success, scheduling repair, or creating an operator obligation. Only the characterization success path submits its coverage result. Later modules extend the evidence without submitting that coverage again.

Malformed strict output, missing capability, and blocked scope create pending work. A failed worker does not erase its supported observations. A subject module without durable results does not advance as completed work.

10. Review the finding

A candidate passes through evidence and source-reference checks. The canonical record preserves equivalent evidence locations. A proposed state change then goes through the disposition gate. Typed receipts explain promotion, rejection, duplicates, deferral, or downgrade. Source-dependent receipts can include hashes for freshness checks.

11. Publish accumulated understanding

Periodic drafts and report workers compile findings, architecture, flows, proof, and remaining questions. Analyst notes preserve developing understanding separately from the formal report. The current report pointer selects the deliverable visible in the workbench. A structural comparison can refuse a replacement that removes populated technical sections.

12. Close with explicit state

Prepare the draft and enrichment before requesting closeout. The gate checks open obligations, validation, disposition receipts, expired claims, context, and the current report. It returns individual blocking checks and warnings. Resolve the blocking conditions or record an operator override receipt.

bash
python3 -m ref_all_executor report draft example
python3 -m ref_all_executor report enrich example
python3 -m ref_all_executor loop closeout example

Source and binary workflows

Main workbench loop

The main loop uses work modules and workbench-local APIs. It combines subject tasks, coverage breadth, finding tasks, and later examination passes. This is the loop launched by the normal Start and Update operations.

Source investigation driver

The standalone source family uses coverage_service.tools.seed_subject and drive_research.py. The seed scanner locates concrete source patterns for commands, parsers, files, network access, secrets, and authorization. It groups anchors into prioritized investigation questions.

A researcher claims a question, reads the anchor, traces data, and examines related callers or callees. Concrete leads can add child investigations to the frontier. The driver runs batches until that frontier is exhausted, a configured limit is reached, or the operator stops it. The specialist prompts use the standalone compatibility coverage service.

Binary investigation driver

The binary family begins with coverage_service.tools.seed_binary. It identifies executable format and imports through radare2. It maps selected dangerous imports to callers and generates function-level investigations.

drive_binresearch.py assigns bounded decompilation work. Workers examine pseudocode, disassembly, arguments, and cross-references. They trace attacker-controlled data to the relevant operation and add concrete branch questions.

drive_binreview.py runs an independent adversarial review of provisional findings. Reviewers reopen the cited function and challenge reachability, bounds checks, mitigations, and severity. They retain exact missing proof targets when a claim remains uncertain.

drive_binvalidate.py selects retained findings without complete runtime proof. Validators prepare a canonical payload, reproduction command, trace, raw output, and observed effect. Subject-specific host actions supply the required execution surface. The driver can publish research visuals and a disclosure package when its validation conditions are satisfied.

These standalone drivers have their own batch lifecycle and compatibility API contracts. They are separate entry points from the module scheduler. Read the driver arguments and subject harness requirements before selecting that family.

Recovery without losing the campaign

An Update request runs one bounded scheduling and polling cycle. It can resume active conversations from stored loop state. Per-target module progress prevents already completed passes from being repeated unnecessarily.

Resume preflight compares recorded source and lane-configuration hashes. A refused resume remains an explicit decision in runtime and pending state. Temporary HTTP failures use transport retries. Recoverable worker failures use bounded repair attempts. Capability and scope failures request decisions instead of repeated execution.

Conversation trajectories and event streams are archived before successful cleanup. The watchdog and supervisor inspect progress and service health. Recovery restores the failed component within the configured policy. Operator holds preserve visibility when repeated recovery is ineffective.

Source map

StageImplementation
Operator controlsWorkbenchOpsPanel
Backend lifecycleOperations
Scheduling and ingestionStanding loop
Per-target workModules
Coverage and decisionsCoverage store
Source frontierSource research
Binary frontierBinary research
Independent binary reviewBinary review
Runtime proofBinary validation
CloseoutCloseout gate
FIELD GUIDE / 0710 MIN READ

REFALL AB technical codex

This guide traces the implementation from operator intent to stored research, report updates, and checked campaign closeout. Read End-to-end workflows for the operator sequence. Read Work modules for subject configuration. Read Control in depth for each enforcement boundary, guidance layer, default, and exception.

1. Operator controls reach backend operations

WorkbenchOpsPanel.svelte loads context, scope, threat model, coverage, ledger records, pending actions, capabilities, runtime evidence, and closeout checks. Its lifecycle controls call the named operation API. The Start control requests a subject launch grant through the capability API when needed. Its browser client does not attach an operator token automatically. When token protection is configured, an authenticated host-side client can establish the grant first.

The production CLI uses RefallLoopManager. Its normal start and update methods call backend operations through ArcticBaseClient. The injectable direct launcher supports controlled embedding and focused tests.

The backend run_command route checks whether named operations are enabled. _execute_named_command validates and dispatches the requested command. _start_subject creates or selects the workbench and requires stored context. _update_subject reads the existing subject and requests a bounded pass.

Sources: Ops panel, CLI manager, backend dispatch.

2. Context becomes a shared input

_context_subject gathers subject material and runs the configured context writer. The writer receives operator direction, metadata, selected source content, and workbench text. Provider command construction is separate from the resulting briefing. The output is Markdown rather than an agent conversation history.

_save_context_artifacts stores the briefing as a subject file and workbench object. The coverage context record mirrors the same research input. The standing loop reads that record when it builds later objectives. The source inventory, scope overlay, and optional threat model remain distinct records.

This separation gives each specialist the same subject identity and declared boundaries. It also lets the operator revise context without reconstructing a worker's private state.

Source: Context generation and persistence.

3. Process launch has an authorization record

_authorize_process_launch constructs a run_local_command request. It includes the standing-loop argument vector, a declared process effect, the requester, and the subject path. The coverage store compares the request with active capability grants. The backend appends the decision to the workbench ledger. A refused decision returns HTTP 403 before process launch.

The capability matcher checks class, expiry, declared requester and grantor strings, and applicable host, port, method, and path conditions. The token-protected grant route supplies an authenticated operator identity only when its token file is configured. The default unset-token mode retains caller-supplied grantor fields. It also checks declared or recognized network, Docker, Codebadger, secret, and destructive effects. The general authorization route records a decision without executing a command. The backend launch caller obeys refusal; direct worker tools do not automatically pass through that route.

_launch_standing_loop verifies the repository path, prepares the subject projection, and checks Agent Canvas. It starts a detached process with the selected workbench URL and loop limits. Continuous loops can install the configured supervisor schedule. Bounded Update passes run without that continuous-loop schedule.

Sources: Launch authorization, capability matching, launcher.

4. Source projection and scratch state stay separate

sync_agent_canvas_workspace checks whether a matching Docker mount already supplies the source. When a mount exists, it records its read-only or writable state. Otherwise, it compares a source snapshot marker and copies the source when necessary. The snapshot marker uses file count, total size, and modification information for synchronization.

The inventory supplies separate content hashes for research identity. A synchronization marker and a source hash therefore serve different purposes.

The dispatcher prepares a deterministic writable directory for each assignment. The path includes the subject, lane, task class, and unit identity or objective hash. The objective identifies the scratch workspace and the subject source root separately. An operator can supply a read-only source mount as an execution boundary. The source-copy fallback does not create that restriction. The dispatcher's read-only source statement is model guidance. Process permissions and configured mounts determine actual file access.

Sources: Workspace projection, workspace naming, dispatcher.

5. Inventory establishes identity and examination units

build_source_inventory walks the subject under explicit exclusion rules. It hashes content in chunks and records size, line count, and kind. Unreadable files remain visible in the skipped list. Symlink observations retain relevant source relationships. Selected configuration directories, such as .github, remain eligible despite their dot prefix.

_auto_enumerate_if_empty stores the inventory and sends file records to coverage enumeration. It runs only when coverage has no recorded lines. A binary-only inventory creates binary_analysis_plan_required instead of synthetic line coverage.

partition_file and build_units turn supplied symbol ranges and uncovered lines into nonoverlapping units. claim_next uses a database transaction to claim an open unit or an expired lease. The default lease is 900 seconds. submit requires file-bearing evidence and writes the characterization fields.

Sources: Inventory builder, automatic enumeration, unit and claim store.

6. Scheduling follows target state

ModuleRegistry.from_config reads the subject's module declaration. The executable default selects security assessment and resolves its built-in characterization dependency. Custom declarations, disabled identifiers, and overrides then shape the registry. Validation rejects unknown classes, invalid concurrency values, missing dependencies, and dependency cycles.

Subject, finding, and later unit selection use eligible_modules. That function checks supplied target properties, dependencies, completion tracking, active work, and module concurrency. Later unit selection supplies the recorded path from its characterized-unit collection. It does not supply the optional unit_state argument. Finding selection supplies its recorded path when available.

Fresh breadth claims a coverage unit directly. _task_for_characterize attaches the first active breadth module, uses task class characterize, and sets 30 iterations. That branch uses the global concurrency limit rather than module path filters or per-module caps. ModuleScheduler maintains subject, unit, and finding progress. The standing loop then applies its dispatch order.

Pending actions and requested threat models take priority. Subject modules supply foundational research. Fresh coverage units supply breadth. Finding and later unit modules supply additional proof and depth.

A module's after dependency applies to the same target. The scheduler can therefore combine different kinds of work without a single global stage counter.

Sources: Registry, eligibility, claim order.

7. Objective construction carries the contract

The task builder combines the stored briefing, scope overlay, blackboard, target, module objective, and routing reason. Later unit passes also receive a prior-work instruction. They must examine stored determinations and saved artifacts before extending the analysis. The dispatcher then adds workspace and approved remote-resource information.

The output contract requests facts, leads, findings, flows, determinations, and durable artifacts. A mapping-oriented subject can select the available no-findings contract. That contract preserves architectural observations and leads without treating them as vulnerability findings.

Sources: Task objective, prior-work context, output contracts.

8. Output capture has several entry paths

An ACP worker can finish on a tool action without a populated final response. read_conversation_agent_text therefore reads the persisted event stream. It collects agent-authored message and action text in order. A final-response request is a fallback.

While a worker runs, _incremental_ingest reads the event stream and workspace. _merge_submission_file accepts structured submission content and embedded research observations. _register_workspace_artifact preserves saved analysis material. All paths converge on parsed findings, facts, leads, and flows.

_persist_new_intel computes stable identities for those records. Incremental ingestion and terminal salvage share the same identity set. A failed backend write removes staged identities so later polling can retry persistence. Persistent submission markers prevent repeated terminal application across retries.

Sources: Event capture, workspace capture, stable persistence.

9. Evidence is retained before outcome handling

_finish_conversation classifies the invocation and parses the result. It checks persistent submission identity before salvaging observations. It then preserves available research output on both successful and failed outcomes.

A failed outcome saves its output artifact and records retry state. A recoverable failure can launch a bounded repair attempt. Other outcomes create capability, scope, or operator-review obligations.

The clean success path applies transitions and persists control requests. Only a completed characterization task submits its coverage unit. Later passes retain evidence without submitting the breadth result again. A subject module with no canonical observations creates an ingestion-review action and remains incomplete.

Completed conversations save trajectory and event archives before cleanup. A failed archive or cleanup has its own recorded state.

Source: Terminal processing.

10. Claim validation and disposition are separate

evaluate_finding_candidate examines evidence, confidence, source membership, scope, and claimed validation material. The coverage store then establishes the canonical finding identity and related locations. New findings begin with provisional or research-needed lifecycle state.

transition_finding validates the proposed current and next states. It separately compares evidence-quality tiers. A VALID transition requires a live reproduction receipt. A quality increase requires the corresponding evidence type. Suppression, duplicate decisions, chain deferral, and downgrades have their own receipt rules.

The store refuses stale source hashes and transitions that contradict relevant pending obligations. It records both accepted and refused attempts. The API layer adds the decision to the evidence trail.

Sources: Candidate gate, transition store.

11. Research memory uses explicit records

RecordResponsibility
Coverage unitExamination identity, lease, determination, evidence, and leads.
Ledger entryFact, lead, finding, validation, artifact, or decision event.
BlackboardCompact shared notes for later tasks.
Module progressCompleted passes and known targets.
Runtime eventRun, task, session, and failure history.
ArtifactDurable file content and metadata.
ObservationProjected claim, evidence, confidence, and limitations.
CheckpointStage state and input identities.
Run manifestCurrent providers, source identity, resources, and stage status.
Analyst notesDeveloping architecture and research narrative.

The JSONL ledger records durable evidence and decisions. Runtime SQLite stores events, projections, retries, manifests, and checkpoints. The evidence graph relates those records. Static call graphs remain separate tool outputs.

Resume preflight compares checkpoint input hashes before resumed work. The manifest links source inventory and scope identifiers with the active run. These records support continuity while the active agent or manager changes.

Sources: Runtime store, projection, resume check.

12. Reports compile persistent evidence

draft_report collects coverage, ledger records, findings, runtime evidence, capabilities, and closeout information. The HTML builder provides detailed finding articles and research sections. The current report identifier is stored in refall_current_report_object_id.

enrich_findings merges equivalent finding locations and groups facts. It also carries runtime observations, artifacts, pending actions, and decisions into the enriched structure. Potential chain groupings remain hypotheses for analyst review.

Before replacement, draft and assembly paths compare populated structural sections. A thinner candidate can be preserved as a diagnostic object while the richer report remains current. This protects architecture, methods, and flow explanations accumulated during earlier research.

The report worker can maintain an analyst-notes artifact alongside the deliverable. The determinations digest records coverage, evidence counts, and module completion counts. Detailed per-unit determinations remain in coverage storage.

Sources: Draft, enrichment, replacement guard.

13. Closeout evaluates the campaign state

Closeout checks evidence-backed finding decisions, pending obligations, context, claims, proof, and report freshness. A missing or stale current report blocks completion. An open action or a deferred action without its receipt also blocks completion.

Coverage below the configured threshold produces a warning in the closeout gate. The gate also reports advisory visual, toolchain, and disclosure conditions. The blocking field distinguishes these checks from completion barriers. Receipt-backed overrides remain explicit ledger records.

The CLI closeout path drafts, enriches, and refreshes the draft before evaluating the gate. That last refresh includes evidence added during enrichment. It prevents a newly updated evidence trail from leaving the report immediately stale.

Sources: Closeout checks, CLI closeout.

FIELD GUIDE / 089 MIN READ

REFALL AB research architecture

The problem the architecture addresses

A long research campaign can outlive the conversations that produced its observations. Different workers can examine related code, repeat earlier work, or disagree about what a result proves. A final report then needs an explanation of scope, method, evidence, and unfinished obligations.

REFALL makes the campaign a persistent system of records. The subject owns the evidence. Workers receive bounded assignments and return material that the system can store, compare, and review. The operator retains decisions about scope, live authority, exceptions, and completion.

Three cooperating planes

Direction and authority

The operator defines the subject, objective, scope, available resources, and necessary approvals. The manager generates context, selects work, examines progress, and resolves questions. The capability broker evaluates the action requests that pass through its route. The disposition and closeout gates evaluate research decisions and completion state.

Execution

The standing loop creates bounded tasks and dispatches configured specialists through Agent Canvas/OpenHands. Source tools, decompilers, harnesses, and remote resources supply the relevant examination surface. The selected module determines the purpose of a pass. The execution environment determines the process and container boundary.

Persistent evidence

The backend stores inventory, coverage, ledger entries, findings, pending actions, capability records, runtime state, artifacts, and report metadata. These records provide input to the next task and the next manager session. The report is a compiled view of this research state.

STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 2STAGE 201Operator scope andauthority02perator03erator04rator05ator06tor07or08r09anager10nager11ager12ger13er14cheduler15heduler16eduler17duler18uler19ler20pecialists21ecialists22cialists23ialists24alists25lists26ists27sts28ts29s30vidence31idence32dence33ence34nce35ce36e37ecisions38cisions39isions40sions41ions42ons43ns44Manager context and workselection45Per-target modulescheduling46Bounded specialistexecution47Persistent evidence andartifacts48Promotion, disposition, and closeout49Report and analyst notes
Control and data flow

Ready Armor and control in depth

Ready Armor public architectures separate what an agent can reason about from what it can authorize. They also separate an agent's action from the evidence supporting its claim. REFALL expresses these principles at the research campaign level.

ConcernREFALL mechanism
What is the assignment?Stored context, scope overlay, target identity, and module objective.
Who owns the work?Coverage claims, pending-action leases, and active task state.
What action is authorized?Scoped capability grants and explicit authorization requests.
What was observed?Facts, flows, artifacts, and source-backed determinations.
What conclusion is supported?Finding promotion checks, quality tiers, and typed transition receipts.
What remains unfinished?Pending actions, refused transitions, validation state, and closeout checks.
What survives the worker?Workbench state, archived events, analyst notes, and reports.

BRA's public architecture focuses on governed assessment execution and materialization. ARA's public architecture focuses on agent runtime policy, engagement composition, procedures, and claim provenance. REFALL focuses on work allocation, research continuity, evidence compilation, report updates, and checked campaign closeout.

The systems share design ideas while retaining separate implementations. REFALL's scheduler, submission parser, ledger, capabilities, and gates are implemented in this repository. Its runtime does not import the ARA or BRA policy engines. A subject-specific ARA mapping contract treats ARA as examined software rather than a runtime dependency.

Build the control stack from actual boundaries

The lowest REFALL operation prepares resources for the configured execution environment. Source projection validates the input directory and destination subject name. It reuses an existing mount or copies source into the container. The existing mount supplies its actual write restrictions. The copy fallback creates an ordinary container directory.

Inventory then records content identity and source relationships. This record supports later membership and freshness decisions. It does not intercept file access, and an off-root file symlink can still supply inventory content.

The capability broker evaluates requests presented to it. Backend lifecycle operations use that result before loop process release. A refused launch request produces HTTP 403 before the process starts. The general authorization route only returns and records its decision. Other callers must obey that decision before they execute an operation.

Grant and revoke authentication activates with REFALL_CAPABILITY_OPERATOR_TOKEN_PATH. Without that setting, the routes retain supplied identities and do not authenticate an operator token. The matcher compares declared requester and grantor strings. Deployment access control and protected credentials establish the actual operator boundary.

Task selection adds a different control. Dependencies, target progress, applicable path filters, and concurrency rules select later work. Fresh breadth uses direct coverage claims and the global conversation limit. Scope and workspace statements in an objective are instructions to the model. Iteration limits are conversation parameters sent to Agent Canvas. The execution environment supplies the actual filesystem, process, network, and tool permissions.

Continuous capture observes execution and preserves material for subsequent decisions. Finding admission checks evidence references, available inventory membership, and explicit external-target scope. Typed transitions require the appropriate receipt before selected status and quality changes. These are record gates after worker activity; they do not retroactively prevent a tool action.

Closeout checks whether campaign records support a closed lifecycle result. Draft and current report objects can already exist when closeout remains blocked. An explicit exception can make identified failed checks nonblocking while preserving the decision record. The override route records a supplied operator identity without the capability-token authentication dependency.

This stack separates action release, assignment, guidance, observation, and record acceptance. An operator can therefore identify the control that refused a request and the authority needed for a different decision. Read Control in depth for the complete layer model and a worked refusal sequence.

Public background: BRA architecture preview, ARA architecture teaser.

Breadth and depth are distinct research work

An inventory gives the campaign a measurable starting surface. Characterization records what a unit does and the evidence for that description. Deeper passes can then trace behavior, review trust boundaries, construct harnesses, or verify findings.

The module scheduler prioritizes fresh coverage before repeated deep work on an initial subset. Per-target dependencies still preserve meaningful order within each component. This combination makes broad examination visible while retaining a route to deeper proof.

Recorded coverage is a progress measure. A high percentage does not establish that every behavior is correct or that every possible vulnerability was found. Findings require their own evidence and disposition checks.

Knowledge accumulates in several forms

A fact describes an observation. A flow describes a relationship between components or data locations. A lead identifies a concrete question and next step. A finding states a supported security issue. A determination summarizes a coverage unit's role, sensitivity, data, and connections.

Artifacts preserve material that would be costly to regenerate. Examples include disassembly, extracted metadata, graph output, traces, payloads, and screenshots. The blackboard gives later tasks compact shared context. Analyst notes retain the evolving architecture and research narrative.

These forms answer different questions. Keeping them separate prevents an open hypothesis from becoming a validated finding through repeated summarization. It also lets architecture mapping produce useful output without manufacturing vulnerabilities.

Completion is a transition

A worker's termination is an execution event. It is not sufficient evidence that its assigned research is complete. REFALL first classifies the invocation and parses its result. After the submission identity check, it preserves observations before handling the classified outcome.

Structured result validation checks the output envelope and evidence references. Coverage completion requires the appropriate characterization result. A subject module without canonical observations remains unfinished. Capability and scope blocks create operator-visible obligations.

The same principle applies to findings. A false-positive decision, duplicate decision, downgrade, or deferred chain needs a typed receipt. A supported primitive can remain preserved even when standalone exploitability has not been demonstrated.

Source identity matters across time

The source inventory binds observations to paths and content hashes. A disposition receipt can identify the exact source or configuration bytes that support it. Resume checkpoints record input hashes alongside stage status. The manifest links the active run to inventory and scope records.

This is useful when code changes during an extended campaign. It gives the manager a basis for distinguishing old proof from current material. The current report freshness check also compares publication time with the evidence trail.

Resources are selected for the subject

The core supplies task generation, result contracts, persistent state, and publication. A subject supplies its source, scope, module configuration, available tools, and necessary execution harnesses.

Source projects can use structural search, semantic navigation, symbol indexes, graphs, and complexity measurements. Binary projects can use decompilation, cross-references, function investigations, and runtime proof. Hardware or platform-specific work requires a configured execution surface and subject harness.

Module declarations add research intent and selection rules. Service definitions add managed tools. Capability grants add authority for specified actions. These are separate configuration decisions.

Models remain execution resources

Task routing distinguishes reasoning, implementation, review, and verification work. Lane patterns can distribute configured provider capacity. Bounded retries can switch providers when a recoverable worker failure prevents a usable result.

The shared workbench remains the source of research continuity. Replacing a model does not replace the subject inventory, evidence identifiers, pending obligations, or current report. The optional Grok configuration uses a host-managed sandbox with a separate material boundary. Its accepted output returns through the manager.

Evidence graphs and analysis graphs

The runtime evidence graph relates stored observations and artifact references. It supports review of where claims and outputs came from. Call graphs, dependency graphs, and property graphs describe the examined software. They are generated by the selected analysis tools.

The two graph families complement each other. An analysis graph can explain a code path. The evidence graph can connect that analysis artifact to the research record and resulting conclusion. Neither relationship substitutes for the proof required by a finding's transition.

Reports are maintained research products

A formal report presents findings, methods, scope, proof, limitations, and remediation. A debrief presents progress, decisions, blockers, and next actions. Analyst notes preserve an evolving technical explanation while the research continues. Visual evidence makes selected paths, relationships, and observed effects easier to examine.

Report generation reads persistent research state. Replacement checks protect populated technical sections from accidental loss. The closeout gate distinguishes unresolved completion barriers from advisory warnings. The operator can therefore see both the campaign result and the conditions under which it was accepted.

Implementation map

PrincipleImplementation
Bounded work and local dependenciesModule scheduler
Evidence before completionTerminal result processing
Input identitySource inventory
Scoped authorityCapability broker
Decisions with receiptsFinding transitions
Persistent continuityRuntime state
Preserved publication detailReport assembly
Explicit completionCloseout gate
FIELD GUIDE / 0911 MIN READ

REFALL AB control in depth

REFALL controls a research campaign through several distinct decisions. It selects work, checks brokered authority, captures results, validates record changes, and evaluates completion. Each decision has a defined boundary.

The execution environment controls process permissions, filesystem mounts, network access, and available tools. REFALL supplies campaign coordination and evidence controls above that environment. This separation makes the reason for each refusal visible. It also shows which component must enforce an execution restriction.

Read the layers from resources to decisions

LayerControl typeDecision or result
Source projectionOperation gateValidate the projection request and prepare the configured container workspace.
InventoryObservationRecord source identity, unreadable files, and symlink relationships.
Capability changesOperator authorityAuthenticate grant and revoke requests when token protection is configured.
Capability matchingPolicy gateAccept or refuse the request presented to the broker.
Backend launchOperation gateRefuse loop process release when authorization fails.
Work selectionTask gateSelect the next eligible assignment and allocate work claims.
Conversation boundsTask gateSend an iteration limit and classify outcomes before retry.
Context and scopeModel guidanceState the assignment, boundaries, resources, and expected evidence.
Resume preflightTask gateCompare checkpoint inputs on supported continuation paths.
Continuous captureObservationPreserve worker results and execution records.
Finding admissionRecord gateCheck evidence references and declared external scope.
Finding dispositionRecord gateCheck typed receipts before status or quality changes.
Pending resolutionRecord gateCheck required receipts for unresolved obligations.
Report replacementRecord gatePreserve populated report sections through the assembly path.
CloseoutRecord gateAccept or block the closed campaign result.
Closeout exceptionsOperator authorityRecord an explicit exception for identified checks.

This order describes responsibility from resources to campaign decisions. A particular command can visit these layers in a different order.

1. Establish the execution resources

A subject has a host source path and a container source location. The projection helper resolves the host path and requires a directory. It rejects empty subject names, slash-containing names, . and ... It also checks that the configured Agent Canvas container runs. A failed check prevents this synchronization operation from completing.

The helper first looks for a matching Docker mount. When a matching mount exists, the result records its actual read-only or writable mode. Otherwise, the helper copies the source into the container with docker cp. A copied directory has the container's ordinary file permissions. The copy path does not create a read-only mount.

Each dispatched task also receives a separate writable scratch directory. Its name derives from the subject, lane, task class, and unit identity or objective hash. The initial message identifies that directory and instructs the worker to preserve the subject source. Read-only mount configuration supplies mechanical write prevention where that restriction is required.

REFALL does not install a general interceptor around worker shell, network, SSH, or external tool calls. Agent Canvas, container configuration, process credentials, and tool services supply those execution boundaries. The standalone source and binary drivers also depend on their configured execution environments.

Sources: projection, workspace creation, workspace instructions.

2. Record source identity

The inventory records file paths, hashes, sizes, line counts, and kinds. Unreadable files remain in a skipped-file record. Symlink observations identify relationships that resolve outside the configured root. Directory traversal does not descend through directory symlinks. A file symlink can still supply content through its resolved target.

Inventory collection establishes examination identity. It does not confine later file access. Finding admission can compare source anchors with this inventory. Receipt validation can compare supplied source hashes with the current files. These controls detect record inconsistencies at their respective acceptance points.

Source: inventory and symlink observations.

3. Give capability changes an authenticated operator boundary

Capability grants define an action class, scope, grantor, expiry, and optional additional requirements. Grant and revoke routes support a host-managed operator token. Set REFALL_CAPABILITY_OPERATOR_TOKEN_PATH to activate this authentication. The request must supply X-REFALL-Operator-Token with the file's token value. Successful authentication sets the grantor to operator. A missing or invalid token gives HTTP 403. An unavailable credential file gives HTTP 503.

The token path defaults to unset. In that mode, grant and revoke routes do not authenticate the operator token. The grant route retains the supplied grantor identity. The browser capability client does not attach the operator token automatically. A protected deployment can issue grants through an authenticated host-side client.

The token file, API reachability, and host-side client belong to the deployment's authority boundary. Restrict their access to the intended operators. String fields such as granted_by and requested_by do not establish authenticated identity by themselves.

Source: operator authentication and capability routes.

4. Evaluate the action presented to the broker

An authorization request includes a capability class, declared effects, command, requester, and target. The broker requires a class and nonempty effects. It searches active grants of the same class and excludes expired grants. Revoked grants are excluded from that search. No matching grant means refusal.

The matcher applies host and port lists when both the relevant request value and grant restriction exist. A restricted HTTP method requires a matching declared method. A local-command request with a target path requires path scope. Applicable path scope compares the requested target with permitted roots. The matcher uses resolved paths, with a textual containment fallback for resolution errors.

Network, Docker, Codebadger, and secret checks connect applicable effects with their capability classes. Recognized destructive commands require destructive_action. Destructive declarations also activate configured dry-run and approval-token checks. These checks examine declared values and recognized command forms. They are not a complete shell parser or an inspection of every later process effect.

The self-grant rule compares requester and grantor strings. Matching nonoperator strings are refused. The protected grant route supplies the authenticated operator identity when token protection is active.

POST /api/workbenches/{slug}/execution/authorize returns the decision and records it in the ledger. It does not execute the proposed command. The caller must use the decision before it releases the action. A direct worker tool invocation does not automatically pass through this route.

Sources: matching, conditions, decision route.

5. Release the backend loop process

The named operation route supplies a concrete caller that obeys the broker decision. Operation routes default to disabled through REFALL_ENABLE_OPS. Named commands also require REFALL_ENABLE_OPS_COMMANDS. The command name must appear in REFALL_OPS_ALLOWED_COMMANDS. These switches enable functionality; they do not authenticate an operator.

Start requires stored, nonempty subject context. Its absence gives HTTP 409. When loop launch is enabled, Start and Update construct a run_local_command request for the standing loop. The request includes the subject path and a declared process effect. The backend records the authorization decision. Refusal gives HTTP 403 before the loop process starts.

Loop launch defaults to enabled once the operation route is enabled. Setting REFALL_ENABLE_LOOP_LAUNCH=false skips actual launch and its authorization check. Direct executor entry points and embedded launcher callbacks have their own calling paths. This launch gate describes the backend lifecycle route.

Source: backend lifecycle release.

6. Bound the assigned work

The scheduler prioritizes pending actions, subject setup, fresh coverage, finding work, and later unit passes. Claims and per-target progress coordinate allocation. Eligible module selection checks dependencies, completed work, active target work, and module concurrency. Recorded paths allow applicable inclusion and exclusion checks.

Fresh characterization claims coverage directly. It uses the characterize task class and a 30-iteration conversation request. It does not use the later module eligibility branch. Later unit selection supplies the recorded path, but it does not supply unit_state. Module fields therefore have specific selection paths, as described in Work modules.

The dispatcher sends the workspace, objective, provider settings, and iteration limit to Agent Canvas. Global loop parallelism defaults to four work items. The runtime classifies recognized capability, scope, and operator-stop outcomes before retry. Those outcomes create decision obligations rather than ordinary automatic retries. Recoverable worker failures use the configured retry limit, which defaults to two attempts.

Context and scope also appear in the worker's objective. These instructions guide reasoning, target selection, resource use, and evidence submission. They do not turn an assignment path into a filesystem or network access restriction.

Sources: selection order, eligibility, dispatch, outcome classification.

7. Check continuation and preserve observations

Resume preflight compares checkpoint input hashes with current source and configuration identities. A returned drift refusal stops the supported continuation path. The runtime retains a visible refusal event and decision state. This comparison requires prior checkpoints and the relevant API support. Missing optional support and some unavailable-service paths continue without a refusal.

During execution, polling reads available conversation events and workspace results. Completion handling first classifies the invocation, parses its output, and checks submission identity. It preserves available observations before handling the classified outcome or scheduling retry. Useful observations can therefore survive a failed conversation. A subject module without durable canonical observations remains unfinished.

Capture records what the worker supplied or what the runtime observed. It does not prove every recorded assertion or prevent the original tool action. Structured terminal submission is optional by default. Enable REFALL_REQUIRE_STRUCTURED_SUBMIT to require its contract during terminal validation.

Sources: resume comparison, resume caller, incremental ingestion.

8. Accept evidence and finding decisions separately

Finding admission requires evidence, confidence, and source or artifact references. When an inventory is available, source anchors must belong to it. Without inventory, anchor membership produces a warning. External URL hosts require explicit inclusion in the scope overlay's external_targets. A candidate outside that declared scope is refused admission. The API records the refusal and creates review work.

This check acts on a proposed finding after worker activity. It does not intercept the network operation that might have produced the evidence. The scope overlay and a capability grant are separate records. Changing one does not automatically change the other.

Existing finding decisions use typed transitions. Direct validation updates cannot change finding status. False-positive, duplicate, downgrade, chain deferral, and live-validation decisions have distinct receipt requirements. Unresolved contradictory obligations can block suppressive transitions. Receipts with supplied hashes also receive source-root containment and current-content checks. An unhashed receipt does not activate those hash checks.

A refused transition preserves finding state and stores the refused attempt. A receipt establishes the required decision record and evidence references. It does not independently prove all factual claims or authenticate its declared author.

Sources: finding admission, typed transitions, hash checks.

9. Require resolution and review completion

Pending actions preserve access requests, blocked scope, incomplete evidence, and decisions that still need attention. A required receipt type must match and include evidence references. Deferral and cancellation require receipt evidence. An untyped action completed as done does not universally require evidence.

Report assembly protects populated sections against structural downgrade. The default assembly path retains the current report and stores a diagnostic candidate when protected sections would disappear. An explicit allow_structural_downgrade argument bypasses that replacement guard. Generic object-content writes use a separate API path.

Closeout evaluates pending work, receipts, refused transitions, finding proof, expired claims, context, and report freshness. Blocking failures produce a blocked lifecycle result. Coverage below the default 95-percent threshold is advisory. Draft reports and current report objects can exist before closeout succeeds. The closeout gate does not control every object write, download, or external publication.

A closeout override requires identified checks and a nonempty receipt. An accepted override converts a matching failed check into a nonblocking warning. The ledger retains the exception and its stated reason. The override route records the supplied operator name without the capability-token authentication dependency. Deployment access control establishes who can reach that authority.

Sources: pending resolution, report protection, closeout, override route.

Worked example: an out-of-scope operation

Assume the subject is alpha. Its write_subject grant permits /tmp/refall-subject. An agent proposes touch /etc/passwd with the target path /etc/passwd.

  1. Submit the action to the authorization route with its class, effects, command, and target.
  2. The broker compares the target with the grant's permitted path.
  3. The broker returns accepted: false and records the refusal.
  4. The calling executor keeps the action unreleased.
  5. The operator examines the requested target and the actual research need.
  6. Keep the refusal, or issue a separate grant for an approved operation.
  7. Submit a new authorization request after any approved change.

The authorization route itself never launches touch. Step 4 belongs to the caller that releases the operation. The backend standing-loop launch implements this release discipline directly. Worker tools need their own runtime enforcement or an integration that obeys broker decisions.

The existing broker tests cover path refusal, host refusal, method restrictions, expiry, and protected grant authentication.

An external validation request uses two additional campaign decisions. The operator can add an approved host to the scope overlay and grant the necessary network capability separately. A later finding can then cite that host under the admission rules. The runtime still needs an executor that obeys authorization and an environment with the intended network restrictions.

How this expresses the Ready Armor approach

A useful control has a named authority, an explicit boundary, and an observable decision. REFALL applies that principle to campaign resources, brokered operations, assignments, evidence, and completion. Its controls make research progress and unfinished authority decisions reviewable across workers and sessions.

ARA and BRA describe related control principles through their own runtime and assessment architectures. REFALL implements its own scheduler, capability matcher, evidence records, and acceptance gates. It does not import their policy engines or supply their execution interception mechanisms. Read Research architecture for the wider system model.

FIELD GUIDE / 105 MIN READ

REFALL AB work modules

A work module declares a bounded research pass. The declaration identifies the work type, target source, prerequisites, selection rules, and additional objective. The scheduler evaluates modules against each target's current state.

Target sources

Source valueTargetTypical use
subjectThe whole subject.Identification, background research, source-tool execution, or synthesis.
coverage_unitsA file, symbol, or line-range unit.Characterization, decomposition, review, or harness creation.
findingsA finding record.Reproduction, verification, or patch proposals.

A subject pass normally runs once for the subject. A later unit pass can run once for each eligible characterized unit. A finding pass can run once for each eligible finding. These selection paths use once_per_target and active-target tracking to prevent duplicate concurrent dispatch. Fresh characterization has a separate coverage-claim path.

Built-in packs

PackModulesPurpose
baselinecharacterizeDescribe units with source anchors.
reverse_engineeringre_identify, re_decompose, re_reviewIdentify the subject, explain components, and review the decomposition.
security_assessmentsec_recon, sec_review, sec_verifyIdentify attack surfaces, examine candidate issues, and verify findings.
researchres_research, res_synthesisResearch the target class and compile the results.
developmentdev_harness, dev_patchDraft harnesses and propose fixes for findings.
dynamic_testingdyn_harness, dyn_verifyBuild an approved harness and record its runtime behavior.
static_analysissa_toolchain, sa_reviewProduce source-tool outputs and examine their meaning.

The default executable selection is security_assessment. Dependency resolution also adds its required characterization module. Select baseline explicitly for a characterization-only run. Packs can be combined. A pack is a convenience group rather than a global execution phase.

Dependencies and order

The after list refers to modules completed on the same target. For example, re_review depends on re_decompose for that unit. It does not require decomposition of the whole repository.

The registry resolves missing built-in dependencies across packs. It rejects unknown dependencies and dependency cycles. Subject selection applies dependencies, completion tracking, active-target tracking, and module concurrency limits. Later unit selection also applies path rules to the recorded unit path. Finding selection applies path rules when a finding path is available. Lower numerical priority runs first within the relevant selection.

Fresh breadth uses coverage claims rather than the module eligibility function. The loop attaches the first active breadth module and always dispatches the characterize task class. That task receives the module objective and a limit of 30 iterations. The global loop concurrency limit controls these tasks. Breadth selection does not apply module path filters or per-module concurrency limits. Use dependency-free coverage modules for characterization. Use modules with an after dependency for filtered later passes.

The standing loop adds a broader dispatch order:

  1. Claim pending actions and requested threat-model work.
  2. Select eligible subject modules.
  3. Claim fresh coverage units for breadth.
  4. Select finding-specific modules.
  5. Select later passes on characterized units.

Fresh breadth takes precedence over repeated deep work on an initial subset. Finding verification and later unit work continue after fresh units are exhausted.

Configure a subject

Store the configuration in the workbench refall_modules custom field. The loader accepts an object or a JSON string. The workbench metadata API represents custom field values as strings. Use JSON text when updating this field through that API.

A combined source-research configuration is:

json
{
  "packs": [
    "reverse_engineering",
    "security_assessment",
    "static_analysis"
  ],
  "overrides": {
    "sec_review": {
      "path_globs": ["src/**", "lib/**"],
      "exclude_globs": ["**/generated/**"],
      "max_parallel": 2
    }
  }
}

This selects subject identification, decomposition, review, source analysis, security review, and finding verification. It limits the security review pass to selected paths and two concurrent tasks.

A custom subject pass can be added as follows:

json
{
  "packs": ["baseline", "reverse_engineering"],
  "modules": [
    {
      "id": "interface_catalog",
      "task_class": "research",
      "source": "subject",
      "priority": 25,
      "max_parallel": 1,
      "objective": "Catalog documented interfaces and anchor each entry to the subject material."
    }
  ]
}

The custom objective becomes part of a bounded task. The module uses the normal context, evidence, persistence, and result-processing path. Its task class must exist in the router. A declaration does not install a new tool or execution service.

Select targets and limits

FieldMeaning
idStable module identifier.
task_classRouter task class.
sourceSubject, coverage units, or findings.
afterPrerequisites on the same target.
unit_stateState predicate for eligibility callers that supply the target state.
path_globsIncluded paths for later unit passes and findings with a recorded path.
exclude_globsExcluded paths for those same selection paths.
priorityNumerical ordering value.
max_parallelModule concurrency limit for subject, finding, and later unit selection.
objectiveAdditional specialist instructions.
enabledWhether the module participates.
once_per_targetWhether completed targets remain excluded.
packModule group label.

The later unit scheduler draws targets from its characterized-unit collection. It does not supply unit_state to the eligibility function. Its target collection and module dependencies establish that path's progress requirements.

The disabled list can disable selected module identifiers. The overrides map can adjust a built-in declaration. Review dependencies when disabling prerequisite modules.

Adapt the research objective

For architecture mapping, select baseline and reverse-engineering modules. Use facts, leads, flows, and saved diagrams as the principal outputs. The refall_no_findings field selects the available mapping-oriented output contract and report behavior.

For a security campaign, combine security assessment with research or static analysis when useful. For remediation work, add development modules and supply the required write authority. For runtime examination, supply a subject harness and the necessary capability grants.

Preserve prior work

The scheduler stores progress in .refall/module_progress.json. It records completed module identifiers, characterized units, and known findings. Later unit objectives tell workers to read prior determinations, findings, shared notes, and saved analysis artifacts. A successful later pass extends the evidence without submitting the characterization result again.

Subject modules must produce durable facts, leads, findings, or flows. A pass without canonical output creates an ingestion-review action and remains unfinished. This requirement keeps a completed task from becoming an empty progress marker.

Sources: WorkModule, registry, scheduler, dispatch order.

FIELD GUIDE / 114 MIN READ

REFALL AB architecture map

REFALL AB connects an operator workbench, backend control services, a persistent research loop, specialist execution, and analysis resources. The subject workbench is the common state boundary. Read Research architecture for the design rationale. Read the Technical codex for the implementation path.

Components

ComponentLocationResponsibility
Operator interfacearcticbase/frontend/src/lib/refall/Context, scope, research controls, evidence, decisions, and closeout views.
Backend operationsarcticbase/backend/src/arctic_base/api/refall_ops_commands.pyContext writing, readiness, authorized lifecycle, and report operations.
Coverage and finding storearcticbase/backend/src/arctic_base/refall/coverage_store.pyUnits, leases, determinations, findings, receipts, pending actions, and capabilities.
Ledgerarcticbase/backend/src/arctic_base/refall/ledger.pyPersistent evidence and decision entries.
Runtime storagearcticbase/backend/src/arctic_base/refall/runtime_state.pyEvents, observations, retries, artifacts, manifests, and checkpoints.
Standing loopref_all_executor/standing_loop.pyScheduling, task construction, polling, ingestion, recovery, and report cadence.
Module registryref_all_executor/work_modules.pySubject-specific task declarations and per-target dependencies.
Agent executionAgent Canvas/OpenHandsSpecialist conversations and their execution workspaces.
Report pipelineref_all_executor/report_*.pyDrafting, enrichment, current-object updates, and replacement protection.
Standalone researchdrive_research.py, drive_bin*.pySource and compiled-binary investigation drivers.
Compatibility coveragecoverage_service/Standalone coverage service, investigation tools, and reusable source or binary helpers.
Analysis toolchaintoolchain/Source tools, indexes, graphs, metrics, and optional MCP services.

The backend implementation retains the ArcticBase name in paths and process names. The product name is REFALL AB.

Service flow

STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 1STAGE 2STAGE 2STAGE 201Operator interface orCLI02perator03erator04rator05ator06tor07or08r09ackend10ckend11kend12end13nd14d15uthority16thority17hority18ority19rity20ity21ty22y23oop24op25p26cheduler27heduler28eduler29duler30uler31ler32er33anvas34nvas35vas36as37s38ngest39gest40est41st42t43tate44ate45te46e47eports48ports49orts50rts51ts52Backend operations53Capability broker54Standing loop55Module scheduler56Agent Canvas/OpenHands57Source, tools, harnesses58Result ingestion59Coverage, ledger,runtime60Reports and analyst notes61Closeout gate
Control and data flow

State ownership

Workbench metadata and objects

The backend stores the subject identity, source path, display theme, custom module settings, and report pointer. Objects hold context, HTML reports, uploaded material, and published artifacts. refall_current_report_object_id identifies the current report explicitly.

Coverage storage

refall_coverage.db stores coverage units, evidence anchors, finding state, transition receipts, pending actions, capabilities, and supporting research records. A coverage unit includes ownership, lease expiry, summary, sensitivity, touched data, flows, and leads. Coverage percentages summarize recorded unit state.

Evidence ledger

refall_ledger.jsonl stores facts, leads, findings, validation, artifacts, and important decisions. Entries identify the workbench and relevant task or lane. This ledger is distinct from the runtime database and agent conversation archive.

Subject runtime

.refall/runtime/state.sqlite stores events, sessions, observations, retry state, artifacts, manifests, and checkpoints. .refall/runtime/artifacts/ holds artifact content. Runtime projections expose observations, evidence relationships, and nonfatal errors through workbench APIs.

Scheduling state

.refall/loop_state.json preserves active conversations and iteration state. .refall/module_progress.json preserves completed passes and known targets. .refall/blackboard.json provides compact shared notes. These records support continuation without making a conversation transcript the research database.

Research work and provider roles

The manager examines context, resources, unresolved work, and current proof. The standing loop turns that state into bounded assignments. Claude and Codex use configured ACP lanes through Agent Canvas. The optional Grok role is controlled by the host manager in a separate sandbox.

The module registry can combine source understanding, security review, research, development, static analysis, and dynamic examination. Each module has a target source and optional local dependencies. The scheduler prioritizes pending actions, subject setup, fresh breadth, finding work, and later unit passes.

Control points

Source projection prepares the configured container location and reports existing mount permissions. Inventory records content identity rather than preventing file access. The capability broker evaluates presented action requests against grants. Backend Start and Update enforce a refusal before loop process release. The generic authorization route returns a decision without executing the command.

Grant and revoke authentication requires REFALL_CAPABILITY_OPERATOR_TOKEN_PATH. The unset default retains caller-supplied grantor identities. Task context supplies scope guidance; Agent Canvas and deployment permissions supply the execution boundary. The promotion gate checks candidate evidence, source identity, and external scope. The disposition gate checks finding transitions and typed receipts. The closeout gate checks the persistent campaign state and report freshness.

These control points govern different transitions. A successful process launch does not establish a valid finding. A valid finding does not establish that the campaign is complete. Draft report objects can exist before the closeout gate passes. Read Control in depth for action gates, record gates, guidance, observations, and explicit exceptions.

Recovery and publication

HTTP retries handle transient service failures. Terminal handling classifies the invocation and preserves output before handling failure or scheduling retry. Checkpoints and manifests connect resumed work to source and configuration identities. The watchdog and supervisor expose stalled or degraded execution and bounded recovery.

Draft and enrichment stages compile stored evidence. A replacement guard protects populated architecture, methods, and flow sections. Closeout then reports blocking obligations and advisory warnings separately.

Additional references

FIELD GUIDE / 122 MIN READ

REFALL AB installation

Requirements

Use Python 3.12 or later for the backend. Use uv to install the backend dependencies. The frontend uses pnpm and Node.js. The Docker build uses Node.js 24 and Python 3.13.

Research execution requires an Agent Canvas/OpenHands service with its Automation API. A manager also requires the selected provider CLI. Docker is necessary for container deployment and Codebadger.

Install the backend

Run these commands from the refall directory.

  1. Install the backend and development dependencies.
bash
   uv sync --project arcticbase/backend --group dev
  1. Install the executor dependencies in the same environment.
bash
   uv pip install --python arcticbase/backend/.venv/bin/python -r coverage_service/requirements.txt
  1. Select the repository directory.
bash
   export REFALL_REPO_ROOT="$PWD"
   export PYTHONPATH="$PWD:$PWD/arcticbase/backend/src"
   export PATH="$PWD/arcticbase/backend/.venv/bin:$PATH"

Build the web interface

  1. Install the frontend dependencies.
bash
   pnpm --dir arcticbase/frontend install --frozen-lockfile
  1. Build the web interface.
bash
   pnpm --dir arcticbase/frontend build

Configure local services

  1. Select the local service address.
bash
   export ARCTIC_BASE_HOST=127.0.0.1
   export ARCTIC_BASE_PORT=7673
   export REFALL_PUBLIC_BASE_URL=http://127.0.0.1:7673
   export REFALL_WORKBENCH_BASE="$REFALL_PUBLIC_BASE_URL"
  1. Select writable storage paths.
bash
   export ARCTIC_BASE_DATA="$PWD/data/arcticbase"
   export REFALL_DATA_ROOT="$PWD/data/refall"
   export REFALL_SUBJECT_ROOT="$REFALL_DATA_ROOT/subjects"
   export ARCTIC_BASE_FRONTEND_DIST="$PWD/arcticbase/frontend/dist"
   export ARCTIC_BASE_SEED_DATA="$PWD/arcticbase/backend/seed"
  1. Set the Agent Canvas address and API key path.
bash
   export REFALL_OPENHANDS_BASE=http://127.0.0.1:8000
   export REFALL_OPENHANDS_API_KEY_PATH="$HOME/.openhands/agent-canvas/api-key.txt"
  1. Enable the operation routes.
bash
   export REFALL_ENABLE_OPS=true
   export REFALL_ENABLE_OPS_COMMANDS=true
  1. Start the backend.
bash
   python -m arctic_base
  1. Open http://127.0.0.1:7673.
  1. Read the health response.
bash
   curl -fsS http://127.0.0.1:7673/api/health

The default backend address uses a detected LAN address when no host override is set. The explicit local address above keeps all local examples consistent. For remote access, set the public URL and listen address for that deployment.

Configure execution

  1. Install and configure Agent Canvas/OpenHands separately.
  2. Mount the repository and subject directories in the agent container.
  3. Set REFALL_WORKSPACE to the container repository path.
  4. Set REFALL_SUBJECT_WORKSPACE_ROOT to the container subject parent.
  5. Configure the selected provider CLI.
  6. Run the subject diagnosis procedure before the first research loop.

The default container repository path is /projects/REFALL-V10. The default container subject parent is /projects. The host source path and container source path must refer to the same subject files.

Optional source tools

Install the analysis tools with toolchain/install.sh. Read Toolchain before you start optional services.

Container deployment

Read Deployment for the Docker Compose configuration. Read Configuration for the complete service settings. Read starthere/ or docs/codex-startup/ for the manager startup sequence.

FIELD GUIDE / 134 MIN READ

REFALL AB configuration

Set environment variables before the process starts. The backend settings are defined in arcticbase/backend/src/arctic_base/config.py. The executor settings are defined in ref_all_executor/config.py.

Backend and storage

VariableDefaultPurpose
ARCTIC_BASE_HOSTauto-lanBackend listen address.
ARCTIC_BASE_PORT7673Backend listen port.
ARCTIC_BASE_DATA./dataWorkbench storage directory.
ARCTIC_BASE_FRONTEND_DIST../frontend/distBuilt frontend directory.
ARCTIC_BASE_SEED_DATA./seedInitial workbench data.
REFALL_DATA_ROOT/var/lib/refallREFALL data parent.
REFALL_SUBJECT_ROOTDerived from the data rootSubject storage parent.
REFALL_ARCTICBASE_DATA/var/lib/refall/arcticbaseDeployment storage path.
REFALL_ARCTICBASE_HOSTauto-lanAdvertised service host.
REFALL_ARCTICBASE_PORT7673Advertised service port.
REFALL_PUBLIC_BASE_URLauto-lanPublic service URL.
REFALL_REPO_ROOTDerived from the backend locationRepository path used by operation commands.

auto-lan selects a private LAN address. The fallback address is 127.0.0.1. Set explicit paths for portable deployments.

Execution and operations

VariableDefaultPurpose
REFALL_ENABLE_OPSfalseEnable operation routes.
REFALL_ENABLE_OPS_COMMANDSfalseEnable named operation commands.
REFALL_ENABLE_LOOP_LAUNCHtruePermit capability-checked process launch.
REFALL_ENABLE_LOOP_SUPERVISOR_CRONtrueEnable supervisor cron support.
REFALL_OPS_ALLOWED_COMMANDSSee the deployment exampleAllowlist for named commands.
REFALL_LOOP_MAX_PARALLEL4Maximum concurrent work items.
REFALL_LOOP_POLL_INTERVAL_S5Poll interval in seconds.
REFALL_LOOP_REPORT_INTERVAL_CYCLES10Report interval in loop cycles.
REFALL_LOOP_STALL_CYCLES36Stall threshold in loop cycles.
REFALL_CAPABILITY_OPERATOR_TOKEN_PATHUnsetOperator token file for capability changes.

The named command allowlist is in deploy/refall-arcticbase.env.example. It includes status, subject, agent, lifecycle, context, coverage, diagnosis, estimate, and log commands.

Operator authentication and action release

Set REFALL_CAPABILITY_OPERATOR_TOKEN_PATH to a host-managed token file to authenticate capability grant and revoke requests. Keep this file outside worker access. Send its value in X-REFALL-Operator-Token from the authenticated operator client. The backend uses constant-time comparison and records the authenticated grantor as operator. An unavailable token file gives HTTP 503; an invalid token gives HTTP 403.

The default unset value disables token authentication on these routes. Grantor fields then come from the request body. The browser capability client does not attach this token automatically. Use a host-side authenticated client to establish grants in a protected deployment. Operation enable switches control functionality rather than caller identity. The closeout override route records the supplied operator name and has no capability-token authentication dependency. Restrict API reachability to the intended callers.

The authorization endpoint returns a decision without executing commands. Backend Start and Update obey that decision before process launch. Worker tools need their actual runtime permissions or an executor integration that obeys the broker. Read Control in depth for applicable scope fields, defaults, and record gates.

Agent execution

VariableDefaultPurpose
REFALL_OPENHANDS_BASELocal port 8000Agent Canvas service URL.
REFALL_OPENHANDS_API_KEY_PATH~/.openhands/agent-canvas/api-key.txt in the executorAPI key file.
REFALL_WORKSPACE/projects/REFALL-V10Container repository path.
REFALL_SUBJECT_WORKSPACE_ROOT/projectsContainer subject parent.
REFALL_AGENT_WORKSPACE_HOST_ROOT~/refallsubHost workspaces for agent output.
REFALL_AGENT_WORKSPACE_CONTAINER_ROOT/projects/refallsubContainer workspaces for agent output.
REFALL_HEAD_PROVIDERcodexManager provider.
REFALL_ALLOWED_HEAD_PROVIDERScodex,claude,grokPermitted manager providers.
REFALL_CODEX_COMMANDcodexCodex executable.
REFALL_CLAUDE_COMMANDclaudeClaude executable.
REFALL_GROK_COMMANDgrokGrok executable.
REFALL_CONTEXT_MANAGER_TIMEOUT_S1800Context generation timeout in seconds.
REFALL_WORKBENCH_BASEPublic URL or detected LAN URLExecutor workbench URL.
REFALL_WORKBENCH_SLUGref-allDefault publishing workbench.

The backend API key path defaults to unset. Configure the same key file for the backend and executor when both call Agent Canvas. The Automation API prefix is /api/automation/v1.

Source projection reuses matching Docker mounts with their existing access mode. Its copy fallback creates an ordinary container directory. Configure read-only source mounts when mechanical write protection is required. The dispatcher identifies separate source and scratch locations in the worker objective. Those instructions do not replace mount permissions or network restrictions.

Result validation and retries

VariableDefaultPurpose
REFALL_REQUIRE_STRUCTURED_SUBMITfalseReject missing or malformed structured results when enabled.
REFALL_RETRY_MAX_ATTEMPTS2Retry limit for recoverable failures.
REFALL_RETRY_BACKOFF_S5Delay between retries in seconds.
REFALL_RETRY_MAX_REPAIR_PROMPT_CHARS4000Maximum prior output in a repair prompt.

Optional services

ServiceAddress or configuration
Compatibility coverage APIhttp://localhost:8900
Coverage from an agent containerhttp://host.docker.internal:8900
Serena MCPhttp://127.0.0.1:20872/mcp
Python LSP127.0.0.1:20871
Codebadger MCPhttp://127.0.0.1:4242/mcp
Codebase-Memory MCPStandard input and output.

REFALL_SSH_RESOURCE_ALIASES selects configured remote resources from the built-in allowlist. The current allowlist contains sectortw and sectortl. Set an empty value to disable these resources. Read Remote resources before remote execution.

FIELD GUIDE / 142 MIN READ

REFALL AB operating procedures

Use these procedures from the repository directory. Complete Installation first. The CLI uses REFALL_WORKBENCH_BASE or REFALL_PUBLIC_BASE_URL to select the workbench service.

Create a subject

A subject is the source tree, binary, or system under examination. A workbench stores the subject context, research records, and reports. Use one workbench for each subject.

  1. Create the subject workbench.
bash
   python3 -m ref_all_executor subject create example --source-path /absolute/path/to/example
  1. Generate the shared context.
bash
   python3 -m ref_all_executor ops command context example
  1. Examine the stored context in the workbench.
  2. Examine the source inventory and scope.
  3. Run the diagnosis command.
bash
   python3 -m ref_all_executor ops command doctor example
  1. Get the workload estimate.
bash
   python3 -m ref_all_executor ops command estimate example

Grant a launch capability

A capability is an operator grant for a specified action and scope. A loop launch requires a subject-scoped run_local_command grant when launch execution is enabled. The operator token comes from REFALL_CAPABILITY_OPERATOR_TOKEN_PATH when that setting is configured.

  1. Submit the launch grant through the workbench capability interface.
  2. Select run_local_command as the action class.
  3. Set the scope to the subject source path.
  4. Record the reason for the grant.

The capability endpoint is /api/workbenches/{slug}/capabilities/grant. Agents cannot grant themselves permission for live actions.

Run research

Stored context is necessary before the first loop launch.

  1. Start the loop.
bash
   python3 -m ref_all_executor loop start example
  1. Examine the operation status.
bash
   python3 -m ref_all_executor ops status
  1. Examine coverage, ledger records, findings, and pending actions in the workbench.
  2. Resolve open capability requests and scope decisions.
  3. Run a bounded update when necessary.
bash
   python3 -m ref_all_executor loop update example

Start Loop and Update in the web interface use the same operation routes. Doctor examines services, paths, capabilities, provider commands, and supporting tools. Estimate measures the work represented by the current subject inventory.

Prepare a report

  1. Create the report draft.
bash
   python3 -m ref_all_executor report draft example
  1. Enrich the report.
bash
   python3 -m ref_all_executor report enrich example
  1. Examine the current report in the workbench.
  2. Examine the closeout gate.
bash
   curl -fsS "$REFALL_PUBLIC_BASE_URL/api/workbenches/example/closeout/gate"
  1. Resolve each open condition.
  2. Request closeout.
bash
   python3 -m ref_all_executor loop closeout example

Report enrichment adds evidence context, related findings, remediation information, and visual evidence. The closeout gate identifies pending actions, refused transitions, incomplete validation, stale reports, and missing receipts.

Examine the ledger

The ledger is a sequence of evidence and decision records. Each significant research result, disposition, capability decision, and report change belongs in the ledger.

bash
curl -fsS "$REFALL_PUBLIC_BASE_URL/api/workbenches/example/ledger/entries"

Stop local services

  1. Examine the shutdown actions.
bash
   python3 refall_down.py --dry-run --include-serena
  1. Stop the local runtime.
bash
   python3 refall_down.py --include-serena

References

The alternate startup sequence is in docs/codex-startup/*.md.

FIELD GUIDE / 154 MIN READ

REFALL Manager Operating Guide

The manager operates REFALL AB and coordinates specialist research. ArcticBase is the backend implementation name. Use REFALL AB as the product name in status messages and procedures.

Read first

  1. Read starthere/ in numeric order.
  2. Read AGENTS.md.
  3. Read docs/manager/AGENTS.md.
  4. Read CODEX.md or CLAUDE.md for the selected role.
  5. Read the subject context before you claim work.

The alternate startup pack is docs/codex-startup/*.md. Read Operating procedures for the command sequence.

Prepare infrastructure

Run these commands from the repository directory.

bash
export REFALL_REPO_ROOT="$PWD"
export REFALL_PUBLIC_BASE_URL=http://127.0.0.1:7673
export REFALL_WORKBENCH_BASE="$REFALL_PUBLIC_BASE_URL"
export REFALL_OPENHANDS_BASE=http://127.0.0.1:8000
  1. Complete the installation procedure.
  2. Read /api/health.
  3. Read /api/refall/ops/status.
  4. Examine /api/refall/ops/infra and /api/refall/ops/agents.
  5. Examine Agent Canvas/OpenHands connectivity.
  6. Examine the selected provider CLI and subject mounts.
  7. Examine remote resource access when the subject requires it.

Operation routes require REFALL_ENABLE_OPS=true. Named commands require REFALL_ENABLE_OPS_COMMANDS=true. The allowlist must include the requested command.

Create the subject

  1. Create the workbench.
bash
   python3 -m ref_all_executor subject create example --source-path /absolute/path/to/example
  1. Examine the source path from the backend host.
  2. Examine the matching source mount from the agent container.
  3. Set the subject scope and exclusions.

Create Workbench Theme

Use PUT /api/workbenches/{slug}/theme to set the workbench appearance. The JSON body contains the complete theme document in raw. The theme document contains YAML frontmatter and Markdown. Read /api/agent/conventions for the required tokens and layout values.

Generate Context

  1. Request context generation.
bash
   python3 -m ref_all_executor ops command context example
  1. Read /api/workbenches/example/coverage/context.
  2. Examine the stored context.md content in info_md.
  3. Examine source inventory, scope overlay, and the threat model.
  4. Run doctor example and estimate example when readiness is uncertain.

A loop cannot start without stored context. The manager uses the context to constrain discovery, validation, and report preparation.

Start Loop

  1. Obtain a subject-scoped run_local_command capability from the operator.
  2. Start the loop.
bash
   python3 -m ref_all_executor loop start example
  1. Examine the operation response and agent state.

With REFALL_ENABLE_LOOP_LAUNCH=true, start launches ref_all_executor.standing_loop with --max-cycles 0. Update launches the same runner with --max-cycles 1. The manager must not replace this runner with undocumented work in chat.

Monitor and recover

Examine these records during each manager cycle:

  • Coverage statistics, claim leases, and the coverage gate.
  • Source inventory and scope overlay.
  • Pending actions and capability requests.
  • Runtime manifest, observations, artifacts, and the evidence graph.
  • Current findings, disposition refusals, and report freshness.

The live diagnosis route is /api/refall/ops/debug/{subject}/live. The run manifest is /api/workbenches/{slug}/runtime/manifest. Local loop state uses .refall/loop_state.json. The nonfatal error route is a read-only projection.

  1. Resolve pending actions before ordinary coverage work.
  2. Examine failures before you request another cycle.
  3. Request a bounded update when recovery is appropriate.
bash
   python3 -m ref_all_executor loop update example
  1. Append important decisions and observations to the ledger.

The watchdog and supervisor preserve liveness information. The recovery policy restores failed services. It does not use service failure as a reason to tear down the stack. Review stale loop state before you start replacement work.

Specialist output

Specialists return fenced refall-submit JSON with schema refall.submit.v1. Use the task_result_submit event and a stable submission_id. Evidence-bearing results require evidence, confidence, limitations, and source anchors or artifact references. Use REFALL_REQUIRE_STRUCTURED_SUBMIT=true to require valid structured output.

The executor ignores duplicate structured submissions. It stores a nested finding before it applies proposed transitions. New findings begin as PROVISIONALLY_VALID or NEEDS_RESEARCH. VALID describes finding status. LIVE_CONFIRMED describes evidence quality. Keep these properties separate.

Submit suppressive changes through /api/workbenches/{slug}/findings/{finding_id}/transitions. Use typed evidence receipts. Include source_hashes when proof depends on exact source or configuration bytes. A stale receipt is refused.

A scope_blocked result creates a scope decision. A capability_missing result creates a capability request. Neither result completes the unit. The specialist cannot authorize its own live action.

Remote SSH Resources

REFALL_SSH_RESOURCE_ALIASES selects remote resources from the configured allowlist. The current allowlist is sectortw,sectortl. Set an empty value to disable remote resources.

  1. Assign a remote resource only when the task requires it.
  2. Run the command with ssh <alias> <command>.
  3. Record the alias, command, evidence, and limitations in the submission.

Grok is not an SSH resource. Its optional worker is host-orchestrated inside a sandbox. It must not receive host credentials. The manager transfers permitted subject material into the sandbox and records its results through REFALL APIs.

Analysis and visual evidence

Run source analysis with toolchain/run.sh when it supplies useful evidence. Start Codebadger for property graphs, data flow, taint analysis, or slicing when necessary. Preserve the command and input identity with analysis artifacts. Link screenshots, graphs, CodeCharta views, and other visual evidence to the supported finding.

Closeout

  1. Resolve pending actions and refused transitions.
  2. Complete the required validation.
  3. Create the report draft.
bash
   python3 -m ref_all_executor report draft example
  1. Enrich the report.
bash
   python3 -m ref_all_executor report enrich example
  1. Read /api/workbenches/example/closeout/gate.
  2. Resolve each condition or record an operator override receipt.
  3. Request closeout.
bash
   python3 -m ref_all_executor loop closeout example

Read Evidence requirements and API reference for the complete contracts.

FIELD GUIDE / 163 MIN READ

REFALL AB API reference

The backend exposes an HTTP API. Use the configured service URL as the base for each route. Get the running schema from /openapi.json. Read interactive API documentation at /docs.

Subjects and operations

MethodRouteFunction
GET/api/healthRead service health.
POST/api/refall/subjectsCreate a subject workbench.
GET/api/refall/ops/statusRead infrastructure state.
GET/api/refall/ops/agentsRead agent state.
POST/api/refall/ops/commandRun an allowed named command.

Named commands include context, coverage, threat-model, doctor, estimate, start, update, and closeout. Operation commands require the corresponding enable settings. Enabled process launches through Start and Update also require a scoped capability grant. A refused authorization returns HTTP 403 before process release.

Workbench state

The following paths are relative to /api/workbenches/{slug}. Replace {slug} with the subject workbench identifier.

PathStored or returned information
/coverage/contextShared subject briefing.
/coverage/source-inventorySource inventory.
/coverage/scope-overlayScope boundaries.
/coverage/threat-modelThreat model.
/coverage/wishlistResource requests and provided-resource records.
/ledger/entriesAppend and query evidence and decision records.
/ledger/entries/{entry_id}Read one ledger entry.
/ledger/statsCounts grouped by kind, lane, and severity.
/pending-actionsUnresolved obligations.
/runtime/eventsRuntime events.
/runtime/manifestCurrent run manifest.
/runtime/nonfatal-errorsRead-only failure projection.
/runtime/observationsProjected observations.
/runtime/artifactsArtifact records.
/runtime/evidence-graphEvidence relationships.
/capabilitiesCapability requests and decisions.
/execution/authorizeAction authorization.
/closeout/gateCloseout conditions.
/refall-reportPinned current report.

Use the running schema for request bodies and supported methods. The coverage API includes claims, research units, findings, investigations, and statistics.

Capability and operator boundaries

POST /api/workbenches/{slug}/execution/authorize returns and records an authorization decision. It does not execute the requested command. Callers must obey the decision before action release.

Capability grant and revoke authentication activates with REFALL_CAPABILITY_OPERATOR_TOKEN_PATH. Protected requests supply X-REFALL-Operator-Token. The unset default accepts caller-supplied identity fields without token authentication. The closeout override route separately records a supplied operator name, check IDs, reason, and receipt. It does not use the capability-token dependency. Read Control in depth for route responsibilities and deployment authority.

Finding transitions

Submit a transition with POST /api/workbenches/{slug}/findings/{finding_id}/transitions. Read transition history with GET on the same route. Read refused transitions at /api/workbenches/{slug}/disposition/refusals.

A transition identifies the current state, proposed state, reason, receipt type, and supporting evidence. Source hashes can bind the receipt to exact source bytes. Read Evidence for the status and receipt models.

Workbench presentation

Use PUT /api/workbenches/{slug}/theme to replace the workbench theme. The request body contains a raw field with YAML frontmatter and Markdown. Read /api/agent/conventions for the theme format.

Reports

REFALL report routes use /api/refall/reports/. The report module serves the specification and report assembly functions. The pinned workbench report route selects the current report explicitly.

Ledger and wishlist procedures

Use POST /api/workbenches/{slug}/ledger/entries to append a supported entry kind. Use GET on the same route to list records. List filters are kind, lane, severity, and since. The ledger exposes lookup and statistics routes without entry update or delete operations.

Use POST and GET on /api/workbenches/{slug}/coverage/wishlist to record and list wishlist rows. List requests can select status. Delivery records use provided; the watchdog appends that row separately from the original open request. The compatibility coverage service has its own /wishlist and /wishlist_resolve routes. Read Ledger and wishlist for payloads and the distinction between the route families.

FIELD GUIDE / 173 MIN READ

REFALL AB evidence requirements

Evidence model

An observation records what the specialist examined or measured. A lead records a question that requires more work. A finding records a supported security conclusion. A receipt records evidence for a decision or state transition.

Each finding includes the affected component, source anchors, reproduction procedure, impact, likelihood, remediation, and verification state. Evidence-bearing submissions also include confidence and limitations. Use artifact references when source anchors cannot represent the proof.

Structured submissions

Return terminal results in a fenced refall-submit JSON block. The schema name is refall.submit.v1. The submission event is task_result_submit. Use submission_id to identify the result.

The executor validates the payload before it applies coverage updates. Duplicate structured submissions do not create duplicate findings. ref_all_executor/contract_payloads.py supplies payload helpers. Backend DTO validation is in arcticbase/backend/src/arctic_base/refall/contracts.py.

Set REFALL_REQUIRE_STRUCTURED_SUBMIT=true to require valid structured results. A missing or malformed result then creates pending work instead of completed coverage. Legacy terminal markers remain a compatibility format.

Finding status and evidence quality

Status and evidence quality describe different properties. Status describes the research workflow. Quality describes the strength of the proof.

StatusMeaning
PROVISIONALLY_VALIDCandidate finding that requires review.
NEEDS_RESEARCHFinding that requires more examination.
VALIDFinding that passed the required checks.
FALSE_POSITIVERejected finding with a supporting receipt.
DUPLICATEFinding equivalent to a canonical finding.
CHAINABLE_DEFERREDPreserved primitive that requires chain analysis.

Evidence quality uses HYPOTHESIS, CODE_CONFIRMED, and LIVE_CONFIRMED. Static confirmation does not establish runtime exploitability.

Disposition receipts

Submit status changes through the finding transition endpoint. Use typed evidence receipts for rejection, duplicates, deferral, severity reduction, and quality reduction.

Receipt typeEvidence
code_anchorSource or binary location that supports the claim.
live_reproReproduced runtime behavior.
non_reachability_proofRouting or configuration proof that excludes the sink.
duplicate_equivalenceSame root cause, sink, and remediation as the canonical finding.
scope_boundaryRecorded scope or exclusion boundary.
downgrade_with_trigger_inventoryRelevant trigger paths and reason for reduced severity.
chain_preservationPreserved primitive and remaining chain questions.
patch_verificationPatch evidence and verification results.
operator_overrideExplicit operator decision and reason.

A duplicate receipt identifies duplicate_of. Receipts can include source_hashes. The backend refuses a receipt when its source or configuration hashes are stale. Accepted and refused transitions remain in the ledger.

Pending actions and capabilities

Pending actions preserve unfinished obligations. Examples include missing validation, exhausted retries, scope decisions, and operator requests. A direct capability_missing result creates a capability request. A direct scope_blocked result creates a scope_decision action. Neither result completes the research unit.

Capabilities restrict live actions to the operator-approved class and scope. The worker cannot approve its own live actions.

Closeout and reports

The closeout gate checks unresolved pending actions, refused transitions, suppressive finding changes, report freshness, and validation proof. Resolve each condition or record an operator override receipt.

Reports include scope, methods, findings, evidence, limitations, remediation, and verification state. A visual evidence pack can contain screenshots, traces, CodeCharta views, Codebadger output, or exploit-flow diagrams. Link each visual to the claim that it supports.

The ledger can record codebadger-toolchain-not-run when that tool did not run. The publication flow records vendor-cna-report-generated for the corresponding report artifact. Read the runtime report specification in docs/superpowers/specs/2026-07-16-refall-report-format-spec.md.

FIELD GUIDE / 182 MIN READ

REFALL AB analysis toolchain

The toolchain creates source indexes, graphs, metrics, and static analysis results. Optional MCP services give specialist agents access to additional analysis tools. Pinned versions are defined in versions.lock.

Install

  1. Open the toolchain directory.
  2. Run ./install.sh.
  3. Examine the installation result.

The installer creates local Python and Node.js environments. It also installs the configured tool shims and supporting resources.

Analyze source

Run these commands from the repository directory.

  1. Examine the proposed analysis steps.
bash
   toolchain/run.sh --dry-run /path/to/subject toolchain/output/example
  1. Run the analysis.
bash
   toolchain/run.sh /path/to/subject toolchain/output/example
  1. Examine run_manifest.json in the output directory.
  2. Link useful results to the subject workbench.

Tools and outputs

ToolOutput or function
SemgrepRule findings in JSON.
ast-grepStructural source matches.
SCIP Python and TypeScriptSymbol indexes.
JediPython symbols.
code2flow and pyan3Call graphs.
dependency-cruiser and madgeJavaScript dependency graphs.
LizardFunction complexity measurements.
RadonPython complexity and maintainability measurements.
GraphvizSVG and PNG graph images.
CodeChartaCode-city data from complexity measurements.
SerenaSemantic source tools through MCP.
Codebase-MemoryPersistent source graph through MCP.
CodebadgerCode property graphs, data flow, taint analysis, and slicing.

Output directories

PathContents
graphs/Graph data and rendered images.
indexes/Symbol, import, and pattern indexes.
metrics/Complexity and package measurements.
codecharta/CodeCharta .cc.json files.
semgrep/Findings and the local report viewer.
scip/SCIP indexes.
logs/Per-tool output.
run_manifest.jsonStep results and return codes.

Manage optional services

bash
toolchain/ctl.sh status
toolchain/ctl.sh up serena
toolchain/ctl.sh down serena
toolchain/ctl.sh up codebadger-mcp
toolchain/ctl.sh restart python-lsp-server
toolchain/ctl.sh down

ctl.sh up preserves an already healthy service. ctl.sh down first sends TERM to the service process group. It sends KILL if the process does not exit. The registry is config/services.json. Only entries in services[] are active.

Codebadger

Start Codebadger when the subject needs property graphs, taint analysis, or slicing. The installer or first start obtains the pinned Codebadger source. Docker supplies its supporting services.

Place local source under toolchain/third_party/codebadger/playground/subjects/<subject>. Use /app/playground/subjects/<subject> as the corresponding container path. The MCP service binds to loopback on port 4242. Its configuration is available through toolchain/bin/codebadger-service mcp-config.

Agent configuration

MCP configuration examples are in config/. Use serena-mcp-config.json, codebase-memory-mcp-config.json, or codebadger-mcp-config.json for the required service. The service registry also contains disabled extension slots. Copy a slot into services[] only after its command and health check are configured.

FIELD GUIDE / 191 MIN READ

REFALL AB deployment

Docker Compose

Run these commands from the repository directory.

  1. Create the deployment environment file.
bash
   cp deploy/refall-arcticbase.env.example deploy/refall-arcticbase.env
  1. Set the storage paths in deploy/refall-arcticbase.env.
  2. Set the public service URL.
  3. Set the Agent Canvas URL and API key path for the deployment.
  4. Start the workbench container.
bash
   docker compose --env-file deploy/refall-arcticbase.env -f deploy/docker-compose.refall-arcticbase.yml up -d --build
  1. Read the health response at the configured service URL.

The Compose file builds the bundled backend and frontend. It mounts persistent workbench storage and the configured subject directory. Agent Canvas, manager executables, and subject execution resources require separate configuration. Process launch requires those resources in the execution environment.

Host deployment

Read Installation for a native backend process. The launchd example is com.gainpalfam.refall-v10.arcticbase.plist.example. Set each path and environment value for the deployment host before you install the service.

Watchdog

The cron/ directory contains watchdog scheduling examples. The watchdog reads subject progress and can request recovery work. Examine the example command and repository path before you install a schedule.

Storage

Use persistent storage for workbench data and subject artifacts. Read Architecture for the state model. Read Configuration for storage settings.

FIELD GUIDE / 202 MIN READ

REFALL AB troubleshooting

The web interface is unavailable

  1. Read /api/health at the configured service URL.
  2. Examine the backend listen address and port.
  3. Examine ARCTIC_BASE_FRONTEND_DIST.
  4. Build the frontend if that directory is absent.
  5. Examine the backend log for startup failures.

A subject loop does not start

  1. Read the subject context.
  2. Generate context if it is absent.
  3. Run python3 -m ref_all_executor ops command doctor example.
  4. Examine REFALL_ENABLE_OPS and REFALL_ENABLE_OPS_COMMANDS.
  5. Examine the named command allowlist.
  6. Examine the subject launch capability.
  7. Examine the Agent Canvas service and API key path.
  8. Examine the provider executable and subject mount paths.

A stored context record and a valid launch grant are necessary when process launch is enabled.

Progress stops

  1. Examine /api/refall/ops/debug/example/live.
  2. Examine coverage claims and pending actions.
  3. Examine the runtime manifest and nonfatal error projection.
  4. Resolve expired or orphaned work through the operation interface.
  5. Request a bounded loop update.

Recoverable failures use the configured retry limit and delay. Exhausted retries remain visible as pending work.

A submission is refused

  1. Examine the structured submission validation result.
  2. Examine the schema, identity, evidence, confidence, and limitations.
  3. Examine the source anchors or artifact references.
  4. Resubmit the corrected result with the appropriate identity.

A finding transition is refused

  1. Read the transition refusal.
  2. Compare the expected current state with the stored finding.
  3. Examine the receipt type and evidence references.
  4. Compare source hashes with the current files.
  5. Collect the missing proof.
  6. Submit a new transition request.

Closeout is blocked

  1. Read the closeout gate response.
  2. Resolve pending actions and refused transitions.
  3. Complete the required validation.
  4. Prepare the report draft.
  5. Enrich the report.
  6. Request closeout again.

Optional analysis tools fail

  1. Run toolchain/ctl.sh status.
  2. Examine the tool output and run_manifest.json.
  3. Examine the service log in toolchain/logs/.
  4. Run toolchain/install.sh to repair installed dependencies when necessary.

Use the toolchain output directory for generated files. Keep subject evidence in the subject workbench and artifact store.

FIELD GUIDE / 211 MIN READ

REFALL AB development

Install development dependencies

  1. Install the backend development environment.
bash
   uv sync --project arcticbase/backend --group dev
  1. Install the executor dependencies.
bash
   uv pip install --python arcticbase/backend/.venv/bin/python -r coverage_service/requirements.txt

Do the tests

Run these commands from the repository directory.

  1. Do the repository tests.
bash
   PYTHONPATH="$PWD:$PWD/arcticbase/backend/src" arcticbase/backend/.venv/bin/python -m pytest -q
  1. Do the backend tests.
bash
   uv run --project arcticbase/backend pytest arcticbase/backend/tests -q

The repository tests cover coverage storage, work claims, executor clients, routing, submissions, reports, and lifecycle behavior. Documentation checks cover navigation and the operating contract. Tests marked live require the external services named by the test.

Frontend build

bash
pnpm --dir arcticbase/frontend install --frozen-lockfile
pnpm --dir arcticbase/frontend build

Source organization

Use ref_all_executor/ for reusable loop behavior. Use coverage_service/ for compatibility coverage behavior. Use arcticbase/backend/src/arctic_base/ for workbench APIs and storage. Use arcticbase/frontend/src/ for the web interface. Keep subject-specific harnesses and generated research results in subject workspaces.

Documentation terms

Use REFALL AB as the product name. Use ArcticBase only for implementation paths and backend component names. Define specialist terms at first use. Write procedures as short commands with one action in each sentence. Keep descriptive sentences at 25 words or fewer. Keep procedural sentences at 20 words or fewer. The writing standard is ASD-STE100 Issue 9.

FIELD GUIDE / 222 MIN READ

REFALL AB documentation

REFALL coordinates bounded research and preserves its evidence across workers, sessions, and report revisions. Use the guides below to understand the product, operate a subject, and trace its implementation.

Understand the product

GuideContents
Product overviewPurpose, practical uses, capabilities, and the Ready Armor approach.
Ready Armor in REFALLDistinctive research composition, practical value, retained knowledge, and actual acceptance boundaries.
Research cyclesCampaign polling, per-target passes, retries, supervision, and closeout.
Ledger and wishlistEvidence history, requested resources, fulfillment, worker briefings, and APIs.
Feature catalogSubject state, scheduling, evidence capture, findings, authority, continuity, and reports.
End-to-end workflowsThe complete operator-to-closeout path and source or binary research families.
Technical codexSource paths, execution order, state ownership, and runtime mechanisms.
Research architectureControl principles, research memory, evidence boundaries, and product relationships.
Control in depthResource boundaries, brokered release, task guidance, evidence gates, defaults, and exceptions.
Architecture mapComponents, service flow, and persistent stores.
Work modulesPacks, target dependencies, custom passes, selection rules, and examples.

Install and operate

GuideContents
InstallationBackend dependencies, frontend build, storage, and execution setup.
Operating proceduresSubject creation, context, readiness, loop commands, reports, and shutdown.
ConfigurationAddresses, storage, providers, execution, and retries.
Manager proceduresDetailed manager operation and decision requirements.
TroubleshootingDiagnosis, blocked work, recovery, and closeout failures.
DeploymentDocker Compose and native service examples.
ToolchainSource analysis, outputs, optional MCP services, and lifecycle control.

Implement and extend

GuideContents
API referenceSubject, coverage, runtime, evidence, capability, and report routes.
Evidence requirementsSubmission identities, quality tiers, receipts, and closeout proof.
Runtime contractsIndividual runtime control references.
DevelopmentEnvironments, existing tests, source organization, and writing conventions.
Report standardRequired report content and deliverable modes.

Fresh manager startup

Read the files in docs/codex-startup/ in numeric order. The starthere/ directory contains the equivalent startup pack. Read the stored subject context before you assign research work.

Interactive edition

Open the technical atlas to explore the architecture, capabilities, operating traces, complete guides, and implementation excerpts. Use Documentation site for the build and publishing procedure.

Implementation file index

Use the source file index to locate implementation modules and their declared interfaces.

FIELD GUIDE / 232 MIN READ

Documentation site

The technical atlas contains the product guides, capability catalog, architecture, operating traces, and source excerpts. All styles, diagrams, scripts, search data, and document content are embedded in docs/index.html. The page works without a product server or an internet connection. The repository includes the generated HTML and docs/.nojekyll marker.

Open the atlas

Open the technical atlas in a browser. Use the left navigation to select a guide. Use Search to find a feature, procedure, concept, or implementation symbol. Use Control in Depth to examine each layer's boundary, activation, decision scope, and code-path example. Select a source reference to read its code excerpt. Use the operating trace tabs to select a campaign cycle, target pass, or resource feedback map. Select a node to read its behavior and source references. Dashed return arrows identify later cycles. Use the color control to select a light or dark theme. Use the browser print command to print the full technical edition.

Publish with GitHub Pages

  1. Push the repository's main branch to GitHub.
  2. Open the repository's Settings.
  3. Select Pages.
  4. Under Build and deployment, set Source to Deploy from a branch.
  5. Select the main branch.
  6. Select the /docs folder.
  7. Select Save.
  8. After publication completes, select Visit site.

GitHub publishes the checked-in HTML from docs/ when changes reach the selected branch. The .nojekyll marker selects the static publication. The HTML is already present; publishing does not require a local build. Read GitHub's publishing settings for the repository controls.

Update the atlas

Update the Markdown guides when product behavior changes. Update docs/site.json when capabilities, operating traces, or source locations change. Run these commands from the repository directory:

sh
python3 -m unittest discover -s scripts -p test_build_docs.py
python3 scripts/build_docs.py

The builder reads the site profile and referenced Markdown chapters. It refreshes the generated source file index from the configured implementation directories. It embeds the guides, diagrams, search data, and source excerpts in docs/index.html. It also creates docs/.nojekyll and preserves the hand-authored guides and assets beside the publication. The build uses the Python standard library.

Commit the updated guides, profile, generated source index, HTML, and static-site marker. Push those changes to main to update the published site. The builder test checks links, unique anchors, embedded assets, escaped source content, operating traces, and retained source files.

Share one file

Copy docs/index.html to another location or device. Open the copied file in a browser. The guides, diagrams, search, operating traces, and source excerpts remain available in that file.

FIELD GUIDE / 2414 MIN READ

Source file index

This index locates the first-party implementation files behind the product.

It groups files by source directory and lists selected declared interfaces.

The table gives line totals, declaration counts, and selected interface names.

The index contains 149 implementation files.

Python definition counts include module functions, classes, and their declared methods.

Other file types list explicit function and class declarations where available.

The interface column shows up to twelve definitions for each file.

ref_all_executor

FileLinesDefinitionsSelected interfaces
ref_all_executor/__init__.py10Module declarations or executable statements
ref_all_executor/__main__.py60Module declarations or executable statements
ref_all_executor/agent_profiles.py302HeadProfile, head_profile
ref_all_executor/agent_workspace.py2189AgentWorkspaceError, sync_agent_canvas_workspace, _safe_subject_name, _source_snapshot, _read_marker, _existing_source_mount, _marker_matches, _remove_existing_workspace, _run
ref_all_executor/arcticbase_client.py25246ArcticBaseClient, ArcticBaseClient.__init__, ArcticBaseClient._req, ArcticBaseClient._raw, ArcticBaseClient._req_once, ArcticBaseClient.create_subject, ArcticBaseClient.get_workbench, ArcticBaseClient.patch_workbench, ArcticBaseClient.list_subjects, ArcticBaseClient.mirror_subject, ArcticBaseClient.ops_status, ArcticBaseClient.ops_agents; 34 additional definitions
ref_all_executor/artifact_store.py671save_artifact
ref_all_executor/auto_client.py10819_urllib_transport, AutoClient, AutoClient.__init__, AutoClient.api_key, AutoClient._req, AutoClient.cron, AutoClient.event, AutoClient.create_prompt_automation, AutoClient.create_code_automation, AutoClient.upload_tarball, AutoClient.dispatch, AutoClient.list_runs; 7 additional definitions
ref_all_executor/cleanup.py15013RefallCleanup, RefallCleanup.__init__, RefallCleanup.run, RefallCleanup._mark_stop, RefallCleanup._active_cids_path, RefallCleanup._read_active_cids, RefallCleanup._stop_conversations, RefallCleanup._cmd, RefallCleanup._stop_containers, RefallCleanup._pgrep, RefallCleanup._lsof_port, RefallCleanup._parse_pids; 1 additional definitions
ref_all_executor/cli.py2782_parse_process_patterns, main
ref_all_executor/config.py28016_private_lan_ipv4, resolve_public_base, session_api_key, local_workspace, remote_workspace, _ssh_resource_aliases, _build_hosts, ssh_resource_instructions, subject_workspace, subject_agent_workspace, prepare_subject_agent_workspace, agent_workspace_host_path; 4 additional definitions
ref_all_executor/context_fingerprint.py1617fingerprint_directory, _sha256, _file_probe, _needs_binwalk, _binwalk_probe, _canonical, _identity_manifest
ref_all_executor/contract_payloads.py462_now, task_result_submit
ref_all_executor/coverage_client.py19928_urllib_transport, CoverageClient, CoverageClient.__init__, CoverageClient._prefix, CoverageClient._req, CoverageClient.enumerate, CoverageClient.claim_next, CoverageClient.submit, CoverageClient.coverage, CoverageClient.gate, CoverageClient.search, CoverageClient.plan_progress; 16 additional definitions
ref_all_executor/doctor.py21315doctor, _network_checks, _filesystem_checks, _arcticbase_subject_checks, _secret_checks, _allowed_providers, _provider_command, _base_url, _openhands_base, _subject_root, _http_get, _add; 3 additional definitions
ref_all_executor/entrypoint.py996Task, Clients, _workspace_for, _objective, execute_task, run_pass
ref_all_executor/estimate.py925estimate_subject, _estimated_open_units, _missing_prerequisites, _dynamic_testing_state, _optional
ref_all_executor/evidence_graph.py181graph_from_deltas
ref_all_executor/head_orchestrator.py1417HeadCycleResult, RefallHeadOrchestrator, RefallHeadOrchestrator.__init__, RefallHeadOrchestrator.run_cycle, RefallHeadOrchestrator._provided_wishlist, RefallHeadOrchestrator._coverage_context, _objective
ref_all_executor/http_retry.py581with_retry
ref_all_executor/infra.py21019RefallInfra, RefallInfra.__init__, RefallInfra.status, RefallInfra.up, RefallInfra.down, RefallInfra.restart, RefallInfra._cmd, RefallInfra._tmux_available, RefallInfra._tmux_has_session, RefallInfra._tmux_start, RefallInfra._port_pids, RefallInfra._coverage_status; 7 additional definitions
ref_all_executor/loop_manager.py2038RefallLoopManager, RefallLoopManager.__init__, RefallLoopManager.start, RefallLoopManager.update, RefallLoopManager.closeout, RefallLoopManager._record, RefallLoopManager._require_context, launch_standing_loop
ref_all_executor/oh_client.py15615_urllib_transport, OHClient, OHClient.__init__, OHClient.api_key, OHClient.request, OHClient._transport_with_retry, OHClient.start_conversation, OHClient.run, OHClient.execution_status, OHClient.conversation_info, OHClient.conversation_probe, OHClient.wait_until_done; 3 additional definitions
ref_all_executor/projection.py1083project_runtime_events, _observation_from_event, _delta_from_observation
ref_all_executor/prompts.py311build_prompt
ref_all_executor/report_assemble.py33815assemble_report, render_enriched_report, _row, _lead, _fact_section, _runtime_section, _runtime_observation_row, _pending_action_row, _capability_rows, _capability_row, _capability_decision_row, _disposition_transition_row; 3 additional definitions
ref_all_executor/report_draft.py154551_req, _optional_req, draft_report, _sev, _finding_sort_key, _finding_refs, _panel, _exec_summary_section, _fstr, _flist, _finding_articles_section, _architecture_section; 39 additional definitions
ref_all_executor/report_enrich.py34515enrich_findings, _is_garbage, _finding_canonical_key, _related_locations_from_entry, _merge_related_locations, _source_anchor_path, _severity, _normalize_title, _resolve_lead_status, _group_facts_into_sections, _runtime_facts, _visual_artifacts; 3 additional definitions
ref_all_executor/router.py532RouteDecision, route
ref_all_executor/runtime_invocation.py1515InvocationOutcome, classify_invocation, record_retry_failure, _submit_failure, _status_failure_class
ref_all_executor/runtime_state.py18413runtime_root, runtime_db_path, connect_runtime, append_event, list_events, list_artifacts, list_observations, list_deltas, _event_row, _observation_row, _delta_row, _load; 1 additional definitions
ref_all_executor/source_inventory.py1926build_source_inventory, _relative, _file_kind, _line_count, _file_facts, _symlink_observation
ref_all_executor/standing_loop.py4006150StandingLoopConfig, AgentTask, DispatchedConversation, PollResult, ParsedOutput, read_conversation_agent_text, _with_resource_context, _with_workspace_context, AgentCanvasDispatcher, AgentCanvasDispatcher.__init__, AgentCanvasDispatcher.start, AgentCanvasDispatcher.poll; 138 additional definitions
ref_all_executor/structured_submit.py1999SubmitParseResult, SubmitParseResult.valid, backfill_submission_envelope, parse_refall_submissions, primary_submission, _validate, validate_submission_contract, _validate_transition_receipt, coverage_paths_from_submission
ref_all_executor/supervisor.py79633SupervisorConfig, SupervisorConfig.paths, resolve_binary, watchdog_active_agents, _run, loop_process_pids, RefallSupervisor, RefallSupervisor.__init__, RefallSupervisor._state, RefallSupervisor._save_state, RefallSupervisor.hold_active, RefallSupervisor._log; 21 additional definitions
ref_all_executor/usage.py342_summary, record_and_delete
ref_all_executor/verify.py513TaskResult, assign_tier, run_verifier
ref_all_executor/watchdog.py139354Thresholds, WatchdogConfig, WatchdogConfig.paths, WatchdogConfig.thresholds, _env_int, _env_float, _env_bool, thresholds_from_env, iso_now, append_jsonl, append_text, request_json; 42 additional definitions
ref_all_executor/wishlist.py282format_provided_wishlist, provided_wishes_from
ref_all_executor/work_modules.py53531ModuleError, WorkModule, WorkModule.validate, WorkModule.to_dict, WorkModule.from_dict, _m, pack_names, builtin_module, ModuleRegistry, ModuleRegistry.from_config, ModuleRegistry.validate, ModuleRegistry._assert_acyclic; 19 additional definitions
ref_all_executor/workbench.py667_urllib_transport, Workbench, Workbench.__init__, Workbench._post, Workbench.write_finding, Workbench.log_run, Workbench.delete

coverage_service

FileLinesDefinitionsSelected interfaces
coverage_service/__init__.py10Module declarations or executable statements
coverage_service/__main__.py60Module declarations or executable statements
coverage_service/app.py49351_list_subjects, EnumReq, ClaimReq, SubmitReq, enumerate_ep, SurfaceReq, surface_ep, claim_ep, submit_ep, release_ep, dead_ep, FindingReq; 39 additional definitions
coverage_service/binary.py1676_r2, classify, dangerous_present, import_callers, build_investigations, seed_binary
coverage_service/dashboard.py1941render
coverage_service/db.py1153connect, _migrate, init_schema
coverage_service/enumerate.py462partition_file, build_units
coverage_service/extract.py503total_lines, extract_python, extract_dir
coverage_service/ids.py61unit_id
coverage_service/project.py1326_state_path, _get, _h, _finding_rows, build_report, publish
coverage_service/seeds.py1675path_penalty, scan_source, scan_dir, summarize, seeds_to_investigations
coverage_service/store.py49340_now, _decode, Store, Store.__init__, Store.upsert_units, Store.set_surface, Store.claim_next, Store.submit, Store.mark_dead, Store._resolve, Store.release, Store.sweep; 28 additional definitions
coverage_service/tools/__init__.py00Empty module marker
coverage_service/tools/final_visual.py344_content_type, verify_rendered_html, publish_final_visual, publish_coverage_visual
coverage_service/tools/report_findings.py26312_get, _clip, _first_sentence, _repro_sentence, _runtime_viability, _attacker_input, _trace_steps, _trace_html, _poc_pack_html, _card, build, publish
coverage_service/tools/seed_binary.py401main
coverage_service/tools/seed_subject.py411main
coverage_service/tools/vendor_disclosure.py41622_get, _maybe_get, _clip, _public_text, _esc, _first, _quoted_subject, _generated_at, _severity_key, _sentence, _extract_command_block, _trace_nodes; 10 additional definitions

arcticbase/backend/src

FileLinesDefinitionsSelected interfaces
arcticbase/backend/src/arctic_base/__init__.py10Module declarations or executable statements
arcticbase/backend/src/arctic_base/__main__.py241main
arcticbase/backend/src/arctic_base/api/__init__.py490Module declarations or executable statements
arcticbase/backend/src/arctic_base/api/agent.py73616agent_landing, conventions, BootstrapBody, BootstrapChecklistItem, BootstrapResponse, _slugify, bootstrap, SelfTestStep, SelfTestResult, self_test, ActivityResponse, activity; 4 additional definitions
arcticbase/backend/src/arctic_base/api/deps.py101get_storage
arcticbase/backend/src/arctic_base/api/errors.py412_problem, install_error_handlers
arcticbase/backend/src/arctic_base/api/objects.py32013ObjectCreate, ObjectPatch, _read_meta_json, list_objects, create_object, get_object, ObjectReorderBody, reorder_objects, patch_object, delete_object, get_object_content, ContentPutResponse; 1 additional definitions
arcticbase/backend/src/arctic_base/api/refall_capabilities.py20110require_capability_operator, CapabilityGrantReq, CapabilityRequestReq, list_capabilities, request_capability, grant_capability, revoke_capability, revoke_capability_compat, authorize_execution, _request_row
arcticbase/backend/src/arctic_base/api/refall_closeout.py543CloseoutOverrideReq, get_closeout_gate, override_closeout_gate
arcticbase/backend/src/arctic_base/api/refall_coverage.py72151EnumerateReq, SurfaceReq, ClaimReq, SubmitReq, ReleaseReq, PlanReq, ContextReq, FindingReq, FindingValidationReq, WishlistReq, AttackPatternReq, ClassCoverageReq; 39 additional definitions
arcticbase/backend/src/arctic_base/api/refall_disposition.py1115TransitionReq, transition_finding, list_finding_transitions, list_disposition_refusals, _source_root_for_transition
arcticbase/backend/src/arctic_base/api/refall_ledger.py554append_ledger_entry, list_ledger_entries, get_ledger_entry, ledger_stats
arcticbase/backend/src/arctic_base/api/refall_ops.py21912require_ops, infra_checks, status_payload, get_status, get_agents, get_infra, get_subjects, active_refall_agents, _subject_roots, _active_from_loop_state, _active_from_event_log, _read_jsonl_tail
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py201377CommandReq, GenerateContextReq, SaveContextReq, require_ops_commands, run_command, generate_context, save_context, _execute_named_command, _audit, _require_subject_arg, _start_subject, _update_subject; 65 additional definitions
arcticbase/backend/src/arctic_base/api/refall_ops_context.py30215get_debug, get_debug_conversation, get_debug_live, _read_jsonl_tail, _discover_logs, _subject_roots, _agent_log_roots, _event_log_paths, _find_conversation_events, _active_conversations, _active_from_loop_state, _recent_conversation_events; 3 additional definitions
arcticbase/backend/src/arctic_base/api/refall_ops_sse.py544record_event, push_event, feed_history, feed
arcticbase/backend/src/arctic_base/api/refall_pending_actions.py17810PendingActionReq, ClaimReq, ResolveReq, list_pending_actions, create_pending_action, claim_pending_action, resolve_pending_action, defer_pending_action, cancel_pending_action, _finish
arcticbase/backend/src/arctic_base/api/refall_reports.py765spec_index, spec_bundle, spec_file, report_template, current_report
arcticbase/backend/src/arctic_base/api/refall_runtime.py44728RuntimeEventReq, RuntimeArtifactReq, RuntimeManifestReq, RuntimeCheckpointReq, ResumePreflightReq, list_runtime_runs, put_runtime_manifest, get_runtime_manifest, put_runtime_checkpoint, get_runtime_checkpoints, runtime_resume_preflight, append_runtime_event; 16 additional definitions
arcticbase/backend/src/arctic_base/api/refall_subjects.py965SubjectCreate, SubjectCreateResponse, MirrorResponse, create_subject, mirror_workbench
arcticbase/backend/src/arctic_base/api/render.py2625_make_md, build_theme_style_block, build_bridge_script, _inject_into_html, render_object
arcticbase/backend/src/arctic_base/api/responses.py1246ResponseAppend, list_responses, latest_response, latest_payload, export_responses, append_response
arcticbase/backend/src/arctic_base/api/templates_data.py6173TemplateMeta, list_templates, get_template
arcticbase/backend/src/arctic_base/api/theme.py1368ThemeRawBody, get_theme, get_theme_raw, put_theme, list_theme_assets, get_theme_asset, put_theme_asset, delete_theme_asset
arcticbase/backend/src/arctic_base/api/workbenches.py60833WorkbenchCreate, WorkbenchPatch, _slugify, list_workbenches, create_workbench, get_workbench, WorkbenchReorderBody, reorder_workbenches, patch_workbench, ArchiveResponse, archive_workbench, TrashEntry; 21 additional definitions
arcticbase/backend/src/arctic_base/config.py1795_is_private_lan_ipv4, _detect_private_lan_ipv4, Settings, Settings.derive_refall_paths, get_settings
arcticbase/backend/src/arctic_base/logging_setup.py1258ActivityRecord, get_recent_activity, JsonFormatter, JsonFormatter.format, configure_logging, RequestIdMiddleware, RequestIdMiddleware.__init__, RequestIdMiddleware.__call__
arcticbase/backend/src/arctic_base/main.py892_seed_if_empty, create_app
arcticbase/backend/src/arctic_base/models.py18314BannerKind, Banner, WorkbenchMeta, WorkbenchMeta._drop_computed_inputs, WorkbenchMeta.view_url, ObjectKind, ObjectMeta, ResponseKind, Submitter, ResponseEvent, TypographyStyle, LayoutTokens; 2 additional definitions
arcticbase/backend/src/arctic_base/refall/__init__.py10Module declarations or executable statements
arcticbase/backend/src/arctic_base/refall/artifact_store.py671save_artifact
arcticbase/backend/src/arctic_base/refall/capabilities.py112grant, authorize
arcticbase/backend/src/arctic_base/refall/closeout_gate.py43214evaluate_closeout_gate, _accepted_transition_finding_ids, _check, _report_check, _subject_path, _runtime_records, _visual_evidence_available, _toolchain_has_run, _vendor_cna_report, _external_scope_blockers, _informational_checks, _latest_entry_time; 2 additional definitions
arcticbase/backend/src/arctic_base/refall/contracts.py27821stable_id, _utc_now, RefallContract, EvidenceReceipt, EvidenceReceipt.require_material, ContextPackage, ContextPackage.set_id, SourceInventoryPackage, SourceInventoryPackage.set_id, ScopeOverlay, ScopeOverlay.set_id, ThreatModelPackage; 9 additional definitions
arcticbase/backend/src/arctic_base/refall/coverage_db.py1713connect, init_schema, _ensure_column
arcticbase/backend/src/arctic_base/refall/coverage_store.py188195_now, _json, _load_json, _now_id, _receipt_has_evidence, _receipt_stale_reasons, _normalize_sha256, _source_anchor_path, _receipt_summary, _finding_has_visual_or_proof_artifact, _pending_identity, _scope_list; 83 additional definitions
arcticbase/backend/src/arctic_base/refall/disposition.py172transition_finding, list_transitions
arcticbase/backend/src/arctic_base/refall/evidence_graph.py181graph_from_deltas
arcticbase/backend/src/arctic_base/refall/ledger.py1136ledger_path, append_entry, list_entries, get_entry, stats, _read_entries
arcticbase/backend/src/arctic_base/refall/pending_actions.py232create, list_actions
arcticbase/backend/src/arctic_base/refall/projection.py1063project_runtime_events, _observation_from_event, _delta_from_observation
arcticbase/backend/src/arctic_base/refall/promotion_gates.py18214evaluate_finding_candidate, _source_refs, _location_refs, _artifact_refs, _evidence_items, _freeform_refs, _inventory_paths, _source_path, _external_hosts, _url_host, _external_host_allowed, _claims_validation; 2 additional definitions
arcticbase/backend/src/arctic_base/refall/runtime_state.py46523runtime_root, connect_runtime, append_event, list_events, list_nonfatal_errors, list_artifacts, list_observations, list_deltas, put_manifest, get_latest_manifest, put_checkpoint, list_checkpoints; 11 additional definitions
arcticbase/backend/src/arctic_base/refall/source_inventory.py1806build_source_inventory, _relative, _file_kind, _line_count, _file_facts, _symlink_observation
arcticbase/backend/src/arctic_base/refall/subjects.py1936SubjectBootstrap, slugify_subject, bootstrap_subject, _create_subject_layout, _create_or_update_workbench, _merge_tags
arcticbase/backend/src/arctic_base/storage/__init__.py5325Storage, Storage.list_workbenches, Storage.get_workbench, Storage.create_workbench, Storage.update_workbench, Storage.archive_workbench, Storage.restore_workbench, Storage.list_trash, Storage.get_theme, Storage.get_theme_raw, Storage.put_theme, Storage.list_theme_assets; 13 additional definitions
arcticbase/backend/src/arctic_base/storage/errors.py236StorageError, NotFound, AlreadyExists, StaleETag, StaleETag.__init__, InvalidTheme
arcticbase/backend/src/arctic_base/storage/filesystem.py69864_utcnow, _hash, _slug_ok, _atomic_write, _dump_model, _flock, FilesystemStorage, FilesystemStorage.__init__, FilesystemStorage._wb_dir, FilesystemStorage._wb_lock, FilesystemStorage._wb_manifest, FilesystemStorage._theme_md; 52 additional definitions
arcticbase/backend/src/arctic_base/theme.py814parse_theme_md, serialize_theme, _split, _validate_required

arcticbase/frontend/src

FileLinesDefinitionsSelected interfaces
arcticbase/frontend/src/App.svelte1922loadEditor, loadThemeEditor
arcticbase/frontend/src/lib/api.ts6792HttpError, req
arcticbase/frontend/src/lib/bridge.ts1334isIncoming, isRemoteSub, poll, installBridge
arcticbase/frontend/src/lib/chrome/Blueprints.svelte1780Module declarations or executable statements
arcticbase/frontend/src/lib/chrome/Dashboard.svelte6518deriveStats, fmtBytes, fmtAgo, pad, rpad, load, create, buttonLabel
arcticbase/frontend/src/lib/chrome/Editor.svelte1977lang, pickImage, handlePaste, handleDrop, uploadAndInsert, save, reload
arcticbase/frontend/src/lib/chrome/FileBrowser.svelte1213load, upload, fmt
arcticbase/frontend/src/lib/chrome/ImageViewer.svelte550Module declarations or executable statements
arcticbase/frontend/src/lib/chrome/ResponsesHistory.svelte1523load, toggle, fmt
arcticbase/frontend/src/lib/chrome/Settings.svelte94524load, loadActivity, openEdit, createSnapshot, restoreSnapshotPrompt, deleteSnapshotPrompt, runSelfTest, fmtBytes, addCustomField, removeCustomField, closeEdit, saveEdit; 12 additional definitions
arcticbase/frontend/src/lib/chrome/StatusBar.svelte730Module declarations or executable statements
arcticbase/frontend/src/lib/chrome/TopNav.svelte1940Module declarations or executable statements
arcticbase/frontend/src/lib/mini-md.ts452renderMiniMd, inline
arcticbase/frontend/src/lib/refall/OpsTerminal.svelte890Module declarations or executable statements
arcticbase/frontend/src/lib/refall/WorkbenchOpsPanel.svelte140843load, runCommand, runDoctor, runEstimate, generateContext, saveContext, saveScopeOverlay, saveThreatModel, requestThreatModel, projectRuntime, claimPending, finishPending; 31 additional definitions
arcticbase/frontend/src/lib/router.ts383parse, navigate, match
arcticbase/frontend/src/lib/theme.ts171themeToCssVars
arcticbase/frontend/src/lib/themed/HtmlViewer.svelte490Module declarations or executable statements
arcticbase/frontend/src/lib/themed/MdViewer.svelte450Module declarations or executable statements
arcticbase/frontend/src/lib/themed/RunbookViewer.svelte3859load, persist, setStatus, setBlocker, addStep, removeStep, moveStep, statusGlyph, nextStatus
arcticbase/frontend/src/lib/themed/ThemeEditor.svelte1511save
arcticbase/frontend/src/lib/themed/ThemeWrapper.svelte210Module declarations or executable statements
arcticbase/frontend/src/lib/themed/WorkbenchDashboard.svelte54713loadAll, reloadObjects, togglePin, toggleArchive, bannerSrc, clickObject, deleteObject, viewResponses, enableRefallFromWorkbench, setRefallSourcePath, moveWithinDrawer, persistOrder; 1 additional definitions
arcticbase/frontend/src/main.ts180Module declarations or executable statements

toolchain/bin

FileLinesDefinitionsSelected interfaces
toolchain/bin/codebadger-service21014usage, die, require_cmd, env_value, mcp_port, ensure_checkout, ensure_env, ensure_stack, deploy, health, status, down; 2 additional definitions
toolchain/bin/jedi-symbol-index900Module declarations or executable statements
toolchain/bin/lizard-to-codecharta1760Module declarations or executable statements
toolchain/bin/module-import-index750Module declarations or executable statements
toolchain/bin/render-dot-graphs480Module declarations or executable statements

toolchain/install.sh

FileLinesDefinitionsSelected interfaces
toolchain/install.sh38714record, have_python_package, have_npm_package, run_logged, sha256_matches, have_codebase_memory, install_codebase_memory, install_codebadger, apt_update_once, apt_install_package, ensure_deb_package, install_semgrep_report_viewer; 2 additional definitions

toolchain/run.sh

FileLinesDefinitionsSelected interfaces
toolchain/run.sh1563usage, steps, run_step

toolchain/ctl.sh

FileLinesDefinitionsSelected interfaces
toolchain/ctl.sh39224usage, die, require_registry, service_names, autostart_service_names, service_exists, selected_services, service_json, service_field, pid_file, service_log_file, service_port; 12 additional definitions

deploy/cron

FileLinesDefinitionsSelected interfaces
deploy/cron/refall-supervisor.sh450Module declarations or executable statements
deploy/cron/refall-watchdog-cron.sh1021cleanup_tmp

drive_context.py

FileLinesDefinitionsSelected interfaces
drive_context.py652_objective, run

drive_characterize.py

FileLinesDefinitionsSelected interfaces
drive_characterize.py1888log, _objective, _dex, ensure_l1, _run_agent, run, status, kill

drive_research.py

FileLinesDefinitionsSelected interfaces
drive_research.py2014_objective, _run_agent, run, research_status

drive_binresearch.py

FileLinesDefinitionsSelected interfaces
drive_binresearch.py2045_objective, _run_agent, ensure_r2, run, research_status

drive_binreview.py

FileLinesDefinitionsSelected interfaces
drive_binreview.py1444run_validation, _objective, _run_agent, run

drive_binvalidate.py

FileLinesDefinitionsSelected interfaces
drive_binvalidate.py2304needs_validation, _objective, _run_agent, run

refall_infra.py

FileLinesDefinitionsSelected interfaces
refall_infra.py100Module declarations or executable statements

refall_down.py

FileLinesDefinitionsSelected interfaces
refall_down.py100Module declarations or executable statements

Read a source location

  1. Select a file from the relevant subsystem.
  2. Read its imports, declarations, and named interfaces.
  3. Use the architecture and workflow guides to identify its callers and stored records.
  4. Compare an observed result with the relevant decision and state change.
FIELD GUIDE / 254 MIN READ

Ready Armor Personal Use License 1.0

Copyright (c) 2026 GainSec LLC. All rights reserved.

Licensing contact: licensing@gainsecmail.com

1. Scope

GainSec LLC is the Licensor. The Software means the code, documentation, and other original materials provided under this license. This license covers only rights that GainSec LLC can grant. Separately licensed components remain subject to Section 6.

2. Free personal-use permission

You may inspect, download, run, make private copies of, and privately modify the Software solely for Personal Use. These permissions are free of charge, nonexclusive, and nontransferable. They apply to this release while you comply with these terms.

Personal Use means use by an individual, acting solely on their own behalf, for personal study, unpaid experimentation, or noncommercial hobby projects. The use must not form part of work or services performed for an employer, client, business, institution, or other organization. It must not involve payment, fees, rewards, sponsorship, or an anticipated commercial application. Private modifications remain subject to this license.

3. Every other use requires a separate license

You must obtain a separate written license from GainSec LLC before any use outside Section 2. This requirement includes:

  • Business use, including internal operations, training, evaluation, research, or development.
  • Employment-related work, consulting, client assessments, or other professional services.
  • Use by or on behalf of a nonprofit, educational institution, research institution, government, or other organization.
  • Paid, sponsored, reward-based, or revenue-generating activity.
  • Hosting or providing the Software's functionality to other people as a service.
  • Incorporating the Software into a product, service, or offering for others.
  • Distribution, redistribution, sublicensing, sale, rental, or publication of the Software or modified copies, except as Section 5 permits.
  • Any other activity that does not meet every condition of Personal Use.

Using a personal device or working without compensation does not make organizational or professional activity Personal Use. Nonprofit status, an educational purpose, or the absence of revenue does not create an exception.

Request a separate license at licensing@gainsecmail.com. That agreement must expressly authorize the proposed use. Its scope, fees, duration, and other terms are determined separately. A request for permission does not itself grant permission.

4. Copies, notices, and modifications

Keep this license and all applicable copyright, attribution, and license notices in permitted copies and private modifications. Do not remove notices or represent modified versions as official GainSec releases. You may not sublicense the permissions granted by this document. Another person must obtain their own applicable permission.

5. Independent rights and outputs

This license does not restrict rights that applicable law independently gives you. It also does not withdraw repository viewing or forking permissions independently granted through the hosting platform's applicable terms. Those permissions do not authorize additional execution, distribution, or other uses under this license.

This license does not claim ownership of your independent results merely because you produced them with the Software. Producing results for a client, employer, organization, or commercial activity still requires the separate license described in Section 3. Copies of the Software included in an output remain subject to their applicable license.

6. Third-party and separately licensed materials

Third-party dependencies, bundled materials, and components with their own explicit licenses retain those licenses. Their copyright notices and license requirements remain in effect. This document does not replace those terms or limit permissions independently granted under them. It does not withdraw rights already granted for copies supplied under an earlier license.

A separate component's license does not grant rights to other GainSec material covered only by this license. Apply each license to the material it covers.

7. Reserved rights

GainSec LLC retains all rights not expressly granted here. Source availability does not provide additional permissions. No trademark, branding, or endorsement permission is granted. A separate written agreement can grant additional rights for its stated scope.

8. Termination

Your permissions under this license end if you breach its conditions. You must then stop the unauthorized use and remove copies held only under these permissions. GainSec LLC may reinstate permission in writing. Termination does not erase obligations or liability arising before termination. Sections 3 through 9 continue to apply as relevant. Independently granted rights described in Sections 5 and 6 remain governed by their own terms.

9. Warranties and liability

The Software is supplied without warranties or support commitments. To the extent applicable law permits, GainSec LLC disclaims express and implied warranties, including merchantability, fitness for a particular purpose, and noninfringement. You accept the risks of using the Software.

Where the law permits, GainSec LLC and contributors are not liable for loss or damage arising from the Software or this license. This includes lost profits, lost data, business interruption, and direct, indirect, incidental, special, or consequential damage. These exclusions do not apply where applicable law prohibits them.

FIELD GUIDE / 261 MIN READ

MIT License

Copyright (c) 2026 Jon 'GainSec' Gaines, GainSec LLC

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

FIELD GUIDE / 271 MIN READ

Remote SSH resources

Remote resources supply a host operating system, service, device, or execution environment. REFALL_SSH_RESOURCE_ALIASES selects aliases from the configured allowlist. The current allowlist contains sectortw and sectortl. Set an empty value to disable remote resources.

  1. Configure the SSH alias on the execution host.
  2. Verify the host identity and access.
  3. Mount the required SSH configuration in the agent container.
  4. Assign the remote resource to the subject task.
  5. Run the task with ssh <alias> <command>.
  6. Record the alias, command, evidence references, and limitations in the submission.

Grok is an optional worker controlled by the host manager. It is not an SSH resource. Keep host credentials outside its sandbox.

FIELD GUIDE / 281 MIN READ

REFALL AB agent startup

Read the numbered startup documents in order. The first four documents cover preparation, infrastructure, subject lifecycle, and closeout. Documents 04 through 10 contain the manager and specialist runtime instructions. Document 11 describes remote resources.

Read the stored subject context before you assign work. Use /api/refall/ops/status to examine the runtime. Use the repository Documentation index for operating procedures and reference material.

FIELD GUIDE / 291 MIN READ

Runtime contracts

These references describe the implemented runtime contracts.

FIELD GUIDE / 302 MIN READ

REFALL AB Report Deliverable

REFALL closeout deliverables use HTML and persistent workbench objects. The default internal report is self-contained HTML. It supports an internal review archive and later conversion to a disclosure package.

Supported deliverable modes

ModePurpose
internal-genericComplete technical report for the operator or client archive.
vendor-cnaSanitized disclosure report with validated findings.
operator-debriefLoop activity, coverage, gaps, and next actions.
finding-deep-diveEvidence and remediation for one finding.
visual-evidence-packOptional supporting diagrams, screenshots, analysis files, and graph slices.

Required sections

  • Executive summary.
  • Tested scope and exclusions.
  • Methodology.
  • Coverage summary.
  • Evidence ledger summary.
  • Closeout gate state and blocker or warning table.
  • Pending actions, deferred actions, and resolution receipts.
  • Validated findings.
  • Provisional findings.
  • False positives and duplicates.
  • Attack chains.
  • Reproduction and PoC notes.
  • Runtime evidence from projected observations, runtime artifacts, and evidence graph counts.
  • Runtime Warnings from nonfatal runtime and ledger error projections.
  • Capability decisions that affected live examination or validation.
  • Source and configuration freshness receipts, including available source_hashes.
  • capability_missing and scope_blocked outcomes that affected validation limits.
  • Remediation guidance.
  • Visual evidence summary when visuals improve review.
  • Workbench object provenance.

Show closeout gate blockers in internal-generic and operator-debrief deliverables. Include only validated, sanitized findings in vendor-cna reports. Exclude speculative leads, internal capability requests, and local paths unless disclosure is explicitly approved.

Pending actions are report inputs. For each open, done, deferred, or cancelled action, include its status and available source event, finding, or unit. Include its receipt or limitation text.

Runtime evidence can include structured observations, retry failures, validation events, visual artifacts, and evidence graph changes. Use the visual-evidence-pack for useful Codebadger, CodeCharta, codegraph, Graphviz, screenshot, request, response, and flow artifacts.

Runtime Warnings describe nonfatal submission failures, refusals, timeouts, and blocked actions. They also describe capability or scope failures that affected the run. Keep these warnings separate from findings.

FIELD GUIDE / 311 MIN READ

Finding disposition

A finding transition requires a current state, proposed state, reason, and typed receipt. The disposition gate refuses unsupported suppression or stale source hashes. Accepted and refused attempts remain available in the ledger.

Read Evidence requirements and API reference for the operating contract.

FIELD GUIDE / 321 MIN READ

Pending actions

Pending actions preserve unresolved work across agent sessions. The action identifies its kind, source event, priority, owner, status, and required receipt. Open actions prevent closeout until they are resolved or explicitly deferred with evidence.

Read Evidence requirements and API reference for the operating contract.

FIELD GUIDE / 331 MIN READ

Closeout gate

The closeout gate examines pending actions, finding transitions, reports, and validation proof. The gate returns explicit conditions for the manager. Resolve each condition before you request closeout again.

Read Evidence requirements and API reference for the operating contract.

FIELD GUIDE / 341 MIN READ

Structured terminal submissions

Specialists return a fenced refall-submit JSON result. The schema is refall.submit.v1. The event is task_result_submit. The executor validates evidence and applies each submission identity once.

Read Evidence requirements and API reference for the operating contract.

FIELD GUIDE / 351 MIN READ

Runtime events

The runtime database stores events, sessions, artifacts, retries, and control signals. The backend projects observations and evidence relationships into workbench APIs. The runtime manifest describes current resources and progress.

Read Evidence requirements and API reference for the operating contract.

FIELD GUIDE / 361 MIN READ

Runtime retries

Recoverable runtime failures use a bounded retry policy. The default limit is two attempts with a five-second delay. Repair prompts carry a bounded portion of the previous output. Exhausted retries preserve pending work.

Read Evidence requirements and API reference for the operating contract.

FIELD GUIDE / 371 MIN READ

Execution capabilities

The capability broker compares presented action requests with stored operator grants and applicable scope conditions. Requests declare their action class, target, command, and effects. A specialist must not authorize its own live action. Specialists must request operator authority for live actions. Missing capabilities create requests for operator action.

Backend loop launch obeys broker refusal before process release. The general authorization endpoint returns a decision without executing the action. Direct specialist tools do not automatically pass through this route. Grant and revoke authentication requires the configured operator token file. Its unset default retains caller-supplied identity fields. Read Control in depth for the actual enforcement boundaries and defaults.

Read Evidence requirements and API reference for the operating contract.

06 / SOURCE MAP

Read the implementation.

Source locations and excerpts connect the capability atlas and operating traces to the code.

arcticbase/backend/src/arctic_base/api/refall_subjects.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  from datetime import UTC, datetime
    5  from typing import Annotated
    6  
    7  from fastapi import APIRouter, Depends
    8  from pydantic import BaseModel, Field
    9  
   10  from arctic_base.api.deps import get_storage
   11  from arctic_base.config import get_settings
   12  from arctic_base.models import ObjectKind, ObjectMeta
   13  from arctic_base.refall.subjects import bootstrap_subject
   14  from arctic_base.storage.filesystem import FilesystemStorage, new_object_id
   15  
   16  router = APIRouter(prefix="/refall/subjects", tags=["refall-subjects"])
arcticbase/backend/src/arctic_base/refall/coverage_store.py L332CoverageStore
  328      coverage_db.init_schema(conn)
  329      return CoverageStore(conn)
  330  
  331  
  332  class CoverageStore:
  333      def __init__(self, conn: sqlite3.Connection) -> None:
  334          self.conn = conn
  335          self._wlock = threading.Lock()
  336  
  337      def upsert_units(self, units: list[dict]) -> int:
  338          cols = [
  339              "id",
  340              "subject",
  341              "type",
  342              "path",
  343              "name",
  344              "line_start",
  345              "line_end",
  346              "parent_id",
  347              "total_lines",
arcticbase/backend/src/arctic_base/refall/ledger.py L41append_entry
   37      storage.get_workbench(slug)
   38      return storage.root / "workbenches" / slug / "refall_ledger.jsonl"
   39  
   40  
   41  def append_entry(storage: FilesystemStorage, slug: str, entry: dict[str, Any]) -> dict[str, Any]:
   42      path = ledger_path(storage, slug)
   43      now = datetime.now(UTC).isoformat()
   44      data = dict(entry)
   45      data.setdefault("kind", "fact")
   46      if data["kind"] not in LEDGER_KINDS:
   47          raise ValueError(f"unsupported ledger kind: {data['kind']}")
   48      data.setdefault("id", f"led_{uuid.uuid4().hex}")
   49      data.setdefault("workbench_slug", slug)
   50      data.setdefault("created_at", now)
   51      data["updated_at"] = now
   52  
   53      path.parent.mkdir(parents=True, exist_ok=True)
   54      with path.open("a", encoding="utf-8") as fh:
   55          fh.write(json.dumps(data, sort_keys=True) + "\n")
   56      storage.audit(slug, "refall.ledger.append", id=data["id"], kind=data["kind"])
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L978_context_subject
  974          return "warn"
  975      return "ok"
  976  
  977  
  978  def _context_subject(
  979      subject: str,
  980      settings: Settings,
  981      storage: FilesystemStorage,
  982      operator_input: str = "",
  983  ) -> dict:
  984      slug = subject.strip()
  985      if not slug:
  986          raise HTTPException(422, "context requires a subject")
  987      workbench = storage.get_workbench(slug)
  988      store = store_for_workbench(storage, slug)
  989      coverage = store.coverage(slug)
  990      ledger_stats = ledger.stats(storage, slug)
  991      workbench_data = workbench.model_dump(mode="json")
  992      root = _subject_context_root(workbench_data)
  993      materials = _context_materials(
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L1269_context_materials
 1265  ```
 1266  """
 1267  
 1268  
 1269  def _context_materials(
 1270      *,
 1271      storage: FilesystemStorage,
 1272      workbench: dict,
 1273      root: Path,
 1274  ) -> str:
 1275      sections = [
 1276          "## Workbench Metadata\n"
 1277          + json.dumps(_context_workbench_metadata(workbench), indent=2, sort_keys=True),
 1278      ]
 1279      try:
 1280          scope = store_for_workbench(storage, str(workbench["slug"])).get_scope_overlay(str(workbench["slug"]))
 1281      except Exception:
 1282          scope = None
 1283      if scope:
 1284          sections.append("## Scope Overlay\n" + json.dumps(scope, indent=2, sort_keys=True))
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L1064_run_context_writer
 1060          )
 1061      return root.resolve()
 1062  
 1063  
 1064  def _run_context_writer(
 1065      *,
 1066      settings: Settings,
 1067      subject: str,
 1068      slug: str,
 1069      root: Path,
 1070      materials: str,
 1071      operator_input: str,
 1072      coverage: dict,
 1073      ledger_stats: dict,
 1074  ) -> dict:
 1075      provider = settings.refall_head_provider.strip().lower()
 1076      allowed = {
 1077          item.strip().lower()
 1078          for item in settings.refall_allowed_head_providers.split(",")
 1079          if item.strip()
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L1419_save_context_artifacts
 1415      lower = name.lower()
 1416      return lower.startswith(".") or lower in {item.lower() for item in SKIP_CONTEXT_NAMES}
 1417  
 1418  
 1419  def _save_context_artifacts(
 1420      *,
 1421      storage: FilesystemStorage,
 1422      slug: str,
 1423      info_md: str,
 1424      generated_by: str,
 1425      root_container: str | None,
 1426  ) -> tuple[dict, dict]:
 1427      context = store_for_workbench(storage, slug).set_context(
 1428          slug,
 1429          info_md,
 1430          generated_by,
 1431          root_container,
 1432      )
 1433      workbench = storage.get_workbench(slug)
 1434      subject_dir = _subject_context_dir(workbench.model_dump(mode="json"))
ref_all_executor/doctor.py L26doctor
   22      "GROK_API_KEY",
   23  )
   24  
   25  
   26  def doctor(
   27      *,
   28      subject: str | None = None,
   29      check_network: bool = True,
   30      arcticbase=None,
   31      repo_root: Path | None = None,
   32  ) -> dict:
   33      root = repo_root or Path(__file__).resolve().parents[1]
   34      checks: list[dict] = []
   35      _add(checks, "arcticbase_base", "ok", f"base URL configured: {_base_url()}")
   36      _add(checks, "agent_canvas", "ok", f"Agent Canvas URL configured: {_openhands_base()}")
   37      if check_network:
   38          _network_checks(checks)
   39      for provider in _allowed_providers():
   40          command = _provider_command(provider)
   41          _add(
ref_all_executor/estimate.py L9estimate_subject
    5  from .arcticbase_client import ArcticBaseClient
    6  from .doctor import doctor
    7  
    8  
    9  def estimate_subject(
   10      subject: str,
   11      *,
   12      arcticbase=None,
   13      check_network: bool = True,
   14  ) -> dict:
   15      client = arcticbase or ArcticBaseClient()
   16      coverage = _optional(lambda: client.coverage_stats(subject), {})
   17      inventory = _optional(lambda: client.get_source_inventory(subject), {})
   18      pending = _optional(lambda: client.list_pending_actions(subject, include_terminal=False), [])
   19      funnel = _optional(lambda: client.coverage_funnel(subject), {})
   20      capabilities = _optional(lambda: client.capabilities(subject), {"grants": [], "requests": []})
   21      closeout_gate = _optional(lambda: client.closeout_gate(subject), {})
   22      readiness = doctor(subject=subject, check_network=check_network, arcticbase=client)
   23      files = inventory.get("files") if isinstance(inventory, dict) and isinstance(inventory.get("files"), list) else []
   24      lines_total = int(coverage.get("lines_total") or sum(int(item.get("total_lines") or 0) for item in files))
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L700_doctor_payload
  696          return {"status": doctor_payload["status"], "command": "doctor", "subject": subject, "doctor": doctor_payload, "ledger_id": entry["id"]}
  697      return {"status": doctor_payload["status"], "command": "doctor", "doctor": doctor_payload}
  698  
  699  
  700  def _doctor_payload(settings: Settings, storage: FilesystemStorage, subject: str | None = None) -> dict:
  701      checks: list[dict] = []
  702      _doctor_check(checks, "arcticbase_ops", "ok", "ArcticBase REFALL Ops is enabled")
  703      _doctor_check(
  704          checks,
  705          "agent_canvas",
  706          "ok" if settings.refall_openhands_base else "warn",
  707          "Agent Canvas URL configured" if settings.refall_openhands_base else "Agent Canvas URL missing",
  708          url=settings.refall_openhands_base,
  709      )
  710      _doctor_check(
  711          checks,
  712          "public_base_url",
  713          "ok" if settings.refall_public_base_url else "warn",
  714          "public ArcticBase URL configured" if settings.refall_public_base_url else "public ArcticBase URL missing",
  715          url=settings.refall_public_base_url,
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L738_estimate_subject
  734          _subject_readiness_checks(checks, settings, storage, subject)
  735      return {"status": _aggregate_doctor_status(checks), "subject": subject, "checks": checks}
  736  
  737  
  738  def _estimate_subject(subject: str, settings: Settings, storage: FilesystemStorage) -> dict:
  739      storage.get_workbench(subject)
  740      store = store_for_workbench(storage, subject)
  741      coverage = store.coverage(subject)
  742      inventory = store.get_latest_source_inventory(subject) or {}
  743      pending = store.list_pending_actions(subject, include_terminal=False)
  744      funnel = store.funnel(subject)
  745      capabilities = {
  746          "grants": store.list_capability_grants(subject),
  747          "requests": _capability_requests(store, subject),
  748      }
  749      closeout_gate = evaluate_closeout_gate(storage, subject)
  750      doctor_payload = _doctor_payload(settings, storage, subject)
  751      files = inventory.get("files") if isinstance(inventory.get("files"), list) else []
  752      lines_total = int(coverage.get("lines_total") or sum(int(item.get("total_lines") or 0) for item in files))
  753      resolved_pct = float(coverage.get("resolved_pct") or 0)
ref_all_executor/source_inventory.py L66build_source_inventory
   62      "gemfile": "ruby",
   63  }
   64  
   65  
   66  def build_source_inventory(subject: str, root: Path | str) -> dict[str, Any]:
   67      root_path = Path(root).expanduser().resolve()
   68      if not root_path.is_dir():
   69          raise ValueError(f"source root is not a directory: {root_path}")
   70      files: list[dict[str, Any]] = []
   71      skipped: list[dict[str, Any]] = []
   72      observations: list[dict[str, Any]] = []
   73      for dirpath, dirnames, filenames in os.walk(root_path, followlinks=False):
   74          dirnames[:] = sorted(
   75              name for name in dirnames
   76              if name not in SKIP_DIRS and (not name.startswith(".") or name in INCLUDE_DOT_DIRS)
   77          )
   78          for filename in sorted(filenames):
   79              path = Path(dirpath) / filename
   80              rel = _relative(root_path, path)
   81              if path.is_symlink():
ref_all_executor/standing_loop.py L1654RefallStandingLoop._auto_enumerate_if_empty
 1650              )
 1651              return "delete_failed"
 1652          return "deleted"
 1653  
 1654      def _auto_enumerate_if_empty(self) -> int:
 1655          stats = self._safe_coverage()
 1656          if stats.get("lines_total", 0):
 1657              return 0
 1658          root = self._source_root()
 1659          if not root or not root.is_dir():
 1660              return 0
 1661          inventory = build_source_inventory(self.workbench, root)
 1662          self.source_inventory = inventory
 1663          if hasattr(self.coverage, "put_source_inventory"):
 1664              try:
 1665                  self.coverage.put_source_inventory(self.workbench, inventory)
 1666              except Exception:
 1667                  pass
 1668          files = [
 1669              {
ref_all_executor/work_modules.py L46WorkModule
   42      """A module definition is not usable."""
   43  
   44  
   45  @dataclass
   46  class WorkModule:
   47      """One slot-in unit of work generation."""
   48  
   49      id: str
   50      task_class: str
   51      source: str = SOURCE_UNITS
   52      # Per-target ordering. NOT a global phase: "run me on a target only after
   53      # these modules have completed on that same target".
   54      after: list[str] = field(default_factory=list)
   55      # Coverage unit state a target must be in (None = any).
   56      unit_state: str | None = None
   57      # Only consider targets whose path matches one of these globs (None = all).
   58      path_globs: list[str] | None = None
   59      # Skip targets whose path matches any of these globs, even if path_globs
   60      # matched. Used to keep deep/expensive modules off bulk media and metadata-
   61      # only corpora (HAF/GPS/image/voice), which should be characterized but not
ref_all_executor/work_modules.py L333eligible_modules
  329  
  330  # --------------------------------------------------------------- selection
  331  
  332  
  333  def eligible_modules(
  334      registry: ModuleRegistry,
  335      *,
  336      target_kind: str,
  337      completed: set[str],
  338      unit_state: str | None = None,
  339      path: str | None = None,
  340      in_flight: dict[str, int] | None = None,
  341      target: str | None = None,
  342      in_flight_targets: set[tuple[str, str]] | None = None,
  343  ) -> list[WorkModule]:
  344      """Modules that may run on one target right now, best first.
  345  
  346      `completed` is the set of module ids already done **for this target**, which
  347      is what makes ordering local rather than a global phase.
  348  
ref_all_executor/work_modules.py L386ModuleScheduler
  382  
  383  SUBJECT_TARGET = "__subject__"
  384  
  385  
  386  class ModuleScheduler:
  387      """Decides the next module/target to dispatch, and tracks what is done.
  388  
  389      Progress is per-target: `completed[target] = {module_ids}`. That is what
  390      makes ordering local (a module's `after` is satisfied per target) rather than
  391      a global phase. State persists so a loop restart resumes where it left off.
  392  
  393      The loop feeds the scheduler facts (a unit was characterized, a finding
  394      exists) and asks for the next thing to run. The scheduler never does I/O
  395      except reading/writing its own JSON state.
  396      """
  397  
  398      def __init__(self, registry: ModuleRegistry, state: dict[str, Any] | None = None) -> None:
  399          self.registry = registry
  400          state = state or {}
  401          # target_key -> list[module_id] completed on that target
ref_all_executor/work_modules.py L522load_subject_config
  518          )
  519      return "\n".join(lines)
  520  
  521  
  522  def load_subject_config(workbench: dict[str, Any] | None) -> dict[str, Any]:
  523      """Read module config off a workbench's custom fields."""
  524      custom = (workbench or {}).get("custom_fields") or {}
  525      raw = custom.get("refall_modules")
  526      if not raw:
  527          return {}
  528      if isinstance(raw, str):
  529          try:
  530              return json.loads(raw)
  531          except json.JSONDecodeError as exc:
  532              raise ModuleError(f"refall_modules is not valid JSON: {exc}") from exc
  533      if isinstance(raw, dict):
  534          return raw
  535      raise ModuleError("refall_modules must be an object or JSON string")
ref_all_executor/work_modules.py L217ModuleRegistry.from_config
  213  
  214      modules: list[WorkModule] = field(default_factory=list)
  215  
  216      @classmethod
  217      def from_config(cls, config: dict[str, Any] | None) -> "ModuleRegistry":
  218          """Build from a subject's module config.
  219  
  220          Shape (all optional)::
  221  
  222              {
  223                "packs": ["baseline", "security_assessment"],
  224                "modules": [ {...custom subloop module...} ],
  225                "disabled": ["sec_verify"],
  226                "overrides": {"characterize": {"priority": 5}}
  227              }
  228  
  229          No config at all yields the baseline pack, which is exactly the loop's
  230          historical characterize-only behaviour.
  231          """
  232          config = config or {}
ref_all_executor/work_modules.py L301ModuleRegistry._assert_acyclic
  297                          f"module {module.id!r} depends on {dep!r}, which is not enabled"
  298                      )
  299          self._assert_acyclic()
  300  
  301      def _assert_acyclic(self) -> None:
  302          by_id = {m.id: m for m in self.modules}
  303          state: dict[str, int] = {}
  304  
  305          def visit(node: str, trail: list[str]) -> None:
  306              if state.get(node) == 2:
  307                  return
  308              if state.get(node) == 1:
  309                  cycle = " -> ".join(trail + [node])
  310                  raise ModuleError(f"module dependency cycle: {cycle}")
  311              state[node] = 1
  312              for dep in by_id[node].after:
  313                  visit(dep, trail + [node])
  314              state[node] = 2
  315  
  316          for module in self.modules:
ref_all_executor/standing_loop.py L590RefallStandingLoop._claim_next_task
  586              "enumerated": enumerated,
  587              "state": final_state,
  588          }
  589  
  590      def _claim_next_task(self, active: list[DispatchedConversation] | None = None) -> AgentTask | None:
  591          # 1. Operator/manager pending actions always preempt.
  592          pending = self._claim_pending_action()
  593          if pending:
  594              if pending.get("kind") == "threat_model_request":
  595                  return self._task_for_threat_model(action=pending, active=active)
  596              return self._task_for_pending_action(pending, active=active)
  597          threat_model = self._claim_threat_model_task(active=active)
  598          if threat_model:
  599              return threat_model
  600  
  601          active = active or []
  602          in_flight = self._module_in_flight(active)
  603          in_flight_targets = self._module_in_flight_targets(active)
  604  
  605          # 2. Subject-scoped modules (identify, research, toolchain, synthesis):
ref_all_executor/router.py L49route
   45      "triage_trivial":("C", False, Tier.T3, "genuinely trivial throwaway (dedup/one-line summary) — Spark only"),
   46  }
   47  
   48  
   49  def route(task_class: str) -> RouteDecision:
   50      if task_class not in _TABLE:
   51          raise ValueError(f"unknown task_class: {task_class!r}")
   52      lane, needs_verify, floor, reason = _TABLE[task_class]
   53      return RouteDecision(lane=lane, needs_verify=needs_verify, tier_floor=floor, reason=reason)
ref_all_executor/config.py L1
    1  import hashlib
    2  import os
    3  import re
    4  import socket
    5  from pathlib import Path
    6  
    7  REFALL_WORKSPACE = os.environ.get("REFALL_WORKSPACE", "/projects/REFALL-V10")
    8  SUBJECT_WORKSPACE_ROOT = os.environ.get("REFALL_SUBJECT_WORKSPACE_ROOT", "/projects")
    9  AGENT_WORKSPACE_CONTAINER_ROOT_DEFAULT = "/projects/refallsub"
   10  AGENT_WORKSPACE_HOST_ROOT_DEFAULT = "~/refallsub"
   11  
   12  OH_BASE = os.environ.get("REFALL_OPENHANDS_BASE", "http://localhost:8000")
   13  AUTO_BASE = OH_BASE + "/api/automation/v1"
   14  def _private_lan_ipv4():
   15      try:
   16          with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
ref_all_executor/standing_loop.py L379RefallStandingLoop._next_dispatch_lane
  375          for lane in self.dispatch_lane_pattern:
  376              targets[lane] = targets.get(lane, 0) + 1
  377          return targets
  378  
  379      def _next_dispatch_lane(
  380          self,
  381          default_lane: str,
  382          active: list[DispatchedConversation] | None = None,
  383      ) -> str:
  384          if not self.dispatch_lane_pattern:
  385              return default_lane
  386          if active:
  387              targets = self._lane_pattern_targets()
  388              current: dict[str, int] = {}
  389              for conversation in active:
  390                  if conversation.task_class == "report":
  391                      continue
  392                  current[conversation.lane] = current.get(conversation.lane, 0) + 1
  393              deficits = [
  394                  (targets[lane] - current.get(lane, 0), index, lane)
ref_all_executor/standing_loop.py L131read_conversation_agent_text
  127      capability_requests: list[dict] = field(default_factory=list)
  128      threat_models: list[dict] = field(default_factory=list)
  129  
  130  
  131  def read_conversation_agent_text(cid: str, conversations_dir: str | None = None) -> str:
  132      """All text the agent produced across the whole conversation.
  133  
  134      Reads the Agent Canvas event stream from disk (bind-mounted at
  135      <conversations>/<cid-without-dashes>/events/event-*.json) and concatenates
  136      every agent-authored message and action text, in order. This is the
  137      authoritative source of an agent's [FACT]/[FLOW]/[LEAD]/```finding /
  138      ```refall-submit output -- the `agent_final_response` endpoint returns empty
  139      for ACP agents, which end on a tool call rather than a final message.
  140  
  141      Parsing EVERY agent response (not just a "final" one) means an agent that
  142      emits intel across several turns is fully captured, and one that emits a
  143      single comprehensive message is captured too.
  144      """
  145      root = Path(conversations_dir or config.OPENHANDS_CONVERSATIONS_DIR)
  146      events = next(
ref_all_executor/standing_loop.py L1115RefallStandingLoop._incremental_ingest
 1111              return None
 1112          return {"facts": len(new.facts), "leads": len(new.leads),
 1113                  "findings": len(new.findings), "flows": len(new.flows)}
 1114  
 1115      def _incremental_ingest(self, conversation: DispatchedConversation) -> None:
 1116          """Pull intel a still-running agent has already emitted, in near real time.
 1117  
 1118          Two live sources: the agent's conversation event stream (every tagged
 1119          response so far) and any files it has saved to its workspace. Both are
 1120          parsed and de-duplicated, so intel appears as it is produced rather than
 1121          only at completion.
 1122          """
 1123          fresh = ParsedOutput()
 1124          stream_text = read_conversation_agent_text(conversation.cid)
 1125          if stream_text.strip():
 1126              stream = parse_agent_output(stream_text, self._submission_defaults(conversation))
 1127              fresh.facts.extend(stream.facts)
 1128              fresh.leads.extend(stream.leads)
 1129              fresh.findings.extend(stream.findings)
 1130              fresh.flows.extend(stream.flows)
ref_all_executor/standing_loop.py L965RefallStandingLoop._merge_submission_file
  961                  "workspace_ingest_error",
  962                  {"subject": self.subject, "cid": conversation.cid, "error": str(exc)[:300]},
  963              )
  964  
  965      def _merge_submission_file(self, path: Path, parsed: "ParsedOutput") -> None:
  966          try:
  967              data = json.loads(path.read_text(errors="replace"))
  968          except Exception:
  969              return
  970          blocks = data if isinstance(data, list) else [data]
  971  
  972          def merge_marker(text: str) -> None:
  973              marker = parse_agent_output(text)
  974              for source, target in (
  975                  (marker.findings, parsed.findings),
  976                  (marker.facts, parsed.facts),
  977                  (marker.leads, parsed.leads),
  978                  (marker.flows, parsed.flows),
  979              ):
  980                  for item in source:
ref_all_executor/standing_loop.py L1010RefallStandingLoop._register_workspace_artifact
 1006                      parsed.flows.append({**flow, "kind": "flow"})
 1007                  elif isinstance(flow, str):
 1008                      merge_marker(flow)
 1009  
 1010      def _register_workspace_artifact(self, conversation: DispatchedConversation, path: Path) -> None:
 1011          if not hasattr(self.arcticbase, "create_runtime_artifact"):
 1012              return
 1013          marker_dir = path.parent / ".refall-ingested"
 1014          marker = marker_dir / path.name
 1015          if marker.exists():
 1016              return  # already ingested on a prior cycle
 1017          try:
 1018              content = path.read_text(errors="replace")
 1019          except Exception:
 1020              return
 1021          try:
 1022              self.arcticbase.create_runtime_artifact(
 1023                  self.workbench,
 1024                  {
 1025                      "kind": "agent-analysis",
ref_all_executor/standing_loop.py L1378RefallStandingLoop._finish_conversation
 1374              "type task_result_submit with status done and concrete source_anchors or artifact_refs when "
 1375              "resolved, or type control_submit if operator input or a capability is still required.",
 1376          ] if part)
 1377  
 1378      def _finish_conversation(
 1379          self,
 1380          conversation: DispatchedConversation,
 1381          result: PollResult,
 1382      ) -> DispatchedConversation | None:
 1383          outcome = classify_invocation(
 1384              result.status,
 1385              result.response or "",
 1386              require_structured_submit=_require_structured_submit(),
 1387              attempt=conversation.attempt,
 1388              max_attempts=_retry_max_attempts(),
 1389          )
 1390          # Parse the agent output once. A duplicate structured submission (same
 1391          # submission_id re-applied on a retry) must not re-persist, so that guard
 1392          # runs BEFORE salvage. Otherwise, salvage intel from EVERY invocation --
 1393          # an agent stopped at max_iterations, or one that ended without a clean
ref_all_executor/standing_loop.py L1038RefallStandingLoop._salvage_intel
 1034              marker.write_text("1")
 1035          except Exception:
 1036              pass
 1037  
 1038      def _salvage_intel(
 1039          self, conversation: DispatchedConversation, parsed: "ParsedOutput", status: str
 1040      ) -> dict | None:
 1041          """Persist ALL intel an agent produced, on every outcome. This is the
 1042          single place facts/leads/findings/flows are written, so a stopped or
 1043          malformed agent still contributes its evidence instead of having it
 1044          discarded. Also pulls in deliverables the agent saved to its workspace,
 1045          since agents commonly save findings to files rather than emitting them.
 1046          """
 1047          self._ingest_workspace_output(conversation, parsed)
 1048          counts = self._persist_new_intel(conversation, parsed)
 1049          if counts and sum(counts.values()):
 1050              self._append_event("intel_salvaged", {"subject": self.subject, "cid": conversation.cid, "status": status, **counts})
 1051          return counts
 1052  
 1053      def _intel_key(self, conversation: DispatchedConversation, kind: str, item: dict) -> tuple:
ref_all_executor/standing_loop.py L1161RefallStandingLoop._block_empty_subject_module_completion
 1157                  "module_completion_error",
 1158                  {"subject": self.subject, "cid": conversation.cid, "error": str(exc)[:300]},
 1159              )
 1160  
 1161      def _block_empty_subject_module_completion(
 1162          self,
 1163          conversation: DispatchedConversation,
 1164          parsed: "ParsedOutput",
 1165          persistence_failed: bool = False,
 1166      ) -> bool:
 1167          """Keep subject-level custom work pending when no canonical intel landed.
 1168  
 1169          Unit modules can legitimately advance coverage through their determination
 1170          payload. Subject modules have no such independent output, so completing one
 1171          with zero facts, leads, findings, or flows would silently discard its work.
 1172          """
 1173          if not (
 1174              conversation.module_id
 1175              and conversation.module_target
 1176              and conversation.unit is None
ref_all_executor/structured_submit.py L142validate_submission_contract
  138      errors.extend(validate_submission_contract(payload))
  139      return errors
  140  
  141  
  142  def validate_submission_contract(payload: dict[str, Any]) -> list[str]:
  143      errors: list[str] = []
  144      submit_type = payload.get("type")
  145      evidence_required = submit_type in {"task_result_submit", "finding_submit", "evidence_delta_submit"}
  146      if not evidence_required:
  147          return errors
  148      if not (payload.get("source_anchors") or payload.get("artifact_refs")):
  149          errors.append("evidence-bearing submission requires source_anchors or artifact_refs")
  150      if submit_type in {"task_result_submit", "finding_submit"}:
  151          parent_evidence = payload.get("evidence") or []
  152          parent_source_anchors = payload.get("source_anchors") or []
  153          parent_artifact_refs = payload.get("artifact_refs") or []
  154          for finding in payload.get("findings") or []:
  155              if not isinstance(finding, dict):
  156                  continue
  157              if not (finding.get("code_paths") or finding.get("source_anchors") or finding.get("artifact_refs")):
ref_all_executor/standing_loop.py L3080parse_agent_output
 3076                  },
 3077              )
 3078  
 3079  
 3080  def parse_agent_output(response: str, defaults: dict | None = None) -> ParsedOutput:
 3081      parsed = ParsedOutput()
 3082      structured = parse_refall_submissions(response, defaults)
 3083      parsed.structured = structured.submissions
 3084      parsed.structured_errors = structured.errors
 3085      parsed.structured_missing = structured.missing
 3086      if structured.valid:
 3087          for submission in structured.submissions:
 3088              parsed.pending_actions.extend(_pending_actions_from_submission(submission))
 3089              for transition in _transitions_from_submission(submission):
 3090                  parsed.transitions.append(transition)
 3091              if submission.get("type") == "task_result_submit":
 3092                  if _structured_status_is_done(submission):
 3093                      parsed.done = str(submission.get("summary") or "")
 3094                  else:
 3095                      parsed.pending_actions.append(_pending_action_from_blocked_submission(submission))
ref_all_executor/standing_loop.py L1222RefallStandingLoop._output_contract
 1218              },
 1219          )
 1220          return True
 1221  
 1222      def _output_contract(self) -> str:
 1223          if self._suppress_findings_for_subject():
 1224              return (
 1225                  "## Output Contract\n"
 1226                  "Always compile evidence as you work: emit facts, leads, and flows, including "
 1227                  "negative results. Do NOT emit ```finding blocks, finding_submit payloads, "
 1228                  "or finding-promotion requests for this subject. Bug-like or security-relevant "
 1229                  "observations must be preserved as [LEAD] and [FACT] mapping evidence with "
 1230                  "source anchors.\n"
 1231                  "Prefer one terminal ```refall-submit JSON block using schema refall.submit.v1.\n"
 1232                  "REQUIRED envelope fields (omitting them will NOT lose your intel, but your task "
 1233                  "cannot be marked complete without them): `schema`, `type` (task_result_submit), "
 1234                  "`subject`, `task_id` (or `unit_id`), `status`, `summary`, `produced_by`, "
 1235                  "`created_at`, `evidence`, `confidence`, `limitations`, and `source_anchors` or "
 1236                  "`artifact_refs`.\n\n"
 1237                  "Source anchors MUST use source-inventory-relative paths rooted at the subject "
arcticbase/backend/src/arctic_base/refall/promotion_gates.py L12evaluate_finding_candidate
    8  VALIDATED_QUALITIES = {"CODE_CONFIRMED", "LIVE_CONFIRMED"}
    9  VALIDATED_REPRO = {"reproduced", "confirmed", "live_reproduced"}
   10  
   11  
   12  def evaluate_finding_candidate(
   13      candidate: dict,
   14      *,
   15      source_inventory: dict | None = None,
   16      scope_overlay: dict | None = None,
   17  ) -> dict:
   18      reasons: list[str] = []
   19      warnings: list[str] = []
   20      refs = _source_refs(candidate)
   21      artifacts = _artifact_refs(candidate)
   22      evidence = _evidence_items(candidate)
   23  
   24      if not evidence and not refs and not artifacts:
   25          reasons.append("missing evidence, source anchors, or artifact refs")
   26      elif not evidence:
   27          reasons.append("missing evidence")
arcticbase/backend/src/arctic_base/refall/coverage_store.py L749CoverageStore.add_finding
  745          if not row:
  746              return None
  747          return _load_json(row["payload"], {})
  748  
  749      def add_finding(self, subject: str, data: dict) -> dict:
  750          now = _now()
  751          finding_id = str(data.get("id") or f"find_{uuid.uuid4().hex}")
  752          code_paths = data.get("code_paths") or []
  753          status = str(data.get("status") or "PROVISIONALLY_VALID")
  754          quality_tier = str(data.get("quality_tier") or "HYPOTHESIS")
  755          if not data.get("title"):
  756              raise ValueError("finding requires title")
  757          if not code_paths:
  758              raise ValueError("finding requires code_paths")
  759          if status not in FINDING_STATUSES:
  760              raise ValueError(f"bad status {status}")
  761          if status not in CREATABLE_FINDING_STATUSES:
  762              raise ValueError(f"status {status} requires disposition transition")
  763          if quality_tier not in QUALITY_ORDER:
  764              raise ValueError(f"bad quality_tier {quality_tier}")
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1008CoverageStore.transition_finding
 1004          if updated is None:
 1005              raise ValueError("no such finding")
 1006          return updated
 1007  
 1008      def transition_finding(self, subject: str, finding_id: str, data: dict) -> dict:
 1009          current = self.get_finding(finding_id)
 1010          if current is None or current.get("subject") != subject:
 1011              raise ValueError("no such finding")
 1012          from_status = str(data.get("from_status") or current.get("status") or "")
 1013          to_status = str(data.get("to_status") or current.get("status") or "")
 1014          from_quality = str(data.get("from_quality") or current.get("quality_tier") or "HYPOTHESIS")
 1015          to_quality = str(data.get("to_quality") or from_quality)
 1016          receipt_type = data.get("receipt_type")
 1017          receipt = data.get("receipt") or {}
 1018          duplicate_of = data.get("duplicate_of")
 1019          reasons: list[str] = []
 1020  
 1021          if from_status and from_status != current.get("status"):
 1022              reasons.append("from_status does not match current state")
 1023          if from_status and from_status not in FINDING_STATUSES:
arcticbase/backend/src/arctic_base/refall/coverage_store.py L112_receipt_stale_reasons
  108              return True
  109      return False
  110  
  111  
  112  def _receipt_stale_reasons(receipt: object, source_root: object) -> list[str]:
  113      if not isinstance(receipt, dict):
  114          return []
  115      hashes: dict[str, object] = {}
  116      for key in ("source_hashes", "source_anchor_hashes", "file_hashes", "config_hashes"):
  117          value = receipt.get(key)
  118          if isinstance(value, dict):
  119              hashes.update(value)
  120      if not hashes:
  121          return []
  122      if not source_root:
  123          return ["stale evidence check requires recorded source root"]
  124      try:
  125          root = Path(str(source_root)).expanduser().resolve()
  126      except OSError:
  127          return ["stale evidence check source root is not resolvable"]
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1458CoverageStore.authorize_execution
 1454              )
 1455              self.conn.commit()
 1456          return self.get_capability_grant(grant_id) or grant
 1457  
 1458      def authorize_execution(self, subject: str, request: dict) -> dict:
 1459          capability_class = str(request.get("capability_class") or request.get("class") or "")
 1460          declared_effects = request.get("declared_effects") or []
 1461          reasons: list[str] = []
 1462          if not capability_class:
 1463              reasons.append("missing capability_class")
 1464          if not declared_effects:
 1465              reasons.append("request omits declared effects")
 1466          grants = [
 1467              grant for grant in self.list_capability_grants(subject)
 1468              if grant.get("class") == capability_class and not _grant_expired(grant)
 1469          ]
 1470          matched: dict | None = None
 1471          for grant in grants:
 1472              grant_reasons = _grant_refusal_reasons(grant, request)
 1473              if not grant_reasons:
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1764_grant_refusal_reasons
 1760      expires_at = grant.get("expires_at")
 1761      return expires_at is not None and float(expires_at) <= _now()
 1762  
 1763  
 1764  def _grant_refusal_reasons(grant: dict, request: dict) -> list[str]:
 1765      reasons: list[str] = []
 1766      scope = grant.get("scope") if isinstance(grant.get("scope"), dict) else {}
 1767      target = request.get("target") if isinstance(request.get("target"), dict) else {}
 1768      capability_class = str(request.get("capability_class") or request.get("class") or "")
 1769      command = [str(part) for part in (request.get("command") or [])]
 1770      declared_effects = {str(effect) for effect in (request.get("declared_effects") or [])}
 1771      requested_by = str(request.get("requested_by") or "")
 1772      granted_by = str(grant.get("granted_by") or "")
 1773  
 1774      if grant.get("class") != capability_class:
 1775          reasons.append("grant class does not match")
 1776      if _grant_expired(grant):
 1777          reasons.append("grant is expired")
 1778      if requested_by and granted_by and requested_by == granted_by and granted_by not in {
 1779          "operator",
arcticbase/backend/src/arctic_base/api/refall_capabilities.py L18require_capability_operator
   14  
   15  router = APIRouter(prefix="/workbenches/{slug}", tags=["refall-capabilities"])
   16  
   17  
   18  def require_capability_operator(
   19      request: Request,
   20      settings: Annotated[Settings, Depends(get_settings)],
   21  ) -> str | None:
   22      """Authenticate capability mutations when a host-only token is configured."""
   23      token_path = settings.refall_capability_operator_token_path
   24      if token_path is None:
   25          return None
   26      try:
   27          expected = token_path.read_text().strip()
   28      except OSError as exc:
   29          raise HTTPException(503, "capability operator credential unavailable") from exc
   30      supplied = request.headers.get("X-REFALL-Operator-Token", "")
   31      if not expected or not secrets.compare_digest(supplied, expected):
   32          raise HTTPException(403, "valid capability operator credential required")
   33      return "operator"
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1161CoverageStore.create_pending_action
 1157              args.append(status)
 1158          sql += " ORDER BY created_at DESC"
 1159          return [_transition_row(row) for row in self.conn.execute(sql, args)]
 1160  
 1161      def create_pending_action(self, subject: str, data: dict) -> dict:
 1162          now = _now()
 1163          if not data.get("kind"):
 1164              raise ValueError("pending action requires kind")
 1165          if not data.get("title"):
 1166              raise ValueError("pending action requires title")
 1167          identity_key = data.get("identity_key") or _pending_identity(subject, data)
 1168          action_id = str(data.get("id") or _now_id("pa"))
 1169          with self._wlock:
 1170              self.conn.execute(
 1171                  "INSERT INTO pending_actions(id,subject,identity_key,kind,title,description,status,"
 1172                  "priority,source_event_id,source_finding_id,source_unit_id,required_receipt_type,"
 1173                  "assigned_lane,claimed_by,lease_expires,resolution_receipt,created_by,created_at,updated_at) "
 1174                  "VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) "
 1175                  "ON CONFLICT(identity_key) DO UPDATE SET updated_at=excluded.updated_at",
 1176                  (
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1243CoverageStore.finish_pending_action
 1239              )
 1240              self.conn.commit()
 1241          return self.get_pending_action(action_id) or action
 1242  
 1243      def finish_pending_action(
 1244          self,
 1245          subject: str,
 1246          action_id: str,
 1247          *,
 1248          status: str,
 1249          receipt_type: str | None,
 1250          receipt: object,
 1251          resolved_by: str | None = None,
 1252      ) -> dict:
 1253          action = self.get_pending_action(action_id)
 1254          if action is None or action.get("subject") != subject:
 1255              raise ValueError("no such pending action")
 1256          if status not in {"done", "deferred", "cancelled"}:
 1257              raise ValueError("invalid pending action terminal status")
 1258          required = action.get("required_receipt_type")
ref_all_executor/standing_loop.py L2837RefallStandingLoop._create_failure_pending_action
 2833                  "pending_actions": 0,
 2834              },
 2835          )
 2836  
 2837      def _create_failure_pending_action(
 2838          self,
 2839          conversation: DispatchedConversation,
 2840          outcome: InvocationOutcome,
 2841          artifact_refs: list[str],
 2842      ) -> bool:
 2843          unit_id = (conversation.unit or {}).get("id")
 2844          if outcome.failure_class == "capability_missing":
 2845              capability_class = _capability_class_from_failure(outcome.last_error)
 2846              if hasattr(self.arcticbase, "request_capability"):
 2847                  self.arcticbase.request_capability(
 2848                      self.workbench,
 2849                      {
 2850                          "class": capability_class,
 2851                          "scope": _capability_scope_from_failure(outcome.last_error),
 2852                          "requested_by": conversation.cid,
ref_all_executor/standing_loop.py L2163RefallStandingLoop._write_state
 2159              return json.loads(path.read_text(encoding="utf-8"))
 2160          except json.JSONDecodeError:
 2161              return {"iteration": 0, "active": [], "last_completion_cycle": 0}
 2162  
 2163      def _write_state(
 2164          self,
 2165          cycle: int,
 2166          active: list[DispatchedConversation],
 2167          status: str,
 2168          last_completion_cycle: int,
 2169      ) -> dict:
 2170          coverage = self._safe_coverage()
 2171          state = {
 2172              "status": status,
 2173              "subject": self.subject,
 2174              "workbench": self.workbench,
 2175              "run_id": self.run_id,
 2176              "iteration": cycle,
 2177              "active": [asdict(item) for item in active],
 2178              "active_count": len(active),
ref_all_executor/standing_loop.py L2187RefallStandingLoop._write_run_manifest
 2183          }
 2184          self._state_path().write_text(json.dumps(state, indent=2, sort_keys=True) + "\n", encoding="utf-8")
 2185          return state
 2186  
 2187      def _write_run_manifest(
 2188          self,
 2189          loop_status: str,
 2190          cycle: int,
 2191          active: list[DispatchedConversation],
 2192          state: dict | None = None,
 2193      ) -> None:
 2194          if not hasattr(self.arcticbase, "put_runtime_manifest") or not self.run_id:
 2195              return
 2196          coverage = self._safe_coverage()
 2197          manifest = {
 2198              "run_id": self.run_id,
 2199              "refall_version": "REFALL-V10",
 2200              "provider_roles": _provider_roles(),
 2201              "runtime_host": {
 2202                  "hostname": socket.gethostname(),
ref_all_executor/standing_loop.py L2250RefallStandingLoop._resume_preflight_refused
 2246              except Exception:
 2247                  pass
 2248          self._write_event_checkpoint(event_type, fields)
 2249  
 2250      def _resume_preflight_refused(self) -> bool:
 2251          if not (
 2252              hasattr(self.arcticbase, "list_runtime_checkpoints")
 2253              and hasattr(self.arcticbase, "resume_preflight")
 2254          ):
 2255              return False
 2256          try:
 2257              prior = self.arcticbase.list_runtime_checkpoints(self.workbench)
 2258          except Exception:
 2259              return False
 2260          if not prior:
 2261              return False
 2262          try:
 2263              result = self.arcticbase.resume_preflight(self.workbench, self._checkpoint_input_hashes())
 2264          except RuntimeError as exc:
 2265              if "409" in str(exc):
arcticbase/backend/src/arctic_base/refall/runtime_state.py L346resume_preflight
  342          )
  343      ]
  344  
  345  
  346  def resume_preflight(conn: sqlite3.Connection, subject: str, current_hashes: dict | None) -> dict:
  347      current = {str(key): str(value) for key, value in (current_hashes or {}).items()}
  348      checkpoints = list_checkpoints(conn, subject, limit=1000)
  349      expected: dict[str, str] = {}
  350      checkpoint_ids: list[str] = []
  351      for checkpoint in reversed(checkpoints):
  352          hashes = checkpoint.get("input_hashes") if isinstance(checkpoint.get("input_hashes"), dict) else {}
  353          if hashes:
  354              checkpoint_ids.append(str(checkpoint["id"]))
  355          for key, value in hashes.items():
  356              expected[str(key)] = str(value)
  357      changes = []
  358      if current:
  359          for key, expected_hash in sorted(expected.items()):
  360              actual_hash = current.get(key)
  361              if actual_hash is None:
ref_all_executor/standing_loop.py L1554RefallStandingLoop._save_trajectory_artifact
 1550              },
 1551          )
 1552          return None
 1553  
 1554      def _save_trajectory_artifact(
 1555          self,
 1556          conversation: DispatchedConversation,
 1557          result: PollResult,
 1558      ) -> list[str]:
 1559          """Persist the agent trajectory before the conversation is reaped.
 1560  
 1561          poll() already downloaded the bytes, so this is the last point at which
 1562          they can be written durably; once the conversation is deleted from the
 1563          agent server they are unrecoverable.
 1564          """
 1565          if not result.trajectory:
 1566              return []
 1567          if not hasattr(self.arcticbase, "create_runtime_artifact"):
 1568              return []
 1569          content = json.dumps(
ref_all_executor/standing_loop.py L1596RefallStandingLoop._archive_conversation_events
 1592              return []
 1593          artifact_id = artifact.get("id")
 1594          return [str(artifact_id)] if artifact_id else []
 1595  
 1596      def _archive_conversation_events(self, conversation: DispatchedConversation) -> str:
 1597          """Copy an agent's event stream out of Agent Canvas storage before reaping.
 1598  
 1599          The canvas keeps events under `<conversations>/<cid without dashes>/events`,
 1600          which is bind-mounted read-write into the container, so deleting the
 1601          conversation also removes them. Copying into `logs/agents/<cid>/events`
 1602          puts them under the first root the REFALL AB debug endpoint searches, so
 1603          the conversation stays inspectable permanently.
 1604          """
 1605          cid_key = conversation.cid.replace("-", "")
 1606          source = Path(config.OPENHANDS_CONVERSATIONS_DIR) / cid_key / "events"
 1607          if not source.is_dir():
 1608              return "not_found"
 1609          target = self.subject_path / "refallsubloop" / "logs" / "agents" / conversation.cid / "events"
 1610          try:
 1611              target.parent.mkdir(parents=True, exist_ok=True)
ref_all_executor/standing_loop.py L1628RefallStandingLoop._reap_conversation
 1624              )
 1625              return "archive_failed"
 1626          return "archived"
 1627  
 1628      def _reap_conversation(self, conversation: DispatchedConversation) -> str:
 1629          """Delete a finished conversation so the agent server can release it.
 1630  
 1631          The agent server holds conversation state, event streams, and
 1632          trajectories in memory for the life of the process; without this the
 1633          backend grows unbounded until it is OOM-killed. Events are archived to
 1634          the subject log tree first so deletion costs no evidence.
 1635          """
 1636          delete = getattr(self.dispatcher, "delete", None)
 1637          if not callable(delete):
 1638              return "unsupported"
 1639          try:
 1640              delete(conversation.cid)
 1641          except Exception as exc:
 1642              self._append_event(
 1643                  "conversation_delete_failed",
ref_all_executor/standing_loop.py L2788RefallStandingLoop._retire_replaced_conversation
 2784          )
 2785          self._retire_replaced_conversation(conversation, retry, outcome)
 2786          return retry
 2787  
 2788      def _retire_replaced_conversation(
 2789          self,
 2790          conversation: DispatchedConversation,
 2791          retry: DispatchedConversation,
 2792          outcome: InvocationOutcome,
 2793      ) -> None:
 2794          # Archive before deleting, exactly as the success path does. A retried
 2795          # conversation is the one whose events matter most -- it is the failed
 2796          # attempt a reviewer needs in order to judge whether the retry was right.
 2797          archive_status = self._archive_conversation_events(conversation)
 2798          delete_status = "unsupported"
 2799          delete = getattr(self.dispatcher, "delete", None)
 2800          if callable(delete):
 2801              try:
 2802                  delete(conversation.cid)
 2803                  delete_status = "deleted"
ref_all_executor/watchdog.py L1299run_watchdog_subjects
 1295      subprocess.run(["crontab", "-"], input=merged, text=True, check=True)
 1296      return {"installed": True, "entry": entry, "marker": CRON_MARKER}
 1297  
 1298  
 1299  def run_watchdog_subjects(
 1300      *,
 1301      subject: str,
 1302      repo_root: Path,
 1303      base_url: str,
 1304      oh_base: str,
 1305      log_path: Path | None,
 1306      jsonl_path: Path | None,
 1307      state_path: Path | None,
 1308      dry_run: bool,
 1309  ) -> list[dict[str, Any]]:
 1310      subjects = discover_subjects(base_url) if subject in {"all", "*"} else [subject]
 1311      if not subjects:
 1312          subjects = [subject]
 1313      samples: list[dict[str, Any]] = []
 1314      for slug in subjects:
ref_all_executor/supervisor.py L561RefallSupervisor.tick
  557          return self.paths["status"]
  558  
  559      # ----------------------------------------------------------------- tick
  560  
  561      def tick(self, mode: str = "full") -> dict[str, Any]:
  562          snapshot: dict[str, Any] = {
  563              "mode": mode,
  564              "subject": self.subject,
  565              "ts": iso_now(),
  566              "actions": [],
  567              "status": STATUS_HEALTHY,
  568          }
  569  
  570          if self.hold_active():
  571              snapshot["status"] = STATUS_ESCALATED
  572              snapshot["needs_operator"] = (
  573                  f"supervisor is holding; clear {self.paths['hold']} after fixing the fault"
  574              )
  575              snapshot["probe"] = {}
  576              self.write_status(snapshot)
ref_all_executor/supervisor.py L278RefallSupervisor.check_escalation
  274          return [r for r in self._state().get("recoveries", []) if float(r.get("ts", 0)) >= cutoff]
  275  
  276      # ----------------------------------------------------------- escalation
  277  
  278      def check_escalation(self) -> dict[str, Any] | None:
  279          """N recoveries inside the window -> stop the loop and hold."""
  280          recent = self.recent_recoveries()
  281          if len(recent) < self.cfg.recover_max:
  282              return None
  283          return self.engage_hold(recent)
  284  
  285      def engage_hold(self, history: list[dict[str, Any]]) -> dict[str, Any]:
  286          pids = loop_process_pids(self.subject)
  287          stopped: list[int] = []
  288          if not self.cfg.dry_run:
  289              for pid in pids:
  290                  rc, _ = self.run_cmd(["kill", "-TERM", str(pid)], 20)
  291                  if rc == 0:
  292                      stopped.append(pid)
  293  
ref_all_executor/supervisor.py L285RefallSupervisor.engage_hold
  281          if len(recent) < self.cfg.recover_max:
  282              return None
  283          return self.engage_hold(recent)
  284  
  285      def engage_hold(self, history: list[dict[str, Any]]) -> dict[str, Any]:
  286          pids = loop_process_pids(self.subject)
  287          stopped: list[int] = []
  288          if not self.cfg.dry_run:
  289              for pid in pids:
  290                  rc, _ = self.run_cmd(["kill", "-TERM", str(pid)], 20)
  291                  if rc == 0:
  292                      stopped.append(pid)
  293  
  294          window_m = self.cfg.recover_window_s // 60
  295          lines = [
  296              "REFALL SUPERVISOR HOLD",
  297              "",
  298              f"escalated : {iso_now()}",
  299              f"subject   : {self.subject}",
  300              f"reason    : {len(history)} recoveries within {window_m}m",
ref_all_executor/standing_loop.py L1711RefallStandingLoop._maybe_write_determinations
 1707              },
 1708          )
 1709          return len(files)
 1710  
 1711      def _maybe_write_determinations(self, cycle: int) -> None:
 1712          """Compile the central 'what is every file, and what did we learn' table.
 1713  
 1714          This runs every report cycle regardless of findings, so even a pure
 1715          characterization run produces a durable, queryable roll-up instead of
 1716          results scattered across unit rows and ledger entries. The units table
 1717          already holds per-file determinations (summary/sensitivity); this
 1718          surfaces them as one artifact plus module/evidence progress.
 1719          """
 1720          if self.config.report_interval_cycles <= 0 or cycle % self.config.report_interval_cycles:
 1721              return
 1722          if not hasattr(self.arcticbase, "create_runtime_artifact"):
 1723              return
 1724          try:
 1725              coverage = self._safe_coverage()
 1726              ledger_stats = self.arcticbase.ledger_stats(self.workbench)
ref_all_executor/standing_loop.py L1804RefallStandingLoop._maybe_dispatch_report_agent
 1800          except Exception:
 1801              return self.last_report_intel
 1802          return int(by_kind.get("fact", 0)) + int(by_kind.get("lead", 0)) + int(by_kind.get("finding", 0))
 1803  
 1804      def _maybe_dispatch_report_agent(self, cycle: int, active: list[DispatchedConversation]) -> None:
 1805          """Reserve one slot for a report-writing agent when enough new intel exists.
 1806  
 1807          The default policy is triggered by accumulated intel volume, not cycle
 1808          count, so the report keeps pace with discovery without interrupting a
 1809          running analysis agent. Subjects with `refall_report_agent_continuous`
 1810          reserve one slot and immediately replace a completed report agent. The
 1811          agent enriches the living report AND maintains a running analyst-notes
 1812          digest from canonical evidence.
 1813          """
 1814          if any(getattr(conversation, "task_class", None) == "report" for conversation in active):
 1815              return
 1816          threshold = self.config.report_agent_intel_threshold
 1817          if len(active) >= self.config.max_parallel:
 1818              return
 1819          intel = self._intel_count()
ref_all_executor/standing_loop.py L1852RefallStandingLoop._report_agent_objective
 1848              "report_agent_dispatched",
 1849              {"subject": self.subject, "cid": conversation.cid, "cycle": cycle, "intel": intel},
 1850          )
 1851  
 1852      def _report_agent_objective(self, intel: int) -> str:
 1853          context_md = str(self._coverage_context().get("info_md") or "").strip()
 1854          if self._suppress_findings_for_subject():
 1855              return "\n\n".join(part for part in [
 1856                  f"## REFALL context.md\n{context_md}" if context_md else "",
 1857                  self._blackboard_text(),
 1858                  f"## Task\nYou are the ARA source-mapping codex report agent for subject {self.subject}.",
 1859                  "## Inputs to read first\n"
 1860                  f"- Existing report: GET /api/workbenches/{self.workbench}/refall-report\n"
 1861                  f"- Source-backed facts/leads/flows via /api/workbenches/{self.workbench}/ledger/entries\n"
 1862                  f"- Per-file determinations via /api/workbenches/{self.workbench}/coverage/search\n"
 1863                  f"- Runtime artifacts via /api/workbenches/{self.workbench}/runtime/artifacts\n"
 1864                  f"- Evidence graph via /api/workbenches/{self.workbench}/runtime/evidence-graph",
 1865                  "## Produce two things — source mapping, not a security report\n"
 1866                  "1. ENRICH the living HTML report as an ARA Source Mapping Codex using the ARA theme "
 1867                  "(deep navy #0a0e18, gold #ffd166, blue #4aa6cc). Include source-backed sections for "
ref_all_executor/report_enrich.py L19enrich_findings
   15  SOFTWARE_FACT_KEYWORDS = {"binary", "elf", "compiler", "rust", "version", "firmware", "build", "archive"}
   16  ARCHITECTURE_FACT_KEYWORDS = {"task", "ipc", "peripheral", "bus", "spi", "i2c", "flash", "memory", "dma"}
   17  
   18  
   19  def enrich_findings(
   20      entries: list[dict],
   21      *,
   22      runtime_observations: list[dict] | None = None,
   23      runtime_artifacts: list[dict] | None = None,
   24      evidence_graph: dict | None = None,
   25      closeout_gate: dict | None = None,
   26      pending_actions: list[dict] | None = None,
   27      capabilities: dict | None = None,
   28  ) -> dict:
   29      canonical_by_key: dict[str, dict] = {}
   30      findings: list[dict] = []
   31      leads: list[dict] = []
   32      facts: list[dict] = []
   33      for entry in entries:
   34          kind = entry.get("kind")
ref_all_executor/report_draft.py L476_structural_downgrade_sections
  472          populated.add(anchor)
  473      return populated
  474  
  475  
  476  def _structural_downgrade_sections(existing: str, candidate: str) -> list[str]:
  477      """Required sections populated in ``existing`` but absent/sparse in candidate."""
  478      lost = _populated_report_sections(existing) - _populated_report_sections(candidate)
  479      order = [anchor for anchor, _label in _REQUIRED_REPORT_SECTIONS]
  480      return [anchor for anchor in order if anchor in lost]
  481  
  482  
  483  def _record_structural_downgrade(
  484      transport,
  485      wb_url: str,
  486      *,
  487      subject: str,
  488      candidate: str,
  489      lost_sections: list[str],
  490      source: str,
  491  ) -> str | None:
ref_all_executor/report_assemble.py L16assemble_report
   12  )
   13  from .workbench import _urllib_transport
   14  
   15  
   16  def assemble_report(
   17      workbench: str,
   18      subject: str,
   19      enriched: dict,
   20      *,
   21      base: str | None = None,
   22      transport=None,
   23      allow_structural_downgrade: bool = False,
   24  ) -> dict:
   25      base = (base or config.WB_BASE).rstrip("/")
   26      transport = transport or _urllib_transport
   27      template = _req(transport, "GET", f"{base}/api/refall/reports/template")
   28      if isinstance(template, bytes):
   29          template_text = template.decode("utf-8", errors="replace")
   30      else:
   31          template_text = str(template)
arcticbase/backend/src/arctic_base/refall/closeout_gate.py L14evaluate_closeout_gate
   10  from arctic_base.refall.runtime_state import connect_runtime, list_artifacts, list_events
   11  from arctic_base.storage.filesystem import FilesystemStorage
   12  
   13  
   14  def evaluate_closeout_gate(
   15      storage: FilesystemStorage,
   16      slug: str,
   17      *,
   18      threshold: float = 0.95,
   19  ) -> dict[str, Any]:
   20      store = store_for_workbench(storage, slug)
   21      coverage = store.coverage(slug)
   22      findings = store.list_findings(slug, limit=10000)
   23      pending = store.list_pending_actions(slug, include_terminal=True)
   24      transitions = store.list_transitions(slug)
   25      entries = ledger.list_entries(storage, slug)
   26      subject_path = _subject_path(storage, slug)
   27      runtime_events, runtime_artifacts = _runtime_records(subject_path, slug)
   28      overrides = _closeout_overrides(entries)
   29      checks: list[dict[str, Any]] = []
arcticbase/backend/src/arctic_base/refall/closeout_gate.py L193_report_check
  189          "details": [item.get("id") or item for item in details],
  190      }
  191  
  192  
  193  def _report_check(storage: FilesystemStorage, slug: str, entries: list[dict]) -> dict[str, Any]:
  194      try:
  195          workbench = storage.get_workbench(slug)
  196      except Exception:
  197          return _check("report-current", False, True, [{"id": "workbench-missing"}])
  198      current_id = workbench.custom_fields.get("refall_current_report_object_id")
  199      if not current_id:
  200          return _check("report-current", False, True, [{"id": "missing"}])
  201      try:
  202          meta = storage.get_object(slug, current_id)
  203      except Exception:
  204          return _check("report-current", False, True, [{"id": current_id}])
  205      latest_entry = _latest_entry_time(entries)
  206      if latest_entry and meta.updated_at < latest_entry:
  207          return {
  208              "id": "report-stale",
ref_all_executor/loop_manager.py L81RefallLoopManager.closeout
   77              "ledger_id": ledger.get("id"),
   78              "loop": loop,
   79          }
   80  
   81      def closeout(self, name: str) -> dict:
   82          slug = name
   83          workbench = self.arcticbase.get_workbench(slug)
   84          subject = (workbench.get("custom_fields") or {}).get("refall_subject") or slug
   85          coverage = self.arcticbase.coverage_stats(slug)
   86          ledger_stats = self.arcticbase.ledger_stats(slug)
   87          draft = (self._draft_report or draft_report)(slug, subject)
   88          enriched = (self._enrich_report or enrich_workbench_report)(slug, subject)
   89          # Enrichment persists artifacts and may append ledger evidence even when
   90          # structural-downgrade protection refuses to replace the richer formal
   91          # report. Refresh the authoritative draft after enrichment so the
   92          # report freshness high-water mark includes those writes before the
   93          # closeout gate is evaluated.
   94          draft = (self._draft_report or draft_report)(slug, subject)
   95          closeout_gate = (
   96              self.arcticbase.closeout_gate(slug)
coverage_service/binary.py L159seed_binary
  155      invs.sort(key=lambda v: v["priority"])
  156      return invs
  157  
  158  
  159  def seed_binary(subject, path, binrel):
  160      """Full pipeline: classify → dangerous imports → callers → investigations. Picks richest slice."""
  161      info = classify(path)
  162      best = max(info["slices"], key=lambda s: s["functions"])
  163      dang = dangerous_present(best["imports"])
  164      callers = import_callers(path, list(dang.keys()))
  165      invs = build_investigations(subject, binrel, callers)
  166      return {"classify": info, "dangerous": dang, "n_callers": sum(len(v) for v in callers.values()),
  167              "investigations": invs}
drive_binresearch.py L27_objective
   23  
   24  _PROVIDED_WISHES = {}
   25  
   26  
   27  def _objective(agent_id, subject, binrel, n, provided_wishes=None):
   28      binpath = f"{config.subject_workspace(subject)}/{binrel}"
   29      delivered = format_provided_wishlist(subject, provided_wishes)
   30      return f"""You are a REF-All BINARY SECURITY RESEARCHER (id: {agent_id}). The subject is a compiled binary.
   31  You hunt memory-corruption / injection / auth-bypass vulns by DECOMPILING functions and tracing
   32  attacker-controlled data. Not source — you read radare2 output.
   33  Do up to {n} investigations, then STOP.
   34  
   35  Service: http://host.docker.internal:8900     Subject: "{subject}"
   36  Binary:  "{binpath}"
   37  radare2: run `export PATH=$PATH:/usr/local/bin` first, then use `r2`. Keep YOUR context lean.
   38  
   39  {delivered}
   40  
   41  0. PREFLIGHT:
   42     - GET http://host.docker.internal:8900/context?subject={subject} (read info_md if present).
drive_binreview.py L31_objective
   27  def run_validation(subject, binrel, claude, codex):
   28      return _run_validation_driver(subject, binrel, claude, codex, publish=False)
   29  
   30  
   31  def _objective(agent_id, subject, binrel, finding_ids, provided_wishes=None):
   32      binpath = f"{config.subject_workspace(subject)}/{binrel}"
   33      ids = ", ".join(finding_ids)
   34      delivered = format_provided_wishlist(subject, provided_wishes)
   35      return f"""You are a REF-All ADVERSARIAL REVIEWER (id: {agent_id}). Assume every finding is a FALSE POSITIVE
   36  until the CODE proves otherwise. Ignore the finder's prose reasoning — judge only the decompilation and the
   37  raw claim. Review EXACTLY these finding IDs for subject "{subject}": {ids}
   38  
   39  Binary: "{binpath}"     radare2: run `export PATH=$PATH:/usr/local/bin` first.
   40  Service: http://host.docker.internal:8900
   41  {delivered}
   42  
   43  Report/evidence handoff: use the REF-All report spec at
   44  {config.REFALL_WORKSPACE}/docs/superpowers/specs/2026-07-16-refall-report-format-spec.md to keep review output useful
   45  for the later validation/disclosure loop. You are not writing the disclosure now, but VALID and NEEDS_RESEARCH
   46  decisions must leave concrete trace/proof guidance rather than vague prose.
drive_binvalidate.py L34needs_validation
   30  NON_REJECTED = {"VALID", "NEEDS_RESEARCH"}
   31  RUNTIME_COMPLETE = {"runtime_confirmed", "reproduced"}
   32  
   33  
   34  def needs_validation(finding):
   35      if finding.get("status") not in NON_REJECTED:
   36          return False
   37      repro_status = (finding.get("repro_status") or "").strip().lower()
   38      return repro_status not in RUNTIME_COMPLETE
   39  
   40  
   41  def _objective(agent_id, subject, binrel, finding_ids, provided_wishes=None):
   42      binpath = f"{config.subject_workspace(subject)}/{binrel}"
   43      ids = ", ".join(finding_ids)
   44      delivered = format_provided_wishlist(subject, provided_wishes)
   45      return f"""You are a REF-All BINARY VALIDATION + VISUALIZATION AGENT (id: {agent_id}).
   46  Your job is not to find new bugs. Your job is to take retained non-rejected findings and make each one
   47  operationally useful: complete trace visualization, reproducibility notes, raw evidence/proof, and runtime
   48  validation viability.
   49  
toolchain/run.sh L1
    1  #!/usr/bin/env bash
    2  set -u
    3  
    4  ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
    5  PATH="$ROOT/bin:$ROOT/.venv/bin:$ROOT/node_modules/.bin:$PATH"
    6  export PATH
    7  export SEMGREP_USER_AGENT_APPEND="refallinfr-toolchain"
    8  
    9  usage() {
   10    cat <<'EOF'
   11  usage:
   12    run.sh <TARGET_DIR> <OUTPUT_DIR>
   13    run.sh --dry-run <TARGET_DIR> <OUTPUT_DIR>
   14    run.sh --help
   15  
   16  Runs the persistent code-understanding toolchain against TARGET_DIR and writes:
toolchain/install.sh L1
    1  #!/usr/bin/env bash
    2  set -u
    3  
    4  ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
    5  LOCK="$ROOT/versions.lock"
    6  LOG_DIR="$ROOT/logs"
    7  SUMMARY="$ROOT/install-summary.txt"
    8  mkdir -p "$ROOT/bin" "$ROOT/config" "$ROOT/viewers" "$LOG_DIR"
    9  
   10  # shellcheck source=/dev/null
   11  source "$LOCK"
   12  
   13  : > "$SUMMARY"
   14  
   15  record() {
   16    printf '%-28s %s\n' "$1" "$2" | tee -a "$SUMMARY"
toolchain/config/services.json L1
    1  {
    2    "services": [
    3      {
    4        "name": "python-lsp-server",
    5        "description": "python-lsp-server (pylsp) in TCP mode for interactive Python code understanding sessions.",
    6        "start_cmd": "exec \"$TOOLCHAIN_ROOT/.venv/bin/pylsp\" --tcp --host 127.0.0.1 --port 20871",
    7        "stop_method": "process_group",
    8        "health_check": {
    9          "type": "tcp",
   10          "host": "127.0.0.1",
   11          "port": 20871,
   12          "timeout_sec": 10
   13        },
   14        "port": 20871,
   15        "log_file": "logs/python-lsp-server.log"
   16      },
arcticbase/backend/src/arctic_base/refall/ledger.py L41–57append_entry
   37      storage.get_workbench(slug)
   38      return storage.root / "workbenches" / slug / "refall_ledger.jsonl"
   39  
   40  
   41  def append_entry(storage: FilesystemStorage, slug: str, entry: dict[str, Any]) -> dict[str, Any]:
   42      path = ledger_path(storage, slug)
   43      now = datetime.now(UTC).isoformat()
   44      data = dict(entry)
   45      data.setdefault("kind", "fact")
   46      if data["kind"] not in LEDGER_KINDS:
   47          raise ValueError(f"unsupported ledger kind: {data['kind']}")
   48      data.setdefault("id", f"led_{uuid.uuid4().hex}")
   49      data.setdefault("workbench_slug", slug)
   50      data.setdefault("created_at", now)
   51      data["updated_at"] = now
   52  
   53      path.parent.mkdir(parents=True, exist_ok=True)
   54      with path.open("a", encoding="utf-8") as fh:
   55          fh.write(json.dumps(data, sort_keys=True) + "\n")
   56      storage.audit(slug, "refall.ledger.append", id=data["id"], kind=data["kind"])
   57      return data
arcticbase/frontend/src/lib/refall/WorkbenchOpsPanel.svelte L953–1034Ledger and Output views
  949          </ul>
  950        {:else}
  951          <p class="muted">Closeout gate has not loaded.</p>
  952        {/if}
  953      {:else if activeView === "ledger"}
  954        {#if ledgerEntries.length}
  955          <ul class="item-list">
  956            {#each ledgerEntries.slice().reverse().slice(0, 80) as entry}
  957              <li>
  958                <div class="item-head">
  959                  <strong>{titleFor(entry)}</strong>
  960                  <span class="badge">{entry.kind || "ledger"}</span>
  961                </div>
  962                <p>{entry.created_at || ""} {entry.status || ""} {entry.provider || entry.lane || ""}</p>
  963                {#if entry.summary}<p>{entry.summary}</p>{/if}
  964              </li>
  965            {/each}
  966          </ul>
  967        {:else}
  968          <p class="muted">No ledger entries have been written for this subject.</p>
  969        {/if}
  970      {:else if activeView === "output"}
  971        <div class="view-head">
  972          <span>Agent output — facts, leads &amp; findings compiled to the ledger</span>
  973          <div class="output-filters">
  974            <button class:active={outputFilter === "all"} on:click={() => (outputFilter = "all")}>
  975              All <em>{outputEntries.length}</em>
  976            </button>
  977            <button class:active={outputFilter === "finding"} on:click={() => (outputFilter = "finding")}>
  978              Findings <em>{outputCounts.finding}</em>
  979            </button>
  980            <button class:active={outputFilter === "lead"} on:click={() => (outputFilter = "lead")}>
  981              Leads <em>{outputCounts.lead}</em>
  982            </button>
  983            <button class:active={outputFilter === "fact"} on:click={() => (outputFilter = "fact")}>
  984              Facts <em>{outputCounts.fact}</em>
  985            </button>
  986          </div>
  987        </div>
  988        {#if shownOutput.length}
  989          <ul class="output-list">
  990            {#each shownOutput as entry, i (entry.id + "::" + i)}
  991              {@const rowKey = entry.id + "::" + i}
  992              {@const sev = sevOf(entry)}
  993              {@const anchors = outputAnchors(entry)}
  994              {@const desc = outputDesc(entry)}
  995              <li class="output-card kind-{entry.kind}" class:open={expandedOutput[rowKey]}>
  996                <button class="output-row" on:click={() => toggleOutput(rowKey)} aria-expanded={!!expandedOutput[rowKey]}>
  997                  <span class="caret" aria-hidden="true">▸</span>
  998                  <span class="k-badge kind-{entry.kind}">{entry.kind}</span>
  999                  {#if sev}<span class="sev sev-{sev}">{sev}</span>{/if}
 1000                  <span class="o-title">{outputTitle(entry)}</span>
 1001                  {#if desc && !expandedOutput[rowKey]}<span class="o-snippet">{desc}</span>{/if}
 1002                </button>
 1003                {#if expandedOutput[rowKey]}
 1004                  <div class="output-body">
 1005                    {#if desc}<p class="o-desc">{desc}</p>{/if}
 1006                    {#if ostr(entry.investigation_plan)}
 1007                      <div class="o-field">
 1008                        <span class="o-label">Next step</span>
 1009                        <p class="o-text">{ostr(entry.investigation_plan)}</p>
 1010                      </div>
 1011                    {/if}
 1012                    {#if anchors.length}
 1013                      <div class="o-field">
 1014                        <span class="o-label">Evidence &amp; code paths</span>
 1015                        <ul class="o-anchors">
 1016                          {#each anchors as a}<li><code>{a}</code></li>{/each}
 1017                        </ul>
 1018                      </div>
 1019                    {/if}
 1020                    <div class="o-field">
 1021                      <span class="o-label">Details</span>
 1022                      <dl class="o-meta">
 1023                        {#if sev}<div class="o-kv"><dt>Severity</dt><dd class="sev-text sev-{sev}">{sev}</dd></div>{/if}
 1024                        {#if ostr(entry.status)}<div class="o-kv"><dt>Status</dt><dd>{ostr(entry.status)}</dd></div>{/if}
 1025                        {#if ostr((entry as Record<string, unknown>).quality_tier)}<div class="o-kv"><dt>Quality</dt><dd>{ostr((entry as Record<string, unknown>).quality_tier)}</dd></div>{/if}
 1026                        {#if ostr(entry.lane)}<div class="o-kv"><dt>Lane</dt><dd>{ostr(entry.lane)}</dd></div>{/if}
 1027                        {#if ostr((entry as Record<string, unknown>).phase)}<div class="o-kv"><dt>Phase</dt><dd>{ostr((entry as Record<string, unknown>).phase)}</dd></div>{/if}
 1028                        {#if ostr((entry as Record<string, unknown>).unit_id)}<div class="o-kv"><dt>Unit</dt><dd class="mono">{ostr((entry as Record<string, unknown>).unit_id)}</dd></div>{/if}
 1029                        {#if ostr(entry.agent)}<div class="o-kv"><dt>Agent</dt><dd class="mono">{ostr(entry.agent).slice(0, 8)}</dd></div>{/if}
 1030                        {#if ostr(entry.created_at)}<div class="o-kv"><dt>Time</dt><dd>{ostr(entry.created_at)}</dd></div>{/if}
 1031                        <div class="o-kv"><dt>Ledger ID</dt><dd class="mono muted">{entry.id}</dd></div>
 1032                      </dl>
 1033                    </div>
 1034                  </div>
ref_all_executor/watchdog.py L903–964_handle_wishlist
  899              "wishlist_provided": lambda: self.ab.list_wishlist(self.subject, status="provided"),
  900          }
  901          return {name: self._safe_call(fn) for name, fn in calls.items()}
  902  
  903      def _handle_wishlist(self, endpoints: dict[str, Any]) -> dict[str, Any]:
  904          open_result = endpoints.get("wishlist_open") or {}
  905          provided_result = endpoints.get("wishlist_provided") or {}
  906          open_items = open_result.get("data") if open_result.get("ok") else []
  907          provided_items = provided_result.get("data") if provided_result.get("ok") else []
  908          if not isinstance(open_items, list):
  909              open_items = []
  910          if not isinstance(provided_items, list):
  911              provided_items = []
  912  
  913          provided_keys = {
  914              wishlist_item_key(item)
  915              for item in provided_items
  916              if isinstance(item, dict) and wishlist_item_key(item)
  917          }
  918          provided_records = []
  919          install_attempts = []
  920          unhandled = []
  921  
  922          for item in open_items:
  923              if not isinstance(item, dict):
  924                  continue
  925              kind = str(item.get("kind") or "").lower()
  926              tool = normalize_wishlist_tool(item.get("item"))
  927              if not tool or (kind and kind not in {"tool", "cli", "binary", "package", "dependency"}):
  928                  unhandled.append({"item": item, "reason": "unsupported_wishlist_kind"})
  929                  continue
  930              if tool in provided_keys:
  931                  continue
  932              if shutil.which(tool):
  933                  provided_records.append(self._record_wishlist_provided(item, tool, {"mode": "already_available"}))
  934                  provided_keys.add(tool)
  935                  continue
  936              if self.dry_run:
  937                  unhandled.append({"item": item, "reason": "dry_run_install_skipped", "tool": tool})
  938                  continue
  939              if not self.install_wishlist_items:
  940                  unhandled.append({"item": item, "reason": "install_disabled"})
  941                  continue
  942              installer = WISHLIST_INSTALLERS.get(tool)
  943              if not installer:
  944                  unhandled.append({"item": item, "reason": "no_allowlisted_installer", "tool": tool})
  945                  continue
  946              install_result = self._install_wishlist_tool(tool, installer)
  947              install_attempts.append({"item": item, "tool": tool, "result": install_result})
  948              if install_result.get("ok") and shutil.which(tool):
  949                  provided_records.append(self._record_wishlist_provided(item, tool, {"mode": "installed", "install_result": install_result}))
  950                  provided_keys.add(tool)
  951              else:
  952                  unhandled.append({"item": item, "reason": "install_failed", "tool": tool, "install_result": install_result})
  953  
  954          return {
  955              "open_count": len(open_items),
  956              "provided_count": len(provided_records),
  957              "provided": provided_records,
  958              "install_attempts": install_attempts,
  959              "unhandled": unhandled,
  960              "endpoint_ok": {
  961                  "open": bool(open_result.get("ok")),
  962                  "provided": bool(provided_result.get("ok")),
  963              },
  964          }
ref_all_executor/watchdog.py L966–993_record_wishlist_provided
  962                  "provided": bool(provided_result.get("ok")),
  963              },
  964          }
  965  
  966      def _record_wishlist_provided(self, source_item: dict[str, Any], tool: str, receipt: dict[str, Any]) -> dict[str, Any]:
  967          payload = {
  968              "subject": self.subject,
  969              "kind": source_item.get("kind") or "tool",
  970              "item": tool,
  971              "reason": f"Watchdog provided requested wishlist item from {source_item.get('id') or 'unknown'}",
  972              "requested_by": WATCHDOG_NAME,
  973              "status": "provided",
  974          }
  975          if self.dry_run:
  976              return {"dry_run": True, "request": payload, "receipt": receipt}
  977          created = self.ab.create_wishlist_item(self.subject, payload)
  978          ledger = _safe_ledger(
  979              self.ab,
  980              self.subject,
  981              {
  982                  "kind": "ops-event",
  983                  "subject": self.subject,
  984                  "action": "watchdog.wishlist.provided",
  985                  "status": "provided",
  986                  "summary": f"Watchdog provided wishlist tool {tool}",
  987                  "wishlist_item_id": source_item.get("id"),
  988                  "provided_wishlist_item_id": created.get("id") if isinstance(created, dict) else None,
  989                  "tool": tool,
  990                  "receipt": receipt,
  991              },
  992          )
  993          return {"source": source_item, "created": created, "ledger": ledger, "receipt": receipt}
ref_all_executor/wishlist.py L4–21format_provided_wishlist
    1  from urllib.parse import quote
    2  
    3  
    4  def format_provided_wishlist(subject, provided_wishes):
    5      endpoint = f"/wishlist?subject={quote(subject)}&status=provided"
    6      lines = [
    7          "DELIVERED WISHLIST ITEMS:",
    8          f"- Check {endpoint} during preflight for the live delivered-tool/access state.",
    9      ]
   10      wishes = list(provided_wishes or [])
   11      if not wishes:
   12          lines.append("- No delivered wishlist items are recorded yet.")
   13      else:
   14          lines.append("- Use these before requesting duplicate tooling. Do not re-request matching items.")
   15          for wish in wishes:
   16              kind = wish.get("kind") or "item"
   17              item = wish.get("item") or "(unnamed)"
   18              requested_by = wish.get("requested_by") or "unknown agent"
   19              reason = wish.get("reason") or "no reason recorded"
   20              lines.append(f"  * {kind}: {item} — requested by {requested_by}; reason: {reason}")
   21      return "\n".join(lines)
ref_all_executor/head_orchestrator.py L40run_cycle
   36          self.usage = usage
   37          self.verifier = verifier
   38          self.reporter = reporter
   39  
   40      def run_cycle(
   41          self,
   42          subject: str,
   43          specialist_lane: str,
   44          *,
   45          refresh_report: bool = False,
   46          verify: bool = False,
   47      ) -> HeadCycleResult:
   48          if specialist_lane not in SPECIALIST_LANES:
   49              raise ValueError(f"unknown specialist lane: {specialist_lane}")
   50          gate = self.coverage.gate(subject)
   51          unit = self.coverage.claim_next(subject, self.profile.name)
   52          provided = self._provided_wishlist(subject)
   53          objective = _objective(subject, specialist_lane, unit, provided, self._coverage_context(subject))
   54          dispatch = self.dispatcher.dispatch(specialist_lane, objective)
   55          self.ledger.append({
ref_all_executor/standing_loop.py L590–636_claim_next_task
  586              "enumerated": enumerated,
  587              "state": final_state,
  588          }
  589  
  590      def _claim_next_task(self, active: list[DispatchedConversation] | None = None) -> AgentTask | None:
  591          # 1. Operator/manager pending actions always preempt.
  592          pending = self._claim_pending_action()
  593          if pending:
  594              if pending.get("kind") == "threat_model_request":
  595                  return self._task_for_threat_model(action=pending, active=active)
  596              return self._task_for_pending_action(pending, active=active)
  597          threat_model = self._claim_threat_model_task(active=active)
  598          if threat_model:
  599              return threat_model
  600  
  601          active = active or []
  602          in_flight = self._module_in_flight(active)
  603          in_flight_targets = self._module_in_flight_targets(active)
  604  
  605          # 2. Subject-scoped modules (identify, research, toolchain, synthesis):
  606          #    once each, foundational, so they run early. A single-target module
  607          #    already running is not re-dispatched (no 4 agents doing one identify).
  608          subject_module = self.scheduler.next_subject_module(in_flight, in_flight_targets)
  609          if subject_module is not None:
  610              return self._task_for_subject_module(subject_module, active=active)
  611  
  612          # 3. Breadth: claim a new coverage unit for the characterize module.
  613          #
  614          # Exhaustive mapping jobs need the denominator covered before the loop
  615          # repeatedly deepens a small initial slice. Keep pending actions and
  616          # one-shot subject setup ahead of breadth, but prefer fresh source units
  617          # over per-target follow-on modules while unclaimed coverage remains.
  618          if self.scheduler.has_breadth():
  619              unit = self._claim_next()
  620              if unit:
  621                  return self._task_for_characterize(unit, active=active)
  622  
  623          # 4. Findings need verification/patching once they exist and no fresh
  624          # coverage unit is waiting.
  625          finding_work = self.scheduler.next_finding_module(in_flight, in_flight_targets)
  626          if finding_work is not None:
  627              module, finding = finding_work
  628              return self._task_for_finding_module(module, finding, active=active)
  629  
  630          # 5. Depth: follow-on assessment on units already characterized.
  631          unit_work = self.scheduler.next_unit_module(in_flight, in_flight_targets)
  632          if unit_work is not None:
  633              module, unit = unit_work
  634              return self._task_for_unit_module(module, unit, active=active)
  635  
  636          return None
ref_all_executor/standing_loop.py L442run
  438                  "module_progress_save_failed",
  439                  {"subject": self.subject, "error": str(exc)[:300]},
  440              )
  441  
  442      def run(self) -> dict:
  443          self._ensure_dirs()
  444          state = self._load_state()
  445          self.run_id = str(state.get("run_id") or f"run_{uuid.uuid4().hex}")
  446          self.checkpoint_ids = []
  447          self.source_inventory = None
  448          self.scope_overlay = None
  449          self.threat_model = None
  450          self.threat_model_dispatched = False
  451          active = [self._active_from_dict(item) for item in state.get("active", [])]
  452          cycle = int(state.get("iteration") or 0)
  453          completed = 0
  454          dispatched = 0
  455          last_completion_cycle = int(state.get("last_completion_cycle") or cycle)
  456          self._latest_source_inventory()
  457          if active:
ref_all_executor/work_modules.py L472next_unit_module
  468              target=SUBJECT_TARGET, in_flight_targets=in_flight_targets,
  469          )
  470          return eligible[0] if eligible else None
  471  
  472      def next_unit_module(
  473          self, in_flight: dict[str, int],
  474          in_flight_targets: set[tuple[str, str]] | None = None,
  475      ) -> tuple[WorkModule, dict[str, Any]] | None:
  476          """Highest-priority follow-on module across all characterized units."""
  477          best: tuple[WorkModule, dict[str, Any]] | None = None
  478          for unit_id, unit in self.characterized.items():
  479              eligible = [
  480                  m for m in eligible_modules(
  481                      self.registry, target_kind=SOURCE_UNITS,
  482                      completed=self._done_set(unit_id), path=unit.get("path"), in_flight=in_flight,
  483                      target=unit_id, in_flight_targets=in_flight_targets,
  484                  )
  485                  if m.after  # breadth (characterize) is claimed separately
  486              ]
  487              if not eligible:
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L366_authorize_process_launch
  362          "loop": loop,
  363      }
  364  
  365  
  366  def _authorize_process_launch(
  367      *,
  368      storage: FilesystemStorage,
  369      slug: str,
  370      settings: Settings,
  371      subject: str,
  372      workbench: str,
  373      max_cycles: int,
  374      subject_path: Path | None,
  375      action: str,
  376  ) -> dict:
  377      store = store_for_workbench(storage, slug)
  378      request = {
  379          "requested_by": "refall-ops-command",
  380          "capability_class": "run_local_command",
  381          "command": _standing_loop_argv(settings, subject, workbench, max_cycles),
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L424_launch_standing_loop
  420          str(max_cycles),
  421      ]
  422  
  423  
  424  def _launch_standing_loop(
  425      *,
  426      settings: Settings,
  427      subject: str,
  428      workbench: str,
  429      max_cycles: int,
  430      subject_path: Path | None,
  431      source_path: Path | None = None,
  432  ) -> dict:
  433      if not settings.refall_enable_loop_launch:
  434          return {
  435              "status": "disabled",
  436              "enabled": False,
  437              "reason": "REFALL_ENABLE_LOOP_LAUNCH=false",
  438          }
  439  
arcticbase/frontend/src/lib/refall/WorkbenchOpsPanel.svelte L1
    1  <script lang="ts">
    2    import { onMount } from "svelte";
    3    import {
    4      api,
    5      type RefallCapabilityState,
    6      type RefallCloseoutGate,
    7      type RefallCoverageContext,
    8      type RefallCoverageStats,
    9      type RefallDoctorPayload,
   10      type RefallEvidenceGraph,
   11      type RefallEstimatePayload,
   12      type RefallLedgerEntry,
   13      type RefallLedgerStats,
   14      type RefallNonfatalError,
   15      type RefallPendingAction,
   16      type RefallRuntimeArtifact,
ref_all_executor/loop_manager.py L36RefallLoopManager.start
   32          # source projection and capability authorization cannot be bypassed.
   33          # A launcher remains injectable for focused unit tests and embedders.
   34          self._loop_launcher = loop_launcher
   35  
   36      def start(self, name: str) -> dict:
   37          created = self.arcticbase.create_subject(name)
   38          slug = created.get("workbench_slug") or created.get("subject") or name
   39          subject = created.get("subject") or slug
   40          self._require_context(slug)
   41          if self._loop_launcher is None:
   42              return self.arcticbase.ops_command(f"start {slug}")
   43          loop = self._loop_launcher(subject=subject, workbench=slug, max_cycles=0)
   44          ledger = self._record(slug, "loop.start", "REFALL loop started", status="started")
   45          return {
   46              "subject": subject,
   47              "workbench": slug,
   48              "status": "started",
   49              "subject_path": created.get("subject_path"),
   50              "ledger_id": ledger.get("id"),
   51              "loop": loop,
ref_all_executor/loop_manager.py L54RefallLoopManager.update
   50              "ledger_id": ledger.get("id"),
   51              "loop": loop,
   52          }
   53  
   54      def update(self, name: str) -> dict:
   55          slug = name
   56          if self._loop_launcher is None:
   57              return self.arcticbase.ops_command(f"update {slug}")
   58          workbench = self.arcticbase.get_workbench(slug)
   59          coverage = self.arcticbase.coverage_stats(slug)
   60          ledger_stats = self.arcticbase.ledger_stats(slug)
   61          subject = (workbench.get("custom_fields") or {}).get("refall_subject") or slug
   62          loop = self._loop_launcher(subject=subject, workbench=slug, max_cycles=1)
   63          ledger = self._record(
   64              slug,
   65              "loop.update",
   66              "REFALL loop state updated",
   67              status="updated",
   68              coverage=coverage,
   69              ledger_stats=ledger_stats,
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L266_start_subject
  262          raise HTTPException(422, f"{command} requires a subject")
  263      return args[0].strip()
  264  
  265  
  266  def _start_subject(subject: str, settings: Settings, storage: FilesystemStorage) -> dict:
  267      boot = bootstrap_subject(settings=settings, storage=storage, subject=subject)
  268      slug = boot.workbench_slug
  269      workbench = storage.get_workbench(slug)
  270      source_path = _loop_source_path(workbench.model_dump(mode="json"), boot.subject_path)
  271      store = store_for_workbench(storage, slug)
  272      context = store.get_context(slug)
  273      if not context or not str(context.get("info_md") or "").strip():
  274          raise HTTPException(
  275              409,
  276              "Generate Context before starting REFALL; context.md is required for the head and specialist agents.",
  277          )
  278      if settings.refall_enable_loop_launch:
  279          _authorize_process_launch(
  280              storage=storage,
  281              slug=slug,
ref_all_executor/agent_workspace.py L17sync_agent_canvas_workspace
   13  class AgentWorkspaceError(RuntimeError):
   14      pass
   15  
   16  
   17  def sync_agent_canvas_workspace(
   18      subject: str,
   19      source_path: str | Path,
   20      *,
   21      runner=subprocess,
   22      container: str | None = None,
   23      workspace_root: str | None = None,
   24      timeout_s: int = 300,
   25  ) -> dict:
   26      """Copy the host subject source into the Agent Canvas /projects workspace."""
   27      source = Path(source_path).expanduser().resolve()
   28      if not source.is_dir():
   29          raise AgentWorkspaceError(f"REFALL source path is not a directory: {source}")
   30  
   31      name = container or os.environ.get("REFALL_OPENHANDS_CONTAINER", DEFAULT_CONTAINER)
   32      root = (workspace_root or os.environ.get("REFALL_SUBJECT_WORKSPACE_ROOT", DEFAULT_WORKSPACE_ROOT)).rstrip("/")
ref_all_executor/agent_workspace.py L103_safe_subject_name
   99          "workspace": dest,
  100      }
  101  
  102  
  103  def _safe_subject_name(subject: str) -> str:
  104      value = str(subject or "").strip()
  105      if not value or "/" in value or value in {".", ".."}:
  106          raise AgentWorkspaceError(f"unsafe REFALL subject name for Agent Canvas workspace: {subject!r}")
  107      return value
  108  
  109  
  110  def _source_snapshot(source: Path) -> dict:
  111      files = 0
  112      total_bytes = 0
  113      latest_mtime_ns = 0
  114      for root, dirs, names in os.walk(source):
  115          dirs.sort()
  116          names.sort()
  117          for name in names:
  118              path = Path(root) / name
ref_all_executor/source_inventory.py L179_symlink_observation
  175          "binary": binary,
  176      }
  177  
  178  
  179  def _symlink_observation(root: Path, path: Path) -> dict[str, Any]:
  180      try:
  181          target = path.resolve()
  182          target.relative_to(root)
  183          off_root = False
  184      except (OSError, ValueError):
  185          target = path
  186          off_root = True
  187      return {
  188          "kind": "symlink",
  189          "path": _relative(root, path),
  190          "target": str(target),
  191          "off_root": off_root,
  192      }
arcticbase/backend/src/arctic_base/api/refall_capabilities.py L23–33require_capability_operator
   19      request: Request,
   20      settings: Annotated[Settings, Depends(get_settings)],
   21  ) -> str | None:
   22      """Authenticate capability mutations when a host-only token is configured."""
   23      token_path = settings.refall_capability_operator_token_path
   24      if token_path is None:
   25          return None
   26      try:
   27          expected = token_path.read_text().strip()
   28      except OSError as exc:
   29          raise HTTPException(503, "capability operator credential unavailable") from exc
   30      supplied = request.headers.get("X-REFALL-Operator-Token", "")
   31      if not expected or not secrets.compare_digest(supplied, expected):
   32          raise HTTPException(403, "valid capability operator credential required")
   33      return "operator"
arcticbase/backend/src/arctic_base/api/refall_capabilities.py L101grant_capability
   97      return request
   98  
   99  
  100  @router.post("/capabilities/grant", status_code=201, operation_id="grantRefallCapability")
  101  def grant_capability(
  102      slug: str,
  103      body: CapabilityGrantReq,
  104      storage: Annotated[FilesystemStorage, Depends(get_storage)],
  105      operator_identity: Annotated[str | None, Depends(require_capability_operator)],
  106  ) -> dict:
  107      store = store_for_workbench(storage, slug)
  108      payload = body.model_dump(by_alias=True)
  109      payload["class"] = body.class_
  110      if operator_identity is not None:
  111          payload["granted_by"] = operator_identity
  112      try:
  113          grant = store.grant_capability(slug, payload)
  114      except ValueError as exc:
  115          raise HTTPException(422, str(exc)) from exc
  116      ledger.append_entry(
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1785–1806_grant_refusal_reasons
 1781          "refall-ops-panel",
 1782      }:
 1783          reasons.append("agents cannot authorize their own live actions")
 1784  
 1785      host = str(target.get("host") or _host_from_command(command) or "")
 1786      if host and _scope_list(scope, "hosts") and host not in {str(item) for item in _scope_list(scope, "hosts")}:
 1787          reasons.append("target host outside grant scope")
 1788      port = target.get("port") or _port_from_command(command)
 1789      if port is not None and _scope_list(scope, "ports"):
 1790          if int(port) not in {int(item) for item in _scope_list(scope, "ports")}:
 1791              reasons.append("target port outside grant scope")
 1792  
 1793      method = str(target.get("method") or request.get("method") or "").upper()
 1794      allowed_methods = {str(item).upper() for item in _scope_list(scope, "methods")}
 1795      if allowed_methods:
 1796          if not method:
 1797              reasons.append("HTTP method required by grant scope")
 1798          elif method not in allowed_methods:
 1799              reasons.append("HTTP method outside grant scope")
 1800  
 1801      path = str(target.get("path") or _path_from_command(command) or "")
 1802      path_scope = _scope_list(scope, "paths")
 1803      if capability_class == "run_local_command" and path and not path_scope:
 1804          reasons.append("target path scope required")
 1805      if path and path_scope and not _path_in_scope(path, path_scope):
 1806          reasons.append("target path outside grant scope")
arcticbase/backend/src/arctic_base/api/refall_capabilities.py L171authorize_execution
  167      return revoke_capability(slug, grant_id, storage, operator_identity)
  168  
  169  
  170  @router.post("/execution/authorize", operation_id="authorizeRefallExecution")
  171  def authorize_execution(
  172      slug: str,
  173      request: Annotated[dict, Body()],
  174      storage: Annotated[FilesystemStorage, Depends(get_storage)],
  175  ) -> dict:
  176      result = store_for_workbench(storage, slug).authorize_execution(slug, request)
  177      ledger.append_entry(
  178          storage,
  179          slug,
  180          {
  181              "kind": "capability-decision",
  182              "subject": slug,
  183              "status": result["status"],
  184              "capability_class": request.get("capability_class"),
  185              "grant_id": result.get("grant_id"),
  186              "summary": "; ".join(result.get("refusal_reasons") or []) or "authorized",
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L115require_ops_commands
  111      slug: str
  112      context: str
  113  
  114  
  115  def require_ops_commands() -> Settings:
  116      settings = require_ops()
  117      if not settings.refall_enable_ops_commands:
  118          raise HTTPException(403, "REFALL ops commands disabled")
  119      return settings
  120  
  121  
  122  @router.post("/command", operation_id="runRefallOpsCommand")
  123  def run_command(
  124      body: Annotated[CommandReq, Body()],
  125      settings: Annotated[Settings, Depends(require_ops_commands)],
  126      storage: Annotated[FilesystemStorage, Depends(get_storage)],
  127  ) -> dict:
  128      parts = body.command.strip().split()
  129      if not parts:
  130          raise HTTPException(422, "command required")
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L123run_command
  119      return settings
  120  
  121  
  122  @router.post("/command", operation_id="runRefallOpsCommand")
  123  def run_command(
  124      body: Annotated[CommandReq, Body()],
  125      settings: Annotated[Settings, Depends(require_ops_commands)],
  126      storage: Annotated[FilesystemStorage, Depends(get_storage)],
  127  ) -> dict:
  128      parts = body.command.strip().split()
  129      if not parts:
  130          raise HTTPException(422, "command required")
  131      name = parts[0]
  132      args = parts[1:]
  133      if name not in KNOWN_COMMANDS:
  134          _audit(storage, name, args, "rejected-unknown")
  135          raise HTTPException(422, f"unknown REFALL ops command: {name}")
  136  
  137      allowed = {item.strip() for item in settings.refall_ops_allowed_commands.split(",") if item.strip()}
  138      if name not in allowed:
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L386–403_authorize_process_launch
  382          "declared_effects": ["process"],
  383          "target": {"path": str(subject_path) if subject_path is not None else ""},
  384          "action": action,
  385      }
  386      result = store.authorize_execution(slug, request)
  387      ledger.append_entry(
  388          storage,
  389          slug,
  390          {
  391              "kind": "capability-decision",
  392              "subject": slug,
  393              "status": result["status"],
  394              "capability_class": request["capability_class"],
  395              "grant_id": result.get("grant_id"),
  396              "summary": "; ".join(result.get("refusal_reasons") or []) or "authorized",
  397              "request": request,
  398          },
  399      )
  400      if not result.get("accepted"):
  401          refusal = ", ".join(result.get("refusal_reasons") or ["refused"])
  402          raise HTTPException(403, f"run_local_command capability refused: {refusal}")
  403      return result
ref_all_executor/work_modules.py L472ModuleScheduler.next_unit_module
  468              target=SUBJECT_TARGET, in_flight_targets=in_flight_targets,
  469          )
  470          return eligible[0] if eligible else None
  471  
  472      def next_unit_module(
  473          self, in_flight: dict[str, int],
  474          in_flight_targets: set[tuple[str, str]] | None = None,
  475      ) -> tuple[WorkModule, dict[str, Any]] | None:
  476          """Highest-priority follow-on module across all characterized units."""
  477          best: tuple[WorkModule, dict[str, Any]] | None = None
  478          for unit_id, unit in self.characterized.items():
  479              eligible = [
  480                  m for m in eligible_modules(
  481                      self.registry, target_kind=SOURCE_UNITS,
  482                      completed=self._done_set(unit_id), path=unit.get("path"), in_flight=in_flight,
  483                      target=unit_id, in_flight_targets=in_flight_targets,
  484                  )
  485                  if m.after  # breadth (characterize) is claimed separately
  486              ]
  487              if not eligible:
ref_all_executor/standing_loop.py L205AgentCanvasDispatcher.start
  201  class AgentCanvasDispatcher:
  202      def __init__(self, oh: OHClient | None = None) -> None:
  203          self.oh = oh or OHClient()
  204  
  205      def start(self, task: AgentTask) -> DispatchedConversation:
  206          workspace = config.prepare_subject_agent_workspace(
  207              task.subject,
  208              lane=task.lane,
  209              task_class=task.task_class,
  210              unit=task.unit,
  211              objective=task.objective,
  212          )
  213          workspace_host = str(config.agent_workspace_host_path(
  214              task.subject, lane=task.lane, task_class=task.task_class,
  215              unit=task.unit, objective=task.objective,
  216          ))
  217          objective = _with_workspace_context(task.subject, workspace, _with_resource_context(task.objective))
  218          cid = self.oh.start_conversation(
  219              workspace=config.local_workspace(workspace),
  220              agent_settings=config.LANES[task.lane],
ref_all_executor/oh_client.py L89OHClient.start_conversation
   85              if self.request_backoff_s:
   86                  self.sleep(self.request_backoff_s * attempt)
   87          raise RuntimeError(f"{method} {url} exhausted {attempts} attempts")  # pragma: no cover
   88  
   89      def start_conversation(self, workspace, agent_settings, text, max_iterations=30):
   90          """workspace: a dict from config.local_workspace()/remote_workspace()."""
   91          body = {
   92              "workspace": workspace,
   93              "agent_settings": agent_settings,
   94              "initial_message": {
   95                  "role": "user",
   96                  "content": [{"type": "text", "text": text}],
   97                  "run": True,
   98              },
   99              "max_iterations": max_iterations,
  100          }
  101          status, data = self.request("POST", "/api/conversations", body)
  102          if status != 201 or not isinstance(data, dict) or "id" not in data:
  103              raise RuntimeError(f"start_conversation failed {status}: {data!r}")
  104          return data["id"]
ref_all_executor/runtime_invocation.py L23classify_invocation
   19      last_error: str = ""
   20      next_action: str = "none"
   21  
   22  
   23  def classify_invocation(
   24      status: str,
   25      response: str,
   26      *,
   27      require_structured_submit: bool,
   28      attempt: int = 1,
   29      max_attempts: int = 2,
   30  ) -> InvocationOutcome:
   31      if status not in {"finished", "complete", "completed"}:
   32          failure_class = _status_failure_class(status)
   33          if failure_class == "capability_missing":
   34              return InvocationOutcome(
   35                  state="pending_capability",
   36                  failure_class=failure_class,
   37                  retryable=False,
   38                  attempt=attempt,
ref_all_executor/standing_loop.py L888RefallStandingLoop._objective
  884              "created_at, and a threat_model object with assets, ingress_points, trust_boundaries, threats, "
  885              "hunt_prompts, and status. Keep this REFALL-owned; do not use VVAH report or encryption specs.",
  886          ])
  887  
  888      def _objective(self, task_class: str, reason: str, unit: dict, extra: str = "") -> str:
  889          context_md = str(self._coverage_context().get("info_md") or "").strip()
  890          scope = self._scope_overlay_text()
  891          blackboard = self._blackboard_text()
  892          path = unit.get("path") or unit.get("name") or "unknown"
  893          line_start = unit.get("line_start") or "?"
  894          line_end = unit.get("line_end") or "?"
  895          return "\n\n".join(part for part in [
  896              f"## REFALL context.md\n{context_md}" if context_md else "",
  897              f"## Scope Overlay\n```json\n{scope}\n```" if scope else "",
  898              blackboard,
  899              f"## Task\nRun {task_class} for REFALL subject {self.subject}.",
  900              f"## Routing\n{reason}",
  901              "## Coverage Unit\n"
  902              f"- ID: {unit.get('id')}\n"
  903              f"- Type: {unit.get('type')}\n"
ref_all_executor/standing_loop.py L187_with_workspace_context
  183          return objective
  184      return f"{resources}\n\n{objective}"
  185  
  186  
  187  def _with_workspace_context(subject: str, workspace: str, objective: str) -> str:
  188      if "## REFALL Workspace Layout" in objective:
  189          return objective
  190      source = config.subject_workspace(subject)
  191      layout = (
  192          "## REFALL Workspace Layout\n"
  193          f"- Writable agent workspace: `{workspace}`\n"
  194          f"- Read-only subject source root: `{source}`\n"
  195          f"Run local analysis commands from the writable workspace, but inspect target files under `{source}`. "
  196          f"Do not write to `{source}`; place scratch files, extracted metadata, notes, and temporary outputs under `{workspace}`."
  197      )
  198      return f"{layout}\n\n{objective}"
  199  
  200  
  201  class AgentCanvasDispatcher:
  202      def __init__(self, oh: OHClient | None = None) -> None:
ref_all_executor/standing_loop.py L180_with_resource_context
  176              texts.append(action["message"])
  177      return "\n\n".join(texts)
  178  
  179  
  180  def _with_resource_context(objective: str) -> str:
  181      resources = config.ssh_resource_instructions()
  182      if not resources or "## Remote SSH Resources" in objective:
  183          return objective
  184      return f"{resources}\n\n{objective}"
  185  
  186  
  187  def _with_workspace_context(subject: str, workspace: str, objective: str) -> str:
  188      if "## REFALL Workspace Layout" in objective:
  189          return objective
  190      source = config.subject_workspace(subject)
  191      layout = (
  192          "## REFALL Workspace Layout\n"
  193          f"- Writable agent workspace: `{workspace}`\n"
  194          f"- Read-only subject source root: `{source}`\n"
  195          f"Run local analysis commands from the writable workspace, but inspect target files under `{source}`. "
arcticbase/backend/src/arctic_base/refall/coverage_store.py L974CoverageStore.update_finding_validation
  970              "by_severity": by_severity,
  971              "by_class": by_class,
  972          }
  973  
  974      def update_finding_validation(self, finding_id: str, agent: str, fields: dict) -> dict:
  975          current = self.get_finding(finding_id)
  976          if current is None:
  977              raise ValueError("no such finding")
  978          if fields.get("status") is not None:
  979              raise ValueError("finding status changes must use disposition transitions")
  980          allowed = {
  981              "production_viability",
  982              "repro_status",
  983              "patch_status",
  984              "critic_reasoning",
  985          }
  986          updates = {key: value for key, value in fields.items() if key in allowed and value is not None}
  987          history = {
  988              "agent": agent,
  989              "ts": _now(),
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1346CoverageStore.trigger_pending_actions_for_runtime_event
 1342                  "created_by": "refall-disposition",
 1343              }))
 1344          return created
 1345  
 1346      def trigger_pending_actions_for_runtime_event(self, subject: str, event: dict) -> list[dict]:
 1347          rule = EVENT_PENDING_RULES.get(str(event.get("type") or ""))
 1348          if not rule:
 1349              return []
 1350          payload = event.get("payload") if isinstance(event.get("payload"), dict) else {}
 1351          finding_id = payload.get("finding_id") or payload.get("source_finding_id")
 1352          unit_id = event.get("task_id") or payload.get("unit_id") or payload.get("source_unit_id")
 1353          title = str(payload.get("title") or payload.get("summary") or rule["title"])
 1354          return [self.create_pending_action(subject, {
 1355              "kind": rule["kind"],
 1356              "title": title,
 1357              "description": payload.get("description") or payload.get("summary") or "",
 1358              "source_event_id": event.get("id"),
 1359              "source_finding_id": finding_id,
 1360              "source_unit_id": unit_id,
 1361              "priority": rule["priority"],
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L570_closeout_subject
  566      except urllib.error.URLError as exc:
  567          raise HTTPException(503, f"OpenHands Agent Canvas API is not reachable: {exc.reason}") from exc
  568  
  569  
  570  def _closeout_subject(subject: str, storage: FilesystemStorage) -> dict:
  571      storage.get_workbench(subject)
  572      store = store_for_workbench(storage, subject)
  573      coverage = store.coverage(subject)
  574      ledger_stats = ledger.stats(storage, subject)
  575      report = _ensure_current_report(storage, subject, coverage, ledger_stats)
  576      closeout_gate = evaluate_closeout_gate(storage, subject)
  577      if not closeout_gate.get("passed"):
  578          entry = _append_ops_event(
  579              storage,
  580              subject,
  581              action="loop.closeout.blocked",
  582              summary="REFALL closeout blocked by required gates",
  583              status="blocked",
  584              coverage=coverage,
  585              ledger_stats=ledger_stats,
arcticbase/backend/src/arctic_base/api/refall_closeout.py L33override_closeout_gate
   29      return evaluate_closeout_gate(storage, slug, threshold=threshold)
   30  
   31  
   32  @router.post("/override", operation_id="overrideRefallCloseoutGate")
   33  def override_closeout_gate(
   34      slug: str,
   35      body: Annotated[CloseoutOverrideReq, Body()],
   36      storage: Annotated[FilesystemStorage, Depends(get_storage)],
   37  ) -> dict:
   38      if not body.receipt or not body.check_ids:
   39          raise HTTPException(422, "closeout override requires check_ids and receipt")
   40      entry = ledger.append_entry(
   41          storage,
   42          slug,
   43          {
   44              "kind": "ops-event",
   45              "subject": slug,
   46              "action": "closeout.override",
   47              "status": "accepted",
   48              "created_by": body.operator,
arcticbase/backend/src/arctic_base/refall/closeout_gate.py L421_apply_override
  417              overrides[str(check_id)] = entry
  418      return overrides
  419  
  420  
  421  def _apply_override(check: dict[str, Any], overrides: dict[str, dict]) -> dict[str, Any]:
  422      override = overrides.get(str(check.get("id") or ""))
  423      if not override:
  424          return check
  425      updated = dict(check)
  426      updated["overridden"] = True
  427      updated["override_ledger_id"] = override.get("id")
  428      updated["override_reason"] = override.get("summary") or override.get("reason")
  429      if updated.get("status") == "fail":
  430          updated["status"] = "warn"
  431          updated["blocking"] = False
  432      return updated
ref_all_executor/standing_loop.py L205start
  201  class AgentCanvasDispatcher:
  202      def __init__(self, oh: OHClient | None = None) -> None:
  203          self.oh = oh or OHClient()
  204  
  205      def start(self, task: AgentTask) -> DispatchedConversation:
  206          workspace = config.prepare_subject_agent_workspace(
  207              task.subject,
  208              lane=task.lane,
  209              task_class=task.task_class,
  210              unit=task.unit,
  211              objective=task.objective,
  212          )
  213          workspace_host = str(config.agent_workspace_host_path(
  214              task.subject, lane=task.lane, task_class=task.task_class,
  215              unit=task.unit, objective=task.objective,
  216          ))
  217          objective = _with_workspace_context(task.subject, workspace, _with_resource_context(task.objective))
  218          cid = self.oh.start_conversation(
  219              workspace=config.local_workspace(workspace),
  220              agent_settings=config.LANES[task.lane],
ref_all_executor/standing_loop.py L1115_incremental_ingest
 1111              return None
 1112          return {"facts": len(new.facts), "leads": len(new.leads),
 1113                  "findings": len(new.findings), "flows": len(new.flows)}
 1114  
 1115      def _incremental_ingest(self, conversation: DispatchedConversation) -> None:
 1116          """Pull intel a still-running agent has already emitted, in near real time.
 1117  
 1118          Two live sources: the agent's conversation event stream (every tagged
 1119          response so far) and any files it has saved to its workspace. Both are
 1120          parsed and de-duplicated, so intel appears as it is produced rather than
 1121          only at completion.
 1122          """
 1123          fresh = ParsedOutput()
 1124          stream_text = read_conversation_agent_text(conversation.cid)
 1125          if stream_text.strip():
 1126              stream = parse_agent_output(stream_text, self._submission_defaults(conversation))
 1127              fresh.facts.extend(stream.facts)
 1128              fresh.leads.extend(stream.leads)
 1129              fresh.findings.extend(stream.findings)
 1130              fresh.flows.extend(stream.flows)
ref_all_executor/standing_loop.py L1378_finish_conversation
 1374              "type task_result_submit with status done and concrete source_anchors or artifact_refs when "
 1375              "resolved, or type control_submit if operator input or a capability is still required.",
 1376          ] if part)
 1377  
 1378      def _finish_conversation(
 1379          self,
 1380          conversation: DispatchedConversation,
 1381          result: PollResult,
 1382      ) -> DispatchedConversation | None:
 1383          outcome = classify_invocation(
 1384              result.status,
 1385              result.response or "",
 1386              require_structured_submit=_require_structured_submit(),
 1387              attempt=conversation.attempt,
 1388              max_attempts=_retry_max_attempts(),
 1389          )
 1390          # Parse the agent output once. A duplicate structured submission (same
 1391          # submission_id re-applied on a retry) must not re-persist, so that guard
 1392          # runs BEFORE salvage. Otherwise, salvage intel from EVERY invocation --
 1393          # an agent stopped at max_iterations, or one that ended without a clean
ref_all_executor/standing_loop.py L1711_maybe_write_determinations
 1707              },
 1708          )
 1709          return len(files)
 1710  
 1711      def _maybe_write_determinations(self, cycle: int) -> None:
 1712          """Compile the central 'what is every file, and what did we learn' table.
 1713  
 1714          This runs every report cycle regardless of findings, so even a pure
 1715          characterization run produces a durable, queryable roll-up instead of
 1716          results scattered across unit rows and ledger entries. The units table
 1717          already holds per-file determinations (summary/sensitivity); this
 1718          surfaces them as one artifact plus module/evidence progress.
 1719          """
 1720          if self.config.report_interval_cycles <= 0 or cycle % self.config.report_interval_cycles:
 1721              return
 1722          if not hasattr(self.arcticbase, "create_runtime_artifact"):
 1723              return
 1724          try:
 1725              coverage = self._safe_coverage()
 1726              ledger_stats = self.arcticbase.ledger_stats(self.workbench)
ref_all_executor/standing_loop.py L1769_maybe_refresh_report
 1765                  "determinations_digest_error",
 1766                  {"subject": self.subject, "error": str(exc)[:300]},
 1767              )
 1768  
 1769      def _maybe_refresh_report(self, cycle: int) -> None:
 1770          if self.config.report_interval_cycles <= 0 or cycle % self.config.report_interval_cycles:
 1771              return
 1772          stats = self.arcticbase.ledger_stats(self.workbench)
 1773          by_kind = stats.get("by_kind") or {}
 1774          if self._suppress_findings_for_subject():
 1775              mapping_intel = sum(int(by_kind.get(kind, 0) or 0) for kind in ("fact", "lead", "flow", "artifact"))
 1776              if mapping_intel <= 0:
 1777                  return
 1778          elif by_kind.get("finding", 0) <= 0:
 1779              return
 1780          try:
 1781              draft = self.draft_reporter(self.workbench, self.subject)
 1782              self.arcticbase.append_ledger(
 1783                  self.workbench,
 1784                  {
ref_all_executor/standing_loop.py L1804_maybe_dispatch_report_agent
 1800          except Exception:
 1801              return self.last_report_intel
 1802          return int(by_kind.get("fact", 0)) + int(by_kind.get("lead", 0)) + int(by_kind.get("finding", 0))
 1803  
 1804      def _maybe_dispatch_report_agent(self, cycle: int, active: list[DispatchedConversation]) -> None:
 1805          """Reserve one slot for a report-writing agent when enough new intel exists.
 1806  
 1807          The default policy is triggered by accumulated intel volume, not cycle
 1808          count, so the report keeps pace with discovery without interrupting a
 1809          running analysis agent. Subjects with `refall_report_agent_continuous`
 1810          reserve one slot and immediately replace a completed report agent. The
 1811          agent enriches the living report AND maintains a running analyst-notes
 1812          digest from canonical evidence.
 1813          """
 1814          if any(getattr(conversation, "task_class", None) == "report" for conversation in active):
 1815              return
 1816          threshold = self.config.report_agent_intel_threshold
 1817          if len(active) >= self.config.max_parallel:
 1818              return
 1819          intel = self._intel_count()
ref_all_executor/standing_loop.py L2163_write_state
 2159              return json.loads(path.read_text(encoding="utf-8"))
 2160          except json.JSONDecodeError:
 2161              return {"iteration": 0, "active": [], "last_completion_cycle": 0}
 2162  
 2163      def _write_state(
 2164          self,
 2165          cycle: int,
 2166          active: list[DispatchedConversation],
 2167          status: str,
 2168          last_completion_cycle: int,
 2169      ) -> dict:
 2170          coverage = self._safe_coverage()
 2171          state = {
 2172              "status": status,
 2173              "subject": self.subject,
 2174              "workbench": self.workbench,
 2175              "run_id": self.run_id,
 2176              "iteration": cycle,
 2177              "active": [asdict(item) for item in active],
 2178              "active_count": len(active),
ref_all_executor/standing_loop.py L429–440_save_module_progress
  425              return json.loads(self._module_progress_path().read_text())
  426          except Exception:
  427              return {}
  428  
  429      def _save_module_progress(self) -> None:
  430          try:
  431              path = self._module_progress_path()
  432              path.parent.mkdir(parents=True, exist_ok=True)
  433              state = self.scheduler.to_state()
  434              state["last_report_intel"] = self.last_report_intel
  435              path.write_text(json.dumps(state, indent=2, sort_keys=True))
  436          except Exception as exc:
  437              self._append_event(
  438                  "module_progress_save_failed",
  439                  {"subject": self.subject, "error": str(exc)[:300]},
  440              )
ref_all_executor/work_modules.py L142–157PACKS.security_assessment
  138          _m(id="re_review", task_class="review", after=["re_decompose"], priority=60,
  139             pack="reverse_engineering",
  140             objective="Review the decomposition for correctness and gaps. What is still opaque?"),
  141      ],
  142      "security_assessment": [
  143          _m(id="sec_recon", task_class="recon", after=["characterize"], priority=30,
  144             pack="security_assessment",
  145             objective="Identify the attack surface reachable through this unit: inputs, "
  146                       "parsers, IPC/network entry, privilege boundaries, persistence, and "
  147                       "trust assumptions. Record negative results explicitly."),
  148          _m(id="sec_review", task_class="review", after=["sec_recon"], priority=45,
  149             pack="security_assessment",
  150             objective="Assess this unit for security vulnerabilities against the surface "
  151                       "recon identified. File findings as PROVISIONALLY_VALID with file:line "
  152                       "evidence, impact, and likelihood. State negative results too."),
  153          _m(id="sec_verify", task_class="verify", source=SOURCE_FINDINGS, priority=20,
  154             pack="security_assessment",
  155             objective="Attempt to reproduce this finding. Give exact repro steps, PoC status, "
  156                       "observed vs expected behaviour, and whether the claim is verified."),
  157      ],
ref_all_executor/work_modules.py L211ModuleRegistry
  207  # ---------------------------------------------------------------- registry
  208  
  209  
  210  @dataclass
  211  class ModuleRegistry:
  212      """The set of modules active for one subject."""
  213  
  214      modules: list[WorkModule] = field(default_factory=list)
  215  
  216      @classmethod
  217      def from_config(cls, config: dict[str, Any] | None) -> "ModuleRegistry":
  218          """Build from a subject's module config.
  219  
  220          Shape (all optional)::
  221  
  222              {
  223                "packs": ["baseline", "security_assessment"],
  224                "modules": [ {...custom subloop module...} ],
  225                "disabled": ["sec_verify"],
  226                "overrides": {"characterize": {"priority": 5}}
ref_all_executor/standing_loop.py L653_task_for_characterize
  649              if conversation.module_id and conversation.module_target:
  650                  pairs.add((conversation.module_id, conversation.module_target))
  651          return pairs
  652  
  653      def _task_for_characterize(
  654          self,
  655          unit: dict,
  656          active: list[DispatchedConversation] | None = None,
  657      ) -> AgentTask:
  658          """The breadth module: claim a new unit and characterize it."""
  659          module = next(iter(self.scheduler.breadth_modules()), None)
  660          task_class = "characterize"
  661          lane, reason = self._route_with_dispatch_pattern(task_class, active)
  662          extra = f"## Module\n{module.objective}" if module and module.objective else ""
  663          objective = self._objective(task_class, reason, unit, extra=extra)
  664          return AgentTask(
  665              subject=self.subject,
  666              task_class=task_class,
  667              lane=lane,
  668              unit=unit,
ref_all_executor/work_modules.py L235–249ModuleRegistry executable default
  231          """
  232          config = config or {}
  233          if not isinstance(config, dict):
  234              raise ModuleError("module config must be an object")
  235  
  236          selected: dict[str, WorkModule] = {}
  237          packs = config.get("packs")
  238          if packs is None:
  239              # A subject with no declared intent gets a security assessment, since
  240              # that is REFALL's most common job. `security_assessment` auto-pulls
  241              # `characterize` (baseline) through its dependencies, so this stays a
  242              # superset of the old characterize-only behaviour, not a replacement.
  243              packs = ["security_assessment"]
  244          if not isinstance(packs, list):
  245              raise ModuleError("packs must be a list")
  246          for pack in packs:
  247              if pack not in PACKS:
  248                  raise ModuleError(f"unknown pack {pack!r}; known: {pack_names()}")
  249              for module in PACKS[pack]:
ref_all_executor/standing_loop.py L676_task_for_unit_module
  672              module_target=str(unit.get("id")),
  673              submits_coverage=True,
  674          )
  675  
  676      def _task_for_unit_module(
  677          self,
  678          module: WorkModule,
  679          unit: dict,
  680          active: list[DispatchedConversation] | None = None,
  681      ) -> AgentTask:
  682          """A follow-on module (recon/review/decompose) on a characterized unit."""
  683          lane, reason = self._route_with_dispatch_pattern(module.task_class, active)
  684          # Do NOT waste the resources already spent characterizing this unit: point
  685          # the agent at the prior determination and artifacts and require it to
  686          # build on them rather than re-read the target from scratch.
  687          reuse = (
  688              "## Build On Prior Work (do not redo)\n"
  689              f"This unit ({unit.get('path') or unit.get('name')}) has already been "
  690              "characterized and may have prior findings, flows, and saved analysis "
  691              "artifacts. Before analyzing:\n"
ref_all_executor/standing_loop.py L2457_persist_findings
 2453                  "created_by": action.get("created_by") or "refall-standing-loop",
 2454              }
 2455              self.arcticbase.create_pending_action(self.workbench, payload)
 2456  
 2457      def _persist_findings(self, parsed: ParsedOutput, conversation: DispatchedConversation) -> None:
 2458          if not parsed.findings or not hasattr(self.coverage, "add_finding"):
 2459              return
 2460          for finding in parsed.findings:
 2461              payload = _finding_payload_for_store(finding, parsed, conversation)
 2462              if not payload:
 2463                  continue
 2464              try:
 2465                  self.coverage.add_finding(self.workbench, payload)
 2466              except RuntimeError as exc:
 2467                  if "finding promotion refused" in str(exc):
 2468                      self._record_finding_promotion_refusal(payload, parsed, conversation, str(exc))
 2469                      continue
 2470                  self.arcticbase.append_ledger(
 2471                      self.workbench,
 2472                      {
ref_all_executor/work_modules.py L493next_finding_module
  489              if best is None or eligible[0].priority < best[0].priority:
  490                  best = (eligible[0], unit)
  491          return best
  492  
  493      def next_finding_module(
  494          self, in_flight: dict[str, int],
  495          in_flight_targets: set[tuple[str, str]] | None = None,
  496      ) -> tuple[WorkModule, dict[str, Any]] | None:
  497          for finding_id, finding in self.findings.items():
  498              eligible = eligible_modules(
  499                  self.registry, target_kind=SOURCE_FINDINGS,
  500                  completed=self._done_set(finding_id), path=finding.get("path"), in_flight=in_flight,
  501                  target=finding_id, in_flight_targets=in_flight_targets,
  502              )
  503              if eligible:
  504                  return eligible[0], finding
  505          return None
  506  
  507  
  508  def describe(registry: ModuleRegistry) -> str:
ref_all_executor/standing_loop.py L744_task_for_finding_module
  740              module_target=SUBJECT_MODULE_TARGET,
  741              submits_coverage=False,
  742          )
  743  
  744      def _task_for_finding_module(
  745          self,
  746          module: WorkModule,
  747          finding: dict,
  748          active: list[DispatchedConversation] | None = None,
  749      ) -> AgentTask:
  750          lane, reason = self._route_with_dispatch_pattern(module.task_class, active)
  751          context_md = str(self._coverage_context().get("info_md") or "").strip()
  752          objective = "\n\n".join(part for part in [
  753              f"## REFALL context.md\n{context_md}" if context_md else "",
  754              self._blackboard_text(),
  755              f"## Task\nRun {module.task_class} for a REFALL finding on subject {self.subject}.",
  756              f"## Routing\n{reason}",
  757              "## Finding\n"
  758              f"- ID: {finding.get('id')}\n"
  759              f"- Title: {finding.get('title')}\n"
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1008transition_finding
 1004          if updated is None:
 1005              raise ValueError("no such finding")
 1006          return updated
 1007  
 1008      def transition_finding(self, subject: str, finding_id: str, data: dict) -> dict:
 1009          current = self.get_finding(finding_id)
 1010          if current is None or current.get("subject") != subject:
 1011              raise ValueError("no such finding")
 1012          from_status = str(data.get("from_status") or current.get("status") or "")
 1013          to_status = str(data.get("to_status") or current.get("status") or "")
 1014          from_quality = str(data.get("from_quality") or current.get("quality_tier") or "HYPOTHESIS")
 1015          to_quality = str(data.get("to_quality") or from_quality)
 1016          receipt_type = data.get("receipt_type")
 1017          receipt = data.get("receipt") or {}
 1018          duplicate_of = data.get("duplicate_of")
 1019          reasons: list[str] = []
 1020  
 1021          if from_status and from_status != current.get("status"):
 1022              reasons.append("from_status does not match current state")
 1023          if from_status and from_status not in FINDING_STATUSES:
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1490–1512add_wishlist_item
 1486              "refusal_reasons": sorted(set(reasons)),
 1487              "request": request,
 1488          }
 1489  
 1490      def add_wishlist_item(self, subject: str, data: dict) -> dict:
 1491          now = _now()
 1492          if not data.get("item"):
 1493              raise ValueError("wishlist item required")
 1494          item_id = str(data.get("id") or f"wish_{uuid.uuid4().hex}")
 1495          with self._wlock:
 1496              self.conn.execute(
 1497                  "INSERT INTO wishlist(id,subject,kind,item,reason,requested_by,status,created_at,updated_at) "
 1498                  "VALUES(?,?,?,?,?,?,?,?,?)",
 1499                  (
 1500                      item_id,
 1501                      subject,
 1502                      data.get("kind"),
 1503                      data.get("item"),
 1504                      data.get("reason"),
 1505                      data.get("requested_by"),
 1506                      data.get("status", "open"),
 1507                      now,
 1508                      now,
 1509                  ),
 1510              )
 1511              self.conn.commit()
 1512          return self.get_wishlist_item(item_id) or {"id": item_id, "subject": subject}
ref_all_executor/watchdog.py L995_install_wishlist_tool
  991              },
  992          )
  993          return {"source": source_item, "created": created, "ledger": ledger, "receipt": receipt}
  994  
  995      def _install_wishlist_tool(self, tool: str, commands: list[list[str]]) -> dict[str, Any]:
  996          attempts = []
  997          for command in commands:
  998              try:
  999                  proc = subprocess.run(
 1000                      command,
 1001                      capture_output=True,
 1002                      text=True,
 1003                      timeout=self.wishlist_install_timeout_s,
 1004                      check=False,
 1005                  )
 1006              except Exception as exc:
 1007                  attempts.append({"command": command, "ok": False, "error": repr(exc)})
 1008                  continue
 1009              attempts.append({
 1010                  "command": command,
ref_all_executor/wishlist.py L4format_provided_wishlist
    1  from urllib.parse import quote
    2  
    3  
    4  def format_provided_wishlist(subject, provided_wishes):
    5      endpoint = f"/wishlist?subject={quote(subject)}&status=provided"
    6      lines = [
    7          "DELIVERED WISHLIST ITEMS:",
    8          f"- Check {endpoint} during preflight for the live delivered-tool/access state.",
    9      ]
   10      wishes = list(provided_wishes or [])
   11      if not wishes:
   12          lines.append("- No delivered wishlist items are recorded yet.")
   13      else:
   14          lines.append("- Use these before requesting duplicate tooling. Do not re-request matching items.")
   15          for wish in wishes:
   16              kind = wish.get("kind") or "item"
   17              item = wish.get("item") or "(unnamed)"
   18              requested_by = wish.get("requested_by") or "unknown agent"
   19              reason = wish.get("reason") or "no reason recorded"
ref_all_executor/standing_loop.py L1987_claim_pending_action
 1983              if "404" in str(exc) or "no open units" in str(exc).lower():
 1984                  return None
 1985              raise
 1986  
 1987      def _claim_pending_action(self) -> dict | None:
 1988          if not (
 1989              hasattr(self.arcticbase, "list_pending_actions")
 1990              and hasattr(self.arcticbase, "claim_pending_action")
 1991          ):
 1992              return None
 1993          try:
 1994              actions = self.arcticbase.list_pending_actions(self.workbench, include_terminal=False)
 1995          except RuntimeError as exc:
 1996              if "404" in str(exc):
 1997                  return None
 1998              raise
 1999          if not isinstance(actions, list):
 2000              return None
 2001          now = time.time()
 2002          candidates = [
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1458authorize_execution
 1454              )
 1455              self.conn.commit()
 1456          return self.get_capability_grant(grant_id) or grant
 1457  
 1458      def authorize_execution(self, subject: str, request: dict) -> dict:
 1459          capability_class = str(request.get("capability_class") or request.get("class") or "")
 1460          declared_effects = request.get("declared_effects") or []
 1461          reasons: list[str] = []
 1462          if not capability_class:
 1463              reasons.append("missing capability_class")
 1464          if not declared_effects:
 1465              reasons.append("request omits declared effects")
 1466          grants = [
 1467              grant for grant in self.list_capability_grants(subject)
 1468              if grant.get("class") == capability_class and not _grant_expired(grant)
 1469          ]
 1470          matched: dict | None = None
 1471          for grant in grants:
 1472              grant_reasons = _grant_refusal_reasons(grant, request)
 1473              if not grant_reasons:
ref_all_executor/config.py L207prepare_subject_agent_workspace
  203      root = os.environ.get("REFALL_AGENT_WORKSPACE_CONTAINER_ROOT", AGENT_WORKSPACE_CONTAINER_ROOT_DEFAULT).rstrip("/")
  204      return f"{root}/{_safe_path_component(subject)}/{agent_workspace_name(lane=lane, task_class=task_class, unit=unit, objective=objective)}"
  205  
  206  
  207  def prepare_subject_agent_workspace(
  208      subject,
  209      *,
  210      lane: str = "",
  211      task_class: str = "",
  212      unit: dict | None = None,
  213      objective: str = "",
  214  ) -> str:
  215      """Create the host-side writable workspace and return its container path."""
  216      host_root = Path(os.environ.get("REFALL_AGENT_WORKSPACE_HOST_ROOT", AGENT_WORKSPACE_HOST_ROOT_DEFAULT)).expanduser()
  217      subject_name = _safe_path_component(subject)
  218      workspace_name = agent_workspace_name(lane=lane, task_class=task_class, unit=unit, objective=objective)
  219      (host_root / subject_name / workspace_name).mkdir(parents=True, exist_ok=True)
  220      return subject_agent_workspace(
  221          subject,
  222          lane=lane,
ref_all_executor/standing_loop.py L676RefallStandingLoop._task_for_unit_module
  672              module_target=str(unit.get("id")),
  673              submits_coverage=True,
  674          )
  675  
  676      def _task_for_unit_module(
  677          self,
  678          module: WorkModule,
  679          unit: dict,
  680          active: list[DispatchedConversation] | None = None,
  681      ) -> AgentTask:
  682          """A follow-on module (recon/review/decompose) on a characterized unit."""
  683          lane, reason = self._route_with_dispatch_pattern(module.task_class, active)
  684          # Do NOT waste the resources already spent characterizing this unit: point
  685          # the agent at the prior determination and artifacts and require it to
  686          # build on them rather than re-read the target from scratch.
  687          reuse = (
  688              "## Build On Prior Work (do not redo)\n"
  689              f"This unit ({unit.get('path') or unit.get('name')}) has already been "
  690              "characterized and may have prior findings, flows, and saved analysis "
  691              "artifacts. Before analyzing:\n"
ref_all_executor/standing_loop.py L1073RefallStandingLoop._persist_new_intel
 1069                      json.dumps(material or item, sort_keys=True, default=str).encode("utf-8")
 1070                  ).hexdigest()
 1071          return (kind, ident)
 1072  
 1073      def _persist_new_intel(self, conversation: DispatchedConversation, parsed: "ParsedOutput") -> dict:
 1074          """Persist only intel not seen before this run -- shared by the real-time
 1075          incremental path and the terminal salvage path, so an item that lands live
 1076          is never re-persisted at completion. Dedup is by stable identity, not by
 1077          object identity, so re-reading the same workspace file is safe.
 1078          """
 1079          new = ParsedOutput()
 1080          staged_keys: list[tuple] = []
 1081          for kind, src, dst in (
 1082              ("finding", parsed.findings, new.findings),
 1083              ("fact", parsed.facts, new.facts),
 1084              ("lead", parsed.leads, new.leads),
 1085              ("flow", parsed.flows, new.flows),
 1086          ):
 1087              for item in src or []:
 1088                  if not isinstance(item, dict):
ref_all_executor/standing_loop.py L2691RefallStandingLoop._structured_submissions_already_applied
 2687              )
 2688          except Exception:
 2689              pass
 2690  
 2691      def _structured_submissions_already_applied(self, keys: list[str]) -> bool:
 2692          try:
 2693              conn = connect_runtime(self.subject_path)
 2694              return all(
 2695                  conn.execute("SELECT 1 FROM control_signals WHERE id=?", (_submission_marker_id(key),)).fetchone()
 2696                  for key in keys
 2697              )
 2698          except Exception:
 2699              return False
 2700  
 2701      def _record_structured_submissions(self, keys: list[str]) -> None:
 2702          try:
 2703              conn = connect_runtime(self.subject_path)
 2704              now = time.time()
 2705              for key in keys:
 2706                  conn.execute(
ref_all_executor/standing_loop.py L2701RefallStandingLoop._record_structured_submissions
 2697              )
 2698          except Exception:
 2699              return False
 2700  
 2701      def _record_structured_submissions(self, keys: list[str]) -> None:
 2702          try:
 2703              conn = connect_runtime(self.subject_path)
 2704              now = time.time()
 2705              for key in keys:
 2706                  conn.execute(
 2707                      "INSERT OR IGNORE INTO control_signals(id,subject,kind,payload,created_at) VALUES(?,?,?,?,?)",
 2708                      (
 2709                          _submission_marker_id(key),
 2710                          self.subject,
 2711                          "structured_submission_applied",
 2712                          json.dumps({"key": key}),
 2713                          now,
 2714                      ),
 2715                  )
 2716              conn.commit()
arcticbase/backend/src/arctic_base/refall/coverage_store.py L910CoverageStore._find_canonical_finding
  906          if found is None:
  907              raise ValueError("finding was not stored")
  908          return found
  909  
  910      def _find_canonical_finding(self, subject: str, dedup_key: str) -> dict | None:
  911          row = self.conn.execute(
  912              "SELECT * FROM findings WHERE subject=? AND dedup_key=? "
  913              "AND (canonical_finding_id IS NULL OR canonical_finding_id=id) "
  914              "ORDER BY created_at ASC LIMIT 1",
  915              (subject, dedup_key),
  916          ).fetchone()
  917          return _finding_row(row) if row else None
  918  
  919      def _promotion_gate_context(self, subject: str) -> dict:
  920          context: dict[str, dict] = {}
  921          source_inventory = self.get_latest_source_inventory(subject)
  922          if source_inventory:
  923              context["source_inventory"] = source_inventory
  924          scope_overlay = self.get_scope_overlay(subject)
  925          if scope_overlay:
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1648_finding_dedup_key
 1644      merged["subject"] = existing.get("subject") or incoming.get("subject")
 1645      return merged
 1646  
 1647  
 1648  def _finding_dedup_key(subject: str, data: dict) -> str:
 1649      parts = [
 1650          subject,
 1651          _normalized_dedup_text(data.get("vuln_class") or "unknown"),
 1652          _normalized_dedup_text(data.get("title") or ""),
 1653          _normalized_dedup_text(data.get("affected_component") or data.get("component") or ""),
 1654      ]
 1655      return "dedup:" + hashlib.sha256("|".join(parts).encode("utf-8")).hexdigest()
 1656  
 1657  
 1658  def _related_locations_from_finding(data: dict) -> list[dict]:
 1659      locations: list[dict] = []
 1660      refs: list[str] = []
 1661      for key in ("code_paths", "source_anchors", "artifact_refs", "refs"):
 1662          refs.extend(str(item) for item in data.get(key) or [] if str(item).strip())
 1663      for item in data.get("evidence") or []:
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1678_merge_related_locations
 1674          locations.append({"path": _source_anchor_path(anchor) or anchor, "anchor": anchor})
 1675      return _merge_related_locations([], locations)
 1676  
 1677  
 1678  def _merge_related_locations(existing: list[dict], new_locations: list[dict]) -> list[dict]:
 1679      merged: list[dict] = []
 1680      seen: set[tuple[str, str]] = set()
 1681      for item in [*existing, *new_locations]:
 1682          if not isinstance(item, dict):
 1683              continue
 1684          path = str(item.get("path") or "").strip()
 1685          anchor = str(item.get("anchor") or item.get("ref") or path).strip()
 1686          if not anchor:
 1687              continue
 1688          normalized = {"path": path or _source_anchor_path(anchor) or anchor, "anchor": anchor}
 1689          key = (normalized["path"], normalized["anchor"])
 1690          if key in seen:
 1691              continue
 1692          seen.add(key)
 1693          merged.append(normalized)
arcticbase/backend/src/arctic_base/refall/projection.py L14project_runtime_events
   10  
   11  PROJECTOR = "refall.runtime.v1"
   12  
   13  
   14  def project_runtime_events(conn: sqlite3.Connection, subject: str) -> dict:
   15      watermark = conn.execute(
   16          "SELECT last_event_created_at FROM projection_watermarks WHERE projector=?",
   17          (PROJECTOR,),
   18      ).fetchone()
   19      since = float(watermark[0]) if watermark else -1.0
   20      events = [
   21          event for event in reversed(list_events(conn, subject, limit=10000))
   22          if float(event.get("created_at") or 0) > since
   23      ]
   24      projected = 0
   25      latest = since
   26      for event in events:
   27          latest = max(latest, float(event.get("created_at") or 0))
   28          if event.get("type") != "worker_submit":
   29              continue
ref_all_executor/work_modules.py L195pack_names
  191      ],
  192  }
  193  
  194  
  195  def pack_names() -> list[str]:
  196      return sorted(PACKS)
  197  
  198  
  199  def builtin_module(module_id: str) -> WorkModule | None:
  200      for modules in PACKS.values():
  201          for module in modules:
  202              if module.id == module_id:
  203                  return module
  204      return None
  205  
  206  
  207  # ---------------------------------------------------------------- registry
  208  
  209  
  210  @dataclass
arcticbase/backend/src/arctic_base/refall/coverage_store.py L240partition_file
  236      key = f"{subject}|{kind}|{path}|{line_start}|{line_end}|{name}"
  237      return hashlib.sha1(key.encode()).hexdigest()
  238  
  239  
  240  def partition_file(total_lines: int, symbols: list[dict]) -> list[dict]:
  241      syms = sorted(
  242          (s for s in symbols if int(s["line_start"]) <= int(s["line_end"])),
  243          key=lambda s: int(s["line_start"]),
  244      )
  245      leaves: list[dict] = []
  246      cursor = 1
  247      for symbol in syms:
  248          ls = max(1, int(symbol["line_start"]))
  249          le = min(total_lines, int(symbol["line_end"]))
  250          if le < cursor:
  251              continue
  252          ls = max(ls, cursor)
  253          if ls > cursor:
  254              leaves.append({"type": "line_range", "line_start": cursor, "line_end": ls - 1, "name": ""})
  255          leaves.append({
arcticbase/backend/src/arctic_base/refall/coverage_store.py L390CoverageStore.claim_next
  386                  )
  387              self.conn.commit()
  388          return {"dimension": dimension, "items": len(items), "covered": len(covered_set)}
  389  
  390      def claim_next(self, subject: str, agent: str, lease_s: int = 900) -> dict | None:
  391          with self._wlock:
  392              self.conn.execute("BEGIN IMMEDIATE")
  393              try:
  394                  row = self.conn.execute(
  395                      "SELECT * FROM units WHERE subject=? AND type IN ('symbol','line_range') "
  396                      "AND (state='open' OR (state='claimed' AND lease_expires < ?)) "
  397                      "ORDER BY (type='symbol') DESC, path, line_start LIMIT 1",
  398                      (subject, _now()),
  399                  ).fetchone()
  400                  if not row:
  401                      self.conn.execute("ROLLBACK")
  402                      return None
  403                  self.conn.execute(
  404                      "UPDATE units SET state='claimed', claimed_by=?, lease_expires=? WHERE id=?",
  405                      (agent, _now() + lease_s, row["id"]),
arcticbase/backend/src/arctic_base/refall/coverage_store.py L413CoverageStore.submit
  409              except Exception:
  410                  self.conn.execute("ROLLBACK")
  411                  raise
  412  
  413      def submit(
  414          self,
  415          unit_id: str,
  416          agent: str,
  417          summary: str,
  418          evidence: list[dict],
  419          sensitivity: str | None = None,
  420          data_touched: object = None,
  421          flows_in: list | None = None,
  422          flows_out: list | None = None,
  423          leads: list | None = None,
  424      ) -> dict:
  425          evidence = list(evidence or [])
  426          if not evidence or not all(item.get("file") for item in evidence):
  427              raise ValueError("submit requires >=1 evidence anchor with a file")
  428          return self._resolve(
arcticbase/backend/src/arctic_base/refall/coverage_store.py L504CoverageStore.coverage
  500                  (unit_id,),
  501              )
  502              self.conn.commit()
  503  
  504      def coverage(self, subject: str) -> dict:
  505          row = self.conn.execute(
  506              "SELECT COALESCE(SUM(total_lines),0) total, "
  507              "COALESCE(SUM(CASE WHEN state IN('characterized','dead') THEN total_lines END),0) resolved, "
  508              "COALESCE(SUM(CASE WHEN state='characterized' THEN total_lines END),0) chars, "
  509              "COALESCE(SUM(CASE WHEN state='dead' THEN total_lines END),0) dead "
  510              "FROM units WHERE subject=? AND type IN('symbol','line_range')",
  511              (subject,),
  512          ).fetchone()
  513          files = self.conn.execute(
  514              "SELECT COUNT(*) FROM units WHERE subject=? AND type='file'",
  515              (subject,),
  516          ).fetchone()[0]
  517          done = self.conn.execute(
  518              "SELECT COUNT(*) FROM units f WHERE f.subject=? AND f.type='file' AND NOT EXISTS "
  519              "(SELECT 1 FROM units c WHERE c.parent_id=f.id AND c.state NOT IN('characterized','dead'))",
ref_all_executor/http_retry.py L30with_retry
   26  DEFAULT_MAX_ATTEMPTS = int(os.environ.get("REFALL_HTTP_MAX_ATTEMPTS", "4"))
   27  DEFAULT_BACKOFF_S = float(os.environ.get("REFALL_HTTP_BACKOFF_S", "3"))
   28  
   29  
   30  def with_retry(
   31      transport: Callable,
   32      *,
   33      max_attempts: int | None = None,
   34      backoff_s: float | None = None,
   35      sleep: Callable[[float], None] | None = None,
   36      label: str = "refall",
   37  ) -> Callable:
   38      """Return `transport` wrapped so 5xx and socket errors retry with backoff."""
   39      attempts = DEFAULT_MAX_ATTEMPTS if max_attempts is None else max_attempts
   40      delay = DEFAULT_BACKOFF_S if backoff_s is None else backoff_s
   41      napper = sleep or time.sleep
   42  
   43      def wrapped(method, url, headers, body):
   44          for attempt in range(1, max(1, attempts) + 1):
   45              last = attempt >= max(1, attempts)
ref_all_executor/standing_loop.py L2720RefallStandingLoop._start_retry
 2716              conn.commit()
 2717          except Exception:
 2718              pass
 2719  
 2720      def _start_retry(
 2721          self,
 2722          conversation: DispatchedConversation,
 2723          outcome,
 2724          artifact_refs: list[str],
 2725      ) -> DispatchedConversation | None:
 2726          # Provider failover: if a Claude (lane A) agent balks at a task -- refuses,
 2727          # truncates, or ends without a valid submit -- retry it on Codex (lane B)
 2728          # instead of re-dispatching to the same provider. Retries are capped, so
 2729          # this switches once and cannot ping-pong.
 2730          retry_lane = _failover_lane(conversation.lane)
 2731          if retry_lane != conversation.lane:
 2732              self._append_event(
 2733                  "retry_provider_switch",
 2734                  {
 2735                      "subject": self.subject,
arcticbase/backend/src/arctic_base/refall/ledger.py L36
   32      "closeout-gate",
   33  }
   34  
   35  
   36  def ledger_path(storage: FilesystemStorage, slug: str) -> Path:
   37      storage.get_workbench(slug)
   38      return storage.root / "workbenches" / slug / "refall_ledger.jsonl"
   39  
   40  
   41  def append_entry(storage: FilesystemStorage, slug: str, entry: dict[str, Any]) -> dict[str, Any]:
   42      path = ledger_path(storage, slug)
   43      now = datetime.now(UTC).isoformat()
   44      data = dict(entry)
   45      data.setdefault("kind", "fact")
   46      if data["kind"] not in LEDGER_KINDS:
   47          raise ValueError(f"unsupported ledger kind: {data['kind']}")
   48      data.setdefault("id", f"led_{uuid.uuid4().hex}")
   49      data.setdefault("workbench_slug", slug)
   50      data.setdefault("created_at", now)
   51      data["updated_at"] = now
arcticbase/backend/src/arctic_base/api/refall_ledger.py L14
   10  
   11  router = APIRouter(prefix="/workbenches/{slug}/ledger", tags=["refall-ledger"])
   12  
   13  
   14  @router.post("/entries", status_code=201, operation_id="appendRefallLedgerEntry")
   15  def append_ledger_entry(
   16      slug: str,
   17      entry: Annotated[dict[str, Any], Body()],
   18      storage: Annotated[FilesystemStorage, Depends(get_storage)],
   19  ) -> dict[str, Any]:
   20      try:
   21          return ledger.append_entry(storage, slug, entry)
   22      except ValueError as exc:
   23          raise HTTPException(422, str(exc)) from exc
   24  
   25  
   26  @router.get("/entries", operation_id="listRefallLedgerEntries")
   27  def list_ledger_entries(
   28      slug: str,
   29      storage: Annotated[FilesystemStorage, Depends(get_storage)],
ref_all_executor/standing_loop.py L590
  586              "enumerated": enumerated,
  587              "state": final_state,
  588          }
  589  
  590      def _claim_next_task(self, active: list[DispatchedConversation] | None = None) -> AgentTask | None:
  591          # 1. Operator/manager pending actions always preempt.
  592          pending = self._claim_pending_action()
  593          if pending:
  594              if pending.get("kind") == "threat_model_request":
  595                  return self._task_for_threat_model(action=pending, active=active)
  596              return self._task_for_pending_action(pending, active=active)
  597          threat_model = self._claim_threat_model_task(active=active)
  598          if threat_model:
  599              return threat_model
  600  
  601          active = active or []
  602          in_flight = self._module_in_flight(active)
  603          in_flight_targets = self._module_in_flight_targets(active)
  604  
  605          # 2. Subject-scoped modules (identify, research, toolchain, synthesis):
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1458
 1454              )
 1455              self.conn.commit()
 1456          return self.get_capability_grant(grant_id) or grant
 1457  
 1458      def authorize_execution(self, subject: str, request: dict) -> dict:
 1459          capability_class = str(request.get("capability_class") or request.get("class") or "")
 1460          declared_effects = request.get("declared_effects") or []
 1461          reasons: list[str] = []
 1462          if not capability_class:
 1463              reasons.append("missing capability_class")
 1464          if not declared_effects:
 1465              reasons.append("request omits declared effects")
 1466          grants = [
 1467              grant for grant in self.list_capability_grants(subject)
 1468              if grant.get("class") == capability_class and not _grant_expired(grant)
 1469          ]
 1470          matched: dict | None = None
 1471          for grant in grants:
 1472              grant_reasons = _grant_refusal_reasons(grant, request)
 1473              if not grant_reasons:
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1008
 1004          if updated is None:
 1005              raise ValueError("no such finding")
 1006          return updated
 1007  
 1008      def transition_finding(self, subject: str, finding_id: str, data: dict) -> dict:
 1009          current = self.get_finding(finding_id)
 1010          if current is None or current.get("subject") != subject:
 1011              raise ValueError("no such finding")
 1012          from_status = str(data.get("from_status") or current.get("status") or "")
 1013          to_status = str(data.get("to_status") or current.get("status") or "")
 1014          from_quality = str(data.get("from_quality") or current.get("quality_tier") or "HYPOTHESIS")
 1015          to_quality = str(data.get("to_quality") or from_quality)
 1016          receipt_type = data.get("receipt_type")
 1017          receipt = data.get("receipt") or {}
 1018          duplicate_of = data.get("duplicate_of")
 1019          reasons: list[str] = []
 1020  
 1021          if from_status and from_status != current.get("status"):
 1022              reasons.append("from_status does not match current state")
 1023          if from_status and from_status not in FINDING_STATUSES:
arcticbase/backend/src/arctic_base/refall/closeout_gate.py L14
   10  from arctic_base.refall.runtime_state import connect_runtime, list_artifacts, list_events
   11  from arctic_base.storage.filesystem import FilesystemStorage
   12  
   13  
   14  def evaluate_closeout_gate(
   15      storage: FilesystemStorage,
   16      slug: str,
   17      *,
   18      threshold: float = 0.95,
   19  ) -> dict[str, Any]:
   20      store = store_for_workbench(storage, slug)
   21      coverage = store.coverage(slug)
   22      findings = store.list_findings(slug, limit=10000)
   23      pending = store.list_pending_actions(slug, include_terminal=True)
   24      transitions = store.list_transitions(slug)
   25      entries = ledger.list_entries(storage, slug)
   26      subject_path = _subject_path(storage, slug)
   27      runtime_events, runtime_artifacts = _runtime_records(subject_path, slug)
   28      overrides = _closeout_overrides(entries)
   29      checks: list[dict[str, Any]] = []
arcticbase/backend/src/arctic_base/refall/coverage_store.py L390
  386                  )
  387              self.conn.commit()
  388          return {"dimension": dimension, "items": len(items), "covered": len(covered_set)}
  389  
  390      def claim_next(self, subject: str, agent: str, lease_s: int = 900) -> dict | None:
  391          with self._wlock:
  392              self.conn.execute("BEGIN IMMEDIATE")
  393              try:
  394                  row = self.conn.execute(
  395                      "SELECT * FROM units WHERE subject=? AND type IN ('symbol','line_range') "
  396                      "AND (state='open' OR (state='claimed' AND lease_expires < ?)) "
  397                      "ORDER BY (type='symbol') DESC, path, line_start LIMIT 1",
  398                      (subject, _now()),
  399                  ).fetchone()
  400                  if not row:
  401                      self.conn.execute("ROLLBACK")
  402                      return None
  403                  self.conn.execute(
  404                      "UPDATE units SET state='claimed', claimed_by=?, lease_expires=? WHERE id=?",
  405                      (agent, _now() + lease_s, row["id"]),
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L366
  362          "loop": loop,
  363      }
  364  
  365  
  366  def _authorize_process_launch(
  367      *,
  368      storage: FilesystemStorage,
  369      slug: str,
  370      settings: Settings,
  371      subject: str,
  372      workbench: str,
  373      max_cycles: int,
  374      subject_path: Path | None,
  375      action: str,
  376  ) -> dict:
  377      store = store_for_workbench(storage, slug)
  378      request = {
  379          "requested_by": "refall-ops-command",
  380          "capability_class": "run_local_command",
  381          "command": _standing_loop_argv(settings, subject, workbench, max_cycles),
arcticbase/backend/src/arctic_base/api/refall_capabilities.py L18
   14  
   15  router = APIRouter(prefix="/workbenches/{slug}", tags=["refall-capabilities"])
   16  
   17  
   18  def require_capability_operator(
   19      request: Request,
   20      settings: Annotated[Settings, Depends(get_settings)],
   21  ) -> str | None:
   22      """Authenticate capability mutations when a host-only token is configured."""
   23      token_path = settings.refall_capability_operator_token_path
   24      if token_path is None:
   25          return None
   26      try:
   27          expected = token_path.read_text().strip()
   28      except OSError as exc:
   29          raise HTTPException(503, "capability operator credential unavailable") from exc
   30      supplied = request.headers.get("X-REFALL-Operator-Token", "")
   31      if not expected or not secrets.compare_digest(supplied, expected):
   32          raise HTTPException(403, "valid capability operator credential required")
   33      return "operator"
arcticbase/backend/src/arctic_base/refall/promotion_gates.py L12
    8  VALIDATED_QUALITIES = {"CODE_CONFIRMED", "LIVE_CONFIRMED"}
    9  VALIDATED_REPRO = {"reproduced", "confirmed", "live_reproduced"}
   10  
   11  
   12  def evaluate_finding_candidate(
   13      candidate: dict,
   14      *,
   15      source_inventory: dict | None = None,
   16      scope_overlay: dict | None = None,
   17  ) -> dict:
   18      reasons: list[str] = []
   19      warnings: list[str] = []
   20      refs = _source_refs(candidate)
   21      artifacts = _artifact_refs(candidate)
   22      evidence = _evidence_items(candidate)
   23  
   24      if not evidence and not refs and not artifacts:
   25          reasons.append("missing evidence, source anchors, or artifact refs")
   26      elif not evidence:
   27          reasons.append("missing evidence")
arcticbase/backend/src/arctic_base/refall/subjects.py L56
   52      return out[:64] or "subject"
   53  
   54  
   55  def bootstrap_subject(
   56      *,
   57      settings: Settings,
   58      storage: FilesystemStorage,
   59      subject: str,
   60      title: str | None = None,
   61      description: str = "",
   62      source_path: str | None = None,
   63      tags: list[str] | None = None,
   64  ) -> SubjectBootstrap:
   65      slug = slugify_subject(subject)
   66      subject_root = settings.refall_subject_root
   67      if subject_root is None:
   68          subject_root = settings.refall_data_root / "subjects"
   69      subject_path = (subject_root / slug).resolve()
   70  
   71      created_path = not subject_path.exists()
ref_all_executor/source_inventory.py L66
   62      "gemfile": "ruby",
   63  }
   64  
   65  
   66  def build_source_inventory(subject: str, root: Path | str) -> dict[str, Any]:
   67      root_path = Path(root).expanduser().resolve()
   68      if not root_path.is_dir():
   69          raise ValueError(f"source root is not a directory: {root_path}")
   70      files: list[dict[str, Any]] = []
   71      skipped: list[dict[str, Any]] = []
   72      observations: list[dict[str, Any]] = []
   73      for dirpath, dirnames, filenames in os.walk(root_path, followlinks=False):
   74          dirnames[:] = sorted(
   75              name for name in dirnames
   76              if name not in SKIP_DIRS and (not name.startswith(".") or name in INCLUDE_DOT_DIRS)
   77          )
   78          for filename in sorted(filenames):
   79              path = Path(dirpath) / filename
   80              rel = _relative(root_path, path)
   81              if path.is_symlink():
arcticbase/backend/src/arctic_base/refall/ledger.py L41
   37      storage.get_workbench(slug)
   38      return storage.root / "workbenches" / slug / "refall_ledger.jsonl"
   39  
   40  
   41  def append_entry(storage: FilesystemStorage, slug: str, entry: dict[str, Any]) -> dict[str, Any]:
   42      path = ledger_path(storage, slug)
   43      now = datetime.now(UTC).isoformat()
   44      data = dict(entry)
   45      data.setdefault("kind", "fact")
   46      if data["kind"] not in LEDGER_KINDS:
   47          raise ValueError(f"unsupported ledger kind: {data['kind']}")
   48      data.setdefault("id", f"led_{uuid.uuid4().hex}")
   49      data.setdefault("workbench_slug", slug)
   50      data.setdefault("created_at", now)
   51      data["updated_at"] = now
   52  
   53      path.parent.mkdir(parents=True, exist_ok=True)
   54      with path.open("a", encoding="utf-8") as fh:
   55          fh.write(json.dumps(data, sort_keys=True) + "\n")
   56      storage.audit(slug, "refall.ledger.append", id=data["id"], kind=data["kind"])
ref_all_executor/standing_loop.py L676
  672              module_target=str(unit.get("id")),
  673              submits_coverage=True,
  674          )
  675  
  676      def _task_for_unit_module(
  677          self,
  678          module: WorkModule,
  679          unit: dict,
  680          active: list[DispatchedConversation] | None = None,
  681      ) -> AgentTask:
  682          """A follow-on module (recon/review/decompose) on a characterized unit."""
  683          lane, reason = self._route_with_dispatch_pattern(module.task_class, active)
  684          # Do NOT waste the resources already spent characterizing this unit: point
  685          # the agent at the prior determination and artifacts and require it to
  686          # build on them rather than re-read the target from scratch.
  687          reuse = (
  688              "## Build On Prior Work (do not redo)\n"
  689              f"This unit ({unit.get('path') or unit.get('name')}) has already been "
  690              "characterized and may have prior findings, flows, and saved analysis "
  691              "artifacts. Before analyzing:\n"
ref_all_executor/work_modules.py L46
   42      """A module definition is not usable."""
   43  
   44  
   45  @dataclass
   46  class WorkModule:
   47      """One slot-in unit of work generation."""
   48  
   49      id: str
   50      task_class: str
   51      source: str = SOURCE_UNITS
   52      # Per-target ordering. NOT a global phase: "run me on a target only after
   53      # these modules have completed on that same target".
   54      after: list[str] = field(default_factory=list)
   55      # Coverage unit state a target must be in (None = any).
   56      unit_state: str | None = None
   57      # Only consider targets whose path matches one of these globs (None = all).
   58      path_globs: list[str] | None = None
   59      # Skip targets whose path matches any of these globs, even if path_globs
   60      # matched. Used to keep deep/expensive modules off bulk media and metadata-
   61      # only corpora (HAF/GPS/image/voice), which should be characterized but not
ref_all_executor/work_modules.py L386
  382  
  383  SUBJECT_TARGET = "__subject__"
  384  
  385  
  386  class ModuleScheduler:
  387      """Decides the next module/target to dispatch, and tracks what is done.
  388  
  389      Progress is per-target: `completed[target] = {module_ids}`. That is what
  390      makes ordering local (a module's `after` is satisfied per target) rather than
  391      a global phase. State persists so a loop restart resumes where it left off.
  392  
  393      The loop feeds the scheduler facts (a unit was characterized, a finding
  394      exists) and asks for the next thing to run. The scheduler never does I/O
  395      except reading/writing its own JSON state.
  396      """
  397  
  398      def __init__(self, registry: ModuleRegistry, state: dict[str, Any] | None = None) -> None:
  399          self.registry = registry
  400          state = state or {}
  401          # target_key -> list[module_id] completed on that target
ref_all_executor/standing_loop.py L1115
 1111              return None
 1112          return {"facts": len(new.facts), "leads": len(new.leads),
 1113                  "findings": len(new.findings), "flows": len(new.flows)}
 1114  
 1115      def _incremental_ingest(self, conversation: DispatchedConversation) -> None:
 1116          """Pull intel a still-running agent has already emitted, in near real time.
 1117  
 1118          Two live sources: the agent's conversation event stream (every tagged
 1119          response so far) and any files it has saved to its workspace. Both are
 1120          parsed and de-duplicated, so intel appears as it is produced rather than
 1121          only at completion.
 1122          """
 1123          fresh = ParsedOutput()
 1124          stream_text = read_conversation_agent_text(conversation.cid)
 1125          if stream_text.strip():
 1126              stream = parse_agent_output(stream_text, self._submission_defaults(conversation))
 1127              fresh.facts.extend(stream.facts)
 1128              fresh.leads.extend(stream.leads)
 1129              fresh.findings.extend(stream.findings)
 1130              fresh.flows.extend(stream.flows)
ref_all_executor/standing_loop.py L1378
 1374              "type task_result_submit with status done and concrete source_anchors or artifact_refs when "
 1375              "resolved, or type control_submit if operator input or a capability is still required.",
 1376          ] if part)
 1377  
 1378      def _finish_conversation(
 1379          self,
 1380          conversation: DispatchedConversation,
 1381          result: PollResult,
 1382      ) -> DispatchedConversation | None:
 1383          outcome = classify_invocation(
 1384              result.status,
 1385              result.response or "",
 1386              require_structured_submit=_require_structured_submit(),
 1387              attempt=conversation.attempt,
 1388              max_attempts=_retry_max_attempts(),
 1389          )
 1390          # Parse the agent output once. A duplicate structured submission (same
 1391          # submission_id re-applied on a retry) must not re-persist, so that guard
 1392          # runs BEFORE salvage. Otherwise, salvage intel from EVERY invocation --
 1393          # an agent stopped at max_iterations, or one that ended without a clean
ref_all_executor/standing_loop.py L1161
 1157                  "module_completion_error",
 1158                  {"subject": self.subject, "cid": conversation.cid, "error": str(exc)[:300]},
 1159              )
 1160  
 1161      def _block_empty_subject_module_completion(
 1162          self,
 1163          conversation: DispatchedConversation,
 1164          parsed: "ParsedOutput",
 1165          persistence_failed: bool = False,
 1166      ) -> bool:
 1167          """Keep subject-level custom work pending when no canonical intel landed.
 1168  
 1169          Unit modules can legitimately advance coverage through their determination
 1170          payload. Subject modules have no such independent output, so completing one
 1171          with zero facts, leads, findings, or flows would silently discard its work.
 1172          """
 1173          if not (
 1174              conversation.module_id
 1175              and conversation.module_target
 1176              and conversation.unit is None
arcticbase/backend/src/arctic_base/refall/coverage_store.py L749
  745          if not row:
  746              return None
  747          return _load_json(row["payload"], {})
  748  
  749      def add_finding(self, subject: str, data: dict) -> dict:
  750          now = _now()
  751          finding_id = str(data.get("id") or f"find_{uuid.uuid4().hex}")
  752          code_paths = data.get("code_paths") or []
  753          status = str(data.get("status") or "PROVISIONALLY_VALID")
  754          quality_tier = str(data.get("quality_tier") or "HYPOTHESIS")
  755          if not data.get("title"):
  756              raise ValueError("finding requires title")
  757          if not code_paths:
  758              raise ValueError("finding requires code_paths")
  759          if status not in FINDING_STATUSES:
  760              raise ValueError(f"bad status {status}")
  761          if status not in CREATABLE_FINDING_STATUSES:
  762              raise ValueError(f"status {status} requires disposition transition")
  763          if quality_tier not in QUALITY_ORDER:
  764              raise ValueError(f"bad quality_tier {quality_tier}")
ref_all_executor/report_draft.py L41
   37          return default
   38      return value
   39  
   40  
   41  def draft_report(
   42      workbench: str,
   43      subject: str,
   44      *,
   45      base: str | None = None,
   46      transport=None,
   47      allow_structural_downgrade: bool = False,
   48  ) -> dict:
   49      base = (base or config.WB_BASE).rstrip("/")
   50      transport = transport or _urllib_transport
   51      wb_url = f"{base}/api/workbenches/{workbench}"
   52      meta = _req(transport, "GET", wb_url)
   53      coverage = _req(transport, "GET", f"{wb_url}/coverage/stats")
   54      ledger_entries = _req(transport, "GET", f"{wb_url}/ledger/entries")
   55      ledger_stats = _req(transport, "GET", f"{wb_url}/ledger/stats")
   56      closeout_gate = _optional_req(transport, "GET", f"{wb_url}/closeout/gate", {})
ref_all_executor/report_assemble.py L16
   12  )
   13  from .workbench import _urllib_transport
   14  
   15  
   16  def assemble_report(
   17      workbench: str,
   18      subject: str,
   19      enriched: dict,
   20      *,
   21      base: str | None = None,
   22      transport=None,
   23      allow_structural_downgrade: bool = False,
   24  ) -> dict:
   25      base = (base or config.WB_BASE).rstrip("/")
   26      transport = transport or _urllib_transport
   27      template = _req(transport, "GET", f"{base}/api/refall/reports/template")
   28      if isinstance(template, bytes):
   29          template_text = template.decode("utf-8", errors="replace")
   30      else:
   31          template_text = str(template)
ref_all_executor/standing_loop.py L442
  438                  "module_progress_save_failed",
  439                  {"subject": self.subject, "error": str(exc)[:300]},
  440              )
  441  
  442      def run(self) -> dict:
  443          self._ensure_dirs()
  444          state = self._load_state()
  445          self.run_id = str(state.get("run_id") or f"run_{uuid.uuid4().hex}")
  446          self.checkpoint_ids = []
  447          self.source_inventory = None
  448          self.scope_overlay = None
  449          self.threat_model = None
  450          self.threat_model_dispatched = False
  451          active = [self._active_from_dict(item) for item in state.get("active", [])]
  452          cycle = int(state.get("iteration") or 0)
  453          completed = 0
  454          dispatched = 0
  455          last_completion_cycle = int(state.get("last_completion_cycle") or cycle)
  456          self._latest_source_inventory()
  457          if active:
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L266
  262          raise HTTPException(422, f"{command} requires a subject")
  263      return args[0].strip()
  264  
  265  
  266  def _start_subject(subject: str, settings: Settings, storage: FilesystemStorage) -> dict:
  267      boot = bootstrap_subject(settings=settings, storage=storage, subject=subject)
  268      slug = boot.workbench_slug
  269      workbench = storage.get_workbench(slug)
  270      source_path = _loop_source_path(workbench.model_dump(mode="json"), boot.subject_path)
  271      store = store_for_workbench(storage, slug)
  272      context = store.get_context(slug)
  273      if not context or not str(context.get("info_md") or "").strip():
  274          raise HTTPException(
  275              409,
  276              "Generate Context before starting REFALL; context.md is required for the head and specialist agents.",
  277          )
  278      if settings.refall_enable_loop_launch:
  279          _authorize_process_launch(
  280              storage=storage,
  281              slug=slug,
ref_all_executor/work_modules.py L460
  456      def has_breadth(self) -> bool:
  457          return bool(self.breadth_modules())
  458  
  459      # -- selection -----------------------------------------------------------
  460  
  461      def next_subject_module(
  462          self, in_flight: dict[str, int],
  463          in_flight_targets: set[tuple[str, str]] | None = None,
  464      ) -> WorkModule | None:
  465          eligible = eligible_modules(
  466              self.registry, target_kind=SOURCE_SUBJECT,
  467              completed=self._done_set(SUBJECT_TARGET), in_flight=in_flight,
  468              target=SUBJECT_TARGET, in_flight_targets=in_flight_targets,
  469          )
  470          return eligible[0] if eligible else None
  471  
  472      def next_unit_module(
  473          self, in_flight: dict[str, int],
  474          in_flight_targets: set[tuple[str, str]] | None = None,
  475      ) -> tuple[WorkModule, dict[str, Any]] | None:
ref_all_executor/standing_loop.py L1987
 1983              if "404" in str(exc) or "no open units" in str(exc).lower():
 1984                  return None
 1985              raise
 1986  
 1987      def _claim_pending_action(self) -> dict | None:
 1988          if not (
 1989              hasattr(self.arcticbase, "list_pending_actions")
 1990              and hasattr(self.arcticbase, "claim_pending_action")
 1991          ):
 1992              return None
 1993          try:
 1994              actions = self.arcticbase.list_pending_actions(self.workbench, include_terminal=False)
 1995          except RuntimeError as exc:
 1996              if "404" in str(exc):
 1997                  return None
 1998              raise
 1999          if not isinstance(actions, list):
 2000              return None
 2001          now = time.time()
 2002          candidates = [
ref_all_executor/standing_loop.py L3451
 3447      except (TypeError, ValueError):
 3448          return False
 3449  
 3450  
 3451  def _pending_action_worker_claimable(action: dict) -> bool:
 3452      """Return whether the standing loop may dispatch this action.
 3453  
 3454      Watchdog stop receipts and source-inventory promotion refusals are manager
 3455      work. Re-dispatching them to worker agents creates an endless lease/stop/
 3456      requeue cycle without changing the underlying decision or evidence.
 3457      """
 3458      kind = str(action.get("kind") or "").strip().lower()
 3459      receipt_type = str(action.get("required_receipt_type") or "").strip().lower()
 3460      description = str(action.get("description") or "").lower()
 3461      if kind == "watchdog_conversation_stopped" or receipt_type == "watchdog_stop_review":
 3462          return False
 3463      if kind == "finding_candidate_review" and (
 3464          receipt_type == "manager_review"
 3465          or "source anchor not in source inventory" in description
 3466      ):
ref_all_executor/work_modules.py L123
  119      module.validate()
  120      return module
  121  
  122  
  123  PACKS: dict[str, list[WorkModule]] = {
  124      # Breadth: build the map. Almost every subject wants this.
  125      "baseline": [
  126          _m(id="characterize", task_class="characterize", priority=100, pack="baseline",
  127             objective="Describe what this unit is and does, with file:line evidence."),
  128      ],
  129      "reverse_engineering": [
  130          _m(id="re_identify", task_class="identify", source=SOURCE_SUBJECT, priority=10,
  131             pack="reverse_engineering",
  132             objective="Identify languages, frameworks, entry points, build system, and "
  133                       "which code-understanding tools apply to this target."),
  134          _m(id="re_decompose", task_class="decompose", after=["characterize"], priority=40,
  135             pack="reverse_engineering",
  136             objective="Break this component into its constituent behaviours and data flows. "
  137                       "Name the abstractions and how control reaches them."),
  138          _m(id="re_review", task_class="review", after=["re_decompose"], priority=60,
ref_all_executor/work_modules.py L217
  213  
  214      modules: list[WorkModule] = field(default_factory=list)
  215  
  216      @classmethod
  217      def from_config(cls, config: dict[str, Any] | None) -> "ModuleRegistry":
  218          """Build from a subject's module config.
  219  
  220          Shape (all optional)::
  221  
  222              {
  223                "packs": ["baseline", "security_assessment"],
  224                "modules": [ {...custom subloop module...} ],
  225                "disabled": ["sec_verify"],
  226                "overrides": {"characterize": {"priority": 5}}
  227              }
  228  
  229          No config at all yields the baseline pack, which is exactly the loop's
  230          historical characterize-only behaviour.
  231          """
  232          config = config or {}
ref_all_executor/work_modules.py L426
  422      @staticmethod
  423      def _compact_unit(unit: dict[str, Any]) -> dict[str, Any]:
  424          return {k: unit.get(k) for k in ("id", "path", "name", "type", "line_start", "line_end")}
  425  
  426      def record_characterized(self, unit: dict[str, Any]) -> None:
  427          unit_id = str(unit.get("id") or "")
  428          if not unit_id:
  429              return
  430          self.characterized[unit_id] = self._compact_unit(unit)
  431          # Every breadth (characterize) module is done for this unit.
  432          for module in self.breadth_modules():
  433              self.mark_done(module.id, unit_id)
  434  
  435      def record_finding(self, finding: dict[str, Any]) -> None:
  436          finding_id = str(finding.get("id") or "")
  437          if finding_id:
  438              self.findings.setdefault(finding_id, {
  439                  "id": finding_id,
  440                  "path": finding.get("path") or finding.get("file"),
  441                  "title": finding.get("title"),
ref_all_executor/work_modules.py L333
  329  
  330  # --------------------------------------------------------------- selection
  331  
  332  
  333  def eligible_modules(
  334      registry: ModuleRegistry,
  335      *,
  336      target_kind: str,
  337      completed: set[str],
  338      unit_state: str | None = None,
  339      path: str | None = None,
  340      in_flight: dict[str, int] | None = None,
  341      target: str | None = None,
  342      in_flight_targets: set[tuple[str, str]] | None = None,
  343  ) -> list[WorkModule]:
  344      """Modules that may run on one target right now, best first.
  345  
  346      `completed` is the set of module ids already done **for this target**, which
  347      is what makes ordering local rather than a global phase.
  348  
ref_all_executor/work_modules.py L435
  431          # Every breadth (characterize) module is done for this unit.
  432          for module in self.breadth_modules():
  433              self.mark_done(module.id, unit_id)
  434  
  435      def record_finding(self, finding: dict[str, Any]) -> None:
  436          finding_id = str(finding.get("id") or "")
  437          if finding_id:
  438              self.findings.setdefault(finding_id, {
  439                  "id": finding_id,
  440                  "path": finding.get("path") or finding.get("file"),
  441                  "title": finding.get("title"),
  442              })
  443  
  444      def mark_done(self, module_id: str, target: str) -> None:
  445          done = self.completed.setdefault(target, [])
  446          if module_id not in done:
  447              done.append(module_id)
  448  
  449      # -- module categories ---------------------------------------------------
  450  
ref_all_executor/standing_loop.py L744
  740              module_target=SUBJECT_MODULE_TARGET,
  741              submits_coverage=False,
  742          )
  743  
  744      def _task_for_finding_module(
  745          self,
  746          module: WorkModule,
  747          finding: dict,
  748          active: list[DispatchedConversation] | None = None,
  749      ) -> AgentTask:
  750          lane, reason = self._route_with_dispatch_pattern(module.task_class, active)
  751          context_md = str(self._coverage_context().get("info_md") or "").strip()
  752          objective = "\n\n".join(part for part in [
  753              f"## REFALL context.md\n{context_md}" if context_md else "",
  754              self._blackboard_text(),
  755              f"## Task\nRun {module.task_class} for a REFALL finding on subject {self.subject}.",
  756              f"## Routing\n{reason}",
  757              "## Finding\n"
  758              f"- ID: {finding.get('id')}\n"
  759              f"- Title: {finding.get('title')}\n"
ref_all_executor/standing_loop.py L2889
 2885                  )
 2886              return True
 2887          return False
 2888  
 2889      def _apply_structured_transitions(self, parsed: ParsedOutput, conversation: DispatchedConversation) -> None:
 2890          if not parsed.transitions or not hasattr(self.arcticbase, "transition_finding"):
 2891              return
 2892          for item in parsed.transitions:
 2893              finding_id = str(item.get("finding_id") or "").strip()
 2894              transition = dict(item.get("transition") or {})
 2895              if not finding_id or not transition:
 2896                  if hasattr(self.arcticbase, "create_pending_action"):
 2897                      self.arcticbase.create_pending_action(
 2898                          self.workbench,
 2899                          {
 2900                              "kind": "operator_decision_needed",
 2901                              "title": "Structured transition needs manager review",
 2902                              "description": "Worker proposed a finding transition without a usable finding_id.",
 2903                              "source_unit_id": (conversation.unit or {}).get("id"),
 2904                              "priority": "high",
ref_all_executor/runtime_invocation.py L23
   19      last_error: str = ""
   20      next_action: str = "none"
   21  
   22  
   23  def classify_invocation(
   24      status: str,
   25      response: str,
   26      *,
   27      require_structured_submit: bool,
   28      attempt: int = 1,
   29      max_attempts: int = 2,
   30  ) -> InvocationOutcome:
   31      if status not in {"finished", "complete", "completed"}:
   32          failure_class = _status_failure_class(status)
   33          if failure_class == "capability_missing":
   34              return InvocationOutcome(
   35                  state="pending_capability",
   36                  failure_class=failure_class,
   37                  retryable=False,
   38                  attempt=attempt,
ref_all_executor/standing_loop.py L2720
 2716              conn.commit()
 2717          except Exception:
 2718              pass
 2719  
 2720      def _start_retry(
 2721          self,
 2722          conversation: DispatchedConversation,
 2723          outcome,
 2724          artifact_refs: list[str],
 2725      ) -> DispatchedConversation | None:
 2726          # Provider failover: if a Claude (lane A) agent balks at a task -- refuses,
 2727          # truncates, or ends without a valid submit -- retry it on Codex (lane B)
 2728          # instead of re-dispatching to the same provider. Retries are capped, so
 2729          # this switches once and cannot ping-pong.
 2730          retry_lane = _failover_lane(conversation.lane)
 2731          if retry_lane != conversation.lane:
 2732              self._append_event(
 2733                  "retry_provider_switch",
 2734                  {
 2735                      "subject": self.subject,
ref_all_executor/standing_loop.py L2837
 2833                  "pending_actions": 0,
 2834              },
 2835          )
 2836  
 2837      def _create_failure_pending_action(
 2838          self,
 2839          conversation: DispatchedConversation,
 2840          outcome: InvocationOutcome,
 2841          artifact_refs: list[str],
 2842      ) -> bool:
 2843          unit_id = (conversation.unit or {}).get("id")
 2844          if outcome.failure_class == "capability_missing":
 2845              capability_class = _capability_class_from_failure(outcome.last_error)
 2846              if hasattr(self.arcticbase, "request_capability"):
 2847                  self.arcticbase.request_capability(
 2848                      self.workbench,
 2849                      {
 2850                          "class": capability_class,
 2851                          "scope": _capability_scope_from_failure(outcome.last_error),
 2852                          "requested_by": conversation.cid,
ref_all_executor/standing_loop.py L1769
 1765                  "determinations_digest_error",
 1766                  {"subject": self.subject, "error": str(exc)[:300]},
 1767              )
 1768  
 1769      def _maybe_refresh_report(self, cycle: int) -> None:
 1770          if self.config.report_interval_cycles <= 0 or cycle % self.config.report_interval_cycles:
 1771              return
 1772          stats = self.arcticbase.ledger_stats(self.workbench)
 1773          by_kind = stats.get("by_kind") or {}
 1774          if self._suppress_findings_for_subject():
 1775              mapping_intel = sum(int(by_kind.get(kind, 0) or 0) for kind in ("fact", "lead", "flow", "artifact"))
 1776              if mapping_intel <= 0:
 1777                  return
 1778          elif by_kind.get("finding", 0) <= 0:
 1779              return
 1780          try:
 1781              draft = self.draft_reporter(self.workbench, self.subject)
 1782              self.arcticbase.append_ledger(
 1783                  self.workbench,
 1784                  {
ref_all_executor/standing_loop.py L1804
 1800          except Exception:
 1801              return self.last_report_intel
 1802          return int(by_kind.get("fact", 0)) + int(by_kind.get("lead", 0)) + int(by_kind.get("finding", 0))
 1803  
 1804      def _maybe_dispatch_report_agent(self, cycle: int, active: list[DispatchedConversation]) -> None:
 1805          """Reserve one slot for a report-writing agent when enough new intel exists.
 1806  
 1807          The default policy is triggered by accumulated intel volume, not cycle
 1808          count, so the report keeps pace with discovery without interrupting a
 1809          running analysis agent. Subjects with `refall_report_agent_continuous`
 1810          reserve one slot and immediately replace a completed report agent. The
 1811          agent enriches the living report AND maintains a running analyst-notes
 1812          digest from canonical evidence.
 1813          """
 1814          if any(getattr(conversation, "task_class", None) == "report" for conversation in active):
 1815              return
 1816          threshold = self.config.report_agent_intel_threshold
 1817          if len(active) >= self.config.max_parallel:
 1818              return
 1819          intel = self._intel_count()
ref_all_executor/standing_loop.py L1919
 1915              f"(~{intel} intel items compiled so far.)",
 1916              self._output_contract(),
 1917          ] if part)
 1918  
 1919      def _maybe_dispatch_visualization(self, cycle: int, active: list[DispatchedConversation]) -> None:
 1920          if cycle % 20:
 1921              return
 1922          if not self._source_coverage_complete():
 1923              return
 1924          stats = self.arcticbase.ledger_stats(self.workbench)
 1925          coverage = self._safe_coverage()
 1926          if (
 1927              not self._suppress_findings_for_subject()
 1928              and (stats.get("by_kind") or {}).get("finding", 0) < self.config.visualization_min_findings
 1929          ):
 1930              return
 1931          if coverage.get("resolved_pct", 0) < self.config.visualization_min_resolved_pct:
 1932              return
 1933          if len(active) >= self.config.max_parallel:
 1934              return
ref_all_executor/oh_client.py L89
   85              if self.request_backoff_s:
   86                  self.sleep(self.request_backoff_s * attempt)
   87          raise RuntimeError(f"{method} {url} exhausted {attempts} attempts")  # pragma: no cover
   88  
   89      def start_conversation(self, workspace, agent_settings, text, max_iterations=30):
   90          """workspace: a dict from config.local_workspace()/remote_workspace()."""
   91          body = {
   92              "workspace": workspace,
   93              "agent_settings": agent_settings,
   94              "initial_message": {
   95                  "role": "user",
   96                  "content": [{"type": "text", "text": text}],
   97                  "run": True,
   98              },
   99              "max_iterations": max_iterations,
  100          }
  101          status, data = self.request("POST", "/api/conversations", body)
  102          if status != 201 or not isinstance(data, dict) or "id" not in data:
  103              raise RuntimeError(f"start_conversation failed {status}: {data!r}")
  104          return data["id"]
ref_all_executor/watchdog.py L658
  654              fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
  655              sample = self._run_unlocked()
  656              return sample
  657  
  658      def _run_unlocked(self) -> dict[str, Any]:
  659          now = float(self.now())
  660          state = load_json(self.state_path, {"subjects": {}, "cids": {}})
  661          endpoints = self._collect_endpoints()
  662          core_endpoint_names = (
  663              "ops_agents",
  664              "ops_debug_live",
  665              "coverage_stats",
  666              "runtime_events",
  667              "runtime_manifest",
  668              "pending_actions",
  669              "coverage_context",
  670          )
  671          core_endpoint_failures = {
  672              name: (endpoints.get(name) or {})
  673              for name in core_endpoint_names
ref_all_executor/watchdog.py L903
  899              "wishlist_provided": lambda: self.ab.list_wishlist(self.subject, status="provided"),
  900          }
  901          return {name: self._safe_call(fn) for name, fn in calls.items()}
  902  
  903      def _handle_wishlist(self, endpoints: dict[str, Any]) -> dict[str, Any]:
  904          open_result = endpoints.get("wishlist_open") or {}
  905          provided_result = endpoints.get("wishlist_provided") or {}
  906          open_items = open_result.get("data") if open_result.get("ok") else []
  907          provided_items = provided_result.get("data") if provided_result.get("ok") else []
  908          if not isinstance(open_items, list):
  909              open_items = []
  910          if not isinstance(provided_items, list):
  911              provided_items = []
  912  
  913          provided_keys = {
  914              wishlist_item_key(item)
  915              for item in provided_items
  916              if isinstance(item, dict) and wishlist_item_key(item)
  917          }
  918          provided_records = []
ref_all_executor/supervisor.py L561
  557          return self.paths["status"]
  558  
  559      # ----------------------------------------------------------------- tick
  560  
  561      def tick(self, mode: str = "full") -> dict[str, Any]:
  562          snapshot: dict[str, Any] = {
  563              "mode": mode,
  564              "subject": self.subject,
  565              "ts": iso_now(),
  566              "actions": [],
  567              "status": STATUS_HEALTHY,
  568          }
  569  
  570          if self.hold_active():
  571              snapshot["status"] = STATUS_ESCALATED
  572              snapshot["needs_operator"] = (
  573                  f"supervisor is holding; clear {self.paths['hold']} after fixing the fault"
  574              )
  575              snapshot["probe"] = {}
  576              self.write_status(snapshot)
arcticbase/backend/src/arctic_base/api/refall_pending_actions.py L58
   54          include_terminal=include_terminal,
   55      )
   56  
   57  
   58  @router.post("", status_code=201, operation_id="createRefallPendingAction")
   59  def create_pending_action(
   60      slug: str,
   61      body: PendingActionReq,
   62      storage: Annotated[FilesystemStorage, Depends(get_storage)],
   63  ) -> dict:
   64      store = store_for_workbench(storage, slug)
   65      try:
   66          action = store.create_pending_action(slug, body.model_dump())
   67      except ValueError as exc:
   68          raise HTTPException(422, str(exc)) from exc
   69      ledger.append_entry(
   70          storage,
   71          slug,
   72          {
   73              "kind": "pending-action",
ref_all_executor/standing_loop.py L2457
 2453                  "created_by": action.get("created_by") or "refall-standing-loop",
 2454              }
 2455              self.arcticbase.create_pending_action(self.workbench, payload)
 2456  
 2457      def _persist_findings(self, parsed: ParsedOutput, conversation: DispatchedConversation) -> None:
 2458          if not parsed.findings or not hasattr(self.coverage, "add_finding"):
 2459              return
 2460          for finding in parsed.findings:
 2461              payload = _finding_payload_for_store(finding, parsed, conversation)
 2462              if not payload:
 2463                  continue
 2464              try:
 2465                  self.coverage.add_finding(self.workbench, payload)
 2466              except RuntimeError as exc:
 2467                  if "finding promotion refused" in str(exc):
 2468                      self._record_finding_promotion_refusal(payload, parsed, conversation, str(exc))
 2469                      continue
 2470                  self.arcticbase.append_ledger(
 2471                      self.workbench,
 2472                      {
arcticbase/backend/src/arctic_base/refall/ledger.py L13
    9  
   10  from arctic_base.storage.filesystem import FilesystemStorage
   11  
   12  
   13  LEDGER_KINDS = {
   14      "fact",
   15      "lead",
   16      "finding",
   17      "observation",
   18      "hypothesis",
   19      "remediation",
   20      "attack-chain",
   21      "ask",
   22      "coverage",
   23      "validation",
   24      "artifact",
   25      "ops-event",
   26      "disposition-transition",
   27      "pending-action",
   28      "capability-grant",
arcticbase/backend/src/arctic_base/refall/contracts.py L24
   20  class RefallContract(BaseModel):
   21      model_config = ConfigDict(extra="allow")
   22  
   23  
   24  class EvidenceReceipt(RefallContract):
   25      receipt_type: str
   26      evidence_refs: list[str] = Field(default_factory=list)
   27      source_anchors: list[str] = Field(default_factory=list)
   28      artifact_refs: list[str] = Field(default_factory=list)
   29      source_hashes: dict[str, str] = Field(default_factory=dict)
   30      commands: list[str] = Field(default_factory=list)
   31      negative_tests: list[str] = Field(default_factory=list)
   32      summary: str = ""
   33  
   34      @model_validator(mode="after")
   35      def require_material(self) -> "EvidenceReceipt":
   36          if not any([
   37              self.evidence_refs,
   38              self.source_anchors,
   39              self.artifact_refs,
ref_all_executor/standing_loop.py L2206
 2202                  "hostname": socket.gethostname(),
 2203                  "subject_path": str(self.subject_path),
 2204              },
 2205              "config_hashes": {"lanes": _stable_json_sha256(config.LANES)},
 2206              "source_hashes": self._source_hashes(),
 2207              "source_inventory_id": (self.source_inventory or {}).get("inventory_id"),
 2208              "scope_overlay_id": (self.scope_overlay or {}).get("overlay_id"),
 2209              "threat_model_id": (self.threat_model or {}).get("threat_model_id"),
 2210              "checkpoint_ids": list(self.checkpoint_ids),
 2211              "git": {"sha": None},
 2212              "dynamic_testing": {"status": "not_applicable"},
 2213              "stage_status": {
 2214                  "loop": loop_status,
 2215                  "cycle": cycle,
 2216                  "active": len(active),
 2217                  "coverage_pct": coverage.get("resolved_pct", 0),
 2218              },
 2219          }
 2220          if state:
 2221              manifest["stage_status"]["state_status"] = state.get("status")
arcticbase/backend/src/arctic_base/refall/subjects.py L114
  110  
  111      refall_dir = subject_path / ".refall"
  112      refall_dir.mkdir(parents=True, exist_ok=True)
  113      state_path = refall_dir / "subject_state.json"
  114      ledger_path = refall_dir / "evidence_ledger.json"
  115      now = datetime.now(UTC).isoformat()
  116  
  117      if not state_path.exists():
  118          state = {
  119              "version": 1,
  120              "subject": subject,
  121              "title": title,
  122              "description": description,
  123              "source_path": source_path,
  124              "tags": tags,
  125              "workbench_slug": workbench_slug,
  126              "status": "created",
  127              "created_at": now,
  128              "updated_at": now,
  129          }
ref_all_executor/standing_loop.py L1053
 1049          if counts and sum(counts.values()):
 1050              self._append_event("intel_salvaged", {"subject": self.subject, "cid": conversation.cid, "status": status, **counts})
 1051          return counts
 1052  
 1053      def _intel_key(self, conversation: DispatchedConversation, kind: str, item: dict) -> tuple:
 1054          if kind == "finding":
 1055              ident = str(item.get("id") or item.get("title") or "")
 1056          elif kind == "flow":
 1057              ident = f"{item.get('from')}->{item.get('to')}:{item.get('relation')}"
 1058          else:
 1059              explicit = item.get("id") or item.get("ref")
 1060              if explicit:
 1061                  ident = str(explicit)
 1062              else:
 1063                  material = {
 1064                      key: item.get(key)
 1065                      for key in ("title", "description", "text", "severity", "investigation_plan")
 1066                      if item.get(key) not in (None, "", [], {})
 1067                  }
 1068                  ident = hashlib.sha256(
ref_all_executor/standing_loop.py L3561
 3557          ),
 3558      ] if part)
 3559  
 3560  
 3561  def _entries_from_parsed(parsed: ParsedOutput, conversation: DispatchedConversation) -> list[dict]:
 3562      base = {
 3563          "subject": conversation.subject,
 3564          "lane": conversation.lane,
 3565          "agent": conversation.cid,
 3566          "unit_id": (conversation.unit or {}).get("id"),
 3567          "phase": conversation.task_class,
 3568      }
 3569      entries: list[dict] = []
 3570      for finding in parsed.findings:
 3571          entries.append({**base, **finding, "kind": "finding"})
 3572      for fact in parsed.facts:
 3573          entries.append({**base, **fact, "kind": "fact"})
 3574      for lead in parsed.leads:
 3575          entries.append({**base, **lead, "kind": "lead"})
 3576      for flow in parsed.flows:
arcticbase/backend/src/arctic_base/refall/ledger.py L60
   56      storage.audit(slug, "refall.ledger.append", id=data["id"], kind=data["kind"])
   57      return data
   58  
   59  
   60  def list_entries(
   61      storage: FilesystemStorage,
   62      slug: str,
   63      *,
   64      kind: str | None = None,
   65      lane: str | None = None,
   66      severity: str | None = None,
   67      since: str | None = None,
   68  ) -> list[dict[str, Any]]:
   69      entries = _read_entries(ledger_path(storage, slug))
   70      out: list[dict[str, Any]] = []
   71      for entry in entries:
   72          if kind and entry.get("kind") != kind:
   73              continue
   74          if lane and entry.get("lane") != lane:
   75              continue
ref_all_executor/report_draft.py L51
   47      allow_structural_downgrade: bool = False,
   48  ) -> dict:
   49      base = (base or config.WB_BASE).rstrip("/")
   50      transport = transport or _urllib_transport
   51      wb_url = f"{base}/api/workbenches/{workbench}"
   52      meta = _req(transport, "GET", wb_url)
   53      coverage = _req(transport, "GET", f"{wb_url}/coverage/stats")
   54      ledger_entries = _req(transport, "GET", f"{wb_url}/ledger/entries")
   55      ledger_stats = _req(transport, "GET", f"{wb_url}/ledger/stats")
   56      closeout_gate = _optional_req(transport, "GET", f"{wb_url}/closeout/gate", {})
   57      runtime_observations = _optional_req(transport, "GET", f"{wb_url}/runtime/observations", [])
   58      runtime_artifacts = _optional_req(transport, "GET", f"{wb_url}/runtime/artifacts", [])
   59      evidence_graph = _optional_req(transport, "GET", f"{wb_url}/runtime/evidence-graph", {"nodes": [], "edges": []})
   60      run_manifest = _optional_req(transport, "GET", f"{wb_url}/runtime/manifest", {})
   61      nonfatal_errors = _optional_req(transport, "GET", f"{wb_url}/runtime/nonfatal-errors", {"errors": []})
   62      capabilities = _optional_req(transport, "GET", f"{wb_url}/capabilities", {"grants": [], "requests": []})
   63      if not isinstance(meta, dict):
   64          raise RuntimeError("workbench metadata response was not JSON")
   65      custom_fields = dict(meta.get("custom_fields") or {})
   66      mapping_mode = str(custom_fields.get("refall_no_findings") or "").strip().lower() in {"1", "true", "yes"}
ref_all_executor/report_enrich.py L276
  272          result[key] = str(path)
  273      return result
  274  
  275  
  276  def enrich_workbench_report(
  277      workbench: str,
  278      subject: str,
  279      *,
  280      root: Path | str | None = None,
  281      base: str | None = None,
  282      transport=None,
  283      assemble: bool = True,
  284      allow_structural_downgrade: bool = False,
  285  ) -> dict:
  286      base = (base or config.WB_BASE).rstrip("/")
  287      transport = transport or _urllib_transport
  288      entries = _req(transport, "GET", f"{base}/api/workbenches/{workbench}/ledger/entries")
  289      if not isinstance(entries, list):
  290          raise RuntimeError("ledger entries response was not JSON list")
  291      runtime_observations = _optional_req(transport, "GET", f"{base}/api/workbenches/{workbench}/runtime/observations", [])
arcticbase/backend/src/arctic_base/api/refall_coverage.py L149
  145      critic_reasoning: str | None = None
  146      status: str | None = None
  147  
  148  
  149  class WishlistReq(BaseModel):
  150      subject: str | None = None
  151      id: str | None = None
  152      kind: str | None = None
  153      item: str
  154      reason: str | None = None
  155      requested_by: str | None = None
  156      status: str = "open"
  157  
  158  
  159  class AttackPatternReq(BaseModel):
  160      id: str | None = None
  161      vuln_class: str
  162      title: str | None = None
  163      description: str | None = None
  164      leads: list[str] = Field(default_factory=list)
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1490
 1486              "refusal_reasons": sorted(set(reasons)),
 1487              "request": request,
 1488          }
 1489  
 1490      def add_wishlist_item(self, subject: str, data: dict) -> dict:
 1491          now = _now()
 1492          if not data.get("item"):
 1493              raise ValueError("wishlist item required")
 1494          item_id = str(data.get("id") or f"wish_{uuid.uuid4().hex}")
 1495          with self._wlock:
 1496              self.conn.execute(
 1497                  "INSERT INTO wishlist(id,subject,kind,item,reason,requested_by,status,created_at,updated_at) "
 1498                  "VALUES(?,?,?,?,?,?,?,?,?)",
 1499                  (
 1500                      item_id,
 1501                      subject,
 1502                      data.get("kind"),
 1503                      data.get("item"),
 1504                      data.get("reason"),
 1505                      data.get("requested_by"),
ref_all_executor/coverage_client.py L20
   16      except urllib.error.HTTPError as e:
   17          return e.code, e.read()
   18  
   19  
   20  class CoverageClient:
   21      """Client for REFALL coverage.
   22  
   23      Default mode is ArcticBase workbench-native coverage. Pass ``mode="legacy"``
   24      or a legacy ``base=`` to use the old standalone coverage service.
   25      """
   26  
   27      def __init__(
   28          self,
   29          base=None,
   30          transport=None,
   31          workbench_base=None,
   32          workbench_slug=None,
   33          mode=None,
   34          retry=True,
   35      ):
arcticbase/backend/src/arctic_base/api/refall_coverage.py L646
  642      )
  643      return updated
  644  
  645  
  646  @router.post("/wishlist", status_code=201, operation_id="createRefallCoverageWishlistItem")
  647  def create_wishlist_item(
  648      slug: str,
  649      body: WishlistReq,
  650      storage: Annotated[FilesystemStorage, Depends(get_storage)],
  651  ) -> dict:
  652      data = body.model_dump()
  653      subject = _subject(slug, data.pop("subject"))
  654      try:
  655          return _store(storage, slug).add_wishlist_item(subject, data)
  656      except ValueError as exc:
  657          raise HTTPException(422, str(exc)) from exc
  658  
  659  
  660  @router.get("/wishlist", operation_id="listRefallCoverageWishlist")
  661  def list_wishlist(
ref_all_executor/watchdog.py L39
   35      "dead_or_stuck_loop_state",
   36      "openhands_agent_server_unreachable",
   37      "refall_api_unreachable",
   38  }
   39  WISHLIST_INSTALLERS = {
   40      "binwalk": [["brew", "install", "binwalk"]],
   41      "exiftool": [["brew", "install", "exiftool"]],
   42      "radare2": [["brew", "install", "radare2"]],
   43      "r2": [["brew", "install", "radare2"]],
   44      "rizin": [["brew", "install", "rizin"]],
   45      "rz-bin": [["brew", "install", "rizin"]],
   46      "yara": [["brew", "install", "yara"]],
   47  }
   48  
   49  
   50  @dataclass
   51  class Thresholds:
   52      stale_agent_s: int
   53      no_progress_s: int
   54      stop_no_progress_s: int
ref_all_executor/watchdog.py L635
  631          self.state_path = Path(state_path or resolved_state_path)
  632          self.dry_run = dry_run or cfg.dry_run
  633          self.enforce = cfg.enforce
  634          self.no_progress_samples = cfg.no_progress_samples
  635          self.install_wishlist_items = (
  636              cfg.install_wishlist_items
  637              if config is not None
  638              else _env_bool("REFALL_WATCHDOG_INSTALL_WISHLIST", True)
  639          )
  640          self.wishlist_install_timeout_s = (
  641              cfg.wishlist_install_timeout_seconds
  642              if config is not None
  643              else _env_int("REFALL_WATCHDOG_WISHLIST_INSTALL_TIMEOUT_S", 900)
  644          )
  645          self.thresholds = cfg.thresholds() if config is not None else thresholds_from_env()
  646          self.ab = arcticbase or ArcticBaseClient(base=self.base_url)
  647          self.oh = openhands or OHClient(base=self.oh_base)
  648          self.now = now or time.time
  649  
  650      def run_once(self) -> dict[str, Any]:
ref_all_executor/watchdog.py L966
  962                  "provided": bool(provided_result.get("ok")),
  963              },
  964          }
  965  
  966      def _record_wishlist_provided(self, source_item: dict[str, Any], tool: str, receipt: dict[str, Any]) -> dict[str, Any]:
  967          payload = {
  968              "subject": self.subject,
  969              "kind": source_item.get("kind") or "tool",
  970              "item": tool,
  971              "reason": f"Watchdog provided requested wishlist item from {source_item.get('id') or 'unknown'}",
  972              "requested_by": WATCHDOG_NAME,
  973              "status": "provided",
  974          }
  975          if self.dry_run:
  976              return {"dry_run": True, "request": payload, "receipt": receipt}
  977          created = self.ab.create_wishlist_item(self.subject, payload)
  978          ledger = _safe_ledger(
  979              self.ab,
  980              self.subject,
  981              {
ref_all_executor/head_orchestrator.py L40
   36          self.usage = usage
   37          self.verifier = verifier
   38          self.reporter = reporter
   39  
   40      def run_cycle(
   41          self,
   42          subject: str,
   43          specialist_lane: str,
   44          *,
   45          refresh_report: bool = False,
   46          verify: bool = False,
   47      ) -> HeadCycleResult:
   48          if specialist_lane not in SPECIALIST_LANES:
   49              raise ValueError(f"unknown specialist lane: {specialist_lane}")
   50          gate = self.coverage.gate(subject)
   51          unit = self.coverage.claim_next(subject, self.profile.name)
   52          provided = self._provided_wishlist(subject)
   53          objective = _objective(subject, specialist_lane, unit, provided, self._coverage_context(subject))
   54          dispatch = self.dispatcher.dispatch(specialist_lane, objective)
   55          self.ledger.append({
ref_all_executor/wishlist.py L4
    1  from urllib.parse import quote
    2  
    3  
    4  def format_provided_wishlist(subject, provided_wishes):
    5      endpoint = f"/wishlist?subject={quote(subject)}&status=provided"
    6      lines = [
    7          "DELIVERED WISHLIST ITEMS:",
    8          f"- Check {endpoint} during preflight for the live delivered-tool/access state.",
    9      ]
   10      wishes = list(provided_wishes or [])
   11      if not wishes:
   12          lines.append("- No delivered wishlist items are recorded yet.")
   13      else:
   14          lines.append("- Use these before requesting duplicate tooling. Do not re-request matching items.")
   15          for wish in wishes:
   16              kind = wish.get("kind") or "item"
   17              item = wish.get("item") or "(unnamed)"
   18              requested_by = wish.get("requested_by") or "unknown agent"
   19              reason = wish.get("reason") or "no reason recorded"
drive_research.py L33
   29  
   30  _PROVIDED_WISHES = {}
   31  
   32  
   33  def _objective(agent_id, subject, n, provided_wishes=None):
   34      delivered = format_provided_wishlist(subject, provided_wishes)
   35      subject_ws = config.subject_workspace(subject)
   36      return f"""You are a REF-All SECURITY RESEARCHER (id: {agent_id}). You hunt REAL vulnerabilities by
   37  claiming sensitive-surface investigations and BRANCHING along the code's connections. This is classical
   38  security research, not summarization. Do up to {n} investigations, then STOP.
   39  
   40  Coverage/Plan Service: http://host.docker.internal:8900     Subject: "{subject}"     Source: {subject_ws}
   41  Toolchain (call graphs/scip/metrics): {config.REFALL_WORKSPACE}/toolchain/output/{subject}/   Serena: http://host.docker.internal:20872
   42  
   43  {delivered}
   44  
   45  0. PREFLIGHT (once, first):
   46     - GET http://host.docker.internal:8900/context?subject={subject} and read info_md — the project briefing
   47       (what this subject IS, its auth model, framework primitives, conventions). Ground every judgement in it;
   48       it is your first defense against false positives.
drive_binresearch.py L27
   23  
   24  _PROVIDED_WISHES = {}
   25  
   26  
   27  def _objective(agent_id, subject, binrel, n, provided_wishes=None):
   28      binpath = f"{config.subject_workspace(subject)}/{binrel}"
   29      delivered = format_provided_wishlist(subject, provided_wishes)
   30      return f"""You are a REF-All BINARY SECURITY RESEARCHER (id: {agent_id}). The subject is a compiled binary.
   31  You hunt memory-corruption / injection / auth-bypass vulns by DECOMPILING functions and tracing
   32  attacker-controlled data. Not source — you read radare2 output.
   33  Do up to {n} investigations, then STOP.
   34  
   35  Service: http://host.docker.internal:8900     Subject: "{subject}"
   36  Binary:  "{binpath}"
   37  radare2: run `export PATH=$PATH:/usr/local/bin` first, then use `r2`. Keep YOUR context lean.
   38  
   39  {delivered}
   40  
   41  0. PREFLIGHT:
   42     - GET http://host.docker.internal:8900/context?subject={subject} (read info_md if present).
coverage_service/app.py L414
  410  def set_context_ep(r: ContextReq):
  411      return STORE.set_context(r.subject, r.info_md, r.generated_by, r.root_container)
  412  
  413  
  414  @app.post("/wishlist")
  415  def add_wish_ep(r: WishReq):
  416      try:
  417          return STORE.add_wish(r.subject, r.item, r.kind, r.reason, r.requested_by)
  418      except ValueError as e:
  419          raise HTTPException(422, str(e))
  420  
  421  
  422  @app.get("/wishlist")
  423  def list_wish_ep(subject: str, status: Optional[str] = None):
  424      return STORE.list_wishes(subject, status)
  425  
  426  
  427  @app.post("/wishlist_resolve")
  428  def resolve_wish_ep(r: WishResolveReq):
  429      return STORE.resolve_wish(r.id, r.status)
coverage_service/store.py L415
  411          return dict(r) if r else None
  412  
  413      # ---- WISHLIST --------------------------------------------------------------
  414  
  415      def add_wish(self, subject, item, kind=None, reason=None, requested_by=None):
  416          if not item:
  417              raise ValueError("wishlist item required")
  418          wid = uuid.uuid4().hex
  419          with self._wlock:
  420              self.conn.execute(
  421                  "INSERT INTO wishlist(id,subject,kind,item,reason,requested_by,status,created_at,updated_at) "
  422                  "VALUES(?,?,?,?,?,?,'open',?,?)",
  423                  (wid, subject, kind, item, reason, requested_by, _now(), _now()))
  424              self.conn.commit()
  425          return self.get_wish(wid)
  426  
  427      def get_wish(self, wid):
  428          r = self.conn.execute("SELECT * FROM wishlist WHERE id=?", (wid,)).fetchone()
  429          return dict(r) if r else None
  430  
coverage_service/dashboard.py L159
  155        '<button class=act onclick=saveCtx()>Save</button> '+
  156        '<button class="act ghost" onclick=genCtx()>Generate with agent</button>'+
  157        '<span id=ctxmsg class=muted style="margin-left:8px;font-size:12px"></span></div>';
  158  
  159      m+='<div class=panel><h3>Wishlist &mdash; what researchers need</h3>'+
  160        (wl.length?wl.map(function(w){return '<div class=wl><div><b>'+h(w.item)+'</b> <span class=chip>'+h(w.kind||'?')+'</span><br>'+
  161          '<small>'+h(w.reason||'')+' &middot; '+h(w.requested_by||'')+'</small></div>'+
  162          (w.status==='open'?'<button class="act ghost" onclick="resolveWish(\''+w.id+'\')">provided</button>':'<span class=chip>'+h(w.status)+'</span>')+
  163          '</div>'}).join(''):'<div class=empty>No requests logged.</div>')+'</div></div>';
  164  
  165      m+='<div class="panel wide" style="margin-top:14px"><h3>Findings</h3>'+
  166        findingsTable(nonRejectedFindings,'No non-rejected findings yet. Seed the plan and run the researcher.')+'</div>';
  167  
  168      m+='<div class=panels style="margin-top:14px"><div class=panel><h3>Per-class exhaustiveness</h3>'+
  169        (cc.length?'<div class=chips>'+cc.map(function(c){return '<span class=chip>'+h(c.vuln_class)+': '+h(c.state)+'</span>'}).join('')+'</div>'
  170          :'<div class=empty>No vuln-class checks recorded yet.</div>')+'</div>';
  171      m+='<div class=panel><h3>Next investigations (by priority)</h3>'+
  172        (inv.length?inv.slice(0,8).map(function(v){return '<div class=wl><div><b>'+h(v.title)+'</b><br><small class=mono>p'+v.priority+' &middot; '+h((v.target_files||[]).join(', '))+'</small></div></div>'}).join('')
  173          :'<div class=empty>No open investigations.</div>')+'</div></div>';
  174      m+='<details class="fp-section"><summary>False positives / duplicates ('+rejectedFindings.length+')</summary>'+
arcticbase/backend/src/arctic_base/api/refall_pending_actions.py L116
  112      )
  113      return action
  114  
  115  
  116  @router.post("/{action_id}/resolve", operation_id="resolveRefallPendingAction")
  117  def resolve_pending_action(
  118      slug: str,
  119      action_id: str,
  120      body: ResolveReq,
  121      storage: Annotated[FilesystemStorage, Depends(get_storage)],
  122  ) -> dict:
  123      return _finish(slug, action_id, "done", body, storage)
  124  
  125  
  126  @router.post("/{action_id}/defer", operation_id="deferRefallPendingAction")
  127  def defer_pending_action(
  128      slug: str,
  129      action_id: str,
  130      body: ResolveReq,
  131      storage: Annotated[FilesystemStorage, Depends(get_storage)],
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L1
    1  from __future__ import annotations
    2  
    3  import html
    4  import json
    5  import re as _re
    6  import os
    7  import shutil
    8  import subprocess
    9  import sys
   10  import tempfile
   11  import urllib.error
   12  import urllib.request
   13  from datetime import UTC, datetime
   14  from pathlib import Path
   15  from typing import Annotated
   16  
ref_all_executor/source_inventory.py L1
    1  from __future__ import annotations
    2  
    3  import hashlib
    4  import json
    5  import os
    6  from pathlib import Path
    7  from typing import Any
    8  
    9  
   10  READ_CHUNK_BYTES = 1024 * 1024
   11  SKIP_DIRS = {".git", ".refall", "refallsubloop", "__pycache__", "node_modules", ".venv"}
   12  INCLUDE_DOT_DIRS = {".github", ".gitlab", ".circleci", ".azure", ".azuredevops", ".devcontainer"}
   13  SOURCE_EXTENSIONS = {
   14      ".bash": "shell",
   15      ".c": "c",
   16      ".cfg": "config",
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1
    1  from __future__ import annotations
    2  
    3  import hashlib
    4  import json
    5  import re
    6  import sqlite3
    7  import threading
    8  import time
    9  import uuid
   10  from pathlib import Path
   11  from urllib.parse import urlparse
   12  
   13  from arctic_base.refall import coverage_db
   14  from arctic_base.refall.contracts import ScopeOverlay, SourceInventoryPackage, ThreatModelPackage
   15  from arctic_base.refall.promotion_gates import evaluate_finding_candidate
   16  from arctic_base.storage.filesystem import FilesystemStorage
ref_all_executor/work_modules.py L1
    1  """Modular, phaseless work generation for the REFALL standing loop.
    2  
    3  The loop used to hardcode `task_class = "characterize"` for every coverage unit,
    4  so 12 of the 16 routed task classes were never dispatched and a run produced no
    5  findings unless an operator hand-fed it pending actions.
    6  
    7  This replaces that with **work modules**. A module is a small declaration that
    8  says: for targets matching this predicate, dispatch this task class with this
    9  extra objective. Modules are independent and evaluated every claim, so:
   10  
   11  - **Phaseless.** There is no global stage counter. A target is eligible for a
   12    module the moment its predicate holds. Two modules can be eligible for the
   13    same target at once.
   14  - **Ordered where it matters.** A module can require `after=["characterize"]`,
   15    which orders it *per target* without imposing a pipeline on the whole run.
   16  - **Combinable.** A subject enables any set of modules. Hyundai can run
ref_all_executor/router.py L1
    1  from dataclasses import dataclass
    2  
    3  from .config import Tier
    4  
    5  # triage_trivial is the ONLY class that may use Spark, and only for throwaway work.
    6  TASK_CLASSES = (
    7      "research", "identify", "select", "enumerate", "characterize", "build_toolchain",
    8      "run_toolchain", "decompose", "recon", "harness_draft", "patch", "review", "verify",
    9      "doc_synthesis", "threat_model", "triage_trivial",
   10  )
   11  
   12  
   13  @dataclass(frozen=True)
   14  class RouteDecision:
   15      lane: str
   16      needs_verify: bool
ref_all_executor/standing_loop.py L932
  928          if not text and result.trajectory:
  929              text = result.trajectory.decode("utf-8", errors="replace")
  930          return parse_agent_output(text, defaults)
  931  
  932      def _ingest_workspace_output(self, conversation: DispatchedConversation, parsed: "ParsedOutput") -> None:
  933          """Read the agent's writable workspace and merge its saved deliverables
  934          into `parsed`. Agents commonly save their real output as files -- a
  935          REFALL_SUBMISSION.json with findings, plus analysis/disassembly files --
  936          and end on a file-save action, leaving final_response empty. Without this
  937          that work is invisible to the loop. Files are also registered as durable
  938          runtime artifacts so the RE (disassembly, strings, notes) is preserved.
  939          """
  940          ws = conversation.workspace_host
  941          if not ws:
  942              return
  943          root = Path(ws)
  944          if not root.is_dir():
  945              return
  946          try:
  947              for path in sorted(root.rglob("*")):
ref_all_executor/structured_submit.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  import re
    5  from dataclasses import dataclass, field
    6  from typing import Any
    7  
    8  
    9  SUBMIT_RE = re.compile(r"```refall-submit\s*\n(.*?)\n```", re.DOTALL | re.IGNORECASE)
   10  SUPPORTED_SCHEMA = "refall.submit.v1"
   11  SUPPRESSIVE_STATUSES = {"FALSE_POSITIVE", "DUPLICATE", "CHAINABLE_DEFERRED"}
   12  
   13  
   14  @dataclass
   15  class SubmitParseResult:
   16      submissions: list[dict[str, Any]] = field(default_factory=list)
arcticbase/backend/src/arctic_base/refall/contracts.py L1
    1  from __future__ import annotations
    2  
    3  import hashlib
    4  import json
    5  from datetime import UTC, datetime
    6  from typing import Any, Literal
    7  
    8  from pydantic import BaseModel, ConfigDict, Field, model_validator
    9  
   10  
   11  def stable_id(prefix: str, payload: dict[str, Any]) -> str:
   12      raw = json.dumps(payload, sort_keys=True, default=str, separators=(",", ":")).encode("utf-8")
   13      return f"{prefix}_{hashlib.sha256(raw).hexdigest()[:24]}"
   14  
   15  
   16  def _utc_now() -> str:
arcticbase/backend/src/arctic_base/refall/promotion_gates.py L1
    1  from __future__ import annotations
    2  
    3  import re
    4  from urllib.parse import urlparse
    5  
    6  
    7  VALIDATED_STATUSES = {"VALID"}
    8  VALIDATED_QUALITIES = {"CODE_CONFIRMED", "LIVE_CONFIRMED"}
    9  VALIDATED_REPRO = {"reproduced", "confirmed", "live_reproduced"}
   10  
   11  
   12  def evaluate_finding_candidate(
   13      candidate: dict,
   14      *,
   15      source_inventory: dict | None = None,
   16      scope_overlay: dict | None = None,
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1161
 1157              args.append(status)
 1158          sql += " ORDER BY created_at DESC"
 1159          return [_transition_row(row) for row in self.conn.execute(sql, args)]
 1160  
 1161      def create_pending_action(self, subject: str, data: dict) -> dict:
 1162          now = _now()
 1163          if not data.get("kind"):
 1164              raise ValueError("pending action requires kind")
 1165          if not data.get("title"):
 1166              raise ValueError("pending action requires title")
 1167          identity_key = data.get("identity_key") or _pending_identity(subject, data)
 1168          action_id = str(data.get("id") or _now_id("pa"))
 1169          with self._wlock:
 1170              self.conn.execute(
 1171                  "INSERT INTO pending_actions(id,subject,identity_key,kind,title,description,status,"
 1172                  "priority,source_event_id,source_finding_id,source_unit_id,required_receipt_type,"
 1173                  "assigned_lane,claimed_by,lease_expires,resolution_receipt,created_by,created_at,updated_at) "
 1174                  "VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) "
 1175                  "ON CONFLICT(identity_key) DO UPDATE SET updated_at=excluded.updated_at",
 1176                  (
ref_all_executor/standing_loop.py L1
    1  from __future__ import annotations
    2  
    3  import argparse
    4  import base64
    5  import hashlib
    6  import json
    7  import os
    8  import re
    9  import shutil
   10  import socket
   11  import time
   12  import uuid
   13  from dataclasses import asdict, dataclass, field
   14  from datetime import UTC, datetime
   15  from pathlib import Path
   16  from typing import Callable
ref_all_executor/watchdog.py L1
    1  from __future__ import annotations
    2  
    3  import argparse
    4  import fcntl
    5  import json
    6  import os
    7  import re
    8  import shutil
    9  import subprocess
   10  import sys
   11  import time
   12  import traceback
   13  import urllib.error
   14  import urllib.request
   15  from dataclasses import dataclass
   16  from datetime import datetime, timezone
ref_all_executor/supervisor.py L1
    1  """REFALL loop supervisor.
    2  
    3  Runs on a tiered cron cadence next to a live standing loop:
    4  
    5  - fast tick (every minute): liveness only. Probes REFALL AB, Agent Canvas, and
    6    the standing loop process, and recovers whichever one is down.
    7  - full tick (every 3 minutes): the fast tick plus the full `RefallWatchdog`
    8    pass (agent health, wishlist installs, token/cost/runtime ceilings), a
    9    subject-relevance check over active agents, and a status report.
   10  
   11  Escalation: if recovery fires `recover_max` times inside `recover_window_s`,
   12  the supervisor stops the loop, writes a HOLD file, and refuses to restart
   13  anything until an operator clears it. That converts a crash *loop* into a
   14  single stop plus a report, which is the behaviour a teardown policy gets wrong.
   15  
   16  The supervisor never tears the stack down and never deletes its own cron entry.
ref_all_executor/report_draft.py L1
    1  from __future__ import annotations
    2  
    3  import argparse
    4  import html
    5  import json
    6  import re
    7  from datetime import UTC, datetime
    8  
    9  from . import config
   10  from .workbench import _urllib_transport
   11  
   12  
   13  def _req(transport, method: str, url: str, body: dict | bytes | None = None) -> dict | bytes:
   14      headers = {"Content-Type": "application/json"}
   15      if isinstance(body, bytes):
   16          data = body
ref_all_executor/report_enrich.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  import os
    5  import re
    6  from pathlib import Path
    7  
    8  from . import config
    9  from .report_assemble import assemble_report
   10  from .report_draft import _optional_req, _req, draft_report
   11  from .workbench import _urllib_transport
   12  
   13  
   14  HIGH_IMPACT = {"CRITICAL", "HIGH"}
   15  SOFTWARE_FACT_KEYWORDS = {"binary", "elf", "compiler", "rust", "version", "firmware", "build", "archive"}
   16  ARCHITECTURE_FACT_KEYWORDS = {"task", "ipc", "peripheral", "bus", "spi", "i2c", "flash", "memory", "dma"}
ref_all_executor/report_assemble.py L1
    1  from __future__ import annotations
    2  
    3  import html
    4  import json
    5  
    6  from . import config
    7  from .report_draft import (
    8      _optional_req,
    9      _record_structural_downgrade,
   10      _req,
   11      _structural_downgrade_sections,
   12  )
   13  from .workbench import _urllib_transport
   14  
   15  
   16  def assemble_report(
arcticbase/backend/src/arctic_base/refall/closeout_gate.py L1
    1  from __future__ import annotations
    2  
    3  from collections import Counter
    4  from datetime import datetime
    5  from pathlib import Path
    6  from typing import Any
    7  
    8  from arctic_base.refall import ledger
    9  from arctic_base.refall.coverage_store import store_for_workbench
   10  from arctic_base.refall.runtime_state import connect_runtime, list_artifacts, list_events
   11  from arctic_base.storage.filesystem import FilesystemStorage
   12  
   13  
   14  def evaluate_closeout_gate(
   15      storage: FilesystemStorage,
   16      slug: str,
drive_research.py L1
    1  #!/usr/bin/env python3
    2  """REF-All SECURITY RESEARCHER driver — the 'juicy path'.
    3  
    4  Classical security research: claim sensitive-surface investigations (seeded by
    5  coverage_service.seeds), audit each anchor, BRANCH along the code's connections
    6  (callers/callees/data-flow via the toolchain graphs), and emit findings (Mantis
    7  schema) with evidence. The plan frontier grows as researchers branch and shrinks
    8  as they complete — the loop runs until the frontier is exhausted or stopped.
    9  
   10  Seed the plan first (inside the container):
   11    PYTHONPATH=/projects/REFALL-V10 python3 -m coverage_service.tools.seed_subject example /projects/example
   12  
   13  Then:
   14    PYTHONPATH=. python3 drive_research.py run --subject example --claude 3 --codex 1 --units 4
   15    PYTHONPATH=. python3 drive_research.py status --subject example
   16    PYTHONPATH=. python3 drive_research.py kill
drive_binresearch.py L1
    1  #!/usr/bin/env python3
    2  """REF-All BINARY security-researcher driver — the juicy path for compiled subjects.
    3  
    4  Same shape as drive_research.py, but each researcher DECOMPILES functions with radare2
    5  (pdc/pdf) instead of reading source, and traces untrusted input through the binary's
    6  xrefs. Seeded by coverage_service.binary (dangerous imports → per-function investigations).
    7  
    8    # seed first (in container): python3 -m coverage_service.tools.seed_binary example "<binrel>"
    9    PYTHONPATH=. python3 drive_binresearch.py run --subject example \
   10        --binrel "bin/target" --claude 2 --codex 1 --units 3
   11    PYTHONPATH=. python3 drive_binresearch.py status --subject example
   12    PYTHONPATH=. python3 drive_binresearch.py kill
   13  """
   14  import os, time, json, threading, argparse
   15  
   16  from ref_all_executor.oh_client import OHClient
drive_binreview.py L1
    1  #!/usr/bin/env python3
    2  """REF-All adversarial REVIEW gate for binary findings (Mantis mantis-review).
    3  
    4  Every PROVISIONALLY_VALID finding is guilty until proven innocent. Reviewers re-decompile
    5  the cited function and apply Mantis's 12 negative constraints — with binary-specific emphasis
    6  on FORTIFY (`__memcpy_chk`) / bounded sinks, real attacker reachability, and severity honesty —
    7  then set VALID / FALSE_POSITIVE / NEEDS_RESEARCH. Reviewers are a different, independent pass
    8  from the finders (they can only judge, never add findings).
    9  
   10    PYTHONPATH=. python3 drive_binreview.py --subject example \
   11        --binrel "bin/target" --claude 2 --codex 2
   12  """
   13  import os, threading, json, argparse
   14  
   15  from ref_all_executor.oh_client import OHClient
   16  from ref_all_executor.coverage_client import CoverageClient
drive_binvalidate.py L1
    1  #!/usr/bin/env python3
    2  """REF-All binary validation/visualization driver.
    3  
    4  After adversarial review, every non-rejected retained finding needs a concrete trace artifact
    5  and runtime proof. This driver shards VALID / NEEDS_RESEARCH findings that are not yet
    6  runtime-confirmed across Claude/Codex agents, so inconclusive PoCs stay in the loop until
    7  they reproduce or are rejected with evidence.
    8  
    9    PYTHONPATH=. python3 drive_binvalidate.py --subject example \
   10        --binrel "bin/target" --claude 2 --codex 2
   11  """
   12  import argparse
   13  import json
   14  import os
   15  import threading
   16  import time
ref_all_executor/loop_manager.py L36
   32          # source projection and capability authorization cannot be bypassed.
   33          # A launcher remains injectable for focused unit tests and embedders.
   34          self._loop_launcher = loop_launcher
   35  
   36      def start(self, name: str) -> dict:
   37          created = self.arcticbase.create_subject(name)
   38          slug = created.get("workbench_slug") or created.get("subject") or name
   39          subject = created.get("subject") or slug
   40          self._require_context(slug)
   41          if self._loop_launcher is None:
   42              return self.arcticbase.ops_command(f"start {slug}")
   43          loop = self._loop_launcher(subject=subject, workbench=slug, max_cycles=0)
   44          ledger = self._record(slug, "loop.start", "REFALL loop started", status="started")
   45          return {
   46              "subject": subject,
   47              "workbench": slug,
   48              "status": "started",
   49              "subject_path": created.get("subject_path"),
   50              "ledger_id": ledger.get("id"),
   51              "loop": loop,
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L192
  188          "ledger_id": entry["id"],
  189      }
  190  
  191  
  192  def _execute_named_command(
  193      name: str,
  194      args: list[str],
  195      settings: Settings,
  196      storage: FilesystemStorage,
  197  ) -> dict:
  198      if name == "status":
  199          return {"status": "ok", "command": name, "result": status_payload(settings)}
  200      if name == "subjects":
  201          subjects = []
  202          for workbench in storage.list_workbenches():
  203              if "refall" in workbench.tags or workbench.custom_fields.get("refall_subject"):
  204                  subjects.append({"slug": workbench.slug, "title": workbench.title})
  205          return {"status": "ok", "command": name, "subjects": subjects}
  206      if name == "agents":
  207          return {
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L978
  974          return "warn"
  975      return "ok"
  976  
  977  
  978  def _context_subject(
  979      subject: str,
  980      settings: Settings,
  981      storage: FilesystemStorage,
  982      operator_input: str = "",
  983  ) -> dict:
  984      slug = subject.strip()
  985      if not slug:
  986          raise HTTPException(422, "context requires a subject")
  987      workbench = storage.get_workbench(slug)
  988      store = store_for_workbench(storage, slug)
  989      coverage = store.coverage(slug)
  990      ledger_stats = ledger.stats(storage, slug)
  991      workbench_data = workbench.model_dump(mode="json")
  992      root = _subject_context_root(workbench_data)
  993      materials = _context_materials(
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1764
 1760      expires_at = grant.get("expires_at")
 1761      return expires_at is not None and float(expires_at) <= _now()
 1762  
 1763  
 1764  def _grant_refusal_reasons(grant: dict, request: dict) -> list[str]:
 1765      reasons: list[str] = []
 1766      scope = grant.get("scope") if isinstance(grant.get("scope"), dict) else {}
 1767      target = request.get("target") if isinstance(request.get("target"), dict) else {}
 1768      capability_class = str(request.get("capability_class") or request.get("class") or "")
 1769      command = [str(part) for part in (request.get("command") or [])]
 1770      declared_effects = {str(effect) for effect in (request.get("declared_effects") or [])}
 1771      requested_by = str(request.get("requested_by") or "")
 1772      granted_by = str(grant.get("granted_by") or "")
 1773  
 1774      if grant.get("class") != capability_class:
 1775          reasons.append("grant class does not match")
 1776      if _grant_expired(grant):
 1777          reasons.append("grant is expired")
 1778      if requested_by and granted_by and requested_by == granted_by and granted_by not in {
 1779          "operator",
arcticbase/backend/src/arctic_base/api/refall_ops_commands.py L424
  420          str(max_cycles),
  421      ]
  422  
  423  
  424  def _launch_standing_loop(
  425      *,
  426      settings: Settings,
  427      subject: str,
  428      workbench: str,
  429      max_cycles: int,
  430      subject_path: Path | None,
  431      source_path: Path | None = None,
  432  ) -> dict:
  433      if not settings.refall_enable_loop_launch:
  434          return {
  435              "status": "disabled",
  436              "enabled": False,
  437              "reason": "REFALL_ENABLE_LOOP_LAUNCH=false",
  438          }
  439  
ref_all_executor/agent_workspace.py L17
   13  class AgentWorkspaceError(RuntimeError):
   14      pass
   15  
   16  
   17  def sync_agent_canvas_workspace(
   18      subject: str,
   19      source_path: str | Path,
   20      *,
   21      runner=subprocess,
   22      container: str | None = None,
   23      workspace_root: str | None = None,
   24      timeout_s: int = 300,
   25  ) -> dict:
   26      """Copy the host subject source into the Agent Canvas /projects workspace."""
   27      source = Path(source_path).expanduser().resolve()
   28      if not source.is_dir():
   29          raise AgentWorkspaceError(f"REFALL source path is not a directory: {source}")
   30  
   31      name = container or os.environ.get("REFALL_OPENHANDS_CONTAINER", DEFAULT_CONTAINER)
   32      root = (workspace_root or os.environ.get("REFALL_SUBJECT_WORKSPACE_ROOT", DEFAULT_WORKSPACE_ROOT)).rstrip("/")
ref_all_executor/config.py L245
  241          lane=lane, task_class=task_class, unit=unit, objective=objective
  242      )
  243  
  244  
  245  def agent_workspace_name(
  246      *,
  247      lane: str = "",
  248      task_class: str = "",
  249      unit: dict | None = None,
  250      objective: str = "",
  251  ) -> str:
  252      unit = unit or {}
  253      identity = str(unit.get("id") or unit.get("path") or unit.get("name") or "").strip()
  254      if not identity:
  255          identity = hashlib.sha1((objective or task_class or "task").encode("utf-8")).hexdigest()[:12]
  256      parts = [lane, task_class, identity]
  257      return "-".join(_safe_path_component(part) for part in parts if str(part or "").strip())
  258  
  259  
  260  def _safe_path_component(value) -> str:
ref_all_executor/standing_loop.py L205
  201  class AgentCanvasDispatcher:
  202      def __init__(self, oh: OHClient | None = None) -> None:
  203          self.oh = oh or OHClient()
  204  
  205      def start(self, task: AgentTask) -> DispatchedConversation:
  206          workspace = config.prepare_subject_agent_workspace(
  207              task.subject,
  208              lane=task.lane,
  209              task_class=task.task_class,
  210              unit=task.unit,
  211              objective=task.objective,
  212          )
  213          workspace_host = str(config.agent_workspace_host_path(
  214              task.subject, lane=task.lane, task_class=task.task_class,
  215              unit=task.unit, objective=task.objective,
  216          ))
  217          objective = _with_workspace_context(task.subject, workspace, _with_resource_context(task.objective))
  218          cid = self.oh.start_conversation(
  219              workspace=config.local_workspace(workspace),
  220              agent_settings=config.LANES[task.lane],
ref_all_executor/standing_loop.py L1654
 1650              )
 1651              return "delete_failed"
 1652          return "deleted"
 1653  
 1654      def _auto_enumerate_if_empty(self) -> int:
 1655          stats = self._safe_coverage()
 1656          if stats.get("lines_total", 0):
 1657              return 0
 1658          root = self._source_root()
 1659          if not root or not root.is_dir():
 1660              return 0
 1661          inventory = build_source_inventory(self.workbench, root)
 1662          self.source_inventory = inventory
 1663          if hasattr(self.coverage, "put_source_inventory"):
 1664              try:
 1665                  self.coverage.put_source_inventory(self.workbench, inventory)
 1666              except Exception:
 1667                  pass
 1668          files = [
 1669              {
arcticbase/backend/src/arctic_base/refall/coverage_store.py L240
  236      key = f"{subject}|{kind}|{path}|{line_start}|{line_end}|{name}"
  237      return hashlib.sha1(key.encode()).hexdigest()
  238  
  239  
  240  def partition_file(total_lines: int, symbols: list[dict]) -> list[dict]:
  241      syms = sorted(
  242          (s for s in symbols if int(s["line_start"]) <= int(s["line_end"])),
  243          key=lambda s: int(s["line_start"]),
  244      )
  245      leaves: list[dict] = []
  246      cursor = 1
  247      for symbol in syms:
  248          ls = max(1, int(symbol["line_start"]))
  249          le = min(total_lines, int(symbol["line_end"]))
  250          if le < cursor:
  251              continue
  252          ls = max(ls, cursor)
  253          if ls > cursor:
  254              leaves.append({"type": "line_range", "line_start": cursor, "line_end": ls - 1, "name": ""})
  255          leaves.append({
ref_all_executor/work_modules.py L211
  207  # ---------------------------------------------------------------- registry
  208  
  209  
  210  @dataclass
  211  class ModuleRegistry:
  212      """The set of modules active for one subject."""
  213  
  214      modules: list[WorkModule] = field(default_factory=list)
  215  
  216      @classmethod
  217      def from_config(cls, config: dict[str, Any] | None) -> "ModuleRegistry":
  218          """Build from a subject's module config.
  219  
  220          Shape (all optional)::
  221  
  222              {
  223                "packs": ["baseline", "security_assessment"],
  224                "modules": [ {...custom subloop module...} ],
  225                "disabled": ["sec_verify"],
  226                "overrides": {"characterize": {"priority": 5}}
ref_all_executor/standing_loop.py L888
  884              "created_at, and a threat_model object with assets, ingress_points, trust_boundaries, threats, "
  885              "hunt_prompts, and status. Keep this REFALL-owned; do not use VVAH report or encryption specs.",
  886          ])
  887  
  888      def _objective(self, task_class: str, reason: str, unit: dict, extra: str = "") -> str:
  889          context_md = str(self._coverage_context().get("info_md") or "").strip()
  890          scope = self._scope_overlay_text()
  891          blackboard = self._blackboard_text()
  892          path = unit.get("path") or unit.get("name") or "unknown"
  893          line_start = unit.get("line_start") or "?"
  894          line_end = unit.get("line_end") or "?"
  895          return "\n\n".join(part for part in [
  896              f"## REFALL context.md\n{context_md}" if context_md else "",
  897              f"## Scope Overlay\n```json\n{scope}\n```" if scope else "",
  898              blackboard,
  899              f"## Task\nRun {task_class} for REFALL subject {self.subject}.",
  900              f"## Routing\n{reason}",
  901              "## Coverage Unit\n"
  902              f"- ID: {unit.get('id')}\n"
  903              f"- Type: {unit.get('type')}\n"
ref_all_executor/standing_loop.py L1222
 1218              },
 1219          )
 1220          return True
 1221  
 1222      def _output_contract(self) -> str:
 1223          if self._suppress_findings_for_subject():
 1224              return (
 1225                  "## Output Contract\n"
 1226                  "Always compile evidence as you work: emit facts, leads, and flows, including "
 1227                  "negative results. Do NOT emit ```finding blocks, finding_submit payloads, "
 1228                  "or finding-promotion requests for this subject. Bug-like or security-relevant "
 1229                  "observations must be preserved as [LEAD] and [FACT] mapping evidence with "
 1230                  "source anchors.\n"
 1231                  "Prefer one terminal ```refall-submit JSON block using schema refall.submit.v1.\n"
 1232                  "REQUIRED envelope fields (omitting them will NOT lose your intel, but your task "
 1233                  "cannot be marked complete without them): `schema`, `type` (task_result_submit), "
 1234                  "`subject`, `task_id` (or `unit_id`), `status`, `summary`, `produced_by`, "
 1235                  "`created_at`, `evidence`, `confidence`, `limitations`, and `source_anchors` or "
 1236                  "`artifact_refs`.\n\n"
 1237                  "Source anchors MUST use source-inventory-relative paths rooted at the subject "
ref_all_executor/standing_loop.py L131
  127      capability_requests: list[dict] = field(default_factory=list)
  128      threat_models: list[dict] = field(default_factory=list)
  129  
  130  
  131  def read_conversation_agent_text(cid: str, conversations_dir: str | None = None) -> str:
  132      """All text the agent produced across the whole conversation.
  133  
  134      Reads the Agent Canvas event stream from disk (bind-mounted at
  135      <conversations>/<cid-without-dashes>/events/event-*.json) and concatenates
  136      every agent-authored message and action text, in order. This is the
  137      authoritative source of an agent's [FACT]/[FLOW]/[LEAD]/```finding /
  138      ```refall-submit output -- the `agent_final_response` endpoint returns empty
  139      for ACP agents, which end on a tool call rather than a final message.
  140  
  141      Parsing EVERY agent response (not just a "final" one) means an agent that
  142      emits intel across several turns is fully captured, and one that emits a
  143      single comprehensive message is captured too.
  144      """
  145      root = Path(conversations_dir or config.OPENHANDS_CONVERSATIONS_DIR)
  146      events = next(
ref_all_executor/standing_loop.py L1073
 1069                      json.dumps(material or item, sort_keys=True, default=str).encode("utf-8")
 1070                  ).hexdigest()
 1071          return (kind, ident)
 1072  
 1073      def _persist_new_intel(self, conversation: DispatchedConversation, parsed: "ParsedOutput") -> dict:
 1074          """Persist only intel not seen before this run -- shared by the real-time
 1075          incremental path and the terminal salvage path, so an item that lands live
 1076          is never re-persisted at completion. Dedup is by stable identity, not by
 1077          object identity, so re-reading the same workspace file is safe.
 1078          """
 1079          new = ParsedOutput()
 1080          staged_keys: list[tuple] = []
 1081          for kind, src, dst in (
 1082              ("finding", parsed.findings, new.findings),
 1083              ("fact", parsed.facts, new.facts),
 1084              ("lead", parsed.leads, new.leads),
 1085              ("flow", parsed.flows, new.flows),
 1086          ):
 1087              for item in src or []:
 1088                  if not isinstance(item, dict):
arcticbase/backend/src/arctic_base/refall/runtime_state.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  import sqlite3
    5  import time
    6  import uuid
    7  from pathlib import Path
    8  
    9  from arctic_base.refall.contracts import RunManifest
   10  
   11  
   12  NONFATAL_EVENT_MARKERS = (
   13      "error",
   14      "failed",
   15      "failure",
   16      "invalid",
arcticbase/backend/src/arctic_base/refall/projection.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  import sqlite3
    5  import time
    6  import uuid
    7  
    8  from arctic_base.refall.runtime_state import list_events
    9  
   10  
   11  PROJECTOR = "refall.runtime.v1"
   12  
   13  
   14  def project_runtime_events(conn: sqlite3.Connection, subject: str) -> dict:
   15      watermark = conn.execute(
   16          "SELECT last_event_created_at FROM projection_watermarks WHERE projector=?",
ref_all_executor/standing_loop.py L2250
 2246              except Exception:
 2247                  pass
 2248          self._write_event_checkpoint(event_type, fields)
 2249  
 2250      def _resume_preflight_refused(self) -> bool:
 2251          if not (
 2252              hasattr(self.arcticbase, "list_runtime_checkpoints")
 2253              and hasattr(self.arcticbase, "resume_preflight")
 2254          ):
 2255              return False
 2256          try:
 2257              prior = self.arcticbase.list_runtime_checkpoints(self.workbench)
 2258          except Exception:
 2259              return False
 2260          if not prior:
 2261              return False
 2262          try:
 2263              result = self.arcticbase.resume_preflight(self.workbench, self._checkpoint_input_hashes())
 2264          except RuntimeError as exc:
 2265              if "409" in str(exc):
ref_all_executor/report_enrich.py L19
   15  SOFTWARE_FACT_KEYWORDS = {"binary", "elf", "compiler", "rust", "version", "firmware", "build", "archive"}
   16  ARCHITECTURE_FACT_KEYWORDS = {"task", "ipc", "peripheral", "bus", "spi", "i2c", "flash", "memory", "dma"}
   17  
   18  
   19  def enrich_findings(
   20      entries: list[dict],
   21      *,
   22      runtime_observations: list[dict] | None = None,
   23      runtime_artifacts: list[dict] | None = None,
   24      evidence_graph: dict | None = None,
   25      closeout_gate: dict | None = None,
   26      pending_actions: list[dict] | None = None,
   27      capabilities: dict | None = None,
   28  ) -> dict:
   29      canonical_by_key: dict[str, dict] = {}
   30      findings: list[dict] = []
   31      leads: list[dict] = []
   32      facts: list[dict] = []
   33      for entry in entries:
   34          kind = entry.get("kind")
ref_all_executor/loop_manager.py L81
   77              "ledger_id": ledger.get("id"),
   78              "loop": loop,
   79          }
   80  
   81      def closeout(self, name: str) -> dict:
   82          slug = name
   83          workbench = self.arcticbase.get_workbench(slug)
   84          subject = (workbench.get("custom_fields") or {}).get("refall_subject") or slug
   85          coverage = self.arcticbase.coverage_stats(slug)
   86          ledger_stats = self.arcticbase.ledger_stats(slug)
   87          draft = (self._draft_report or draft_report)(slug, subject)
   88          enriched = (self._enrich_report or enrich_workbench_report)(slug, subject)
   89          # Enrichment persists artifacts and may append ledger evidence even when
   90          # structural-downgrade protection refuses to replace the richer formal
   91          # report. Refresh the authoritative draft after enrichment so the
   92          # report freshness high-water mark includes those writes before the
   93          # closeout gate is evaluated.
   94          draft = (self._draft_report or draft_report)(slug, subject)
   95          closeout_gate = (
   96              self.arcticbase.closeout_gate(slug)
ref_all_executor/config.py L207
  203      root = os.environ.get("REFALL_AGENT_WORKSPACE_CONTAINER_ROOT", AGENT_WORKSPACE_CONTAINER_ROOT_DEFAULT).rstrip("/")
  204      return f"{root}/{_safe_path_component(subject)}/{agent_workspace_name(lane=lane, task_class=task_class, unit=unit, objective=objective)}"
  205  
  206  
  207  def prepare_subject_agent_workspace(
  208      subject,
  209      *,
  210      lane: str = "",
  211      task_class: str = "",
  212      unit: dict | None = None,
  213      objective: str = "",
  214  ) -> str:
  215      """Create the host-side writable workspace and return its container path."""
  216      host_root = Path(os.environ.get("REFALL_AGENT_WORKSPACE_HOST_ROOT", AGENT_WORKSPACE_HOST_ROOT_DEFAULT)).expanduser()
  217      subject_name = _safe_path_component(subject)
  218      workspace_name = agent_workspace_name(lane=lane, task_class=task_class, unit=unit, objective=objective)
  219      (host_root / subject_name / workspace_name).mkdir(parents=True, exist_ok=True)
  220      return subject_agent_workspace(
  221          subject,
  222          lane=lane,
ref_all_executor/standing_loop.py L187
  183          return objective
  184      return f"{resources}\n\n{objective}"
  185  
  186  
  187  def _with_workspace_context(subject: str, workspace: str, objective: str) -> str:
  188      if "## REFALL Workspace Layout" in objective:
  189          return objective
  190      source = config.subject_workspace(subject)
  191      layout = (
  192          "## REFALL Workspace Layout\n"
  193          f"- Writable agent workspace: `{workspace}`\n"
  194          f"- Read-only subject source root: `{source}`\n"
  195          f"Run local analysis commands from the writable workspace, but inspect target files under `{source}`. "
  196          f"Do not write to `{source}`; place scratch files, extracted metadata, notes, and temporary outputs under `{workspace}`."
  197      )
  198      return f"{layout}\n\n{objective}"
  199  
  200  
  201  class AgentCanvasDispatcher:
  202      def __init__(self, oh: OHClient | None = None) -> None:
arcticbase/backend/src/arctic_base/api/refall_capabilities.py L171
  167      return revoke_capability(slug, grant_id, storage, operator_identity)
  168  
  169  
  170  @router.post("/execution/authorize", operation_id="authorizeRefallExecution")
  171  def authorize_execution(
  172      slug: str,
  173      request: Annotated[dict, Body()],
  174      storage: Annotated[FilesystemStorage, Depends(get_storage)],
  175  ) -> dict:
  176      result = store_for_workbench(storage, slug).authorize_execution(slug, request)
  177      ledger.append_entry(
  178          storage,
  179          slug,
  180          {
  181              "kind": "capability-decision",
  182              "subject": slug,
  183              "status": result["status"],
  184              "capability_class": request.get("capability_class"),
  185              "grant_id": result.get("grant_id"),
  186              "summary": "; ".join(result.get("refusal_reasons") or []) or "authorized",
arcticbase/backend/src/arctic_base/refall/runtime_state.py L346
  342          )
  343      ]
  344  
  345  
  346  def resume_preflight(conn: sqlite3.Connection, subject: str, current_hashes: dict | None) -> dict:
  347      current = {str(key): str(value) for key, value in (current_hashes or {}).items()}
  348      checkpoints = list_checkpoints(conn, subject, limit=1000)
  349      expected: dict[str, str] = {}
  350      checkpoint_ids: list[str] = []
  351      for checkpoint in reversed(checkpoints):
  352          hashes = checkpoint.get("input_hashes") if isinstance(checkpoint.get("input_hashes"), dict) else {}
  353          if hashes:
  354              checkpoint_ids.append(str(checkpoint["id"]))
  355          for key, value in hashes.items():
  356              expected[str(key)] = str(value)
  357      changes = []
  358      if current:
  359          for key, expected_hash in sorted(expected.items()):
  360              actual_hash = current.get(key)
  361              if actual_hash is None:
arcticbase/backend/src/arctic_base/refall/coverage_store.py L112
  108              return True
  109      return False
  110  
  111  
  112  def _receipt_stale_reasons(receipt: object, source_root: object) -> list[str]:
  113      if not isinstance(receipt, dict):
  114          return []
  115      hashes: dict[str, object] = {}
  116      for key in ("source_hashes", "source_anchor_hashes", "file_hashes", "config_hashes"):
  117          value = receipt.get(key)
  118          if isinstance(value, dict):
  119              hashes.update(value)
  120      if not hashes:
  121          return []
  122      if not source_root:
  123          return ["stale evidence check requires recorded source root"]
  124      try:
  125          root = Path(str(source_root)).expanduser().resolve()
  126      except OSError:
  127          return ["stale evidence check source root is not resolvable"]
arcticbase/backend/src/arctic_base/refall/coverage_store.py L1243
 1239              )
 1240              self.conn.commit()
 1241          return self.get_pending_action(action_id) or action
 1242  
 1243      def finish_pending_action(
 1244          self,
 1245          subject: str,
 1246          action_id: str,
 1247          *,
 1248          status: str,
 1249          receipt_type: str | None,
 1250          receipt: object,
 1251          resolved_by: str | None = None,
 1252      ) -> dict:
 1253          action = self.get_pending_action(action_id)
 1254          if action is None or action.get("subject") != subject:
 1255              raise ValueError("no such pending action")
 1256          if status not in {"done", "deferred", "cancelled"}:
 1257              raise ValueError("invalid pending action terminal status")
 1258          required = action.get("required_receipt_type")
arcticbase/backend/src/arctic_base/api/refall_closeout.py L33
   29      return evaluate_closeout_gate(storage, slug, threshold=threshold)
   30  
   31  
   32  @router.post("/override", operation_id="overrideRefallCloseoutGate")
   33  def override_closeout_gate(
   34      slug: str,
   35      body: Annotated[CloseoutOverrideReq, Body()],
   36      storage: Annotated[FilesystemStorage, Depends(get_storage)],
   37  ) -> dict:
   38      if not body.receipt or not body.check_ids:
   39          raise HTTPException(422, "closeout override requires check_ids and receipt")
   40      entry = ledger.append_entry(
   41          storage,
   42          slug,
   43          {
   44              "kind": "ops-event",
   45              "subject": slug,
   46              "action": "closeout.override",
   47              "status": "accepted",
   48              "created_by": body.operator,
arcticbase/backend/tests/test_refall_capability_broker.py L1
    1  from pathlib import Path
    2  import time
    3  
    4  import pytest
    5  from fastapi.testclient import TestClient
    6  
    7  from arctic_base.api.deps import get_storage
    8  from arctic_base.main import app
    9  from arctic_base.storage.filesystem import FilesystemStorage
   10  
   11  
   12  @pytest.fixture
   13  def client(tmp_path: Path) -> TestClient:
   14      storage = FilesystemStorage(tmp_path / "data")
   15      app.dependency_overrides[get_storage] = lambda: storage
   16      with TestClient(app) as c:
ref_all_executor/__init__.py L1
    1  __version__ = "0.1.0"
ref_all_executor/__main__.py L1
    1  import sys
    2  
    3  from .cli import main
    4  
    5  if __name__ == "__main__":
    6      sys.exit(main())
ref_all_executor/agent_profiles.py L1
    1  from __future__ import annotations
    2  
    3  from dataclasses import dataclass
    4  
    5  
    6  HEAD_PROVIDERS = {"codex", "claude", "grok"}
    7  SPECIALIST_LANES = {
    8      "source-review",
    9      "binary-review",
   10      "dynamic-validation",
   11      "exploit-poc",
   12      "report-draft",
   13      "report-enrich",
   14      "visualization",
   15      "tooling",
   16      "cleanup",
ref_all_executor/agent_workspace.py L1
    1  from __future__ import annotations
    2  
    3  import os
    4  import json
    5  import subprocess
    6  from pathlib import Path
    7  
    8  
    9  DEFAULT_CONTAINER = "refall-agent-canvas"
   10  DEFAULT_WORKSPACE_ROOT = "/projects"
   11  
   12  
   13  class AgentWorkspaceError(RuntimeError):
   14      pass
   15  
   16  
ref_all_executor/arcticbase_client.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  from urllib.parse import urlencode
    5  
    6  from . import config
    7  from .http_retry import with_retry
    8  from .workbench import _urllib_transport
    9  
   10  
   11  class ArcticBaseClient:
   12      def __init__(self, base: str | None = None, transport=None, *, retry: bool = True) -> None:
   13          self.base = (base or config.WB_BASE).rstrip("/")
   14          # REFALL AB restarts (launchd kickstart, reload) must not end a run.
   15          raw = transport or _urllib_transport
   16          self.raw_transport = raw
ref_all_executor/artifact_store.py L1
    1  from __future__ import annotations
    2  
    3  import hashlib
    4  import sqlite3
    5  import time
    6  import uuid
    7  from pathlib import Path
    8  
    9  from .runtime_state import runtime_root
   10  
   11  
   12  def save_artifact(
   13      conn: sqlite3.Connection,
   14      subject_path: Path,
   15      *,
   16      subject: str,
ref_all_executor/auto_client.py L1
    1  import json
    2  import urllib.request
    3  import urllib.error
    4  
    5  from . import config
    6  
    7  
    8  def _urllib_transport(method, url, headers, body):
    9      req = urllib.request.Request(url, data=body, method=method, headers=headers)
   10      try:
   11          with urllib.request.urlopen(req, timeout=120) as r:
   12              return r.status, r.read()
   13      except urllib.error.HTTPError as e:
   14          return e.code, e.read()
   15  
   16  
ref_all_executor/cleanup.py L1
    1  import os
    2  import subprocess
    3  from pathlib import Path
    4  
    5  from .oh_client import OHClient
    6  
    7  
    8  REPO_ROOT = Path(__file__).resolve().parents[1]
    9  DEFAULT_DATA_DIR = REPO_ROOT / "coverage_service" / "data"
   10  DEFAULT_CONTAINER_PATTERNS = ("refall-agent-canvas", "openhands")
   11  DEFAULT_PROCESS_PATTERNS = {
   12      "coverage_service": r"python.*-m coverage_service",
   13      "drivers": r"drive_(characterize|research|binresearch|binreview|binvalidate|context)\.py",
   14  }
   15  SERENA_PROCESS_PATTERN = r"serena start-mcp-server|VueLanguageServer|typescript-language-server|tsserver\.js"
   16  
ref_all_executor/cli.py L1
    1  import argparse
    2  import json
    3  import os
    4  import sys
    5  from pathlib import Path
    6  
    7  from .auto_client import AutoClient
    8  from .arcticbase_client import ArcticBaseClient
    9  from .cleanup import RefallCleanup
   10  from .doctor import doctor
   11  from .estimate import estimate_subject
   12  from .infra import RefallInfra
   13  from .loop_manager import RefallLoopManager
   14  from .prompts import build_prompt
   15  from .report_draft import draft_report
   16  from .report_enrich import enrich_workbench_report
ref_all_executor/context_fingerprint.py L1
    1  """Deterministic, read-only subject fingerprinting for REFALL context generation."""
    2  
    3  from __future__ import annotations
    4  
    5  import hashlib
    6  import os
    7  import re
    8  import shutil
    9  import subprocess
   10  from pathlib import Path
   11  from typing import Any
   12  
   13  from .source_inventory import READ_CHUNK_BYTES, SKIP_DIRS
   14  
   15  DEFAULT_TIMEOUT_SECONDS = 30
   16  BINWALK_TIMEOUT_SECONDS = 120
ref_all_executor/contract_payloads.py L1
    1  from __future__ import annotations
    2  
    3  from datetime import UTC, datetime
    4  
    5  
    6  def _now() -> str:
    7      return datetime.now(UTC).isoformat()
    8  
    9  
   10  def task_result_submit(
   11      *,
   12      subject: str,
   13      task_id: str,
   14      produced_by: str,
   15      summary: str,
   16      source_anchors: list[str],
ref_all_executor/coverage_client.py L1
    1  import json
    2  import os
    3  import urllib.request
    4  import urllib.error
    5  from urllib.parse import quote, urlencode
    6  
    7  from . import config
    8  from .http_retry import with_retry
    9  
   10  
   11  def _urllib_transport(method, url, headers, body):
   12      req = urllib.request.Request(url, data=body, method=method, headers=headers)
   13      try:
   14          with urllib.request.urlopen(req, timeout=60) as r:
   15              return r.status, r.read()
   16      except urllib.error.HTTPError as e:
ref_all_executor/doctor.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  import os
    5  import shutil
    6  import urllib.error
    7  import urllib.request
    8  from pathlib import Path
    9  
   10  from . import config
   11  
   12  
   13  PROVIDER_ENV = {
   14      "codex": "REFALL_CODEX_COMMAND",
   15      "claude": "REFALL_CLAUDE_COMMAND",
   16      "grok": "REFALL_GROK_COMMAND",
ref_all_executor/entrypoint.py L1
    1  """The multi-lane REF-All loop. In Phase 1b this runs as the code-automation entrypoint;
    2  it can also be driven directly. Real analysis goes to Claude/Codex conversations (subscriptions);
    3  findings are written to the workbench deterministically; the blackboard lives in the native KV store."""
    4  from dataclasses import dataclass
    5  
    6  from . import config
    7  from .oh_client import OHClient
    8  from .workbench import Workbench
    9  from .router import route
   10  from .verify import run_verifier, assign_tier, TaskResult
   11  
   12  
   13  @dataclass
   14  class Task:
   15      task_class: str
   16      objective: str
ref_all_executor/estimate.py L1
    1  from __future__ import annotations
    2  
    3  import math
    4  
    5  from .arcticbase_client import ArcticBaseClient
    6  from .doctor import doctor
    7  
    8  
    9  def estimate_subject(
   10      subject: str,
   11      *,
   12      arcticbase=None,
   13      check_network: bool = True,
   14  ) -> dict:
   15      client = arcticbase or ArcticBaseClient()
   16      coverage = _optional(lambda: client.coverage_stats(subject), {})
ref_all_executor/evidence_graph.py L1
    1  from __future__ import annotations
    2  
    3  
    4  def graph_from_deltas(deltas: list[dict]) -> dict:
    5      nodes: dict[str, dict] = {}
    6      edges: list[dict] = []
    7      for delta in deltas:
    8          operation = delta.get("operation")
    9          if operation == "upsert_node":
   10              node_id = str(delta.get("node_id"))
   11              nodes[node_id] = {
   12                  "id": node_id,
   13                  "type": delta.get("node_type"),
   14                  "properties": delta.get("properties") or {},
   15              }
   16          elif operation == "upsert_edge":
ref_all_executor/head_orchestrator.py L1
    1  from __future__ import annotations
    2  
    3  from dataclasses import dataclass
    4  
    5  from .agent_profiles import SPECIALIST_LANES, head_profile
    6  from .wishlist import format_provided_wishlist, provided_wishes_from
    7  
    8  
    9  @dataclass(frozen=True)
   10  class HeadCycleResult:
   11      provider: str
   12      subject: str
   13      specialist_lane: str
   14      unit_id: str | None
   15      status: str
   16  
ref_all_executor/http_retry.py L1
    1  """Shared retry wrapper for REFALL service transports.
    2  
    3  Every REFALL service the standing loop depends on -- REFALL AB, the coverage
    4  service, and the Agent Canvas -- can drop out briefly: a launchd restart, a
    5  container bounce, an OOM kill of a backend process. Without a retry those blips
    6  raise straight through the loop's request call and terminate the whole run,
    7  losing every in-flight conversation.
    8  
    9  Wrap a transport with `with_retry` to make 5xx responses and raw socket errors
   10  survivable. The final attempt is passed through unchanged so callers keep their
   11  normal error handling: a 5xx is returned for the caller to raise on, and a
   12  socket error is re-raised.
   13  """
   14  
   15  from __future__ import annotations
   16  
ref_all_executor/infra.py L1
    1  import os
    2  import subprocess
    3  from pathlib import Path
    4  
    5  from .cleanup import DEFAULT_DATA_DIR, RefallCleanup
    6  
    7  
    8  REPO_ROOT = Path(__file__).resolve().parents[1]
    9  OPENHANDS_CONTAINER = "refall-agent-canvas"
   10  COVERAGE_SESSION = "refall-coverage"
   11  TOOLCHAIN_SESSION = "refall-toolchain"
   12  TOOLCHAIN_REQUIRED_SERVICES = ("python-lsp-server", "serena")
   13  
   14  
   15  class RefallInfra:
   16      """Umbrella lifecycle helper for local reusable REF-All infrastructure."""
ref_all_executor/loop_manager.py L1
    1  from __future__ import annotations
    2  
    3  import os
    4  import subprocess
    5  import sys
    6  from pathlib import Path
    7  from typing import Callable
    8  
    9  from .arcticbase_client import ArcticBaseClient
   10  from . import config
   11  from .report_draft import draft_report
   12  from .report_enrich import enrich_workbench_report
   13  
   14  
   15  ReportCallback = Callable[[str, str], dict]
   16  LoopLauncher = Callable[..., dict]
ref_all_executor/oh_client.py L1
    1  import json
    2  import os
    3  import time
    4  import urllib.request
    5  import urllib.error
    6  
    7  from . import config
    8  
    9  TERMINAL = {"finished", "error", "stopped", "paused"}
   10  
   11  
   12  def _urllib_transport(method, url, headers, body):
   13      req = urllib.request.Request(url, data=body, method=method, headers=headers)
   14      try:
   15          with urllib.request.urlopen(req, timeout=120) as r:
   16              return r.status, r.read()
ref_all_executor/projection.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  import sqlite3
    5  import time
    6  import uuid
    7  
    8  from .runtime_state import list_events
    9  
   10  
   11  PROJECTOR = "refall.runtime.v1"
   12  
   13  
   14  def project_runtime_events(conn: sqlite3.Connection, subject: str) -> dict:
   15      watermark = conn.execute(
   16          "SELECT last_event_created_at FROM projection_watermarks WHERE projector=?",
ref_all_executor/prompts.py L1
    1  RE_RESEARCH_LOOP_PROMPT = """You are the REF-All research agent running autonomously inside an OpenHands automation.
    2  
    3  TARGET: {target_desc}
    4  
    5  Run ONE pass of the REF-All reverse-engineering / fuzzing research loop:
    6  1. INTAKE — inspect the target materials available in this sandbox.
    7  2. CONTEXT — note languages, formats, dependencies, and likely attack surfaces.
    8  3. CLASSIFY — pick the single highest-value investigative subtask for this pass.
    9  4. ANALYZE — do that subtask (static read, harness sketch, or repro attempt).
   10  5. VERIFY — if you make an operating claim (a bug, a working harness, a repro),
   11     try to confirm it by real execution. Do NOT accept a claim on inspection alone.
   12  6. TIER — label each finding: tier_1_live_confirmed (executed+reproduced),
   13     tier_2_code_confirmed (strong static evidence), or tier_3_hypothesis (lead).
   14  
   15  Then WRITE the finding back to the REF-All workbench so the operator sees it. Use the
   16  terminal to POST it (this is an append to the workbench, safe to run):
ref_all_executor/runtime_invocation.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  import sqlite3
    5  import time
    6  import uuid
    7  from dataclasses import dataclass
    8  
    9  from .structured_submit import parse_refall_submissions
   10  
   11  
   12  @dataclass
   13  class InvocationOutcome:
   14      state: str
   15      failure_class: str | None = None
   16      retryable: bool = False
ref_all_executor/runtime_state.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  import sqlite3
    5  import time
    6  import uuid
    7  from pathlib import Path
    8  
    9  
   10  SCHEMA = """
   11  CREATE TABLE IF NOT EXISTS runs (
   12    id TEXT PRIMARY KEY, subject TEXT NOT NULL, workbench TEXT, created_at REAL
   13  );
   14  CREATE TABLE IF NOT EXISTS task_epochs (
   15    id TEXT PRIMARY KEY, run_id TEXT, task_id TEXT, created_at REAL
   16  );
ref_all_executor/usage.py L1
    1  import json
    2  import os
    3  import time
    4  
    5  
    6  DATA_DIR = os.path.join(os.path.dirname(os.path.dirname(__file__)), "coverage_service", "data")
    7  USAGE_LOG = os.path.join(DATA_DIR, "agent_usage.jsonl")
    8  
    9  
   10  def _summary(info):
   11      return {
   12          "execution_status": info.get("execution_status"),
   13          "created_at": info.get("created_at"),
   14          "updated_at": info.get("updated_at"),
   15          "title": info.get("title"),
   16          "current_model_id": info.get("current_model_id"),
ref_all_executor/verify.py L1
    1  from dataclasses import dataclass
    2  
    3  from .config import LANES, Tier
    4  
    5  
    6  @dataclass
    7  class TaskResult:
    8      task_class: str
    9      lane: str
   10      objective: str
   11      response: str
   12      trajectory: bytes
   13      status: str
   14      tier: str
   15      verified: bool
   16      reason: str
ref_all_executor/wishlist.py L1
    1  from urllib.parse import quote
    2  
    3  
    4  def format_provided_wishlist(subject, provided_wishes):
    5      endpoint = f"/wishlist?subject={quote(subject)}&status=provided"
    6      lines = [
    7          "DELIVERED WISHLIST ITEMS:",
    8          f"- Check {endpoint} during preflight for the live delivered-tool/access state.",
    9      ]
   10      wishes = list(provided_wishes or [])
   11      if not wishes:
   12          lines.append("- No delivered wishlist items are recorded yet.")
   13      else:
   14          lines.append("- Use these before requesting duplicate tooling. Do not re-request matching items.")
   15          for wish in wishes:
   16              kind = wish.get("kind") or "item"
ref_all_executor/workbench.py L1
    1  import json
    2  import ssl
    3  import sys
    4  import urllib.request
    5  import urllib.error
    6  
    7  from . import config
    8  
    9  _UNVERIFIED = None  # set once if the workbench cert is expired/invalid (its own LAN service)
   10  
   11  
   12  def _urllib_transport(method, url, headers, body):
   13      global _UNVERIFIED
   14      req = urllib.request.Request(url, data=body, method=method, headers=headers)
   15      try:
   16          with urllib.request.urlopen(req, timeout=60, context=_UNVERIFIED) as r:
coverage_service/__init__.py L1
    1  __version__ = "0.1.0"
coverage_service/__main__.py L1
    1  import uvicorn
    2  from .app import app
    3  
    4  if __name__ == "__main__":
    5      # 0.0.0.0 -> reachable over the LAN at http://<host-LAN-IP>:8900
    6      uvicorn.run(app, host="0.0.0.0", port=8900)
coverage_service/app.py L1
    1  import os
    2  import asyncio
    3  import json as _json
    4  from typing import List, Optional
    5  from fastapi import FastAPI, HTTPException
    6  from fastapi.middleware.cors import CORSMiddleware
    7  from fastapi.responses import HTMLResponse, StreamingResponse
    8  from pydantic import BaseModel
    9  from . import db as dbmod
   10  from .store import Store
   11  from .enumerate import build_units
   12  from .dashboard import render
   13  
   14  DB_PATH = os.environ.get("COVERAGE_DB", os.path.join(os.path.dirname(__file__), "data", "coverage.db"))
   15  os.makedirs(os.path.dirname(DB_PATH), exist_ok=True)
   16  # Subjects live here on the host; each maps to /projects/<name> in the agent container.
coverage_service/binary.py L1
    1  """Binary (Mach-O) intake — the seed finder for compiled subjects.
    2  
    3  Front-end swap for the source seed finder: instead of grepping source text, we
    4  enumerate a binary's functions and its **dangerous imported functions** (the binary
    5  analog of sinks: memcpy/strcpy/sprintf/system/ippReadIO/httpRead/…), then seed one
    6  investigation per function that calls a dangerous import — priority by risk. Uses
    7  LIEF for parsing and radare2 for function analysis + xrefs. Deterministic, no tokens.
    8  
    9  The researcher then decompiles each function (r2 `pdc`/`pdg`) and audits it.
   10  """
   11  import os
   12  import subprocess
   13  
   14  try:
   15      import lief
   16  except ImportError:
coverage_service/dashboard.py L1
    1  """REF-All operator console — a self-contained SPA served at :8900/.
    2  
    3  Subject selector + per-subject: coverage, plan progress, findings + funnel, context (view/edit/
    4  generate), wishlist, per-class exhaustiveness. Fetches every endpoint client-side and polls to
    5  stay live. Theme-aware (--wb-* vars); no external assets.
    6  """
    7  
    8  
    9  def render():
   10      return r"""<!doctype html><html lang=en><head><meta charset=utf-8>
   11  <meta name=viewport content="width=device-width,initial-scale=1">
   12  <title>REF-All — Operator Console</title><style>
   13  :root{--bg:var(--wb-bg,#0d1020);--surf:var(--wb-surface,#171b31);--surf2:var(--wb-surface-2,#1e2440);
   14  --ink:var(--wb-ink,#f4f7ff);--mut:var(--wb-muted,#9aa6c7);--acc:var(--wb-accent,#e51b2d);
   15  --acc2:var(--wb-accent-2,#6ee85a);--rule:var(--wb-rule,#2b3156);
   16  --crit:#ff4d5e;--high:#ff9f40;--med:#ffce54;--low:#7a83a6}
coverage_service/db.py L1
    1  import sqlite3
    2  
    3  SCHEMA = """
    4  CREATE TABLE IF NOT EXISTS units (
    5    id TEXT PRIMARY KEY, subject TEXT NOT NULL, type TEXT NOT NULL,
    6    path TEXT, name TEXT, line_start INTEGER, line_end INTEGER, parent_id TEXT,
    7    total_lines INTEGER, indexed INTEGER DEFAULT 1, indexed_src TEXT,
    8    state TEXT DEFAULT 'open', claimed_by TEXT, lease_expires REAL,
    9    summary TEXT, sensitivity TEXT, data_touched TEXT, flows_in TEXT, flows_out TEXT,
   10    leads TEXT, characterized_by TEXT, updated_at REAL
   11  );
   12  CREATE INDEX IF NOT EXISTS ix_units_subject_state ON units(subject, state);
   13  CREATE INDEX IF NOT EXISTS ix_units_parent ON units(parent_id);
   14  CREATE INDEX IF NOT EXISTS ix_units_path ON units(subject, path);
   15  CREATE TABLE IF NOT EXISTS evidence (
   16    id INTEGER PRIMARY KEY AUTOINCREMENT, unit_id TEXT NOT NULL,
coverage_service/enumerate.py L1
    1  from .ids import unit_id
    2  
    3  
    4  def partition_file(total_lines, symbols):
    5      """Return disjoint leaf units covering lines 1..total_lines exactly once.
    6  
    7      A leaf is either a `symbol` (a symbol body) or a `line_range` (the gaps
    8      between symbols: imports, module-level code, comments, blanks). Overlapping
    9      or nested symbols are clamped to the running cursor so the partition stays
   10      disjoint (first-wins), which keeps covered_lines exact.
   11      """
   12      syms = sorted((s for s in symbols if s["line_start"] <= s["line_end"]),
   13                    key=lambda s: s["line_start"])
   14      leaves, cursor = [], 1
   15      for s in syms:
   16          ls, le = max(1, s["line_start"]), min(total_lines, s["line_end"])
coverage_service/extract.py L1
    1  import ast
    2  import os
    3  
    4  TEXT_EXTS = {".py", ".ts", ".tsx", ".js", ".jsx", ".md", ".txt", ".json", ".yml",
    5               ".yaml", ".sh", ".html", ".css", ".cfg", ".toml", ".ini"}
    6  SKIP_DIRS = {".venv", "node_modules", ".git", "__pycache__", ".pytest_cache", "dist", "build"}
    7  
    8  
    9  def total_lines(src):
   10      if not src:
   11          return 0
   12      return src.count("\n") + (0 if src.endswith("\n") else 1)
   13  
   14  
   15  def extract_python(src):
   16      """Top-level symbols only -> non-overlapping by construction (a class is one unit)."""
coverage_service/ids.py L1
    1  import hashlib
    2  
    3  
    4  def unit_id(subject, type, path, line_start, line_end, name=""):
    5      key = f"{subject}|{type}|{path}|{line_start}|{line_end}|{name}"
    6      return hashlib.sha1(key.encode()).hexdigest()
coverage_service/project.py L1
    1  """Project the Coverage Service knowledge base into other toolchain layers.
    2  
    3  Layer 1 (now): a LIVE operator report on the ref-all workbench, generated from the coverage DB,
    4  so the expensive characterization understanding shows up for the operator — not siloed in :8900.
    5  Updates the SAME workbench object in place (mime text/html per the report spec).
    6  """
    7  import json
    8  import html
    9  import os
   10  import urllib.request
   11  import urllib.error
   12  
   13  WB = os.environ.get("REFALL_WORKBENCH_BASE", "http://127.0.0.1:7673")
   14  SLUG = os.environ.get("REFALL_WORKBENCH_SLUG", "ref-all")
   15  COV = os.environ.get("COVERAGE_BASE", "http://localhost:8900")
   16  
coverage_service/seeds.py L1
    1  """Mechanical sensitive-anchor finder — the seed set for the 'juicy path'.
    2  
    3  Classical security research starts at the interesting surface (dangerous sinks,
    4  untrusted sources, secrets, auth) and branches out along the connections. This
    5  module finds those anchors deterministically (regex/AST-free scan, no model, no
    6  tokens) so researchers can start from real attack surface instead of file order.
    7  
    8  Output feeds the Coverage Service `investigations` plan: one investigation per
    9  (file, category) cluster, priority = category risk (lower = more dangerous =
   10  claimed first). Recall over precision — the researcher validates each seed.
   11  """
   12  import os
   13  import re
   14  
   15  from .extract import TEXT_EXTS, SKIP_DIRS
   16  
coverage_service/store.py L1
    1  import json
    2  import time
    3  import threading
    4  import uuid
    5  
    6  
    7  def _now():
    8      return time.time()
    9  
   10  
   11  # Mantis schema.json enums (finding lifecycle).
   12  FINDING_STATUS = {
   13      "VALID",
   14      "FALSE_POSITIVE",
   15      "PROVISIONALLY_VALID",
   16      "NEEDS_RESEARCH",
coverage_service/tools/__init__.py empty
(Empty file)
coverage_service/tools/final_visual.py L1
    1  """Final visual-report gate for REF-All research loops."""
    2  import urllib.request
    3  
    4  from coverage_service import project
    5  from . import report_findings
    6  
    7  
    8  def _content_type(headers):
    9      if hasattr(headers, "get_content_type"):
   10          return headers.get_content_type()
   11      return (headers.get("Content-Type", "") or "").split(";", 1)[0].strip().lower()
   12  
   13  
   14  def verify_rendered_html(oid, *, timeout=30):
   15      url = f"{report_findings.WB}/api/workbenches/{report_findings.SLUG}/objects/{oid}/render"
   16      req = urllib.request.Request(url, method="GET")
coverage_service/tools/report_findings.py L1
    1  """Publish a polished, evidence-anchored findings report for a subject to the workbench.
    2  
    3    PYTHONPATH=. python3 -m coverage_service.tools.report_findings example \
    4        --title "Example Subject - Security Research"
    5  """
    6  import argparse
    7  import html
    8  import json
    9  import os
   10  import urllib.request
   11  import urllib.error
   12  
   13  WB = os.environ.get("REFALL_WORKBENCH_BASE", "http://127.0.0.1:7673")
   14  SLUG = os.environ.get("REFALL_WORKBENCH_SLUG", "ref-all")
   15  COV = os.environ.get("COVERAGE_BASE", "http://localhost:8900")
   16  STATE_DIR = os.path.join(os.path.dirname(os.path.dirname(__file__)), "data")
coverage_service/tools/seed_binary.py L1
    1  """Seed a binary subject's plan from its dangerous imports + call sites (radare2 + LIEF).
    2  
    3  Run inside the agent container:
    4    PYTHONPATH=/projects/REFALL-V10 python3 -m coverage_service.tools.seed_binary example \
    5      "bin/target"
    6  """
    7  import json
    8  import sys
    9  import urllib.request
   10  
   11  from coverage_service import binary
   12  
   13  COV = "http://host.docker.internal:8900"
   14  
   15  
   16  def main(subject, binrel, root=None, load=True, cov=COV):
coverage_service/tools/seed_subject.py L1
    1  """Scan a subject's source for sensitive anchors and load them as the seed plan.
    2  
    3  Run inside the agent container (where the subject source lives):
    4    PYTHONPATH=/projects/REFALL-V10 python3 -m coverage_service.tools.seed_subject example /projects/example
    5  """
    6  import json
    7  import sys
    8  import urllib.request
    9  
   10  from coverage_service.seeds import scan_dir, summarize, seeds_to_investigations
   11  
   12  COV = "http://host.docker.internal:8900"
   13  
   14  
   15  def main(subject, root, load=True, cov=COV):
   16      anchors = scan_dir(root)
coverage_service/tools/vendor_disclosure.py L1
    1  """Build and publish a self-contained vendor disclosure report for a subject.
    2  
    3    PYTHONPATH=. python3 -m coverage_service.tools.vendor_disclosure example \
    4        --title "Example Subject - Vendor Disclosure Report"
    5  """
    6  import argparse
    7  import datetime as _dt
    8  import html
    9  import json
   10  import os
   11  import re
   12  import urllib.error
   13  import urllib.parse
   14  import urllib.request
   15  
   16  WB = os.environ.get("REFALL_WORKBENCH_BASE", "http://127.0.0.1:7673")
arcticbase/backend/src/arctic_base/__init__.py L1
    1  __version__ = "0.1.0"
arcticbase/backend/src/arctic_base/__main__.py L1
    1  """Entry point: `python -m arctic_base` or the `arctic-base` console script.
    2  
    3  Reads host/port from settings (env-driven) so the default is auto-lan:7673.
    4  """
    5  
    6  from __future__ import annotations
    7  
    8  import uvicorn
    9  
   10  from arctic_base.config import get_settings
   11  
   12  
   13  def main() -> None:
   14      settings = get_settings()
   15      uvicorn.run(
   16          "arctic_base.main:app",
arcticbase/backend/src/arctic_base/api/__init__.py L1
    1  from fastapi import APIRouter
    2  
    3  from arctic_base.api import (
    4      agent,
    5      objects,
    6      refall_capabilities,
    7      refall_closeout,
    8      refall_coverage,
    9      refall_disposition,
   10      refall_ledger,
   11      refall_ops,
   12      refall_ops_commands,
   13      refall_ops_context,
   14      refall_ops_sse,
   15      refall_pending_actions,
   16      refall_reports,
arcticbase/backend/src/arctic_base/api/agent.py L1
    1  """Agent-facing endpoints.
    2  
    3  Designed to minimize context cost: `/agent` is a redirect to the OpenAPI spec, the conventions
    4  doc is tight and only covers what OpenAPI can't express, and the bootstrap endpoint replaces a
    5  multi-step setup with one call.
    6  """
    7  
    8  from __future__ import annotations
    9  
   10  from datetime import UTC, datetime
   11  from typing import Annotated, Literal
   12  
   13  from fastapi import APIRouter, Body, Depends, Request
   14  from fastapi.responses import PlainTextResponse, RedirectResponse
   15  from pydantic import BaseModel, Field
   16  
arcticbase/backend/src/arctic_base/api/deps.py L1
    1  from functools import lru_cache
    2  
    3  from arctic_base.config import get_settings
    4  from arctic_base.storage.filesystem import FilesystemStorage
    5  
    6  
    7  @lru_cache(maxsize=1)
    8  def get_storage() -> FilesystemStorage:
    9      settings = get_settings()
   10      return FilesystemStorage(settings.data)
arcticbase/backend/src/arctic_base/api/errors.py L1
    1  """RFC 7807-style problem responses for storage-layer exceptions."""
    2  
    3  from fastapi import FastAPI, Request
    4  from fastapi.responses import JSONResponse
    5  
    6  from arctic_base.storage.errors import (
    7      AlreadyExists,
    8      InvalidTheme,
    9      NotFound,
   10      StaleETag,
   11  )
   12  
   13  
   14  def _problem(status: int, title: str, detail: str, type_: str = "about:blank") -> JSONResponse:
   15      return JSONResponse(
   16          {"type": type_, "title": title, "status": status, "detail": detail},
arcticbase/backend/src/arctic_base/api/objects.py L1
    1  from datetime import UTC, datetime
    2  from typing import Annotated
    3  
    4  from fastapi import APIRouter, Body, Depends, Header, HTTPException, Request
    5  from fastapi.responses import Response
    6  from starlette.datastructures import UploadFile
    7  from pydantic import BaseModel, Field
    8  
    9  from arctic_base.api.deps import get_storage
   10  from arctic_base.config import get_settings
   11  from arctic_base.models import ObjectKind, ObjectMeta
   12  from arctic_base.storage.filesystem import FilesystemStorage, new_object_id
   13  
   14  router = APIRouter(prefix="/workbenches/{slug}/objects", tags=["objects"])
   15  
   16  
arcticbase/backend/src/arctic_base/api/refall_capabilities.py L1
    1  from __future__ import annotations
    2  
    3  import secrets
    4  from typing import Annotated
    5  
    6  from fastapi import APIRouter, Body, Depends, HTTPException, Request
    7  from pydantic import BaseModel, Field
    8  
    9  from arctic_base.config import Settings, get_settings
   10  from arctic_base.api.deps import get_storage
   11  from arctic_base.refall import ledger
   12  from arctic_base.refall.coverage_store import store_for_workbench
   13  from arctic_base.storage.filesystem import FilesystemStorage
   14  
   15  router = APIRouter(prefix="/workbenches/{slug}", tags=["refall-capabilities"])
   16  
arcticbase/backend/src/arctic_base/api/refall_closeout.py L1
    1  from __future__ import annotations
    2  
    3  from typing import Annotated
    4  
    5  from fastapi import APIRouter, Body, Depends, HTTPException
    6  from pydantic import BaseModel, Field
    7  
    8  from arctic_base.api.deps import get_storage
    9  from arctic_base.refall import ledger
   10  from arctic_base.refall.closeout_gate import evaluate_closeout_gate
   11  from arctic_base.storage.filesystem import FilesystemStorage
   12  
   13  router = APIRouter(prefix="/workbenches/{slug}/closeout", tags=["refall-closeout"])
   14  
   15  
   16  class CloseoutOverrideReq(BaseModel):
arcticbase/backend/src/arctic_base/api/refall_coverage.py L1
    1  from __future__ import annotations
    2  
    3  from typing import Annotated
    4  
    5  from fastapi import APIRouter, Body, Depends, HTTPException
    6  from pydantic import BaseModel, Field, ValidationError
    7  
    8  from arctic_base.api.deps import get_storage
    9  from arctic_base.refall.contracts import (
   10      ContextPackage,
   11      FindingCandidate,
   12      ScopeOverlay,
   13      SourceInventoryPackage,
   14      ThreatModelPackage,
   15  )
   16  from arctic_base.refall import ledger
arcticbase/backend/src/arctic_base/api/refall_disposition.py L1
    1  from __future__ import annotations
    2  
    3  from pathlib import Path
    4  from typing import Annotated
    5  
    6  from fastapi import APIRouter, Body, Depends, HTTPException
    7  from pydantic import BaseModel, Field
    8  
    9  from arctic_base.api.deps import get_storage
   10  from arctic_base.refall import ledger
   11  from arctic_base.refall.coverage_store import store_for_workbench
   12  from arctic_base.storage.filesystem import FilesystemStorage
   13  
   14  router = APIRouter(prefix="/workbenches/{slug}", tags=["refall-disposition"])
   15  
   16  
arcticbase/backend/src/arctic_base/api/refall_ledger.py L1
    1  from __future__ import annotations
    2  
    3  from typing import Annotated, Any
    4  
    5  from fastapi import APIRouter, Body, Depends, HTTPException
    6  
    7  from arctic_base.api.deps import get_storage
    8  from arctic_base.refall import ledger
    9  from arctic_base.storage.filesystem import FilesystemStorage
   10  
   11  router = APIRouter(prefix="/workbenches/{slug}/ledger", tags=["refall-ledger"])
   12  
   13  
   14  @router.post("/entries", status_code=201, operation_id="appendRefallLedgerEntry")
   15  def append_ledger_entry(
   16      slug: str,
arcticbase/backend/src/arctic_base/api/refall_ops.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  import shutil
    5  from pathlib import Path
    6  from typing import Annotated
    7  
    8  from fastapi import APIRouter, Depends, HTTPException
    9  
   10  from arctic_base.api.deps import get_storage
   11  from arctic_base.config import Settings, get_settings
   12  from arctic_base.storage.filesystem import FilesystemStorage
   13  
   14  router = APIRouter(prefix="/refall/ops", tags=["refall-ops"])
   15  
   16  
arcticbase/backend/src/arctic_base/api/refall_ops_context.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  from pathlib import Path
    5  from typing import Annotated
    6  
    7  from fastapi import APIRouter, Depends, HTTPException
    8  
    9  from arctic_base.api.deps import get_storage
   10  from arctic_base.api.refall_ops import require_ops
   11  from arctic_base.config import Settings
   12  from arctic_base.refall import ledger
   13  from arctic_base.refall.coverage_store import store_for_workbench
   14  from arctic_base.storage.errors import NotFound
   15  from arctic_base.storage.filesystem import FilesystemStorage
   16  
arcticbase/backend/src/arctic_base/api/refall_ops_sse.py L1
    1  from __future__ import annotations
    2  
    3  import asyncio
    4  import json
    5  from collections import deque
    6  from datetime import UTC, datetime
    7  from typing import Annotated, Any
    8  
    9  from fastapi import APIRouter, Body, Depends
   10  from fastapi.responses import StreamingResponse
   11  
   12  from arctic_base.api.refall_ops import require_ops
   13  from arctic_base.config import Settings
   14  
   15  router = APIRouter(prefix="/refall/ops", tags=["refall-ops-events"])
   16  
arcticbase/backend/src/arctic_base/api/refall_pending_actions.py L1
    1  from __future__ import annotations
    2  
    3  from typing import Annotated
    4  
    5  from fastapi import APIRouter, Depends, HTTPException
    6  from pydantic import BaseModel, Field
    7  
    8  from arctic_base.api.deps import get_storage
    9  from arctic_base.refall import ledger
   10  from arctic_base.refall.coverage_store import store_for_workbench
   11  from arctic_base.storage.filesystem import FilesystemStorage
   12  
   13  router = APIRouter(prefix="/workbenches/{slug}/pending-actions", tags=["refall-pending-actions"])
   14  
   15  
   16  class PendingActionReq(BaseModel):
arcticbase/backend/src/arctic_base/api/refall_reports.py L1
    1  from __future__ import annotations
    2  
    3  from pathlib import Path
    4  from typing import Annotated
    5  
    6  from fastapi import APIRouter, Depends, HTTPException
    7  from fastapi.responses import PlainTextResponse
    8  
    9  from arctic_base.api.deps import get_storage
   10  from arctic_base.storage.filesystem import FilesystemStorage
   11  
   12  router = APIRouter(tags=["refall-reports"])
   13  
   14  REPORT_ROOT = Path(__file__).resolve().parents[1] / "refall_report"
   15  SPEC_DIR = REPORT_ROOT / "spec"
   16  TEMPLATE_PATH = REPORT_ROOT / "template.html"
arcticbase/backend/src/arctic_base/api/refall_runtime.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  from datetime import UTC, datetime
    5  from pathlib import Path
    6  from typing import Annotated
    7  
    8  from fastapi import APIRouter, Body, Depends, HTTPException, Response
    9  from pydantic import BaseModel, ConfigDict, Field, ValidationError
   10  
   11  from arctic_base.api.deps import get_storage
   12  from arctic_base.models import ObjectKind, ObjectMeta
   13  from arctic_base.refall import ledger
   14  from arctic_base.refall.artifact_store import save_artifact
   15  from arctic_base.refall.coverage_store import store_for_workbench
   16  from arctic_base.refall.evidence_graph import graph_from_deltas
arcticbase/backend/src/arctic_base/api/render.py L1
    1  """Themed render endpoints.
    2  
    3  `GET /api/workbenches/{slug}/objects/{oid}/render` returns:
    4    - For `kind=md`: themed HTML body fragment (CSS vars + rendered MD).
    5    - For `kind=html` or `approval-html`: the raw HTML with theme CSS vars and the
    6      `window.workbench` postMessage bridge injected near the top of the document.
    7  """
    8  
    9  from typing import Annotated
   10  
   11  from fastapi import APIRouter, Depends
   12  from fastapi.responses import HTMLResponse, Response
   13  from markdown_it import MarkdownIt
   14  
   15  from arctic_base.api.deps import get_storage
   16  from arctic_base.models import ObjectKind, Theme
arcticbase/backend/src/arctic_base/api/responses.py L1
    1  import json
    2  from datetime import UTC, datetime
    3  from typing import Annotated, Any
    4  
    5  from fastapi import APIRouter, Body, Depends
    6  from fastapi.responses import Response
    7  from pydantic import BaseModel, Field
    8  
    9  from arctic_base.api.deps import get_storage
   10  from arctic_base.models import ResponseEvent, ResponseKind, Submitter
   11  from arctic_base.storage.filesystem import FilesystemStorage
   12  
   13  router = APIRouter(prefix="/workbenches/{slug}/objects/{oid}/responses", tags=["responses"])
   14  
   15  
   16  class ResponseAppend(BaseModel):
arcticbase/backend/src/arctic_base/api/templates_data.py L1
    1  """HTML templates served to agents via /api/agent/templates.
    2  
    3  Each template is a self-contained HTML doc that:
    4  
    5  - references `--wb-*` CSS variables so it auto-themes when served from a workbench;
    6  - bakes in fallback values so it still works opened standalone (per the review-HTML rules);
    7  - includes `window.workbench` feature-detection so state mirrors to the API when present and to
    8    `localStorage` always;
    9  - exposes a JSON-export button for the canonical recorded state.
   10  
   11  Agents are expected to fetch the template, replace the marked sections (questions, prose, etc.),
   12  and upload the result via POST /api/workbenches/{slug}/objects.
   13  """
   14  
   15  from __future__ import annotations
   16  
arcticbase/backend/src/arctic_base/api/theme.py L1
    1  from typing import Annotated
    2  
    3  from fastapi import APIRouter, Body, Depends, UploadFile
    4  from fastapi.responses import PlainTextResponse, Response
    5  from pydantic import BaseModel
    6  
    7  from arctic_base.api.deps import get_storage
    8  from arctic_base.models import Theme
    9  from arctic_base.storage.filesystem import FilesystemStorage
   10  
   11  router = APIRouter(prefix="/workbenches/{slug}/theme", tags=["theme"])
   12  
   13  
   14  class ThemeRawBody(BaseModel):
   15      raw: str
   16  
arcticbase/backend/src/arctic_base/api/workbenches.py L1
    1  import json
    2  from datetime import UTC, datetime
    3  from typing import Annotated
    4  
    5  from fastapi import APIRouter, Body, Depends
    6  from fastapi.responses import Response
    7  from pydantic import BaseModel, Field
    8  
    9  from arctic_base.api.deps import get_storage
   10  from arctic_base.models import Banner, WorkbenchMeta
   11  from arctic_base.storage.filesystem import FilesystemStorage
   12  
   13  router = APIRouter(prefix="/workbenches", tags=["workbenches"])
   14  
   15  
   16  class WorkbenchCreate(BaseModel):
arcticbase/backend/src/arctic_base/config.py L1
    1  import os
    2  import socket
    3  from ipaddress import ip_address
    4  from pathlib import Path
    5  
    6  from pydantic import Field, model_validator
    7  from pydantic_settings import BaseSettings, SettingsConfigDict
    8  
    9  AUTO_LAN_HOST = "auto-lan"
   10  DEFAULT_REFALL_REPO_ROOT = Path(__file__).resolve().parents[4]
   11  
   12  
   13  def _is_private_lan_ipv4(value: str) -> bool:
   14      try:
   15          address = ip_address(value)
   16      except ValueError:
arcticbase/backend/src/arctic_base/logging_setup.py L1
    1  """Structured JSON logging to stdout. Request IDs propagated via contextvar."""
    2  
    3  from __future__ import annotations
    4  
    5  import contextvars
    6  import json
    7  import logging
    8  import time
    9  import uuid
   10  from collections import deque
   11  from typing import TypedDict
   12  
   13  from starlette.requests import Request
   14  from starlette.types import ASGIApp
   15  
   16  _request_id: contextvars.ContextVar[str] = contextvars.ContextVar("request_id", default="-")
arcticbase/backend/src/arctic_base/main.py L1
    1  import logging
    2  import shutil
    3  from pathlib import Path
    4  
    5  from fastapi import FastAPI
    6  from fastapi.responses import FileResponse, JSONResponse, RedirectResponse
    7  from fastapi.staticfiles import StaticFiles
    8  
    9  from arctic_base import __version__
   10  from arctic_base.api import agent_top, router as api_router
   11  from arctic_base.api.errors import install_error_handlers
   12  from arctic_base.config import get_settings
   13  from arctic_base.logging_setup import RequestIdMiddleware, configure_logging
   14  
   15  
   16  def _seed_if_empty(data: Path, seed: Path) -> None:
arcticbase/backend/src/arctic_base/models.py L1
    1  from datetime import datetime
    2  from enum import StrEnum
    3  from typing import Any
    4  
    5  from pydantic import BaseModel, ConfigDict, Field, computed_field, model_validator
    6  
    7  ManifestVersion = 1
    8  
    9  
   10  class BannerKind(StrEnum):
   11      upload = "upload"
   12      url = "url"
   13  
   14  
   15  class Banner(BaseModel):
   16      kind: BannerKind
arcticbase/backend/src/arctic_base/refall/__init__.py L1
    1  """REFALL-specific ArcticBase extensions."""
arcticbase/backend/src/arctic_base/refall/artifact_store.py L1
    1  from __future__ import annotations
    2  
    3  import hashlib
    4  import sqlite3
    5  import time
    6  import uuid
    7  from pathlib import Path
    8  
    9  from arctic_base.refall.runtime_state import runtime_root
   10  
   11  
   12  def save_artifact(
   13      conn: sqlite3.Connection,
   14      subject_path: Path,
   15      *,
   16      subject: str,
arcticbase/backend/src/arctic_base/refall/capabilities.py L1
    1  from __future__ import annotations
    2  
    3  from arctic_base.refall.coverage_store import CoverageStore
    4  
    5  
    6  def grant(store: CoverageStore, subject: str, data: dict) -> dict:
    7      return store.grant_capability(subject, data)
    8  
    9  
   10  def authorize(store: CoverageStore, subject: str, request: dict) -> dict:
   11      return store.authorize_execution(subject, request)
arcticbase/backend/src/arctic_base/refall/coverage_db.py L1
    1  from __future__ import annotations
    2  
    3  import sqlite3
    4  from pathlib import Path
    5  
    6  
    7  SCHEMA = """
    8  CREATE TABLE IF NOT EXISTS units (
    9    id TEXT PRIMARY KEY, subject TEXT NOT NULL, type TEXT NOT NULL,
   10    path TEXT, name TEXT, line_start INTEGER, line_end INTEGER, parent_id TEXT,
   11    total_lines INTEGER, indexed INTEGER DEFAULT 1, indexed_src TEXT,
   12    state TEXT DEFAULT 'open', claimed_by TEXT, lease_expires REAL,
   13    summary TEXT, sensitivity TEXT, data_touched TEXT, flows_in TEXT, flows_out TEXT,
   14    leads TEXT, characterized_by TEXT, updated_at REAL
   15  );
   16  CREATE INDEX IF NOT EXISTS ix_units_subject_state ON units(subject, state);
arcticbase/backend/src/arctic_base/refall/disposition.py L1
    1  from __future__ import annotations
    2  
    3  from arctic_base.refall.coverage_store import CoverageStore
    4  
    5  
    6  def transition_finding(store: CoverageStore, subject: str, finding_id: str, data: dict) -> dict:
    7      return store.transition_finding(subject, finding_id, data)
    8  
    9  
   10  def list_transitions(
   11      store: CoverageStore,
   12      subject: str,
   13      *,
   14      finding_id: str | None = None,
   15      status: str | None = None,
   16  ) -> list[dict]:
arcticbase/backend/src/arctic_base/refall/evidence_graph.py L1
    1  from __future__ import annotations
    2  
    3  
    4  def graph_from_deltas(deltas: list[dict]) -> dict:
    5      nodes: dict[str, dict] = {}
    6      edges: list[dict] = []
    7      for delta in deltas:
    8          operation = delta.get("operation")
    9          if operation == "upsert_node":
   10              node_id = str(delta.get("node_id"))
   11              nodes[node_id] = {
   12                  "id": node_id,
   13                  "type": delta.get("node_type"),
   14                  "properties": delta.get("properties") or {},
   15              }
   16          elif operation == "upsert_edge":
arcticbase/backend/src/arctic_base/refall/ledger.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  import uuid
    5  from collections import Counter
    6  from datetime import UTC, datetime
    7  from pathlib import Path
    8  from typing import Any
    9  
   10  from arctic_base.storage.filesystem import FilesystemStorage
   11  
   12  
   13  LEDGER_KINDS = {
   14      "fact",
   15      "lead",
   16      "finding",
arcticbase/backend/src/arctic_base/refall/pending_actions.py L1
    1  from __future__ import annotations
    2  
    3  from arctic_base.refall.coverage_store import CoverageStore
    4  
    5  
    6  def create(store: CoverageStore, subject: str, data: dict) -> dict:
    7      return store.create_pending_action(subject, data)
    8  
    9  
   10  def list_actions(
   11      store: CoverageStore,
   12      subject: str,
   13      *,
   14      status: str | None = None,
   15      kind: str | None = None,
   16      include_terminal: bool = True,
arcticbase/backend/src/arctic_base/refall/source_inventory.py L1
    1  from __future__ import annotations
    2  
    3  import hashlib
    4  import os
    5  from pathlib import Path
    6  from typing import Any
    7  
    8  from arctic_base.refall.contracts import SourceInventoryPackage
    9  
   10  
   11  READ_CHUNK_BYTES = 1024 * 1024
   12  SKIP_DIRS = {".git", ".refall", "refallsubloop", "__pycache__", "node_modules", ".venv"}
   13  INCLUDE_DOT_DIRS = {".github", ".gitlab", ".circleci", ".azure", ".azuredevops", ".devcontainer"}
   14  SOURCE_EXTENSIONS = {
   15      ".bash": "shell",
   16      ".c": "c",
arcticbase/backend/src/arctic_base/refall/subjects.py L1
    1  from __future__ import annotations
    2  
    3  import json
    4  from dataclasses import dataclass
    5  from datetime import UTC, datetime
    6  from pathlib import Path
    7  
    8  from arctic_base.config import Settings
    9  from arctic_base.models import WorkbenchMeta
   10  from arctic_base.storage.errors import AlreadyExists, NotFound
   11  from arctic_base.storage.filesystem import FilesystemStorage
   12  
   13  
   14  SUBJECT_DIRECTORIES = (
   15      "Notes",
   16      "Screenshots",
arcticbase/backend/src/arctic_base/storage/__init__.py L1
    1  """Storage Protocol — the canonical interface every storage backend implements.
    2  
    3  v1 ships only `FilesystemStorage`. Future SQLite or hybrid backends drop in here.
    4  """
    5  
    6  from __future__ import annotations
    7  
    8  from typing import Protocol, runtime_checkable
    9  
   10  from arctic_base.models import ObjectMeta, ResponseEvent, Theme, WorkbenchMeta
   11  
   12  
   13  @runtime_checkable
   14  class Storage(Protocol):
   15      # --- workbenches ---
   16      def list_workbenches(self) -> list[WorkbenchMeta]: ...
arcticbase/backend/src/arctic_base/storage/errors.py L1
    1  class StorageError(Exception):
    2      """Base for storage-layer failures."""
    3  
    4  
    5  class NotFound(StorageError):
    6      pass
    7  
    8  
    9  class AlreadyExists(StorageError):
   10      pass
   11  
   12  
   13  class StaleETag(StorageError):
   14      """Raised when If-Match precondition fails on object content writes."""
   15  
   16      def __init__(self, expected: str, actual: str) -> None:
arcticbase/backend/src/arctic_base/storage/filesystem.py L1
    1  """Filesystem-backed Storage implementation.
    2  
    3  Layout:
    4      data/
    5        workbenches/{slug}/
    6          .lock                       advisory lock for manifest/object-meta writes
    7          manifest.json
    8          theme.md
    9          theme/
   10          objects/{id}/
   11            .lock                     advisory lock for response monotonicity
   12            meta.json
   13            content
   14          responses/{id}/
   15            {version}.json
   16            latest.json
arcticbase/backend/src/arctic_base/theme.py L1
    1  """Parse and serialize the workbench `theme.md` format.
    2  
    3  The file is YAML frontmatter (between `---` fences) followed by a Markdown prose body.
    4  """
    5  
    6  from __future__ import annotations
    7  
    8  import io
    9  
   10  from ruamel.yaml import YAML
   11  
   12  from arctic_base.models import (
   13      REQUIRED_CORE_TOKENS,
   14      REQUIRED_ROUNDED,
   15      REQUIRED_SPACING,
   16      REQUIRED_TYPOGRAPHY,
arcticbase/frontend/src/lib/api.ts L1
    1  // Arctic Base API client. Single-origin; relative URLs.
    2  
    3  export type Banner =
    4    | { kind: "upload"; ref: string }
    5    | { kind: "url"; ref: string };
    6  
    7  export interface WorkbenchMeta {
    8    version: number;
    9    slug: string;
   10    title: string;
   11    description: string;
   12    url: string | null;
   13    ip: string | null;
   14    local_path: string | null;
   15    tags: string[];
   16    banner: Banner | null;
arcticbase/frontend/src/lib/bridge.ts L1
    1  // Parent-side window.workbench bridge handler. Listens for postMessage events from served-HTML
    2  // iframes and routes them to the API. Mounted once at the SPA root.
    3  
    4  import { api } from "./api";
    5  
    6  interface IncomingMessage {
    7    __ab: string; // request id
    8    type: "save" | "saveAll" | "load" | "loadAll" | "submit";
    9    payload: unknown;
   10    ctx: { workbench: string; object: string };
   11  }
   12  
   13  interface RemoteSubscribe {
   14    __ab_remote_subscribe?: boolean;
   15    __ab_remote_unsubscribe?: boolean;
   16    ctx: { workbench: string; object: string };
arcticbase/frontend/src/lib/mini-md.ts L1
    1  // Minimal markdown → HTML renderer for inline use (workbench descriptions).
    2  // Handles: **bold**, *italic*, `code`, [text](url), line breaks, and bullet lists (- item).
    3  // HTML escapes everything else; no <script>/<iframe>/raw HTML pass-through.
    4  
    5  export function renderMiniMd(src: string): string {
    6    if (!src) return "";
    7    const escaped = src
    8      .replace(/&/g, "&amp;")
    9      .replace(/</g, "&lt;")
   10      .replace(/>/g, "&gt;");
   11  
   12    const lines = escaped.split(/\r?\n/);
   13    const out: string[] = [];
   14    let inList = false;
   15    for (const raw of lines) {
   16      const m = raw.match(/^\s*-\s+(.*)$/);
arcticbase/frontend/src/lib/router.ts L1
    1  // Minimal hash-free pathname router.
    2  
    3  import { writable, type Readable } from "svelte/store";
    4  
    5  export interface Route {
    6    path: string;
    7    params: Record<string, string>;
    8    query: URLSearchParams;
    9  }
   10  
   11  function parse(): Route {
   12    const url = new URL(window.location.href);
   13    return { path: url.pathname, params: {}, query: url.searchParams };
   14  }
   15  
   16  const _route = writable<Route>(parse());
arcticbase/frontend/src/lib/theme.ts L1
    1  import type { Theme } from "./api";
    2  
    3  export function themeToCssVars(theme: Theme): string {
    4    const lines: string[] = [];
    5    for (const [k, v] of Object.entries(theme.tokens.core)) lines.push(`--wb-${k}: ${v};`);
    6    for (const [k, v] of Object.entries(theme.tokens.rounded)) lines.push(`--wb-rounded-${k}: ${v};`);
    7    for (const [k, v] of Object.entries(theme.tokens.spacing)) lines.push(`--wb-space-${k}: ${v};`);
    8    for (const [k, v] of Object.entries(theme.tokens.custom)) lines.push(`--wb-${k}: ${v};`);
    9    for (const [name, style] of Object.entries(theme.tokens.typography)) {
   10      if (style.fontFamily) lines.push(`--wb-font-${name}: ${style.fontFamily};`);
   11      if (style.fontSize) lines.push(`--wb-font-size-${name}: ${style.fontSize};`);
   12      if (style.fontWeight) lines.push(`--wb-font-weight-${name}: ${style.fontWeight};`);
   13      if (style.lineHeight) lines.push(`--wb-line-height-${name}: ${style.lineHeight};`);
   14      if (style.letterSpacing) lines.push(`--wb-letter-spacing-${name}: ${style.letterSpacing};`);
   15    }
   16    return lines.join(" ");
arcticbase/frontend/src/main.ts L1
    1  import { mount } from 'svelte'
    2  import '@fontsource/bungee/400.css'
    3  import '@fontsource/black-ops-one/400.css'
    4  import '@fontsource/vt323/400.css'
    5  import '@fontsource/space-grotesk/400.css'
    6  import '@fontsource/space-grotesk/500.css'
    7  import '@fontsource/space-grotesk/700.css'
    8  import '@fontsource/jetbrains-mono/400.css'
    9  import '@fontsource/jetbrains-mono/500.css'
   10  import '@fontsource/jetbrains-mono/700.css'
   11  import './app.css'
   12  import App from './App.svelte'
   13  
   14  const app = mount(App, {
   15    target: document.getElementById('app')!,
   16  })
toolchain/ctl.sh L1
    1  #!/usr/bin/env bash
    2  set -u
    3  
    4  ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
    5  REGISTRY="$ROOT/config/services.json"
    6  RUN_DIR="$ROOT/run"
    7  LOG_DIR="$ROOT/logs"
    8  PATH="$ROOT/bin:$ROOT/.venv/bin:$ROOT/node_modules/.bin:$PATH"
    9  export PATH
   10  export TOOLCHAIN_ROOT="$ROOT"
   11  
   12  usage() {
   13    cat <<'EOF'
   14  usage:
   15    ctl.sh up [service...]
   16    ctl.sh down [service...]
deploy/cron/refall-supervisor.sh L1
    1  #!/bin/zsh
    2  # REFALL loop supervisor cron wrapper.
    3  #
    4  #   refall-supervisor.sh fast   -> liveness + recovery only (cheap, every minute)
    5  #   refall-supervisor.sh full   -> + watchdog pass, wishlist, relevance, report
    6  #
    7  # Installed and removed automatically when a standing loop starts/stops. It
    8  # never tears the stack down and never edits its own crontab entry.
    9  set -u
   10  
   11  mode="${1:-full}"
   12  subject="${2:-${REFALL_SUPERVISOR_SUBJECT:-all}}"
   13  
   14  export REFALL_REPO_ROOT="${REFALL_REPO_ROOT:-/opt/refall/REFALL-V10}"
   15  
   16  # Config lives in a sidecar env file, NOT inline on the crontab line. macOS
deploy/cron/refall-watchdog-cron.sh L1
    1  #!/bin/zsh
    2  set -u
    3  
    4  export REFALL_REPO_ROOT="${REFALL_REPO_ROOT:-/opt/refall/REFALL-V10}"
    5  export REFALL_PUBLIC_BASE_URL="${REFALL_PUBLIC_BASE_URL:-http://127.0.0.1:7673}"
    6  export REFALL_WORKBENCH_BASE="${REFALL_WORKBENCH_BASE:-$REFALL_PUBLIC_BASE_URL}"
    7  export REFALL_OPENHANDS_BASE="${REFALL_OPENHANDS_BASE:-http://127.0.0.1:8000}"
    8  export REFALL_WATCHDOG_SUBJECT="${REFALL_WATCHDOG_SUBJECT:-all}"
    9  export REFALL_PYTHON="${REFALL_PYTHON:-/opt/homebrew/bin/python3}"
   10  export PYTHONPATH="$REFALL_REPO_ROOT${PYTHONPATH:+:$PYTHONPATH}"
   11  
   12  mkdir -p "$REFALL_REPO_ROOT/logs"
   13  cd "$REFALL_REPO_ROOT" || exit 1
   14  
   15  tmp_out="$(mktemp -t refall-watchdog.XXXXXX)"
   16  cleanup_tmp() {
drive_context.py L1
    1  #!/usr/bin/env python3
    2  """Generate a subject's INFO.md context briefing (deepsec-style) and store it in the service.
    3  
    4  One strong agent reads the subject and writes a lean, concrete project briefing that every
    5  researcher reads first — cutting false positives by giving them the subject's auth model,
    6  conventions, and known-safe patterns up front.
    7  
    8    PYTHONPATH=. python3 drive_context.py --subject example
    9  """
   10  import argparse
   11  
   12  from ref_all_executor.oh_client import OHClient
   13  from ref_all_executor.coverage_client import CoverageClient
   14  from ref_all_executor import config
   15  from ref_all_executor.usage import record_and_delete
   16  from drive_characterize import log
drive_characterize.py L1
    1  #!/usr/bin/env python3
    2  """Sustained characterize driver for the REF-All Coverage Service.
    3  
    4  Loops: dispatch batches of parallel Claude `characterize` agents (each claims+characterizes
    5  several coverage units with evidence) until the subject's gate opens (fully characterized) or
    6  a STOP is requested. Killable — see kill().
    7  
    8  Usage:
    9    PYTHONPATH=. python3 drive_characterize.py run   --subject example --agents 5 --units 10
   10    PYTHONPATH=. python3 drive_characterize.py status --subject example
   11    PYTHONPATH=. python3 drive_characterize.py kill          # graceful+forceful stop
   12  """
   13  import os, sys, time, json, threading, signal, argparse
   14  
   15  from ref_all_executor.oh_client import OHClient
   16  from ref_all_executor.coverage_client import CoverageClient
refall_infra.py L1
    1  #!/usr/bin/env python3
    2  """Convenience wrapper for `python3 -m ref_all_executor infra`."""
    3  
    4  import sys
    5  
    6  from ref_all_executor.cli import main
    7  
    8  
    9  if __name__ == "__main__":
   10      raise SystemExit(main(["infra", *sys.argv[1:]]))
refall_down.py L1
    1  #!/usr/bin/env python3
    2  """Convenience wrapper for `python3 -m ref_all_executor.cli down`."""
    3  
    4  import sys
    5  
    6  from ref_all_executor.cli import main
    7  
    8  
    9  if __name__ == "__main__":
   10      raise SystemExit(main(["down", *sys.argv[1:]]))

Search the atlas

Source reference

PRODUCT VIEW

Screenshot