Capturing content

Updated

Every reporting application chooses its own capture mode with ASSAY_CAPTURE. The mode decides whether prompt and response content leaves the application at all. Assay itself never inspects, redacts or classifies what it receives, so the decisions on this page are made in the reporting application.

Usage capture

usage is the default. It records:

record identifiers and how records link into a run tree (agents, steps, attempts, tool calls, sub-agents);
the operation and agent class;
the provider, the requested model and the model that actually responded;
tool names, timings, finish reasons, outcomes, approval outcomes and failure classes;
the usage metrics the provider reported: input, output, cache-read, cache-write, reasoning, image input and image output tokens, audio seconds and search units.

It does not record instructions, messages, model output, structured output, tool arguments or results, or exception messages. Usage capture is never sampled: every run is recorded.

Full capture

ASSAY_CAPTURE=full adds the supported content: instructions, messages, responses, structured output, tool arguments and results, exception messages, and the inputs of non-agent operations.

Some things are never sent in either mode: attachments and media bytes or their locations, embedding vectors, credentials, headers, endpoints, provider options, continuation tokens, replay blocks and raw provider responses.

Captured content is stored in your Assay database. Only people with content access can read it; see Access, retention and erasure. Assay's own guide at /assay/risk sets out what full capture stores and who can see it. Read it before you switch an application to full.

Sampling

In full mode, ASSAY_SAMPLE_RATE sets the share of runs whose content is captured, from 0 to 1. It defaults to 1. To use a different rate for particular agents, publish the client config and add exact agent-class entries to agent_sample_rates:

'agent_sample_rates' => [
    App\Ai\SupportAgent::class => 0.1,
],

The decision is made once, when the root of a run starts, and applies to the whole tree, including sub-agents and linked non-agent operations. Sampling only affects content. Usage is still recorded for every run.

Capturing content when a run fails

ASSAY_ALWAYS_ON_FAILURE defaults to true. For an agent run that was not sampled, the client still sends its usage straight away, and keeps the filtered content in memory, up to 512 KiB per run by default (ASSAY_FAILURE_BUFFER_BYTES), dropping the oldest content first.

If the run succeeds, the buffered content is discarded.
If an agent or step fails, the buffered content is sent once, attached to the records already sent. The failed run is marked complete or truncated depending on whether anything was dropped from the buffer.

Standalone non-agent operations, such as a single embeddings or image call, have no failure event, so they are not covered. Set ASSAY_ALWAYS_ON_FAILURE=false to turn this off.

Subjects

A subject is an opaque string you put in Laravel Context at assay.subject before a run starts, such as user:123 or team:42. It lets you break usage down per subject and erase one subject's content later. Assay never looks the value up. Missing subjects are stored and shown as unknown; the Pipeline Health table reports what share of each application's runs have no subject.

Filtering content before it is sent

To mask, hash, suppress or drop content, rebind ArtisanBuild\BuiltForCloudContracts\PayloadFilter in a service provider of the reporting application. The default binding passes everything through. The filter runs after the client has picked out what it is allowed to send and before anything is queued.

A filter sees payloads for every Built for Cloud product the application talks to, so it must check the product first and return other products' payloads unchanged:

use ArtisanBuild\BuiltForCloudContracts\OutboundPayload;
use ArtisanBuild\BuiltForCloudContracts\PayloadDisposition;

public function filter(OutboundPayload $payload): ?OutboundPayload
{
    if ($payload->product !== 'assay') {
        return $payload;
    }

    $data = $payload->data;
    unset($data['content']);

    return new OutboundPayload($payload->product, $payload->kind, $payload->schemaVersion, PayloadDisposition::Droppable, $data, $payload->attributes);
}

That example keeps usage and drops all content. The risk guide at /assay/risk has the complete recipes for the four supported patterns: drop one agent, mask known content fields, hash known identifiers, and keep usage while dropping content.

Rules the client enforces:

A filter may change content only. Identifiers, tree links, usage, timings and capture metadata are restored from the client's own copy after the filter runs.
Returning null drops the record. A filter that throws also drops the record. Either way nothing is sent unfiltered, and the drop is added to the hook-drop count shown in Pipeline Health.