SDK specification
What a recorder in any language must do. The Node SDK is one implementation of
this document, and its language-specific half is the appendix,
NODE_SDK_SPEC.md.
1. Scope and conformance
Section titled “1. Scope and conformance”An SDK is a library embedded in somebody else’s application that records journey events and delivers them to an ingestion endpoint.
The documents it sits against:
EVENT_PROTOCOL.md, the event’s fields and their meanings.INGESTION_CONTRACT.md, normative for the routes, the limits, the refusals and idempotency. Where this document and that one disagree about the wire, that one is right.- The conformance fixtures under
packages/protocol/conformance/.
The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be interpreted as described in RFC 2119 and RFC 8174, and only when they appear in capitals.
Conformance means two things. Every applicable sdk fixture passes: record
the case’s calls, capture the request body your SDK sent, compare it with
expect.wire, then send those exact bytes to POST /v1/events/batch?dryRun=true
and compare expect.results[].stored. And the requirements in section 14, which
no fixture can express, are covered by your own tests.
Every requirement below carries a stable identifier and a source, so a reader
can see which decision it comes from and nothing arrives without provenance.
tests/docs-truth.test.ts fails if a requirement has no source, or names a
fixture that does not exist, or says a fixture cannot check it without appearing
in section 14.
The propagation specification is pending. Every requirement in it is a name: header names, queue attribute names, the value grammar, the identifier prefix. Those names carried the product’s name, which changed on 2026-09-17 (ADR-057), so freezing them here before the rename would have meant publishing a contract and breaking it in the same month. Section 10 states only the rules that do not depend on a name. Header names, queue attribute names and environment variable names are therefore not specified in this document yet.
2. Host safety
Section titled “2. Host safety”A recorder is code somebody else runs inside their application. It has to be impossible for it to break that application.
- SDK-1. A recorder failure MUST NOT reach host code. Every public entry point MUST be guarded.
- SDK-2. A wrapper MUST return the callback’s value unchanged.
- SDK-3. A wrapper MUST rethrow the callback’s exact error value, not a copy and not a wrapper around it.
- SDK-4. In a language where a synchronous call and an asynchronous one are different things, a wrapper MUST preserve which one it was given. Instrumenting a synchronous call must not turn a handled error into an unhandled rejection.
- SDK-5. Telemetry MUST NOT block the host on the network.
- SDK-6. Startup MUST NOT fail over a configuration value that can be reported instead. A recorder that refuses to start breaks the application it exists to observe.
| ID | Source | Checked by |
|---|---|---|
| SDK-1 | ADR-007; core invariant 1 | section 14 |
| SDK-2 | ADR-007 | section 14 |
| SDK-3 | ADR-007 | section 14 |
| SDK-4 | ADR-007; NODE_SDK_SPEC section 4 | section 14 |
| SDK-5 | AGENTS.md, SDK reliability rules | section 14 |
| SDK-6 | ADR-007; packages/sdk-node/src/config.ts | section 14 |
3. Identifiers and idempotency
Section titled “3. Identifiers and idempotency”- SDK-7. Event ids MUST be generated by the client, from a UUID drawn from a cryptographic random source.
- SDK-8. Journey ids MUST be unpredictable, for the same reason: a journey id that can be guessed is a way to learn whether a journey exists.
- SDK-9. A retry MUST resend the same event id with byte-identical content. That is what makes ingestion idempotent, and it is why a resend is answered as a duplicate rather than stored twice.
| ID | Source | Checked by |
|---|---|---|
| SDK-7 | core invariant 3; ADR-038 | sdk/generated-ids |
| SDK-8 | ADR-038 | sdk/generated-ids |
| SDK-9 | core invariant 3; ADR-036 | section 14 |
4. The event
Section titled “4. The event”Required fields, their meanings and their limits are in EVENT_PROTOCOL.md and
in the generated schema; this section states only what an SDK must do about
them.
- SDK-10. An SDK SHOULD type the eleven operations rather than accept free text. Anything else is refused at ingestion, and a refused event leaves a timeline that is not empty but wrong.
- SDK-11.
timestampMUST be when the operation started for a wrapper, and call time for an unwrapped record. A step must not sort after the work it caused. - SDK-12.
durationMsMUST be whole milliseconds and MUST fit in a signed 32-bit integer. A larger value passes the wire schema nowhere and used to fail the insert. - SDK-13.
attemptis supplied by the caller. An attempt above one MUST recordretriedrather than the natural operation, with the attempt in metadata. - SDK-14. A failed delivery MUST record the attempt’s own operation with
errorpopulated.failedis for a terminal journey or branch failure, such as a dead-letter transition. - SDK-15. Aliases are a top-level field, and an identify call MUST emit its
own
identifiedevent carrying them.
| ID | Source | Checked by |
|---|---|---|
| SDK-10 | EVENT_PROTOCOL section 3 | wire/invalid-event-unknown-operation |
| SDK-11 | ADR-031 | section 14 |
| SDK-12 | packages/protocol/src/event.ts | wire/duration-int4-boundary |
| SDK-13 | ADR-022 | sdk/retried-attempt |
| SDK-14 | ADR-022 | sdk/retried-attempt |
| SDK-15 | ADR-023 | sdk/identify |
5. Capture and redaction
Section titled “5. Capture and redaction”- SDK-16. Capture MUST be synchronous at the call. A later mutation of the captured object must not change what was recorded, or the diff lies.
- SDK-17. Redaction MUST run before the event enters the queue, so a secret never sits in memory in the clear waiting for a flush.
- SDK-18. Redaction MUST match the built-in secret names at any depth and
inside arrays, in each of the shapes
SECURITY.mdsection 4 lists: an object key, a[name, value]pair, a{name, value}or{key, value}element, an interleaved header list, and a CRLF-delimited header block. - SDK-19. Names MUST be compared with case,
-and_ignored, soapiKey,api-key,API_KEYandAPIKeyare one name. - SDK-20. A matched value MUST be replaced rather than deleted. Evidence that a value existed is part of the record.
- SDK-21. Operator paths MUST be applied beside the built-in list, never substituted for it. Adding one path must not silently disable the rest. The one exception is effective full capture, where the built-in list alone applies, because full capture is a request to store the payload and the built-in names are the part that cannot be turned off.
- SDK-22.
error.messageMUST be masked by shape and bounded to the protocol’s 4,096 characters; a stack to 16,384. An SDK SHOULD NOT send a stack at all. - SDK-23. Values the wire format cannot carry MUST be repaired rather than dropped, and an SDK MUST NOT lose the event because it could not capture the payload.
- SDK-24. Capture modes MUST use the protocol’s names, and the server’s policy is authoritative: an SDK may capture less than the server would store, never more.
The built-in secret names
Section titled “The built-in secret names”These apply in every mode that stores a payload at all, including full capture,
and cannot be disabled. They are listed here because an implementer in another
language cannot import the file; tests/docs-truth.test.ts fails if this list
and packages/payload-security/src/default-secrets.ts stop being equal.
authorizationproxy-authorizationcookieset-cookiex-api-keypasswordaccess_tokenrefresh_tokenclient_secretapi_keysecretstripe-signaturex-hub-signaturex-hub-signature-256x-slack-signaturex-hubspot-signaturex-hubspot-signature-v3x-twilio-signaturex-shopify-hmac-sha256The list is deliberately narrow: each name means a secret in essentially every payload it appears in. The last eight are webhook signature headers: a signature stored beside its body is a request the receiver accepts, and GitHub’s carries no timestamp, so the pair stays valid for as long as the signing secret (ADR-055). A name that is only sometimes a secret belongs in an operator’s own configuration, where over-redaction is their call to make.
The repairs
Section titled “The repairs”| Value | Becomes |
|---|---|
| a cycle | [CIRCULAR] at the point the loop closes; the rest is kept |
| the same object twice | expanded both times; a shared reference is not a cycle |
| a string over 65,536 UTF-16 code units | its start and [TRUNCATED: <n> characters removed], 65,536 code units in all; counted, and the event is still sent |
| a payload that cannot fit the event’s budget, even cut, or nested or wide past the limits | [PAYLOAD_TOO_LARGE], counted, and the event is still sent |
| a value that cannot be read | [UNCAPTURABLE], and the event is still sent |
| an integer beyond the language’s safe range | its decimal string, digits intact |
| a non-finite number | null |
| a NUL byte | removed |
| an unpaired surrogate | repaired |
The last two exist so that the server never has to answer unstorable_payload
for an event this SDK sent. The wire fixtures show the server refusing those
values and the SDK fixtures show the same values arriving repaired, so the pair
documents both halves.
| ID | Source | Checked by |
|---|---|---|
| SDK-16 | ADR-034 | section 14 |
| SDK-17 | ADR-035; SECURITY.md section 4 | sdk/secrets-at-depth |
| SDK-18 | ADR-035; SECURITY.md section 4 | sdk/header-pairs |
| SDK-19 | ADR-039 | wire/secret-name-spellings |
| SDK-20 | SECURITY.md section 4 | sdk/secrets-at-depth |
| SDK-21 | ADR-035 | section 14 |
| SDK-22 | ADR-046 | sdk/error-masked-and-bounded |
| SDK-23 | ADR-034; ADR-036 | sdk/oversize-payload |
| SDK-24 | ADR-018 | wire/metadata-only-capture |
6. Buffering
Section titled “6. Buffering”- SDK-25. The queue MUST be bounded. Prefer dropping events over exhausting the host’s memory.
- SDK-26. A full queue MUST drop the oldest event and count the drop. The newest event is the one describing what is happening now.
This narrows NODE_SDK_SPEC.md section 8, which offered drop-newest as a
configurable policy that was never built.
| ID | Source | Checked by |
|---|---|---|
| SDK-25 | AGENTS.md, SDK reliability rules | section 14 |
| SDK-26 | NODE_SDK_SPEC section 8, narrowed to drop-oldest | section 14 |
7. Transport
Section titled “7. Transport”- SDK-27. An SDK MUST send through the batch route. It is the only one with per-event verdicts.
- SDK-28. An SDK MUST read the response body. A 2xx does not mean the events were stored.
- SDK-29. A batch MUST hold at most 100 events.
- SDK-30. A send MUST retry with capped exponential backoff and jitter, behind a circuit breaker, and MUST NOT retry forever.
- SDK-31. A whole-request 4xx is permanent: it MUST NOT be retried and MUST NOT count toward the breaker. A whole-request 5xx or a transport failure is retried, bounded by the queue.
- SDK-32. A send in which anything was stored MUST NOT count toward the breaker.
- SDK-33. Per-event verdicts MUST follow the rule in
INGESTION_CONTRACT.mdsection 4: below 500 is permanent and the event is never sent again; 500 or above is transient and that event alone is sent again, for up to 30 seconds from its first refusal or 10 sends, whichever comes first; a refusal with no status is permanent; and an event the response gives no verdict for is not sent again, because the request succeeded and the server may have stored it. - SDK-34. Concurrent sends MUST be capped.
- SDK-35. An SDK SHOULD report an unencrypted endpoint and MUST still start.
- SDK-36. An SDK MUST NOT use the dry run in normal operation.
Defaults
Section titled “Defaults”Every number here is a SHOULD, taken from the merged Node SDK. An implementation may differ; it should be able to say why.
| Setting | Default |
|---|---|
| batch size | 50 |
| flush interval | 1,000 ms |
| request timeout | 1,500 ms |
| queue | 1,000 events |
| event budget | 262,144 bytes |
| concurrent sends | 4, clamped to 1 to 16 |
| attempts per send | 3 |
| backoff | 100 ms base, 2,000 ms maximum |
| breaker | 5 consecutive failures, open for 30 seconds |
| per-event retry budget | 30 seconds, or 10 sends |
| ID | Source | Checked by |
|---|---|---|
| SDK-27 | INGESTION_CONTRACT section 1 | section 14 |
| SDK-28 | INGESTION_CONTRACT section 1 | section 14 |
| SDK-29 | packages/protocol MAX_BATCH_EVENTS | sdk/hundred-and-one-events |
| SDK-30 | AGENTS.md, SDK reliability rules | section 14 |
| SDK-31 | packages/sdk-node/src/transport.ts | section 14 |
| SDK-32 | packages/sdk-node/src/transport.ts | section 14 |
| SDK-33 | INGESTION_CONTRACT section 4 | section 14 |
| SDK-34 | packages/sdk-node/src/config.ts | section 14 |
| SDK-35 | packages/sdk-node/src/diagnostics.ts | section 14 |
| SDK-36 | ADR-050 | section 14 |
8. Shutdown
Section titled “8. Shutdown”- SDK-37. Shutdown MUST stop accepting new events, then drain until the queue is empty, until a pass makes no progress, or until the timeout expires, whichever comes first, aborting what is in flight.
- SDK-38. Shutdown MUST count every event it could not deliver, exactly once.
sent + rejected + droppedMUST equal the events recorded. - SDK-39. Shutdown MUST NOT hang, and MUST NOT hold the process open after the drain finishes.
| ID | Source | Checked by |
|---|---|---|
| SDK-37 | packages/sdk-node/src/recorder.ts | section 14 |
| SDK-38 | packages/sdk-node/src/accounting.test.ts | section 14 |
| SDK-39 | packages/sdk-node/src/recorder.ts | section 14 |
9. Diagnostics
Section titled “9. Diagnostics”- SDK-40. An SDK MUST be silent by default. Debug output is opt-in. The exceptions are the warnings SDK-56 and SDK-60 allow, each at most once per process, the one SDK-61 allows, at most once per process and name, and the one SDK-63 allows, at most once per process, field and value shape.
- SDK-41. A printed diagnostic MUST NOT contain a payload, an API key, a message from the server, or the endpoint’s path or query. A path or a query can carry a credential.
- SDK-42. An SDK SHOULD expose counters: recorded, sent, rejected, dropped, and payloads omitted and truncated, counted separately. It SHOULD also expose dropped by cause, one count per cause with every cause present from the start at zero, summing to dropped, so that a collector slower than the shutdown timeout reads apart from one that answers with the wrong body (ADR-063).
| ID | Source | Checked by |
|---|---|---|
| SDK-40 | AGENTS.md, SDK reliability rules | section 14 |
| SDK-41 | SECURITY.md section 12 | section 14 |
| SDK-42 | packages/sdk-node/src/diagnostics.ts; ADR-063 | section 14 |
10. Propagation
Section titled “10. Propagation”Only the rules that do not depend on a name. The names, the value grammar and the test vectors are the pending propagation specification.
- SDK-43. There are three levels, and the default MUST be the middle one: the journey and the entity type propagate, and the entity id does not.
- SDK-44. Aliases MUST NOT propagate at any level. They are other people’s identifiers.
- SDK-45. The entity id MUST propagate only at the highest level, which an operator opts into.
- SDK-46. An SDK MUST NOT write
traceparent. Reading one is section 11; writing one would put this product in the middle of somebody else’s tracing. - SDK-47. A journey MUST NOT cross an environment boundary. A context that arrives from another environment starts a new journey rather than extending the old one, because the server refuses the merge.
| ID | Source | Checked by |
|---|---|---|
| SDK-43 | SECURITY.md section 10 | section 14 |
| SDK-44 | SECURITY.md section 10 | section 14 |
| SDK-45 | SECURITY.md section 10 | section 14 |
| SDK-46 | ADR-010 | section 14 |
| SDK-47 | ADR-038 | wire/cross-environment-journey |
11. Optional trace correlation
Section titled “11. Optional trace correlation”- SDK-48. An SDK MAY read an active trace id and span id and attach them.
- SDK-49. An SDK MUST NOT require an OpenTelemetry installation. No deployment of this product requires OpenTelemetry.
| ID | Source | Checked by |
|---|---|---|
| SDK-48 | ADR-010 | section 14 |
| SDK-49 | ADR-010; ADR-049 | section 14 |
12. Configuration
Section titled “12. Configuration”Four settings are required, given here by role rather than by name because the names carry the product’s:
| Role | What it is |
|---|---|
| endpoint | where to send |
| key | an API key for one project and one environment |
| service | the name of the service doing the recording |
| environment | which environment this process is |
Two more are optional: the journey id secret, used only by SDK-55, and the known-safe names, used only by SDK-62.
- SDK-50. An SDK SHOULD NOT read ambient environment variables of its own. The application decides where its configuration comes from.
| ID | Source | Checked by |
|---|---|---|
| SDK-50 | ADR-012 | section 14 |
13. Fitting the limits, and the helpers running it found missing
Section titled “13. Fitting the limits, and the helpers running it found missing”Requirements added after the first edition, from instrumenting a real service. They are numbered after the rest so that no identifier above moved.
Fitting an event to the server’s limits
Section titled “Fitting an event to the server’s limits”- SDK-51. An SDK MUST apply the limits in
INGESTION_CONTRACT.mdsection 3 to the whole envelope before sending it, so that a limit never refuses an event it sent. A string over the length limit MUST be cut to its start and[TRUNCATED: <n> characters removed], where<n>counts the code units removed and the result is exactly the limit long. When the string held a CRLF, the marker MUST follow a CRLF, so that the server still masks a secret header line the SDK could not recognise. A payload that still does not fit, or is nested or wide past the limits, MUST be replaced with[PAYLOAD_TOO_LARGE], the larger ofinputandoutputfirst, andmetadataleft off last. The same holds for the schema’s caps on keys and short fields: a top-level metadata key or an alias type over 128 code points, or an alias value that is not a string of at most 512, MUST be left off and reported, with"[KEY_TOO_LONG]": <n>added to metadata for the keys dropped from it (never a marker among aliases, which would be stored and searchable). The event MUST still be sent. - SDK-52. Truncation MUST run after redaction, and a truncated payload MUST be reported and counted separately from an omitted one. A payload cut and then omitted is an omission.
| ID | Source | Checked by |
|---|---|---|
| SDK-51 | ADR-051; packages/protocol/src/event.ts | sdk/long-string-truncated |
| SDK-52 | ADR-051 | section 14 |
Projections
Section titled “Projections”- SDK-53. A wrapper SHOULD accept a projection of its input and of its
output, so a host can record a view of a value while its own code receives
the value itself. The input projection SHOULD run before the callback, and
its result MUST be captured when it runs, so a callback that changes what the
projection shares cannot change what is recorded. A
projection that fails, or that does not return synchronously, MUST NOT affect
the host’s call, its return value or its error; the SDK MUST record
[UNCAPTURABLE]in that payload’s place and report it.
| ID | Source | Checked by |
|---|---|---|
| SDK-53 | ADR-007; docs/superpowers/specs/2026-09-16-dogfood-gaps-design.md | sdk/projection-throws |
One operation on many journeys
Section titled “One operation on many journeys”- SDK-54. An SDK SHOULD let a host record one operation on several journeys in one call. Each journey MUST get its own event with its own id, the events MUST share one timestamp and one duration, a wrapper MUST run its callback once, and a journey named twice MUST be recorded once. Nothing on the wire changes: the server receives ordinary events.
| ID | Source | Checked by |
|---|---|---|
| SDK-54 | docs/superpowers/specs/2026-09-16-dogfood-gaps-design.md | sdk/across-journeys |
Deriving a journey id from the entity
Section titled “Deriving a journey id from the entity”A host with nowhere to keep a journey id between runs can derive one from the
entity. The derivation is keyed, because an unkeyed one is predictable, and a
predictable journey id is the risk INGESTION_CONTRACT.md section 5 describes.
- SDK-55. An SDK SHOULD offer a journey id derived from an entity under a
secret the host configures, of at least 32 bytes. When it does, it MUST
compute HMAC-SHA256, keyed with the secret’s UTF-8 bytes, over four fields in
this order: the label
journey-id/v1, the recorder’s environment, the entity type and the entity id, each written as a 4-byte big-endian length followed by its UTF-8 bytes. The id MUST be the journey id prefix followed by the first 32 lowercase hex characters of the MAC. An SDK MUST NOT derive for an entity whose type or id is empty, which the server refuses, or is not well-formed text (in UTF-16, one holding an unpaired surrogate): encoding would replace the bad code unit and give it the id of the replacement, and the server refuses such an id anyway. It MUST reproduce every vector inpackages/protocol/fixtures/journey-id-derivation.json. The SDK MUST NOT read the secret from an environment variable of its own. - SDK-56. Deriving without a usable secret, or for an entity that SDK-55 refuses or whose type and id are not non-empty strings, MUST NOT throw and MUST NOT fail startup. The SDK MUST report it, and MUST return a fresh unpredictable journey id rather than an unkeyed derivation. A secret too short to use MUST be reported when the recorder is created, and MUST NOT be used. Because a missing secret splits every derived journey without anybody noticing, an SDK SHOULD also print one warning for it per process even when debug output is off.
| ID | Source | Checked by |
|---|---|---|
| SDK-55 | ADR-052; packages/protocol/fixtures/journey-id-derivation.json | section 14 |
| SDK-56 | ADR-052; ADR-007 | section 14 |
Displayable aliases
Section titled “Displayable aliases”- SDK-57. An SDK SHOULD let the host list, when it states aliases, the alias
types a reader may see in full, and send them as
displayableAliaseson the same event. The default MUST be none. An SDK MUST NOT add a type the host did not list, and SHOULD document that the list has to accompany every statement of the alias, because the server keeps an alias displayable only while every statement lists it.
| ID | Source | Checked by |
|---|---|---|
| SDK-57 | ADR-053; docs/SECURITY.md section 6 | sdk/identify-displayable |
Journey labels
Section titled “Journey labels”A label is the journey’s name on the Journeys page, where partial text finds it
(EVENT_PROTOCOL.md section 3).
- SDK-58. An SDK SHOULD let a host set a label on a journey, sent as
journeyLabel. When it does, it MUST send only labels the protocol accepts. A label over 200 code points MUST be cut, at a code point boundary so that no surrogate pair is split, to at most 200 code points, and reported; the events carrying it MUST still be sent. A label that is empty, consists only of whitespace (Unicode White_Space, line terminators and U+FEFF, as ECMAScript’strimremoves), or is not a string, MUST NOT be sent and MUST be reported, and the event it would have been on MUST still be sent. Setting a label records nothing by itself. An SDK SHOULD carry the label on every later event of the journey, not only the next one: the server keeps the label of the event that started last, so repeating it changes nothing, and an event that is lost would otherwise take the label with it. The Node SDK cuts to 199 code points followed by…(U+2026), and leaves an earlier label in place when a later one is refused. - SDK-59. An SDK SHOULD document that a label is stored, shown and matched in plain text, is never redacted, and so should not hold personal data or anything else a reader of the journey list may not see.
| ID | Source | Checked by |
|---|---|---|
| SDK-58 | EVENT_PROTOCOL section 3; packages/protocol/src/limits.ts; docs/superpowers/specs/2026-09-16-journeys-browse-design.md section 1 | sdk/journey-label, sdk/journey-label-blank, sdk/journey-label-cut, sdk/journey-label-empty |
| SDK-59 | docs/superpowers/specs/2026-09-16-journeys-browse-design.md section 1 | section 14 |
Configuration that cannot be used
Section titled “Configuration that cannot be used”- SDK-60. A configuration value that is missing, cannot be read, has the
wrong type or is out of range MUST NOT fail startup (SDK-6). An SDK MUST
report each such value when the recorder is created, naming the setting and
never its value, and MUST NOT coerce one type into another. An optional
setting SHOULD take its default, or be clamped into range. A required
setting (section 12) has no default, so nothing recorded reaches the server
until it is fixed; an SDK SHOULD therefore print one warning for it per
process even when debug output is off. An SDK SHOULD print one warning per
process for an optional setting it rejected too: the recorder runs on a
default the operator did not choose while every event it was meant to bound
or enrich keeps flowing, and with debug output off and no callback read
nothing else says so. An SDK SHOULD also let a host read which settings were
rejected, not only how many, so that a test or a health check can name one,
and SHOULD keep the settings refused at creation apart from options refused
on a later call, so that one odd call site never reads as a misconfigured
process (F-038). A setting made of parts SHOULD be reported by the part that
was refused, with the setting itself named as well when nothing of it is
used, so a partial refusal reads differently from a total one (F-031). A required string setting that is
empty, or holds only whitespace, MUST be treated as missing, reported and
printed the same way, since an unset environment variable often arrives as
"". A setting given under a name the SDK no longer reads SHOULD be reported the same way, naming the setting that replaced it, and printed like a required one, because its value is otherwise lost without a sign. When debug output is on, every problem found at creation MUST be printed, whatever rate limit applies to other output, so that one cannot hide another.
| ID | Source | Checked by |
|---|---|---|
| SDK-60 | ADR-007; ADR-052; ADR-062; packages/sdk-node/src/config.ts | section 14 |
Secret-looking names no rule covers
Section titled “Secret-looking names no rule covers”Redaction matches names, so a credential filed under a name neither the built-in list nor the operator’s rules cover is sent in the clear. An SDK warns about it and never redacts on a guess, because a diff must not change on one (ADR-055).
A name looks like a secret by this rule. An implementation reproduces it
exactly, so that every SDK and doctor agree:
- Fold it as SDK-19 does: lower case,
-and_removed. - Drop trailing ASCII digits and, if any were dropped, one
vbefore them. - Its end matches one of these terms, and the name is either the term alone
or the term after any other characters:
token(except afterpage,next,continuation,pagination,sync,client,clientrequest,idempotency,resume,cancel,cursor,start,stop,bos,eos,pad,unk,sep,clsormask),secret,password,passwd,passphrase,passcode,credential,credentials,authorization,auth,bearer,cookie,cookies,signature(except afteremail),jwt,otp,cvv,cvc,apikey,accesskey,secretkey,privatekey,signingkey,encryptionkey,masterkey,sessionkey,authkey,hmackey,sharedkey,subscriptionkey,sessionid,sessid,secretstring,secretvalue,codeverifier,clientassertion,authcode,authorizationcode,otpcode,mfacode,recoverycode,connectionstring,databaseurl,dsn,passwordconfirmation. - Or its end is
pin, alone or aftercard,atm,user,account,security,login,new,oldorcurrent; orpwdafterdb,user,admin,rootordatabase, never alone; or the name is exactlyhmac.
Personal data such as ssn or cardNumber is not on the list: the rule is
about credentials, and whether personal data is captured is the capture mode’s
question.
A value under such a name could be a credential when it is a number, or a
string that is not empty, not [REDACTED], and not, once trimmed and in lower
case, one of true, false, none, basic, bearer, oauth, required,
optional; and, when the term matched is auth, at least 8 characters long.
The minimum applies to auth alone because PINs, card codes and one-time codes
are real secrets of 3 to 6 characters.
The table is packages/payload-security/src/secret-name.ts, and its test
holds the names from real APIs it was checked against.
- SDK-61. When redaction keeps a name that looks like a secret with a value
that could be a credential, an SDK SHOULD report it: under an object key, and
in each positional header shape of SDK-18 (a pair, a name-value object, an
interleaved list, a header block line). The report names the payload field,
the name, and its path with every array index written
[*], and MUST NOT include the value. An SDK SHOULD report only for a payload the event it sends still carries, SHOULD report each folded name once per recorder, SHOULD print one warning per process and name even when debug output is off, saying how to cover the name with a redaction rule, or that no rule can name it, or how to mark it known-safe, and MUST send the event unchanged. The report handed to the host’s own diagnostic callback MAY carry the name as written, bounded and unmasked, as other diagnostics do; the printed line is masked (SDK-41). It SHOULD find these during the redaction walk rather than in a second one, and SHOULD bound how many names it remembers and how much of each it keeps. - SDK-62. An SDK that implements SDK-61 SHOULD accept a list of key names, compared as SDK-19 compares names, that it does not warn about, including names no redaction rule can express. The list MUST NOT change what is redacted, and an entry that is not a non-empty string MUST be ignored and reported as SDK-60 reports a setting.
| ID | Source | Checked by |
|---|---|---|
| SDK-61 | ADR-055; ADR-007 | sdk/unredacted-secret-name |
| SDK-62 | ADR-055 | section 14 |
Personal data in a value a reader sees in full
Section titled “Personal data in a value a reader sees in full”A journey label (SDK-58) and an alias the host marked displayable (SDK-57) are both stored, shown and matched in plain text and are never redacted. SDK-59 says an SDK documents that for a label, and the same is true of a displayable alias, but documentation alone is missed: a design review approved a label of a company and a person’s full name, and nothing in any SDK would have said a word (F-006, F-012).
-
SDK-63. An SDK SHOULD report a journey label, an alias value the host marked displayable, or an error’s message, that looks like personal data. An error message is masked for credential shapes (SDK-22) and for nothing else, and a timeline shows it to every reader (F-041). It MUST NOT change the value, refuse it, or stop marking the alias displayable: this warns on a guess, as SDK-61 does, and a value changed on a guess is the failure ADR-055 refuses. The report names which of the three it was and what the value looked like, and MUST NOT include the value. An error message is examined as it is sent, after masking and bounding; a stack is not examined. An SDK SHOULD report once per process, field and value shape, and SHOULD print one warning per process, field and shape even when debug output is off, for the reason SDK-61 gives: the value is stored in the clear. Per field, so that a noisy field, as error text is, never silences a warning about a quieter one (ADR-062). The email shape is narrowed further for this reason: a local part holds no
/,:,[or], and a domain followed by:,/or@is a host, not an address; and a+followed by a real timezone offset (hours 00 to 14, minutes 00, 15, 30 or 45) is not a telephone number. An SDK SHOULD say nothing about an alias the host did not mark displayable, because it is masked when it is read.The rule is deliberately narrow, so that it does not fire on ordinary text:
- Something shaped like an email address anywhere in the value: characters
that are not whitespace or
@, an@, more of the same, a., and at least two letters. - Or something shaped like an international telephone number (ADR-063): a
+at the start of the value or after whitespace or one of<,>,(,),[,",',,,;,=,:, never after a letter, a digit or.; then the run of digits, spaces,(,),.and-after it, at most 20 characters; not a real timezone offset, a+with hours 00 to 14 and minutes 00, 15, 30 or 45 and no fifth digit; holding 8 to 15 digits (E.164’s own bound) when a separator stands between two of them, and 10 to 15 when they are one unbroken run. Sophone=+19195551234,tel:+19195551234and{"phone":"+19195551234"}are found, andReceived +12345678 bytesis not.
Nothing else. A person’s name, a customer number, a national telephone number written without a
+and a postal address are all personal data this does not catch, and an SDK MUST still document the rule that a label and a displayable alias are public text (SDK-59). The Node SDK examines the first 1,024 characters of a value. - Something shaped like an email address anywhere in the value: characters
that are not whitespace or
| ID | Source | Checked by |
|---|---|---|
| SDK-63 | ADR-055; ADR-060; ADR-053; ADR-062; ADR-063 | section 14 |
Which SDK recorded an event
Section titled “Which SDK recorded an event”Nothing in an event said which SDK recorded it, and two builds of the Node SDK
both called themselves 0.1.0, so during a server-first upgrade nothing could
say which services still ran the old one (F-046).
- SDK-64. An SDK SHOULD send
runtime.language,runtime.versionandruntime.sdk(name,version, andcommitwhen it has one) on every event, so a reader can say which services run which SDK build, and MUST NOT take any of them from host settings or the host’s environment: they describe the SDK, which the host cannot know better, and a value the host could set would answer the question wrongly. The version and commit are those of the build, fixed when the SDK is built. It SHOULD NOT sendruntime.hostnameorruntime.processIdfor this: a hostname is a new identifier on every event, often a person’s name on a laptop (F-046, ADR-063).
| ID | Source | Checked by |
|---|---|---|
| SDK-64 | ADR-063; wire/runtime-sdk; wire/runtime-sdk-name-empty | section 14 |
A reply with no verdict
Section titled “A reply with no verdict”A collector that answers 2xx with a body that is not a verdict, which is what a misconfigured proxy in front of the API does, reset the breaker on every send: Leadline measured 16,000 events recorded and 16,000 dropped with the breaker never opening (F-048).
- SDK-65. A send in which no attempt got a verdict for any of its events MUST count toward the breaker, as a send that failed does. An attempt got a verdict when at least one of its events was accepted, refused for good, or refused for now. A send that stored anything still resets the count (SDK-32), as does one that got at least one verdict and left nothing unsent, and a whole-request 4xx still leaves it as it was (SDK-31). A collector that answers 2xx with the wrong body otherwise loses every event it is sent with the breaker never opening (F-048, ADR-063). A send of no events is not a send: it neither counts toward the breaker nor resets it.
| ID | Source | Checked by |
|---|---|---|
| SDK-65 | ADR-063; packages/sdk-node/src/transport.ts | section 14 |
14. Conformance, and what the fixtures cannot check
Section titled “14. Conformance, and what the fixtures cannot check”To run the fixtures, follow INGESTION_CONTRACT.md section 9. In short: drive
your recorder through each sdk case’s calls against a stub endpoint, compare
the request body it sent with expect.wire, then send those bytes to the dry run
and compare expect.results[].stored. Report every case you skip, with the
reason; a skip nobody sees is a case that quietly stopped running.
These requirements need tests of your own, because no fixture can express them. A fixture observes a request body and a stored row; none of these is visible in either.
| Requirement | What your test has to do |
|---|---|
| SDK-1, SDK-2, SDK-3, SDK-4 | Make the recorder fail inside a wrapper and assert the host is unaffected, that the callback’s value comes back unchanged, that the exact error value is rethrown, and that a synchronous wrapper stays synchronous. |
| SDK-5 | Assert a record call returns without waiting on the network. |
| SDK-6 | Start a recorder with an unusable configuration value and assert it starts. |
| SDK-9 | Force a retry and assert the resent event has the same id and identical content. |
| SDK-11 | Assert a wrapper’s timestamp is the moment the callback started, not the moment it finished. |
| SDK-16 | Mutate a captured object after the call and assert the recorded value did not change. |
| SDK-21 | Configure one redaction path and assert the built-in names still apply. |
| SDK-25, SDK-26 | Fill the queue past its bound and assert the oldest events are the ones dropped, and that the drop is counted. |
| SDK-27, SDK-28 | Assert the SDK reads the response body of a 2xx in which every event was refused. |
| SDK-30, SDK-31, SDK-32 | Assert the backoff is capped and jittered, that a whole-request 4xx is not retried and does not open the breaker, and that a partially stored send does not either. |
| SDK-33 | Assert a 5xx per-event refusal is resent and a 4xx one is not, and that both budgets end it. |
| SDK-34 | Assert concurrent sends never exceed the cap, including during an explicit flush. |
| SDK-35 | Point a recorder at an unencrypted endpoint and assert it reports and still starts. |
| SDK-36 | Assert nothing in normal operation sets the dry-run parameter. |
| SDK-37, SDK-38, SDK-39 | Assert shutdown drains, aborts what is in flight, counts every undelivered event exactly once so the totals reconcile, and returns. |
| SDK-40, SDK-41, SDK-42 | Assert nothing is printed by default, and that a printed line carries no payload, key, server message, path or query. |
| SDK-43, SDK-44, SDK-45, SDK-46 | Assert the default level, that aliases never propagate, that the entity id propagates only at the highest level, and that traceparent is never written. |
| SDK-48, SDK-49 | Assert trace correlation works with the tracing library present and that the SDK works without it. |
| SDK-50 | Assert the recorder reads no ambient environment variable of its own. |
| SDK-52 | Record a payload with a secret-named field holding a string over the limit and assert it arrives masked and is not counted as truncated; cut a payload and then force its omission and assert it is counted once, as omitted. |
| SDK-55 | Reproduce every vector in packages/protocol/fixtures/journey-id-derivation.json, and assert the result is accepted by your own propagation extraction. |
| SDK-56 | Derive without a secret, with a short one, for each entity the fixture’s refused and refusedEmpty lists name, and for an entity that is not a pair of strings; assert nothing throws, each is reported, the ids differ call to call, a short secret is reported at creation, and a missing or short secret prints one warning per process with debug output off. |
| SDK-58 | Set a label that is not a string, including one whose conversion to text throws, and assert nothing throws, it is reported, and the event is sent without it. |
| SDK-59 | Check that the documentation of the label says it is stored and shown in plain text and must not hold personal data. |
| SDK-60 | Start a recorder with a required setting missing and an optional one of the wrong type; assert it starts, both are reported without their values, the required one prints once per process with debug output off, and both print with it on. Repeat with a required setting that is "" and one that is only whitespace, and assert each is reported and printed as missing. Start a recorder with every setting valid, make a call with an unusable option, and assert the settings the host can read are still empty and the option is readable apart from them. Configure a deployment with one field too long and assert that field alone is named and the rest is sent; with every field refused, assert the field and the setting are both named. |
| SDK-61, SDK-62 | Record a secret-looking name twice from two recorders with debug output off and assert one report per recorder and one printed line in all, without the value; assert a name the redaction rules cover and a known-safe name are not reported, that a known-safe name that is also a rule is still redacted, and that a known-safe entry that is not a string is reported; assert a payload the event budget omits reports nothing; record many distinct very long names and assert the memory kept is bounded. |
| SDK-63 | Set a journey label holding an email address and assert one report naming the label and the shape, with debug output off, one printed line, and neither carrying the value; assert the label the event carries is the one that was set; assert a second label with an email address reports nothing more, and one with an international telephone number reports once; assert a label that looks like neither reports nothing; mark an alias displayable whose value is an email address and assert the same report names the alias, and that an alias not marked displayable reports nothing; then, in the same process, record a failure whose message holds an email address and assert a report naming the error message although the label already warned for that shape, that the message the event carries is unchanged, and that a stack holding one reports nothing; assert an error message holding a module path under node_modules/@scope/, a git remote git@host:org/repo.git or a date with +0000 reports nothing; assert an error message holding phone=+19195551234 or tel:+19195551234 reports the telephone shape and one holding Received +12345678 bytes reports nothing. |
| SDK-64 | Record events of every kind and assert each carries runtime.language, runtime.version and runtime.sdk with the SDK’s own name and version, and no hostname or processId; set host settings and environment variables that name another version or commit and assert nothing changes; assert the server’s schema accepts the event. |
| SDK-65 | Answer every request 2xx with a body that is not JSON, and again with JSON that holds no results; assert the breaker opens after the threshold of sends with no transport error reported, and that events are dropped as no verdict until then. Answer with verdicts for half the events and assert it never opens. |