Scheduling a chain export to S3

Goal

Pull yesterday's Hash Chain segment from Adjudon and push it to an S3 bucket on a daily schedule. The exported bundle is a self-contained verification artefact: a regulator (or the operator's internal audit team) can re-run the chain verification against the bundle months later without round-tripping to Adjudon's API.

This is the standard archival pattern for BaFin's seven-year retention requirement: keep the live trace store on a 365-day window in Adjudon, push the daily segment to your own WORM-bucket for the long tail.

You'll need

An Adjudon Governance plan or above (the hashChainAudit feature gate)
A workspace API key (adj_live_*)
An S3 bucket (or any object store; the recipe is cloud-agnostic)
Python 3.9+ with requests and boto3

pip install requests boto3
export ADJUDON_API_KEY="adj_live_..."
export S3_BUCKET="acme-audit-archive"
export S3_PREFIX="adjudon/hash-chain"

Code

export_chain_segment.py
import os
import json
import hashlib
from datetime import datetime, timedelta, timezone
import requests
import boto3

ADJUDON_API_KEY = os.environ["ADJUDON_API_KEY"]
S3_BUCKET       = os.environ["S3_BUCKET"]
S3_PREFIX       = os.environ["S3_PREFIX"]

# ── 1. Pull yesterday's chain segment ──────────────────────────────────
yesterday = datetime.now(timezone.utc).date() - timedelta(days=1)
since = f"{yesterday.isoformat()}T00:00:00Z"
until = f"{yesterday.isoformat()}T23:59:59Z"

resp = requests.get(
    "https://api.adjudon.com/api/v1/hash-chain/export",
    params={"since": since, "until": until},
    headers={"Authorization": f"Bearer {ADJUDON_API_KEY}"},
    timeout=30,
)
resp.raise_for_status()
bundle = resp.json()["data"]

# ── 2. Compute a digest of the bundle for the manifest ─────────────────
bundle_bytes = json.dumps(bundle, separators=(",", ":"), sort_keys=True).encode()
bundle_sha256 = hashlib.sha256(bundle_bytes).hexdigest()

# ── 3. Push to S3 ──────────────────────────────────────────────────────
s3 = boto3.client("s3")
key_bundle   = f"{S3_PREFIX}/{yesterday}/bundle.json"
key_manifest = f"{S3_PREFIX}/{yesterday}/manifest.json"

s3.put_object(
    Bucket=S3_BUCKET,
    Key=key_bundle,
    Body=bundle_bytes,
    ContentType="application/json",
    ServerSideEncryption="AES256",
)

manifest = {
    "date":           yesterday.isoformat(),
    "fromSequence":   bundle["fromSequence"],
    "toSequence":     bundle["toSequence"],
    "entryCount":     len(bundle["entries"]),
    "bundleSha256":   bundle_sha256,
    "exportedAt":     datetime.now(timezone.utc).isoformat(),
}
s3.put_object(
    Bucket=S3_BUCKET,
    Key=key_manifest,
    Body=json.dumps(manifest, indent=2).encode(),
    ContentType="application/json",
    ServerSideEncryption="AES256",
)

print(f"Exported {manifest['entryCount']} entries → s3://{S3_BUCKET}/{key_bundle}")

Run it once interactively to verify, then schedule the script:

# Cron — every day at 02:00 UTC
0 2 * * * /usr/bin/python3 /opt/adjudon/export_chain_segment.py >> /var/log/adjudon-export.log 2>&1

For Lambda / Cloud Run: package the script as a job and trigger on a daily EventBridge / Cloud Scheduler rule.

What just happened

GET /api/v1/hash-chain/export returned a JSON bundle covering yesterday's chain segment: every entry's sequence number, trace ID, content hash, previous-hash link, and the segment's boundary fromSequence / toSequence. The script wrote two objects to S3: the bundle itself and a manifest with the bundleSha256 digest plus the entry count.

The two-file shape lets you confirm the bundle is intact at retrieval time without re-parsing the whole thing — re-fetch the manifest, re-fetch the bundle, recompute sha256(bundle) and compare against manifest.bundleSha256. A bit-rot in S3 surfaces immediately on the comparison; an auditor doing the same comparison years from now reaches the same answer.

Why the bundle is self-contained

Every entry in the exported bundle carries the cryptographic fields needed to verify the chain offline: the entry's own content hash, the previous entry's hash (the link), the sequence number, and the timestamp. A consumer who pulls yesterday's bundle from S3 can recompute every chained hash and confirm the structure matches the public Hash Chain contract documented at /api-reference/hash-chain without any further round-trip to Adjudon. This is what makes the archive an independent audit artefact: the operator's own auditor can verify chain integrity from the S3 bucket alone, even years after Adjudon's API has moved to a different version or sunset the legacy mount.

The trade-off is that an attacker with write access to the S3 bucket could rewrite the bundle. Defence in depth: store the manifest digest in a second location (a separate bucket with different IAM, an internal notarisation service, an internal ledger) so a tampered bundle is detectable even if the primary storage is compromised.

Verifying a stored bundle

verify_stored_bundle.py
import boto3, json, hashlib, requests

s3 = boto3.client("s3")

def verify_day(date_str: str):
    bundle  = json.loads(s3.get_object(Bucket=S3_BUCKET,
              Key=f"{S3_PREFIX}/{date_str}/bundle.json")["Body"].read())
    manifest = json.loads(s3.get_object(Bucket=S3_BUCKET,
              Key=f"{S3_PREFIX}/{date_str}/manifest.json")["Body"].read())

    expected = manifest["bundleSha256"]
    actual   = hashlib.sha256(json.dumps(bundle, separators=(",", ":"),
                              sort_keys=True).encode()).hexdigest()
    if expected != actual:
        raise RuntimeError(f"BUNDLE TAMPERED: {date_str}")

    # Optional: round-trip-verify the chain via the live API
    resp = requests.post("https://api.adjudon.com/api/v1/hash-chain/verify",
        json={"fromSequence": bundle["fromSequence"], "toSequence": bundle["toSequence"]},
        headers={"Authorization": f"Bearer {ADJUDON_API_KEY}"},
    )
    return resp.json()["data"]["verified"]

Operating considerations

Idempotent re-runs. The script writes to deterministic S3 keys (<prefix>/<date>/bundle.json); re-running for the same day overwrites the prior export. Combine with S3's If-Match precondition or Object Lock if your retention regime forbids overwrites.
Failed exports. A network failure between Adjudon and the script means yesterday's bundle is missing in S3. Detect this with a daily check against the manifest list; alert on any date where the manifest object is absent.
Retention trim. If you keep traces in Adjudon under the default 90-day retention, the live API still answers /export for historical segments only as long as the underlying entries exist. Once the trace store has trimmed past a date, the chain segment for that date is no longer live-fetchable; your S3 archive becomes the only copy. Run this script reliably from day one of the retention regime.

Going further

WORM-mode S3. For the BaFin retention pattern, write the bucket with Object Lock in compliance mode so the objects cannot be deleted or overwritten before the retention window closes — not even by the bucket owner.
Cloud-agnostic. Swap boto3 for the GCS / Azure / Cloudflare R2 SDK; the bundle structure is the same.
Hourly instead of daily. Pass fromSequence and toSequence resolved from a sequence-tracker (e.g. "everything since I last successfully exported") rather than a calendar window. Useful for high-volume orgs where a daily bundle would exceed S3's per-object size bounds.
Anchor proofs. Include the per-anchor proof (GET /hash-chain/anchors/:id/proof) alongside the bundle if your retention regime requires the anchor's own attestation in the archive, not just the chain entries.

Goal​

You'll need​

Code​

What just happened​

Why the bundle is self-contained​

Verifying a stored bundle​

Operating considerations​

Going further​

See also​