Skip to content

Integration — S3 export

Public · Status: ✅ available · access scoping 🔄 proposed

1. Purpose

Consume the full per-convoy bundle from object storage — for data lakes, batch ETL, ML, or evidence archives. Best when you want the raw artifacts (point clouds, height-maps, photos, report), not just the scalar measurements.

2. When to use it

  • You have a data-lake / warehouse pipeline that ingests files.
  • You want the evidence (MP4, snapshot JPEGs, the PDF report) or the dense data (MCAP, Parquet).

3. Layout

Each completed convoy is uploaded under the device prefix:

s3://{bucket}/{prefix}/{sensor_id}/data/recordings/{convoy_id}/
File Contents
payload.json the exact measurement payload (convoy + cars) — the canonical record
convoy.mcap sensor recording (LiDAR / camera / events) for the passage
signals.parquet per-frame derived signal time-series
<car>.mp4 · snaps/<car>/*.jpg evidence video + snapshot images
rasters/car<N>.png + car<N>.json per-car height-map (16-bit PNG) + geometry sidecar

Start from payload.json (the structured record → data model); the other files are addressed by deterministic bundle-relative names.

4. Access (🔄 proposed)

You don't get raw bucket credentials. Instead, your API key mints time-limited presigned URLs scoped to your organization's bundles — so access is auditable, revocable, and never crosses tenants.

  1. List/locate a convoy with the REST API.
  2. Ask for its artifacts:
curl -H "X-API-KEY: bsp_live_xxx" \
  https://bsp.ita.builders/api/v1/convoys/{convoy_id}/artifacts


{
  "convoy_id": "cvy_8f3a",
  "expires_at": "2026-06-15T12:15:00Z",
  "files": [
    { "name": "payload.json", "url": "https://…signed…" },
    { "name": "convoy.mcap",  "url": "https://…signed…" },
    { "name": "signals.parquet", "url": "https://…signed…" }
  ]
}
3. GET each presigned url directly (no auth header — the signature carries it) before expires_at (default ~15 min); re-request to refresh.

For continuous ingestion, poll the REST list endpoint (or use 🔄 proposed completion webhooks) and pull each new convoy's artifacts. A dedicated read-only IAM principal or a mirrored bucket can be arranged for high-volume data-lake pipelines — ask the BulkScanPro team.