AU Logistics Platform — Reliability & Security Hardening
liveEnterprise "ruggedisation" of a warehouse-management and delivery-tracking platform ahead of expansion across ~11 Australian distribution centres — closing the gap between a startup-scale system and one an enterprise rollout could depend on. Client and platform names are withheld under a standing contractor NDA; the engineering below is not.
Problem
Duplicate stock-allocation errors were reaching production, traced to unbounded retries against three third-party integrations (order management, invoicing, and delivery tracking) with no idempotency boundary — a retry after a slow or failed response could re-apply the same allocation. The platform also had no correlation-ID tracing across services, so diagnosing a failed job meant reconstructing its path from unlinked logs.
Constraints
- No downtime window during the fix — the platform stayed live across all existing distribution centres throughout
- Three external integrations (order management, invoicing, delivery tracking), none of which could be changed — the fix had to sit entirely on this side of each API boundary
- Everything shipped to NZ production first, behind a security regression suite, before touching AU
Decisions
Deduplication keys over distributed locking
Idempotent job processing with deduplication keys, rather than a lock service — avoids adding a coordination dependency to jobs that were already tolerant of at-least-once delivery. Trade-off: correctness depends on every write path deriving its key the same way, which is a discipline cost, not a runtime one.
Bounded retries + dead-letter queue, not indefinite retry
Circuit breakers around each of the three integrations, with bounded retries feeding a dead-letter queue instead of retrying forever. An indefinitely-retrying job hides the failure; a bounded one surfaces it and gives replay tooling a queue to work from once the upstream integration recovers.
Correlation-ID propagation before more logging
Correlation IDs across services and jobs, structured JSON logging to CloudWatch and Sentry, came before adding more log volume — more logs without a way to link them across a job's lifecycle would have made diagnosis slower, not faster.
Terraform migration with remote state, not a rewrite
AWS resources moved to Terraform with S3/DynamoDB remote state and a multi-account layout, migrating existing infrastructure rather than rebuilding it — the goal was a repeatable, rollback-tested path to provision new Australian environments, not a green-field re-architecture.
Rollout
Delivered in fortnightly UAT sprints with a Brisbane-based delivery team. Unit, integration, and Playwright E2E coverage on the critical picking/dispatch workflows became a required CI gate before merge. Shipped to NZ production behind the security regression suite first, then AU, with rollback procedures tested ahead of each environment provision — not assumed.
Disclosure — Client, platform, and the three integration vendors are withheld under a standing contractor NDA. Nothing above names them, reproduces their code, schemas, or credentials, or discloses operational metrics (incident counts, request volume, deployment frequency) that were not already public in the source résumé. This case study is deliberately silent on numeric outcome — an unverifiable percentage is worse than none; ask for a walkthrough if useful.