The seafood company in this series needs to reserve stock when an order is accepted. It also needs to analyse production yield over several years. Both jobs move data, and the same cloud product may be capable of parts of both. I would still design their recovery paths separately because a warehouse load and a stock reservation have different consequences when they run twice.
The boundary I care about is the business action. A pipeline that reloads yesterday’s production data can be useful. A replay that books the same truck again needs a different set of controls. Product names alone tell us very little about whether those controls exist.
Work backwards from the booking
For transport booking, I would define a request identity that remains stable through retries. The integration records the request before calling the carrier and stores the carrier’s booking reference when it receives an answer. If the call times out, recovery queries the carrier using that identity where the API supports it.
An uncertain outcome stays visible until it has been reconciled.
Booking and recovery
An Azure Service Bus queue and a .NET worker are a plausible implementation. The queue holds accepted work while the carrier is unavailable. The worker owns the carrier contract and retry behaviour. A separate process table records pending, confirmed, rejected and uncertain outcomes. Those states let operations distinguish a request waiting to be sent from a booking that may already exist.
Status and reporting
The ERP can expose the booking status to users. The data warehouse can receive confirmed booking events through its own consumer. This allows reporting to catch up after an outage without becoming part of the carrier call’s success path.
Give the analytical load its own recovery model
For production analysis, I would keep the original extracted records in a restricted landing area with a source identifier, extraction time and source revision. Azure Data Factory could orchestrate extraction and copying into storage. SQL transformations or another data-processing runtime could then build the reporting tables. The exact runtime depends on volume, team skills and the transformations required.
Publish an accepted load
A load should have a runId and a manifest. I would validate row counts, required fields and key uniqueness before publishing the result. If a partition is replaced during recovery, consumers should see a complete accepted version. The reporting model also needs explicit rules for corrections: a revised batch weight may require recomputing yield for an earlier day.
Duplicates and late data require deliberate treatment in analytics too. A duplicated production row can change a forecast just as surely as it changes a report. An upsert keyed by source record and revision, or a deduplicated transformation over retained history, makes the rule inspectable. Choosing between them depends on whether the report needs current state, an audit history, or both.
Define the timestamps and quantities
For a batch record, I would carry batchId, producedAt, receivedAt, sourceRevision, weight and unit. Quality approval should have its own status and approval time. Production completion and quality approval describe different events; the data model needs room for both.
The stock measure needs equally careful naming. Physical stock, reserved stock and available-to-promise stock answer different questions. I would keep their definitions beside the dataset and test them against a few known orders. A forecast trained on physical stock when its users expect available stock starts from the wrong quantity even if every pipeline is green.
Keep independent progress for each consumer
Billing, transport and analytics should each be able to pause and recover within their agreed limits. With a topic-based design, each consumer gets its own subscription and dead-letter handling. For a retained event stream, consumers need independent checkpoints and a retention period long enough for their recovery requirements.
This independence has a storage cost. A stopped consumer accumulates work until its retention limit is reached. I would alert on its lag and agree how to rebuild it from the source or retained history. A line on a diagram labelled replay is incomplete until somebody knows where the replay data comes from and which side effects it may repeat.
Sharing a platform can still be practical
The same team, deployment tooling or cloud estate can support both workloads. I would separate identities, access permissions, capacity limits and recovery procedures where a failure could cross the boundary. A large historical reload should have a way to slow down before it exhausts resources needed by today’s orders.
A data platform can also produce an operational recommendation, such as a proposed replenishment order. In that case I would expose the recommendation with its calculation time and model version, then submit an explicit command through the operational application’s validation rules. The application checks current stock, limits and approvals before accepting it. Recomputing a prediction should have no accidental purchasing effect.
A useful first test
I would stop the analytical consumer for two hours while continuing to accept orders and book transport in a test environment. After restoring it, I would reconcile the reporting totals and inspect any duplicates or corrections. Separately, I would force a carrier timeout after acceptance and check that recovery finds the existing booking.
Those two tests make the separation tangible. They tell us whether analytical recovery can proceed independently and whether operational retries preserve the intended business result. That is enough direction to evaluate a proposed platform without requiring every team to buy a different product.
