Running three warehouse systems isn’t the hard part. The hard part is that each one is tied to the sales channels it serves, so you can’t send an order to wherever the stock is. What keeps it working day to day is a shared integration contract, a team that writes everything down, and monitoring that treats every single order as important.
The situation
We run three warehouse management systems, plus integrations to several 3PLs. Two are bespoke, one of them a monolith first built in 2001. The third is an off-the-shelf product. Between them they cover fulfilment for most of the globe, for both B2B and B2C.
Nobody sets out to build an estate like this. As a business grows and changes, you end up with several storefronts, more than one order management system and several warehouse systems, all stitched together with integration services.
The oldest bespoke system has grown with the business for 25 years. More recently the focus has been on reducing what it’s responsible for, as the business moves other processes onto the best tool for each job.
Each warehouse system is tightly coupled to the sales channels it serves. Orders go one to one, from a sales channel to its fulfilment channel, so routing is a basic mapping with a little extra logic kept in one place. That also means there’s no flexible fulfilment yet: an order can’t be moved to another warehouse that has the stock.
There are also two order management systems in play.
What we did
One contract, a bridge per system
Every warehouse system integrates differently. Some we push events to or receive events from, others we have to poll. Some offer REST APIs, others XML over SOAP.
So we abstracted the middleware as far as we could. It’s event-driven, running on cloud workers and a message bus. Each system has its own bridge service that owns the integration with that platform.
The key decision was an internal enterprise contract: a definition of each object, such as an order or a shipment, that isn’t tied to any platform. The bridges at the edge translate between that contract and whatever the platform on the other side speaks.
I learned why that kind of design matters back in 2015, at an earlier company. Lightning struck the data centre and we lost all visibility, and the website with it. Once connectivity was restored, everything picked up where it had left off. No messages or events were lost; the systems simply caught up. That was down to how the system was architected. It was a stressful day, but a proud one.
storefronts -> order management -> [ order / shipment contract on the message bus ]
| | |
bridge A bridge B bridge C
| | |
bespoke WMS bespoke WMS packaged WMS (+ 3PL bridges)
A lean team that writes everything down
One very lean team looks after all of our integration. That only works because our knowledge is documentation-led. Everything we do is written down and shared across the team and with operations.
We also built the platforms to support the people running them, which meant putting time into dashboards and metrics across all the core services. If someone is off, anyone else can find where an issue is and work through to the root cause.
More recently we’ve adopted AI tooling heavily. It has been a game changer for our rate of change, but it means agents have to follow our standards too. We do that with agent rules in a markdown file, so any change an agent makes follows the team’s guidelines.
Every order matters
In a high-volume B2C business, a 0% failure rate isn’t realistic. In my experience some people accept a failure rate because “it’s only one order”. I’ve always taken the opposite view: every single order is important. Look after the pennies and the pounds look after themselves.
With this many systems, no single platform is the truth, and no single platform can tell you where an order is, especially when an integration has failed. So we agreed an SLO for how long an order should take to get from the storefront to the warehouse, and built a tool that monitors every order against it. It checks every storefront, the middleware, the core business systems, and the warehouse and logistics tools, and confirms each event has happened in time. If a single order misses the SLO, it’s categorised and escalated to the right team.
That tool deserves an article of its own, so I’ll come back to it.
Testing in layers
We test in layers, and each layer costs a different amount:
- Unit and acceptance tests with the code change. Quick and cheap.
- Functional platform tests, mocking out as many of the external integrations as we can. This layer gives us the most feedback on regressions and full platform features.
- System-to-system tests once a change is promoted to an integrated environment.
- Full end-to-end runs overnight. Slow and expensive.
We decide how much testing belongs in each layer. We don’t test every feature and scenario overnight, because it wouldn’t be cost-effective. Instead we built confidence early that every unique scenario is covered somewhere.
A few years ago we tried to count how many tests it would take to cover every unique scenario for just placing an order. The answer was 3.2 million functional tests. That number is why the layers matter.
Keeping the 25-year-old system
The oldest bespoke system is still there because it works, and it works specifically for this business. It isn’t “legacy”. It has been shaped and matured alongside the business for 25 years, which is a rare position for any platform to be in.
People shy away from radical changes to a system that’s been baked for 25 years: if it isn’t broken, don’t fix it. AI changed that for us. It let us draw on the knowledge built into that application over the years and understand, at a very low level, how the business actually works. With the full context of the application, we built a lot of safety around it, and that gave us the confidence to change it quickly, extend it and modernise it.
The clearest example is its user interface. Using AI under engineering direction, we refreshed the whole thing, more than 800 pages with French and German support, in three weeks. That simply wasn’t possible before, and it changes the conversation about whether a system like this has to be replaced at all.
What worked, what didn’t
What worked:
- The internal contract. Each bridge only has to understand its own platform and the contract. Nothing else needs to know how a given warehouse system works.
- Writing everything down, and the dashboards to go with it. That’s what lets a lean team cover three warehouse systems and the 3PLs.
- Monitoring that follows each order, not each system.
- Testing in layers, with the expensive tests kept for what only they can catch.
- Using AI to understand the old system before changing it.
What hasn’t been solved yet:
- Flexible fulfilment. With each warehouse system coupled to its channels, an order can’t go to the warehouse that has the stock.
- No single place for routing decisions. Two order management systems means no one place to make them.
- No guarantee the stock is there. Without one system in control, another system or process may already have taken the stock at the destination.
- No single source of truth for the state of an order.
What comes next
The first step is to consolidate order capture so that every order funnels through one order management system: all orders will go straight from the sales channels into it. Once it has stock visibility, it can route fulfilment to one or more warehouse systems as needed, and assign, replace or redirect fulfilment requests.
The ERP stays the system of record. It will be told about every transaction and every decision so the financial records are captured.
That gives a single source of truth for the state of any order, so other systems can read order data with confidence. A single view of stock then lets the business optimise fulfilment and, in the end, give customers a better experience.
If you inherit three warehouse systems
- Define the boundaries of responsibility. Be clear about what each system owns.
- Build loosely coupled integrations. A contract in the middle and bridges at the edge.
- Assess each warehouse system honestly. Can it provide an enterprise-grade service?
What I’d do differently
Done is better than perfect. It’s something I’m still learning as my career goes on. Engineers, and I was one of them, want the perfect system. But if you’re always chasing perfect, you don’t deliver much tomorrow. Know when to cut corners, and accept that technical debt isn’t always a bad thing.
We all have mortgages. They’re a necessity. If I saved up to buy a house outright, I’d be homeless for a long time.