Most integration projects between ERP and online shop do not fail because of the code. They fail because nobody can test safely before go-live. The ERP has no spare tenant, the test data is a copy of production containing real customer addresses, and the first bulk load runs unthrottled against the live system. The German Federal Office for Information Security states the countermeasure as a basic requirement: software MUST be tested in a test and development environment that is separate from the production environment (BSI IT-Grundschutz CON.8). For personal test data there is an additional rule: it MUST be pseudonymised at minimum (BSI IT-Grundschutz OPS.1.1.6). This article describes an environment strategy for ERP shop integrations that meets these requirements and still tests close to reality: sandbox tenants and their limits, test data between data minimisation and meaningfulness, contract tests against mocks, load and backfill tests, plus release and cutover criteria including a rollback plan.
Key takeaways
- Separate development, test and production environments are a baseline requirement of the German BSI IT-Grundschutz (CON.8.A7), not a nice-to-have. Testing in production advances number ranges, creates documents and sends confirmation emails to real recipients.
- A Business Central sandbox allows 300 OData requests per minute versus 600 in production (Microsoft Learn) and runs on a different performance tier. Meaningful load tests therefore require a dedicated environment of the production type.
- Copying production into a sandbox stops the job queue, disables webhooks, clears SMTP settings, switches off document exchange, blocks outbound HTTP calls and removes database export (Microsoft Learn). Mail, documents and webhooks need a substitute path.
- Personal data used for testing must be pseudonymized at minimum and anonymized where possible (BSI OPS.1.1.6.A11), and Article 5(1)(c) GDPR requires limitation to what is necessary. An anonymization pipeline belongs between production and test, not a backup restore.
- A small synthetic master data set covers every edge case: special pricing, drop shipping, tax exemption, credit limits, bills of material, packaging units. Contract tests against mocks replace one slow integration test with two sets of fast tests (Pact Foundation).
- Backfill and load belong before sign-off: 120,000 items at 100 operations per $batch add up to 1,200 calls (Microsoft Learn), plus resuming after an abort. The acceptance plan under OPS.1.1.6.A10 names release criteria and a path for refusal.
Why integration projects fail at testing
The pattern repeats across projects. The connection between ERP and shop is technically finished, the mappings are agreed, authentication is in place. Then comes the question of where to try it all out – and too often the answer is: in the live system, but carefully. In practice, carefully means a developer creating individual orders during off-peak hours while sales hopes no real invoice falls out of it. That is not a test. That is an experiment with customer data.
The BSI describes exactly this pattern in the threat assessment of building block OPS.1.1.6. If software tests are performed with production data, confidential information can be seen by people who were merely assigned to the test; and if testing uses the original production data rather than copies, that data can be changed or deleted unintentionally (BSI IT-Grundschutz OPS.1.1.6). Building block CON.8 adds that unintended changes to production data also arise when software is tested or operated incorrectly, that such a change may not be detected promptly, and that these errors can affect other applications accessing the same data sets (BSI IT-Grundschutz CON.8).
For an ERP shop interface this effect is particularly unpleasant, because the side effects do not stay inside the system. A test order in the production tenant advances a number series, possibly creates a document, triggers a confirmation email to a real address and travels on into accounting. The BSI also names the root cause of this imbalance: while software is being tested, the focus is not on protecting the test data but on whether the software behaves as intended (BSI IT-Grundschutz CON.8). That is precisely why a robust integration project starts with an environment question, not a code question.
The first bulk run does not belong in production
Separate environments are a basic requirement, not a luxury
The rule is stated unambiguously. In building block CON.8 Software Development, requirement A7 reads: software MUST be tested in a test and development environment that is separate from the production environment (BSI IT-Grundschutz CON.8). This requirement is one of the building block's 8 basic requirements (BSI IT-Grundschutz CON.8), and basic means, in IT-Grundschutz terms, to be implemented as a priority rather than optionally. The same requirement contains two points that tend to get lost in day-to-day project work: test data SHOULD be carefully selected and protected, and it MUST be tested whether the system prerequisites for the intended software are adequately dimensioned (BSI IT-Grundschutz CON.8). The second point is the load question – so it is not at the bottom of a nice-to-have list, it sits in a basic requirement.
The operations building block OPS.1.1.6 Software Tests and Approvals sharpens this. It comprises 6 basic requirements (BSI IT-Grundschutz OPS.1.1.6) and, in standard requirement A13, calls for the separation of test and production environments: software SHOULD only be tested in a test environment provided for that purpose, operated separately from production; the architectures and mechanisms used SHOULD be documented, as SHOULD the procedure for handling the test environment after the software tests have concluded (BSI IT-Grundschutz OPS.1.1.6). That last clause is easily skipped over: a test environment that keeps running with an old production copy after the project is a data risk with no remaining benefit.
Development environment
This is where code works against mocks and a small synthetic master data set. No personal data, no external systems, and feedback in seconds rather than minutes.
Test environment
This is where integration and regression run against a real ERP test tenant with anonymised data. Operated separately, documented, with a defined way of handling it after the project.
Production environment
This is where document output, mail dispatch and number series are live. Whatever happens here for the first time was not a test beforehand – it was an assumption.
In practice, the place where this separation is technically enforced is usually the middleware: it knows the target systems per environment, keeps credentials apart and prevents a test configuration from accidentally pointing at a production endpoint. Point-to-point connections, where the target URL is maintained separately in every script, make exactly this mistake likely – and it often surfaces only when a test order shows up in the real order intake.
Sandbox tenants and their limits
Modern cloud ERP systems ship with test environments. In Dynamics 365 Business Central, the Essential and Premium subscriptions each include one production environment and three sandbox environments free of extra charge; every additional production environment purchased comes with three further sandbox environments and 4 GB of additional tenant-wide database capacity (Microsoft Learn). That is a good starting point – as long as you know how a sandbox differs from production. And the differences are precisely the ones that affect an interface.
The first difference is the limits. For OData requests, Microsoft documents a per-environment rate of 300 requests per minute in the sandbox tenant versus 600 per minute in production (Microsoft Learn). Anyone testing a bulk sync against the sandbox while running close to the limit is measuring half the truth. In addition, per-user limits have applied since late 2023: 6,000 requests per sliding 5-minute window, 5 concurrently processed requests, 95 queued requests and 100 simultaneous connections (Microsoft Learn). If the rate or the queue is exceeded, the service responds with 429 Too Many Requests; queued requests time out after 8 minutes with 503 Service Temporarily Unavailable, and a single OData operation is cancelled after 8 minutes with 408 Request Timeout (Microsoft Learn). The API reference additionally names a limit of 10 minutes of execution time, after which a request ends with 504 Gateway Timeout (Microsoft Learn).
| Characteristic | Sandbox environment | Production environment |
|---|---|---|
| Included in subscription (Essential/Premium) | 3 | 1 |
| Documented OData rate per environment | 300 requests/minute | 600 requests/minute |
| Performance tier in Azure | different, not for benchmarks | target tier of live operations |
| Database export | not possible | possible |
| Outbound HTTP calls from extensions | blocked by default | approved per extension |
| Debugging endpoint | open by default | not intended for it |
The values in this overview come from the product documentation (Microsoft Learn). The second major difference concerns what is deliberately switched off when a production environment is copied into a sandbox. Microsoft documents a series of such precautions: tasks in the job queue are stopped automatically, agents are copied along but turned off in the target environment, all webhooks are set to inactive, the entries of the service connection table and the Exchange synchronisation are disabled, base application integration settings are cleared, SMTP server details are deleted and email accounts of the types Microsoft 365 and Current User are removed; the setup for document exchange and the VAT registration number checking service are disabled as well (Microsoft Learn).
For an integration project this cuts both ways. These precautions are the reason a sandbox copy can be used safely at all: they stop test runs from triggering real emails, real documents and real webhook calls. At the same time they mean the test does not cover the complete process. Document output, mail dispatch and webhook delivery are exactly the parts that only go live again in production. If you do not replicate them in the test through a deliberately built substitute path – a recipient on a test domain, a webhook endpoint under your own control – then you simply have not tested them.
What the sandbox tenant does not cover
The third difference matters most for load tests. Sandbox environments run on a different Azure performance tier than production environments and are not reliable for performance tests or comparable benchmarking; for that purpose Microsoft explicitly recommends a dedicated environment of the production type in order to get the same experience and performance as in actual live operations (Microsoft Learn). So if you want to verify a volume profile, you need a second production environment, not the sandbox included in the subscription. That is a budget decision, and it should be made early in the project rather than three days before cutover.
On the SAP side the picture is similarly layered. SAP Integration Suite offers a simulation of integration flows that lets you check a subset of a flow without deployment and without explicitly enabling tracing. The limits are documented: a maximum of 10 integration flow steps per simulation run and a maximum of 1 MB payload (SAP Help Portal). More important for the test strategy: during simulation no backend or connector calls take place; instead the receiver adapter's answer is supplied as a simulation response (SAP Help Portal). That is a mock at heart, built into the tool – and a good indication that the platform vendor itself does not assume you drive against the real backend at every intermediate step.
Anyone planning an SAP integration or a connection to Dynamics 365 Business Central should know these limits before the test plan is written. Which API generation is the target is a decision of its own – we have covered the migration to the Business Central API v2.0 separately. If test and production environments sit in the cloud, it is also worth looking at the portability and switching obligations that the EU Data Act brings for ERP cloud interfaces.
Test data between data minimisation and realism
The most convenient test data strategy is the production copy. It is realistic, immediately available and contains every special case the company has ever built. It is also the variant that holds up least well in data protection terms. Article 5(1)(c) of the General Data Protection Regulation requires personal data to be adequate, relevant and limited to what is necessary in relation to the purposes of processing (EUR-Lex, Regulation (EU) 2016/679). A functional test of an order interface needs neither real names nor real delivery addresses nor real bank details. It needs data that behaves like the real thing.
Article 32 adds the technical side. It explicitly names the pseudonymisation and encryption of personal data as an appropriate measure (Article 32(1)(a)) and requires a process for regularly testing, assessing and evaluating the effectiveness of technical and organisational measures (Article 32(1)(d)) (EUR-Lex, Regulation (EU) 2016/679). Test data management is therefore not a side issue for development, it is part of the measures an organisation has to demonstrate anyway.
The BSI translates this into a basic requirement. OPS.1.1.6.A11 states: if production data containing information worth protecting is used for software tests, that test data MUST be protected appropriately; if it contains personal information, it MUST be pseudonymised at minimum, and where possible test data with a personal reference SHOULD be fully anonymised. If a personal reference could be derived from the test data, the data protection officer MUST be consulted, and under some circumstances the staff representation as well (BSI IT-Grundschutz OPS.1.1.6). The order is notable: pseudonymisation is the floor, anonymisation is the goal.
A production copy is not a test environment. It is production in a place where nobody works with production diligence.
In practice this means: what belongs between production and the test environment is an anonymisation pipeline, not a backup restore. That pipeline replaces personal fields while keeping the structural properties the interface reacts to. It is a piece of software that is maintained and versioned – and it is the reason a test data state is reproducible rather than merely grown over time.
- Replace names and addresses, keep the structure: a real delivery address becomes a synthetic one – with the same country code, the same postcode logic and the same special characters. That is what mappings trip over, not the name itself.
- Re-key identifiers consistently: customer numbers are replaced using the same mapping across all tables. Otherwise the references between order, document and payment fall apart and the test data set becomes worthless in business terms.
- Route contact channels into the void: email addresses and phone numbers are rewritten to a controlled test domain so that no test run reaches a real recipient – not even if someone accidentally enables mail dispatch.
- Remove payment and bank data rather than distorting it: whatever does not need testing does not belong in the test environment. That is data minimisation under Article 5(1)(c) in its simplest form (EUR-Lex, Regulation (EU) 2016/679).
- Document the re-keying, keep the key separate: anyone who retains the mapping from real to synthetic still has a personal reference in play. That mapping does not belong in the test environment but in the custody of the function accountable for it.
- Set a deletion deadline: OPS.1.1.6.A13 calls for a documented procedure for handling the test environment after the tests have concluded (BSI IT-Grundschutz OPS.1.1.6). An expiry date per test data state saves that discussion later.
Synthetic master data with real edge cases
Anonymised production data solves the data protection problem, but not the coverage problem. It contains what happened in the past – and the very cases that topple an interface are rare. That is why the anonymised copy should be accompanied by a small, deliberately built master data set containing every known edge case exactly once. Small enough to maintain; complete enough to serve as a regression baseline. OPS.1.1.6.A1 sets the bar: test cases MUST be selected so that they check all functions of the software as representatively as possible, and negative tests SHOULD be taken into account (BSI IT-Grundschutz OPS.1.1.6).
Special prices and price lists
Customer-specific terms, tiered prices and time-limited promotions are the classic case where shop and ERP produce different results. One test customer per pricing rule covers it.
Drop-shipment items
Line items shipped directly by the supplier travel differently through stock, documents and shipping notifications than warehouse goods. Without a drop-shipment case in the test data, that path stays untested.
Tax exemption
Intra-community supply, reverse charge, export and the customer with a valid VAT ID: each constellation needs its own test customer with a matching address and flag.
Credit limits and blocks
A customer at the limit, a blocked customer and one required to prepay show whether the checkout processes the ERP's answer correctly instead of ignoring it when in doubt.
Bills of material and sets
Sales BOMs, sets and variants create more line items in the ERP than the shop sent. One set in the test data covers this quantity logic and how it is reported back.
Units of measure and packaging
Pack sizes, minimum order quantities and differing sales units are a frequent source of errors in quantity conversion between shop and ERP – and a rewarding negative test.
This synthetic data set is at the same time the bridge to master data management. What shows up here as an edge case is usually a rule that ought to be described properly in master data management. A test data set containing edge cases often reveals gaps in data maintenance faster than any analysis – because it forces them out rather than searching for them. And because it is synthetic, it may live in any environment, including a developer's machine.
Contract tests against mocks instead of full integration on every commit
Once the environments are in place, the question becomes how often to test against the real ERP. The obvious answer – on every commit – is usually the wrong one. A full integration test occupies two systems, is slow, depends on test data states and fails as soon as the ERP tenant is being updated. Failures caused by environment issues are more expensive than no test at all, because the team starts ignoring red runs. A test nobody believes any more is ballast.
Consumer-driven contract testing inverts the relationship. The consumer – here the shop or the middleware – writes a test against a mock provider. That test produces a contract describing exactly the requests and responses the consumer actually uses. The provider then replays that contract against its real implementation. The Pact documentation describes the core like this: a slow integration test is replaced by two sets of fast unit tests that give fast feedback (Pact Foundation). The side effect is just as valuable: only the parts of the communication that consumers actually use get tested – provider behaviour no consumer uses is free to change without breaking tests (Pact Foundation).
The arguments against full integration tests in the pipeline are in the same documentation: end-to-end tests are slow, slow build times result in batching of changes, and batching is bad for continuous delivery; end-to-end complexity is non-linear and gets harder and messier over time (Pact Foundation). Contract tests, by contrast, execute faster than the integration tests they replace, allow failures to be debugged locally on a developer's machine rather than through system logs, and clearly identify where the problem is when they fail (Pact Foundation). For an ERP integration this means concretely: a changed mandatory field in the order header surfaces in seconds, not in the nightly run.
# On every commit -- seconds to minutes, no external systems
- unit-tests # mapping, rounding and tax logic
- contract-tests # consumer test against a mock provider creates the pact
- provider-verification # replay the pact against the real API implementation
# Nightly -- against the sandbox tenant
- integration-suite # real orders, real documents, anonymised test data
- regression-suite # edge cases: special price, drop shipment, tax exemption
# Before release -- against an environment on the production tier
- load-test # volume profile of live operations
- backfill-test # initial load with the full catalogueIt matters to know the boundary. Contract tests check the integration between two sides, not the business logic and not performance (Pact Foundation). Whether the tax amount is right belongs in a unit test; whether 120,000 items fit into a time window belongs in a load test. And a provider-side contract test alone, without consumer tests, gives no test-based assurance that the consumers are calling the provider in the correct manner (Pact Foundation). The method does not replace a test stage – it sorts them.
Why mocks are not self-deception
These test stages interlock with what the interface has to withstand in operation. How to catch retries and duplicate deliveries cleanly is covered in our article on idempotency and retry strategies; how error cases are handled in a structured way is covered in error handling at interfaces. For contracts to hold across releases, sound API versioning belongs with it – and API development itself should be cut from the start so it can be checked without external systems.
Load and backfill tests with real volume profiles
The test run most frequently missing in projects is the initial load. In ongoing operation, maybe a few dozen orders and a few hundred stock changes flow per hour. On go-live day the entire catalogue flows at once. Anyone who only tested ongoing operation is testing the backfill for the first time in production – on the very day with the least time for surprises.
Work through the volume profile beforehand. An example: a catalogue with 120,000 items, transferred via OData batch with a maximum of 100 operations per $batch request (Microsoft Learn), yields 1,200 batch requests. At the documented rate of 300 requests per minute per sandbox environment that is 4 minutes on paper, and in production at 600 per minute roughly 2 minutes (Microsoft Learn) – assuming no other load shares the quota. On the read side, the maximum page size of 20,000 entities per OData response additionally limits (Microsoft Learn) how many records a single query can return. Numbers like these are established within a quarter of an hour and spare you the project's most unpleasant surprise.
One detail deserves particular attention: Microsoft has since moved the per-environment rates to per-user limits and explicitly points out that the earlier per-environment rates were not strictly enforced, whereas the current per-user ones are (Microsoft Learn). If you run your entire integration through a single technical user or service principal, you will reach these limits accordingly quickly; as a remedy the documentation names distributing the load across multiple users or service principals, for instance in a round-robin rotation (Microsoft Learn). For test planning this means: a bulk run under a single account behaves differently from one that spreads the load – and both variants should have been measured once before go-live.
- Does the volume fit the window? Catalogue size, batch size and rate produce a duration. That duration has to fit into the planned maintenance window, with reserve for retries and rework.
- What happens if it aborts halfway? A backfill that fails at record 80,000 has to be resumable without writing the first 80,000 again. That is a test case of its own, not a side effect.
- Are 429 responses handled correctly? On a 429 the client has to back off and retry after a cool-off period (Microsoft Learn) instead of hammering on in a loop.
- Is normal operation still running in parallel? If the backfill consumes the quota, stock sync and order retrieval stall. Both loads belong measured together, not one after the other.
- Is the result correct? A reconciliation of record counts and field-level samples between source and target belongs in the acceptance record – traceable, not as a gut feeling.
Such runs are also the moment when it becomes clear whether observability is up to the job. A backfill without metrics is a black box; what to measure is described in our article on observability and monitoring at interfaces. With replacements there is the added fact that old and new systems run in parallel for a while – the typical pitfalls are covered in the article on ERP migration from legacy systems.
The load test is a budget question, not a diligence question
Release, cutover and rollback plan
Tests without release criteria end in a debate about whether the result is good enough. The BSI has the acceptance plan for this: under OPS.1.1.6.A10 it SHOULD document the types of tests to be carried out, the test cases and the expected results; in addition, the acceptance plan SHOULD contain the release criteria and define an approach for the situation in which a release is refused (BSI IT-Grundschutz OPS.1.1.6). That last clause is the most important one. An acceptance plan without a refusal path is a statement of intent – because under deadline pressure the easiest thing without a rule is exactly what gets done: release it.
Two basic requirements from the same building block go with it. A3 requires that the results of the software tests be evaluated and the evaluation documented, with a target-actual comparison against defined specifications (BSI IT-Grundschutz OPS.1.1.6). A4 requires that the department responsible in business terms releases the software and documents the release in the form of a release declaration – and in doing so checks whether testing was carried out according to the requirements and whether the results match the expectations defined in advance (BSI IT-Grundschutz OPS.1.1.6). Translated into project reality: development does not release, the business department does, and in writing.
- All edge cases of the synthetic master data set run green, documented with test case number and result.
- Regression tests have run in full; omitted test cases are justified and documented (BSI IT-Grundschutz OPS.1.1.6).
- The backfill has handled the volume profile within the planned window, including a successful resume after an artificially triggered abort.
- The load test on an environment with a production tier held the target profile without normal operation coming to a halt (Microsoft Learn).
- The outbound paths disabled in the sandbox tenant – mail dispatch, document output, webhooks – have been checked via a substitute path (Microsoft Learn).
- The rollback plan has been rehearsed once, not merely written down.
The rollback plan is the point at which most projects fall silent. It answers three questions: how do we recognise that we have to go back – which metric over which period? Who decides that, and until when is the decision possible at all? And what happens to the data created between cutover and rollback? The third question in particular determines whether a rollback is a real option. If the new process has already created documents and drawn number series, going back without rework is out. Knowing that beforehand leads to a different cutover plan – for instance a read-only lead-in before the writing processes go live.
After cutover one task remains that A13 names explicitly: the documented procedure for handling the test environment after the software tests have concluded (BSI IT-Grundschutz OPS.1.1.6). Test environments holding old customer data that nobody touches any more are a risk with no remaining value. And because the acceptance itself is evidence: what was tested, released and changed when belongs in the GoBD process documentation for ERP shop interfaces – the point where the test strategy meets the duty to provide evidence.
What a sound test strategy changes in a project
The effort for environments, anonymisation and test data comes at the start; the benefit shows at the end. That is why these line items are readily struck from quotes, and at the same time why go-live dates slip. An interface that has run under realistic conditions beforehand goes live with a different set of expectations than one that has only been seen in the development environment. The difference is rarely technical – it is a question of preparation.
The building blocks are manageable: separate development, test and production environments, as CON.8.A7 requires as a basic requirement (BSI IT-Grundschutz CON.8); test data from an anonymisation pipeline rather than from a backup, pseudonymised at minimum under OPS.1.1.6.A11 (BSI IT-Grundschutz OPS.1.1.6); a small synthetic master data set with every known edge case; contract tests in the pipeline and full integration at wider intervals; load and backfill tests on an environment with a production tier (Microsoft Learn); and an acceptance plan with release criteria and a refusal path under OPS.1.1.6.A10 (BSI IT-Grundschutz OPS.1.1.6). None of these points is demanding on its own. Together they decide whether the go-live is a date or an event.
We build ERP shop integrations with exactly this environment strategy: a reproducible test environment, documented provenance of the test data, contract tests against mocks and an acceptance record that carries the release. Which environments your ERP brings along, which ones you additionally need and what that costs is something we clarify before the first line of code – a look at our ERP integration services shows how we cut a project like this.
Sources and studies