Skip to content
Shop integration & processes

Umlauts, Accents, Field Lengths: Names Between Shop and ERP

Why customer names arrive garbled in the ERP: define the character set per system, normalize to NFC, transliterate only for labels, truncate with a warning.

21 min read Daten-MappingZeichensatzStammdatenMiddleware

In short and in plain language

A shop passes names and addresses on to other programs. Some programs do not know all letters. Then a letter turns into a question mark.

Some names look the same. But they are stored in a different way. A program can first bring them into the same form. Then it finds them again.

Some fields are too short for long names. The program should then show a warning. It should not cut the name off without a word.

In the shop and in the business system the name stays correct. Only the label on the parcel may use a simpler form.

A list of test names shows if everything works. We help you set it up.

Suppose the name is right in the shop: Zoë Łęcka-Dumitrașcu, delivery to Große Bahnhofstraße. In the ERP it arrives as “Zoë ??cka-Dumitra?cu”, the shipping label is missing the second half of the surname, and the order with a grinning-face emoji in the note has been stuck in the error list since the morning. None of these errors is exotic. They appear at the boundaries between systems that store characters differently, count them differently and have fields of different lengths. This article shows where names and addresses get damaged on the way from the shop to the ERP and on to the label, how to define the character repertoire for each system, why normalization belongs at the system boundary and what a truncation looks like that reports instead of hiding. DIN 91379, the standard for characters in names, serves as the reference.

Key takeaways

  • Garbled names appear at system boundaries: UTF-8 encodes a character with 1 to 4 bytes (RFC 3629), while older single-byte code pages do not know letters such as Ł or ő at all and replace them with question marks or drop them.
  • DIN 91379 defines the characters for names of people, companies, products, places and streets (DIN Media). IT procedures for data exchange with citizens and businesses had to comply by November 1, 2024 (IT-Planungsrat); for shops it is a reference, not a duty.
  • Names that look identical can be different character sequences, for example é as one character or as e plus an accent. Normalizing to NFC at the system boundary stores and compares them consistently (Unicode Standard Annex #15).
  • Transliteration such as Ł to L belongs only in outputs for downstream systems such as the label or payment file. The master record keeps the original, just as a passport shows diacritics next to its machine-readable line (ICAO Doc 9303).
  • Field limits need rules instead of silent truncation: cut by characters rather than bytes, shorten at word boundaries, mark the truncation and report it as a warning. A test set with names from every delivery language checks this on every change.

How a name turns into question marks in the ERP

Anyone who reads through the error list of a shop integration finds recurring patterns in names and addresses. They have different causes, which is why a single stopgap such as replacing all special characters across the board does not help. The following five patterns can usually be told apart from the data alone:

  • Garbled characters: “Müller” becomes “Müller”. The bytes were written as UTF-8 and read in a Western European single-byte code page, so every umlaut shows up as two foreign characters.
  • Question marks or gaps: “Łęcka” becomes “??cka” or “cka”. The target system does not know the letters and replaces or drops them.
  • Duplicate customers: The same name appears twice in the customer master, because “é” was stored once as one character and once as “e” followed by an accent. The duplicate check compares bytes and finds no match.
  • Cut-off address lines: The street fits into the shop field but not into the ERP or label field. The rest is missing without notice, and the house number with it.
  • Aborted import: A four-byte emoji in the order note does not fit into a database column that stores only three bytes per character, and the entire order gets stuck.

On top of that, there is damage that only becomes visible later: if a system replaces unknown characters when saving, the original spelling is lost there. Every later correction has to be entered by hand, provided a source with the correct spelling still exists. Usually that is the shop, which keeps the customer's input unchanged.

How often such names occur depends on the customer base. The Central Register of Foreign Nationals in Germany gives a reference point for domestic business: at the end of 2025 it recorded around 14.07 million (Destatis) people with a foreign nationality, including 903,770 (Destatis) from Romania, 839,660 (Destatis) from Poland and 1,520,400 (Destatis) from Turkey. Romanian, Polish and Turkish use letters such as Ș, Ł or ğ that are missing from the Western European code page Windows-1252. Add to that deliveries abroad and company names with characters such as & or €. The basics of field assignment are covered in the article on data mapping between ERP and shop; this one is about the name and address fields themselves.

Where characters get lost: code pages and bytes

A character and the way it is stored are two different things. Unicode defines which characters exist; with version 17.0 there are 159,801 (Unicode Consortium). An encoding defines how these characters are stored as bytes. In UTF-8 a character takes 1 to 4 bytes (RFC 3629): a letter without an accent takes one, umlauts and letters such as Ł, ő or ș take two, the euro sign three and characters outside the so-called Basic Multilingual Plane four. Many emojis belong to this last group. As soon as a name contains a letter beyond the English alphabet, it is therefore longer in bytes than in characters in UTF-8.

Older ERP systems, warehouse programs and label printers sometimes still work with single-byte code pages in which every character takes exactly one byte. They cover only a small section. The Western European code page Windows-1252, for example, lacks Ł, ő, ș and ğ, while ë, é and the euro sign are present. There is also a trap in the labels: what one piece of software calls “Latin-1” is not the same everywhere. Under the WHATWG Encoding Standard, “latin1” is just another name for Windows-1252, and the byte 0x80 (WHATWG Encoding Standard) is read there as the euro sign. Other libraries return a control character or an error for the same byte. Two systems that both claim Latin-1 can therefore disagree on something as simple as the euro sign in a company name.

UTF-8 is not always the same UTF-8 either. The MySQL database knows two variants: utf8mb3 uses at most three bytes (MySQL 8.4 Reference Manual) per character, utf8mb4 up to four. A character outside the Basic Multilingual Plane needs four bytes in utf8mb4, and according to the manual a utf8mb3 column cannot store it at all (MySQL 8.4 Reference Manual). The vendor now lists utf8mb3 as deprecated. When reviewing an interface, the question is therefore not only whether the ERP supports UTF-8, but which variant is set in which table and which column.

Error patternTypical causeHow to recognize it
“Müller” instead of “Müller”written as UTF-8, read as a single-byte code pagecharacter pairs such as ü or ö in the data
“?” instead of “Ł”target code page does not know the letterquestion marks in names that are correct in the shop
letter missing entirelyconverter discards unknown charactersname in the ERP shorter than in the shop
import aborts on a notecolumn stores only three bytes per charactererror message about an invalid character value
duplicates despite identical namesdecomposed and precomposed spellings mixeddisplay identical, byte comparison fails
rest of the street missingfield limit in the ERP or on the labelhouse number missing, delivery fails

Measure the state of existing data with a few queries before anyone touches it: names with “Ô followed by another character point to double-encoded umlauts, question marks in name fields to lost letters, combining accents to decomposed spellings. Compare samples against the shop, which still holds the customer's original input. Such measurements belong in a test environment with a copy of the data, not in live operation.

Define the character repertoire for each system

The first step is an inventory: which characters can each system along the way store, display, print and pass on? Shop and middleware usually work with UTF-8 today. The ERP, shipping software, label printer, payment file and the EDI connection to a wholesaler, on the other hand, sometimes have limits of their own. Only once these limits are written down can you decide what happens to a name at which point. When connecting an ERP or inventory system, this overview therefore belongs in the requirements, not in troubleshooting after go-live.

DIN 91379, edition 2022-08 (DIN Media), is a suitable reference for the character repertoire. According to DIN Media, the standard defines the characters needed for names in the broader sense. These include the civil names of natural persons as well as names of legal entities, products, places and streets (DIN Media). Taken as a whole, it covers the official EU languages, plus the official languages of Iceland, Liechtenstein, Norway and Switzerland and the German minority languages (DIN Media). That is the language area a shop delivering to the European Economic Area and Switzerland should cover: customer names, company names, streets and places.

The standard is binding for a different group. In November 2022 the IT-Planungsrat, the German federal and state IT planning council, decided that all IT procedures used for data exchange between the federal and state levels and with citizens and businesses must comply with DIN 91379 by November 1, 2024 (IT-Planungsrat) at the latest. This decision creates no obligation for private shops and ERP systems. For them the standard is a reference: it provides a finished, reasoned list instead of every project deciding for itself which letters a name may contain. Anyone who passes data to public administration procedures also meets recipients that must comply under the decision.

The standard offers two more practical aids. According to its change note, the statements on conformity of IT procedures were extended to cover Normalization Form C (NFC) and UTF-8 (DIN Media), and a digital annex maps the characters to legacy code pages (DIN Media). For an ERP that does not yet work with Unicode throughout, this describes how characters can be handled at the transition without every project inventing its own table.

SystemCharacter repertoire (example)Rule at the boundary
ShopUTF-8, all characters of the inputnormalize input to NFC, remove control characters
MiddlewareUTF-8, DIN 91379 as referencecheck, report, convert per target
ERP master dataUTF-8 or a single-byte code page, depending on the versionstore the original spelling where possible; otherwise log the deviation
Shipping labelcharacter set of the printer or carriertransliteration and truncation only in the output
Payment filecharacter set as required by payment rulestransliterate payee names, keep the original in the document
EDI partnercharacter set as declared in the message headeragree on and test the character set per partner

The table shows at which boundary a character can get lost, and it becomes the basis for acceptance testing. If middleware is part of your architecture, it is the right place for the rules in this article: it sees every message, knows every target and can report before a record arrives damaged. Without middleware, the shop export and the ERP import take on this task, but then in two places with twice the maintenance.

Normalization at the system boundary

Unicode allows two spellings for many letters. The “é” can be stored as a single character or as “e” followed by a combining accent. Both look the same and are canonically equivalent under Unicode (Unicode Standard Annex #15). For a database they are still different byte sequences: a byte-level comparison does not find the decomposed entry “René”, the duplicate check does not fire, and the measured length differs by one character.

Normalization solves this. When an application keeps its strings in a normalized form, equivalent strings have a unique binary representation according to Unicode (Unicode Standard Annex #15). For general text, the Unicode FAQ names NFC as the suitable form because it is more compatible with strings converted from legacy encodings (Unicode FAQ). DIN 91379 also bases its conformity statements on NFC.

Where normalization happens is what matters. It belongs at the boundary where a name enters a system: in the middleware before writing to the ERP, and in the import for data flowing back from the ERP into the shop. Decomposed forms show up in unexpected places; the Unicode FAQ mentions the macOS Finder, which still decomposes file names (Unicode FAQ). And even joining two strings can break a normalized form again (Unicode FAQ). Anyone who combines first and last name into one label field should therefore normalize the result once more.

name-check.php
// Middleware: prepare a name field before export to the ERP
function prepareName(string $raw): array
{
    if (!mb_check_encoding($raw, 'UTF-8')) {
        return ['ok' => false, 'reason' => 'invalid_utf8'];
    }
    $name = Normalizer::normalize($raw, Normalizer::FORM_C);
    $name = preg_replace('/\p{Cc}+/u', ' ', $name);
    $name = trim(preg_replace('/\s+/u', ' ', $name));

    return ['ok' => true, 'value' => $name, 'length' => mb_strlen($name, 'UTF-8')];
}

The example shows the order that matters: first check that valid UTF-8 arrives at all, then normalize, then clean up control characters and double spaces, and only then measure the length. Measuring before normalization counts one character too many for decomposed letters. Invalid byte sequences are not silently repaired but reported as errors. Proper error handling then holds back only that one order and lets the rest continue.

Transliteration only for downstream systems

Not every target can display every letter. The reflex of allowing only simple letters in the shop or the ERP for that reason moves the problem to the wrong place: the original spelling is then lost for good. The passport shows a better way. In its visual inspection zone, diacritics are expressly permitted (ICAO Doc 9303), while the machine-readable zone below uses a transliteration. For “Ä”, the ICAO recommendation lists “AE” or “A” (ICAO Doc 9303). Both versions sit side by side, and neither replaces the other.

Applied to the interface, this means the master record in the ERP keeps the original spelling. The transliteration is created only in the output for a target that needs it, and it is not written back. DIN 91379 contains a table for this that maps Latin letters to base letters in line with ICAO (DIN Media). A separate replacement table for each project is then usually unnecessary, and all outputs can follow the same rule.

What happens when each system decides for itself was described by the European Payments Council for payments in 2009: instead of rejecting a payment, some receiving banks replaced unrecognized characters with blanks or other characters or simply deleted them (European Payments Council). The council recommended a common conversion table so that banks convert characters the same way when needed. The same idea applies to a shop integration: one rule, in one place, with a log. Anyone who matches incoming payments in the ERP knows the consequences of diverging names from payment reconciliation.

  • Transliteration happens in the output for a specific target, not in the master record.
  • One replacement table for all outputs, derived from the mapping to base letters in DIN 91379.
  • Every transliteration is logged: original value, result, target and time.
  • Feedback from a target, such as shipment data, is matched by customer or order number, not by the transliterated name.
  • Apply uppercase for labels before the length check: under the Unicode rules, “ß” becomes “SS” in uppercase (Unicode Consortium), so the name gets longer.

Where the label is created and which data a carrier expects is covered in the article on shipping and logistics interfaces. For transliteration, one thing matters most: it is a property of the output, not of the customer. If the target changes, for example because a carrier starts printing more characters, only the output changes and the customer master stays untouched.

Field lengths: truncate with a warning, not silently

The second big error pattern is field limits. A shop accepts long names and streets, the ERP has its own limits for name, name 2 and street, and a carrier's label has others again. The W3C advises against limiting the field size for names in the database (W3C). But because downstream systems have limits you cannot change, each of these limits needs its own rule.

The unit matters. Some systems count field lengths in characters, others in bytes. According to the W3C, a four-character Japanese name in UTF-8 is likely to need not four but 12 bytes (W3C). For European names the difference is smaller but real: every Ł, ő or ș takes two bytes in UTF-8 (RFC 3629). A field that counts bytes therefore holds fewer letters the more special characters a name contains. A name that just fits in the shop can be too long for the ERP, even though both fields are the same size on paper.

The tool has to be right as well. In PHP, a character in a string is the same as a byte (PHP Manual), so the plain substr cuts by bytes. Only mb_substr works multibyte-safe based on the number of characters, according to the manual (PHP Manual). Truncating with the wrong function splits a two-byte character in the middle and produces an invalid byte sequence on which the receiving system fails. Even counting in characters does not always match what a person sees as a letter: a decomposed “é” consists of two characters. Normalizing to NFC before truncating solves this for every letter that exists as a single character.

The passport again shows what marked truncation looks like. The name field in its machine-readable zone holds 39 characters (ICAO Doc 9303). If the name does not fit, it is shortened according to fixed rules, and a letter in the last position of the field indicates that truncation has occurred (ICAO Doc 9303). Anyone reading the field therefore knows that the name may not be complete there. This is exactly the property an interface lacks when it truncates silently: someone only finds out when a parcel comes back.

SituationRuleMarking
name too long for the ERP name fieldmove the rest to name 2, split at a word boundarynote in the log, no truncation
name too long for the labelshorten first names to initials, keep the surname in fullwarning in the error list with the original value
street too longdo not shorten the house number, move additions to the address supplementwarning, hold the shipment if it is still too long
company name too longabbreviate the legal form from a stored listwarning, approval by sales or shipping
cut would split a charactercut by characters only, not by bytestechnical error, hold back the record

The marking is the core. A warning with original value, target value and affected field goes into the error list, and for addresses where the house number no longer fits, the shipment is held instead of printed. This way the business team learns which customers are affected, and the ERP team sees whether a field limit is permanently too tight. Which system leads for the name and where corrections flow is defined by master data synchronization.

Emojis and control characters in free-text fields

Order notes, delivery instructions and company additions are free text. That is where characters end up that rarely appear in names: emojis, line breaks, tabs, invisible formatting characters and typographic quotation marks. Emojis from the Unicode block Emoticons lie in the range U+1F600 to U+1F64F (Unicode Consortium), outside the Basic Multilingual Plane, and take four bytes in UTF-8. A utf8mb3 column cannot store them (MySQL 8.4 Reference Manual), and depending on the database settings the entire write operation then fails.

The rule for this: a free-text field must not hold up the whole order. If the ERP can store four-byte characters, the note passes through unchanged. If it cannot, the middleware replaces the character with a readable placeholder and reports the case as a warning with the order number; the original note stays in the shop. The note text itself does not belong in the log, as the article on customer data in integration logs shows. Line breaks and tabs in address lines become spaces, because a line break in the street shifts every following line on the label.

Placeholders instead of question marks

A placeholder such as “[symbol]” stays readable for the warehouse and shows that something was replaced here. A question mark, by contrast, looks like a typo by the customer and leads to questions nobody in the warehouse can answer.

The approach deliberately differs from the one for names. A name is master data and must be kept complete, because invoice, dunning and delivery depend on it. A note is information for the moment; a placeholder with a warning loses nothing that matters for delivery or invoicing. Typographic quotation marks and dashes, finally, are not errors: they are kept as long as the target can store them and are otherwise converted using the same replacement table as letters.

A test set with names from every delivery language

A test set that runs on every change to the shop, middleware or ERP shows whether the rules work. It contains no real customer data, only constructed records, each covering a specific risk. The names come from all languages you deliver to; the character list of DIN 91379 serves as the template.

The test set does not start with exotic cases but with the obvious ones. The W3C points out that even English names may contain non-ASCII characters and gives Zoë as an example (W3C). For each delivery language, add the letters that occur there in everyday use, plus the special cases from normalization, length and free text.

Test caseInputExpected result
German special charactersJürgen Größmann, street with ßunchanged in the ERP, on the label according to the target's rule
PolishŁucja WąsowskaŁ and ą are kept in the ERP
RomanianȘtefan Țurcanu, comma below the letterȘ with comma stays, no confusion with Ş with cedilla
HungarianKőváry Ödönő stays, does not become õ or ö
Turkishİlknur Yıldızİ and ı remain distinguishable
decomposed spellingRené with e plus combining accentidentical to the single-character form after NFC, no duplicate
excess lengthdouble surname with three first namesname 2 used, label with marked truncation, warning
free textdelivery note with emoji and line breakorder goes through, placeholder and warning
special characters in company namescompany with &, apostrophe and €characters kept, correctly escaped in XML

Each test case has an expected output per target: ERP record, label data, payment file and, where relevant, EDI message. The check compares bytes, not the display, because decomposed and precomposed forms look the same on screen. For trading partners connected via EDI, the test set belongs in the agreement on the message format; notes on this are in the article on EDI integration.

Rollout in five steps

Queries for double-encoded umlauts, question marks in name fields and combining accents show how many records are affected today and in which system the error occurs.

Legacy data that is already damaged can only be partly repaired. Double-encoded umlauts can usually be reversed because the original bytes are still there. Question marks and deleted letters cannot; the only help there is a comparison with the shop, which still has the original spelling, or asking the customer. We build the correction run from shop to ERP as a separate interface with a log so that every change stays traceable. Which other services around the integration are needed depends on your system landscape; for an initial assessment, a short inquiry listing the systems involved is enough.

The name stays, the output adapts

A customer name is stored correctly once and compared the same way everywhere: in UTF-8, normalized to NFC, with DIN 91379 as the reference for the character repertoire. Transliteration and truncation happen only where a target requires them, and each of them leaves a warning instead of a silent gap.
This article is based on data from: DIN Media, product page DIN 91379:2022-08 “Characters and defined character sequences in Unicode for the electronic processing of names and data exchange in Europe”; IT-Planungsrat, decision 2022/51 of November 10, 2022; Unicode Consortium, announcement of the Unicode Standard 17.0 and Blocks.txt and SpecialCasing.txt of version 17.0.0; Unicode Standard Annex #15 “Unicode Normalization Forms”, revision 57; Unicode FAQ “Normalization”; RFC 3629 “UTF-8, a transformation format of ISO 10646”; WHATWG Encoding Standard; MySQL 8.4 Reference Manual, sections 12.9.1 and 12.9.2; W3C Internationalization, “Personal names around the world”; PHP Manual, Strings and mb_substr; European Payments Council, EPC217-08, best practices for an extended character set (2009); ICAO Doc 9303, Part 3 and Part 4, Eighth Edition 2021; Federal Statistical Office of Germany (Destatis), foreign population by selected nationalities, Central Register of Foreign Nationals.

Related Articles

Law & compliance

EU Customs Reform 2026: Customs Data from the ERP

Since 1 July 2026 the 150-euro duty relief is gone. Commodity code, origin and net weight from the ERP through middleware into shop and customs declaration.

13 min read
Law & compliance

PPWR 2026: Packaging Data from ERP to Your Shop

From 12 August 2026 the EU Packaging Regulation applies directly. How packaging master data, volume logic and reporting emerge from ERP and shop data.

17 min read
Law & compliance

eFTI from July 2027: Digital Freight Data from the ERP

From 9 July 2027, authorities must accept eFTI data, while paper stays allowed for merchants for now. Which fields the ERP supplies and what middleware checks.

21 min read