What if EAN/UCC numbers didn't exist?

I was in the process of setting up a mapping for a new EDI (Electronic Data Interchange) customer. One of the first things I had to do was to take the senders ID from the message envelope and create a look-up link to the customers account on our ERP system. Now just lately I have become obsessed by leaner, simpler EDI. So I looked at this "simple" process with new eyes.

The senders ID, and by extension any party ID, has to be agreed to in advance. This is because it is no good if Acme Inc uses an ID of ACM001 if it is also used by The Association of Comics & Mimes. The receiver wont know which party it is. Now a human will be able to deduce who it is from if the message includes a full address and this is displayed for manual processing. But EDI is not about manual processing.

One solution is for the sender to use the receivers’ allocated ID for the sender (themselves). But they won't know this unless they communicate first.

The most common solution is to use a neutral and authoritative third party’s' allocated ID. The most commonly used are EAN/UCC numbers. However this is not very ubiquitous. Ask any big company CEO or any small one person trader, what their company’s EAN/UCC number is, you will get a blank look. Ask them what their tax number is and they would be able to give you an answer quite quickly. Tax numbers are not a universal solution as there are occasions when they would be not applicable.

But as a receiver of unsolicited messages, a new, never before seen EAN/UCC number is not much use. They are not routinely stored on our ERP customer/supplier database. I can't do a Google search to tell me who is 555987654321 or what is Acme's EAN/UCC. I am stuck with asking and potential EDI partner what their EAN/UCC number is. We are back to prior communication again.

The EDIFACT format standard, NAD segment, lists 300+ organisations that can be used to supply ID numbers. Tax offices, Banking organisations and Standard bodies dominate (EAN/UCC is number 9). None of those listed stood out to me as a better solution so I guess EAN/UCC dominates as it is the least worst.

At this point I realised something. These messages from this sender are not being delivered by a traditional EDI method and as a result our system already knows who they are from. You see they are already linked with an Internet domain name. Now this strikes me as a very good pre-existing "identity" code. It is unique. It is well known. It is in common use. For me it passes the "Fit for purpose" test and the "Ubiquitous" test. I searched the EDIFACT code list but apart from the catch all ZZZ (= Mutually Defined), none seemed applicable.

But then where in all the EDIFACT standard does it refer to a URL, a URI, or an email address? I guess the relevant parts (as they are pretty fundamental) probably date from the early history of the standard, circa 1989. I think I sent my first email on JANET around about then. (That is much earlier than anyone else I know. Whoops! I guess that ages me.)

It is twenty years later now and I think EDI has some catching up to do.

Electronic Data Interchange Needs Electronic Paper

There is probably a word for it. You know, when you are struggling to express an idea and try several times without being satisfied that you are getting your point across. Or indeed not sure what your point is. Then someone else hits the nail on the head. Then you feel the ping of recognition, the swell of confirmed pride and the annoying jealousy of "I wish I had said that".

I want EDI to be easier. Correction, I want EDI to be easy.

In a previous article I was poking fun at the format explosion and buzz word bingo that goes on in the EDI world. The idea that standards are good so we need more of them. The dis-ing of everything that has gone before as "legacy". The constant re-invention of the wheel without leaning the lessons of history.

As a throw-away remark I said in a future article I would announce a solution based on "Facsimile Technology". I knew what I was referring to but up until now I hadn't been able to express it clearly. Then I read it else where.

At The Register, Chris Mellor has an Article entitled "The latest EDI money saver? Paper invoices - Use humans, save money".

I suspect the title is being deliberately provocative. He describes a system where a Scanner is used with OCR to create "electronic copies" of invoices that are machine readable. This is not new and there are other scanner/OCR solution providers out there. If you can't get all your suppliers to send EDI invoices, it is a great way to deal with paper invoices.

The people selling this system explain their thinking...

EDI leaves IT departments perpetually recoding (changing standards, used in differing ways, new formats, different communication protocols).
No all-embracing EDI standard is going to emerge.
The number of paper invoices still being produced after all these years of EDI effort - is massive.

... so if EDI is a failure, accept it and learn to deal with the paper.


I don't want EDI to be a failure, yet I can't believe in a "paperless office". So lets study what paper has going for it.

Paper is ubiquitous, cheap and can be put to many uses.
It can be supplied by many sources and what differences exist (size, weight, color) don't cause tie in.
Organisations are already adapted to handle it.
Humans interface with it easily.


If we look at it the other way, as a sender of invoices. If we accept that some receivers will simply convert (map) the files to paper. Is there an electronic form that EDI enabled receivers can use, yet is just as good as paper for those that aren't enabled? To come close I believe the receiving user should be able to view a list of invoices in a computer folder, like any shared folder. A double click should instantly display the same image as the old paper format. Click print, and it prints. In the folder view, highlight several, Left click, select print, and several print.

HTML & PDF files do this. XML with style sheets come close. ODF & OOXML are related to XML. DOC is closed secret and proprietary.

The file types that don't do this are X12, Edifact, Eancom, Tradacom, Odette (sorry traditional EDI).

The file format is only part of the story. The delivery mechanism needs to be as universal as snail mail. Is there an electronic protocol that EDI enabled receivers can use, yet is just as good as snail mail for those that aren't enabled?

Email is close to this. However it is a bit like using post cards with no certainty of delivery.


In my view EDI needs to be so simple and fault tolerant, that it can be done with a laptop office productivity software. That may mean we have to make changes to our office productivity software. But we definitely need to make changes to our EDI.

Why isn't EDI easier? Part I - One Standard to Rule Them All

EDI (Electronic Data Interchange) really should be less difficult than it is. If I am a start-up company and I want to purchase timber, metal, paper, widgets or some other commodity item, EDI is just too hard.

The overhead on establishing a relationship, agreeing a standard, and testing communications is huge. EDI is supposed to reduce cost but if it results in a supplier tie-in then it will produce unwelcome influences.

If it takes a department of expensive-to-employ / prickerly-to-manage IT geeks to support, then this "cost saving" thing called EDI, just got too expensive.

The first thing that springs to most peoples mind is Standards. How may ways do we need to represent a Purchase Order? Why are there many Standards? Wouldn't it be easier to just all agree on one standard? The answer to the last question is, not necessarily.

To explain this, consider the classic example of Purchase Order Delivery date. On a multi-line purchase order, a customer will probably want all the items delivering together. But while some customers will send a delivery date in the order header section of the message, some will send delivery dates in the order line sections. Some will send both. Some will have differing dates in the detail line section. Some customers will omit dates altogether indicating the goods are required ASAP.

Don't stop me, I'm on a roll...

Sometimes it will be appropriate to send an Earliest and a Latest date range. This could be in either section. Given a free text field some will write 9/11/01, some will write 11/9/2001 and some computers will generate 20010911. By the way, not everyone in the world agrees what year it is or how many months there are.

At the other extreme, timing might be important. For example when ordering services like an aircraft flight time, or an insurance period start and finish. This brings in the question of time zones and daylight saving adjustments.

• So if a Standard is to be universal, it has to be large and complex.

• But if it is large and complex, it won't be easy to implement.

• And your next partner will use the Standard in a way that is at least slightly different to all your others.

When you read "X12 is popular in North America, EDIFACT in Europe, TRADACOM in Britain" remember, just agreeing one Standard doesn't solve anything. One Standard used in two different ways is like two Standards. So the incentive to converge isn't there.

I have been watching the ODF v OOXML Standards dust up with interest. The difference with EDI is that an EDI Standard has many thousands of individual implementations. If we accept Standards overlapping as much as ODF/OOXML damage each other. If interpretation of the specification will lead to differing implementations, which will lead to interchange problems between office applications. Then we shouldn't be surprised if EDI is in trouble.

I think this is the real source of the excitement over XML. It gives structure to data even without a standard. If we accept that even with a standard, there is a need for a "mapping" function, then maybe we should aim to make this as easy as possible.

EDI System Design - Overview

Part I - Message Control
Part II - Export Mapping
Part III - Import Mapping
Part IV - Transaction Processing

EDI System Design Part III - Import Mapping

OK, I know I've missed something out. Between EDI (Electronic Data Interchange) Exporting and EDI Importing is Communicating. But to understand that, it's best to first have an understanding of both the inputs and outputs. Besides, it's my Blog.
What are we going to do with these EDI messages? Possibilities include,

• Print them out in human readable form
• Covert them into an email alert
• Store them for querying and reporting
• Store them for transaction processing

The first option of printing the message out is surprisingly common. I hope you can spot what is wrong with this approach. You see this sort of configuration has a technical name. It is called a Facsimile machine (or a Fax for short). Joking apart, it would be a shame to have spent a large effort of time and resources to discover we have re-invented the Fax machine (and an expensive one at that).

Now it should be obvious what is wrong with the second approach. Our trading partner can send email alerts and cut out the "EDI" system.
Querying and reporting is fine for some types of data. For example if a customer is updating us on EPOS sales data from their branches. We would probably want to run various analysis reports, maybe comparing against forecast, or projecting future stock replenishment orders. This means getting the information into our database. For most this means SQL compatible tables.

If the message is transactional in nature, and we want to action a transaction on our ERP system, then there are also sound reasons for storing the message in our database first. I don't want to go into too much detail (see Part IV), but one reason is you may have transactions sourced from many different EDI message types/standards, and our database table is a good linga franca to pass to the transaction routine.

Another reason is, we should be phasing in EDI software functionality with little updates often, rather than a long wait, followed by a big bang implementation. We can implement a "glorified fax machine", running reports from our database tables, first. Then move to a more functioning system later. I know I ridiculed this arrangement early on, but the point is this should be a transitional stage. We don't want to leave things like this for long.

So our EDI Import Mapping routine will take each message and loop through each segment updating our object and maybe a list of object details. When we reach a message header marker we write the object to the database. Different message types of the same encoding standard will update different object tables but it is worth all these routines updating an "Envelope" table with message meta data, and putting the id field on the object table. This will enable us at a later date to translate various partner identity codes (products, branches etc.) through a look up with a sender identity.

Once these updates have been done we can delete the EDI message from the "pending" folder and move it to an "archive" folder. There will be times when you get unexpected output (or no output) and we will want to check what our trading partner sent in the raw.

EDI System Design Part II - Export Mapping

I'm not going to go into the details of creating the text of EDI (Electronic Data Interchange) messages in certain formats. Well not in this post anyway. It is a big subject and the last post was long enough. What I am trying to do is lay out the System Design and APIs.


So you have a routine in your ERP system that created a document or a transaction e.g. a Sales Invoice. The options you have are,

1. Edit the routine to - Check EDI control. Pass the document to a export mapping routine. Store the returned EDI message in a folder for later batched communication.


2. Edit the routine to - Check EDI control. Store the document reference in a folder for later batched export mapping and communication.


3. Create an external routine to - Periodically select all unchecked documents. Check EDI control. Pass the document to a export mapping routine and communicate.

If you don't have access to your existing source code then your only option is 3. If you do have access to the source code you may still want to do option 3. There is less chance of introducing error and, in these days of Sarbanes-Oxley, the administrative work in getting changes approved, in something like an invoicing routine, must be considered. It doesn't mean you can't move to option 1 or 2 at a later date.


For traditional EDI formats (e.g. EDIFACT, TRADACOM etc.) there will be sections of messages that are common to several documents. Consider the senders address. For each document, this will probably be read in from the same place. The format may say addresses must be presented in a certain way. The first line of the address may be limited to a fixed number of characters but our system allows longer lines. We can't chop off the end of lines, so we will have to move the excess to the second line. This isn't that difficult a task but we don't really want to have to type out this code several times, testing and debugging each one. And if a format version change means a code change is required, then a small task will have just turned into a code hunt.

For all these reasons we want a single export mapping routine per format. Things like, senders address, can then be a single local function or subroutine. One of the calling parameters is document type. The next calling parameter will be a list of documents objects, or pointers to documents - probably the key fields in our SQL table. The returning parameter will be a list of formatted EDI documents. The calling program can then take care of writing to the pending folder or passing them to the communication routine.

There is one other less obvious parameter needed by the export mapping routine and that is the Communication Type data. Even though we have designed a system to separate mapping form communication, this flag is still needed. The reason is envelopes and addressing. Especially in traditional EDI formats, the message document is not complete without an envelope wrapper and we will need the communication type.


Part I