Green Coffee XML - Revisited

I wrote before about my frustration at EDI Standards Proliferation. I also said if there was anything more fustrating than EDI Standards Proliferation, it was EDI Standards Secrecy.

Well done to the Green Coffee Organisation. I can now find reference material for Green Coffee XML on the internet for free. This link leads you to a 124 page PDF document, Contract Terms & Conditions. There listed in appendix A-D is the specification.

[It is there at time of writing. Whether it is moved after the next site redesign is another issue.

So long as it is findable by the major search engines.]

I can't make head or tail of it though (excuse the pun). No worked examples. While I am not an expert on XML, I believe most definitions use DTD or XML Schema. This does not look like either. But still, that is there right and freedom.

The licence is "...royalty-free, worldwide..." and covers copies "...all or part... ..in any form..." provided the copyright notice is retainned and patent truce is maintained.

Well done Green Coffee Org.

When Lost, it helps to remember where you want to go

I have the occasional high school student pass through my office on work experience. They do the rounds going to each department. When they come to me they have already spent time in the Sales and Purchasing departments. I am expected to give them a 30 minute overview of EDI (I don't just do EDI, but no one else understands it, or wants to understand it, so that is what they ask me to do). Then they move on to the help desk where they occupy them with tasks, like prepping desktops. They always like that, but I can't claim any are enthusiastically interested in what I have to say. I find the exercise helps to keep my feet firmly on the ground and head out of the stars, as they sit there wondering (despite my best efforts) what I'm talking about.

Well, the other day it was the turn of the one of my IT colleagues offspring. Keen to make a good impression, I was spurred on to think up a new and better explanation than I had used before. I was pleased with the results so I thought I would reproduce a version of it here.

***

When you enter Sales Order Processing department, you will see people opening the mail and extracting customer purchase order forms. Some of them are hand written, some are computer printed.

There is also have a Fax machine dedicated to customer orders that spews out a steady stream of order forms. Again, some of them are hand written, some are copies of computer generated order forms.

We also have a team of people who take orders verbally over the phone.

Finally, some customer orders come in as emails. Usually as PDF or Microsoft Word attachments. All these orders are read and the details typed into our Computer system.

This raises 2 questions, and leads to a 3rd great big "What If" question.

Q1. Why aren't all orders emailed?

It is cheaper not to have labor deployed in the mail room, or manning the phones, or maintaining the fax line and fax machine. The cost of receiving emails is tiny in comparison. When you scale up to hundreds, thousands, tens of thousands a day - for emails, the cost graph raises far less steeply. The same is true for our customers. Most of them are companies and organisations. It is cheaper for them not to pay for postage, or phone calls.

So I repeat the question. Why aren't all orders emailed? The answer? I don't know, you should ask the customers. But the point is, whatever the real answer (and it might be different for different customers) take a note of how difficult it is to change our customers behaviour. We can't afford to turn away business by not accepting the orders and we don't want, or to make it hard to order from us.

Q2. Why do all orders have to be re-typed into our Computer system?

Every product is made up of many components and materials. All of which have to be purchased. Every product requires many different sorts of resources, people and machines. All have to be planned and instructed what to do. Controlling all these aspects takes a complicated Computer program (called an ERP system) and that is why they have to be entered.

Q3. What if, instead of having to re-type everything, you could click and drag the email attachment, drop it on the ERP icon on your desktop, and the software processed it for you?

I will tell you "what if". If you could do that you will have achieved what businesses and organisations have been trying to achieve for decades. This is the ultimate aim of EDI.

Why is this so difficult? To a computer, text, is text, is text. How does a computer tell one piece of text is a product description and another is an address, one is a quantity and another is a price, one a delivery date another is a created date? To humans this is easy. For computers this is hard.

Ever tried to open a Microsoft Word document with Notepad? That is what the file looks like to a programmer. What do all the non-text characters mean? Only those who have signed a non-disclosure contract with Microsoft know, and none of them have permission to tell you.

First thing you have to do is agree a document format that is open to everyone. Then everyone has to agree where to place each bit of data or how to label it. In short, globally speaking, we can't agree. Many have been expecting one of the many formats to emerge as the dominant one, like the way every one uses MP3 instead of WMA or WAV, but this hasn't happened.

The next thing is product identity. We distribute a catalogue describing all our products and product options along with product codes (you see these bar-coded on the packaging). We encourage our customers to quote these in the order so that there is no miss-understanding. How do our customers (and their computer systems) know this information? If they order the same product from different suppliers, each will have their own code.

At least sending the document is easy. I mean everyone uses email, right? How else would you send electronic data from A to B? Er.. hmm... well... No. We have been moving electronic data around since before email and the Internet. There are dedicated EDI networks. There are Extranet web sites. Third party hosted web app clearing houses. There are different ways of securing emails. There are different encryption solutions. Security Certificates. Web of trust. etc. etc. etc.

So what are we to do? We could pick one solution and tell all our customers to communicate in that way. But remember these are the same customers, some of whom are still using post and fax. Remember how difficult it is to change some customers behaviour? Some of the big customers might prefer a different solution. We won't turn them away, but very quickly we find ourselves supporting many different forms of EDI. The cost of implementing another one has to be weighed against the volume of business (and cost saving) we can expect in the future.

Now turn your point of view around. When we order from our suppliers, we become the customer. The complexity just doubled. I have only talked about orders, but there are many other documents that are exchanged in business.

***

Are you Lost? Can you remember where we were heading? Any bright ideas?

What if EAN/UCC numbers didn't exist?

I was in the process of setting up a mapping for a new EDI (Electronic Data Interchange) customer. One of the first things I had to do was to take the senders ID from the message envelope and create a look-up link to the customers account on our ERP system. Now just lately I have become obsessed by leaner, simpler EDI. So I looked at this "simple" process with new eyes.

The senders ID, and by extension any party ID, has to be agreed to in advance. This is because it is no good if Acme Inc uses an ID of ACM001 if it is also used by The Association of Comics & Mimes. The receiver wont know which party it is. Now a human will be able to deduce who it is from if the message includes a full address and this is displayed for manual processing. But EDI is not about manual processing.

One solution is for the sender to use the receivers’ allocated ID for the sender (themselves). But they won't know this unless they communicate first.

The most common solution is to use a neutral and authoritative third party’s' allocated ID. The most commonly used are EAN/UCC numbers. However this is not very ubiquitous. Ask any big company CEO or any small one person trader, what their company’s EAN/UCC number is, you will get a blank look. Ask them what their tax number is and they would be able to give you an answer quite quickly. Tax numbers are not a universal solution as there are occasions when they would be not applicable.

But as a receiver of unsolicited messages, a new, never before seen EAN/UCC number is not much use. They are not routinely stored on our ERP customer/supplier database. I can't do a Google search to tell me who is 555987654321 or what is Acme's EAN/UCC. I am stuck with asking and potential EDI partner what their EAN/UCC number is. We are back to prior communication again.

The EDIFACT format standard, NAD segment, lists 300+ organisations that can be used to supply ID numbers. Tax offices, Banking organisations and Standard bodies dominate (EAN/UCC is number 9). None of those listed stood out to me as a better solution so I guess EAN/UCC dominates as it is the least worst.

At this point I realised something. These messages from this sender are not being delivered by a traditional EDI method and as a result our system already knows who they are from. You see they are already linked with an Internet domain name. Now this strikes me as a very good pre-existing "identity" code. It is unique. It is well known. It is in common use. For me it passes the "Fit for purpose" test and the "Ubiquitous" test. I searched the EDIFACT code list but apart from the catch all ZZZ (= Mutually Defined), none seemed applicable.

But then where in all the EDIFACT standard does it refer to a URL, a URI, or an email address? I guess the relevant parts (as they are pretty fundamental) probably date from the early history of the standard, circa 1989. I think I sent my first email on JANET around about then. (That is much earlier than anyone else I know. Whoops! I guess that ages me.)

It is twenty years later now and I think EDI has some catching up to do.

Electronic Data Interchange Needs Electronic Paper

There is probably a word for it. You know, when you are struggling to express an idea and try several times without being satisfied that you are getting your point across. Or indeed not sure what your point is. Then someone else hits the nail on the head. Then you feel the ping of recognition, the swell of confirmed pride and the annoying jealousy of "I wish I had said that".

I want EDI to be easier. Correction, I want EDI to be easy.

In a previous article I was poking fun at the format explosion and buzz word bingo that goes on in the EDI world. The idea that standards are good so we need more of them. The dis-ing of everything that has gone before as "legacy". The constant re-invention of the wheel without leaning the lessons of history.

As a throw-away remark I said in a future article I would announce a solution based on "Facsimile Technology". I knew what I was referring to but up until now I hadn't been able to express it clearly. Then I read it else where.

At The Register, Chris Mellor has an Article entitled "The latest EDI money saver? Paper invoices - Use humans, save money".

I suspect the title is being deliberately provocative. He describes a system where a Scanner is used with OCR to create "electronic copies" of invoices that are machine readable. This is not new and there are other scanner/OCR solution providers out there. If you can't get all your suppliers to send EDI invoices, it is a great way to deal with paper invoices.

The people selling this system explain their thinking...

EDI leaves IT departments perpetually recoding (changing standards, used in differing ways, new formats, different communication protocols).
No all-embracing EDI standard is going to emerge.
The number of paper invoices still being produced after all these years of EDI effort - is massive.

... so if EDI is a failure, accept it and learn to deal with the paper.


I don't want EDI to be a failure, yet I can't believe in a "paperless office". So lets study what paper has going for it.

Paper is ubiquitous, cheap and can be put to many uses.
It can be supplied by many sources and what differences exist (size, weight, color) don't cause tie in.
Organisations are already adapted to handle it.
Humans interface with it easily.


If we look at it the other way, as a sender of invoices. If we accept that some receivers will simply convert (map) the files to paper. Is there an electronic form that EDI enabled receivers can use, yet is just as good as paper for those that aren't enabled? To come close I believe the receiving user should be able to view a list of invoices in a computer folder, like any shared folder. A double click should instantly display the same image as the old paper format. Click print, and it prints. In the folder view, highlight several, Left click, select print, and several print.

HTML & PDF files do this. XML with style sheets come close. ODF & OOXML are related to XML. DOC is closed secret and proprietary.

The file types that don't do this are X12, Edifact, Eancom, Tradacom, Odette (sorry traditional EDI).

The file format is only part of the story. The delivery mechanism needs to be as universal as snail mail. Is there an electronic protocol that EDI enabled receivers can use, yet is just as good as snail mail for those that aren't enabled?

Email is close to this. However it is a bit like using post cards with no certainty of delivery.


In my view EDI needs to be so simple and fault tolerant, that it can be done with a laptop office productivity software. That may mean we have to make changes to our office productivity software. But we definitely need to make changes to our EDI.

Freedom !

I have just read Craig's post on EDITalk and was motivated to leave a long comment. So I have decided to re-post it here.

Quoting Braveheart is like a red rag to a bull to me. I have got to respond.

I am going to go on a bit about history here, but bear with me. I will come back to EDI (Electronic Data Interchange) at the end.

For the average Joe, the freedom Wallace (and Mel Gibson) was fighting for was to be lorded over by a Scotsman (Bruce) rather than an Englishman (Edward). In both cases Joe remained a peasant/serf with no right to land ownership, an obligation to pay taxes (without representation) and with little hope of a evenly applied "rule of law". Is the race of the King significant?

The attempts by various English Kings of this period (1200-1350 ?) to acquire Scotland was a MAJOR historical cock up that delayed the creation of a unified nation (Britain) by hundreds of years. For all the historical inaccuracies of the Braveheart film, one point it did correctly portray was that the Scottish ruling elite owned significant landholdings in England.

Inter-marriage and cross-cultural exchanges was bringing both countries together. If things had been left alone and there hadn’t been the many wars, unification might have happened anyway by joint inheritance. This eventually happened in 1603 when Elizabeth I of England was succeeded by James VI of Scotland (her fathers, sisters, great-grandson - or first cousin, twice removed - I think!)

Even then, things were not settled. The last big rebellion happened in 1745 when "Bonnie Prince Charlie" was defeated at the battle of Colloden. This event holds an emotional place in the culture of Scots and is seen by many as the "death of nationhood". However in this battle there were more Scots fighting on the "English" side than English. The alternative view is that this event marks the end of "tribalism" on this island and the rise of the modern Nation State.

Back to EDI.

From this perspective the freedom to choose a format looks like a freedom to choose to be enslaved by a ruling elite.

The freedom to depart from the standard is the freedom to not to be bothered with the law - that is for little people (or suppliers).

The freedom to reject the established standards and formats completely and develop your own is like sailing off to the New World in hope of a better life. You know it is not going to be a bed of roses but you think it is the future.

So how do we avoid repeating the mistakes of history? I think it is important understand what is important and makes a difference to our lives.

A N other encoding format is no better than one we already support. The new one might be easier to utilise but is requires effort to get to there from here. Using what we have is easier.

A single dominating format that was accepted by all, would produce future savings and improvements. This would be disruptive to the status quo. So to succeed it would need additional benefits beyond simple convergence, or we are in for a long wait.

CSVML - Accept No Compromise

Can't decide between XML and CSV? I have the answer, and it is not JSON as I previously thought.

I have seen the light.

The answer is here. Hilarious! (well it is hilarious if you are a geek).

In a future post I shall show how the future of EDI (Electronic Data Interchange) is a merger of Edifact, X12 & Tradacom by using Facsimile technology...

Alternative EDI Formats Part II – JSON & Protocol Buffers

In the previous post I wrote how a large amount of EDI (that is Electronic Data Interchangein the widest sense) is done, without using a strict formalised standard, using CSV formats. Now Google has released details of how they execute server-to-server/program-to-program message interchange using Protocol Buffers. You won’t see the term EDI any where on Google but then the term doesn’t have a sexy web 2.0 image.

Google rejected the use of XML. I am all for that. To be fair, I think this is more to do with the desire for a binary format for super fast, supper scalable encoding and decoding. Inter-company EDI is universally text based. I can’t see that changing.

The first thing I noticed about the .proto files is their similarity to JSON. Their use seems to have pre-dated the popularisation of JSON. In other areas I have seen Google use YAML for similar definition purposes.

The .proto files are not message files. They are not sent as part of a message, ever. They are used to automatically compile programs to handle messages in the format defined by these files.

Now this struck me, because I think this is one area where CSV beats traditional EDI standards. That first row, of column headings, is like the file definition. If a trading partner adds new columns (or removes columns, or moves columns) the next time he sends the same type of message, it doesn’t matter. We don’t need to agree beforehand. The reciever can identify which cell is which piece of information by locating the column heading position.

Stripping the .proto example down to equate it with our first simplified JSON message data from the previous post, we get the following,

[[‘Jodie Foster’,1,’jfoster@silence.com’,’555-1234’],
[‘Sigourney Weaver’,2,’sweaver@alien.org’,’555-9876’],
[‘Drew Barrymore’,3,’dbarrymore@angel.net’,’555-2468’]]

message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
repeated PhoneNumber phone = 4;
}

The field list is in a [Modifier – Type – Field Name – Sequence] format. Modifier and Type wouldn’t make much sense in JSON which is not restrictive in its type usage. Incorporating the sequence number into our JSON definition section gives us a useful ability.

{‘definition’:{‘name’:0,’id’:1,’email’:2}}

MessageObject.definition[‘name’] returns 0
Or,
MessageObject.definition.name returns 0

MessageObject.data[0][MessageObject.definition[‘name’]] returns Jodie Foster

Now we have the same ability to cope with our trading partners adding, moving and removing fields without the format losing its meaning.

<aside> Did you notice Google started numbering at 1 and not 0? What is that about? That is Muggle thinking! </aside>

What happens when we expand the pone field into a sub-table like before? On its own this sub-table would have a definition of,

{‘phonenumber’:0,’type’:1}

but we can't just slot this in and replace the existing phone field definition becuase we would lose the positional data. What Protocol Buffers does is list the definitions separately.

{‘definition’:{‘person’:{‘name’:0,’id’:1,’email’:2,’phone’:3},
'phone':{’phonenumber’:0,'type':1}},
‘data’:[[‘Jodie Foster’,1,’jfoster@silence.com’,
[[’555-1234’,'home'],
['555-777','mobile'],
['555-1235','fax']]],
[‘Sigourney Weaver’,2,’sweaver@alien.org’,
[[’555-9876’,'home'],
['555-0101','office']]],
[‘Drew Barrymore’,3,’dbarrymore@angel.net’,
[[’555-2468’,'home']]]]}

a=MessageObject.definition['person']['phone']
b=MessageObject.definition['phone']['phonenumber']
c=MessageObject.definition['phone']['phonetype']
MessageObject.data[0][a][2][b] returns 555-1235
MessageObject.data[0][a][2][c] returns fax

In this way the sender can omit any fields they like and the field sequence is no longer important. The receiver can still parse the message and extract the data segments. The message file size is kept to a minimum. returns

This is not JSONML (althogh that is intresting in it's own right) . This is about efficiently transporting a (potentially large) list of data objects of the same type.

Alternative EDI Formats Part I – CSV & JSON

I have been meaning to make this post for a long time, then Google came along with Protocol Buffers and the world moves on. So in this post I am going to outline how CSV files are used and how I thought JSON would be an improvement. In another post I will write about what I think can be learnt from Protocol Buffers.

A lot of data is communicated from machine to machine by CSV file format. It might not be strict EDI but it is electronic data interchange. It almost feels like an uncomfortable little secret no one likes to talk about (OK I admit it. I am trying to avoid the Elephant cliché).

To show what I mean, look at the number of responses to these keyword searches on Google. I know it isn't an accurate measure (compare Tradacom with Tradacom & EDI !?!?) but this is just for indicative purposes.

Keywords	Number of Google Links
X12	46,400,000
X12 & EDI	295,000
EDIFACT	802,000
EDIFACT & EDI	241,000
Tradacom	4,410
Tradacom & EDI	5,350
XML	650,000,000
XML & EDI	451,000
JSON	8,680,000
JSON & EDI	21,000
CSV	52,400,000
CSV & EDI	1,040,000

Note that CSV out ranks all the other terms when combined with EDI. It even out ranks the unqualified EDIFACT search - the ‘UN’ standard for EDI.

Why? Well CSV is easy. It is human readable. It can be output from spreadsheet programs. Most of all, the columns and rows closely resemble the way data is stored in RDBMS tables which is the destination of most EDI data.

Taking inspiration from Google’s Protocol Buffer example, an address book could be represented as follows…

name,id,email,phone
Jodie Foster,1,jfoster@silence.com,555-1234
Sigourney Weaver,2,sweaver@alien.org,555-9876
Drew Barrymore,3,dbarrymore@angel.net,555-2468

All the programmer needs is a ‘splitting’ function to slice the file up, first by carriage returns, then by commas. In JSON format this same data may be represented as follows…

[{‘name‘:’Jodie Foster’,’id’:1,’email’:’jfoster@silence.com’,’phone’:’555-1234’},
{‘name‘:’Sigourney Weaver’, ‘id’:2, ‘email’:’sweaver@alien.org’, ‘phone’:’555-9876’},
{'name‘:’Drew Barrymore’, ‘id’:3, ‘email’:’dbarrymore@angel.net’, ‘phone’:’555-2468’}]

MessageObject[0].name returns Jodie Foster

However the file size has just ballooned. To overcome this, it could be represented in JSON another way to produce a much smaller file…

{‘definition’:[‘name’,’id’,’email’,’phone’],
‘data’:[[‘Jodie Foster’,1,’jfoster@silence.com’,’555-1234’],
[‘Sigourney Weaver’,2,’sweaver@alien.org’,’555-9876’],
[‘Drew Barrymore’,3,’dbarrymore@angel.net’,’555-2468’]]}

MessageObject.definition[0] returns name
MessageObject.data[0][0] returns Jodie Foster

Now suppose Ms Foster is good enough to give us her mobile and fax number in addition. The ‘phone’ field becomes a list. For the CSV file, another delimiter is needed.

name,id,email,phone
Jodie Foster,1,jfoster@silence.com,555-1234/555-777/555-1235
Sigourney Weaver,2,sweaver@alien.org,555-9876
Drew Barrymore,3,dbarrymore@angel.net,555-2468

But what if we want to hold phone number type as well (home, mobile, office, fax etc.)? We have 3 options…
1. add another field, also sub-delimited, where the sequencing matches the other field. 555-1234/555-777/555-1235,home/mobile/fax
2. turn the ‘phone’ field into a compound field. 555-1234]home/555-777]mobile/555-1235,home. The column heading becomes phone/type.
3. create a separate table for the fields. Rows in this new table need a unique identifier to rows in the original table.

At this point the CSV format is beginning to creek. Beyond 1 nested table, options 1 & 2 will require ever more different delimiters. So let us concentrate on option 3. In isolation this new sub-table would look like this,

id,phone,type
1,555-1234,home
1,555-777,mobile
1,555-1235,fax
2,555-9876,home
2,555-0101,office
3,555-2468,home

These files can be sent separately. If they are to be combined into 1 message then we need to indicate in some way what table each row is part of. Typically this is done by reserving the first column. In this example it could contain phoneheader-definition, phoneheader-data, phonedetail-definition, phonedetail-data.

How would we represent this in our JSON format?

{‘definition’:[‘name’,’id’,’email’,[’phone’,'type']],
‘data’:[[‘Jodie Foster’,1,’jfoster@silence.com’,
[[’555-1234’,'home'],
['555-777','mobile'],
['555-1235','fax']]],
[‘Sigourney Weaver’,2,’sweaver@alien.org’,
[[’555-9876’,'home'],
['555-0101','office']]],
[‘Drew Barrymore’,3,’dbarrymore@angel.net’,
[[’555-2468’,'home']]]]}

MessageObject.definition[3][0] returns phone
MessageObject.data[0][3][2][0] returns 555-1235
MessageObject.data[0][3][2][1] returns fax

While this encodes and represents the same message, is it better than CSV?
It is more extendable, it is slightly bigger, it is probably equally as human readable, and probably equally as machine readable. I already thought JSON was a good candidate for being the next CSV for EDI. In the next post I will write about how taking inspiration from Google’s Protocol Buffers, I think it can be improved further.

Green Coffee XML

I am not kidding (pdf). Some might think this is great. Some might think is shows how wonderful XML is. I don't. To me it represents a lot of what is mixed up about EDI (Electronic Data Interchange). I want to make 2 points...



What is so special about Green Coffee that it needs it's own schema?

• Well reading the docs it seems coffee dealers are a bit fussy about defining when ownership of the product and ownership of the risk (associated with product delivery) is transferred. So they have 9 different order types.
• As well as the buyer and seller, they need to be precise about the Broker and the Shipper.
• The quality of the product is defined by a standard and is reflected in the product codes.
• Pricing can be by formula.
• Unit of measure is usually Kgs but when it comes to weighing coffee it seems to be important who weighs, when, and who pays for the weighing. I count 8 weighing types.
• The journey coffee makes can be long and the value of the coffee at different stages changes so it seems the "place of tender" is important. A simple "delivery date" is not precise enough and must be qualified.

Phew! Complicated. But excuse me. Is any one of these points unique to coffee? Maybe the combination is unique. Maybe it is more sophisticated than Acme retail EDI. But what does it gain us to reject all that has gone before in 60 years of EDI and create new EDI ghettos ?


I hope they didn't. I hope they just defined some extra tags and specified some extra attribute values, and added them on to some existing, already utilised and proven XML order standard. Which brings me to my next point.


How (for the love of coffee!) can I implement this?


I went in search of the technical details. The PDF document listed 4 XML Appendices on the contents page. They seem to be missing from the web. I went to root URL and clicked around. I couldn't even find my way back to the document. I used Google to search the site - zilch. I used Google to search the web for "Green Coffee XML", no luck.

How can you expect a schema to be used if you wont tell anybody the details? If you want it to succeed make it freely available! Have you not heard of Peer Review?

Why isn't EDI easier? Part I - One Standard to Rule Them All

EDI (Electronic Data Interchange) really should be less difficult than it is. If I am a start-up company and I want to purchase timber, metal, paper, widgets or some other commodity item, EDI is just too hard.

The overhead on establishing a relationship, agreeing a standard, and testing communications is huge. EDI is supposed to reduce cost but if it results in a supplier tie-in then it will produce unwelcome influences.

If it takes a department of expensive-to-employ / prickerly-to-manage IT geeks to support, then this "cost saving" thing called EDI, just got too expensive.

The first thing that springs to most peoples mind is Standards. How may ways do we need to represent a Purchase Order? Why are there many Standards? Wouldn't it be easier to just all agree on one standard? The answer to the last question is, not necessarily.

To explain this, consider the classic example of Purchase Order Delivery date. On a multi-line purchase order, a customer will probably want all the items delivering together. But while some customers will send a delivery date in the order header section of the message, some will send delivery dates in the order line sections. Some will send both. Some will have differing dates in the detail line section. Some customers will omit dates altogether indicating the goods are required ASAP.

Don't stop me, I'm on a roll...

Sometimes it will be appropriate to send an Earliest and a Latest date range. This could be in either section. Given a free text field some will write 9/11/01, some will write 11/9/2001 and some computers will generate 20010911. By the way, not everyone in the world agrees what year it is or how many months there are.

At the other extreme, timing might be important. For example when ordering services like an aircraft flight time, or an insurance period start and finish. This brings in the question of time zones and daylight saving adjustments.

• So if a Standard is to be universal, it has to be large and complex.

• But if it is large and complex, it won't be easy to implement.

• And your next partner will use the Standard in a way that is at least slightly different to all your others.

When you read "X12 is popular in North America, EDIFACT in Europe, TRADACOM in Britain" remember, just agreeing one Standard doesn't solve anything. One Standard used in two different ways is like two Standards. So the incentive to converge isn't there.

I have been watching the ODF v OOXML Standards dust up with interest. The difference with EDI is that an EDI Standard has many thousands of individual implementations. If we accept Standards overlapping as much as ODF/OOXML damage each other. If interpretation of the specification will lead to differing implementations, which will lead to interchange problems between office applications. Then we shouldn't be surprised if EDI is in trouble.

I think this is the real source of the excitement over XML. It gives structure to data even without a standard. If we accept that even with a standard, there is a need for a "mapping" function, then maybe we should aim to make this as easy as possible.

When did XML become a good Idea?

It is so easy to list what is wrong with XML for EDI (Electronic Data Interchange).

• It produces large files
• It is very processor intensive to parse
• Needs another new standard
• Or can be used/abused without a standard
• It isn't even very human readable (despite what some say)

And yet... a lot of people find it very compelling.

To understand why, imagine you are talking to a developer who knows nothing about EDI. Show them an EbizXML file, an EDIFACT file, a Tradacom file and a X12 file. All for the same document. The chances are they will immediately be able to tell you the first file is XML as they know what that is. They will probably even be able to begin to translate its contents.

Explain to this developer you want a subroutine / function / method, to extract and return certain elements from the document. They will not blink as they reach to the keyboard and fire up their perfered language tools. They will have a first draft ready in less time than it will take to explain the mere structure of any of the other files.

Any non-developer who has just read this is now scatching their head wondering why file structures would have to be explained. They are also wondering why the problems listed at the beginning are... problems?

EDI System Design - Overview

Part I - Message Control
Part II - Export Mapping
Part III - Import Mapping
Part IV - Transaction Processing

EDI Overview

Part I - Formats
Part II - Communication
Part III - Why

Overview Part I - Formats

There are 4 elements to EDI (Electronic Data Interchange).

1. Composing a message
2. Sending a message
3. Receiving a message
4. Processing a message

Steps 1 & 4 are sometimes refered to as "translating" or "mapping". Steps 2 & 3 are the communication.

By translating you take a document on your computer system, say an order or an invoice, and convert it into another format. Or you take a document that someone has sent you from their computer system, and convert into a document on your system.

Senders Format --> Another Format --> Receivers Format

Why use this third format? Why not translate directly into the receivers format? Well this is quite a lot of work. Think VHS --> Betamax. Think Paralell port --> USB port. If you have lots of receivers all with their own formats then you are going to have to write a lot of different translation routines. The big idea of EDI was, if we could all agree on one standard format, and we could all write our own translation routine into, and out of, the standard and save a lot of work.

[SAP / Salesforce.com / Oracle] --> Standard --> [Oracle/ Salesforce.com / SAP]

Unfortunately everyone couldn't agree. There are lots of standards. Edifact, Ebiz-XML, Tradacom... and many more. Some are tailored to specific industries. Some are more popular in certain countries. In IT invovation is very fast. People are always thinking up new "improvements".

So when your trading partner tells you they will be using the new "DLite buzzword compliant standard", ignore the "standard" word. Don't try and enter into a conversation/debate about the merits of it and others. Just think of it as their format. Oh did I say "Trading Partner"? That is another term you have to get used to. It means customer or supplier. If they are a customer, you will be accepting their "standard". If they are a supplier, they will be accepting your "standard".

Part II, Part III