Monday 3 December 2007

EDI System Design Part III - Import Mapping

OK, I know I've missed something out. Between EDI (Electronic Data Interchange) Exporting and EDI Importing is Communicating. But to understand that, it's best to first have an understanding of both the inputs and outputs. Besides, it's my Blog.
What are we going to do with these EDI messages? Possibilities include,

  • Print them out in human readable form
  • Covert them into an email alert
  • Store them for querying and reporting
  • Store them for transaction processing

The first option of printing the message out is surprisingly common. I hope you can spot what is wrong with this approach. You see this sort of configuration has a technical name. It is called a Facsimile machine (or a Fax for short). Joking apart, it would be a shame to have spent a large effort of time and resources to discover we have re-invented the Fax machine (and an expensive one at that).

Now it should be obvious what is wrong with the second approach. Our trading partner can send email alerts and cut out the "EDI" system.
Querying and reporting is fine for some types of data. For example if a customer is updating us on EPOS sales data from their branches. We would probably want to run various analysis reports, maybe comparing against forecast, or projecting future stock replenishment orders. This means getting the information into our database. For most this means SQL compatible tables.

If the message is transactional in nature, and we want to action a transaction on our ERP system, then there are also sound reasons for storing the message in our database first. I don't want to go into too much detail (see Part IV), but one reason is you may have transactions sourced from many different EDI message types/standards, and our database table is a good linga franca to pass to the transaction routine.

Another reason is, we should be phasing in EDI software functionality with little updates often, rather than a long wait, followed by a big bang implementation. We can implement a "glorified fax machine", running reports from our database tables, first. Then move to a more functioning system later. I know I ridiculed this arrangement early on, but the point is this should be a transitional stage. We don't want to leave things like this for long.

So our EDI Import Mapping routine will take each message and loop through each segment updating our object and maybe a list of object details. When we reach a message header marker we write the object to the database. Different message types of the same encoding standard will update different object tables but it is worth all these routines updating an "Envelope" table with message meta data, and putting the id field on the object table. This will enable us at a later date to translate various partner identity codes (products, branches etc.) through a look up with a sender identity.

Once these updates have been done we can delete the EDI message from the "pending" folder and move it to an "archive" folder. There will be times when you get unexpected output (or no output) and we will want to check what our trading partner sent in the raw.