When do Standards become Standards ?

I want to promote good sense when I read it. Adam Bosworth has a very good article on standards. He is not talking about EDI standards, but technical and software standards in general. Yet everything he says I feel is applicable to EDI.

In summary, standards should be simple, readable, focused, precise, implemented, forgiving and free.

Adam's standards background is ODBC, AJAX and XML. These technologies are used so computer can talk to computer, servers and clients can talk to clients and servers, so Electronic Data can be... Interchanged.

Creating an FTP Script

In the last post I discussed using a web FTP service as an alternative EDI service, so I thourght I should also provide some help with FTP scripts. You can use a graphical tool like windows explorer for example by typing ftp://someserver.net in the address box. However to eliminate the work of the human operator as far as possible, scripts are needed.

That said the best first step to achieve this is to make sure you can do everything manually first. So drop to the command line. On a windows system click Start > Run, and type CMD. Now try and replicate something like the session below, the local machine prompts are in red, the ftp server responces are in blue, inputs in black.

C:\Documents and Settings\ACME>
C:\Documents and Settings\ACME>ftp ltoons.com
Connected to ltoons.com.
220 ready.
User: ACME
331 Password required for ACME.
Password:
230 User ACME logged in.

ftp> cd /outbox
250 CWD command successful.
ftp> put acme-bbunny-321.txt acme-bbunny-321.tmp
200 PORT command successful.
150 Opening ASCII mode data connection for put acme-bbunny-321.txt.
226 Transfer complete.
ftp> rename acme-bbunny-321.tmp acme-bbunny-321.txt
350 File exists, ready for destination name.
250 RNTO command successful.

ftp> cd /inbox
250 CWD command successful.
ftp> ls
200 PORT command successful.
150 Opening ASCII mode data connection for file list.
  bbunny-acme-123.txt
  dduck-acme-88.txt
226 Transfer complete.
ftp> get bbunny-acme-123.txt
200 PORT command successful.
150 Opening ASCII mode data connection for bbunny-acme-123.txt (999 bytes).
226 Transfer complete.
ftp> del test.tmp
250 DELE command successful.

ftp> quit
221 Goodbye.
C:\Documents and Settings\ACME>

This follows the procedure laid out in the previous post. Sending a file, listing files to be fetched and fetching them. If you can do this then you are well on your way. If not then any amount of program testing and debugging wont help you.

The next stage is some quick and dirty code just to get the job done. This example is in Python.

from ftplib import FTP

host = 'toons.com'
user = 'acme'
password = 'password'
filestosend =["acme-bbunny-321.txt"]

# Log on

ftp = FTP(host)
ftp.login(user,password)

# send

ftp.cwd('/outbox')
for filename in filestosend:
fileinput = open(filename, 'r')
ftp.storlines('STOR tempory.tmp',fileinput)
ftp.rename('tempory.tmp',filename)

# fetch

ftp.cwd('/inbox')
filestofetch = ftp.nlst()
for filename in filestofetch:
if filename[-4:] != '.tmp':
  fileoutput = open(filename, 'w').write
  ftp.retrlines('RETR ' + filename,fileoutput)
  ftp.delete(filename)

ftp.quit()

Does this work? If it does then well done, but your work isn't finished. This script has no error traps, no try statements. The 4 variable assignments at the beginning need to be parametrised in some way. (Exercise for the reader)

How to use an FTP Host as an Alternative EDI Network

Web site hosting services are very competative now. Easy to use and ever cheaper for both individuals and companies. These servers and systems can also easily provide FTP connections. Dozens of user logins and Giga bytes of storage can be had for an annual cost of less than the purchase price of a desktop computer. With the aid of some relatively simple scripts your computer system can reach out across the internet and use these as a stripped down, bargin basement, alternative EDI (Electronic Data Interchage) network. Here's how...

Create a folder for each partner. Within each folder create 2 sub-folders, inbox & outbox. See below

tom

tom\inbox

tom\outbox

dick

dick\inbox

dick\outbox

harry

harry\inbox

harry\outbox


You could call these sub folders something else, like upload & download, or simply in & out. However using the inbox/outbox terms helps to relate them to their function and models the familiar email arrangement.

Assign each partner a user id & password and set their "home" folder. When they log on they should only be able to access their own folder.

To send you a file, the partner will log on and "put" (in FTP terms) the file in the postbox folder. To receive any files from you they will list ("ls") the mailbox and "get" anything they find then "del".

To send a file to a partner, log on and "put" the file in the partner\mailbox folder for them to collect. remember as host you start from the root folder. To receive any files from your partners, list all outbox folders and "get" anything you find then "del".

This is a simple arrangement, but there are some suttle potential gotchas.

• All files sent and received should be have unique names. If you send a daily file (for example orders.csv) and the receiver doesn't pick it up before the next one is sent, it will get overwritten. To avoid this the sender keeps a counter updated on the generating system and the number is incorparated into the file name e.g. orders123.csv

• If both sender and receiver are logged on at the same time then it is possible to list a file that is only partly transmitted. To avoid receiving part files, the sender should "put" using a tempory name (typically with a .tmp extension), then "rename" it to its correct name. The receiver just egnores any listed .tmp file.

• The FTP protocol has 2 modes. ASCII & Binary. Most EDI files are text files. On Windows/Mac/Linux/Unix the end of a line (EOL) in a text file is marked by different characters. New lines, form feed etc. If sender and receiver have different EOL styles then this can cause problems. This is solved if the FTP is done in ASCII mode (which is usually the default). However for things like excel and jpeg's is would cause data curruption so binary mode must be used.

It doesn't really matter which party is the host. If you are at the client end of the relationship and have to deal with several different hosted sites, the overhead for each additional FTP connection and session is small. Address, username & password. No complicated configuration.

If Tom, Dick, Harry need to exchange files with each other, as well as yourself, then this can also be achieved. In traditional EDI the "to" & "from" envelope data is found by reading the beginning of the message. This method could be used but it would require each file to be moved to your system so it could be read and then imediately retured to the host server in the correct folder. A better way that does not restrict the files to formal EDI formats, is to require the file names to be in the form from-to-count.extn e.g. tom-dick-99.txt or harry-tom-321.xls