Using AttachFS To Test Format Descriptions

Format descriptions are one of the handiest features of the OIE workflow engine - no more writing code to read/write all the various files that are received or need to be created; OIE formats can read and write fixed-record length files, delimited files, XLS files, and DIF files. Already built-in to the various format descriptions are various work-arounds for the ways in which each of these files can be mangled. OIE's XLS reading, for example, can deal with spreadsheets where those numbers have actually been occasionally entered as strings, and the delimited file reader can cope with randomly placed escape characters dropped in by applications unaware of what an escape character is. But how do you know, other than trying to run the full process to import the data, if your format description matches the file? Or later on how do you know that the vendor/supplied/department/customer hasn't changed their format without informing you - again without running the process? Answer: Format descriptions can be tested via the AttachFS protocol.

Note: That this usage of formats is intended for debugging use and for use by workflow developers maintaining format descriptions; processing of the test data is synchronous and may not scale to extremely large data sets. If your data is very large test with a subset. For example take the first several thousand lines of a fixed-record length file, etc... This feature is not intended as a production means for translating data; that is the purpose of OIE's readAction and writeAction workflow actions.

The URL to access format description tests is "/attachfs/workflow/format/formatName/verb" where formatName is the name of the format description and verb is the action to be performed using the format: read, write, or readwrite.

• read : Read the uploaded data via the format description producing StandardXML. The Content-Type of the response will be "application/xml" with the filename "standard.xml".
• write : Write the uploaded data, typically StandardXML, via the the format description. The Content-Type of the response will be dependentTesting Format Descriptions
• readwrite : Reads the submitted data via the format description and then writes it out using the same format. This provides a means to test the round trip validity of a format description. The Content-Type of the response will be dependent on the MIME type produced by the format class. The filename of the response data will be “convertedOutput.data”.

If the posted data can be processed by the specified format the response will be HTTP/200 and the payload of the response will be the result of the processing format. The file-name for the result will be included in both the disposition of the response and as the value of the X-OpenGroupware-Filename header. While Etag headers are provided in the format test responses they are nothing more than a time-stamp in order ensure no content from tests is cached by clients or intermediate proxies.

$ curl -vvv -o response.data -u fred -T APNETCOM \
    http://coils.example.com/attachfs/workflow/format/CTABSAPPaymentsReceipt/read

Text 1: PUT the contents of the local file APNETCOM in order to test reading the data via the CTABSAPPaymentReceipt format description

If the processing the posted data via the format fails a response of HTTP/418 “Teapot” will be returned to the client. In this case the payload of the response will be a text/plain stack trace of the exception which interrupted processing. A response of HTTP/500 indicates an error occurred outside the format operation.

A GET request may also be performed for the URL "/attachfs/workflow/format/formatName", where formatName is the name of the format description; this retrieves the raw YAML markup of the format description.