I’ve just come back from the UK, where Richard Morris and myself attended DevWeek 2011. The DevWeek conference was hosted at the Barbican center in London, and covered a multitude of various Microsoft / Windows technologies. As it wasn’t hosted by Microsoft, there were no significant goodies/toys given away, but we did gain a lot of great info on developing state-of-the-art user interfaces, and lose a few pounds climbing 4 flights of stairs between sessions.
After the conference, Richard and I visited a couple of customers and had planned to spend a couple of days performing analysis work for a prospective PSG project. However, a few hours into this analysis task, we got a cry for help from another UK customer that Richard had been working with.
This customer was in the process of moving their application from OpenVMS to Windows. They had already moved the executables to Windows and were using xfServer to access the data on OpenVMS. Over the weekend that followed DevWeek, Richard had helped them do a test migration of the data from OpenVMS RMS files to Windows SDBMS isam files, and it all appeared to be working fine. However, during their testing, a few files were showing spurious data in a few fields – hence the cry for help.
It turns out that the bad data was actually caused by integer fields, that everybody had assumed were decimal fields. Now, when you unload data files, most people just use the standard isload/fconvert (or Convert on OpenVMS) to generate a sequential file, which works great for Synergy alpha and decimal fields, but when you have integer (aka binary) fields, regular sequential files no longer “cut the mustard”. One of the issues you have to deal with is this. If you have an integer field that contains the value 13, (aka a Carriage Return character) in a sequential file, the operating system assumes that it’s an end-of-record character and treats the record as truncated. So now the next part of the same record is now treated as a new truncated record, and you have two “bad” records in the file, instead of one “good” one.
The way to resolve this class of issue is to use a “counted” file. This is a special form of sequential file where the length of the record is included with each record, so when re-loading data, the application knows how many characters there are in the record, and it will ignore any “spurious” control characters in the middle of the record. Job done, you would think, but no. Being OpenVMS it wasn’t as simple as that, as we also had to use the OpenVMS Convert utility to change the sequential file into a stream file, so we can preserve the file as a counted file after it was FTP’d to Windows.
With Synergy version 9, the language supports several new data types (e.g. boolean, enum) which are really integer fields, so if you start to store these new data types in your Synergy DBMS files, you need to start using counted files, instead of regular sequential files when loading/unloading data. Fortunately, (in this case) both OpenVMS and Windows are little endian systems, so there was no issue about changing endian type, but this is something that you may need to consider, should you consider moving integer data between systems.
Now, to come back to the topic I started with of “attending conferences” (and to issue a shamless plug), if you come to the 2011 SPC in Chicago or Oxford, you’ll be able to hear all about managing binary data in Synergy DBMS files.
Hope to see you there.