By Steve Ives, Posted on August 26, 2019 at 9:58 pm
For some time CodeGen has shipped with a utility called CreateFile that (given an appropriately configured repository) could be used to create an ISAM file based on a repository FILE or STRUCTURE definition.
A customer recently asked me to create a mechanism that they could use to create multiple files, load data into those files, and also support creating both RELATIVE and ISAM files. Rather than writing something completely new I decided to extend the CreateFile utility, and that new feature has now been released along with the latest version of CodeGen (5.4.2).
Briefly, there is a new command line option -in that allows you to specify an input file containing the details of one or more files to be created and and populated. The file uses a JSON format. If you’re interested in finding out more you can find additional information in the CodeGen documentation.
It’s been a while since a GodeGen release was announced on this BLOG, but rest assured they have been coming thick and fast; in fact there have been 9 interim releases since we last announced here.
So why all the frenetic activity in CodeGen land? Well the primary driving force for so many releases and so many new features is to provide support for the ever evolving Harmony Core RESTful Web Services framework. If you’re not familiar with Harmony Core then you should check it out. More and more Synergy developers are starting to take advantage of it as a way to expose their applications data and business logic in whole new ways and to support a myriad of use cases.
So what’s new in CodeGen 5.4.0? Well the answer to that question is … a LOT! I won’t list all the new features here but if you want to check out the specifics then you can head over to the CodeGen Releases page where you’ll find a detailed change log for this and previous versions. And if you need more information than that then check out the CodeGen Documentation.
We recommend that all developers who are using CodeGen upgrade to this latest version, but particularly if you’re developing with Harmony Core.
By Steve Ives, Posted on September 23, 2018 at 4:43 pm
Just a quick note to announce that CodeGen 5.3.6 has been released and is available for immediate use. This latest version represents the culmination of a series of recent releases which together have added significant new features across the entire product, including the ability to generate code based on metadata found in the xfServerPlus Synergy Method Catalog.
Many of the new features were added specifically to support code generation for the new Harmony Core RESTful web services framework that we are excited to be introducing at the upcoming DevPartner Conference in New Orleans next month.
If you are joining us for the post-conference Harmony Core workshop then you will need to have this new version of CodeGen installed on your development system; you can download it here. And even if you’re not intending to use the new Harmony Core framework, there are many new features that may be useful to all CodeGen users; we recommend this release for general use.
We’re looking forward to seeing you all in New Orleans between October 8th and 12th. If you haven’t signed up yet, it’s not too late! You can still sign up here.
The Synergex Resource Center Community site has a number of great features including the Answers and Ideas portals. Ideas is the place to post your ideas for improving Synergy and related products. You can vote for ideas and provide comment and feedback to help the whole community be more successful. The Synergex community site had an idea posted recently:
The Synergy DBMS Manager allows you to query and manage data in your Synergy data files using Symphony Data Objects.
Code generating Symphony Data Objects for your files is a simple process. Ensure you have your Synergy Repository configured. You need to define the structure and associated file definition for each file you wish to be able to manage using the utility. There are full instructions at the Symphony Framework page which walk you through the few simple steps to building your data object assembly in Synergy .Net. Not sure how to build Synergy .NET assemblies – then send me your repository and I’ll do it for you!
To query data you simply issue the required select command and define the response data object;
Simple, and you can scroll through the results. The idea posted was to be able to then export the selected data to Microsoft Excel so that further review and analysis can be performed.
A new toolbar button has been added and the GemBox Spread assembly used to easily create the Excel document. Now you can export all the selected rows to an Excel document:
Whenever I work with customers code there is almost always a need to “run” their applications. That means I need data. To ensure that Synergex protects our customers data and that we conform to the various data security requirements of todays world there is usually a need to cleanse the data before I get access to it. This cleansing process is like redacting words or phrases in a document – to prevent the consumer (in this case me) from seeing the real data.
With Synergy today there are basically two ways to do this; write a program to clear or set the data fields within the records in the file to a specific value or configure Synergy xfODBC and use a database management tool. If you don’t want to write code, or license and configure xfODBC you can use the Synergy DBMS Manager.
The Synergy DBMS Manager is a simple utility that can be used to redact data in your Synergy DBMS files. Your can download and install the Synergy DBMS Manager utility by visiting the downloads page. The utility uses Symphony Data Objects to describe the data in your files – these are easy to code generate.
Code generating Symphony Data Objects for your files is a simple process. Ensure you have your Synergy Repository configured. You need to define the structure and associated file definition for each file you wish to be able to manage using the utility. There are full instructions which walk you through the few simple steps to building your data object assembly in Synergy .Net. Not sure how to build Synergy .Net assemblies – then send me your repository and I’ll do it for you!
Synergy DBMS Manager
Once you have the data object library built, you simple run the utility:
You’ll need to locate the “Data object assembly” that you have just built, and following that select the “Table mapping method”. Enter the password and you’ll be in the utility and ready to manage your Synergy DBMS data.
You can perform simple queries to locate and review the data in a file;
Now you can easily redact the data in the file – remember you are affecting the ACTUAL data in the file, so make sure this is a copy of the data file and NOT your LIVE data, there is no rollback functionality!
Simply issue an update command;
Notice the result count () at the bottom of the screen. Because we didn’t specify a where clause then all records in the file were affected by the requested update. Now you can perform a simple query and see the results;
This data is now cleaned. If you need to send data to Synergex Support to assist then to resolve an issue you have, this is a great way to redact and protect that data before sending it.
By Steve Ives, Posted on December 1, 2017 at 11:18 pm
We are pleased to announce the release of CodeGen version 5.2.3. The new release includes new features, addresses some issues found with previous releases, and also paves the way for once again being able to use CodeGen on non-Windows platforms through experimental support for the .NET Core environment.
As always, you can download the latest version of CodeGen from here.
CodeGen Version 5.2.3 Release Notes
Added a new experimental utility to the distribution. The Code Converter utility can be used to automate bulk searches within and bulk edits to an applications code. This utility is in a usable form. However, it is still a work in progress and is likely to undergo substantial changes as it evolves.
Added two new utility routines (IsDate.dbl and IsTime.dbl) that are referenced by some of the supplied sample template files.
Corrected a regression that was introduced in the previous release which caused the field loop expansion token <FIELD_SQL_ALTNAME> not to default to using the actual field name if no alternate name was present.
Performed an extensive code review and cleanup, updating the code in several areas to take advantage of new features available in the Synergy compiler, and also improving efficiency.
Fixed an issue that was causing the CreateFile utility -r (replace file) option to fail, an existing file would not be replaced even if the -r option was specified.
Fixed an issue in the CreateFile utility that would result in an unhanded exception in the event that invalid key information was passed to XCALL ISAMC.
Made some minor code changes to allow CodeGen to be built in a .NET Core environment and we hope to be able to leverage .NET Core to once again support the use of CodeGen on non-Windows systems (starting with Linux) in the near future.
This version of CodeGen was built with Synergy/DE 10.3.3d and requires a minimum Synergy version of 10.1.1 to operate.
Symphony Framework Components
We no longer ship the Symphony Framework sample templates with CodeGen. You can obtain the latest Symphony Framework templates from the Symphony Framework web site (http://www.symphonyframework.net).
There were no Symphony Orchestrator changes in this release.
There were no Symphony Framework CodeGen Extensions changes in this release.
By Steve Ives, Posted on October 25, 2017 at 3:00 pm
We are delighted to announce the availability of the CodeGen version 5.2.2 release that includes the following enhancements and changes:
CodeGen Version 5.2.2 Release Notes
Added a new field loop expansion token <FIELD_FORMATSTRING> which can be used to access a fields format string value.
Added a new command-line option -utpp which instructs CodeGen to treat user-defined tokens as pre-processor tokens. This means that user-defined tokens are expanded much earlier during the initial tokenization phase, which in turn means that other expansion tokens may be embedded within the values of user-defined tokens.
Removed the RpsBrowser utility from the distribution; it was an experimental project that was never completed.
This version of CodeGen was built with Synergy/DE 10.3.3d and requires a minimum Synergy version of 10.1.1 in order to operate.
By Steve Ives, Posted on August 24, 2017 at 2:10 pm
We are pleased to announce the release of CodeGen 5.2.1 which includes several new features that significantly extend the possibilities for what can be generated. The core goal for this release was to introduce new features that make it possible to sub-divide the processing of large loops into smaller units of code, and by doing so make it possible to achieve new things, especially when working with very large structures.
For example, the reference code that demonstrates how to implement the replication of ISAM data to a relational database (https://github.com/SteveIves/SqlReplicationIoHooks) previously had a restriction that it could only deal with files (tables) with up to 252 fields (columns). The underlying reason for this related to the maximum number of arguments that can be passed to an external routine, %SSC_BIND in this case. The restriction of 252 available data parameters to %SSC_BIND meant that bad code would be produced for any structure with more than that number of fields. Now however, using some of the new features in CodeGen 5.2.1, the template has been re-structured so that %SSC_BIND will be called multiple times if necessary, removing the previous restriction.
But that’s just one example, there are doubtless many more. Here are the release notes for the new version:
We added several a new generic expression token <IF COUNTER_n_op_value> that allows you to write conditional template code based on testing the current value of the two internal template counters against a simple numeric expression. Here is an example of using this new expression:
Code to include for every 10th item
We added a new file header token <REQUIRES_CODEGEN_VERSION> that allows you to specify the minimum version of CodeGen that is required to successfully process a template. Here is an example of using one this new expression:
We made a minor correction to the documentation of the -f l command line option. The documentation previously stated that this option caused fields marked as “Excluded by Language” to be EXCLUDED from field loop processing, but actually such fields are excluded by default. The -f l command line option actually suppresses this behavior, causing such fields to be INCLUDED in field loop processing.
This version of CodeGen was built with Synergy/DE 10.3.3c and requires a minimum Synergy run-time version of 10.1.1.
The latest version of CodeGen can always be downloaded here.
This year’s DevPartner conference was held in Atlanta, Georgia and was a blast. I guess I say “the best ever”” each year, so I won’t this time – but it was. The whole conference ran flawlessly, although throughout the week I did see a few strange things in the corner of my eye – but you know how it is. You think you are seeing things… But maybe not.
We had lots of new faces in the audience seeing what Synergy has to offer which is always great, and they got to see a few new faces presenting at the DevPartner conference for the first time.
Although Phil could be considered a veteran now – his third conference – he was joined on stage by first-timer Tate, a fellow support engineer. They presented a cool session on BOTS – not those things you sit on, the automated robots that answer whatever questions you have. There are loads of bots around and the duo showed us what it’s all about and how to build you own!
Another new face was Jacklin who presented all the ways to get fully up to speed with Synergy today including all the various media feeds and the cool YouTube channel we have that’s being loaded with great training videos – and of course you can always re-watch all the conference sessions.
At the beginning of March I blogged about “Physio My Way” which is an app to monitor a patients compliance to perform their prescribed upper limb exercises. Ashley presented the full Physio My Way story and received great feedback – if you get the time check out the Synergex YouTube channel in the coming days to see the full video! It’s a complete mobile solution written end to end in Synergy.
And so to my post-conference workshop. The goal was to write, from scratch, a complete application called “On Vinyl”. “On Vinyl” is a vinyl collection management and inquiry system. The system is written entirely in Synergy .Net and uses the Symphony Framework.
We started off by code generating the base data objects – the classes that expose the Synergy Repository structures as classes with full get/set properties for the fields. We also code generated classes to provide complete data management using Symphony Harmony.
Next we built a simple maintenance program to allow us to manage our collection of vinyl albums. To save typing we also implemented the ability to perform a Google search passing in the album barcode and returning/displaying all the album information, including the cover artwork. I’ll blog about just how easy it was to implement Google searching soon.
Next we hosted all the server logic on a remote cloud server behind Symphony Bridge and showed how to manage the data remotely using the Synergy DBMS Manager (available on the download page at www.symphonyframework.net).
Next we moved to the client portion of the project – to build and deploy an app on the Android and iOS platforms to enable us to check if we have a particular vinyl in our collection. We built up the technology stack. First layer was the client data layer – code generated data objects built into a portable class library. Next we created the connection library which used commands to search for vinyl details using the barcode:
If we didn’t have the album in our collection we could call our stored procedure to perform the required Google search:
And if we chose to buy the album we could insert it into our collection directly from the device:
Our final portable library was the common UI code – using Xamarin Forms. This provides the ability to write the UI once and deploy to different target devices such as Android and iOS.
Up to this point, although there were a couple of “what was that…” moment’s things all built and ran just fine.
The final piece of the puzzle was to write the device specific code – and our first target was Android. We created the project, added the required resource (images, etc.) and after a couple of minor typo’s the project built. But something was not quite right and we could not get the program to successful deploy and run.
We’d hit the end of the day so aborted the attempt and regrouped to try to find out what had gone wrong. Once we have the solution I’ll post the entire project to GitHub and blog about it!
Now I don’t really believe in code gremlins, ghosts or phantoms, but throughout the week there were some strange goings-on, and to be honest I was having flash-backs to conference time 2009! Surely he was not back to try to mess up my workshop again?
I am pleased to announce that we have just released a new version of CodeGen (5.1.9) that contains some new features that were requested by customers. The changes in this release are as follows:
We added two new structure expansion tokens <FILE_ODBC_NAME> and <FILE_RPS_NAME> that expands to the repository ODBC table name or structure name of the first file definition that is assigned to the structure being processed.
We made a slight change to the way that the multiple structures command line option (-ms) is processed, allowing it to be used when only one repository structure is specified. This allows for templates that use the <STRUCTURE_LOOP> construct to be used when only one structure is being processed.
We also fixed an issue that was causing the <FIELD_SPEC> token to produce incorrect values for auto-sequence and auto-timestamp fields. Previously the value 8 would be inserted, now the correct value i8 is inserted.
This version of CodeGen was built with Synergy/DE 10.3.3c and requires a minimum Synergy runtime version of 10.1.1. You can download the new version directly from the CodeGen Github Repository.
Working with the many customers we have is a great opportunity to advance the software we write. Some time ago I blogged about Symphony Bridge – a server based utility that allows you to expose Synergy data and logic through Symphony Harmony using an SQL-like syntax. In essence you can write queries on the client like “select * from part” and the bridge server will return a collection of part objects that are based on a structure you have created in your repository. You can then bind these to grids or input controls on you windows or mobile applications or display them in web pages. In fact you can use the data in any Synergy .Net or non-Synergy application.
Another aspect of Symphony Harmony/Symphony Bridge is the ability to call a remote procedure using the syntax “exec @MyNameSpace.MyClass.MyMethod”. You can pass in arguments and again a data object or a collection of data objects are returned.
During a recent visit with one of our customers who is currently utilizing symphony Harmony and Symphony Bridge to provide third-party access to their systems to allow enquiries on orders, etc. I was asked about “providing the ability to return a standard object that contains the call status AND the response data”. Simple, no problem: Create an RPS structure that defines your response details – call status, error codes, and error message for example. Code generate the required data object and then customize it by exposing a collection of <DataObjectBase> (the base class of all data objects!) As I said – simple.
But the customer was not overly impressed – and they had an idea! Oh dear me thinks – these are seasoned C# developers and I’m not convinced I’ll be able to implement their ideas in Synergy .Net! But how wrong I was…. and without any changes to the Symphony Framework. The basic requirement is to have a “response” class that they can extend with any serializable “type” – be that a single instance or a collection of “type” – without any restrictions.
The starting point was the same – create the required response structure in the repository and code-generate the “response” class. This is the resulting class definition;
public partial class Server_response_Data extends Symphony.Conductor.Model.DataObjectBase
The code generation process creates all the required properties and the logic to serialize the data into JSON or BSON.
The next step was to create a second class with the same name as the response class but this time it will accept a generic – generics are quite cool and allow you to defer the specification of a type until that type is instantiated. So our generic response class definition looks like this;
public class Server_response_Data<T(new)> extends Server_response_Data
By extending the base code-generated class we get all the required Symphony capabilities. The “T” denotes the generic “type”. The (new) ensures that any type you provide must have a parameter-less constructor which ensures that this class can creates an instance of it. We can expose the property so it’s serializable;
public property Result, T
And in the constructor we can ensure there is an instance (to prevent null object exceptions);
public method Server_response_Data
Result = new T()
And that is about it. You can then write your server methods to create an instance of your response class, populate the generic “response.Result” with the required “type” and set the response properties accordingly. For example;
data response = new Server_response_Data<Part_Data>()
Will create a response class with the “Result” as a type of Part_Data data object. If you need to return a collection of Part_Data objects;
data response = new Server_response_Data<List<Part_Data>>()
It’s really as easy as that. On the client you simply make the remote EXEC call through Symphony Harmony;
result = DataExecute.RunDataExecute(connector,
& "exec @MyNameSpace.MyClass.MyMethod",
& new Server_response_Data<List<Part_Data>>()).Result
You can also await the call on a separate thread.
Big shout-out to Gareth for the idea and his persistence to make me get it working!
I’ll be using these techniques in my DevPartner 2017 post conference workshop as we build a mobile app from start to finish using Symphony Harmony/Symphony Bridge to access remote data and logic.
By Steve Ives, Posted on February 7, 2017 at 10:25 am
We are pleased to announce that Professional Services has just released CodeGen 5.1.7. The main feature of the release is the addition of experimental support for generating code for the MySQL and PostgreSQL relational databases. Developers can use a new command line option -database to specify their database of choice. This causes the SQL-compatible data types that are injected by the field loop expansion token <FIELD_SQLTYPE> to be customized based on the chosen database. The default database continues to be Microsoft SQL Server.
Before we consider support for these new databases to be final we would appreciate any feedback from developers working with MySQL or PostgreSQL to confirm whether we have chosen appropriate data type mappings. Additional information can be found in the CodeGen documentation.
By Steve Ives, Posted on July 29, 2016 at 10:42 am
We are pleased to announce that we have just released CodeGen V5.1.4. The main change in this version is an alteration to the way that CodeGen maps Synergy time fields, i.e. TM4 (HHMM) and TM6 (HHMMSS) fields, to corresponding SQL data types via the <FIELD_SQLTYPE> field loop expansion token. Previously these fields would be mapped to DECIMAL(4) and DECIMAL(6) fields, resulting in time fields being exposed as simple integer values in an underlying database. With this change it is now possible to correctly export time data to relational databases.
We also made a small change to the CodeGen installation so that the changes that it makes to the system PATH environment variable occur immediately after the installation completes, meaning that it is no longer necessary to reboot the system after installing CodeGen on a system for the first time.
This version of CodeGen is built with Synergy/DE 10.3.3a and requires a minimum Synergy runtime version of 10.1.1.
For some time now we have published various examples of how to replicate ISAM data to a relational database such as SQL Server in near to real time. Until now however, all of these examples have required that the ISAM files that were to be replicated needed be modified by the addition of a new “replication key” field and the addition of a corresponding key in the file. Generally this new field and key would be populated with a timestamp value that was unique to each record in the file. While this technique guarantees that every ISAM file can be replicated, it also made it hard work to do so because the record layout and key configuration of each ISAM file needed to be changed.
However, almost all ISAM files already have at least one unique key, and when that is the case one of those existing those keys could be used to achieve replication without requiring changes to the original record layouts or files. When this technique is combined with the capabilities of I/O hooks it is now possible to achieve data replication with only minimal effort, often with no changes to the ISAM files being replicated, and with only minimal modification of the original application code.
I am pleased to announce that I have just published a new example of doing exactly that. You can find the example code on GitHub at https://github.com/SteveIves/SqlReplicationIoHooks. Of course if you are interested in implementing data replication to a relational database but need some assistance in doing so, then we’re here to help; just contact your Synergex account manager for further information.
The Symphony Harmony namespace allows access to data and logic through an SQL like syntax. For example you can select records from a file using a query such as “SELECT ID, DESCRIPTION FROM PART WHERE QUANTITY > 12 ORDER BY QUANTITY”. All matching records are returned from the query in the form of Symphony Data Objects. The data can be local or accessed via Synergy xfServer. The Symphony Bridge utility allows you to expose your query-able database via a standard Windows Communication Foundation (WCF) web service. So far so good.
Steve Ives and I recently had the opportunity to spend a week working together in the UK to “bang” heads together. Steve has always been an exponent of providing RESTful services to access logic and data which can be consumed by just about anything. So we set about using CodeGen to build a standard Restful service that will utilize the Symphony Framework to enable dynamic access to data and ultimately logic.
We soon had the basic service up and running. Out first implementation handled the standard GET verb – and returned all the records in the file. No filtering, no selection, just all the records retuned as a JSON collection. This is the standard API;
Now remember that Symphony Harmony allows you to filter the data you are requesting, so we next implemented the ability to provide the “where” clause to the query. So for example;
And using ARC (Advanced Rest Client which is a Google Chrome plug-in) we can test and query the service;
And we get back just the selected customer details – all those customers where CUSTST has value of CA.
As well as being able to filter the data we can also limit the results returned by Harmony to just the fields we need; this has the benefit of reducing the data being brought across the wire. But how can our REST server build the required data objects to just include the fields we select? By doing runtime code generation! Within our code generated data objects we added the ability to dynamically build the response data object to only include those fields requested by the client. The calling syntax, as provided by the API, is;
And again using ARC to test our server we can issue a command like;
This is requesting all records from CUSMAS where the CUSNM2 field contains the word “LAWN” and limiting the response data object to just three fields. The response JSON looks like;
Two perfectly formed data object that are limited by the fields in the selection list. If your Symphony Harmony connection to your data is via xfServer then only those selected fields will have been loaded from the file, again improving performance.
We also added the ability to limit the amount of data retuned by adding a “maxrows” option;
We have already added the ability to the Symphony Harmony namespace to perform inserts, updates and deletes using standard SQL syntax and we’ll be adding these capabilities to the appropriate rest verbs POST, PUT and DELETE. Watch this blog feed for more information.