Jeff and Steve give a short update on the Harmony Core project, including what will be changing with Synergy/DE 11. They also introduce the xfBBQ project. The team then reviews an issue and answers several questions on the project.
If you’ve ever been coached or trained in public speaking, you’ve probably been made painfully aware of your use of the dreaded filler words—words you say almost automatically when speaking and nervous. Everyone has their own flavor of it, but for me, it’s the word “like.” If I’m really nervous, I can end up sounding like Moon Unit Zappa in her single Valley Girl. (Millennials, ask your parents. Gen Z, ask your grandparents.)
But the word “like” gets a bad rap and in some uses is fine and perfectly acceptable, like making a comparison or at the beginning of a list talking about the uses of like. Today, though, I’m talking about like in the context of new features added to Synergy/DE in version 11. Which introduces the new system-supplied class Select.Where.Like.
If you aren’t familiar with the Select class in Synergy, where have you been? We’ve been harping on about this thing, like, forever. It’s a wonderful feature in the language for reading data in a SQL-like way. Well, the Select class now has even more functionality with the addition of the Like statement, allowing you to filter your alpha results based on a matching pattern. It’s similar to Where.Contains in that you can do the equivalent of Where.Contains with Where.Like, plus so much more. A Where.Contains is a Where.Like that has a “%” sign in the front and back, telling it to search the results for that string of characters anywhere in the field.
How does Select.Where.Like work?
|Zero or more characters||%||bl% finds bl, black, blue, and blob|
|Single character||_||h_t finds hot, hat, and hit|
|Single character in a set||||h[oa]t finds hot and hat but not hit|
|Single character not in a set||[^]||h[^oa]t finds hit but not hot and hat|
|Single character in a range||[-]||c[a-c]t finds cat, cbt, and cc but not cot|
Using the above wildcards, you can build your Where statement to look for a pattern inside a record field. For example, say you have a city address in the form CityName, ST(ate). Now let’s say you want all results from entries from CA where the city starts with “SAN”. What would this look like?
Breaking this down, the first three characters are “San”, then there are one or more characters between “San” and our next part “,CA”, which can have one or more characters after it. This query should return results like San Francisco, San Diego, Santa Monica, and San Jose.
Note that you can do a Where.Like that is not case sensitive with NoCaseWhere.Like. The above statement wouldn’t find a city if “san” was in the body of the city name, like Pleasanton, CA. This is most helpful if your data is not uniform or consistent (like sometimes the state is ca or cA). To get the most use out of Where.Like, you should be familiar with your dataset and how information is entered and modified.
Getting an idea how this works
Let’s try something a little tougher. Let’s say you have data that contains postal codes, but the postal codes can be from different countries. Now you could combine your Where.Like with an AND to a field that has a country, but you could also handle it all within the Where.Like. For instance, let’s say the data has postal codes from three different countries: United States, United Kingdom, and Sweden. An example of each one would be
US: 99999 Sweden: 999 999 UK: varies but most commonly AA9 9AA
where 9 is a number and A is a letter.
If we wanted to be completely explicit, we could use these Where.Like statements:
Sweden: [0-9][0-9][0-9] [0-9][0-9][0-9]
UK: [A-Z][A-Z][0-9] [0-9][A-Z][A-Z]
This is a manual way to handle it, but maybe we can do it with less. If we want only US and Sweden, we can use the fact that UK postal codes have letters to our advantage and go with a simple
US & Sweden: [^A-Z]%
This essentially equates to any postal code that doesn’t begin with a letter, which works for our simplified version of these countries’ postal codes. The opposite, [A-Z]%, also works if you want UK only.
If we want to do Sweden and the UK, we note that both contain a space in their postal codes, so
UK & Sweden: %[ ]%
This includes any postal code with a space in it. Now this is tricky, as these could be Synergy alpha types, and there might be trailing spaces at the end. To fix this scenario, we use a wildcard to say there is a number or letter after the space:
UK & Sweden: %[ ][0-9]%
This means that we are looking for a space with a number immediately following. However, unlike the last example, we can’t use the opposite of this to get US only, because %[^ ][^0-9]% actually works for all three countries if there are trailing spaces; a letter or number at the end followed by a non-numeric character works. For example, a UK postal code ending in “G” with trailing spaces satisfies this condition.
What’s unique about US postal codes, though, is that there are a series of four numbers in a row, and they have fewer total characters than the other two. Therefore, %[0-9][0-9][0-9][0-9]% would work, as would _ _ _ _ _ % (i.e., five wildcard spaces followed by a literal “ ”), as there would be a space afterwards if this were an alpha field, whereas it would be a numeric or alpha character in the UK or Swedish postal codes. Of course, this would be quite different in practice, because your searches would look more like “98_ [0-9]%” if you were looking for Swedish postal codes starting with 98. But the purpose here is to get you thinking with wildcards. This new feature has a lot of potential to optimize searches as it does when used inside of SQL. Be sure to try it out now in Synergy 11 and read more about this feature in our online documentation.
Synergex recently released REV11 licensing, a new revision to our licensing system on Windows and Unix. We’re excited to roll it out to you as it’s going to greatly simplify your management of Synergy product keys. REV11 licensing is included with Synergy/DE 11, and it also supports Synergy/DE 9.3 – 10.3 via the REV11 licensing upgrade package. I.e., you don’t have to upgrade to Synergy/DE 11 to use REV11 licensing. We also announced that REV11 licensing is required for Synergy/DE DevPartner (development) license renewals. So before your next DevPartner subscription renewal, download and install the licensing upgrade package on the license server(s) for your development systems. After your renewal, any new DevPartner licenses you add will use REV11 licensing.
After you install the REV11 licensing upgrade package, License Manager on your server will automatically send your licensee name and current registration string to the Synergy License Web Service, where the information will be used to locate the license for your system. Product keys are then generated, and License Manager will automatically download and install them.
Currently REV11 licensing is only required on your development systems. It is recommended but not required for deployment (production) systems. To learn more about REV11 licensing, view our REV11 licensing web page, our documentation, or watch the video below. If you have any questions or would like assistance with REV11 licensing, please contact Synergy/DE Developer Support at firstname.lastname@example.org, your account executive at email@example.com, or either team at (916) 635-7300.
Some time ago a customer posted in the Synergy/DE Ideas forum suggesting that a Synergy DBL language plug-in should be created for the increasingly popular Visual Studio Code editor. Over course of the next few months the post received a significant number of votes, and we also started hearing the same request from others.
Later we heard from the original customer that they had started to define some basic language integration themselves, offered to share that initial code, and agreed that it could be open sourced.
At this time we are pleased to announce that initial support for DBL in VS Code is live! We have developed this functionality as an open source project on GitHub (https://github.com/Synergex/vscode-synergydbl) and welcome your feedback and your contributions.
If you just want to try the extension you can search for “Synergy” in the extension marketplace inside of VS Code, or download and install the VSIX from the GitHub Releases page. If you do try the extension out, and have questions or problems, please use the GitHub Issues page to let us know.
The initial scope of the extension is fairly limited, but we have been using the colorization locally for a while now and it makes a huge difference when you just need to do a quick edit to a source file. We are looking forward to seeing what everyone else has to contribute.
We’re very pleased to announce the release of Synergy/DE 11. Version 11 delivers many features, including important security updates, exciting ISAM resilience features, and runtime performance improvements. It also includes new Synergy DBL and Visual Studio integration features. See our Synergy/DE 11 page for a full list of features.
Synergy/DE 11 also includes our new REV11 licensing, which automates your product key updates. REV11 licensing is used with Synergy/DE 11, and it also supports Synergy/DE 9.3 – 10.3 with a licensing upgrade package. See our REV11 licensing page for more information.
Many of the new features in Synergy/DE 11 were suggested on our Ideas site. We would like to thank everyone for their input, and we encourage you to continue to post, comment, and vote on that site to let us know what features are most important to you.
Be sure to review the new requirements before you upgrade to Synergy/DE 11. For example, version 11 doesn’t support Windows 7, Windows Server 2008 R2, or Visual Studio 2015.
We strongly encourage you to upgrade to Synergy/DE 11 for enhanced application security, resilience, and performance; a better developer experience; and a much easier process for managing product keys. Take a look, and let us know if you have any questions or if you need any assistance.
I am pleased to announce that another new version of CodeGen has been released, is available for immediate use, and contains several powerful new features:
- We added a new structure expression token <IF STRUCTURE_PII_FIELDS> that indicates whether any field in a structure has been tagged as containing PII (Personally Identifiable Information). We also added a new field loop expression token <IF PII_SCRUB> that indicates whether a field has been tagged as containing PII data, and a new field loop expansion token <FIELD_PII_TYPE> which inserts the type of PII data a field contains.
- We added a new field loop expression token <IF USED_IN_RELATION> that can be used to determine whether the field currently being processed is involved in any relations to other structures.
- We added a new custom field loop expression token <IF HARMONYCORE_CUSTOM_FIELD> and a new custom field loop expansion token <HARMONYCORE_CUSTOM_FIELD_TYPE> to the Harmony Core extensions assembly. In a future release Harmony Core will use these new features to allow developers to easily implement support for custom field types that may be implemented within their applications data set.
- We added a new tag loop expansion token <TAGLOOP_FIELD_SNTYPE> and relation key segment loop expansion token <LITERAL_SEGMENT_SNTYPE>.
Additionally, because the Symphony Framework is no longer maintained by Synergex, and is no longer an open source development effort, we have decided to discontinue the distribution of the “Symphony Framework CodeGen Extensions” assembly that was formerly distributed along with CodeGen. We continue to support the addition of custom extensions and the Symphony Framework extensions can be added back at any time by any Synergy developer with the necessary Synergy .NET development tools.
This version of CodeGen was built with Synergy/DE 11.1.1 and requires a minimum of version 10.1.1 to operate.
Jeff and Steve give an update on the Harmony Core project. Jeff also validates resolving issues that have been raised in GitHub. Lastly, Jeff and Steve discuss adding documentation to let users know how to enable STDOUT logging inside IIS.
The Harmony Core project has been steadily gaining momentum as we improve our processes, work on support issues, and continue with user engagements.
To start with, we’ve been working on our continuous integration and continuous delivery (CI/CD) pipeline in Azure DevOps. This pipeline serves a dual purpose. It allows us to demonstrate software development best practices applicable to Harmony Core users, and it enables our team to be more productive. Whenever code is committed or a pull request is created, the full test suite is run to ensure that the code is good and that there are no regressions. Automating this has improved our ability to take pull requests from outside contributors.
Significant effort has gone into offering a nearly seamless experience for xfServerPlus users who are looking to expose their Synergy routines via RESTful web services. We have updated CodeGen, our CodeGen templates, and the underlying support libraries. CodeGen can now read SMC interface definitions, and there are new tokens for iterating over xfServerPlus methods and parameters. Additionally, CodeGen now uses SMC interface definitions as input when determining which repository structures to iterate over. We use this functionality to generate the data objects required to wrap structures passed as parameters.
As part of a consulting engagement, we built a mechanism to support custom loading and custom validation of data objects and transactions (Issue 107). We enhanced CodeGen and the CodeGen templates in Harmony Core to support certain types of relations to ensure that required relations have the necessary resources in the database or the current transaction.
Traditional Bridge Improvements
Up to this point, we’ve been exclusively promoting RESTful web services, but now we’re introducing an alternative. We’ve created CodeGen templates for exposing Traditional Bridge services via SignalR, a technology provided by Microsoft. SignalR allows for two-way communication in or outside of the browser without compromising performance, reach, or standards. We’re recommending SignalR in cases where an application can’t operate statelessly or the server needs to push updates to clients rather than requiring them poll or refresh for changes.
On the xfServerPlus front, we implemented encoding in Traditional Bridge for all currently known scenarios and data types that are supported for xfServerPlus. This includes arrays, handles, ArrayLists, and scalar types. We also implemented a secondary calling convention that uses the Synergy routine call block API for those of you that use the optional parameter support offered only in xfNetLink COM clients. And we’re actively working on porting the full test suite used by xfServerPlus and xfNetLink.NET to work as unit tests for Traditional Bridge. Swagger Docs
Over the last few weeks, we’ve been working on improving our API version support. We found an open-source Microsoft library for this, and we have created integration samples. This library does two things for us: it takes care of versioning, and it produces Swagger docs for versions, enabling us to move away from statically generated Swagger files. This dramatically improves the quality of Harmony Core Swagger documentation for items that fall outside the normal set of GET and POST entities. And it has the added benefit of allowing you to expose source code doc comments as part of your API documentation.
Traditional Runtime Updates
The traditional Synergy runtime for version 11 includes support for reading and writing JSON using a subset of the new System.Text.Json namespace from .NET Core 3.0. We have decided to use this functionality because it is designed for speed. Even for moderately-sized JSON documents, we’re seeing upwards of 15x faster parsing, and the number gets larger for larger documents. In the coming weeks, we’ll be implementing a dual pathway for JSON reading and writing in Traditional Bridge. If you’re on a runtime older than 11, we’ll fall back to the implementation written in Synergy DBL. But if you’re using the latest runtime, you’ll be able to take advantage of the massive performance increase.
A New Harmony Core Example
We’ve received some requests for a complete end-to-end Harmony Core example that demonstrates how to develop, test, and deploy a web application backed by a Harmony Core web service. We’ve come up with an idea for an application we’re calling xfBBQ. At our offices during the summer, the development team frequently hosts barbeques for the company, and one of the challenges we face is taking orders. This is currently managed using a Google Sheets spreadsheet, which isn’t ideal. But it gave us the idea to build an example app to improve this process. xfBBQ will be implemented as a React + Redux Single Page Application. We’re planning to pack large amounts of sometimes gratuitous functionality, but we’ll stay within the bounds of best practices.
Utilizing xfBBQ for planning company BBQs
Resolving Your Harmony Core Issues
There are two things we need to know when prioritizing Harmony Core development: what issues you have and what features you need. And we can more quickly resolve your issues and implement the features you need when we know what they are. The following are the best ways to reach us.
We host online office hours every six weeks. During this time, we deliver a short update on what we have improved in the Harmony Core framework. At the end of each session, we answer questions and work through issues you may have. This is a great opportunity to get direct and live feedback from us. During our last online office hours session, we fielded some support questions that we were subsequently able to resolve. We will continue to offer these sessions every six weeks. If you have any questions for us, please sign up for the next office hours session so we can answer your questions directly.
The next best way to reach us is through GitHub issues. Here we can easily keep track of your issues and make sure they are resolved as quickly as possible. This is also helpful because it creates a repository of answers for other users to search through. Recently we were able to resolve questions regarding ISMFILE.txt and .XDL files and where to put logicals by directing the user to our wiki documentation.
For some time CodeGen has shipped with a utility called CreateFile that (given an appropriately configured repository) could be used to create an ISAM file based on a repository FILE or STRUCTURE definition.
A customer recently asked me to create a mechanism that they could use to create multiple files, load data into those files, and also support creating both RELATIVE and ISAM files. Rather than writing something completely new I decided to extend the CreateFile utility, and that new feature has now been released along with the latest version of CodeGen (5.4.2).
Briefly, there is a new command line option -in that allows you to specify an input file containing the details of one or more files to be created and and populated. The file uses a JSON format. If you’re interested in finding out more you can find additional information in the CodeGen documentation.
Synergy/DE 11 is bringing many new features, and among those additions are some exciting new abilities within the String class. All of these powerful new features are available in both traditional and Synergy .NET.
When dealing with strings, keep in mind that a string is immutable (fancy word for unchangeable), so any time a string variable is changed, a new string variable is created.
First, let me list the new methods added, along with existing methods that include new overloads to the String class:
String.IsNullorEmpty(@string) , boolean
- Parameter: String to interrogate for null or empty value
- Return value: Boolean value indicating whether the string is null or empty
String.IsNullorWhitespace(@string) , boolean
- Parameter: String to interrogate for null or white-space characters
- Return value: Boolean value indicating whether the string contains null or white-space characters
New overloads to give precise control on how a string is trimmed:
String.Trim() ,string (exists prior to version 11)
- Parameter: None
- Return value: String with leading and trailing white-space characters removed
- Parameter: Character(s) to be removed from the string’s beginning and end
- Return value: Returned string with passed-in character(s) removed from the beginning and end
String.TrimStart(@string) ,string String.TrimEnd(@string) ,string
- Parameter: Character(s) to remove from either the string’s start or end
- Return value: String with all instances of character(s) removed from start or end
New methods to control how characters are removed or inserted into a string:
- Parameter: Starting position in the string to remove all characters until its end
- Return value: String with all characters from passed starting position to end removed
- Parameter: Starting position in the string to remove all characters until its end
- Return value: String with all characters from passed starting position to end removed
String.Insert(N, @string) ,string
- Starting position to insert string passed as parameter 2
- String to be inserted
- Return value: Returned string with sting passed in parameter 2 inserted
New overloads to give more control on concatenating strings together:
String.Concat(@string, @string) ,string (exists prior to version 11) String.Concat(@string, @string, @string) ,string String.Concat(@string, @string, @string, @string) ,string String.Concat([#]@string) ,string
- Two strings to concatenate together
- Three strings to concatenate together
- Four strings to concatenate together
- Dynamic string array to concatenate all elements together
- Return value: Returned string after concatenation operation
New overloads to locate the position of a substring in a string:
String.IndexOf(A1) ,int (exists prior to version 11) String.IndexOf(A1, N) ,int String.IndexOf(A1, N, N) ,int (exists prior to version 11)
- Character(s) to find in string
- Character(s) to find in string starting at position specified in parameter 2
- Character(s) to find in string starting at position specified in parameter 2 for the length of parameter 3
- Return value: Integer value with first position of passed character(s)
New overloads to find the last occurrence of a substring in a string:
String.LastIndexOf(A1) ,int (exists prior to version 11) String.LastIndexOf(A1,N) ,int String.LastIndexOf(A1,N,N) ,int (exists prior to version 11)
- Numeric value of the starting position to begin returning the substring
- Numeric value of the length of the substring to return
- Return value: Returned substring with value from the specified starting and length values
New methods to create an array from a delimited string:
String.Split(A1) ,[#]string String.Split(A1, StringSplitOptions) ,[#]string (see below for the StringSplitOptions enumeration)
- Character(s) to use a delimiter
- Enumeration StringSplitOptions to control if empty elements are created
- Return value: Dynamic string array to be populated with the contents of the passed string
New overload to use the replace method on alphas:
String.Replace(A, A) ,string
- Alpha value to be replaced by parameter 2
- Alpha value to replace all occurrences of parameter 1 in string
- Return value: Returned string with all occurrences of parameter 1 replaced by parameter 2
String.Replace(@string, @string) ,string (exists prior to version 11)
- String value to be replaced by parameter 2
- String value to replace all occurrences of parameter 1 in string
- Return value: Returned string with all occurrences of parameter 1 replaced by parameter 2
And finally, the enumeration StringSplitOptions, which is used with the Split method to control if empty elements are created when creating the string array:
public enum StringSplitOptions None ,0 RemoveEmptyEntries ,1
You can find details on all String members in the System.String topic in the Synergy/DE documentation.
As I mentioned earlier, some of the above are brand new methods (Insert, IsNullorWhitespace, Remove, Split, TrimEnd, TrimStart) added to the System.String class, while others are new overload methods added to existing methods to provide additional functionality. Interestingly, the runtime itself will make use of the String.Concat method when strings are added together (e.g., string1 + string), as it is efficient and generates fewer temporary variables.
Now let’s take a look at one of the new methods. Using the new Split method, we can populate a dynamic array from a delimited string. Say we have a string containing the following:
mystring = “Bob, Mary, Joe, Jack,,,, Fred, Henry, Tom”
We’d like for the values between the commas to be elements in an array, but we don’t want the commas without values between them to become empty elements in the array. This can easily be accomplished with the String.Split method using the syntax below:
stringarray = mystring.split(“,” , StringSplitOptions.RemoveEmptyEntries)
Passing the quoted comma as the first parameter tells the Split method that a comma is the delimiter to look for in the string. Also, for even more control, the delimiter can consist of any number of characters. There is no need to instantiate the dynamic array, as the dynamic array will automatically be instantiated by the Split method. The result of the above operation will be that the dynamic array stringarray will contain elements with the names that are separated by commas in the string mystring and no empty elements (due to passing StringSplitOptions.RemoveEmptyEntries as the second parameter) for the commas with no values between them:
stringarray = “Bob” stringarray = “Mary” stringarray = “Joe” stringarray = “Jack” stringarray = “Fred” stringarray = “Henry” stringarray = “Tom”
As you can see, the enhancements to the String class are more than enough reason to upgrade to Synergy/DE 11 when it’s released. If you can’t wait and want to try these new capabilities now, the version 11 beta is available in the Resource Center for download. Don’t forget to set your Synergy/DE documentation to the “Beta” version in the version drop-down near the top of the screen.
And wait, there’s more! Now that you’re aware of the new and powerful methods added to the String class, be sure to also check out the new StringBuilder class added to version 11.
Jeff Greene discusses some recent updates to Harmony Core. Jeff also asks the audience if there is any interest in an Entity Framework Core project template, and guidance for stand-alone use.
Thanks to those of you who joined us last week for our security webinar. We talked about some of the big security threats we all face today, and how to protect ourselves from ransomware/malware, endpoint attacks, and phishing. Keeping current with security updates is one important step, and this includes operating systems, applications, firewalls, and other components. Also, if you’re using HTTPS or SSL communication, you may need to encrypt your data between computers and when writing to disk (as well as when transferring externally).
If you use Windows, make sure you have a plan to upgrade any Windows 7 or Server 2008R2 systems. Microsoft is ending support for these at the end of this year (and for some of their other products soon). This means there will be no more security patches. This will increase the likelihood that your systems will be compromised with ransomware. Also, if you’re using OpenSSL on Windows, you’ll need to upgrade to Synergy/DE 11 (scheduled for release later this year) as it supports the new OpenSSL requirements. Synergy/DE 11 has other security updates, including the ability for services to be configured to run as a user other than system/root.
Here’s an article from our last Synergy-e-News for additional security information. And our Synergy/DE 11 page has for more information about our upcoming release. We’ll continue to provide you with information about these security topics and others. In the meantime, please contact us if you have questions or if you would like more information.
It’s been a while since a GodeGen release was announced on this BLOG, but rest assured they have been coming thick and fast; in fact there have been 9 interim releases since we last announced here.
So why all the frenetic activity in CodeGen land? Well the primary driving force for so many releases and so many new features is to provide support for the ever evolving Harmony Core RESTful Web Services framework. If you’re not familiar with Harmony Core then you should check it out. More and more Synergy developers are starting to take advantage of it as a way to expose their applications data and business logic in whole new ways and to support a myriad of use cases.
So what’s new in CodeGen 5.4.0? Well the answer to that question is … a LOT! I won’t list all the new features here but if you want to check out the specifics then you can head over to the CodeGen Releases page where you’ll find a detailed change log for this and previous versions. And if you need more information than that then check out the CodeGen Documentation.
We recommend that all developers who are using CodeGen upgrade to this latest version, but particularly if you’re developing with Harmony Core.
Happy 4th Everyone!
Jeff and Steve discuss some recent updates to Harmony Core and review a few use cases used by others.
Jeff Greene and Steve Ives talk to Chris about the progress of his project. Steve also presents a demonstration on the basics of Harmony Core to Deborah.