In the first article in this series I discussed an issue with how versioning is done in Identity Manager Packages, for Designer.

Packages are great for both Micro Focus to deliver content, but also for people like me, a consultant to deliver content. They are far from perfect, so it's best to understand weaknesses and how to work with them.

The way versioning works is one major pain point. You work in your development lab project, finalize your project, and then finally Build it, hitting the Release button, which locks it, so you can then Publish it to a web site for your customer to go get.

But the act of Releasing a package, resets the date string in the version. I almost always forget to push that specific package version back out to the Dev tree. This is because I am hesitant to Deploy, which would update everything since there is some bug in Designer I cannot quite put my finger on well enough to report that causes deploy events to cause issues. Not always, but sometimes. (Usually when someone else has changed a Driver set Global Configuration Variable and your project is not up to date).

In order for your new package version to be pushed to the tree, every instance of DirXML-pkgGUID on every piece of packaged content needs to be updated. Ok, there is only one such attribute per object, but you get what I mean.

However, assume you did not do that, the ship has sailed. The version stamp is from when you started working on the project 1 month ago, you just built it, and now the date stamp in the version string is one month later.

That string could look like this:
       0.0.1.20170911103600

But the one in the Dev tree is:
       0.0.1.20170811103600

You come back a few month later to fix a new issue, and import a fresh Designer project. You have picked up via the Check for Package Update option all the latest versions, and you have the Sept 2017 version, but the Aug 2017 is non-existent. It is gone. How do you recover it?

One approach is take the Sept 2017, add it to the imported driver, and push that back out to the tree.

But if you know you are in this instance, I recently realized there is a way that you can recover the specific version number.

First thing to realize, is how is a package structured? We see hundreds (are we at a thousand yet? My Designer has 1262 packages, so yep, we are at a thousand) of JAR files. But what is inside them?

Well JAR is really just a ZIP file. In fact any Java file, that is a JAR, can be unzipped. If you are running on Winders, which is foolishly reliant on extensions to define file types, then the simplest next step is just rename the file to .zip, and then open it with any ZIP tool (WinRAR, 7Zip, heck Windows built in Compressed folders).

Inside a JAR, I will pick the largest most complex one as an example which is of course the User App Base package. It is 1.13MB in size, the largest by far, next biggest is a JMS driver base package around 250K. So almost 5 times bigger!

There are two files, package_import.xml and plugin.xml, and a directory META-INF, which contains a single file MANIFEST.MF.

The files are all text. The package_import.xml is basically all the components of the package as if you had done an iManager driver export just for those components. If you open the file you will see the XML that you would expect. Alas, if there is XML as the value for any particular field, it is Base 64 encoded so you would have to decode it to look at it. Additionally, some of the User App Provisioning features are actually Base 64 encoded inside XML fields, you might decode field to find that there is another encoded element within it. Just get used to it, tis the way of the world.

So experimenting with this, I first tried changing the MANIFEST.MF file. As a manifest would imply it controls the content.

The contents look something like this:

Manifest-Version: 1.0
Require-Bundle: com.novell.idm.packagemanager
Bundle-Vendor: Novell, Inc.
Short-Name: NOVLUABASE
Supported-Drivers: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4
 8c3VwcG9ydGVkLWRyaXZlcnM Cgk8ZGVmaW5pdGlvbiBkaXNwbGF5LW5hbWU9IlVzZXIg
 QXBwbGljYXRpb24gNC41IERyaXZlciIgZHJpdmVyLWlkPSJOUHJvdiBEcml2ZXIgNC41L
 jAiIGlkPSJVQVBST1YiLz4KPC9zdXBwb3J0ZWQtZHJpdmVycz4=
Dependencies: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz48ZGVw
 ZW5kZW5jaWVzPgoJPGRlcGVuZGVuY3kgbmFtZT0iUHJvdmlzaW9uaW5nIE5vdGlmaWNhd
 GlvbiBUZW1wbGF0ZXMiIHBhY2thZ2UtaWQ9IkhGS09JMzhNXzIwMTAwNzI3MTIwNjI1MD
 Q5NiIgdHlwZT0iNCI CgkJPG1pbi12ZXJzaW9uIHZhbHVlPSIyLjAuMCIvPgoJPC9kZXB
 lbmRlbmN5PgoJPGRlcGVuZGVuY3kgbmFtZT0iUGFzc3dvcmQgTWFuYWdlbWVudCBOb3Rp
 ZmljYXRpb24gVGVtcGxhdGVzIiBwYWNrYWdlLWlkPSJCSktNSE1DTl8yMDEwMDcyNzEyM
 TYzNTA1OTQiIHR5cGU9IjQiLz4KPC9kZXBlbmRlbmNpZXM 
Internal-Version: 50
Bundle-Version: 4.5.1.20150602213315
Base-Package: true
Features: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz48ZmVhdHVy
 ZXM Cgk8bWFuZGF0b3J5Lz4KCTxvcHRpb25hbC8 CjwvZmVhdHVyZXM 
Bundle-Name: User Application Base
Bundle-ManifestVersion: 2
Type: 2
Bundle-SymbolicName: com.novellinc.novluabase; singleton:=true

So it seemed like the Bundle-Version might control it.
       Bundle-Version: 4.5.1.20150602213315

I tried changing the date after the 4.5.1 to a different value to see if it imported into my Package Catalog with that proper date, but alas, it kept the shipping date. (For clarity, in testing, I actually used a custom package of my own, not the NetIQ shipping User App base package, since that would be of no value to change. I was testing this for a client with a specific package, and a specific missing version).

Then I figured the plugin.xml file, would contain useful information on importing it:

<?xml version="1.0" encoding="UTF-8"?>
<?eclipse version="3.2"?>
<plugin>
    <extension point="com.novell.idm.packagemanager.packageregistration">
        <packageregistration>
		       <package id="B5PAGQ5E_201005261601510810" version="4.5.1.20150602213315"/>
            <display name="NOVLUABASE_4.5.1.20150602213315"/>
            <symbolic name="com.novellinc.novluabase"/>
        </packageregistration>
    </extension>
</plugin>

There we have a version XML attribute, (in XPATH plugin/extension/packageregistration/package/@version) so maybe changing this one and possibly the display name ( in XPATH plugin/extension/packageregistration/display/@name ) would make a difference and it too did not help.

At this point, it had me a bit worried, but I went back to look more carefully at the package_import.xml and noticed the very first node in the XML:

<?xml version="1.0" encoding="UTF-8"?><package base-package="true" category="Provisioning" category-folder="User Application" checksum="3461721773" directive-checksum="2124674124" id="B5PAGQ5E_201005261601510810" name="User Application Base" symbolic-name="com.novellinc.novluabase" type="2" version="4.5.1.20150602213315">

There is a name for the bit at the beginning, but I can never remember it, but it basically declares the type of XML we are looking at:

<?xml version="1.0" encoding="UTF-8"?>

But inside the package node, there are two very interesting XML attributes:
       checksum="3461721773"
       version="4.5.1.20150602213315"

Now I would love to see the DTD for this XML, but as far as I know it is not published. (I actually was thinking they probably actually define it in one of the JAR files in Designer, since most XML parsers expect a DTD to follow the rules and validate arbitrary XML against but there are so many JAR's I am not sure where to look.)

I know from understanding Packages that the checksum matters, as it is used to make sure the package is not dirty. I.e. Designer looks at the checksum from the package for each element and then calculates it on the various components it sees in a driver, and if the two numbers do not match, then it is dirty.

Thus when you use the very very cool Compare Customizations (Driver Properties, Packages side tab, select your dirty package, and instead of Upgrade or Downgrade, try Compare Customizations and it shows you all the components that are dirty (not in the Provisioning space though, that is mostly broken and unsupported) and what each looks like, then does an XML side by side compare just like you would see when you compared against the live tree). This is super handy to see if anything was actually changed.

There was a bug a while ago where on a Windows vs Linux Designer instance, ECMA script objects had different line feed/carriage return characters so the entire contents was dirty since every line was different but that was fixed a few versions ago.

So the checksum is key, even after much nagging and begging I was not able to find out how it is calculated, since then I could edit a package, make the change I need, reset the checksum and all would be good. But of course that would defeat the entire purpose. (Mostly I was curious to see if I could). I am sure I could track down the Java function that does it, but I do not have the time, considering how big Designer is in Java.

Interestingly there is also the XML attribute: directive-checksum="2124674124"

I am not sure what this checksum includes as opposed to the checksum XML attribute but clearly there is a division. I suspect that they break the notion of the policies and resource objects into one set, and then the meta information about how they should be installed and the package itself into a separate set.

For example, a Base package can include driver initial settings like an image file for iManager to show a custom icon. It can include the Filter if you want, but that is bad form, instead you should use a Filter Extension. Thus I suspect this checksum is for that sort of stuff in the package, while the other is for everything in its entirety.

But my key mistake in thinking this fix was impossible was I thought the checksum value would include the version string. But I see that the version XML attribute and checksum XML attribute are at the same level. This likely means that version is not part of the text being checksummed. If it was, then checksum would likely be part and how could you generate a checksum that includes the calculated checksum, without knowing it before you checksum it? Very chicken and egg like problem. (Of course the answer is egg.) Of course they could have had the checksum function include everything BUT the checksum XML attribute but that seemed unlikely.

So I tried changing the version to match the version I needed, and viola, I rezipped the file, renamed it back to a JAR and imported it into Designer.

Now it shows up in my Package Catalog with the proper value. Excellent.

However, I think looked at the JAR it had imported, and checked the MANIFEST.MF and plugin.xml and they did not have the values updated. So I would probably not distribute this one wide and far, since it is slightly inconsistent. It is useful for getting the version you need properly imported, when you have lost a build version but not really for running long term. The next step is to upgrade to the released version and update the Dev lab with that proper version number. Additionally this is proper since then Dev and Prod would look the same, which is what you really want.

I did notice that after the package node there were some interesting fields that might have been an issue. But since these were inside the checksum'ed text I do not think you can successfully change them.

	<idm-buildtime>20150602213436</idm-buildtime>
	<idm-creationtime>201506022047440346</idm-creationtime>
	<idm-internalversion>50</idm-internalversion>

Here you can see the idm-buildtime, which is usually the date stamp of when it was last built, but interesting that does not quite agree with the package version date stamp, which is, 4.5.1.20150602213315 being off by almost 3 and a quarter minutes. Not sure what is going on there. The id XML attribute looks like this:
       id="B5PAGQ5E_201005261601510810"

That I think is when the first User App Base package was created, from which all other versions have been spawned.

The idm-creationtime is interesting to see when it was started, but I assumed that would be the version first version was created but this is the same day as the build time so that probably means something else.

There are nodes for the min and max version of IDM and min and max application versions that the GUI exposes when you make a package, but I notice the idm-internalversion value of 50 is interesting. I have no idea what that means.

I looked through the rest of the XML and did not find any other version instances, so it looks like this really is the controlling one. Testing it looked pretty good. I passed on a version to my client with the specific version he needed and all was well.

Now I happen to think that the simple fix for this issue would be to simply allow the current timestamp on the package to stay the same, when you build it each time, even if it was a preference setting, and this whole issue would go away.