I have been discussing the Identity Governance product in terms of interesting things I have noticed using it. In trying to resolve these issues, I have not usually found the answer in the documentation, since it is impossible to document everything. Additionally, the kind of information I need, working within the product, doing foolish things that most people would never consider doing, is not the sort of thing documentation writers usually provide. It is sort of the difference between the developers perspective and the guy in the field using the product perspective. They are different but complimentary. Thus I am glad to write these sorts of articles and I hope that these views from the field will be helpful.

If you run into some issues of this sort, please consider writing it up as an article, as it is not ever going to get out there to help other people otherwise.

Display Name Issues:

I was working with a customer on Identity Governance, and we ran into several issues. Some of them were data consistency issues or else unexpected data.

The first was, our Identity source was eDirectory, and the Given Name and Surname attributes held multi valued data. Now normally people are of but one name. That name may have many components. For example, John Jacob Jingleheimer Schmidt, is that one first name, and three last name components? Not important, because so long as the two attributes (Given Name and Surname) have one value each, all is well.

In this system, alas, there were users who had two or more values in eDirectory for those attributes. It was not just those attributes, rather it was all over the place in all sorts of attributes.

Now in eDirectory, this is completely legal. (I did not look and see how their Identity Manager drivers were dealing with this, since some of the targets are not multi valued for these attributes, and I probably should. But I am on the Identity Governance project, IDM is theirs to work on).

But in Identity Governance, the default attributes in the database, expect single values for most of the attributes. You cannot in fact edit those attributes. So what do you do?

The reason this matters is that the users with multi valued data, being stored in a single valued field, did not store it as a JSON array as it is represented in the collected raw data, but rather errored on the user and refused to import them. We had about 1000 users in this state so that was a problem. We need the users in the Identity collection, so can't just throw those away.

Our first approach was to define new entries for First name and Last Name but this time defined as allowing multiple values. Then we renamed the original attributes out of scope and did not collect them at all.

This worked, we got the data into the system and were starting to work with it. However at some point we finally noticed that when listing Identities, or looking at Accounts in the catalog, the Display Name field was empty.

This is not good. In fact, we would see a display of blank lines, with only a Display Name column and all empty. As noted in previous articles in this series I enabled a lot of tracing and I got the REST messages into trace, and I saw that when I refreshed that page, it made a REST call, and it got the data back, with an empty value for Display Name. Thus I knew there was data there and being returned, just the Display Name field is empty.

Ok, this is not that big a deal, except for the GUI issue that the hyper linked field, wherein you can drill down further into a catalog object is the Display Name field. If it is empty, there is no hyperlink so you lose your access to the user data in the catalog.

Looking at the definition of the Display Name field, we see that it looks like it reads displayName as its attribute key. Except eDirectory does not have a displayName attribute. Active Directory does. Ok, this makes it look like this is a strange design decision that I do not get.

Our next thought was well, this is eDirectory and we can fix this easy peasy. Simply pick the LDAP server we are reading data from for our Identity collection, and change the Attribute Map in iManager so that an LDAP client searching for displayName, (which is not in eDirectory schema and thus should return nothing, always) and set it to return CN (which in our case, in this tree is what we want. Oddly tree has uses named by uid, and CN holding the full name data. Whatever.). Thus we can fix it in eDirectory. This is a super useful feature of eDirectory that makes it really easy sometimes to solve application problems.

Some applications have very simplistic views of how LDAP should work, that is really just because they developed against a single directory service and never considered other valid and legal approaches. eDirectory makes this easy to work around, which is great.

However, we did not get around to making this change (We were collecting from Prod, and needed to wait for permission first). But before that we found out what was the cause of the real issue.

It turns out that Display Name is NOT a read of the attribute displayName from the Identity source, instead it is actually pulled from the database, and is configured in configutil.sh



Load up configutil.sh in GUI mode (default, to make it console, you need to add the switch -console when you run it) and switch to the Miscellaneous Settings tab. Amusingly during the formal training on this product they said, you should never need to touch these settings most of the time.

In this case our answer was the first item, User Display Name, that had the value:

COALESCE(first_name,n' ')||' '||COALESCE(last_name,n'')

First thing of note, the fields for the shipping first and last names, do not have the underscore in their names. This makes me think these are database references. Thus the next step is to figure out which database table this is coming from, find the proper column for our extended attributes. I am not sure that the database is particularly well documented so I did not want to do that, so we decided to move on to our next idea.

Collecting multi valued attributes into a single valued field:

Our second idea was to use ECMA to process the incoming data in the fields that hold multi valued data. So what ECMA do we need? Well there is a document (PDF format) that shows some default transforms to help get you started, clearly written by a consultant, not a document writer.

https://www.netiq.com/documentation/identity-governance-30/pdfdoc/references/Collector-Data-Transformation.pdf

Now there is also a link that looks perfect, DaaS Multi-valued transformations, here:
https://www.netiq.com/documentation/identity-governance-30/pdfdoc/references/Multi-value-transform-DaaS.pdf

That should be exactly what I am looking for!! Alas, that is meant for when your collection data input, is a set of values, that you wish to merge somehow and generate some output from it. Again, offers useful insights, since I find examples are often the best way to learn how to use a system. (Have you ever read an RFC to learn how to use a product, but then you see an example and it is so much clearer?)

The GUI itself, when you click on an attribute, on the left, the name of the field is a hyperlink which shows you the 'schema' definition for the attribute. But on the right of the field you can enter an attribute name there is a {..} icon that expands the field, and allows an ECMA processing field.

Here they cleverly give you some hints, such that the collected value will be stored in a variable known as inputValue and you need to send the results into a variable called outputValue. Ok, that is helpful and useful. This gives you the basics of how you get your data delivered (inputValue variable) and how you return your result (contents of outputValue variable).

So what format does data come in, via the collector? Well again, this product has some very clever features at the bottom of the Collection definition page, there is a Test Collection button.

This takes you to a screen where you can run a test collection and download the Raw Data. Use the Action button that has a drop down menu where Download is an option.

This told me that the data provided to the Collector from the DaaS (Directory as a Service) service comes in as a JSON array. Now is that a JSON array for real or just a string representation?

Looking at the Test data downloaded, I see it means the collected data would look something like:

["Geoffrey","Geoff","Jeff"]

So my first approach was simple. Try to take the first node simply using the position notation of [position] that is so very common in most languages.

outputValue=inputValue[0];

However that was returning the string '[', that is the first character of the string with a JSON array. Ok, so it is a string representation of a JSON array and I need to convert it to a JSON array first.

Next approach uses the JSON object to parse the inputValue into an array. Then take the first value of that array.

var jsonObj=JSON.parse(inputValue);
outputValue=jsonObj[0];

That looked like it worked in testing. So we kicked that out into the full collector, and guess what happens when you have a non-multi valued input or I guess you would say, single valued input? You get problems. You get the first character value of the single valued input. Thus we need to handle two cases, one input value, and more than one. This is a case where some REST endpoints use JSON in a more consistent fashion, and all results are arrays. That array might have 0, 1, or more values, but it is always in the format of an array which makes dealing with it simpler. However, Identity Governance does not quite do that, so we need to be more clever.

Ok, so lets test and see if this is an array and if so, take the first value, else just return the current value. How can we tell if it is an array? Well any isArray style test, expects a parsed object first to then test for an array.

   
	if( inputValue[0]=='[' ) {
			var jsonObj = JSON.parse(inputValue);
			result=jsonObj[0];
			} else {
			result = inputValue;
	}
	outputValue=result;

Check and see if the first character of inputValue (inputValue[0]) is equal to open square bracket ([) which means this is an array in string form and not single valued.

In that case, JSON parse it into a proper array and then the [0]th element of jsonObj is the first element that we want. Otherwise, just return the current value, since it is single valued and what we want.

I suppose we could do something clever and pick the last entry or the second but first seems like the simplest approach to implement.

This worked and let us collect the data from the multi valued fields into the standard single valued fields, and we had to run a couple of collection cycles so we could find all the attributes that Identity Governance expects to be single valued, but were actually multi valued in our directory.

What is nice is that after a collection is complete, there is a report that shows you the first 100 errors it found. This allowed us to track down the various attributes, where just copied in the ECMA we developed into all of them. All good. After each pass we got the number of errors down (since we could only see the first 100 errors), until there were none left as we properly handled all the attributes.)

Then we properly got Display Name when we looked at the Catalog. Yay!

This was an interesting problem and provided interesting insights into the inner workings of Identity Governance so I thought it would be a really good topic to discuss.