Novell Identity Manager is a big product with lots of moving parts. It can be quite daunting to get started working with it. Just as bad, the deeper you dig, the more complexity you discover. Now this is not necessarily a bad thing, it is just something you have to deal with. There is a learning curve.

Thus my attempt here to try and lower that curve. First off I have an article with stuff you should start off reading to get over the first hump in the learning curve. If you have not already read this stuff, take a few minutes and do so, it will pay off ten fold as you work through IDM: FAQ: Getting started with IDM, what should you read?

On top of the learning curve, there are some reasonably well known common problems that beginners in the product will run into. I figured that trying to write them down can only help. If nothing else, Google ought to find it, if you search for it.

Thus I started this series, with the first article that you can read here: Common Mistakes Newcomers to IDM Make - Part 1

In that article I covered the concepts:

• Using the forums

• Basic articles to read to understand the engine

• Default DN formatting

• Verb vs Noun tokens in Argument Builder

• Time Conversions

• Built in variables (Local, global, and error)

• Active Directory driver and SSL

In the second article, Common Mistakes Newcomers to IDM Make - Part 2 I covered examples of:

• Getting a driver started (migration)

• Designer vs iManager versions of the project

• Case sensitivity issues when specifying shim class

• Move token, specify destination container

• The Attribute tokens

In the third article Common Mistakes Newcomers to IDM Make - Part 3 I covered examples of:

• XPATH

• Using the Remote Loader instead of Local

• Restarting eDir after a code change

• Tracing to a file per driver

In the fourth article Common Mistakes Newcomers to IDM Make - Part 4 I covered examples of:

• Per Replica attributes

• DirXML-Associations

Now on to part 5 (will this ever end? Alas I suspect not for a while, there is so much complexity to the product, and so many thing just not clear from a simple reading of the documentation that I imagine I will keep doing this for a while!)

Variables $ vs $$ vs ~~ vs XPATH

Something that generates a lot of confusion, and is actually reasonably handled in the documentation is the myriad ways you can address variables in Policy.

There is squiggle notation (~GCVName~), single dollar sign notation ($VarName or $GCVName) and then double dollar sign ($VarName$ or $GCVName$) notation.

Ok, so when do you use each type, and does it matter?

The simplest answer is that single dollar sign is used in XPATH expressions only. This can provide a Local variable ($LocalVarNam) or a Global variable ($GCVName). This is not referring to the scoping of Policy of Driver on a local variable, rather a local variable vs a Global Configuration Variable.

You use the single dollar sign to use directly in XPATH or to pass to ECMA via an es:functionName($localVarName) like call. Same for making a Java call and passing it data.

As of IDM 3.6 most places where it makes sense in DirXML Script Policy, the ability to use a variable was added. That is, say you want to do a Set Destination Attribute token. Previously you of course could provide the data that goes into the destination attribute using the Local Variable token. But with variable replacement, even the attribute name can use a local variable, if you use the double dollar sign notation.

This is great, since you might have a rule that needs to clear a long list of attributes if the user is deleted. Well you could easily write a condition if Login Disabled is changing to true, that then fires one action for each attribute in the list (Clear destination attribute token). But that is really repetitive and if your list changes over time (adding new attributes, removing old ones from the list) it can be a pain to maintain.

Instead, you could store the list in a GCV that is type List, add each entry. Now for a trick, if you set a local variable as a nodeset with the GCV data, you get a nodeset of the data. If you set the local variable as a string, you get a string separated by the delimiter you specified in the GCV's configuration. If you set a nodeset local variable, you can then do a for each loop over that variable, and then inside, have a single action of clear destination attribute $current-node$ and all in the list will be processed.

I call this configuration vs code as an approach. Do as much as you can in configuration, so that changes do not require additional coding. Makes life easier and more maintainable.

A double dollar sign will work for a local variable or a global configuration variable equally well.

So if that is where we use single and double dollar signs, what is the point of squiggles, (Ok their real name is tilde, but I like squiggle better)?

Well squiggles are interesting as they can expand to a GCV only, but the way they work is very different.

A single or double dollar sign is evaluated at run time. That is, when the code is executed the values are expanded. The squiggle ones are replaced at driver start time. To the point if you reference a GCV in squiggle notation and it does not exist, the driver will not start. If you do that in single or double dollar sign notation you do not get an error until the engine hits that part of the code.

Also, the squiggles will be expanded even in comment fields! This has a nasty side affect that if you want to talk about a GCV in comments and use squiggles, the GCV better exist of the driver will not start!

There was a bug in IDM that is sort of fixed in 4.0 and is set to be back ported to 3.6.1 in FP5 I think, related to funny characters in a variable. Specifically characters you would use in a Regular Expression. The Replace All token uses Regular Expressions, so you can do some amazing things. (I once saw someone do a replace all token to do time format conversions using capture groups! I never thought of that approach but it does work! Crazy to maintain and Convert Time is so much easier to use). If you tried using a double dollar sign variable, you would get a binary character replacing your Regular expressions special character which means it just plain did not work. However the bug was in run time expansion, not in the at driver start time expansion, so if you could store it in a GCV you could use squiggle notation to replace it. With IDM 4 it is sort of fixed. The expansion appears to be correct but it does not quite work in the Regular Expressions. I need to test it some more to be certain. This was funny as I wrote an example policy in Designer 4 and tested it in Simulator which worked. Then I tried it on my 3.6.1 tree and it failed. That is because Simulator on Designer uses the IDM engine on your workstation to simulate your policy, and Designer 4 is using the IDM 4 engine with all its bug fixes (and possibly new bugs).

Regular Expressions:

Now that I have raised the notion of using regular expressions, might as well address them as well.

Regex (as Regular Expressions are commonly known) are very powerful and a tool used to match patterns in data. There is tons of stuff on this on the web. If you know Perl a lot of Perl's find, substitute and replace functionality relies on Regex. If you use less in a Unix host to look at your DStrace logs, then you should know that forward slash (/ search forward) and question mark (? search backwards) usually support Regex in your searches as well. The primary consequence is that Regex special characters need to be escaped with a backslash if you want to search for a literal string. I.e. To find ^Bob via less, you would actually need to type it as \^Bob\ since a caret (^) and a plus sign ( ) mean something special in Regex.

The power of Regex is astounding, and you can find examples on the inter tubes that do literally magic with a soup of comic book curse words. IDM supports almost all of it, which is cool. As I said above, I saw someone use capture groups to pick apart a date string into groups of components (one group for the year, one for the month, one for the date) and then put them back together in another order. That is a ludicrous use, since Convert Time() is a built in token that is way easier to use and maintain. But his approach does work for the limited case.

There are several books on Regex, from OReilly, and the Dummies series that can get you started. There are tons of web based resources, from tutorials to online testers. I.e. Enter a Regex, enter a match and/or replace string and see if it works.

My tool of choice is called RegeX Buddy and costs about $50 and is totally worth it. It has a GUI to let you pop in the various available functions in Regex without having to memorize them or look them up. It has an explain mode that tries to tell you what the Regex is doing, and it has a simulator to show you what is being matched or replaced. Great tool, very helpful and faster to test with than iterating in Simulator.

Now where can you use a Regular Expression in Identity Manager?

Alas, one place it would be really helpful, but you cannot directly use it is in an XPATH expression. You would need to call an ECMA or Java function that supports Regex. Now as it turns out, David Gersic wrote an article explaining just how to do that!

Adding Regular Expression Matching to XPath in Identity Manager

Basically you need to add an ECMA function something like this:

/** Return true or false based on a Regular Expression match
* @param {String} s1 the string to test
* @param {String} s2 the regex string to test it with
* @type Boolean
* @return a boolean true or false
*/
function matches(s1,s2)
{
 // Build search and replace options.
 var options = "";

 var re = new RegExp(s2, options);
 if (s1.match(re)) {
  return true;
 } else {
  return false;
 }
}

You get to use this by calling his matches() function in a XPATH token, with something like es:matches($SOURCE,$PATTERN) where es: means its an ECMA function call. (Do remember to link the ECMA object to your driver that wants to use it). Then provide either the literal string in quotes, or else use a variable name it is stored in for the string. Same thing for the pattern, that you want to match.

This is great as now you can easily use Regex in XPATH as well which can be very helpful at times.

In almost all of the condition tests, if there is an equality option, then the default is case insensitive for the compare type, but you can change that to Regular Expression (Or numeric or case sensitive, etc as your data suggests). In which case the string you provide stops being a literal string compare and instead becomes a regular expression compare, which is immensely more powerful.

The simplest example I can think of would be in the if operation token, you can set the equality test to Regular expression, and match on the string add|modify|delete so you can in a single token catch the add, modify or delete events in a single test. (Obviously in that example, the pipe symbol (|) is the OR indicator.)

The Replace First and Replace All tokens in Argument Builder allow you to use a Regular expression for both the Search and Replace strings.

As I look through the tokens, I think that may be all the locations. Seems like only a few, but the ability to use it in almost all the condition tests is really powerful, and the Replacement tokens are great for changing the data around as needed.

Going back to an earlier example I mentioned, of using the Replace Token to reformat dates, here is the crazy

<do-set-local-variable name="trialtermstart">
	<arg-string>
		<token-replace-all regex="(\d\d\d\d)-(\d\d)-(\d\d)" replace-with="$1$2$3000000Z">
			<token-xpath expression="$current-node/termBeginDt"/>
		</token-replace-all>
	</arg-string>
</do-set-local-variable>

Quickly diving into this example, you see that the replace statement is (\d\d\d\d)-(\d\d)-(\d\d), which is looking for a very specific pattern of 2011-12-11 since \d means any numeric digit, grouped. So you can see that it is assuming a date pattern of 2011-11-11 and then it is replacing it with the Regular expression of $1$2$3000000Z which uses capturing groups, so the first group ($1) goes first then the second group ($2), then the third capture group ($3), followed by a bunch of zeros and the letter Z.

You can see this is a silly example, but also its sort of an extreme crazy example of how you could use Regular expressions if you really wanted too.

Some quick Regular expression tips:

^ matches the start of the line
$ matches the end of the string
. matches any single character that is not a line break
modifies the previous match to match between 1 and many times.
* modifies the previous match to match between 0 and many times.

Thus .* means 0 or more of any non-line break characters, where . means 1 or more non-line break characters.

There are a whole stack of backslashed flags for things like tab (\t), carriage returns (\r), line feeds (\n), and so on.

Then there are some that are shortcuts for character classes. Like \d which represents a digit from 0-9. (\d would be one or more digits). This is actually a short cut for [0-9] which is how a class would be defined.

I could go on for hours, but go find a good web resources and read up. Or ask in the support forums and someone will be glad to help out. Some people (**cough*** Aaron **cough** seem to get great joy in coming up with crazy Regex solutions. In fact the Perl guys consider it a challenge to write obfuscated Regex/Perl code).

As always if you have some other examples of things you think a beginner in IDM should know, or a common pitfall you or others have fallen into, please send them on to me in any of the various systems you can find me. (Forums, Cool Solutions, email, whatever).