#FIM2010 MPR Integrity Checks

I recently had reason to suspect that there were a number of MPRs which had become corrupted in a lab environment due to the deletion of set objects.

FIM 2010 doesn’t complain when you delete a set, but it will leave any associated MPRs in an invalid state.  Obviously this is not desirable, and you wouldn’t intentionally be doing this.  However, it is possible that someone who doesn’t know any better could make this mistake, and if they have, how would you know?

I decided I’d write a couple of xpath queries which could probably be useful as MPR search scopes – they identified a number of faulty MPRs, and they may be worth running on your own environments now for that extra peace of mind!

  • Rights-granting policy where the PRINCIPAL set reference is missing
/ManagementPolicyRule[not(PrincipalSet=/*) and GrantRight=true and not(starts-with(PrincipalRelativeToResource,'%'))]
  • Non-rights-granting policy where the FINAL set reference is missing
/ManagementPolicyRule[not(ResourceFinalSet=/*) and not(GrantRight=true) and not(starts-with(PrincipalRelativeToResource,'%')) and not(starts-with(ActionType,'Transition'))]
  • Transition IN policy where the FINAL set reference is missing
/ManagementPolicyRule[not(ResourceFinalSet=/*) and not(GrantRight=true) and not(starts-with(PrincipalRelativeToResource,'%')) and (starts-with(ActionType,'TransitionIn'))]
  • Transition OUT policy where the CURRENT set reference is missing
/ManagementPolicyRule[not(ResourceCurrentSet=/*) and not(GrantRight=true) and not(starts-with(PrincipalRelativeToResource,'%')) and (starts-with(ActionType,'TransitionOut'))]

Note that the above queries are not likely to be a definitive set, and I’d be keen to add to them over time.  They also are written on the premise that MPRs which are rights-granting do not invoke any action workflows (a “best practice” I stick to religiously).

Hope this sparks some other ideas on FIM policy integrity checks.  Let me know if you come up with any others, or variations on the above.

 

Posted in Uncategorized | Tagged , , | Leave a comment

Identifying #FIM2010 Database Index Fragmentation

I want to share the following SQL script which I have adapted for FIM from the original here.

If you read the blog post you will understand that both FIM databases meet the criteria the author describes (GUID cluster keys) as a cause for high index fragmentation, leading to poor FIM performance in a variety of ways (even leading to SQL timeouts in extreme cases).  If only the GUID keys were able to be sequential to be able to avoid this problem – but alas they are not.  Hence the need to do something about them – and regularly!

When troubleshooting poor application performance where SQL is involved, my approach (with both FIM services) is to open the following script in SQL Server Management Studio, selecting the FIMService database in the toolbar drop-down list:

SELECT TOP 20 
 OBJECT_NAME (ips.[object_id]) AS [Object Name],
 si.name AS [Index Name],
 ROUND (ips.avg_fragmentation_in_percent, 2) AS [Fragmentation],
 ips.page_count AS [Pages],
 ROUND (ips.avg_page_space_used_in_percent, 2) AS [Page Density],
 CASE 
 WHEN ips.avg_page_space_used_in_percent = 0
 THEN ips.page_count * ROUND (ips.avg_fragmentation_in_percent, 2)/100
 ELSE ips.page_count * ROUND (ips.avg_fragmentation_in_percent, 2) / ROUND (ips.avg_page_space_used_in_percent, 2)
 END AS [Weighting]
FROM sys.dm_db_index_physical_stats (DB_ID ('FIMService'), NULL, NULL, NULL, 'DETAILED') ips
--FROM sys.dm_db_index_physical_stats (DB_ID ('FIMSynchronizationService'), NULL, NULL, NULL, 'DETAILED') ips
CROSS APPLY sys.indexes si
WHERE
 si.object_id = ips.object_id
 AND si.index_id = ips.index_id
 AND ips.index_level = 0
 AND si.name IS NOT NULL
ORDER BY CASE 
 WHEN ips.avg_page_space_used_in_percent = 0
 THEN ips.page_count * ROUND (ips.avg_fragmentation_in_percent, 2)/100
 ELSE ips.page_count * ROUND (ips.avg_fragmentation_in_percent, 2) / ROUND (ips.avg_page_space_used_in_percent, 2)
 END DESC
GO

The results I will get back (after about a minute running the first time) will show the tables which I have rated as the most in need of defragmentation (highest weighting) at the top.  I then proceed down the list to simply locate each offending table (e.g. ObjectValueReference is a prime candidate), expand the database table in the Object Explorer treeview, and select the Rebuild All option from the RH mouse menu (at this point I am a tad heavy-handed, and prefer to rebuild ALL of the indexes not just the one that shows the highest weighting).  For particularly large FIM databases it is often best to stop the FIMService first before doing this, but I find I don’t always have to do this.  Once I have completed this exercise I repeat it once or twice until I am satisfied that the remaining fragmentation is acceptable and not likely to cause further problems for now.

I then select the FIMSynchronizationService database in the drop-down list, comment the line referencing FIMService, uncomment the corresponding FIMSynchronizationService line, and repeat the above process for the FIM Sync database (with far less tables here you will find this far quicker).

At some stage I plan to implement something along the lines of the SQL Server Maintenance Solution , but in the meantime I am using this spot fix approach – particularly after the initial system data load on deployment, or after periods of high data volatility such as the beginning of a school year at an education site :).  I like the weighting idea because the generic > 30% fragmentation rule sometimes used (or similar) doesn’t necessarily highlight those fragmented table indexes which are having the greatest performance impact but may be below your threshold.

Posted in FIM (ForeFront Identity Manager) 2010, ILM (Identity Lifecycle Manager) 2007 | Tagged , , , , , | 5 Comments

#FIM2010 R2 SP1 (W2012) Oracle Management Agent requires additional Oracle driver

My good friend Henry from infoWAN in Germany asked me to post the following for him on my blog (in lieu of setting up one of his own at least for now).  Here is the tip that Henry discovered and was so keen to share …

My task was to replace an ILM 2007 Server with FIM 2010 R2 SP1 running on Server 2012. The newly configured FIM Server is being stood up alongside other systems connected to an Oracle Database Version 11g.  Here was my approach

  1. Checked the Server compatibility with the release notes of FIM 2010 R2 SP1.
  2. Checked Oracle compatibility with the Management Agent list.
  3. Installed FIM and the Oracle Client Software 11.2.0.x and tried to create the Oracle Management Agent ,,,

This was as far as I got before I the install failed, reporting the Client Software could not be found as displayed below:

Image

Error connecting to Oracle database’

At this point I checked Oracle Database connection using SqlPlus and was able to open the View in question. Given this proved the client was installed correctly and the settings in the tnsnames file were also fine, I then checked file permission on the Oracle Client directory for the FIMSync Service Account as others had suggested in the same situation.

I then dug a bit deeper, using the Process Monitor to look up what was happening behind the scenes. The most important clue that led me the right direction was the highlighted access to a CLSID registry key which could not be found by the miisserver.exe:

Image

Using the process monitor to identify a missing registry key

This registry key was not found on the server, so I searched for this ID {3F63C36E-51A3-11D2-BB7D-00C04FA30080} in the Internet and found references to an OLEDB DLL provided by Oracle:

 Image

In the end I discovered that this OLEDB driver was NOT included in the Oracle client Software package. Instead it is included in 64-bit Oracle Data Access Components (ODAC) which can be separately downloaded at the Oracle web site.  This is the 64-bit ODAC 11.2 Release 5 (11.2.0.3.20) for Windows x64 – it contains 64-bit Oracle Provider for OLE DB 11.2.0.3.0.

Having installed the software, the registry key is clearly now available on the server and references the Oracle OLEDB dll – thereby enabling me to create the Oracle Management Agent on my FIM Box:

Oracle DLL registry settings

Oracle DLL registry settings

As it happens, I have a feeling I’ll be needing just this piece of vital info myself in the coming weeks …

Posted in FIM (ForeFront Identity Manager) 2010 | Tagged , | 1 Comment

Uncovering #FIM2010 Service Set Correction Requests

When responding to this FIM forum post tonight it occurred to me that monitoring for and troubleshooting these events is something I’ve probably not rated highly enough on the priority list.

Digging a bit further I stumbled upon this TechNet WIKI article from Markus – and it reinforced the thought that behind every recurring set correction you are likely to uncover a policy design flaw that’s probably going to be a pain in the !@&*#! to track down.  This is right up there with the failed FIM request that occurs when two multiple workflow instances are spawned attempting to concurrently apply the same action on a FIM object – where one succeeds and the other fails with a “denied” exception.  These types of errors are really the hardest to pin down, and it’s why I’m bothering to post about them.

Markus explains a scenario which can cause the set correction condition to occur – I had to read it a couple of times before I understood this.  Maybe you will too – in which case the following variation may help:

The end result of multiple updates for the same FIM resource may well be that the resource satisfies a set criteria, but each not after each request individually.  If requests are processed on a single thread sequentially, then the last request would be expected to cause the criteria to be satisfied.  However, in periods of high volatility and multi-threading, if the individual requests are processed concurrently it is possible for all requests to be fully processed without the set condition having been satisfied.  When this happens set correction is required.

Of course there are going to be other reasons for set corrections being required too – such as exceptions occurring evaluating complex set criteria (particularly when the FIMService database indexes become overly fragmented, or when your criteria is just too complex for FIM to handle).  There is always a trade-off here:

  • reducing the number of set definitions you need at risk of increased complexity (and relying on a defrag/index rebuild regime)
    versus
  • using additional nested sets to simplify individual set criteria but (arguably) reduce solution maintainability and risk running foul of stated best practices in this regard.

Note to self – whenever monitoring the health of the FIM Service, look not only at the exceptions and the failures, but also for the presence of set corrections.  Of course if you have SCOM and the FIM 2010 Management Pack (my customer chooses not to) you will no doubt already have the following in hand:

The key monitoring scenarios covered by this management pack are listed below:

  • End-User Availability
  • Synchronization Service Availability
  • FIM Service and Portal Availability
  • FIM Portal Errors Shown to End Users
  • FIM Portal Configuration Errors
  • FIM Service Internal State
  • FIM Service Set Corrections
  • FIM Service Connectivity with Exchange
  • FIM Synchronization Service Configuration Errors
Posted in FIM (ForeFront Identity Manager) 2010 | Tagged , | 1 Comment

Optional Synchronization Rule Parameters

Recently I needed to extend a simple outbound sync rule (FIM 2010 R1) to provision a business email address to an HR system.  In the target HR system, multiple contact records can be recorded for a user, and under normal conditions a “business” contact was to be set with the exchange email address from AD.  However, in a test environment where “new starter” emails are to be sent from the HR system I didn’t want to use “real” email addresses but a test mailbox instead.

I figured I simply needed a means of overriding an EAF in a sync rule with a constant email address – purely to support my testing needs.  Under normal circumstances there should be no override, so I figured I could use a workflow parameter and only set a value in the test scenario.  The override idea seemed to work well – I could have identical sync policy in each of my DEV/TEST/PROD environments, but this way I could support this testing requirement without having to actually change the sync rule itself.  Test emails were indeed sent to the test mailbox as required.

I set up my EAF in my sync rule like this (CS and MV prefixes for explanatory purposes only):

CS.email = IIF(Eq(Trim($EmailOverride),""),MV.email,$EmailOverride)

It seemed like a perfectly reasonable thing to expect to work – I assumed that if I simply didn’t supply any parameter value when I added the sync rule to the target user object, that the above logic would result in Eq(Trim($EmailOverride) returning a TRUE value.  I was wrong …

I only noticed there was a problem when I removed the override value and noticed that the pending exports subsequently produced had no email address value at all!  This broke my HR exports and indicated that I had a lingering problem with the above EAF.  This was confirmed when I compared the corresponding ERE for two different users – one created when the constant email value was present (which worked), and one when the value was removed (which failed).  What I noticed was that there was only an XML value in the Synchronization Parameter binding on the ERE when there was a value specified on the workflow which attached my sync rule.  When I specified an override email I ended up with this in the SR parameter :

<sync-parameter><name>EmailOverride</name><value>dummy.mailbox@mydomain.com</value></sync-parameter>

… but when there was no value specified, rather than getting this:

<sync-parameter><name>EmailOverride</name><value></value></sync-parameter>

… I actually got no SR parameter at all (i.e. no XML whatsoever).  This was not what I expected, and explained why my EAF wasn’t working.

I then tried each of the following without success:

  • Eq($EmailOverride,Null())
  • IsPresent($EmailOverride)

I finally had to settle for this:

CS.email = IIF(Eq(Trim($EmailOverride),”NONE”),MV.email,$EmailOverride)

and resort to having to specify “NONE” as my default workflow parameter rather than an empty string.

So the upshot of this post is to make the point that (for FIM2010 R1 at least) there is effectively no such concept as an “optional sync rule parameter”.  Why?  Because there doesn’t appear to be a way to successfully test for the (lack of) presence of a value in a parameter.

I would be interested to find out if anyone has observed this same behaviour for R2?

Posted in FIM (ForeFront Identity Manager) 2010 | Tagged , | Leave a comment

MS TechEd 2013

Finally on my way this morning to join friends and colleagues on the annual pilgrimage to the Microsoft tech-fest on Australia’s premier convention location, the sunny Gold Coast. My colleague Carol will be presenting both of the sessions listed against the Forefront Identity Manager technology stream, with her “Identity Jigsaw Puzzle” topic proving the most popular so get in early for that one. It’s not hard to understand why …

Don’t be fooled by the latest biometrics hype – there’s definitely nothing sexy about Identity Management – at least in my experience. This is always reinforced at events like this, where the focus is always on the latest glittering release of Windows Server or Azure technology. Yet when the buzz has died down and you’re faced with pulling it all together, you find you’re faced with the same old challenges we always have – they’ve just morphed into a few new forms.

Get along to Carol’s session to make your own connections between the pieces in your own puzzle, and come and have a chat to either of us. You will find me at our joint OptimalIdM/UNIFY booth this year, so come and introduce yourself and we will no doubt find some common ground on what is often a complex and challenging topic. See how the “Microsoft Stack” has extended well beyond the shrink-wrapped FIM 2010 R2 software.

See you there!

Posted in FIM (ForeFront Identity Manager) 2010 | Tagged | Leave a comment

Error saving the FIM Sync key set

Ran into something unusual just now with the FIM R2 Sync install – at the end of the install after having selected a folder to save the *.bin file to, there was a delay of about 30 seconds before I got the following error dialog:

“The Forefront Identity Manager Synchronization Service setup wizard was unable to back up the key set. <hr=0×80131904> … try again?”

I figured there was a permissions problem with the target folder location, so I tried several times, including creating c:\temp, giving full access permissions to everyone, and saving it there – but it didn’t matter how many times I tried I got the same error.  Eventually I selected “no” and the installer completed with a success status (nothing in the error log or any warning that the key was yet to be saved).

At this point I simply ran the Synchronization Service Key Management program and successfully saved my miiskeys-1.bin file without a problem.

So I still have no idea what the error actually was, but if anyone does have the same experience, rest assured that you can still happily skip this bit so long as you remember to do it straight afterwards using the utility.

Posted in FIM (ForeFront Identity Manager) 2010 | Tagged , | 1 Comment