Tuesday, December 3, 2013

QT015: Use Filter Recurring Nodes to Apply XPath Predicates within Map Activities

Quick Tip #015: Use Filter Recurring Nodes to Apply XPath Predicates within Map Activities

The Cast Iron Map Activity is a powerful and easy to use tool for transforming XML documents.  For most transformations the simple point and click / drag and drop interface for mapping fields and inserting functions is intuitive and for the most part self explanatory.  However, there are a few features that are not very well known and are sometimes forgotten about because they can only be accessed by right click menus.  This article will cover the usage of one such feature, the Filter Recurring Nodes feature.

What is Filter Recurring Nodes?

The Filter Recurring Nodes option, which is accessible by right clicking a recurring node on the target side of a Map, allows you to apply an XPath predicate to the source document as part of the map.  XPath is a language for navigating XML hierarchies, it has a simple syntax and includes various functions for transforming data.  An XPath predicate is a filter that can be applied within an XPath expression to filter nodes that meet the criteria expressed in the XPath predicate.  Predicates can use sub-path expressions functions and comparison operators to identify the nodes that should be included in the filter.

How do I use Filter Recurring Nodes?

The best way to understand how to use Filter Recurring Nodes is with an example.  For this example we will use a common design pattern when working with the Salesforce.com Connector.  When working with the Salesforce.com Connector you may have noticed that in order to determine whether or not your operation completed successfully you must check the results output parameter of the Salesforce.com Activity.  The results output parameter is an XML document which contains a result element for each sObject passed to the input parameter of the connector.  Each result contains a boolean flag indicating whether or not the operation on the associated record was successful.  (Results are outputted in the same order as the input data)  Assuming you want to report on all the records that errored, you will need to collect those records and write the error messages to a database or send them in an email.
Without filter recurring nodes you might use a for-each loop on the results object with an if-then and an expand occurrences map.  This is a very inefficient way to collect these records (expand occurrences maps are very inefficient), also your orchestration would be cluttered with a lot of unnecessary activities which increases the potential for mistakes and hinders readability.  Fortunately, filter recurring nodes provides an efficient and compact way to accomplish this goal.
First, we will create a Salesforce.com Upsert Activity and go to the Map Outputs Pane.  Create two variables based on the results output parameter of the activity and name them successes and failures.  Add the newly created variables to the map.  Now you can drag the recurring result node of the output parameter to the recurring result node of both the successes and failures variables on the target side of the map.  Next right click on the result recurring node of the successes variable and choose Filter Recurring Nodes.  This will open the filter recurring nodes dialog.  You will see an XPath expression with an empty predicate (/results/result[]).  Now you can fill in the predicate to complete the expression and filter for only nodes where success equals true.  To do this enter the following text into the box:
*:success = true()  
Repeat this filter recurring nodes step for the failures variable and use the XPath predicate:
*:success = false()

That's it you now have a variable called successes that contains all the success records and a variable called failures which contains all the errors.  The failures variable can now be transformed to an email message or logged to a database, etc.

How do I know if Filter Recurring Nodes has been Applied?

When reviewing an orchestration, especially one created by another developer, it is important to understand whether or not this feature is being used and which XPath predicate has been applied.  When a map contains a Filter Recurring Nodes condition, an icon is displayed next to the recurring node where it was applied.  You may also hover over the the node to see the XPath predicate that was used.  See the screenshot below, the Filter Recurring Nodes icon and the predicate are highlighted in green.

Notes on XPath predicates

You may have noticed a couple things about the predicates above.  First, the *: before the success fieldname.  This is a namespace wildcard and is a common idiom in XPath expressions such as this because it is often difficult to know which namespace prefixes have been declared.  Be careful when using this wildcard that you do not happen to have two fields in your document with the same name and different prefixes.  Second, the use of the true() function instead of a literal true.  XPath does not reserve a true keyword because XML does not reserve true as a keyword.  Therefore the literal string true would match to a node named true rather than the boolean value.  To get around this limitation XPath provides the true() and false() functions which return their respective boolean values.

Further Reading

In order to use this feature effectively you will want to have a good background in XPath predicates the following resources may be helpful in understanding XPath:


  1. I have read your blog, it was good to read & I am getting some useful info's through your blog keep sharing... Informatica is an ETL tools helps to transform your old business leads into new vision. Learn
    Informatica training in chennai from corporate professionals with very good experience in informatica tool.
    Best Informatica Training In Chennai|Informatica courses in Chennai

  2. The data warehouse concept you have furnished above was very useful to me. Thanks for sharing.
    Informatica training in chennai|Best Informatica Training In Chennai|Informatica training center in Chennai