Thursday, February 28, 2013

CIOS 6.3.0.1 Is Now Available


Cast Iron Version 6.3.0.1

IBM has released a new version of CIOS version 6.3.0.1 was released on 02/28/2013.  See the Release Notes for more information.

What do you need to know about this Release?

This is a FixPack release, it rolls up the prior iFix releases including a security vulnerability for those customers using LDAP.  This release also contains some enhancements to connectors, if you are experiencing timeouts with the Netsuite connector there is a fix for that.  Also, there is an enhancement for the Domino Connector to allow you to specify the database as a configuration parameter. Finally, there is also an important fix for an issue that may cause the WMC to become inaccessible when the disk fills up.

If you are experiencing any of those issues with 6.3.0.0 or would like to try out the enhancements to the domino connector you can get the new release here.  For more information on minor version upgrades see this post.

Monday, February 25, 2013

QT004: Cast Iron Hypervisor Edition

Quick Tip 004: Installing IBM WebSphere Cast Iron Hypervisor Edition

In this post we will cover the steps necessary to configure VMWare for Cast Iron Hypervisor Edition.

Installing ESXi Server

In order to run Cast Iron Hypervisor edition you will need a physical machine running ESX or ESXi version 4.0.0, Build 164009 or later.  The ESXi software runs on a variety of platforms and is installed as the base operating system for the server, there is no need to install Linux or Windows.  If you do have an HA requirement, all you need is the ESXi Server to run Cast Iron Hypervisor Edition.  The installation process is extremely simple, download the version that corresponds to your hardware insert your bootable media into the machine and the installation will only ask you a few simple questions before installing and rebooting the machine.  Once the software is installed configure the network parameters and download vSphere Client.  vSphere Client is the tool which you will use to deploy the CIOS Virtual Machine and vSphere client is available as a download once you have installed ESXi server by navigating to the url presented at the end of the setup process.

Installing vCenter Server

If you have a requirement for High Availability and want to create a cluster of ESXi nodes, you will need to install vCenter Server. An ESXi Cluster is a set of ESXi servers that run in tandem and allow you to migrate VMs between nodes manually or automatically even while they are running.  vCenter Server is also very easy to install,  for small deployments you can even run vCenter Server as a virtual machine deployed to your ESXi Cluster.  vCenter Server is available for installation on top of Windows Server or as a linux based Virtual Appliance.

vMotion, vSphere HA, and Fault Tolerance

vMotion is the tool for migrating VMs from one node to another while they are running with no perceivable interruption to other systems.  vMotion will copy the memory of your system while it is running to another server and then pause the system long enough to capture a delta of the memory changes that happened during the copy process and then resume the machine on the new server.  vMotion will also take care of the networking changes in a way that is transparent above layer 2, so even network connections are not interrupted by vMotion.  vSphere HA is a technology that monitors servers virtual machines and even applications running on your VMs for failures and responds to failures by restarting VMs or moving them to another node in your cluster.  Fault Tolerance provides another level service that actually provides zero downtime by continuously replicating the state of your VM to a slave instance on another server, if the the master instance fails the slave will take over with virtually no interruptions.  The fault tolerance architecture most closely represents the level of service that the Cast Iron hardware based High Availability Pair is designed to provide.

Server and Network Requirements for vMotion, vSphere HA, and Fault Tolerance

In addition to the networks required to support your Virtual Machines you will need a separate physical network for vMotion and HA, and if you intend to use Fault Tolerance a separate set of NICs should be used for Fault Tolerance.  VMWare allows for redundant NICs so, if you want to use vMotion, vSphere HA and Fault Tolerance you will want a minimum of 8 physical NICs to create 4 separate networks (VM Management, VM Kernal for vMotion, Fault Tolerance, and finally the network for your virtual machines) 

Installing Cast Iron

Cast Iron Hypervisor Edition is distributed for VMWare as an OVA file which can be loaded using vSphere Client.  Follow these instructions in the Cast Iron documentation on how to load the OVA file.  For information on the 6.3.0.0 release see this post.  You will most likely want to apply the latest patches further information can be found here.  (Note: As of the time of this post, there appears to be a problem with the download for the latest iFix for CIOS 6.3.0.0, it appears that the Hypervisor Edition patch is actually for a physical appliance, you will likely have to contact IBM Support in order to obtain the latest patch.)

Monday, February 18, 2013

QT003: Migrating from a Pre-IBM Version of CIOS

Quick Tip 003: Migrating from a Pre-IBM Version of CIOS

In this post we will review some of things that you need to know if you are moving from a Dell based Cast Iron Solution to the latest IBM WebSphere Cast Iron.  We will cover some differences in terminology as well as the new choices available in the latest version of Cast Iron.  First, if you are still running a legacy version of CIOS and are looking to upgrade, there are a couple of new alternatives to the standard hardware based appliance:
Hypervisor Edition:  IBM Cast Iron offers a virtual appliance that is designed to run on VMWare or the open source Xen virtualization platform.  The Hypervisor Edition is a good solution for customers who are already making an investment in virtualization, it provides a simple solution for high availability as well.
Cast Iron Live: IBM Cast Iron also offers a cloud based solution for customers who don't want or need an on premise solution.  Cast Iron live does support a hybrid cloud solution with the Cast Iron Secure connector which is an agent that users can run within there private network to allow Cast Iron live to interact with local databases, web services, CRMs etc.
DataPower Appliance: The most direct upgrade path is the XH40 DataPower appliance which is the new IBM hardware based solution.  It works much the way the Dell based product worked, its a physical box that is for the most part a direct replacement for the Dell based solution.  HA Pair configurations continue to be supported by the DataPower appliances as well.

Upgrading Orchestrations

Cast Iron Studio automatically performs upgrades when you open a project from a prior version of CIOS.  Although its not strictly required, we do recommend stepping through major version upgrades to ensure maximum compatibility.  For example, when going from say version 4.5 to version 6.1, you can open and save the project in the latest version 5 studio before moving to version 6.1.
Some new features may not be available in upgraded projects until you refresh your connectors.  For example, the Salesforce connector was upgraded in version 6 to allow foreign key upserts.  For upgraded projects, you will not see the drop down for external id fields until you refresh the connector schema in studio.  This is because studio needs to connect to salesforce to discover and leverage external id relationships.

Going Cloud and Using the Secure Connector

Cast Iron Live is the cloud offering from IBM Cast Iron.  For the most part, projects and orchestrations built for other versions of Cast Iron are compatible with the cloud version.  The main exception being the obvious fact that on premise systems cannot be directly accessed from the cloud.  In order to connect to on premise systems, you will want to install the Cast Iron Secure connector.  (We will cover more about the Secure Connector, how to install it and how it works in a future post.)  Connector operations work much the same way in a hybrid Cloud / On Premise solution you simply need to tell the connector which secure connector to use when connecting to the resource.

Web Services in the Cloud

Cast Iron Live provides several new ways of exposing web services:
  • you can provide web services to your internal private network via the secure connector
  • you can also provide external web services authenticated using a query string in the Url. ( this will be covered in more detail in a later post on using Salesforce Outbound Messaging with Cast Iron Live.)
  • webapi.castiron.com is the latest tool from IBM Cast Iron for exposing web services to third parties. ( This will also be covered in more detail in a later post.)

Hypervisor Edition and High Availability

If you are migrating from a High Availability Pair you may want to consider the new virtual appliance as it is fully compatible with VMWare vCenter High Availability.  With HA and vMotion it is possible to load balance and provide high availability in your virtualization cluster.  VMWare High Availability continuously monitors both host and guest operating systems in a virtualized environment and will detect failures in real time and move virtual machines to a healthy node in your cluster.  CIOS also provides a persistence layer which allows it to seamlessly recover in the event of such a failure.  When VMWare moves the CI appliance to a new node in the cluster, CIOS will attempt to resume all jobs from their previous state. Do note that ability to recover in flight transactions does require support from the endpoint and varies by connector. It is therefore advisable to ensue that interactions with your endpoints are idempotent because if a transaction is in process when a fail over occurs it will likely be restarted.  (We will cover configuring VMWare for the CI Hypervisor Edition in a future post.)

Notes for Power Users

If you have ever used the the system shell command on the Dell based appliance you may be disappointed to know that you are no longer allowed to access the linux shell.  There are some new commands in the CLI to access the system logs, however for security reasons, there is no longer access to the shell.  See the online help for the CLI for more details on the new CLI commands for Debugging.
Also, somewhere along the way, there was a change in CIOS to use the internal persistence layer as a paging store for large documents, it is therefore not advisable to turn persistence off for any orchestrations unless you really understand the consequences.  If you had been running with persistence off in a legacy version, you will want to turn it back on when upgrading to avoid potentially dramatic performance degradation.

Monday, February 11, 2013

FR001: Support for RESTful Web Services and JSON in CIOS 6.3.0.0

Feature Review #001: Support for RESTful Web Services and JSON in CIOS 6.3.0.0 

Starting with CIOS Version 6.3.0.0 IBM Cast Iron has expanded support for RESTful web services including new activities for parsing the JSON message format.  Web APIs built around RESTful architectural patterns are becoming more and more prevalent in Cloud Based IT Systems.

First a Little bit of Background on REST and JSON

REST

REpresentational State Transfer (REST) is a concept that was laid out by Roy Fielding one of the authors of HTTP in his Ph.D. dissertation.  It is the now ubiquitous architectural pattern that the modern Internet is built around.  Chapter 5 of Fielding's dissertation is the best resource for the clinical definition of REST.  However, REST is an architectural pattern not a protocol and the REST architectural pattern is not limited to the modern interpretation of a RESTful web service.

JSON

JavaScript Object Notation is a message format that is derived from the JavaScript syntax for defining objects.  JSON is often used in Web APIs because it is a lightweight human readable alternative to XML which does not require a parser to be interpreted in JavaScript.  Because of those properties it is a very popular message format for Web APIs that are intended to be called from a web browser.

Modern RESTful Web APIs

Whereas SOAP based Web Services strive for interoperability through rigidity, RESTful Web APIs strive for interoperability through simplicity.  Rather than using a complex well defined protocol like SOAP, RESTful Web APIs use simple message formats based on XML or JSON that are exchanged using standard HTTP methods.  This lightweight simplicity grew from necessity, modern Web APIs evolved as a means for building web applications in JavaScript,  so they can easily be called from web browsers and mobile platforms.  Because RESTful APIs use simple HTTP requests and self describing data formats (typically XML or JSON) and because there is as of yet no formal standard for documenting these APIs, working with RESTful APIs is often an autodidactic experience.  If you need to know what fields a call will return you can make the call and see what comes out, etc.


SOAP based Web Service are typically implemented on top of of the HTTP protocol (There are some platforms that support SOAP over JMS or even SMTP, but by nature they are a level on top of the underlying transport).  RESTful Web APIs are implemented at the HTTP layer rather than on top of it.  They allow clients to interact with the server using the features if HTTP, including HTTP methods (GET,  PUT,  DELETE,  POST, HEAD, OPTIONS, etc.), URIs, query strings, headers, etc.
  • URI: Web APIs generally use Uniform Resource Identifiers (URI) to identify resources that a client can interact with.  Addresses are typically hierarchical and are used to identify a class of resources with its name, and a single entity underneath it by its unique identifier.  For example you my have something like /books/{isbn}, or even a deeper hierarchy /library/branches/{branch-id}/books/{isbn}, where branch-id uniquely identifies a branch within a library system and isbn uniquely identifies a book.
  • Methods: Generally, the actions that can be performed on a resource are mapped to HTTP methods.  GET to retrieve a resource, PUT or POST to create or change a resource, DELETE to delete it, etc.  See RFC2616 Section 9 for more details on HTTP/1.1 methods. 
  • Query Strings: Query strings are the standard way to pass parameters in HTTP GET requests.  A query string is the part of the URL between the ? and either a # or the end of the URL.  And typically take the form name=value&name2=value2.  Values passed via query string must be URL encoded.  See RFC3986 for more details on URLs. 

What's New in Version 6.3?

Version 6.3 includes several enhancements to existing activities as well as new activities to better support RESTful Web APIs. 

New Transform Activities for JSON


Read JSON Activity

The Read JSON Activity parses JSON input and converts it to xml.  This new activity works much the same way the Read XML or Read Flat File Activities work.  You select a schema and then you map a flat string into the Map Inputs, the Map Outputs will come in the form that you specified.  For the Read JSON activity you also have the option of specifying a sample JSON or XML message and the activity will determine whether your have specified JSON or XML, parse the sample message and learn the schema.  See below for an example of specifying a sample JSON message:


The sample message functionality also supports sample XML messages.

Write JSON Activity

The write JSON activity converts XML to the JSON format.  Much like the Read JSON activity you can specify either an existing schema in your project or a sample message.  Below is an example of an XML sample message:

New HTTP Activities


Invoke Activities

As you can see above there are a lot more HTTP activities than there were before.  These new activities are not fundamentally different than there counterparts in previous versions of Cast Iron.  However, the new activities retain the functionality that was there before while adding new features that make writing RESTful Web APIs much easier.  The Invoke Request activity is the general HTTP Request activity and functions much like the old HTTP Post Request Activity with some usability enhancements for path parameters, query strings, and HTTP Headers, see the Receive Request below for more details on what the changes look like and how they are used.  The Get Request, Post Request, Head Request, Put Request, and Delete Request all function similarly to Invoke Request, however there is no need to specify the HTTP method as it is implied by the activity name.

Receive Request

The Receive Request activity has been updated to more easily support RESTful Web APIs.  Most of the functionality was there already but there have been a number of usability enhancements that make it easier to support things like path parameters, query strings, and HTTP Headers.  See the screenshots below for examples of the changes and how they are used:

The new configure pane allows users to specify path parameters and query string values as well as allowing the user to choose which methods this activity will listen on.  Here we specify a path parameter isbn, as well as the query parameters title and year.
The Request Headers Pane allows users to specify headers to give easier access to the headers  that clients specify.  Here we add two headers, username and token, which can be specified by clients calling this service.


Finally, putting it all together you can see that the  username and token headers are exposed in the httpheaders section, the isbn path parameter is exposed in the new pathParameters node and the query parameters title and year are exposed in the queryParameters node.  There is no longer a need to write javascript to parse query strings, or xpath expressions to search for headers.

Send Response

The Send Response activity functions much as it did before, with some usability enhancements for setting response headers.

Putting it all Together

These new activities are not major changes to how Cast Iron works, but all in all they provide much needed functionality for RESTful Web APIs.  The HTTP enhancements provide huge usability upgrades that make it much faster and easier to deal with path parameters, query strings, and HTTP headers and methods.  The Read and Write JSON activities supply the necessary tools for dealing with the JSON format which along with XML are the standard message formats in RESTful Web APIs.  All of these small improvements lead to a long awaited major gain in the ability to call and and provide Web APIs.

Monday, February 4, 2013

QT002: CIOS Minor Version Upgrades

Quick Tip 002: CIOS Minor Version Upgrades

The process for performing minor upgrades to your Cast Iron environment is fairly straight-forward, but our customers often have trouble navigating the IBM site to find appropriate files and instructions.

Before you get Started

It is important to note before starting that even minor upgrades can contain regressions or fixes to features that may impact the behavior of your integrations.  You should treat any upgrade seriously and due diligence requires regression testing of upgrades in your test environment before applying them to your production system.  Also, note that your version of Studio must be in sync with your appliance version.  If you upgrade your appliance you should also install the corresponding version of studio.  Projects are automatically updated when you open them in a new version of studio, however, you should archive the old version because they cannot be downgraded.

Upgrades are Platform Specific

First you must determine which Platform you are using, the upgrade process is different depending on whether you have a HyperVisor Edition or a physical appliance or HA pair.  If you have an old Dell appliance, you may need to upgrade your hardware as well.

Cast Iron® Cast Iron XH40 Integration Appliance

  • First consult the instructions appropriate to your version of CIOS on upgrades.  Direct Link for Version 6.3.0.0
  • Download the appropriate patch from IBM at the Cast Iron Cloud Integration Support Home.  
    • From the Cast Iron Support home you can either click the "Fixes (downloads)" link and specify the version you would like or click the availability announcement for the desired version which you will usually find in the featured links.  If you choose the "Fixes (downloads)" link you will be taken directly to a list of files.  If you choose the availability announcement you will be taken to a page with more information about the release such as release notes as well as a link to a page with direct download links towards the bottom of the overview tab.  Note: If you choose the second option the links provided will be for fix packs, if you want to install an interim fix you must choose option one to list all downloads for a release.  See below for more details on Fix Packs and Interim Fixes.
    • For the Cast Iron XH40 Integration Appliance patches end with a .scrypt2 extension.  See Below for more information on the different files available for download.

WebSphere Cast Iron® HyperVisor Edition

  • First consult the instructions appropriate to your version of CIOS on upgrades.  Direct Link for Version 6.3.0.0
  • Download the appropriate patch from IBM at the Cast Iron Cloud Integration Support Home.  (The above instructions refer you to fix central, but the patches are easier to find from CI Support home)
    • From the Cast Iron Support home you can either click the "Fixes (downloads)" link and specify the version you would like or click the availability announcement for the desired version which you will usually find in the featured links.  If you choose the "Fixes (downloads)" link you will be taken directly to a list of files.  If you choose the availability announcement you will be taken to a page with more information about the release such as release notes as well as a page with direct download links.  Note: If you choose the second option the links provided will be for fix packs, if you want to install an interim fix you must choose option one to list all downloads for a release.  See below for more details on Fix Packs and Interim Fixes.
    • For HyperVisor Edition patches end with a .vcrypt2 extension.  See Below for more information on the different files available for download.

Which File do I Need?

Once you navigate to the downloads page there are several files available below is brief overview of available files.

Fix Packs Vs Interim Fixes

Both Fix Packs and Interim Fixes are cumulative for upgrades, they incorporate all previous fixes.  The difference is, a fix pack is a scheduled release, whereas an interim fix is the most up to date version that contains fixes incorporated between releases.  The interim fix is mostly to incorporate critical fixes for specific customers.

File Types

There are four different file types:
  1. *.studio:  This is the Windows installer for Cast Iron Studio.
  2. *.scrypt2:  This is the patch upgrade for the Cast Iron XH40 Integration Appliance.  (Hardware Appliance)
  3. *.vcrypt2: This is the patch upgrade for the Cast Iron HyperVisor Edition.  (Hardware Appliance) 
  4. *.ova:  This is the fresh install for VMWare, .ova is an Open Virtual Appliance file for installation on VMWare ESX.  For more info on the VMWare install go here.
  5. *-xen:  This is the fresh install file for the Xen virtualization environment.  For more info on the Xen Install go here.

Additional Resources

IBM WebSphere Cast Iron planned maintenance release dates
WebSphere Cast Iron Support Lifecycle