FUSE ESB (4)


Accessing RPC/Encoded Web services in Servicemix / Fuse ESB

If there’s one issue I’ve found with ServiceMix, it’s that it tends to enthusiastically embrace a certain way of doing things while letting other methods fall to the wayside.  A prominent example of this is when constructing code to access web services.  By default, Servicemix uses Apache CXF as its web service layer.  CXF is a thoroughly modern framework for constructing web service clients and services, but because it’s so modern it doesn’t support some of the older web services definitions.  A particular example is ‘RPC/Encoded’ web services. If you’re not sure what an RPC/Encoded web service is, but you’ve run into the dreaded ‘rpc encoded wsdls are not supported in jaxws’ error message that is reported when you try to use CXF in this case, there’s a good (if rather dry) explanation of the various types of web services here but all you really need to know is that most people moved away from RPC/Encoded services a while ago, and most modern frameworks are dropping support for this style.  This is all well and good, but what should you do if you need to access a piece of 3rd party software using this style of web service?  CXF doesn’t support it at all, and servicemix doesn’t offer an alternative web framework that does support it.  If you google this issue, you find a load of posts form mailing groups and forums discussing it, but none offering a real solution.

Well, as with most things relating to servicemix there’s a couple of ways of approaching this.

The first, and most time consuming is to simply construct XML Requests and / or responses yourself and then slot your data in before sending it off to the remote endpoint using CXF.  SoapUI is useful here to work out what your XML should look like, and you can use the camel-freemarker component to add in your data and make requests.  For responses, you need some way of parsing the xml that comes back and extracting the relevant data.  If you only have a few requests / responses to make, this is probably a good method.  Once you start adding more requests / responses though it very quickly becomes unwieldy and time consuming to write the parsers and request xml templates for this.

So, what’s the better way of doing this?  Well, before CXF came along there was another Java based web services framework called Axis.  And, fortunately for us, Axis does support RCP/Encoded web services.  However, there are some issues with getting Axis support into servicemix.  Firstly, it’s not supported at all.  There’s no nice FuseSource provided packages available and, because Axis makes use of some of the same wiring as CXF, having Axis present in your OSGI environment can cause all kinds of issues with packages using Axis dependencies instead of CXF dependencies.

Luckily for you, I’ve already done the work to get this working, and All you need to do is copy the methodology below for a nice working RCP/Encoded client API.

For ease of use, I’ve split my project into two separate areas.  A ‘generator’ bundle, which produces our Axis API code, and an ‘API’ bundle which makes use of that code.

Here’s the POM.xml for the Generator bundle:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <groupId>com.tall-paul</groupId>
 <artifactId>my-rpc-generator</artifactId>
 <packaging>pom</packaging>
 <version>1.0-SNAPSHOT</version>
 <name>${project.artifactId}</name>
 <properties>
 <camel-version>2.9.0</camel-version>
 <activemq-camel-version>5.5.1</activemq-camel-version>
 <servicemix-version>2011.02</servicemix-version>
 <maven-bundle-plugin-version>2.3.4</maven-bundle-plugin-version>
 <build-plugins-plugin-version>2.5</build-plugins-plugin-version>
 <build-helper-plugin-version>1.7</build-helper-plugin-version>
 <maven-resources-plugin-version>2.5</maven-resources-plugin-version>
 <maven-compiler-plugin-version>2.3.2</maven-compiler-plugin-version>
 <osgi-import-package>*,org.apache.camel.osgi</osgi-import-package>
 <osgi-export-package/>
 </properties>
 <dependencies>
 <dependency>
 <groupId>org.apache.axis</groupId>
 <artifactId>axis</artifactId>
 <version>1.4</version>
 </dependency>
 <dependency>
 <groupId>org.apache.axis</groupId>
 <artifactId>axis-jaxrpc</artifactId>
 <version>1.4</version>
 </dependency>
 <dependency>
 <groupId>org.apache.axis</groupId>
 <artifactId>axis-saaj</artifactId>
 <version>1.4</version>
 </dependency>
 <dependency>
 <groupId>axis</groupId>
 <artifactId>axis-wsdl4j</artifactId>
 <version>1.5.1</version>
 </dependency>
 <dependency>
 <groupId>commons-discovery</groupId>
 <artifactId>commons-discovery</artifactId>
 <version>0.4</version>
 </dependency>
 <dependency>
 <groupId>log4j</groupId>
 <artifactId>log4j</artifactId>
 <version>1.2.14</version>
 </dependency>
 </dependencies>
 <build>
 <pluginManagement>
 <plugins>
 <!-- used to generate the MANIFEST-FILE of a bundle -->
 <plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-compiler-plugin</artifactId>
 <configuration>
 <source>1.5</source>
 <target>1.5</target>
 </configuration>
 </plugin>
 <plugin>
 <groupId>org.codehaus.mojo</groupId>
 <artifactId>axistools-maven-plugin</artifactId>
 <configuration>
 <sourceDirectory>src/main/resources/</sourceDirectory>
 <packageSpace>my.package.namespace</packageSpace>
 <testCases>false</testCases>
 <serverSide>true</serverSide>
 <subPackageByFileName>false</subPackageByFileName>
 </configuration>
 <executions>
 <execution>
 <goals>
 <goal>wsdl2java</goal>
 </goals>
 </execution>
 </executions>
 </plugin>
 </plugins>
 </pluginManagement>
 </build>
</project>

There’s a couple of things you’ll need to change here for your project.  As well as the usual maven artifactID etc, you’ll need to change the ‘packageSpace’ element to reflect what package you want the generated code to fall under.  Your RPC/Encoded wsdl needs to go in your src/main/resources directory.  Do a maven build, and you should find a load of generated code in your target directory (yay!).  Now we have to actually do something with that code.  Here, I copy the generated code into my ‘Api’ project, the pom for which looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.tall-paul</groupId>
  <artifactId>my-rpc-api</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>bundle</packaging>
  <name>my-rpc-api</name>
  <description>My RPC Api</description>
  <properties>
    <camel-version>2.9.0</camel-version>
    <activemq-camel-version>5.5.1</activemq-camel-version>
    <servicemix-version>2011.02</servicemix-version>
    <maven-bundle-plugin-version>2.3.4</maven-bundle-plugin-version>
    <build-plugins-plugin-version>2.5</build-plugins-plugin-version>
    <build-helper-plugin-version>1.7</build-helper-plugin-version>
    <maven-resources-plugin-version>2.5</maven-resources-plugin-version>
    <maven-compiler-plugin-version>2.3.2</maven-compiler-plugin-version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.apache.axis</groupId>
      <artifactId>axis</artifactId>
      <version>1.4</version>
    </dependency>
    <dependency>
      <groupId>org.apache.axis</groupId>
      <artifactId>axis-jaxrpc</artifactId>
      <version>1.4</version>
    </dependency>
    <dependency>
      <groupId>org.apache.axis</groupId>
      <artifactId>axis-saaj</artifactId>
      <version>1.4</version>
    </dependency>
    <dependency>
      <groupId>javax.xml</groupId>
      <artifactId>jaxrpc-api-osgi</artifactId>
      <version>1.1-b01</version>
    </dependency>
    <dependency>
      <groupId>axis</groupId>
      <artifactId>axis-wsdl4j</artifactId>
      <version>1.5.1</version>
    </dependency>
    <dependency>
      <groupId>commons-discovery</groupId>
      <artifactId>commons-discovery</artifactId>
      <version>0.4</version>
    </dependency>
    <dependency>
      <groupId>commons-logging</groupId>
      <artifactId>commons-logging</artifactId>
      <version>1.0.2</version>
    </dependency>
    <dependency>
      <groupId>javax.xml</groupId>
      <artifactId>jaxrpc-api</artifactId>
      <version>1.1</version>
    </dependency>
    <dependency>
      <groupId>log4j</groupId>
      <artifactId>log4j</artifactId>
      <version>1.2.14</version>
    </dependency>   
  </dependencies>
  <build>
    <defaultGoal>install</defaultGoal>
    <plugins>
      <plugin>
        <groupId>org.apache.felix</groupId>
        <artifactId>maven-bundle-plugin</artifactId>
        <version>${maven-bundle-plugin-version}</version>
        <extensions>true</extensions>
        <configuration>
          <instructions>
            <Bundle-SymbolicName>TESTING</Bundle-SymbolicName>
            <Export-Package>my-package-namespace,javax.xml.rpc</Export-Package>
            <Embed-Dependency>
							axis;scope=compile|runtime,
							axis-jaxrpc;scope=compile|runtime,
							axis-saaj;scope=compile|runtime,
							jaxrpc-api-osgi;
							scope=compile|runtime,
							axis-wsdl4j;scope=compile|runtime,
							commons-discovery;scope=compile|runtime,
							jaxrpc-api;scope=compile|runtime,
							log4j;scope=compile|runtime,
							commons-logging;scope=compile|runtime
						</Embed-Dependency>
            <Embed-Transitive>true</Embed-Transitive>
            <Bundle-ClassPath>.,{maven-dependencies}</Bundle-ClassPath>
            <Import-Package>
							*;resolution:=optional,
							com.sun.jdmk.comm;resolution:=optional,
							com.sun.jimi.core;resolution:=optional,
							com.sun.net.ssl;resolution:=optional,
							com.sun.net.ssl.internal.ssl;resolution:=optional,
							org.apache.bsf;resolution:=optional,
							org.apache.log;resolution:=optional,
							sun.awt.image.codec;resolution:=optional,
							sun.security.provider;resolution:=optional,
							org.exolab.castor.xml;resolution:=optional,
							org.apache.camel.osgi,
							</Import-Package>
          </instructions>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${maven-compiler-plugin-version}</version>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

 

The real magic here is the ‘Embed-Depency’ element, which ensures the Jars referenced here are downloaded and bundled into our project, rather than being imported from the OSGI container at runtime.

And that’s pretty much it, you can now use the generated axis code in much the same way as similar CXF code.




FUSE ESB Development: the build environment

The first thing we need to do when starting to work with fuse ESB is set up our build environment.  For this we’ll be using Eclipse, with the FUSE IDE plugin and maven. I’ll also cover downloading and installing Servicemix, and then generating our first FUSE project.

I’ll be using a windows box for the majority of this tutorial, but it should be fairly straightforward to get this working on your favoured flavour of *nix if you’re so inclined.

There’s a lot to get through here so lets get started.

Maven.

Download the latest version of maven from here: http://maven.apache.org/download.html

Extract the downloaded zip somewhere sensible (I use c:\program files\maven)

To make sure we can use maven from the command line we need to add it to our ‘Path’ environment variable.  On windows 7  you can do this by right clicking ‘Computer’, selecting ‘advanced system properties’ then click the ‘environment variables’ button.  Once the box opens scroll down until you find ‘Path’ under system variables, click edit and add ‘c:\program files\maven\bin’ to your string.

That’s maven setup, lets move onto something a little more challenging.

JDK installation

Before we can install our FUSE products we need to grab the Java Development Kit (JDK) and ensure that Eclipse and servicemix know where to find it by setting our ‘JAVA_HOME’ environment variable.

grab the JDK from here: http://www.oracle.com/technetwork/java/javase/downloads/jdk-6u25-download-346242.html and run the installer.

Once it’s installed you need to go back into the ‘Environment variables’ box we were in during the Maven installation and add a new variable called ‘JAVA_HOME’ pointing at the Directory your just installed the JDK into (by default it should be something like ‘C:\Program Files\Java\jdk1.6.0_25’).

Eclipse

Download Eclipse Helios from here: http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/heliossr2

Personally I’ve had no problems using the later ‘Indigo’ version of eclipse, but FUSE recommend Helios for working with their IDE so we’ll stick with that.  Run the installer as usual.  Once it’s installed we need to get the FUSE IDE plugin.

Go here: http://fusesource.com/products/fuse-ide/ to download it.  You need to register as part of the ‘Fuse community’, then add the update site listed on that page to your eclipse installation to add the IDE plugin.

Once that’s done the final part of the Eclipse installation is to make Eclipse use our previously installed maven repository rather than the one that comes bundled with Eclipse.  In the Eclipse main menu open ‘Window’->’Preferences’.  Then select ‘Maven’->’Installations’.  Click ‘Add’ and browse to your maven directory (in my case it’s ‘c:\program files\maven’).  Click OK and you’re done.

ServiceMix

For our ServiceMix installation I’m going to use the packages provided by FuseSource as ‘Fuse ESB#.  You can get the latest version from here: http://fusesource.com/products/enterprise-servicemix/.

Download the latest Zip file and extract it somewhere obvious (c:\servicemix for example).

And…that’s pretty much it, you should now be able to start Servicemix from the command line by changing to the servicemix directory and running the command ‘bin\servicemix.bat‘.  If everything goes well you should see something like this:

Generating our first project

Now everything’s setup lets try generating and deploying a simple test project.

To generate our project structure we can use the maven archetype generator as follows:

First create a new, empty directory where we’ll be working (I’m using c:\testproject)

Next run the command ‘mvn archetype:generate‘. this will prompt maven to download the latest list of archetypes and display them as a huge list.  Luckily we can filter this list if we know the name of the archetype we want. Type ‘karaf-blueprint-archetype‘ to return our filtered list, which should have 1 entry.  press 1 to select this entry and then from the list of versions that follows select the latest one (at time of writing this is 2.2.7).

You’ll now be prompted for some information about the project, use the following (or replace with your own values as required):

Once you’ve entered the details a simple ‘Y’ will confirm them and generate our project.

Import project into eclipse

The next stage is to import our project into Eclipse.  In the Eclipse main menu select ‘File’->’Import’, then from the import box scroll down and select ‘Maven’->’Existing Maven Projects’.

In the Import window browse to the location of your test project (notice maven has generated it in a subdirectory of our root directory) as shown:

Click ‘Finish’ to import the project.  You may get a ‘NullPointerException’ here for some reason, but the project should still import.

Configure a ‘Hello World’ Route

Now let’s make our project actually do something.

If you open up your project structure, you should see something like this:

There’s a few files and folders here to take note of, and we’ll go into these in more detail in further installments of this series, but for now let’s just jump in and start hacking around.

First of all we can remove the sample java beans that Maven gives us, so delete the two files under your java class path.  In my case this is ‘test-project\src\main\java\uk\co\tall-paul’.

We can also get rid of the sample camel context (in the file ‘my-service.xml’) and replace it with our own.  So remove that file and create a new one in the same place called ‘hello-world.xml’ containing the following:

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:camel="http://camel.apache.org/schema/blueprint"
    xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0"
    xmlns:cxf="http://camel.apache.org/schema/blueprint/cxf"
    xsi:schemaLocation="
       http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
       http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/spring/camel-blueprint.xsd">       
	
	<camelContext id="hello-world-context" xmlns="http://camel.apache.org/schema/blueprint">
	
	<route id="hello-world-route">
		<from uri="timer:test?period=5000"/>
		<log message="hello world!"/>
		<to uri="mock:result"/>
	</route>
		
	</camelContext>
</blueprint>

For more info on what that xml actually does have a look at the camel website, and specifically the information abou the various components which you can find here: http://camel.apache.org/components.html

This simple route basically uses a timer to fire off a message every 5 seconds, then logs ‘hello world’ to the console.

Build the Project

To build the project we can run maven from the command line or from within eclipse.

From the command line, navigate to the project folder (c:\testproject\test-project) and run the command ‘mvn clean install‘ (you can find more information about various maven commands here: http://maven.apache.org/maven-1.x/reference/command-line.html , but really ‘mvn clean install’ is all you’ll need for now).  If all goes well you should (eventually) see something like the following:

From Eclipse you can do pretty much the same thing by right clicking on your project and selecting ‘Run as’->’Maven install’ as shown here:

Our project has now been compiled and installed into our maven repository, which you can see if you browse to the location of the repository (by default this is at c:\users\YOURUSERNAME\.m2\repository):

Deploy the project into servicemix

Thanks to the integration of maven into servicemix, deploying our project is trivial using the ‘install’ command withing servicemix.

From your servicemix consolerun the following command:

install mvn:\uk.co.tall-paul\test-project\1.0-SNAPSHOT

Hopefully you should be given a bundle ID, something like ‘bundle ID: 200’.  Now type the following command (substituting the bundle ID you were given for the 200 shown here)

start 200

And that should be it.  As long as you didn’t see any error messages, your bundle should now be running.  To see the output from our route type the following commands (the first clears the log file for clarity, the second displays the current log)

log:clear
log:tail

after a few seconds you should see something like this:

You can see that every 5 seconds our timer is going off and logging our message to the console. You can also see here the name of the component the message originated from (remember we put ‘test’ as the first parameter for our timer component) and the route in which the message is running (here it’s ‘hello-world=route’), which indicates the importance of naming your routes and components sensibly.

Conclusion

Hopefully this brief tutorial has shown how easy it is to get up and running with Fuse ESB development.  Next time we’ll look at how to use the environment we’ve set up here to do some real integration work.




FUSE ESB Development Introduction

Today I’ve decided to move on from the magento stuff I’ve been posting recently, and start looking at working with FUSE ESB.  This is an enterprise service bus based on apaches servicemix platform.

There’s a lot of new stuff here, the majority of the code for servicemix is written in Java and plugged together using the Blueprint XML language and apache camel.  For the build environment we’ll be using apache maven, eclipse and subversion (for source control).  Over the coming days (weeks) I’ll describe how to set all this up from scratch and develop a web services API that will allow you to access data from a lot of different systems using one interface.  I’d never used Java or Maven before getting started on this project, so that should be a good indicatoof how easy this stuff is to pick up.

For more info on what fuse ESB is, and what it does you should go and read my post from a few weeks ago: Overview of FUSE ESB.  As a quick summary the idea of an ESB is to allow various systems to interoperate in a standard, maintainable way.  There’s a lot more to it than that of course, but that’s the basic idea.

As a starting point for this series of tutorials, lets have a look at what we’ll be covering (subject to change, I’ll update this list if I change things around)

1: Setting up the build environment

1a: downloads

1b: maven setup

1c: subversion setup

1d: eclipse setup

1e: servicemix / FUSE ESB setup

1f: creating our project structure

2: Defining our APIs

2a: creating a wsdl

2b: generating our API code

3c: generating code from a 3rd party wsdl (magento)

3: Implementing our API

3a: creating our camel route

3b: plugging it all together

4: Useful resources

Okay, that’s all for today, check back over the weekend when I’ll hopfully have written up the ‘setting up the build environment’ section.




Overview of FUSE ESB

One of the big projects coming up in my new role is implementing an Enterprise service bus (ESB) as the backend of our IT operations.  I’d never even heard of an ESB before starting here, so this has all been new ground for me which makes it all the more interesting. The first thing I needed to do was understand what an ESB is and what it does.  This part is pretty simple once you understand the problem that an ESB is designed to solve,namely how to enable many disparate systems, written in different languages and running on different machines to share information and form an integrated business platform.

Tranditionally applications have shared information (if at all!) directly, by one application (app A) using the API of another (app B) to push information directly from A to B.  We can see thwat with just 2 applications we only need 1 interface (A-B).  However if we introduce another application the number of interfaces required for full integration jumps to 3 (A-B, B-C, A-C), and with 4 apps we need 5 interfaces (A-B,B-C,C-D,B-D,C-D).  More importantly each of these interfaces requires custom code to translate between the different formats and interfaces required. You can see that as the number of applications increases we have more interfaces with custom code.  And what happens if we decide to change one of the applications?  Each interface to that application now needs rewriting, which is a huge amount of work.

An ESB aims to simplify this problem of integration by becoming a ‘hub’ that sits in the middle of all your applications and facilitates message routing between them.  Instead of every applicaiton interfacing with every other application, each application now just has one interface to the ESB.  the ESB is responsible for translating messages to / from a common format and routing them to destinations.  The major saving in this approach comes if you have to replace any of your applications.  Instead of writing a whole host of new interfaces, you now only have one to worry about (between your application and the ESB).

The ESB we’re going to be using is ‘FUSE ESB‘ which is a packaged and supported version of Apache servicemix.  Servicemix is a (as the name suggests) made up of several difefrent parts, which come together to provide an end to end ESB solution.  The major parts are:

  • Apache Karaf:  An OSGI application server kernel.  This is basically the ‘container’ in which your routes will sit, you can think of this as an application and the routes you write as scripts for that application
  • Apache Camel: An integration framework.  Camel provides the ‘bits’ (components, chunks of processing code) that you can plug together to make up messaging routes within your integration solution
  • Apache ActiveMQ: A messaging solution.  ActiveMQ provides a simple queueing framework which is easy to integrate with Camel

With these 3 frameworks we have the basis of an incredibly powerful and flexible integration solution.  I’ll be looking at some use cases for this technology in future articles.