By default OSGi services are only visible and accessible in the OSGi container where they are published. Distributed OSGi allows to define services in one container and use them in some other (even over machine boundaries).

For this tutorial we use the DOSGi sub project of CXF which is the reference implementation of the OSGi Remote Service Admin specification, chapter 122 in the OSGi 4.2 Enterprise Specification).

Example on github

Introducing the example

Following the hands on nature of these tutorial we start with an example that can be tried in some minutes and explain the details later.

Our example is again the tasklist example from Part 1 of this tutorial. The only difference is that we now deploy the model and the persistence service on container A and model and UI to container B and we install the dosgi runtime on bother containers.

Full Size

As DOSGi should not be active for all services on a system the spec defines that the service property "osgi.remote.interfaces" triggers if DOSGi should process the service. It expects the interface names that this service should export remotely. Setting the property to "*" means that all interfaces the service implements should be exported. The tasklist persistence service already sets the property so the service is exported with defaults.

Installing the service

To keep things simple we will install container A and B on the same system.

• Download Apache Karaf 2.2.10
• Unpack into folder container_a
• Copy etc/jre.properties.cxf into etc/jre.properties
• Start bin/karaf
• config:propset -p org.apache.cxf.dosgi.discovery.zookeeper zookeeper.port 2181
• config:propset -p org.apache.cxf.dosgi.discovery.zookeeper.server clientPort 2181
• features:chooseurl cxf-dosgi 1.4.0
• features:install cxf-dosgi-discovery-distributed cxf-dosgi-zookeeper-server
• features:addurl mvn:net.lr.tasklist/tasklist-features/1.0.0-SNAPSHOT/xml
• features:install example-tasklist-persistence

After these commands the tasklist persistence service should be running and be published on zookeeper.

You can check the wsdl of the exported service http://localhost:8181/cxf/net/lr/tasklist/model/TaskService?wsdlBy starting the zookeeper client zkCli.sh from a zookeeper distro you can optionally check that there is a node for the service below the osgi path.

Installing the UI

• Unpack into folder container_b
• Copy etc/jre.properties.cxf into etc/jre.properties
• Start bin/karaf
• config:propset -p org.ops4j.pax.web org.osgi.service.http.port 8182
• config:propset -p org.apache.cxf.dosgi.discovery.zookeeper zookeeper.port 2181
• features:chooseurl cxf-dosgi 1.4.0
• features:install cxf-dosgi-discovery-distributed
• features:addurl mvn:net.lr.tasklist/tasklist-features/1.0.0-SNAPSHOT/xml
• features:install example-tasklist-ui

The tasklist client ui should be in status Active/Created and the servlet should be available on http://localhost:8182/tasklist. If the ui bundle stays in status graceperiod then DOSGi did not provide a local proxy for the persistence service.

How does it work

Full Size

The Remote Service Admin spec defines an extension of the OSGi service model. Using special properties when publishing OSGi services you can tell the DOSGi runtime to export a service for remote consumption. The CXF DOSGi runtime listens for all services deployed on the local container. It only processes services that have the "osgi.remote.interfaces" property. If the property is found then the service is either exported with the named interfaces or with all interfaces it implements.The way the export works can be fine tuned using the CXF DOSGi configuration options.

By default the service will be exported using the CXF servlet transport. The URL of the service is derived from the interface name. The servlet prefix, hostname and port number default to the Karaf defaults of "cxf", the ip address of the host and the port 8181. All these options can be defined using a config admin configuration (See the configuration options). By default the service uses the CXF Simple Frontend and the Aegis Databinding. If the service interface is annotated with the JAX-WS @WebService annotation then the default is JAX-WS frontend and JAXB databinding.

The service informations are then also propagated using the DOSGi discovery. In the example we use the Zookeeper discovery implementation. So the service metadata is written to a zookeeper server.

The container_b will monitor the local container for needed services. It will then check if a needed service is available on the discovery impl (on the zookeeper server in our case). For each service it finds it will create a local proxy that acts as an OSGi service implementing the requested interface. Incoming request are then serialized and sent to the remote service endpoint.

So together this allows for almost transparent service calls. The developer only needs to use the OSGi service model and can still communicate over container boundaries.

On thursday I had a talk about Apache Camel at W-JAX in Munich. Like on the last conferences there was a lot of interest in Camel and the room was really full. You can find the slides "Integration ganz einfach mit Apache Camel" here and the sources for the examples on github.

On Friday I joined the Eclipse 4 RCP workshop from Kai Tödter. Learned a lot about the new Eclipse. At last Eclipse RCP programming is becoming easier.

I just did my ApacheCon talk about OSGi best practices.It was the last slot but the room was still almost full. In general the OSGi track had a lot of listeners and there were a lot of talks that involved Apache Karaf. So I think that is a nice sign for greater adoption of OSGi and Karaf.

You can find the Slides at google docs.

The demo application can be found inside my Karaf tutorial code at github.

Practical Camel example that polls from a database table and sends the contents as XML to a jms queue. The route uses a JTA transaction to synchronize the DB and JMS transactions. An error case shows how you can handle problems.

Route and Overview

Full Size

from("jpa://net.lr.tutorial.karaf.camel.jpa2jms.model.Person").id("jpa2jms")
.onException(Exception.class).maximumRedeliveries(3).backOffMultiplier(2).handled(true).to("file:error")
.transacted()
.marshal(df)
.bean(new ExceptionDecider())
.to("jms:person");

The route starts with a jpa endpoint. It is configured with the fully qualified name of a JPA @Entity. From this entity camel knows the table to poll and how to read and remove the row. The jpa endpoint polls the table and creates a Person object for each row it finds. Then it calls the next step in the route with the Person object as body. The jpa component also needs to be set up separately as it needs an EntityManagerFactory.

The onException clause makes the route do up to 3 retries with backoff time increasing by factor 2 each time. If it still fails the message is passed to a file in the error directory.

The next step transacted() marks the route as transactional it requires that a TransactedPolicy is setup in the camel context. It then makes sure all steps in the route have the chance to participate in a transaction. So if an error occurs all actions can be rolled back. In case of success all can be committed together.

The marshal(df) step converts the Person object to xml using JAXB. It references a dataformat df that sets up the JAXBContext. For brevity this setup is not shown here.

The ExceptionDecidet bean allows to trigger an exception if the name of the person is error. So this allows us to test the error handling later.

The last step to("jms:person") sends the xml representation of person to a jms queue. It requires that a JmsComponent named jms is setup in the camel context.

from("jms:person").id("jms2log")
.transacted()
.convertBodyTo(String.class)
.to("log:personreceived");

This second route simply listens on the person queue, reads and displays the content. In a production system this part would tpyically be in another module.

Person as JPA Entity JAXB class

The Person class acts as a JPA Entity and as a JAXB annotated class. This allows us to use it in the camel-jpa component as well as during the marshalling. Keep in mind though that this
would rather be a bad practice in production as it would tie the DB model and the format of the JMS message together. So for real integrations it would be better to have separate beans for JPA and JAXB and do a manual
conversion between them.

@Entity
@XmlType
@XmlRootElement
public class Person {
    private String name;
    private String twitterName;

    public String getName() { return name; }
    public void setName(String name) { this.name = name; }
    public String getTwitterName() { return twitterName; }
    public void setTwitterName(String twitterName) { this.twitterName = twitterName; }
}

DataSource and ConnectionFactory setup

We use an XADataSource for Derby (See https://github.com/cschneider/Karaf-Tutorial/blob/master/db/datasource/datasource-derby.xml). As the default ConnectionDactory provided by ActiveMQ in Karaf is not XA ready we define the broker and ConnectionFactory definition by hand (See https://github.com/cschneider/Karaf-Tutorial/blob/master/cameljpa/jpa2jms/localhost-broker.xml). Together with the Karaf transaction feature these provide the basis to have JTA transactions.

JPAComponent, JMSComponent and transaction setup

An important part of this example is to use the jpa and jms components in a JTA transaction. This allows to roll back both in case of an error.
Below is the blueprint context we use. We setup the JMS component with a ConnectionFactory referenced as an OSGi service.
The JPAComponent is setup with an EntityManagerFactory using the jpa:unit config from Aries JPA. See Apache Karaf Tutorial Part 6 - Database Access for how this works.
The TransactionManager proviced by Aries transaction is referenced as an OSGi service, wrapped as a spring PlattformTransactionManager and injected into the JmsComponent and JPAComponent.

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0"
    xmlns:jpa="http://aries.apache.org/xmlns/jpa/v1.1.0"
	xsi:schemaLocation="
  http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
  http://www.osgi.org/xmlns/blueprint-ext/v1.1.0 https://svn.apache.org/repos/asf/aries/tags/blueprint-0.3.1/blueprint-core/src/main/resources/org/apache/aries/blueprint/ext/blueprint-ext.xsd
  http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd
  http://aries.apache.org/xmlns/jpa/v1.1.0 http://aries.apache.org/schemas/jpa/jpa_110.xsd
  ">

    <reference id="connectionFactory" interface="javax.jms.ConnectionFactory" />

    <bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
        <property name="connectionFactory" ref="connectionFactory"/>
        <property name="transactionManager" ref="transactionManager"/>
    </bean>

    <bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
        <argument ref="jmsConfig"/>
    </bean>

    <reference id="jtaTransactionManager" interface="javax.transaction.TransactionManager"/>

    <bean id="transactionManager" class="org.springframework.transaction.jta.JtaTransactionManager">
        <argument ref="jtaTransactionManager"/>
    </bean>

    <bean id="jpa" class="org.apache.camel.component.jpa.JpaComponent">
        <jpa:unit unitname="person2" property="entityManagerFactory"/>
        <property name="transactionManager" ref="transactionManager"/>
    </bean>

    <bean id="jpa2jmsRoute" class="net.lr.tutorial.karaf.camel.jpa2jms.Jpa2JmsRoute"/>

    <bean id="PROPAGATION_REQUIRED" class="org.apache.camel.spring.spi.SpringTransactionPolicy">
        <property name="transactionManager" ref="transactionManager"/>
     </bean>

    <camelContext id="jpa2jms" xmlns="http://camel.apache.org/schema/blueprint">
        <routeBuilder ref="jpa2jmsRoute" />
    </camelContext>

</blueprint>

Running the Example

You can find the full example on github : JPA2JMS Example
Follow the Readme.txt to install the necessary Karaf features, bundles and configs.

Apart from fthis example we also install the dbexamplejpa. This allows us to use the person:add command defined there to populate the database table.
Open the Karaf console and type:

person:add "Christian Schneider" @schneider_chris
log:display

You should then see the following line in the log:

2012-07-19 10:27:31,133 | INFO  | Consumer[person] | personreceived ...
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<person>
    <name>Christian Schneider</name>
    <twitterName>@schneider_chris2</twitterName>
</person>

So what happened

We used the person:add command to add a row to the person table. Our route picks up this record, reads and converts it to a Person object. Then it marshals it into xml and sends to the jms queue person.
Our second route then picks up the jms message and shows the xml in the log.

Error handling

The route in the example contains a small bean that reacts on the name of the person object and throws an exception if the name is "error".
It also contains some error handling so in case of an exception the xml is forwarded to an error directory.

So you can type the following in the Karaf:

person:add error error
log:display

This time the log should not show the xml. Instead it should appear as a file in the error directory below your karaf installation.

Summary

In this tutorial the main things we learned are how to use the camel-jpa component to write as well as to poll from a database and how to setup and use jta transactions to achieve solid error handling.

Back to Karaf Tutorials

Yesterday evening I did a talk about Apache Karaf and OSGi best practice together with Achim Nierbeck. Achim did the first part about OSGi basics and Apache Karaf and I did the second part about OSGi best practices.

One slide from the presentation about Karaf shows the big number of features that can be installed easily. So while the Karaf download is just about 8 MB you can install additional features transparently using maven that make it a full blown integration or enterprise application server.

OSGi best practices

In my part I showed how blueprint, OSGi Services and the config admin service can be used together to build a small example application consisting of the typical modules model, persistence and UI like shown below.

Except for the UI the example was from my first Karaf tutorial. While in the tutorial I used a simple Servlet UI that merely is able to display the Task objects I wanted to show a fancier UI for this talk. Since I met the makers of Vaadin on the last W-JAX conferences I got interested in this simple but powerful framework. So I gave it as spin. I had only about two days to prepare for the talk so I was not sure if I would be able to create a good UI with it. Fortunately it was really easy to use and it just took me about a day to learn th basics and build a full CRUD UI for my Task example complete with data binding to the persistence service.

One additional challenge was to use vaadin in OSGi. The good thing is that it is already a bundle. So a WAB (Web application bundle) deployment of my UI would have worked. I wanted it to be pure OSGi though so I searched a bit a found the vaadinbrige from Neil Bartlet. It allows to simply create a vaadin Application and factory class in a normal bundle and publish it as a service. The bridge will then pick it up and publish it to the HttpService.

The end result looks like this:

So you have a table with the current tasks (or to do items). You can add and delete tasks with the menu bar. When you select a task you can edit it in the form below. Any changes are directly sent to the service and updated in the UI.
The nice thing about vaadin is that it handles the complete client server communication and databinding for you. So this whole UI takes only about 120 lines of code. See ExampleApplication on github.

So the general idea of my part of the talk was to show how easy it is to create nice looking and architecturally sound applications using OSGi and Karaf. Many people still think OSGi will make your live harder for normal applications. I hope I could show that when using the right practices and tools OSGi can even be simpler and more fun than Servlet Container or Java EE focused development.

I plan to add a little more extensive Tutorial about using Vaadin on OSGi to my Karaf Tutorial series soon so stay tuned.

Links

Presentation: ApacheKaraf.pdf

Source Code:

Vaadin UI: https://github.com/cschneider/Karaf-Tutorial/tree/master/vaadin

Tasklist Model and Persistence: https://github.com/cschneider/Karaf-Tutorial/tree/master/tasklist

Achim adapted another Vaadin OSGi example from Kai Tödter to Maven and Karaf: https://github.com/ANierbeck/osgi-vaadin-demo

Nach dem Talk auf der letzten W-JAX hatte ich nun die Gelegenheit, auch auf der JAX über Apache Camel zu sprechen. Diesmal hatte ich einen grösseren Raum zur Verfügung, der mit fast 200 Zuhörern auch gut gefüllt war. Dies zeigt das grosse Interesse an Apache Camel. Die Präsentation ist direkt im Anhang verfügbar. Diesmal ging ich stärker auf OSGi und Apache Karaf als Ablaufumgebung ein. Ich hatte auch nur 20 Folien und verwendete einen größeren Teil der Zeit für Live Demos. Der Vortrag wurde auch gefilmt und sollte bald auf der JAX Website verfügbar sein. Ich werde dann eine Aktualisierung mit Link posten.

Nach dem geplanten Ende des Vortrags war ein freier Zeitblock. Viele der Zuhörer blieben noch um Fragen zu stellen und ich zeigte auch noch einige tiefer gehende Beispiele zu Bean Integration und Pojo Messaging. Als Resumé kann ich sagen, dass Apache Camel sehr beliebt ist und besonders Entwickler und Architekten den Einsatz treiben während das Management noch oft auf große Kommerzielle Frameworks setzt. Apache Karaf wird als sehr interesante Deploymentumgebung wahrgenommen. In den meisten Fällen gibt es allerdings Schwierigkeiten mit Operations bei der produktiven Einführung, da Apache Karaf und OSGi noch recht wenig verbreitet sind und damit eine zusätzliche Serverlandschaft darstellen.

Präsentation: Apache Camel JAX12.pdf

Beispiele: https://github.com/cschneider/camel-webinar/tree/master/part1

Today the 2.2.6 Version of Karaf was released. It incorporates more than 80 fixed jira issues.

One important usability improvement is the features:chooseurl command. It allows to add feature files for well known products like Apache Camel or Apache CXF in a much simpler way than the features:addurl command.

For example to install the current Apache Camel you just have to type:

features:chooseurl camel 2.9.1
features:install camel-core

Besides camel we currently already support activemq, cxf, jclouds, openejb and wicket.

In fact the command simply uses a config file etc/org.apache.karaf.features.repos.cfg that maps a product name to the feature file url and sets it with the given version number. So if some product you would like to use is missing you can simply add it yourself. If it is interesting for others too then please create a jira issue so we add it in the next distro.

Currently the chooseurl command already has completion for the product name but not for the version number. We plan to add completion for the version number in one of the next Karaf releases by evaluating the version in the maven repos.

Btw. for camel and cxf you still have to replace the etc/jre.properties with the etc/jre.properties.cxf file to change some package exports of the system bundle.

This wednesday on the 4th of April I will give a talk about the open source integration framework Apache Camel at the Java User Group in Karlsruhe. I will start with an overview of Camel and give some insight in the Camel Architecture. The main part of the Talk will be live coding showing how easy integration can be with the Camel DSL.

See the webpage of the JUG Karlsruhe for some more details: http://jug-karlsruhe.mixxt.de/networks/events/show_event.55045

Camel has many options for deployment. If I have the freedom of choice I prefer to run Camel on Karaf but the typical case at customers is that they have a certain app server and we have to fit in. In this case the platform was JBoss 5.1. Before Camel 2.8 this was quite complicated as camel tried to scan for typeconverters on the classpath and that part failed because of the JBoss class loader. I used camel 2.8.4 and so this was no issue except for a little problem I will come back to later.

Packaging Camel integrations as a war and using camel-servlet.

The most suitable deployment option on JBoss is to package your integration in a war archive and install it e.g. using the deploy folder. The camel-example-servlet-tomcat is the best starting point for that kind of project. It shows how to create wars using maven and also shows how to start camel from a spring application context in a servlet environment. If you know the older camel servlet examples you will notice that the way camel is started has changed recently. In older versions you had to configure the spring context xml in the camel servlet which is a rather uncommon way to start spring. The current example now shows how to start camel with the default spring context loader listener which is the much better solution.

See https://svn.apache.org/repos/asf/camel/tags/camel-2.8.4/examples/camel-example-servlet-tomcat/src/main/webapp/WEB-INF/web.xml

Installing and using the ActiveMQ connection factory using jndi

The easiest way to install a connection factory for a camel integration is to define the connection factory in your spring context. This has some drawbacks though. One is that you then depend directly on ActiveMQ and can not simply replace it by another broker. The other is that
the developer has access to the password of the connection factory. Of course you can use a property file to extract this still it is not ideal. So the prefered way to find a connection factory in an app server environment is to look it up in jndi. In the spring context this is quite simple using the jee namespace:

<jee:jndi-lookup id="connectionFactory" jndi-name="activemq/XAConnectionFactory" />

A bigger problem is how to install the connection factory in JBoss so it is available in jndi. There are two ways to achieve this. One is to install it as a JEE resource adapter (see http://activemq.apache.org/jboss-integration.html). I don´t like this solution too much as it is quite complicated and requires the special activemq-ra.rar.

After a lot of searching on the net I found a nice solution.

<?xml version="1.0" encoding="UTF-8"?>
<deployment xmlns="urn:jboss:bean-deployer:2.0">
    <aop:lifecycle-configure xmlns:aop="urn:jboss:aop-beans:1.0"
        name="DependencyAdvice"
        class="org.jboss.aop.microcontainer.aspects.jndi.JndiLifecycleCallback"
        classes="@org.jboss.aop.microcontainer.aspects.jndi.JndiBinding"
        manager-bean="AspectManager"
        manager-property="aspectManager">
    </aop:lifecycle-configure>

    <bean name="ActiveMQXAConnectionFactory" class="org.apache.activemq.ActiveMQXAConnectionFactory">
        <annotation>@org.jboss.aop.microcontainer.aspects.jndi.JndiBinding(name="activemq/XAConnectionFactory", aliases={"java:/activemq/XAConnectionFactory"})</annotation>
        <property name="brokerURL">tcp://localhost:61616</property>
    </bean>
</deployment>

This JBoss mbean description initializes an ActiveMQXAConnectionFactory and installs it in the jndi context. It obviously needs the activemq-core-5.5.0.jar which I simply installed in the lib dir of the default server. The problem with this is of course that it is then available on the classpath of all wars you also install. So if some activemq specialist knows a better solution to have it only on the classpath for the mbean config I would really like to know how to do this.

When I experimented with this setup I first used the activemq-all-5.5.0.jar. The problem was that it contains an older camel version. So when I installed my camel war it was not able to start
because of problems loading typeconverters from this jar. So remember to only use the core jar.

Example project

To make it easy for you to test this out yourself I have put the code of a simple producer and consumer project on github.
To install do the following steps:

• download activemq-5.5.0, extract and start
• download jboss 5.1 and extract it
• download the example projects and build them using mvn install
• copy the jms-jboss-beans.xml to the default/deploy folder
• copy activemq-5.5.0.jar to the default/lib folder
• checkout or download https://github.com/cschneider/cameljbossha
• Build the example project with mvn clean install
• copy the war files /consumer/target/consumer-1.0.0.war and /producer/target/producer-1.0.0.war to the default/deploy folder
• start jboss

The producer offers a servlet where we can trigger a message. So open a browser and go to http://localhost:8080/producer-1.0.0/camel/tojms . The jboss log will show that the request is handled and a jms message is sent. The consumer will pick up the message and write a log entry "Message received from jms".

Summary

We have seen how to produce war projects with Apache Camel and how the usage of the camel servlet has changed recently. This part is independent from JBoss. Next we have seen that current Camel versions allow to deploy on JBoss without any special tweaks. We learned how to reference a connection factory in jndi and how to install it in JBoss.

The projects producer and consumer will be reused in my next post where I look into concepts for high availability with Camel and ActiveMQ.

CXF 2.6.0 will bring a lot of improvements for deployment in OSGi. Till now cxf was bundled in one OSGi bundle. Either with all features or with a minimal feature set. Thanks to Dan Kulp cxf is now delivered as individual bundles. So it can be installed with only the needed features. Besides the smaller size in many use cases this also means that we have less optional dependencies which make installation difficult. Each bundle defines the imports it really needs. This makes it much easier to get the dependencies right. Of course the Karaf feature file will still be provided to make it easy to install CXF in Apache Karaf.

Based on the work of Dan I recently started to optimize the imports of the typical bundles most people will use from cxf. At the start we had many dependencies like spring, velocity, neethi, .., that I felt should not be needed and make cxf quite big. By refactoring some of the modules I was able to slim these down to the bare minimum. The current code on trunk already reflects these changes.

If you want to try this yourself you can easily install the snapshot of cxf in karaf 2.2.5. As the feature file is not yet changed I uploaded a gist of the commands you need to execute. Remember to also use the jre.properties.cxf for karaf to disable some default java apis so CXF can replace them with newer versions.

So after this install the karaf list -u command shows the following bundles:

 karaf@root> list -u
START LEVEL 100 , List Threshold: 50
   ID   State         Blueprint      Level  Update location
[  87] [Active     ] [            ] [   60] mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.jaxb-impl/2.2.1.1_1
[  88] [Active     ] [            ] [   60] mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.jaxb-xjc/2.2.1.1_1
[  89] [Active     ] [            ] [   60] mvn:org.codehaus.woodstox/stax2-api/3.1.1
[  90] [Active     ] [            ] [   60] mvn:org.codehaus.woodstox/woodstox-core-asl/4.1.1
[  91] [Active     ] [            ] [   60] mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.wsdl4j/1.6.2_3
[  92] [Active     ] [            ] [   60] mvn:org.apache.ws.xmlschema/xmlschema-core/2.0.1
[  93] [Active     ] [            ] [   60] mvn:org.apache.cxf/cxf-common-utilities/2.6.0-SNAPSHOT
[  94] [Active     ] [Created     ] [   60] mvn:org.apache.cxf/cxf-rt-core/2.6.0-SNAPSHOT
                                       Fragments: 95
[  95] [Resolved   ] [            ] [   60] mvn:org.apache.cxf/cxf-api/2.6.0-SNAPSHOT
                                       Hosts: 94
[  96] [Active     ] [Created     ] [   60] mvn:org.apache.cxf/cxf-rt-bindings-soap/2.6.0-SNAPSHOT
[  97] [Active     ] [            ] [   60] mvn:org.apache.cxf/cxf-rt-bindings-xml/2.6.0-SNAPSHOT
[  98] [Active     ] [Created     ] [   60] mvn:org.apache.cxf/cxf-rt-frontend-simple/2.6.0-SNAPSHOT
[  99] [Active     ] [            ] [   60] mvn:org.apache.cxf/cxf-rt-databinding-jaxb/2.6.0-SNAPSHOT
[ 100] [Active     ] [Created     ] [   60] mvn:org.apache.cxf/cxf-rt-frontend-jaxws/2.6.0-SNAPSHOT
[ 102] [Active     ] [Created     ] [   60] mvn:org.apache.cxf/cxf-rt-transports-http/2.6.0-SNAPSHOT
[ 103] [Active     ] [Created     ] [   60] mvn:org.apache.cxf/cxf-rt-frontend-jaxrs/2.6.0-SNAPSHOT

This installation of CXF is ready for SOAP/HTTP and REST with JAX-WS and JAXB on the java side which reflects what most people will need.

To test the features I recommend to install the example from my Karaf Tutorial about CXF.

Shows how to access databases from OSGi applications running in Karaf and how to abstract from the DB product by installing DataSources as OSGi services. Some new Karaf shell commands can be used to work with the database from the command line. Finally JDBC and JPA examples show how to use such a DataSource from user code.

Installing the driver

The first step is to install the driver jar(s) for your database system into Karaf. Most drivers are already valid bundles and available in the maven repo. So this is tpyically only
one Karaf command. If the driver is available in maven but no bundle we can most times use the wrap: protocol of Karaf to make it a bundle on the fly. If the driver is not even
in the repo we have to install the file into the maven repo first.

For derby the following command will work

> install -s mvn:org.apache.derby/derby/10.8.2.2

See the db/datasource folder on github for installation instructions for (db2, derby, h2, mysql and oracle).

Installing the datasource

In Java EE servers and servlet containters you typically use JNDI to install a DataSource on the server level so the application can just refer to it and so does not have to know the specific driver or database url. In OSGi JNDI is replaced by OSGi services. So the best way to decouple your application from the database is to offer a DataSource as an OSGi service.

As we can deploy simple blueprint xml files in Karaf this is really easy. We define a bean with the class of the specific datasource impl and configure it. Then we publish that bean as an OSGi service with the interface a javax.sql.DataSource. This works because Karaf uses dynamic imports when it deploys blueprint context files so all classes are available.

For each database flalour you can find a suitable blueprint.xml in db/datasource.

Browsing the database using the Karaf db:* commands

As part of this tutorial I created some simple Karaf commands to work with databases from the Karaf shell. These commands proved to be quite handy so I will try to move them to the Karaf project.

db:select <name>

When called without parameters the command shows all available DataSources.
Example:

karaf@root> db:select
Sel | Name         | Product      | Version              | URL
------------------------------------------------------------------------------------
    | jdbc/derbyds | Apache Derby | 10.8.2.2 - (1181258) | jdbc:derby:test
    | jdbc/mysqlds | MySQL        | 5.5.17               | jdbc:mysql://localhost:3

When called with the name of a DataSource it will select the DataSource:

Example:

db:select jdbc/derbyds

db:exec "<sql>"

Executes a SQL command.
Example:

karaf@root> db:exec "create table person (name varchar(100), twittername varchar(100))"
karaf@root> db:exec "insert into person (name, twittername) values ('Christian Schneider', '@schneider_chris')"

This creates a table person and adds a row to it.

db:tables

Shows the current tables in the database.

Example:

db:tables
TABLE_CAT | TABLE_SCHEM | TABLE_NAME       | TABLE_TYPE   | REMARKS | TYPE_CAT | TYPE_SCHEM | TYPE_NAME | SELF_REFERENCING_COL_NAME | REF_GENERATION
----------------------------------------------------------------------------------------------------------------------------------------------------
          | SYS         | SYSALIASES       | SYSTEM TABLE |         |          |            |           |                           |
          | SYS         | SYSVIEWS         | SYSTEM TABLE |         |          |            |           |                           |
          | SYSIBM      | SYSDUMMY1        | SYSTEM TABLE |         |          |            |           |                           |
          | APP         | PERSON           | TABLE        |         |          |            |           |                           |

db:query

Executes a query and shows the results.

Example:

karaf@root> db:query "select * from person"
NAME                | TWITTERNAME
--------------------------------------
Christian Schneider | @schneider_chris

Accessing the database using JDBC

The project db/examplejdbc shows how to use the datasource we installed and execute jdbc commands on it. The example uses a blueprint.xml to refer to the OSGi service for the DataSource and injects it into the class
DbExample.The test method is then called as init method and shows some jdbc statements on the DataSource.The DbExample class is completely independent of OSGi and can be easily tested standalone using the DbExampleTest. This test shows how to manually set up the DataSource outside of OSGi.

Build and install

Build works like always using maven

> mvn clean install

In Karaf we just need our own bundle as we have no special dependencies

> install -s mvn:net.lr.tutorial.karaf.db/db-examplejdbc/1.0-SNAPSHOT
Using datasource H2, URL jdbc:h2:~/test
Christian Schneider, @schneider_chris,

After installation the bundle should directly print the db info and the persisted person.

Accessing the database using JPA

For larger projects it is quite typical that JPA is used instead of hand crafted SQL. Using JPA has two big advantages over JDBC. You need to maintain less SQL code and JPA provides dialects for the subtle differences in databases that else you would have to code yourself. For this example we use OpenJPA as the JPA Implementation. On top of it we add Apache Aries JPA which supplies an implementation of the OSGi JPA Service Specification and blueprint integration for JPA.

The project examplejpa shows a simple project that implements a PersonService that manages Person objects.
Person is just a java bean annotated with JPA @Entitiy. As OpenJPA needs to enhance the bytecode of the classes we need to add the openjpa-maven-plugin to the pom.xml which prepares the classes for JPA.

Additionally the project implements two Karaf shell commands person:add and person:list that allow to easily test the project.

persistence.xml

Like in a typical JPA project the peristence.xml defines the DataSource lookup, database settings and lists the persistent classes. The datasource is refered using "osgi:service/javax.sql.DataSource/(osgi.jndi.service.name=jdbc/derbyds)". This makes a lookup for an OSGi service with the given interface and properties.

The OSGi JPA Service Specification defines that the Manifest should contain an attribute "Meta-Persistence" that points to the persistence.xml. So this needs to be defined in the config of the maven bundle plugin in the prom. The Aries JPA container will scan for these attributes
and register an initialized EntityMangerFactory as an OSGi service on behalf of the use bundle.

blueprint.xml

We use a blueprint.xml context to inject an EntityManager into our service implementation and to provide automatic transaction support.
The following snippet is the interesting part:

  <bean id="personService" class="net.lr.tutorial.karaf.db.examplejpa.impl.PersonServiceImpl">
      <jpa:context property="entityManager" unitname="person" />
      <tx:transaction method="*" value="Required"/>
  </bean>

This makes a lookup for the EntityManagerFactory OSGi service that is suitable for the persistence unit person and injects an EnityManager into the
PersonServiceImpl. Additionally it wraps each call to a method of PersonServiceImpl with code that opens a transaction
before the method and commits on success or rollbacks on any exception thrown.

Build and Install

The project builds with mvn clean install like usual. A prerequisite is that the derby datasource is installed like described above. Then we have to install the bundles for openjpa, aries (jpa, transaction, proxy and jndi) and of course our db-examplejpa bundle.
See ReadMe.txt for the exact commands to use.

Test

person:add 'Christian Schneider' @schneider_chris

This should create a person object with the above data and persist it to the database. Unfortunately this currently does not work. I guess I still have an error somewhere. So instead we use the db commands to populate the DB by manually:

db:select jdbc/derbyds
db:exec "insert into person (id, name, twittername) values (1, 'Christian Schneider', '@schneider_chris')"

Then we list the persisted persons

karaf@root> person:list
Christian Schneider, @schneider_chris

Using pooling for datasources

I any real world scenario you will need pooling for the DataSource. To achieve this you have two good options:

1. Use a pooling datasource from the vendor:

2. Use the PoolingDataSource from dbcp like described in this gist by Andreas Pieber: https://gist.github.com/2761628

Summary

In this tutorial we learned how to work with databases in Apache Karaf. We installed drivers for our database and a DataSource. We were able to check and manipulate the DataSource using the db:* commands. In the examplejdbc we learned how to acquire a datasource
and work with it using plain jdbc. This is really easy but a bit verbose. You might want to try the spring JdbcTemplate to remove all the cleanup code. Last but not least we also used jpa to access our database.

In theory JPA and OSGi work together really well. Keep in mind though that JPA support for OSGi is still quite fresh. It took me quite a while to get it all running. The documentation is quite sparse and I still have not been able to fix the persist issue. I will update the code and blog entry as soon as I have the jpa persist working.

Back to Karaf Tutorials

Shows how to run your camel routes in the OSGi server Apache Karaf. Like for CXF blueprint is used to boot up camel. The tutorial shows three examples - a simple blueprint route, a jms2rest adapter and an order processing example.

Installing Karaf and making Camel features available

• Download Karaf 2.3.1 and unpack to the file system
• Start bin\karaf.bat or bin/karaf for unix

In Karaf type:

features:chooseurl camel 2.10.4
features:list

You should see the camel features

Getting and building the examples

You can find the examples for this tutorial on github Karaf Tutorial - camel.

So either clone the git repo or just download and unpack the zip of it.To build the code do:

cd camel
mvn clean install

Starting simple with a pure blueprint deployment

Our first example does not even require a java project. In Karaf it is possible to deploy pure blueprint xml files. As camel is well integrated with blueprint you can define a complete camel context with routes in a simple blueprint file.

simple-camel-blueprint.xml

The blueprint xml for a camel context is very similar to the same in spring. Mainly the namespaces are different. Blueprint discovers the dependency on camel so it will automatically require the at least the camel-blueprint feature is installed. The camel components in routes are discovered as OSGi services. So as soon as a camel component is installed using the respective feature it is automatically available for usage in routes.

So to install the above blueprint based camel integration you only have to do the following steps:

features:install camel-blueprint camel-stream

Copy simple-camel-blueprint.xml to the deploy folder of karaf. You should now see "Hello Camel" written to the console every 5 seconds.

The blueprint file will be automatically monitored for changes so any changes we do are directly refelcted in Karaf. To try this open the simple-camel-blueprint.xml file from the deploy folder in an editor, change "stream:out" to "log:test" and save. Now the messages on the console should stop and instead you should be able to see "Hello Camel" in the Karaf log file formatted as a normal log line.

JMS to REST Adapter (jms2rest)

Full Size

This example is not completely standalone. As a prerequisite install the person service example like described in Karaf Tutorial 4.

The example shows how to create a bridge from the messaging world to a REST service. It is simple enough that it could be done in a pure blueprint file like the example above. As any bigger integration needs some java code I opted to use a java project for that case.

Like most times we mainly use the maven bundle plugin with defaults and the packaging type bundle to make the project OSGi ready. The camel context is booted up using a blueprint file blueprint.xml and the routes are defined in the java class Jms2RestRoute.

Routes

The first route watches the directory "in" and writes the content of any file placed there to the jms queue "person". It is not strictly necessary but makes it much simpler to test the example by hand.

The seconds route is the real jms2rest adapter. It listens on the jms queue person and expects to get xml content with persons like also used in the PersonService. In the route the id of the person is extracted from the xml and stored in a camel message header. This header is then used to build the rest uri. As a last step the content from the message is sent to the rest uri with a PUT request. So this tells the service to store the person with the given id and data.

Use of Properties

Besides the pure route the example shows some more tpyical things you need in camel. So it is a good practice to externalize the url of services we access. Camel uses the Properties component for this task.

 This enables us to write {{personServiceUri}} in endpoints or ${properties:personServiceUri} in the simple language.

In a blueprint context the Properties component is automatically aware of injected properties from the config admin service. We use a cm:property-placeholder definition to inject the attributes of the config admin pid "net.lr.tutorial.karaf.cxf.personservice". As there might be no such pid we also define a default value for the personServiceUri so the integration can be deployed without further configuation.

JMS Component

We are using the camel jms component in our routes. This is one of the few components that need further configuration to work. We also do this in the blueprint context by defining a JmsComponent and injecting a connection factory into it. In OSGi it is good practice to not define connection factories or data sources directly in the bundle instead we are simply refering to it using a OSGi service reference.

Deploying and testing the jms2rest Adapter

Just type the following in Karaf:

features:chooseurl activemq 5.8.0
features:chooseurl camel 2.10.4
features:install  camel-blueprint camel-jms camel-http activemq-blueprint
activemq:create-broker --type blueprint
install -s mvn:net.lr.tutorial.karaf.camel/example-jms2rest/1.0-SNAPSHOT

This installs the activemq and camel feature files and features in karaf. The activemq:create command creates a broker defintions in the deploy folder. This broker is then automatically started. The broker defintion also publishes an OSGi service for a suitable connection factory. This is then referenced later by our bundle.

As a last step we install our own bundle with the camel route.

Now the route should be visible when typing:

> camel:route-list
Route Id             Context Name         Status
[file2jms          ] [jms2rest          ] [Started           ]
[personJms2Rest    ] [jms2rest          ] [Started           ]

Now copy the file person1.xml to the folder "in" below the karaf directory. The file should be sent to the queue person by the first route and then sent to the rest service by the second route.

In case the personservice is instaleld you should now see a message like "Update request received for ...". In case it is not installed you should see a 404 in the karaf error when accessing the rest service.

Order processing example

Full Size

The business case in this example is a shop that partly works with external vendors.

We receive an order as an xml file (See: order1.xml). The order contains a customer element and several item elements. Each item specifies a vendor. This can be either "direct" when we deliver the item ourself or a external vendor name. If the item vendor is "direct" then the item should be exported to a file in a directory with the customer name. All other items are sent out by mail. The mail content should be customizeable. The mail address has to be fetched from a service that maps vendor name to mail address.

How it works

This example again uses maven to build, a blueprint.xml context to boot up camel and a java class OrderRouteBuilder for the camel routes. So from an OSGi perspective it works almost the same as the jms2rest example.

The routes are defined in net.lr.tutorial.karaf.camel.order.OrderRouteBuilder. The "order" route listens on the directory "orderin" and expects xml order files to be placed there. The route uses xpath to extract several attributes of the order into message headers. A splitter is used to handle each (/order/item) spearately. Then a content based router is used to handle "direct" items different from others.

In the case of a direct item the recipientlist pattern is used to build the destination folder dynamically using a simple language expression.

recipientList(simple("file:ordersout/${header.customer}"))

If the vendor is not "direct" then the route "mailtovendor" is called to create and send a mail to the vendor. Some subject and to address are set using special header names that the mail component understands. The content of the mail is expected in the message body. As the body also should be comfigureable the velocity component is used to fill the mailtemplate.txt with values from the headers that were extracted before.

Deploy into karaf

The deployment is also very similar to the previous example but a little simpler as we do not need jms. Type the following in karaf

features:addurl mvn:org.apache.camel.karaf/apache-camel/2.9.0/xml/features
features:install camel-blueprint camel-mail camel-velocity camel-stream
install -s mvn:net.lr.tutorial.karaf.camel/example-order/1.0-SNAPSHOT

To be able to receive the mail you have to edit the configuration pid. You can either do this by placing a properties file
into etc/net.lr.tutorial.karaf.cxf.personservice.cfg or editing the config pid using the karaf webconsole. (See part 2 and part 3 of the Karaf Tutorial series).

Basically you have to set these two properties according to your own mail environment.

mailserver=yourmailserver.com
testVendorEmail=youmail@yourdomain.com

Test the order example

Copy the file order1.xml into the folder "ordersin" below the karaf dir.

The Karaf console will show:

Order from Christian Schneider

Count: 1, Article: Flatscreen TV

The same should be in a mail in your inbox. At the same time a file should be created in ordersout/Christian Schneider/order1.xml that contains the book item.

Wrapping it up and outlook

The examples show that fairly sophisticated integrations can be done using camel and be nicely deployed in an Apache Karaf container. The examples also show some best practices around configuration management, jms connection factories and templates for customization. The examples should also provide a good starting point for you own integration projects. Many people are a bit hesitant using OSGi in production. I hope these simple examples can show how easy this is in practice. Still problems can arise of course. For that case it is advisable to think about getting a support contract from a vendor like Talend. The whole Talend Integration portfolio is based on Apache Karaf so we are quite experienced in this area.

I have left out one big use case for Apache Camel in this tutorial - Database integrations. This is a big area and warrants a separate tutorial that will soon follow. There I will also explain how to handle DataSources and Connection Factories with drivers that are not already OSGi compliant.

Back to Karaf Tutorials

Shows how to publish and use a simple REST and SOAP service in karaf using cxf and blueprint.

To run the example you need to install the http feature of karaf. The default http port is 8080 and can be configured using the
config admin pid "org.ops4j.pax.web". You also need to install the cxf feature. The base url of the cxf servlet is by default "/cxf".
It can be configured in the config pid "org.apache.cxf.osgi".

PersonService Example

The "business case" is to manage a list of persons. As service should provide the typical CRUD operations. Front ends should be a REST service, a SOAP service and a web UI.

Full Size

The example consists of four projects

• model: Person class and PersonService interface
• server: Service implementation and logic to publish the service using REST and SOAP
• proxy: Accesses the SOAP service and publishes it as an OSGi service
• webui: Provides a simple servlet based web ui to list and add persons. Uses the OSGi service

You can find the full source on github: https://github.com/cschneider/Karaf-Tutorial/tree/master/cxf/personservice

Installation and test run

First we build, install and run the example to give an overview of what it does. The following main chapter then explains in detail how it works.

Installing Karaf and preparing for CXF

We start with a fresh Karaf 2.3.1.

• Unpack Karaf 2.3.1 from http://karaf.apache.org/index/community/download.html
• If you use Karaf 2.2.x you will have to copy etc/jre.properties.cxf to jre.properties
• Run Karaf using bin/karaf

Installing CXF

In Karaf Console run

features:chooseurl cxf 2.7.4
features:install http cxf

Changes in commands for karaf 3 features:chooseurl -> feature:repo-add features:install -> feature:install

Build and Test

Checkout the project from github and build using maven

> mvn clean install

Install service and ui in karaf

install -s mvn:net.lr.tutorial.karaf.cxf.personservice/personservice-model/1.0-SNAPSHOT
install -s mvn:net.lr.tutorial.karaf.cxf.personservice/personservice-server/1.0-SNAPSHOT
install -s mvn:net.lr.tutorial.karaf.cxf.personservice/personservice-proxy/1.0-SNAPSHOT
install -s mvn:net.lr.tutorial.karaf.cxf.personservice/personservice-webui/1.0-SNAPSHOT

Test the service

The person service should show up in the list of currently installed services that can be found herehttp://localhost:8181/cxf/

List the known personshttp://localhost:8181/cxf/person
This should show one person "chris"

Now using a firefox extension like Poster or Httprequester you can add a person.

Send the following xml snippet:

 <?xml version="1.0" encoding="UTF-8"?>
<ns1:person xmlns:ns1="http://person.jms2rest.camel.karaf.tutorial.lr.net">
    <id>1001</id>
    <name>Christian Schneider</name>
    <url>http://www.liquid-reality.de</url>
</ns1:person>

with Content-Type:text/xml using PUT:http://localhost:8181/cxf/person/1001
or to this url using POST:http://localhost:8181/cxf/person

Now the list of persons should show two persons.

Test the proxy and web UI

http://localhost:8181/personui

You should see the list of persons managed by the personservice and be able to add new persons.

How it works

Defining the model

The model project is a simple java maven project that defines a JAX-WS service and a JAXB data class. It has no dependencies to cxf. The service interface is just a plain java interface with the @WebService annotation.

@WebService
public interface PersonService {
    public abstract Person[] getAll();
    public abstract Person getPerson(String id);
    public abstract void updatePerson(String id, Person person);
    public abstract void addPerson(Person person);
}

The Person class is just a simple pojo with getters and setters for id, name and url and the necessary JAXB annotations. Additionally you need an ObjectFactory to tell JAXB what xml element to use for the Person class.
There is also no special code for OSGi in this project. So the model works perfectly inside and outside of an OSGi container.

The service is defined java first. SOAP and rest are used quite transparently. This is very suitable to communicate between a client and server of the same application. If the service is to be used by other applications the wsdl first approach is more suitable. In this case the model project should be configured to generate the data classes and service interface from a wsdl (see cxf wsdl_first example pom file). For rest services the java first approach is quite common in general as the client typically does not use proxy classes anyway.

Service implementation (server)

PersonServiceImpl is a java class the implements the service interface and contains some additional JAX-RS annotations. The way the class is defined allows it to implement a REST service and a SOAP service at the same time.

The server project also contains a small starter class that allows the service to be published directly from eclipse. This class is not necessary for deployment in karaf.

The production deployment of the service is done in src/main/resources/OSGI-INF/blueprint/blueprint.xml.

As the file is in the special location OSGI-INF/blueprint it is automatically processed by the blueprint implementation aries in karaf. The REST service is published using the jaxrs:server element and the SOAP service is published using the jaxws:endpoint element. The blueprint namespaces are different from spring but apart from this the xml is very similar to a spring xml.

Service proxy

The service proxy project only contains a blueprint xml that uses the CXF JAXWS client to consume the SOAP service and exports it as an OSGi Service. Encapsulating the service client as an OSGi service (proxy project) is not strictly necessary but it has the advantage that the webui is then completely independent of cxf. So it is very easy to change the way the service is accessed. So this is considered a best practice in OSGi.

See blueprint.xml

Web UI (webui)

This project consumes the PersonService OSGi service and exports the PersonServlet as an OSGi service. The pax web whiteboard extender will then publish the servlet on the location /personui.
The PersonServlet gets the PersonService injected and uses to get all persons and also to add persons.

The wiring is done using a blueprint context.

Some further remarks

The example uses blueprint instead of spring dm as it works much better in an OSGi environment. The bundles are created using the maven bundle plugin. A fact that shows how well blueprint works
is that the maven bundle plugin is just used with default settings. In spring dm the imports have to be configured as spring needs access to many implementation classes of cxf. For spring dm examples
take a look at the Talend Service Factory examples (https://github.com/Talend/tsf/tree/master/examples).

The example shows that writing OSGi applications is quite simple with aries and blueprint. It needs only 153 lines of java code (without comments) for a complete little application.
The blueprint xml is also quite small and readable.

Back to Karaf Tutorials

The Apache integration day at W-JAX was a big success. We had sessions about CXF, Camel, Karaf, TESB and Continuous Delivery. Now finally most presentations and examples are available.

In enterprise environments a typical requirement is that an integration has to be highly available. Typically you will use at least two nodes to achieve that. Depending on the requirements you will either want all nodes to be active or only one. The problem with having more than one active node is that messages can get out of order. So if your requirement is that your messages keep in sequence then sometimes the only way to achieve that is to make sure only one node is active at any time.

By default Apache Camel has no mechanism to achieve this. So as I had this requirement from some customers I decided to create an addition to apache camel to achieve this.

SimpleCluster

The idea is to use a database table lock to synchronize the locking between the nodes. The reason is that databases are really good at such things and so a database lock is really very reliable. As I did not want to reinvent the wheel I started with the database lock code from ActiveMQ (http://activemq.apache.org/jdbc-master-slave.html). Basically the idea is to do a "select * from mytable for update" in a transaction. This locks the table so only one node can acquire the lock.

Full Size

This is encapsulated in the class DbLockManager. The interface FailoverHandler then allows to register a callback into your own code to be notified that you should start or stop. This part is completely independent of Apache Camel and can also be used for other use cases.

Behaviour

The node that is started first will acquire the lock and start the route. All nodes will try to get the lock after the sleep interval. If the lock can not be achieved the db call blocks till the transaction timeout is reached. So the interval between two tries to get the lock is a little larger then the sleep interval. In case of a connection or db failure the node will stop. So if the DB goes down all nodes will stop. That means you should make sure the DB is also HA.

Configuration

The DbLockManager is configured like this:

<bean id="lockManager" init-method="start" destroy-method="stop">
<property name="dataSource" ref="dataSource" />
<property name="handler" ref="failoverPolicy" />
</bean>

You can also set the sleep time and the lock table name.

The Camel integration is done with a RoutePolicy. Such a policy can be easily added to a camel route and can control the status of the route. So the FailoverRoutePolicy simply needs to be added as a bean:

<bean id="failoverPolicy" class="net.lr.simplecluster.example.FailoverRoutePolicy"/>

So the only thing that remains is to add the policy to the route and to make sure the route does not start on its own:

  from("file:target/test").noAutoStartup().routePolicyRef("failoverPolicy").to("log:test2");

Code

https://github.com/cschneider/simplecluster