WSO2 ESB 5.0.0: Data Mapper

One of the features i missed within the WSO2 Developer Studio was a data mapper.
We had to write our own XSLT or PayloadFactory within the product.
But everything is going to change with the release of ESB 5.0.0 !

Or is it ? ....

There is a first blog item on the datamapper that i read and tried:

A very good blog indeed. However when i tried the new developer studio, i was somehow disappointed. I am positive about the fact that the Mapper is a real resource (even project) now. But i miss a lot of features (like for example XPath functions, if-then-else-construct, constraints) within the mapper.

As you can see from the picture below, only the following operators are supported:

  • Concat
  • Split
  • LowerCase
  • UpperCase

And when you investigate the generated project files, you will notice that JScript? is generated to do the actual mapping.

So my first impression is disappointed but it is a good start ! Hopefully new features and operations will be added soon, because at the moment i doubt if it is production ready.

But hey that's my opinion ;-)


WSO2 API Management in DTAP

Using the WSO2 API Manager in a DTAP environment is slightly different than for example the ESB product. Nadeesha has written a good article about it: http://wso2.com/library/articles/2016/03/article-architecting-a-multi-environment-api-manager-deployment-with-wso2-api-manager/

My conclusion of the article and some take aways:

  • API Manager does not use Carbon Archive files because API Manager artifacts are both stored on file system and database
  • Use import/export facility to migrate APIs
  • There is also a bulk tool available to import/export a bulk of APIs
  • Environment specific attributes must be given as parameters
The downside of that last point is that each time new parameters are added, you must restart the API server.

Each backend will have its own endpoints, so this will require new parameters.
I would recommend a seperate file that must be imported within the wso2server.bat file.


WSO2 Conference Asia


Unfortunately i did not went to the WSO2 Conference in Asia but i went through the presentations and this blog item describes some interesting topics i came accross.

There were a lot of different tracks:

  • API Management 
  • Cloud
  • Strategy
  • Integration'
  • Security
  • Governance
  • Architecture
  • Analytics
  • DevOps
  • Internet of Things (IoT)


API (Management) is a hot topic. Almost all reference architectures use APIs to expose functionality, either to mobile apps or other apps. It is used within Microservices as contracts, it is used in connected enterprises that want to expose their APIs into the API economy system and it is used internally to manage and govern services.


Internet of Things is a also a big thing, so also at the WSO2 conference.
The WSO2 IoT Server was announced in 2016 Q2.

Features of this product:
  • Implementation of WSO2 IoT Reference Architecture
  • Device Management
  • Device Analytics
  • Device adapters
  • Supported protocols: MQTT, XMPP, WebSockets, HTTP
  • Device Control
It will also be part of WSO2 Cloud offering.

WSO2 Microservices Framework for Java (MSF4J)

Another interesting product is MSF4J, which is a lightweight, high performance framework for building microservices in Java.
  • It uses Java annotations to define microservices APIs and metrics
  • Out-of-the-box integration with WSO2 DAS for analytics and metrics of the APIs
  • Transport based on Netty 4.0
  • Low memory footprint
  • Fast startup
  • Swagger definition to code in Developer Studio

Analytics family

Analytics is becoming more attention and WSO2 has several products to support this:
Especially the Machine Learner becomes more interesting when "R" is integrated.

Cloud Offerings

Managed Cloud
One of the new cloud offerings of WSO2 is Managed Cloud. As the name suggests, your WSO2 environment is managed by WSO2. The environment is placed in the Amazon VPC cloud. This can be a viable solution for organisations how can not manage their environment themselves.
The regions that are supported: US East, US West, Asia Pacific, South America and Europe (where?).

Public Cloud
Cloud offering that offers WSO2 products in the public cloud. Currently only API Manager and App Manager (Beta) available. Will be extended with Identity, Device and Analytics platforms.

Cloud strategies
When dealing with Cloud you need to think of:
* Which Cloud services to use (there are a lot !) -> WSO2 Integration connectors
* Federated cloud identities (Facebook, LinkedIn, Google+, etc) -> WSO2 Identity Server
* Development in cloud -> WSO2 App Factory
* Running apps in the cloud  -> WSO2 Managed Cloud


WSO2 has a new product called the Enterprise Mobility Manager. I did not go through those slides yet as i am focussing only on Middleware platform for the moment.


A few new features in ESB 4.9:
* Dynamic inbound channels
* Kafka, MQTT and RabbitMQ protocol support

New in upcoming ESB 5.0 release:
* Mediation debugger
* Data Mapper
* JMS 2.0 support
* Websocket protocol support

New WSO2 Gateway product with high performance message handling is used in several other products (API Manager, ESB,  App Manager, Load balancer).


Integration and API management remain an important topic within the WSO2 product suite. Some interesting added products are MSF4J, Enterprise Mobility Manager and IoT Server.
Also analytics become more important within the suite and security remains an important aspect of todays IT with BYOD and federated authentications.



Today i received a mail from one of my Blog readers. I think it is a general question about becoming a SOA architect and therefore i post his question to my blog, so that anyone can help him.
I will also do my share.

Hello Roger, 

I came across your blog and it is pretty helpful for individuals who are pursuing on a career path in SOA and integration area. 

I myself have been in the same path since the past 5 years associated with the iWay ESB. I am looking into strengthen my skills and grow into a role of an SOA consultant/Architect after having dealt with a variety of SOA based scenarios in the organizations where i have worked. But i have always found it challenging when we go into the job market as with many ESB tools in the market, its tough to market one self for that particular position as people will be looking for individuals having experience in the their own esb  tools. 

Even though the underlying functionality and concept is the same for every tool, still its tough to convince the companies to hire.

So how do i go about it and what sorts of skills should i pursue if i am looking to grow into as an SOA and Integration Consultant. I am looking to pursue the SOA Architect based certification from SOA School. More Knowledge regarding SOA i can gain from these certifications definitely, but how do i gain experience in the same if the industry keeps asking for experience on other tools.  

I was introduced to iWay from my first company and initially worked as a part of the support team and slowly grew into a developer's role and from then on i have been building my experience working on various integration scenarios using the iWay Service bus. I do indeed want to branch out and get to know how other industry based esb and etl tools handle these scenarios. 

I would be grateful to you if you could guide me with the same. 


My first reaction is: a tool is just a tool. Anyone with good skills can learn a tool. Another oneline: a fool with a tool is still a fool. And what i mean here, is that you have to know the concepts of doing SOA and integration, before you can use the tool correctly ! I would rather hire a consultant with good conceptual knowledge and some practical knowledge with a tool.
The challenge is to map that conceptual knowledge to the implementation of the tool !
* Use of a Common Data Model
* Use of eai patterns (i.e. store-and-forward, data-mapping)
* What to implement on the ESB and what to implement into a BPM tool
* How to expose the services (REST/XML/JSON/WebService)
* How to make the ESB artifacts testable
* How to divide the ESB implementation into manageable packages
* How to deal with technical and functional errors
* How to monitor
* What security is needed

With these skills you will become a far more valuable consultant, than just knowing the gritty nitty details of the tool....

Hope this helps !


API Management

I currently work for a customer that wants to expose its "services" to partners and other external consumers. They do not want to build websites themselves anymore, but instead have the services published and used by partners. So how can this be done?

Always curious and looking for good solutions i started to think about this.
I saw one of the following options:

  1. Just expose the (web)services through a web gateway and use http basic authentication over SSL for security
  2. Make REST adapters and expose those 


However soon i also discovered:
  • Consumers have to be managed. You have to manage the client credentials. For a few consumers this is doable, but for a couple of hundred or thousands this can be cumbersome
  • Consumers have to find the services somewhere. What is the functionality of those services? Is it soap or REST based?
  • Consumers want to test the services within a sandbox before actually using them
  • As a service provider you have to manage the life cycle of those services. You can not just make a service obsolete, because clients may still use that version
  • As a service provider you do not know how many times the service is used. You may want to throttle the usage, based on i.e. the clients or the services.
  • The services may also be used internally (SOA based services) and these services may be different. Internal maybe may use more data than external. How do you handle this?
  • As a service provider you want to expose services using different technical interfaces. For example through REST, soap or maybe even JMS. So you need (data) transformation functionality.

Functional components

The more i read about it, the more i came to the conclusion that API Management might be a good solution for this company. Such a platform (or product) comes with basically the following functional components:

  • Portal Configurator
    As a service provider you publish the APIs (services) and the documentation.
  • API Configurator
    Within the API Configurator you can define the exposed services and the integration with your backends.
  • Traffic Configurator
    Here you can define and monitor the security policies of your APIs.
  • Developer Portal
    Here the clients of your APIs can search for APIs and test them within a sandbox environment. Here the consumers can subscribe for APIs as well.
  • Traffic Manager
    All API calls run through a API gateway to canalize the traffic. Here the API keys are checked against the policies. May this user actually use this API.

Product selection

The next step in the process is to actually select a product. Because the API Management products are fairly new, this selection can not be based on my personal experience with one of the products. I did two things:
* Read a lot about API Management on the internet
* Talked to other people with more practical experience with products

So what to look for when selecting a product. This depends of course on the functionality you would like to have and the priority and weight of those requirements.
You can think of:
  • Which API Keys must be supported (i.e. OAuth, http basic authentication, certificates)
  • What are the backend systems you have to integrate with
  • Do you need manual and or automatic approval of API subscriptions
  • Do you want a sandbox environment for consumers
  • Which API interfaces do you want to be supported (i.e. REST/XML/JSON/Soap/JMS)
  • Which search features of the developer portal do you want
  • What governance supported? Multiple versions of the API
  • etc
You must also think about non-functional requirements, like
  • Costs
  • Is the community of the (freeware) product active or not
  • Do you have local support available
  • How is the documentation of the product
  • Do you want a local and or cloud version of the product
  • How mature is the product

Some products:
  • Layer7
  • Apigee
  • Mashery
  • WSO2
  • Azure API Management
  • IBM
  • SOA Software

Example: WSO2

Let me first say that i do favor WSO2 perse, but it is an open source product that is already very mature with its API Management product. It has more components that can be integrated as separate components. So whatever you want to use, you can plug in.