2016/08/23

WSO2 ESB - Design for testability

Introduction


Testability is one of the underestimated qualities of software. This is also the case for WSO2 ESB projects. However it is important to design the integrations for testability and this starts with the way you setup the proxies. This blog gives some guidelines which you can use to design for testability.

Sequences

Sequences are a way in WSO2 to group Mediators. A proxy has by default an in-Sequence, out-sequence and optionally an error-sequence. These sequences can be split up in sub-sequences and this is a good way for reuse, but also a way to split up the design. A proxy usually contains the following functional parts:
  • Validation
  • Transformation
  • Sending to an endpoint
 









These parts can be put in separate sequences. This has the advantage that the part scan be reused in other proxies. This is also the way to enable testing for these parts. 

Design for testability

The sequences are the basis for the testability of WSO2 ESB proxies. The following design guidelines and steps can be used for testing your proxies.
  1. Configure separate testable parts within a separate sequence
    A good guideline in splitting is that the parts should be as independant as possible from other parts. Good examples are: input validation, data transformation, sending a message to an endpoint, a step within an iteration.

  2. Define a separate developer studio project for the test proxies
    This will be the proxies that will contain a testable sequence and can be called from soapUI for example. This way also the test package can be deployed separately. This way the test project is not deployed on production.

  3. Define a soapUI project for testing the component
    It is wise to define a separate soapUI project for each component (WSO2 proxy) you want to test. Note that this can also be used within a Continuous Integration environment for automatic testing.

Example

The following is an example in which a data transformation is tested.
Step 1 – Define data transformation in a separate sequence



  
Step 2 – Define separate ESB project with test proxy
Note that the sequence is referenced and the result of the sequence is returned with respond mediator.
This result can be checked within soapUI using Asserts later on.

 


Step 3 - Define a soapUI project with test cases
In the request message put a soap message as expected by the sequence. Note that the ESB uses soap as its common message format within the mediation. The response can be checked with assertions.
Request



Response





Conclusion

The sequence split can be used to configure testable parts within a WSO2 ESB implementation.

This may not always be easy todo but hopefully you can use it to test your ESB implementations.
Feel free to comment on this blog ! All feedback is welcome

2016/05/23

WSO2 ESB 5.0.0: Data Mapper

One of the features i missed within the WSO2 Developer Studio was a data mapper.
We had to write our own XSLT or PayloadFactory within the product.
But everything is going to change with the release of ESB 5.0.0 !

Or is it ? ....

There is a first blog item on the datamapper that i read and tried:
https://nuwanpallewela.wordpress.com/2016/04/10/how-to-use-wso2-data-mapping-mediator-in-esb/

A very good blog indeed. However when i tried the new developer studio, i was somehow disappointed. I am positive about the fact that the Mapper is a real resource (even project) now. But i miss a lot of features (like for example XPath functions, if-then-else-construct, constraints) within the mapper.

As you can see from the picture below, only the following operators are supported:

  • Concat
  • Split
  • LowerCase
  • UpperCase



And when you investigate the generated project files, you will notice that JScript? is generated to do the actual mapping.



So my first impression is disappointed but it is a good start ! Hopefully new features and operations will be added soon, because at the moment i doubt if it is production ready.

But hey that's my opinion ;-)

2016/03/21

WSO2 API Management in DTAP

Using the WSO2 API Manager in a DTAP environment is slightly different than for example the ESB product. Nadeesha has written a good article about it: http://wso2.com/library/articles/2016/03/article-architecting-a-multi-environment-api-manager-deployment-with-wso2-api-manager/

My conclusion of the article and some take aways:

  • API Manager does not use Carbon Archive files because API Manager artifacts are both stored on file system and database
  • Use import/export facility to migrate APIs
  • There is also a bulk tool available to import/export a bulk of APIs
  • Environment specific attributes must be given as parameters
The downside of that last point is that each time new parameters are added, you must restart the API server.


Each backend will have its own endpoints, so this will require new parameters.
I would recommend a seperate file that must be imported within the wso2server.bat file.

2016/02/26

WSO2 Conference Asia


Introduction

Unfortunately i did not went to the WSO2 Conference in Asia but i went through the presentations and this blog item describes some interesting topics i came accross.

There were a lot of different tracks:

  • API Management 
  • Cloud
  • Strategy
  • Integration'
  • Security
  • Governance
  • Architecture
  • Analytics
  • DevOps
  • Internet of Things (IoT)

API

API (Management) is a hot topic. Almost all reference architectures use APIs to expose functionality, either to mobile apps or other apps. It is used within Microservices as contracts, it is used in connected enterprises that want to expose their APIs into the API economy system and it is used internally to manage and govern services.

IoT

Internet of Things is a also a big thing, so also at the WSO2 conference.
The WSO2 IoT Server was announced in 2016 Q2.












Features of this product:
  • Implementation of WSO2 IoT Reference Architecture
  • Device Management
  • Device Analytics
  • Device adapters
  • Supported protocols: MQTT, XMPP, WebSockets, HTTP
  • Device Control
It will also be part of WSO2 Cloud offering.

WSO2 Microservices Framework for Java (MSF4J)

Another interesting product is MSF4J, which is a lightweight, high performance framework for building microservices in Java.
  • It uses Java annotations to define microservices APIs and metrics
  • Out-of-the-box integration with WSO2 DAS for analytics and metrics of the APIs
  • Transport based on Netty 4.0
  • Low memory footprint
  • Fast startup
  • Swagger definition to code in Developer Studio









Analytics family

Analytics is becoming more attention and WSO2 has several products to support this:
Especially the Machine Learner becomes more interesting when "R" is integrated.

Cloud Offerings

Managed Cloud
One of the new cloud offerings of WSO2 is Managed Cloud. As the name suggests, your WSO2 environment is managed by WSO2. The environment is placed in the Amazon VPC cloud. This can be a viable solution for organisations how can not manage their environment themselves.
The regions that are supported: US East, US West, Asia Pacific, South America and Europe (where?).

Public Cloud
Cloud offering that offers WSO2 products in the public cloud. Currently only API Manager and App Manager (Beta) available. Will be extended with Identity, Device and Analytics platforms.

Cloud strategies
When dealing with Cloud you need to think of:
* Which Cloud services to use (there are a lot !) -> WSO2 Integration connectors
* Federated cloud identities (Facebook, LinkedIn, Google+, etc) -> WSO2 Identity Server
* Development in cloud -> WSO2 App Factory
* Running apps in the cloud  -> WSO2 Managed Cloud

Mobility

WSO2 has a new product called the Enterprise Mobility Manager. I did not go through those slides yet as i am focussing only on Middleware platform for the moment.

Integration

A few new features in ESB 4.9:
* Dynamic inbound channels
* Kafka, MQTT and RabbitMQ protocol support

New in upcoming ESB 5.0 release:
* Mediation debugger
* Data Mapper
* JMS 2.0 support
* Websocket protocol support

New WSO2 Gateway product with high performance message handling is used in several other products (API Manager, ESB,  App Manager, Load balancer).

Conclusion

Integration and API management remain an important topic within the WSO2 product suite. Some interesting added products are MSF4J, Enterprise Mobility Manager and IoT Server.
Also analytics become more important within the suite and security remains an important aspect of todays IT with BYOD and federated authentications.


2015/05/16

Question

Today i received a mail from one of my Blog readers. I think it is a general question about becoming a SOA architect and therefore i post his question to my blog, so that anyone can help him.
I will also do my share.

Mail:
Hello Roger, 

I came across your blog and it is pretty helpful for individuals who are pursuing on a career path in SOA and integration area. 

I myself have been in the same path since the past 5 years associated with the iWay ESB. I am looking into strengthen my skills and grow into a role of an SOA consultant/Architect after having dealt with a variety of SOA based scenarios in the organizations where i have worked. But i have always found it challenging when we go into the job market as with many ESB tools in the market, its tough to market one self for that particular position as people will be looking for individuals having experience in the their own esb  tools. 

Even though the underlying functionality and concept is the same for every tool, still its tough to convince the companies to hire.

So how do i go about it and what sorts of skills should i pursue if i am looking to grow into as an SOA and Integration Consultant. I am looking to pursue the SOA Architect based certification from SOA School. More Knowledge regarding SOA i can gain from these certifications definitely, but how do i gain experience in the same if the industry keeps asking for experience on other tools.  

I was introduced to iWay from my first company and initially worked as a part of the support team and slowly grew into a developer's role and from then on i have been building my experience working on various integration scenarios using the iWay Service bus. I do indeed want to branch out and get to know how other industry based esb and etl tools handle these scenarios. 

I would be grateful to you if you could guide me with the same. 

Regards,
Ravi


My first reaction is: a tool is just a tool. Anyone with good skills can learn a tool. Another oneline: a fool with a tool is still a fool. And what i mean here, is that you have to know the concepts of doing SOA and integration, before you can use the tool correctly ! I would rather hire a consultant with good conceptual knowledge and some practical knowledge with a tool.
The challenge is to map that conceptual knowledge to the implementation of the tool !
Examples:
* Use of a Common Data Model
* Use of eai patterns (i.e. store-and-forward, data-mapping)
* What to implement on the ESB and what to implement into a BPM tool
* How to expose the services (REST/XML/JSON/WebService)
* How to make the ESB artifacts testable
* How to divide the ESB implementation into manageable packages
* How to deal with technical and functional errors
* How to monitor
* What security is needed
Etc

With these skills you will become a far more valuable consultant, than just knowing the gritty nitty details of the tool....

Hope this helps !