2008/10/27

The Role of the Enterprise Service Bus

This was a presentation given by Mark Richards and describes the role of an ESB and particularly what it provides. It is an old presentation (23 oct 2006) but still very true, in the fact that there are still a lot of different definitions of an ESB within the world.
This is a short recap of this presentation.

Capabilities
He emphasizes that we need to understand the capabilities of an ESB. The capabilities are defined from an architectural context and he first explains why an ESB is used (loose coupling, agility, location transparency, services reuse, separate business services from service providers).

The following core capabilitities are described.
Note that not all capabilities need to be present within the ESB, but only use the one the customer needs. Next the capabilities are described in short.

Routing
The ability to channel a request to a particular provider based on routing criteria (types are: deterministic, content-based, policy-based, complex rules-based).

Message Transformation
The ability to convert the structure and the format of incoming request to provider needs.

Message Enhancement
The ability to add or modify the information contained in the message as required by the provider.

Protocol Transformation
The ability to accept one type of protocol and communicate to the provider through a different protocol (i.e. SOAP/JMS -> IIOP, XML/HTTP -> RMI).

Service Mapping
The ability to translate a business service into the corresponding service implementation and provide binding and location information.

Message Processing
The ability to manage state and perform request management by accepting an input request and ensuring delivery back to the client via message synchronization.

Transaction Management
The ability to provide a single unit of work for a business service request by providing a framework for the coordination of multiple resources across multiple disparate services (WS-Coordination, JSR-95). It is difficult to propagate transaction through for example MQ and HTTP. The solution would be to use an aggregate service.

Security
The ability to protect enterprise services from unauthorized access (authentication, authorisation, auditing) through security manager.

The two where you probably have the most discussion about are Process Choreography and Orchestration. Choreography is the ability to manage complex business processes that require the coordination of multiple business services to fulfill a single business request. Whereas orchestration manages multiple implementation services (aggregation of services).

Components
When looking at architecture components of the ESB with its responsibilities/capabilities he defines 4 components.
Mediator: routing, communication, message transformation, message enhancement, protocol transformation, message processing, error handling, service orchestration, transaction management, security.
Service Registry: service mapping.
Rules Engine: rules-based routing, message transformation, message enhancement.
Choreographer: Process choreography and only in case of entry point for the ESB: transaction management, security and message processing.

He then describes the relation between the mediator and choreographer and advises to use the mediator as the entry point and the choreographer underneath it. He advices to use a JMS resource as the entry point for an ESB because of its reliable delivery, monitoring and performance.
The advantages to set it up this way are: performance, scalability, only services that need choreography go through the choreographer.

JSR-208 JBI Specification
This specification of the Java community reached its final version in August 2005. It is an architecture and set of interfaces for providers of integration products to integrate with each other within a JBI container. It is highly flexible in which you can configure just the components you need within your ESB solution (by using Service Components). The way communication is handled with the bus is done with Binding Components.

The advantage is a vendor-indepedant ESB, flexible, light weight solution.

Open Source
There are two open source ESBs
  • Mule
    Light weight messaging framework that uses POJOs for implementing the routing, message enhancements etc, so you have to implement a lot.
  • ServiceMix
    Open source JBI compliant ESB with some out-of-the-box service- and binding components (like WSIF, HTTP, XSLT, Drools)


Opinion
I would like to give my opinion on the presentation:
  • In my opinion choreography is not part of the ESB capabilities, but is done within BPM. Orchestration can be done within ESB and is often seen as choreography. Or should the choreography be part of the ESB? Because you could also say that a choreography is a business service itself and it should be transparent for a service client where and how the business service is implemented.
  • The capabilities list can nicely be used to investigate which product is suitable for a customer. It is a nice checklist.

2008/10/24

BEA WebLogic: The NodeManager

The NodeManager is an important part of a weblogic server and this blog will give a short introduction and more detailed information on system administration can be found here.

Node Manager is a WebLogic Server utility that enables you to start, shut down, restart, and monitor remote WebLogic Server instances. To enable these capabilities, you run an instance of Node Manager on each physical machine in your domain.
WebLogic Server provides two versions of Node Manager, Java-based and script-based, with similar functionality. However, each version has different configuration and security considerations.
  • Java-based Node Manager runs within a Java Virtual Machine (JVM) process (start as a service). BEA provides native Node Manager libraries for Windows, Solaris, HP UX, Linux on Intel, Linux on Z-Series, and AIX operating systems.
  • A script based version is provideded for UNIX and Linux systems. This script is based on UNIX shell scripts, but uses SSH for increased security. SSH uses user-id based security.
Access Node Manager
The node manager can be accessed from:
  • Administration Server (Console or JMX utilities that you write)
  • WLST commands and scripts


What can you do with the Node Manager?

  • Start, Stop, Restart admin server
  • Start, shut down, suspend and restart Managed Servers
  • Monitor servers and view log files
    Node Manager creates a log file for the Node Manager process and a log file of server output for each server instance it controls.

2008/10/16

Gartner: About Oracle's Fusion Middleware

See here

The bottom line:
"The full integration of the BEA products into Oracle's middleware will not come out until somewhere in 2009"

Gartner analysts recommend waiting for Oracle Fusion Middleware (OFM) 11g, due in the next six to 12 months.

2008/10/12

Bea WebLogic Server JMS - Part 1 Basic JMS

This is the first blog in a series about WebLogic JMS (WLS 10.0). Full description can be found here.

The Java Message Service (JMS) is a standard API for accessing enterprise messaging systems.

WebLogic Server is compliant with the Java Platform, Enterprise Edition (Java EE) Version 5.0 specification and fully compliant with the JMS 1.1 Specification and can be used in production.

The major components of the WebLogic JMS are:
  • JMS servers that hosts a set of modules and any associated persistent storage that reside on a WebLogic Server instance.
  • JMS modules contains configuration resources conform the weblogic-jmsmd.xsd schema.
  • JNDI (Java Naming and Directory Interface), which provides a resource lookup facility. JMS resources such as connection factories and destinations are configured with a JNDI name.
  • WebLogic persistent storage (file store or JDBC-accessible) for storing persistent message data.
Messaging Models
JMS supports two messaging models: point-to-point (PTP) and publish/subscribe (pub/sub). Multiple queue senders and queue receivers can be associated with a single queue, but an individual message can be delivered to only one queue receiver.
Unlike with the PTP messaging model, the pub/sub messaging model allows multiple topic subscribers to receive the same message. JMS retains the message until all topic subscribers have received it.

Message Persistence
Messages can be specified as persistent or non-persistent.
  • A persistent message is guaranteed to be delivered once-and-only-once. The message cannot be lost due to a JMS provider failure and it must not be delivered twice. It is not considered sent until it has been safely written to a file or database.
  • Non-persistent messages are not stored. They are guaranteed to be delivered at-most-once, unless there is a JMS provider failure, in which case messages may be lost, and must not be delivered twice. If a connection is closed or recovered, all non-persistent messages that have not yet been acknowledged will be redelivered. Once a non-persistent message is acknowledged, it will not be redelivered.

WebLogic JMS extensions
In addition to fully supporting XA transactions, WebLogic JMS also features high availability through its clustering and service migration features, while also providing seamless interoperability with other versions of WebLogic Server and third-party messaging providers.
See weblogic.jms.extensions.

JMS API
To create a JMS applications, use the javax.jms API. For a complete description of all JMS classes, see the javax.jms or weblogic.jms.extensions Javadoc.

ConnectionFactory
A ConnectionFactory encapsulates connection configuration information, and enables JMS applications to create a Connection. You can use the preconfigured default connection factories provided by WebLogic JMS, or you can configure one or more connection factories to create connections with predefined attributes that suit your application. The default connection factories are: weblogic.jms.ConnectionFactory and weblogic.jms.XAConnectionFactory.
An XA factory is required for JMS applications to use JTA user-transactions, but is not required for transacted sessions. Another distinction when using the default connection factories is that you have no control over targeting the WebLogic Server instances where the connection factory may be deployed. However, you can disable the default connection factories on a per-server basis. A connection factory supports concurrent use, enabling multiple threads to access the object simultaneously.

Connection
A Connection represents an open communication channel between an application and the messaging system, and is used to create a Session for producing and consuming messages.

Session
A Session object defines a serial order for the messages produced and consumed, and can create multiple message producers and message consumers. The same thread can be used for producing and consuming messages. If an application wants to have a separate thread for producing and consuming messages, the application should create a separate session for each function. WebLogic JMS does not support having both types of MessageConsumer (QueueConsumer and TopicSubscriber) for a single session.

Destinations
A Destination object can be either a queue or topic, encapsulating the address syntax for a specific provider. Administrators can also configure a distributed destination, which is a single set of destinations (queues or topics) that are accessible as a single, logical destination to a client. The members of the set are typically distributed across multiple servers within a cluster, with each member belonging to a separate JMS server. Applications that use a distributed destination are more highly available than applications that use standalone destinations because WebLogic JMS provides load balancing and failover for the members of a distributed destination in a cluster.

Producer and Consumer
A MessageProducer sends messages to a queue or topic. A MessageConsumer receives messages from a queue or topic.

Message
A Message encapsulates the information exchanged by applications and it has header- and property fields and a body. Some header fields:

JMSCorrelationID
Specifies one of the following: a WebLogic JMSMessageID, an application-specific string, or a byte[] array. The JMSCorrelationID is used to correlate messages and is set directly on the message by the application before send().
There are two common applications for this field.
The first application is to link messages by setting up a request/response scheme, as follows:
  1. When an application sends a message, it stores the JMSMessageID value assigned to it.
  2. When an application receives the message, it copies the JMSMessageID into the JMSCorrelationID field of a response message that it sends back to the sending application.
The second application is to use the JMSCorrelationID field to carry any String you choose, enabling a series of messages to be linked with some application-determined value.

JMSDeliveryMode
Specifies PERSISTENT or NON_PERSISTENT messaging. This field is set on the producer or as parameter sent by the application before send().
JMSMessageID
Contains a string value that uniquely identifies each message sent by a JMS Provider.This field is set internally by send().
JMSReplyTo
Specifies a queue or topic to which reply messages should be sent. This field is set directly on the message by the application before send().

A message body contains the content being delivered from producer to consumer and can be one of the following types:
Bytes, Map (name/value pairs), Object (serializable), Stream (same as Bytes only primitive types allowed), Text, XML (Weblogic extension and facilitates message filtering which is more complex when done on a test message that happens to contain XML data)


2008/10/08

XA Transactions

I came across a good article about what XA transactions here.

Common Data Model on ESB

This item is about a CDM a.k.a CMM (Common Message Model) on the Bus, why is it wise and how can it be done.

First of all when you face a integration challenge the systems to be integrated are heterogeneous and use different data models (syntactically and semantically). So the data needs to be mapped from requester data to provider data.
1) Direct mapping, this results in n * m mappings (n=#requestors, m=#providers) and is very costly when a new provider or requester is added.
2) CDM, this results in n + m mappings
A CDM gives you:
  • Message consistency
  • Message maintainability
A CDM is a set of data representing the business entities used in all messages on the Bus. This does not mean that each provider or requestor uses the same set of messages but that the messages are all based on the same types.

Possible implementations for CDM on ESB
Option 1: ESB translates
  • The requestor and provider keep their own models
  • Only CDM is used on ESB
  • Existing services do not have to be changed (especially B2B)
  • Only m+ n translations to be implemented and managed
  • Easier to reuse mediation logic because it can use the CDM data instead of proprietary message models of the requestors/providers
  • Processing overhead? ESB becomes single point of failure?
Option 2: The services translate

  • The ESB is not responsible for data mapping
  • Transformations run in the request's/provider's environment
Option 3: The requestors/providers all use the CDM data
  • No transformations needed
  • Services can adopt the CDM data model
  • Reduces load and management on the ESB
  • Mostly unrealistic, hard to reach consensus on model, applications are legacy and cannot be changed
Option 4: Hybrid (combination of option 1 and 2)
  • It may lead to complex maintenance and management of the transformations.

Reference
http://www.ibm.com/developerworks/webservices/library/ar-esbpat3/index.html?ca=drs-

2008/10/06

Message Oriented Middleware

With Message Oriented Middleware you can have several different communication mechanisms. The most common are Queues and Publish-Subscribe. Queues are mostly point-to-point, so therefore not loosely coupled, location-based addressing. What does Pub-Sub give you?
  • Decoupling of Producers and Consumers
  • Producers and consumers do not know each other
  • Flexible number of producers and consumers

Addressing Models

With pub-sub there are a number of addressing models.
  • Channel or Topic based
  • Subject based
  • Content based
  • Concept based (not further described here, but tries to describe interest on a high level)

Topic based addressing

This is simple but less powerfull. Example JMS topics.You basically get every message that is published on a particular topic. However you have the possibility to set a filter when a subscription is taken that will filter the messages on attributes in the header of the JMS message.
This is supported by BEA AquaLogic Service Bus.

Subject based addressing

It avoids the use of of physical network addresses. Messages are labeled with subject and consumers can subscribe on subjects. Producers do not need to know how messages are consumed and consumers do not need to know the source. There must be agreement on subject names and subjects can be added dynamically.The names consist of a hierarchy of elements: element.subelement.subsubelement.
Consumers can use wildcards: BOOK.* matches BOOK.TITLE and BOOK.> matches BOOK.TITLE.DESCR.
The disadvantage is that subject hierarchies can not be changed easily.

This type of addressing is supported Tibco's Rendezvous.

Content based addressing

A content based filter is a predicate on the content of the messages.


JMS

The JMS publish and subscribe is a combination of Topics and expressions on envelope's attributes. Factories, Destinations are identified via JNDI.
The message header is used for addressing and extra message properties, which are name value pairs, can be set. A selector (SQL-like expressions) can be used by consumer to filter messages based on the header's properties.