2009/12/29

JDBC error when deploying

I encountered the following exception when deploying a JDBC configuration to Weblogic Server:
 com.rsa.jsafe.JSAFE_PaddingException: Could not perform unpadding: invalid pad byte.

After some analysis this error occurs when you encrypted the password for the JDBC connection in another domain as where you deploy the JDBC configuration.
Conclusion: You should always encrypt on the same Domain as you deploy the configuration !

PS The password is encrypted within the password-encrypted element.

2009/12/04

Oracle Workspace Studio: Validation failed with diagnostic messages

The last couple of days i run into Oracle Workspace Studio problems.
They are of the form:
Validation failed with diagnostic messages. Diagnostics for WSDL xxxxx
    ERROR: <0> Validation of xxx  failed:

The only remedy i found so far is to create a new Workspace and import the project(s).

2009/11/25

Automatic XQuery unit tests for OSB projects

Within the OSB 10.3 you use XQuery code to perform data transformations. These transformations can be very complex. I am in favor of TDD and i want to use automatic tests for XQueries. You can use the OSB Console to test XQueries but this is not very practical with complex Queries and takes a lot of time.You can also use XQuery and JDeveloper to run XQueries and this inspired me to write a simple XQUnit module to test xqueries.
This blog shows you how to set up automatic tests with the use of JUnit, XMLUnit and Oracle XQuery engine.

The contraints for me were:
1) The XQuery code should be as-is, so no changes with scripts or whatever. The XQueries are used within the test framework as used within the OSB.
2) The XQuery engine of the WLS must be used. I could use some other XQuery engine, but this would not test the real WLS engine and still errors could occur runtime.
3) The test framework can be used within Java unit tests. This way it can also be used within Maven for instance.


XQuery transformations
XQueries are used within the OSB to perform data transformations. These XQuery modules use external variables that are binded to OSB $variables.

An example of such an XQuery (test.xq) is:
declare namespace wn = "http://www.example.com/werknemer/v1";
declare namespace xf = "http://tempuri.org/test/";
declare namespace np = "http://www.example.com/natuurlijkpersoon/v1";

declare function xf:Medewerker2Werknemer($medewerker as element())
    as element(wn:Werknemer) 
{
  
    
       {concat(data($medewerker/np:Voorvoegsels), " ",  
        data($medewerker/np:Achternaam))}
    
  
};

declare variable $medewerker external;
xf:Medewerker2Werknemer($medewerker)

As mentioned already i used some ideas from Biemond's blog to run an xquery. I added some constructions from XMLUnit and developed the following XQUnit code.
The libraries needed are:
* JUnit.jar (3.7 or higher is good)
* Xmlunit-1.3.jar (Downloaded from XMLUnit site)
* XML Parser v1
* XML Query


package unittest;


import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.Reader;
import java.io.StringWriter;
import javax.xml.namespace.QName;

import javax.xml.transform.stream.StreamSource;

import oracle.xml.parser.v2.DOMParser;
import oracle.xml.parser.v2.XMLDocument;
import oracle.xml.parser.v2.XMLNode;
import oracle.xml.parser.v2.XMLParseException;
import oracle.xml.xqxp.datamodel.XMLSequence;
import oracle.xquery.PreparedXQuery;
import oracle.xquery.XQueryContext;
import oracle.xquery.exec.Utils;

import org.custommonkey.xmlunit.DetailedDiff;
import org.custommonkey.xmlunit.Diff;

import org.custommonkey.xmlunit.NodeInputStream;
import org.custommonkey.xmlunit.jaxp13.Validator;
import org.custommonkey.xmlunit.XMLAssert;

import org.w3c.dom.Document;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;

import org.xml.sax.SAXException;


public class XQUnit {

   public XQUnit() {

   }
   
    public static boolean checkSchema(Document doc, String element, String schemaLocation) {
        NodeList list = doc.getElementsByTagName(element);
        Node bodyNode = list.item(0);
        Validator bodyValidator = new Validator();
        File orgschema = new File(schemaLocation);
        bodyValidator.addSchemaSource(new StreamSource(orgschema));
        NodeInputStream bodyStream = new NodeInputStream(bodyNode);
        StreamSource src = new StreamSource(bodyStream);
        boolean valid = bodyValidator.isInstanceValid(src);
        
        return valid;
    }
    
    public static boolean checkSchema(String file, String element, String schema) {
        boolean result = false;
        try {
            DOMParser prs = new DOMParser();
            InputStream is = Utils.getStream(file);
            prs.parse(is);
            XMLDocument doc = prs.getDocument();
            result = checkSchema(doc, element, schema);
        } catch (SAXException e) {
            // TODO
        } catch (IOException e) {
            // TODO
        }
        
        return result;
    }

   public String runXQuery(String xq, String testdoc, String extvar) {
        return runQuery(xq, testdoc, extvar);
    }
   
   public void testXQuery(String xq, String testdoc, String var, String expected) {
     boolean testresult = false;
     try {
       // Run the query
       String resultxml = runQuery(xq, testdoc, var);
                     
       // Create expected result XMLDocument
        int ch;
        StringBuffer strContent = new StringBuffer("");
        FileInputStream fin = null;
         fin = new FileInputStream(expected);
         while( (ch = fin.read()) != -1)
           strContent.append((char)ch);
        fin.close();

       // Check the difference      
       Diff dif = new Diff(strContent.toString(), resultxml);
       XMLAssert.assertXMLEqual(new DetailedDiff(dif), true);

       } catch (FileNotFoundException e) {
        e.printStackTrace();
       } catch (IOException e) {
         e.printStackTrace();
       } catch (SAXException e) {
         e.printStackTrace();
       }   
   }

   private String runQuery(String xq, String testdoc, String extvar)  {
     String resultxml = null;
     try {
       // Create XQuery
       XQueryContext ctx = new XQueryContext();
       Reader strm = new FileReader(xq);
       PreparedXQuery xquery = ctx.prepareXQuery(strm);
       
       // Create test XMLDocument
       DOMParser prs = new DOMParser();
       InputStream is = Utils.getStream(testdoc);
       prs.parse(is);
       XMLDocument doc = prs.getDocument();
       
       // Run XQuery
       xquery.setNode( new QName(extvar), (XMLNode)doc.getDocumentElement());
       XMLSequence seq = xquery.executeQuery();
       
       // Convert to String
       seq.next();
       StringWriter wr = new StringWriter();
       seq.getCurrentItem().getNode().print(wr);
       resultxml = wr.toString();
     } catch (FileNotFoundException e) {
        e.printStackTrace();
        resultxml = null;
     } catch (SAXException e) {
        resultxml = null;
     } catch (IOException e) {
        resultxml = null;
     }
     System.out.println("runXQuery result: " + resultxml);
     return resultxml;

   }

   public static void main(String[] args) {
       XQUnit testXQuery = new XQUnit();
       testXQuery.testXQuery("test.xq", "TestXML.xml",
        "medewerker", "TargetWerknemer1.xml");
   }
}
The following example XML file (Medewerker1.xml) is used as a test example:
 
        Kimmenade
        van de


The expected result of the test xquery is (TargetWerknemer1.xml):

   van de Kimmenade



The test code to test is then (AllTests.java):
package unittest;

import junit.framework.TestSuite;
import junit.textui.TestRunner;

public class AllTests {
    
    public static void main(String[] args) {
        AllTests tests = new AllTests();
        TestRunner.run(tests.suite());
    }
    
    public static TestSuite suite() {
        System.out.println("Start Werknemer2WerkgeverTest");
        
        TestSuite suite = new TestSuite("All");
        suite.addTestSuite(unittest.AchternaamTest.class);
        
        return suite;
    }
}
The next is the actual test case for Achternaam:
package unittest;

import org.custommonkey.xmlunit.XMLTestCase;

public class AchternaamTest extends XMLTestCase {

    protected void setUp() {}
   
    public void testTransformation1() throws Exception {
        XQUnit xq = new XQUnit();
        xq.testXQuery("test.xq", "Medewerker1.xml",
         "medewerker", "TargetWerknemer1.xml");
    }
}

Only test part of the result

In case you want to test only some parts of the result XQuery you can use some handy XMLUnit constructions. The following example shows it.
package unittest;

import java.util.HashMap;

import org.custommonkey.xmlunit.SimpleNamespaceContext;
import org.custommonkey.xmlunit.XMLTestCase;
import org.custommonkey.xmlunit.XMLUnit;

public class VoorkeurnaamTest extends XMLTestCase {

    private SimpleNamespaceContext ctx;

    protected void setUp() {
        HashMap m = new HashMap();
        m.put("wn", "http://www.example.com/werknemer/v1");
        ctx = new SimpleNamespaceContext(m);
        XMLUnit.setXpathNamespaceContext(ctx);
        
    }

    public void testAchternaam() throws Exception {
        XQUnit xq = new XQUnit();
        
        String res = xq.runXQuery("test.xq", "Medewerker1.xml",
         "medewerker");
        assertXpathEvaluatesTo("van de Kimmenade", "/wn:Werknemer/wn:Achternaam", res);
    }
}

Conclusions
* The XQUnit framework is not complete, but can be extended to support multiple external variables.
* The XQUnit framework also supports the check of XMLSchema, but this is not shown in an example.
* At the moment the JDeveloper XQuery engine is used (Somebody knows which WLS jar to include to use the real WLS XQuery engine?) 


Feel free to comment on this blog item!

2009/11/16

Technical CDM model considerations

There were probably already hundreds of discussions about versioning XML schemas and WSDL documents. This Blog considers the strategy described in Thomas Erl's "Web Service Contract Design and Versioning for SOA".
How would you use these practices within a CDM? This Blog considers some options you have when developing XML Schemas out of you logical CDM.
Consider a small CDM in which the following model is used.

LegalEntity <-- NaturalPerson <-- Employee
                  <-- NonNaturalPerson

Namespaces

For these entities you have some possibilities to use namespaces:
1) A group of Entities share the same namespace
Example: http://www.blog/organisation/v1

2) Each Entity gets its own namespace
Example:
http://www.blog/organisation/legalentity/v1
http://www.blog/organisation/naturalperson/v1

The advantages of option 1 are
  • Only 1 namespace needed, namespaces are hell as we all know
  • The entities belong together so this seems logical
The disadvantages
  • The CDM entities are used within Service Contracts (WSDL) and when the version of the namespace changes all Services that use one of these Entities is changed. In fact all Service Consumers have to be adapted.
  • When schema elements are "qualified" all data transformations (XSLT and XQueries) have to be changed, because these use the same namespace within the XPath constructions.
So this option leads to a lot of rework when the major version changes.

Lets consider option1 in which each Entity gets its own namespace.
In this case the disadvantages are
  • You will get a lot of namespaces.
But the advantages are
  • When one Entity changes its namespace only those Services that use this Entity changes.
  • Not all data transformations have to be changed, only those that use this Entity.
  • Better governance on the Entities.
Separate XSD files
As also described as a SOA pattern by Thomas Erl, it is best practice to have each Entity within a separate XSD file.
  • Better reuse of Entities
  • Better governance of the Entities
So you have LegalEntity_v1.xsd and NaturalPerson_v1.xsd files.

Entity inheritance relations
As described by the logical model LegalEntity is a base of NaturalPerson.
You have the possiblity to design this as an xml:extension.

When you have an InternalEmployee and ExternalEmployee Entity how would you consider a Service in which an Employee is needed as input. You could consider to use the xml:choice construct within the message (1). Or you could define two different operations: AddInternalEmployee and AddExternalEmployee (2).

The disadvantage of option (1) is that within your Service (or on the ESB) you will have if-constructions to check with which kind of Employee you are dealing with.

This Blog described some options you have when dealing with the technical CDM.
Please leave you comments on how you are dealing with it...




2009/11/04

JPA problem: Column override on fields

Today I was struggling with WLS 10.3, Workshop 10.3, JPA and Hibernate to get an Entity persistent to a Database. I got the following error:


I used the following code (which was acutally generated by Workshop!):

@Entity()
@Table(name="werknemers")
public class Werknemer implements Serializable {
 //default serial version id, required for serializable classes.
 private static final long serialVersionUID = 1L;
 
 @Id
 private String persNummer; 
 private String emailInternet;
    public Werknemer() {
    }

 @Basic()
 @Column(name="email_internet", nullable=false, length=128)
 public String getEmailInternet() {
  return this.emailInternet;
 }
 public void setEmailInternet(String emailInternet) {
  this.emailInternet = emailInternet;
 }
}

As you can see from the error code the column name "email_internet" (as indicated by @Column) was not taken, but instead the "emailInternet" property was taken.
I could not find the problem, but the following helped. Move the JPA annotation above the private property, like this:

        @Basic()
 @Column(name="email_internet", nullable=false, length=128)
 private String emailInternet;

This solved my problem, but not sure if this is a Hibernate, JPA bug?




2009/11/02

International SOA Symposium 2009



On the 22th and 23th of october 2009 the second SOA Symposium and first Cloud symposium took place in Rotterdam. For me it was the first SOA Symposium. The main theme of this symposium was the introduction of the SOA Manifesto by some respectful people of the SOA community like Thomas Erl, Anne Thomas Manes and Grady Booch.

There were 6 SOA tracks, 2 Cloud computing tracks and a working group that worked on Candidate SOA Design patterns and the SOA Manifesto.

The first day I followed:
The Convergence and Unification of SOA, EDA & BPM, by Manas Deb, Oracle
This presentation was about to think about the business events that occur within your business and to take these events into account early. So it should be natural to combine events, services and processes to create the need for the business. Events were also used to split complex processes.
At the end of the presentation his new book was given away and I got hold of one piece (see also here for a book review).

Service Modelling: Making Sure Your Services Deliver Value, by Richard Watson, Burton Group

This talk was about focus on service design principles rather than technology. The identification and definition of sharable and reusable services is still more art than science. He believes that developers must start thinking out-of-the-box instead of in there technical world.

I agree with him to some extend, but that's what they are hired to do, maybe the role of the architect is more relevant here. I also see that architects just design point-2-point interfaces without thinking "Services".

SOA and Cloud Computing - A Match Made in Heaven? by David Chappell, Oracle


I followed this presentation to get some feeling about what Cloud computing is, and what the relationship with SOA is.
He began his talk about the weeks/months it takes to deploy new applications/services on hardware. He then introduces the Cloud in which hardware and other resources can be managed by configuration. This significantly increases the speed of deployment.
Then he also introduces the Grid in which data is made persistent and available for all services. There I get lost a little and wonder if this is the way you would want to go. I would expect this to be hided from the Service Consumer, and that this data is accessible through a Service. He foresees public- and private clouds.

Mission impossible? Applying Agile to the World of SOA and ERP, by Sander Hoogendoorn (Capgemini) and Twan van den Broerk (Ciber)
This was a great real example of how Agile can help (even) SAP projects to deliver on time and what the customer wants. This presentation was about the use of Smart Usecases and Scrum within a SAP project.


 I like the use of Scrum or other iterative and agile approaches when doing a software project. I also strongly believe in its value in integration and SOA based projects. This presentation showed its value in real example project.




 


Next Generation SOA: A Web-Centric Perspective, by Stefan Tilkov, InnoQ

Stefan Tilkov is a pioneer on exposing Services as REST services, so this talk was also on REST. After an introduction on resources, URI and representations he talked about Atom.
He concluded with some recommendations:
* Ensure your webapps are RESTful
* Expose machine readable info via HTTP Get
* Manage your metadata with RESTful HTTP
* Use WS-* if policies or legacy forces you

I personally like the REST style of interfacing, but only if this is the "best" way. I really like the fact that each resource object gets its own URI that can be used for linking. I also like the fact that a resource can have multiple representations. So a resource can be represented as a WebPage, POX (Plain Old XML), JSON or whatever. The WS-* versus the REST continues ;)

Between the sessions..
Between the sessions I had the opportunity to talk to some interesting people. I spoke among others with Brian Lokhorst who works for the DTCA ("Belastingdienst") and he also uses SOA Design Patterns to encapsulate legacy systems.
Furthermore I talked with a Belgian company (Collibra) that developed a product for managing "semantic data". This product manages the enterprise data, not only syntactically but also semantically and can generate technical models and transformations. At runtime it will keep track of semantic correctness.


There was a funny presentation by Gray Booch who could not be physically present and did his presentation via SecondLife.

Day 2

Introducing the modern ESB, by Satadru Roy from Sun and Brain Loesgen from Microsoft
The only interesting and funny part was that SUN and Microsoft did this presentation together. There was a demo of Biztalk but further nothing new.

 

Understanding SOA Governance by Harold van Aalst, Progress Software
This was a presentation about Progress Actional governance tool of Progress.
I wonder if these tool can really help you. There were only a few Dutch customers using the tool at that moment.

Real-life use cases of SOA Design patterns at the Dutch Tax and Customs Administration, by Brian Lokhorst
This talk was about the usage of two SOA design patterns used within the DTCA.

Introducing Transactions to REST, by Michael Musgrove, Red Hat
This talk was about transactions (Compensations and Atomic) the REST way. A demo was given using the
RESTeasy framework.

It was a bit ackward interface of which the client must have knowledge. It remains a complex problem to solve also within the WS-* world in which the standard is not used very much (yet)?

SOA Manifesto
But the main theme of the symposium was the introduction of the SOA Manifesto which can be seen here. I hope this will help take SOA on a higher level, let SOAlize the enterprise !!

2009/10/29

Book review: Getting Started with Oracle SOA Suite 11g R1


I got this book on the second SOA Symposium in Rotterdam from one of the authors (Manas Deb) who happen to give a presentation there on SOA, BPM and EDA. He signed the book, so I have a unique piece :)

This book gives you a prefect head start with the Oracle SOA Suite 11g. It explains the concepts of SCA, Database-, JMS- and File Adapter, BPEL, Human Tasks, Business Rules, OSB, Unit test of composite SCA applications, Security, Exception Handling, BAM, Event Delivery Network, SDO and B2B.
All these concepts are explained with small tutorial examples.

If you want to have in depth knowledge of the SOA Suite, this is not the correct book.

2009/10/21

Should a SOA Architect be certified?

I am currently following the SOA Certified Architect traject at SOASchool and have finished my first module (jippie). For me the reason to be certified are:
  • It gives you a good basis to understand what SOA is all about
    This way you can put this into better perspective when using or selecting SOA tools like ESB and BPM. I always compare it with learning first Object Orientation before diving into C++ or Java. This way I truely believe you get better (reusable, maintainable) code.
     
  • Just that little more focus on the content then by just reading a good SOA book
    It forces you to really study the material.

  • It helps when planning an architecture roadmap
    I know that you always get presented the ideal SOA picture, but at least it helps you when you have to define a SOA roadmap. It also helps you where to give attention and what kind of discussions you have to do. This way it will help to change the mindset of developers/designers/architects/business.

However the question remains "What is the value of this certification within the market and how is it valued by customers?". I still do not discover the questions by customers, but Java certifications are valued. Maybe it is a question of time?
Anyone?

2009/10/12

Book review: Understanding SCA

I bought this book to get a global overview of the possibilities of SCA, because it is also implemented within the Oracle SOA Suite 11g. It is the first book on the fairly new SCA standard.



If you want to get a global overview the first chapter will already do. The next chapters go deeper into the concepts with Java as the example programming language.

Chapter 1 - Introducing SCA

The book starts with a global introduction of SCA and its place w.r.t. SOA, J2EE, .NET, Spring and distributed computing. SCA focuses on being able to describe assemblies of components which have been written in a variety programming models and protocols. It does not cover presentation and persistency.

Within this chapter the main terms are explained: Service, Component, Composite and Domain.
Why another standard? Because there are still some issues with J2EE and .NET, namely complexity and reusability. You have to understand a lot of different technologies like JAX-WS, JMS, EJB, ASP.NET, .NET Messaging.

Chapter 2 - Assembling and deploying a composite
This chapter describes the creation of composites and deploying them into a domain as a contribution. Fabric3 (open source Java SCA implementation) with Java is taken as example language.

Chapter 3 - Service-Based development Using Java
This chapter shows how to create Java services (asynchronous and synchronous) and it is described how the life cycle of components are handled.

Chapter 4 - Conversational Interactions Using Java
Chapter about implementing conversational interactions in which several interface calls belong together as one conversation.


Chapter 5 - Composition
This chapter describes in more detail how applications are assembled into composites. Its about service bindings, performance, properties, including other composites.

Chapter 6 - Policy
This chapter is about policies wich include authentication, encryption, signing, reliability, transaction propagation.

Chapter 7 - Wires
This focuses more in details the possibilities with wiring components. It uses the Inversion of Control pattern to inject dependencies. This can be automatically done by the SCA runtime or at configuration level.

Chapter 8 - Bindings
Bindings are used to expose the services of components to external clients. This chapter describes it in more detail. It covers the usage of WSDL and JMS.

Chapter 9 - The Domain
The domain is the SCA runtime in which management, policies administration, communication infrastructure and resource sharing is handled. There are local, distributed and federated domains. Components are installed as contributions within the domain.

Chapter 10 - Service-Based development using BPEL
The BPEL is also taken as an example to develop SCA components. Some SCA extensions to bpel are covered.

Chapter 11 - Persistence
This chapter describes a way to incorporate persistency into the services. Persistency is not part of the SCA standard. It covers JDBC and JPA.

Chapter 12- Presentation
Also the presentation layer is not part of the SCA. This chapter describes the way to integrate with technologies like JSP and servlets.



Conclusion
SCA is Spring in the large that uses the Dependency Injection pattern to wire components into composite services. Note however that Spring also has Spring Integration for integrating distributed components.

Oracle SOA Suite 11g supports the SCA way of working, so that is a good base for its adaptation, but I wonder if you always need this kind of flexibility of protocols used between components. And the configuration based approach may also be a reason for some not to use it.
I have not seen it used within the projects i worked for, but i am curious in using it within the Oracle SOA Suite.
It is a great book to get you introduced into the SCA standard.

Some interesting links on SCA

2009/10/11

What happened with SOA?

I came accross this article about the question "What happened to SOA?"

It is stated that the technical SOA part is a succes but within the whole enterprise it fails (most of the time).
It fails most of the time because its still is damn difficult to close the gap between business and IT and SOA does not help either as hard numbers, blogs and experience show.

What came to my mind were two things,
  1. enterprise architecture must give you control over your IT (applications, hardware and interfaces). If you reach this point, it does not matter how it is setup (SOA may help). As long as you can keep the operational costs low. So make posters and communicate the architecture, something i see rarely done.
  2. Projects are too complex and big, so maybe we should start thinking small features/processes and use Agile methologies and start having multidiscipline teams with business representatives in it, instead of staying in the ivory towers of the IT department.

2009/10/10

Why not do Agile (EAI/SOA) projects?

The Agile approach is not new anymore, but still I see problems with trying to do projects the Agile way.
What I see:
  1. Business plans have to be made with functionality and ROI upfront
    This does not fit the Agile way (Customer Collaboration over contract negotiation). The ROI is sometimes hard to calculate and what you see are naive numbers just to get the project approved. Furthermore it is a long and costly phase. Of course you have to discuss if a functionality is worthwhile.

  2. The business wants to know upfront what the costs are and the needed resources
    This is because resources (people) need to be reserved and hired. They want to know the investments needed in advance because usually budgets are given each (half) year.

  3. A project is divided in several subteams which must be coordinated
    The teams only communicate in the beginning (for the requirements) and at the end (integration testing). They do not communicate during the project and do not integrate during the project. I call this the big bang approach.

  4. Projects do not want to pay for future advantages for the enterprise
    Especially in the beginning architectural decisions, POCs and implementations have to be made. Think of product selection for BPEL, BPM, ESB, etc. The business does not want to pay for all these (needed!) upfront investmentsfor the sake of the enterprise.

  5. Documents are made just to satisfy the QA process
    Technical design, unit test plan for example are written upfront without review or written afterwards. These documents are most of the time outdated and are not maintained.
Ad 1
Begin with small projects where the ROI is easy to calculate and small. Take a small part of a business process. Maybe we should even call it a feature/business process plan instead of project plan and keep them small and manageable.

Ad 2
You could reserve some budget for feature (enhancements) and after each iteration/sprint (or whatever method you use) choose to continue or stop. This way you never go over budget.

Ad 3
If you keep the features small a lot of communication may be circumvented and communication is easier. However in lots of cases the places in which the team work are different. Maybe modern tools like wikis, twitter, use groops, google talk etc may help to communicate better and often.

Ad 4
Some budget may be reserved not only for the business but also for enterprise architecture in general. This way such POCs and product selection trajects may be funded. This means management support is really needed, but this is always the case in such trajects.

Ad 5
I like the use of TDD (Test Driven Development) in which automatic tests are made and ran during builds. It takes some effort but it really is very helpfull during refactoring and regression tests. It also makes designs better because it forces you to think about the testability of the component. Also document only the most important design decisions and why some decisions are made. Those decisions can be very valuable in the future, instead of describing attributes of classes in detail for example. These can be (and should be) documented in the code.

I am curious to hear from you in other issues and/or solutions for these issues!

2009/10/08

Installing Oracle SOA Suite/JDeveloper 11g

This post guides you through the installation of the Oracle SOA Suite 11g with JDeveloper 11 with the use of Oracle XE database (for Windows).
Note: It is assumed that Oracle XE database is already installed.


Required downloads

The detailed steps are nicely described here.
The next picture gives a nice overview of the directory structure created.




Step1 -  Create Schemas for Oracle SOA Suite and Oracle BAM (RCU)

  • Unzip the files
  • Run bin/rcu.bat
  • At the Database Connection Details fill in the XE details. If you encounter the next fault you have probably not started the XE database.

  •   Ignore all warnings

  • Click OK
  • Select all components

  • I ran into the following problem:

When you click Ok you see the following error:

And this is about the Processes DB Init parameter that is wrong.
To fix this goto the Database Home page, Sql Editor and execute the following statements:
alter system set PROCESSES=500 scope=SPFILE;
alter system set open_cursors=500 scope=SPFILE; 
If everything is ok you should see the next screen.
 
  • Further ignore errors from portal installation

Step 2 - Install Oracle WebLogic Server and Create the Middleware Home
  • Run wls1031_win32.exe
  • Choose Middleware Home (example: d:\oracle\middleware)
  • Register for security updates, leave empty, Next
  • Choose Typical install type
  • Choose Weblogic Server Home (example: d:\oracle\middleware\wlserver_10.3)
  • The next screens choose Next and the installation progress screen is shown
  • After a while you are done with this step
Step 3 - Installing SOA Suite and BAM
  • Unzip ofm_soa_generic_11.1.1.1.0_disk1_1of1.zip
  • Run setup.exe within Disk1 directory
  • When asked for the Java jre use middleware home (d:\oracle\middleware\jdk160_11\jre)
  • When the checks are finished the installation locations are asked:
    Use the Middleware Home (selected by default) and use Oracle_SOA1 (also default)
  • The following components are installed: BPEL PM, Mediator, Rules, B2B, Human Workflow, BAM

Step 4 - Creating Domain

  • Run Configuration Application Server within SOA
  • Create a New Domain
  • Select all products to be supported
  • Enter a Domain name
  • Configure JDBC Component Schema

  • Select all optional configuration

  • Configure the Administration Server, leave unchanged
  • Configure Managed Servers, leave unchanged
  • Configure Clusters, leave unchanged
  • Configure Machines, leave unchanged
  • Assign Servers to Machines, leave unchanged
  • Target Deployments to Clusters or Servers, leave unchanged
  • Target Services to Clusters or Servers, leave unchanged
  • Configure JMS File Stores, leave unchanged
  • Configure RDBMS Security Store Database, leave unchanged
  • Summary
  • Create
Step 5 - Install JDeveloper
  • Run jdevstudio11111install.exe
  • Choose the existing Middleware Home directory
  • The rest is Next, Next, ...
  • After JDeveloper is installed start JDeveloper
  • Choose Help -> Check for Updates
  • On the Source page choose:

  • On the Update page select the extensions to download

  • Note: You need an Oracle Web Account. 
  • After the download is finished you have to restart JDeveloper.
That's it and now you can start developing SOA components using SCA, but that is an item for another blog item ;)

2009/10/05

SOA and Agile a good match?

Everytime when I see efforts in trying to implement a SOA within an enterprise you see the gap stays between the IT department and the business. It is very hard to sell SOA to the business, even if the concept of Services is close to the mindset of the business.
So can Agile development be a good approach?


I mean Agile has Customer collaboration as one of its key principles in its Manifesto. I see a lot of big project contracts made which are stopped or constantly changing, so why not collaborate more with the customer (business) and establishing the Services together?!
This will perhaps change the gap, and the Agile approach is not only for user interface design as I also hear a lot. Why not for integrating systems and implementing business processes? These processes can also be broken down in small peaces.
And why should the business be in different work locations as the IT department?
So I think that the Agile approach can be a good way to bringing the business and IT together.

2009/09/30

Integrating Subversion into Eclipse 3.x

This blog is about the steps to take to integrate Subclipse into Oracle Workshop for Weblogic 10.3.

1) Goto Help -> Software Updates -> Find and Install ...
2) Check "Search for new features to install
 
3) Click on the "New Remote Site.." button
4) Enter the next information in the dialogue:

5) Select the new site and click Finish

6) The screen shows the features that can be selected, check as shown below and click Next


7) Accept the license agreement, click Next
8) Click the Finish button on the Installation dialogue
9) The software is downloaded.


10) At the Feature Verification dialogue, click Install All
11) You have to Restart Eclipse, so choose Yes
12) The choose Window -> Open Perspective -> Other and choose SVN Repository Exploring

13) Now you can add your repositories.
14) You can change the SVN preferences, under Window -> Preferences -> Team -> SVN



And thats it.

You are now also able to check out projects from the repository:
1) Goto the Project Explorer and choose Import

2) Select to import from SVN


3) Choose an existing Repository or create a new one (see step 13)
4) Choose Next and off you go.

2009/09/25

Oracle SOA Suite 11g Introduction

If you want to have a good overview of the SOA Suite products you can visit this page.
Also some links to tutorials are given.

2009/09/23

Moving away from file interfaces

In the company I work for, I see a lot of batch oriented, file interfaces. An example is within the proces of where an employee gets employed.
This new employee ie entered within the PeopleSoft system and once a day an export is ran of all employees. This file contains the action that has to be taken towards other systems (add, delete, update). The other systems are phonebook system and authorisation system in which the new employee must be registered too. There is even a separate database that keeps track of the files already processed!

This is not what you want, but how would you integrate the systems? Should you use an ESB? Should you use a Common Data Model? Should you include BPM? Should you add another presentation layer?



All this creates a lot of overhead for a secondary process, just to SOAlize/BPM it. So what are the benefits to use this elegant IT solution?

I think:
* Monitoring of the process, so better maintenance
* Easier to change the process (for example other authorisation levels for different employees)
* Easier to change an implementation for a system
* Realtime update of the employee information instead of waiting a day or more

What's your experience in these kind of situations?

2009/09/22

"Not valid in an Oracle Service Bus Configuration project"

This is the error I got when importing an existing project into Workspace (version 10.3).


I found nothing to circumvent the error and the only way was to create a new Configuration and copy the projects into it.

2009/09/21

Reuse XQuery within OSB 10gR3

XQuery is a great language that can be used within the OSB for translations and other expressions. Unfortunately the OSB does not support the module construction of XQuery. A module construction can be perfectly reused for code reuse.
This post shows how this omission in the OSB can be solved.

This example shows how a message with a header and a body can be created of which the header code is reused.

1) First the header function is created (without parameters, but these can also be added).


2) Add another XQuery that creates the message with the header and body content as parameters.


3) Within a Stage or where you want to use the construction add an Assign.


4) The expression must be the Header XQuery.


5) Add another Assign for the message XQuery and add the $body and $header as binding variables.


And that's it, so the trick is to use parameters. This may be not always possible but it is a construction you could use.

2009/09/17

Integrating PeopleSoft in a SOA

I came accross this post about exposing PeopleSoft functionality as a webservice.
This can be very useful when you want to use this functionality within a BPM process, like for example "EmployeeEmployment".

2009/09/11

OSB 10gR3 Service versioning

There has been a lot written and discussed about how to do Service versioning.
On LinkedIn there is a nice discussion about the WSDL versioning (read here). Also on InfoQ there is a nice article.

I like the strategy in which a minor and major version is used. A minor change is backwards compatible and only updates the documentation element within the WSDL. A major change, also changes the namespace (http://.../v2).

The following structure within the OSB can help to implement this strategy.


The advantages of this structure are:
* The different versions are nicely seperated and can be governed and maintained seperatly. Instead of having only one proxy for all versions and doing some content based routing.
* The two versionings can live together and in case the old version is deprecated the proxy can be deleted. This way "old" consumers do not have to migrate in case a major update is published, so they have the time to plan it.

2009/09/10

OSB 10gR3 project structure

This blog describes a way you can structure your OSB project structure in a SOA project.


Within you OSB domain you have one configuration file that contains several projects. each project represents a Service.
The shared resources directories contain common resources that are used within the other projects. This can be a good place to store the CDM XSDs of the domain.

2009/09/08

Use of Oracle Adapters (OSB 11g)

In this Oracle post it is stated that new Adapters are available for the Oracle Service Bus.
But do we really want these Adapters on the OSB within a SOA?
I propose not to implement Services this way, because:

Adapters do not belong on the OSB but on the system the Service is actually implemented, because this goed against a known service-oriented design principle of "Service Autonomy". This belongs and should be governed by the Service and not by the OSB (as the Service provider).

2009/09/03

SOASchool: Certified SOA Architect

I am currently studying for the first module of the 5 module certification traject for becoming a SOA Certified Architect.
What I like about the traject is that you have the option to go to workshops or order self study kits. The workshops can be followed also at the SOA Symposium in Rotterdam in october 2009.
Furthermore the exams are based on the books of Thomas Erl a respected author in the SOA field.

I have ordered the self study kit and it took a couple of weeks to arrive. It contains two reading materials, one handout of the presentation (fortunately with an audio CD) and one what to study and example questions (unfortunately just three). The audio CDs are a pain to listen to. It looks like generated from text with hardly any tonations.
The best way for me was to read the SOA Principles of Service Design book. Tomorrow (4 sept 2009) is my exam for this first module. Keep you up-to-date about the level required.

What I am curious about though is how this certification is seen by customers. Has it reached the image of the Java certifications for example yet? That still must be seen, i guess.

2009/09/02

OSB 10.3.1 Database Adapter

Edwin Biemond wrote a great post on using the DB Adapter within OSB 10.3.1. You can find his article here and I followed the steps nicely.
However I did not get it working. I made a mistake by giving the connection factory the same JNDI name as the JDBC Data Source!

So please be aware of the JNDI traps.

2009/08/14

How do you establish reliable Services?

I was wondering what kind of protocols/mechanisms/standards are used to establish reliable Services?
Just to get started:

* HTTP
But this is by definition not reliable so some extra mechanisms are needed on API level.
* JMS (Queues or Topics)?
Is reliable but only used within Java technology. I like the Topic idea so that other components can listen on the Service entrance.
* WS-ReliableMessaging
It's a standard defined on top of SOAP, but which tools, platforms already use it?
* Other?

I am curious in your experiences!

2009/08/11

Data integrity and transactions within SOA

I came accross this article about managing transactions within a SOA environment. There Vinay Singla mentions 4 techniques:
1) Perform the operations that support transactions before the operations that don’t support transactions (technique 2 can help)
2) Use compensating transactions (services must support compensating capabilities)
3) Break the transaction into multiple decoupled transactions
4) Execute the transaction as a long-running transaction

Maybe you think of WS-Transaction and WS-Reliability but these standards have not yet been mature enough.

Another technique not mentioned is the use of Functional/Technical maintenance. In this case when a step is not fulfilled, it will be send to a queue. This will be picked up by maintenance to correct the problem.

I always fancy the most simple solution that decreases complexity for situations that may occur only once in a while, but this depends of course on the requirements.

Examples for SOA 11g and 10g

Are you looking for SOA examples? Maybe you can find them here.

2009/08/10

Dynamic transformations in OSB 10gR3

Here is an interesting post from Chris about dynamic transformations within the OSB.
It looks great, but think about this:
You loose the strong typing of the messages the proxy can receive (as described in the blog itself). This may lead to faults that are difficult to trace because the service contract is in fact far more coarse grained.

How about maintenance? I like the approach that for a different major version of a service with different XML data, use a different proxy. This way services of different versions can live together and services can become obsolete when clients are no longer using the old service. If you try to put all versions of the service in one proxy service it can soonly become unmanageable.

But it can still be useful to have dynamic transformations.

2009/08/08

Federated SOA

This week I had a discussion about a federated ESB and functional domains. It is widespread known that a CDM within an enterprise is hard to achieve and that's why the enterprise is divided in functional domains. However a common mistake often made, is that this division is done on a 1-by-1 mapping to the organisational divisions.
Reasons I hear:
* "Then we can make this department responsible for those processes"
* ".. and those department is responsible for those systems/applications"
This again is the silo-based concept of thinking.
This division is furthermore also visible in the division of domains within the IT landscape (federated ESBs). But if there is one variable that often changes it will be the organisation. Also most processes will span multiple areas and it is very difficlut to keep that in one domain. However the business of the organisation and the functions the business must offer is more stable.

So divide your SOA landscape on a functional basis is my personal opinion and not on a organisational basis.
What's your experience?

2009/07/20

XQuery use within OSB, watch out!

I came accross an issue when using an XQuery resource within an assign. I could not see the binding variable, it turned out that the external variable within the .xq code was not used within the XQuery code. In this case the OSB does not show the variable!

I entered the next code:


Then I wanted to use this .xq within an Assign and opened the XQuery Expression Editor. However the Variable Binding (which is used to assign a value to the externally defined XQuery variable) is not shown:


When I now actually use the external variable:



The OSB now shows the variable:



So watch out when tou are setting up the code, that you actually use the external variable otherwise the OSB does not show you the variable!

2009/07/07

SCA Introduction

To get an overview of the possibilities of the Software Component Architecture (SCA) standard you should read this blog.

Also Oracle supports the SCA standard.
This standard let you define new components by using other components no matter which technology and protocol used. The wiring is done through configuration.
Curious if this will be used a lot by the community!

Note: The Oracle SOA Suite 11g does not contain the AquaLogic Bus, this is a separate product from Oracle.

2009/06/28

CDM where to transform?

This is a presentation I made about "why to use CDM?" and "where transformation can be done".
It is in Dutch, please feel free to comment on it !

2009/06/21

CDM or not?

Again I am participating in a project that must decide to use a Canonical Data Model or not. In theory this is generally a good idea as already described by Wolf's Enterprise Integration Patterns. Does this also hold for a SOA? Yes when we must believe Thomas Erl, in fact it is one of his SOA pattern (Canonical Schema)

We all know the advantages

* Less transformation needed (this only holds when you have NxM relations)
* Data reuse between Services within the same Service Inventory

What do others say?

Steve Jones describes three ways in his blog to setup a CDM. In Common demoninator (1) the minimum set of data that can be used to effectively communicate between areas on a consistent basis. The goal here isn't that this should be used on 100% of occasions but that it represents 70-80% of the interactions. In a Superset (2) the goal is to capture a canonical form that represents 100% of the possible interactions between services. Thus if a service might need 25 fields of product information then the canonical form has those 25 fields. The last one is the Enterprise beast (3) and this one creates a single data model that represents not only the superset of interactions, but the superset of interals as well.

And what do I think?

  1. I do not see it happen to get one big enterprise model.
  2. Try to make a CDM per business domain. And use the data attributes that really are necessary. This makes you think good about the model and will cut out the bad fruit
  3. Make use of XML Schema to define the data entities and do not CSV,TXT or other non-standard formats
  4. Make a document that will describe and defines the data entities so that everybody has a good understanding of the entities.
  5. Make the data available to others

2009/06/18

Soap over JMS W3C Recommendation

As you may know Soap over JMS is not a standard yet.
W3C has just released W3C Candidate Recommendation SOAP over Java Message Service 1.0 which specifies how SOAP (both SOAP 1.1 and SOAP 1.2) should bind to a messaging system that supports the Java Message Service (JMS).

If you do not want to read specifications this article on InfoQ might be a good starting point.

Now it is waiting for our Vendors to support it.

2009/05/27

Ken je Web Services standaarden!

De volgende tabel geeft een mooi overzicht van WS-* standaarden die gevolgd kunnen worden voor Web Services.










StandaardAdviesAlternatieven
OrchestrationBPELWS-Choreography, WS-CDL
Management
WS-DistributedManagement, WS-Provisioning, WS-Management
SecurityWS-SecurityWS-Trust, WS-Federation, WS-SecureConversation, WS-SecurityPolicy
TransactionWS-Transaction, WS-CoordinationWS-CompositeApplicationFramework (WS-CAF), WS-Context (WS-Ctx), WS-CoordinationFramework (WS-CF)
ReliabilityWS-ReliableMessagingWS-Reliability
DescriptionWSDL, UDDIWS-Inspection, Disco, WS-Discovery, WS-PolicyFramework, WS-MetaDataExchange
MessagingXML, SOAPWS-Addressing, WS-Notification, WS-ResourceFramework, WS-Eventing, WS-Policy, SOAP with Attachment
TransportHTTP, JMS, RMI-IIOPTCP, UDP, Jabber, SMTP
InteroperabilityWS-I Basic Profile

Mochten er toevoegingen zijn, dan voel je vrij om een Comment te plaatsen.

2009/05/26

SOA, EAI, BPM job trends

Op verschillende sites lees je dat de hype van SOA voorbij is. Dat geldt wellicht als marketing term, maar ik heb eens gezocht naar de vacatures die open staan.
Op de site indeed.com kun je de job trends bekijken (in amerika).


Nederland heeft ook een site indeed.nl. Helaas laat die site niet de trendlijn zien.
Maar dit geeft de volgende resultaten:
SOA: 483
EAI: 230
BPM: 133

De hype is wellicht voorbij maar het momentum is daar voor SOA om zich als distributed computing platform te bewijzen.

2009/05/20

Data Services

Het gebruik van Data Services (ook wel Entity Services genoemd) wordt goed uitgelegd in het volgende artikel: http://www.infoq.com/articles/narayanan-soa-data-services

Als tip zou ik ook nog meegeven: kijk ook eens naar een REST interface!

Gaat SOA slagen?

Ik lees veel artikelen over het Service Oriented Computing paradigma. Ik heb OO zien "slagen", in die zin dat het wereldwijd omarmd wordt. Ik heb Component Based Development toch al een stuk minder zien slagen. SOA gaat weer een stapje verder (enterprise wide) en ik ben benieuwd of deze vorm van architectuur kunst gaat slagen. OO is nog goed te overzien omdat het vaak binnen een applicatie blijft. Met herbruikbare componenten werken binnen je afdeling of zelfs binnen de organisatie vergt toch een hele andere vorm van samenwerken. Ditzelfde geldt ook voor SOA. Er is niet voor niets zoveel te lezen over Governance.
Net als de overgang van structured analysis/design (SA/SD) naar OO werd aangenomen dat dat alles zou oplossen. Dit is natuurlijk niet zo, je kunt er nog steeds een puinhoop van maken. Dit geldt ook voor de overgang naar een SOA, alleen dan nog een paar gradaties erger. Daarom denk ik dat het een zeer goede practice is om kleinschalig te beginnnen en eerst maar eens proberen om een enkele applicatie (Silo) te "SOAliseren". Dus verticaal i.p.v. horizontaal en dan beetje bij beetje meer zaken gaan toevoegen. Maar op deze manier doe je al ervaring op met Services en het herbruikmaar definieren van de Services. Ook kom je er dan achter dat er verschillende Service levels zijn (Entity, Task en Utility Services). Entity is een service die een business term omvat (dus zeer herbruikbaar), een task service is meer expliciet voor een stukje van een specifiek business process bedoeld (dus vaak ook minder herbruikbaar) en een utility service is meer een horizontale service zoals bijvoorbeeld voor logging of error handling.
Dus eerst maar eens terug naar de basis en een nette 3-tier applicatie bouwen dat is denk ik voor veel organisatie al een uitdaging op zich.