How to build WSO2 ESB from source and remote debugging the source code

In this post I'll describe how to build the WSO2 ESB from the source code and how to remote debug the WSO2 ESB code using IntelliJ IDEA. 

Remote debugging is important to understand how the product is working.

This post contains five main parts.

1. Setup the environment.
2. Clone source code base from Git Hub.
3. Building the product from source.
4. Running the product.
5. Remote debugging WSO2 ESB code base using  IntelliJ IDEA.

1. Setup the environment.

Before building and remote debugging we need to have following things in our computer. All these are free. We can easily download and install.

1. JDK

2. Maven - https://maven.apache.org/ 

3. intelliJ IDEA (Community Edition) - https://www.jetbrains.com/idea/download/

Note : we need to set following environment variable :

MAVEN_OPTS="-Xms1024m -Xmx4096m -XX:MaxPermSize=1024m"

This is to avoid the Maven OutOfMemoryError.

For more information please read the documentation :

https://docs.wso2.com/display/ESB481/Building+from+Source

2. Clone source code base from Git Hub.

We need to clone the source for the WSO2 ESB from Git Hub. It is 100% open source and released under Apache license.

The code is in three repositories as

1. wso2-synapse
https://github.com/wso2/wso2-synapse

2. carbon mediation
https://github.com/wso2/carbon-mediation

3. product esb.
https://github.com/wso2/product-esb

Clone each of these repositories one by one. (If you are planing to send pull requests before cloning fork the repo)

You can copy the clone URL from the right hand side box in Git Hub repository page.

Change the directory where you want to clone the source. Then use “git clone” command to clone the source.

Eg:
wso2-synapse
git clone https://github.com/wso2/wso2-synapse.git

carbon-mediation
git clone https://github.com/wso2/carbon-mediation.git

product-esb
https://github.com/wso2/product-esb.git

After cloning all you would see three repositories named

wso2-synapse
carbon-mediation
product-esb

Open up each repository and you will see a project object model (POM) file inside each of them. This file is used by Maven  to build the source.

3. Building the product from source.

We need to build all three repositories in the following order.

1. wso2-synapse
2. carbon-mediation
3. product-esb

There are three commands that we can use to build the source.

1. mvn clean install -Dmaven.test.skip=true

This command will generate the binary and source distributions, without running any of the unit tests.

2. mvn clean install

This command will generate the binary and source distributions, with running unit tests.

If you run this command most of the time test cases might get failed so I prefer the first command.

3. After building the product once using a above command you can use
mvn clean install -Dmaven.test.skip=true -o

This command will generate binary and source distributions, without running any of the unit tests, in off line mode.

So now navigate to each folder in above order and run  mvn clean install -Dmaven.test.skip=true in a terminal.

After a successful build you should see a BUILD SUCCESS message.

For an example after building wso2-synapse you would see following lines at the bottom of the terminal.

INFO] Apache Synapse .................................... SUCCESS [3.774s]
[INFO] Apache Synapse - Secure vault ..................... SUCCESS [5.571s]
[INFO] Apache Synapse - Commons classes .................. SUCCESS [6.501s]
[INFO] Apache Synapse - Transports ....................... SUCCESS [1.058s]
[INFO] Apache Synapse - Non-blocking HTTP/s Transport .... SUCCESS [9.755s]
[INFO] Apache Synapse - PIPE Transport ................... SUCCESS [2.606s]
[INFO] Apache Synapse - VFS Transport .................... SUCCESS [4.536s]
[INFO] Apache Synapse - Tasks classes .................... SUCCESS [1.478s]
[INFO] Apache Synapse - Core ............................. SUCCESS [36.741s]
[INFO] Apache Synapse - FIX Transport .................... SUCCESS [4.929s]
[INFO] Apache Synapse - Extensions ....................... SUCCESS [4.734s]
[INFO] Apache Synapse - Samples .......................... SUCCESS [4.049s]
[INFO] Apache Synapse - Patches .......................... SUCCESS [3.552s]
[INFO] Apache Synapse - Experimental code ................ SUCCESS [1.628s]
[INFO] Apache Synapse - Web Application .................. SUCCESS [28.018s]
[INFO] Apache Synapse - Handler Module ................... SUCCESS [6.053s]
[INFO] Apache Synapse - XAR Maven Plugin ................. SUCCESS [7.084s]
[INFO] Apache Synapse - Configuration Migrator ........... SUCCESS [1.440s]
[INFO] Apache Synapse - Distribution ..................... SUCCESS [19.387s]
[INFO] Apache Synapse - Package skeleton ZIP ............. SUCCESS [1.238s]
[INFO] Apache Synapse - Package archetype ................ SUCCESS [4.550s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:44.279s
[INFO] Finished at: Fri May 01 21:27:46 IST 2015
[INFO] Final Memory: 147M/630M
[INFO] ------------------------------------------------------------------------

After building all three code bases to find the product (WSO2 ESB) open product-esb >> modules >> distribution >> target.
In there you should see a zip archive named wso2esb-4.9.0-M8-SNAPSHOT.zip.

This name can change based on  the version. This is the binary distribution of the WSO2 ESB you can  now run the built product.

4. Running the product.

If you want to test run the product just built, extract the zip archive and change directory to ESB_HOME >> bin and run sh wso2server.sh . Shutdown the server using ctrl+c
For more information about running the product please visit: https://docs.wso2.com/display/ESB481/Running+the+Product

5. Remote debugging WSO2 ESB code base using  IntelliJ IDEA.

In this part I will describe how to remote debug the source using intelliJ IDEA. For this demonstration I'll use ESB with sample 0 configuration. However you can remote debug any of the code in the same way based on your requirement.
Here we have to do three things. First open the code base form inteliJ IDEA, edit run configuration and run the ESB sample 0.

• Open wso2-synapse code from intelliJ IDEA

First open the J IDEA and select File >> Open. Then navigate to the location where you cloned the wso2-synapse  code base and select pom.xml inside the wso2-synapse maven project. Hit OK to open the project.

After opening the project you will be able to see the project structure like this.

• Edit run configurations

Now we need to edit run configurations of the Run/debug Configurations JIDEA to remote debug. Select Run >> Edit configurations.



Click the + button in the top left corner of the and select Remote. To edit remote debug configuration.

In the configuration give a suitable name and keep all other fields as default.

 Click Apply and OK to save the changes.

• Running the sample 0.

Now we need to run ESB with sample 0.

First deploy SimpleStockQuoteService sample in the axis2server.

Run axis2server.
Navigate to ESB_HOME >> samples >> axis2Server and in a terminal execute
sh axis2server.sh to run the axis2 server.

Deploy  SimpleStockQuoteService.
Navigate to  ESB_HOME >> samples >> axis2Server >> src >> SimpleStockQuoteService and in a terminal run ant to deploy the service in the server.

You can find more information about running sample 0 from here :

https://docs.wso2.com/display/ESB481/Sample+0%3A+Introduction+to+ESB

Start the ESB with sample 0 configuration with listening to the port 5005.
Navigate to ESB_HOME >> bin and execute sh wso2server.sh -sn 0 -debug 5005

You would see starting the ESB has been paused and the ESB is listening to port 5005. (Check the console)

Listening for transport dt_socket at address: 5005

Now open up intelliJ IDEA. From the Run menu you can select the debug option. In my case it is “Debug  wso2_esb_remote_debug”.

Or else you can simply start remote debug by just clicking the “bug” icon in the menu bar.

Make sure that you have selected the correct configuration.

When the debug session is starting you would see Connected to the target VM, address: 'localhost:5005', transport: 'socket' in intelliJ IDEA console. Then the starting of ESB will continue.

 

Next we need to place a break point in the source code.

I select SynapseCallbackReceiver class inside the synapse-core >> src >> main >> java >> org.apache.synapse.core.axis2 package.

I will place a break point at the receive method in SynapeCallbackReciver class. I selected this method because when  a new response message is received this method get invoked. So we can easily start debug from here.

To run stockQuote client navigate to ESB_HOME >> samples >> axis2Client and in a terminal run following command to execute the stockQuote client in the smart client mode.

ant stockquote -Daddurl=http://localhost:9000/service/SimpleStockQuoteService -Dtrpurl=http://localhost:8280/

Now you will see that IntelliJ IDEA will get focused as soon as execution has been reached  the breakpoint.

Now from here you can test the execution steps using the options provided by intelliJ IDEA. (F8 F9 etc.)

Programmatically clone Synapse message context in WSO2 ESB.

If you are a WSO2 ESB developer sometimes you might want to programmatically create an exact copy of message context with all the information attached with that. This is called as message cloning in ESB terms.

Mainly message cloning is useful for storing and resending purposes, but there are other uses of this. 

In WSO2 ESB to clone and create an exact copy of message context we can use MessageHelper class in the WSO2-Synpse, under org.apache.synapse.util package. 

You can clone the source for WSO2 synapse from following repo.
https://github.com/wso2/wso2-synapse

After getting the source code for WSO2 synapse open with Jidea you will find MessageHelper class under WSO2-Synapse > modules > core > src > main > java > org.apache.synapse.util package. 

In the MessageHelper class there is a method named cloneMessageContext. This method will clone the message context and will return an exact copy of the passed message. 

If you examine the code and comments carefully of this method you will easily understand what is going on. So here I'm not going to explain the code of that method. 

The use of cloneMessageContext is pretty easy and straightforward. We just want to pass the synapse message context that we want to clone, then it will return the clone message context. See the following example code line :

org.apache.synapse.MessageContext clonedMessageContext =
                        MessageHelper.cloneMessageContext(synapseMessageContext);

If you want only to clone the axis2 message context then you can use cloneAxis2MessageContext method in the same class. The use of that class is also similar. The only thing is you want to pass an axis2 message context to the method.