How to cluster WSO2 4.2.X products with latest Hazelcast clustering

Given below are the steps to setup the simplest scenario of clustering with two WSO2 Application Server nodes (without fronting by an ELB).

Configurations for the AS-node1

Download and extract the wso2as-5.2.0.zip to a prefered location.

axis2.xml configuration changes

1. axis2.xml can be found at [AS_HOME]/repository/conf/axis2.

2. Enable clustering by making the given below config ‘ true ’.

<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" span=""></clustering>

               enable="true">

3. We recommend to use well known address (wka) as membership scheme. Therefor the membershipScheme parameter should be configured as follows.

<parameter name="membershipScheme">wka</parameter>

4. You have to specify a domain name for the AS cluster.

<parameter name="domain">wso2.as.domain</parameter>

5. Provide a hostname for the AS node 1 as given below.

<parameter name="localMemberHost">wso2.as.node1</parameter>

6. A unique value should be given to the localMemberPort of each service in order to identify the node when clustering.

<parameter name="localMemberPort">4000</parameter>

7. The subdomain by default comes as ‘ worker ‘ now. You can either keep it or can comment out. For this scenario subdomain will be kept as ‘worker’.

<parameter name="properties"></parameter>

           <property name="backendServerURL" value="https://${hostName}:${httpsPort}/services/"></property>

           <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"></property>

           <property name="subDomain" value="worker"></property>

  

8. Since we have used ‘wka’ as membership scheme it is possible to specify the node 2 as a wka member. When both the servers are well known to each other if any node would go down and then recover it will join back to the cluster. But in this scenario I would not add any members to node1. Only the node2 will get joined to Node1. In such scenario if node1 would go down and recover it will not be joined to the cluster, whereas node 2 would do. Therefor our recommendation is that each cluster member should be well known to at least two other members on the cluster. But for this setup the configuration for node1 is going to be kept as below for the simplicity.

<members>

           <!--<member>

               <hostname>127.0.0.1</hostName>

               <port>4000</port>

           </member>-->
</members>

   

carbon.xml configuration changes

If both the server nodes are going to be in the same machine set the port-offset in carbon.xml

<offset>1</offset>

Configurations for the AS-node 2

Since most of the configurations going to be same for the node 2 we can take a copy of the configured Application Server node1 (AS-node1) and the rename it as AS-node2.

axis2.xml configuration changes

1. The hostname for the AS-node 2 should be changed as given below.

<parameter name="localMemberHost">wso2.as.node2</parameter>

2. The given to the localMemberPort should be changed to a different unique value.

<parameter name="localMemberPort">4001</parameter>

3. In the members configuration you have to specify AS-node1 as the wka member as given below. The should be the local member port of AS-node1.

<members>          

               <hostname>wso2.as.node1</hostname>

               <port>4000</port>
</members>

          

carbon.xml configuration changes

change the port-offset in carbon.xml to a different value.

<offset>2</offset>

The host names provided for both servers should be added to host entries against the IPs of the two server nodes. In linux you can add the host names to /etc/hosts file whereas in windows host names should be added to /windows/system32/drivers/etc/hosts.

Starting the two Application Server nodes.

1.  First start the AS-node1. You will see following logs before starting since we have enabled clustering.

INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} -  Cluster domain: wso2.as.domain

INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} -  Using wka based membership management scheme

INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} -  Hazelcast initialized in 230ms

INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} -  Local member: [54062d80-430d-40eb-aa8a-75b836d0d123] - Host:10.xxx.1.xxx, Remote Host:null, Port: 4000, HTTP:9764, HTTPS:9444, Domain: wso2.as.domain, Sub-domain:worker, Active:true

INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} -  Added member: Host:10.xxx.1.xxx, Remote Host:null, Port: 4000, HTTP:9764, HTTPS:9444, Domain: wso2.as.domain, Sub-domain:worker, Active:true

INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} -  Cluster initialization completed

2. And once AS-node1 is successfully started  then start the AS-node2 . And before AS-node2 completed its start you can see logs of AS-node2 getting joined to AS-node1.

INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} -  Cluster domain: wso2.as.domain

INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} -  Using wka based membership management scheme

INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} -  Added member: Host:wso2.as.node1, Remote Host:null, Port: 4000, HTTP:-1, HTTPS:-1, Domain: null, Sub-domain:null, Active:true

INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} -  Hazelcast initialized in 8209ms

INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} -  Added member: Host:10.xxx.1.xxx, Remote Host:null, Port: 4000, HTTP:9764, HTTPS:9444, Domain: wso2.as.domain, Sub-domain:worker, Active:true

INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} -  Local member: [7229415c-e4c9-4634-a24b-5353440d902e] - Host:10.xxx.1.xxx, Remote Host:null, Port: 4001, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true

INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} -  Added member: Host:10.xxx.1.xxx, Remote Host:null, Port: 4001, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true

INFO {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} -  Cluster initialization completed

3. And you can see following logs on AS-node1 which says the node2 has joined to the cluster.

INFO {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -  Member joined [7229415c-e4c9-4634-a24b-5353440d902e]: wso2.as.node2/10.xxx.1.xxx:4001

INFO {org.wso2.carbon.core.clustering.hazelcast.util.MemberUtils} -  Added member: Host:10.xxx.1.xxx, Remote Host:null, Port: 4001, HTTP:9765, HTTPS:9445, Domain: wso2.as.domain, Sub-domain:worker, Active:true