This blog describes the steps how to configure a WSO2 BPS cluster(version wso2bps-2.1.0).
WSO2 BPS cluster deployment diagram
Pre-requisite:
Download and extract the wso2bps-2.1.0.zip file and make two copies as Master/Slave
Download and extract the wso2greg-4.1.1.zip file
All WSO2 products are shipped embedded H2 registry, for clustering purpose we need to configure with a database, using MySQL DB for this setup. First we need to configure the database configuration for the registry and then BPS instances. I’m referred Greg home as GREG_HOME, BPS master node as BPS_MASTER_HOME and BPS slave node as BPS_SLAVE_HOME.
BPS cluster database configuration
1. First will create all necessary databases for this setup according to above diagram.
create database greg;
create database bpsmaster;
create database bpsslave
create database bpsds;
2. Now will configure the G-Reg, for that Open the GREG_HOME/repository/conf/registry.xml file comment the original dbConfig element and add following database configuration. (IP, url, username and password should change).
3. For all the cluster nodes (G-Reg, BPS master and slave), same user store is using and first will configure the G-Reg node user store details.
Open GREG_HOME/repository/conf/user-mgt.xml file and change the database configuration as follows (IP, url, username and password should be changed).
Now G-Reg configurations are completed
4. Now will configure the BPS master node, Open the BPS_MASTER_HOME/repository/conf/registry.xml file and comment the original dbConfig element and add the following database configuration. (IP, url, username and passwords should change).
5. Now we have to configure the mount configuration to master node, for that we need to add more parameters to registry.xml file. Add the following database configuration to the after the. (IP, url, username and passwords should change).
We have mounted the /_system/config -> /_system/bpsConfig and /_system/governance -> /_system/governance. In this configuration you can see bps config is mounting to bpsConfig and governance mount to the governance, that’s because the governance details should be same across the cluster like user store and bpsConfig and registry config shouldn’t be shared that’s two different products.
6. Next will configure the BPS slave node, open the BPS_SLAVE_HOME/repository/conf/registry.xml file and comment the original dbConfig element and add the database configuration as follows. (IP, url, username and password should change).
7. Now we have to configure the mount configuration to the slave node, Open the BPS_SLAVE_HOME/repository/conf/registry.xml file and change the database configuration as follows (IP, url, username and password should change).
We have configured the mount configuration to the slave node and we are making the master node as Read/Write node by making the readOnly, by setting false. Slave node makes to readOnly by setting the readOnly value to true. This is the only difference between master and slave node.
8. For all cluster nodes (G-Reg, BPS master and slave), same user store is using and earlier we have configured the G-Reg and now we have to configure BPS nodes.
Open BPS_MASTER_HOME/repository/conf/user-mgt.xml file and change the database configuration as follows (IP, url, username and password should be changed).
And open the BPS_SLAVE_HOME/repository/conf/user-mgt.xml and change database configuration.
9. Now we have to configure the BPEL database for BPS nodes.
Open the BPS_MASTER_HOME/repository/conf/datasources.properties file and change the database configuration as follows (ips, ports, usernames and passwords should change)
Do the same change to the BPS_SLAVE_HOME/repository/conf/datasources.properties node as well. If both nodes are running on the same machine you have to change the JNDI port (providerPort) as well. If yes change the BPS_SLAVE_HOME JNDI port value to 2299.
10. Now we have to configure the axis2 clustering and membership schemes in BPS nodes. There two membership schemes supported by axis2 clustering, namely "multicast" and "wka".
First will enable the axis2 clustering, Open BPS_MASTER_HOME/repository/conf/axis2.xml file and enable the clustering.
Default membership Scheme is set to multicast, so there’s no change to membershipScheme.
Do the same change to the BPS_SLAVE_HOME as well.
11. Now will configure the cache configuration for all the nodes in the cluster including the G-Reg node.
Open GREG_HOME/repository/conf/cache.xml file and change the cache configuration as follows.
Comment the following cache configuration.
Uncomment the following cache configuration.
Make same changes to BPS master and slave nodes as well.
12. If you are running WSO2 Product Instances on same server, then you have to configure the different ports on each instance. Default ports are HTTP port – 9763 and HTTPS port - 9443 for all WSO2 products and you can easily change all ports in WSO2 product by setting an offset value in the carbon.xml file. $CARBON_HOME/repository/conf/carbon.xml by using port offset
0 -> HTTP port – 9763 / HTTPS port – 9443 and other ports
1 -> HTTP port – 9764 / HTTPS port – 9444 and other ports
This sample was configured in same server; therefore we need to change the ports.
13. Before start the server we have to copy the MySQL jdbc driver library to all instances, So copy the MySQL jdbc driver library to the following directory.
GREG_HOME/repository/components/lib directory
Copy the MySQL jdbc driver library same location of the BPS nodes as well.
14. Now all configurations are completed and we need to start servers in following manner for the first time. All necessary tables structure will create from –Dsetup option
I. Go to GREG_HOME/bin
./wso2server.sh -Dsetup
II. Go to BPS_MASTER/bin
./wso2server.sh -Dsetup
III. Go to BPS_SLAVE/bin
./wso2server.sh
WSO2 BPS cluster deployment diagram
Download and extract the wso2bps-2.1.0.zip file and make two copies as Master/Slave
Download and extract the wso2greg-4.1.1.zip file
All WSO2 products are shipped embedded H2 registry, for clustering purpose we need to configure with a database, using MySQL DB for this setup. First we need to configure the database configuration for the registry and then BPS instances. I’m referred Greg home as GREG_HOME, BPS master node as BPS_MASTER_HOME and BPS slave node as BPS_SLAVE_HOME.
BPS cluster database configuration
| Instance Name | Registry Space | Instance Name Registry Space Mysql database name/mounted path |
|---|---|---|
| BPS Master | local | bpsmaster |
| config | greg /_system/BPSConfig | |
| governance | greg /_system/governance | |
| user-manager | greg | |
| BPS Slave | local | bpsslave |
| config | greg /_system/BPSConfig | |
| governance | greg /_system/governance | |
| user-manager | greg | |
| G-Reg | All | greg |
1. First will create all necessary databases for this setup according to above diagram.
create database greg;
create database bpsmaster;
create database bpsslave
create database bpsds;
2. Now will configure the G-Reg, for that Open the GREG_HOME/repository/conf/registry.xml file comment the original dbConfig element and add following database configuration. (IP, url, username and password should change).
<currentDBConfig>wso2registry</currentDBConfig> <readOnly>false</readOnly> <registryRoot>/</registryRoot> <dbConfig name="wso2registry"> <url>jdbc:mysql://<IP>:3306/greg?autoReconnect=true</url> <userName>root</userName> <password>root</password> <driverName>com.mysql.jdbc.Driver</driverName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <minIdle>5</minIdle> </dbConfig>
3. For all the cluster nodes (G-Reg, BPS master and slave), same user store is using and first will configure the G-Reg node user store details.
Open GREG_HOME/repository/conf/user-mgt.xml file and change the database configuration as follows (IP, url, username and password should be changed).
<Property ame="url">jdbc:mysql://<IP>:3306/greg?autoReconnect=true</Property> <Property name="userName">root</Property> <Property name="password">root</Property> <Property name="driverName">com.mysql.jdbc.Driver</Property> <Property name="maxActive">50</Property> <Property name="maxWait">60000</Property> <Property name="minIdle">5</Property>
Now G-Reg configurations are completed
4. Now will configure the BPS master node, Open the BPS_MASTER_HOME/repository/conf/registry.xml file and comment the original dbConfig element and add the following database configuration. (IP, url, username and passwords should change).
<currentDBConfig>wso2registry</currentDBConfig> <readOnly>false</readOnly> <registryRoot>/</registryRoot> <dbConfig name="wso2registry"> <url>jdbc:mysql://IP:3306/bpsmaster?autoReconnect=true</url> <userName>root</userName> <password>root</password> <driverName>com.mysql.jdbc.Driver</driverName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <minIdle>5</minIdle> </dbConfig>
5. Now we have to configure the mount configuration to master node, for that we need to add more parameters to registry.xml file. Add the following database configuration to the after the. (IP, url, username and passwords should change).
<dbConfig name="bpsMountRegistry"> <url>jdbc:mysql://<IP>:3306/greg?autoReconnect=true</url> <userName>root</userName> <password>root</password> <driverName>com.mysql.jdbc.Driver</driverName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <minIdle>5</minIdle> </dbConfig> <remoteInstance url="https://<IP of G-Reg>:<HTTPS prot of G-Reg>/registry"> <id>Mount1</id> <dbConfig>bpsMountRegistry</dbConfig> <readOnly>false</readOnly> <registryRoot>/</registryRoot> </remoteInstance> <mount path="/_system/config" overwrite="true"> <instanceId>Mount1</instanceId> <targetPath>/_system/bpsConfig</targetPath> </mount> <mount path="/_system/governance" overwrite="true"> <instanceId>Mount1</instanceId> <targetPath>/_system/governance</targetPath> </mount>
We have mounted the /_system/config -> /_system/bpsConfig and /_system/governance -> /_system/governance. In this configuration you can see bps config is mounting to bpsConfig and governance mount to the governance, that’s because the governance details should be same across the cluster like user store and bpsConfig and registry config shouldn’t be shared that’s two different products.
6. Next will configure the BPS slave node, open the BPS_SLAVE_HOME/repository/conf/registry.xml file and comment the original dbConfig element and add the database configuration as follows. (IP, url, username and password should change).
<currentDBConfig>wso2registry</currentDBConfig> <readOnly>false</readOnly> <registryRoot>/</registryRoot> <dbConfig name="wso2registry"> <url>jdbc:MySQL://<IP>:3306/bpsslave?autoReconnect=true</url> <userName>root</userName> <password>root</password> <driverName>com.mysql.jdbc.Driver</driverName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <minIdle>5</minIdle> </dbConfig>
7. Now we have to configure the mount configuration to the slave node, Open the BPS_SLAVE_HOME/repository/conf/registry.xml file and change the database configuration as follows (IP, url, username and password should change).
<dbConfig name="bpsMountRegistry"> <url>jdbc:MySQL://<IP>:3306/greg?autoReconnect=true</url> <userName>root</userName> <password>root</password> <driverName>com.mysql.jdbc.Driver</driverName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <minIdle>5</minIdle> </dbConfig> <remoteInstance url="https://<IP of G-Reg>:<HTTPS port of G-Reg>/registry"> <id>Mount1</id> <dbConfig>bpsMountRegistry</dbConfig> <readOnly>true</readOnly> <registryRoot>/</registryRoot> </remoteInstance> <mount path="/_system/config" overwrite="true"> <instanceId>Mount1</instanceId> <targetPath>/_system/bpsConfig</targetPath> </mount> <mount path="/_system/governance" overwrite="true"> <instanceId>Mount1</instanceId> <targetPath>/_system/governance</targetPath> </mount>
We have configured the mount configuration to the slave node and we are making the master node as Read/Write node by making the readOnly, by setting false. Slave node makes to readOnly by setting the readOnly value to true. This is the only difference between master and slave node.
8. For all cluster nodes (G-Reg, BPS master and slave), same user store is using and earlier we have configured the G-Reg and now we have to configure BPS nodes.
Open BPS_MASTER_HOME/repository/conf/user-mgt.xml file and change the database configuration as follows (IP, url, username and password should be changed).
<Property name="url">jdbc:mysql://<IP>:3306/greg?autoReconnect=true</Property> <Property name="userName">root</Property> <Property name="password">root123</Property> <Property name="driverName">com.mysql.jdbc.Driver</Property> <Property name="maxActive">50</Property> <Property name="maxWait">60000</Property> <Property name="minIdle">5</Property>
And open the BPS_SLAVE_HOME/repository/conf/user-mgt.xml and change database configuration.
9. Now we have to configure the BPEL database for BPS nodes.
Open the BPS_MASTER_HOME/repository/conf/datasources.properties file and change the database configuration as follows (ips, ports, usernames and passwords should change)
synapse.datasources=bpsds
synapse.datasources.icFactory=com.sun.jndi.rmi.registry.RegistryContextFactory
synapse.datasources.providerPort=2199
synapse.datasources.bpsds.registry=JNDI
synapse.datasources.bpsds.type=BasicDataSource
synapse.datasources.bpsds.driverClassName=com.mysql.jdbc.Driver
synapse.datasources.bpsds.url=jdbc:MySQL://<ip>:3306/bpsds?autoReconnect=true
synapse.datasources.bpsds.username=root
synapse.datasources.bpsds.password=root
synapse.datasources.bpsds.dsName=bpsds
synapse.datasources.bpsds.maxActive=100
synapse.datasources.bpsds.maxIdle=20
synapse.datasources.bpsds.maxWait=10000
Do the same change to the BPS_SLAVE_HOME/repository/conf/datasources.properties node as well. If both nodes are running on the same machine you have to change the JNDI port (providerPort) as well. If yes change the BPS_SLAVE_HOME JNDI port value to 2299.
synapse.datasources.providerPort=2299
10. Now we have to configure the axis2 clustering and membership schemes in BPS nodes. There two membership schemes supported by axis2 clustering, namely "multicast" and "wka".
First will enable the axis2 clustering, Open BPS_MASTER_HOME/repository/conf/axis2.xml file and enable the clustering.
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
Default membership Scheme is set to multicast, so there’s no change to membershipScheme.
<parameter name="membershipScheme">multicast</parameter>
Do the same change to the BPS_SLAVE_HOME as well.
11. Now will configure the cache configuration for all the nodes in the cluster including the G-Reg node.
Open GREG_HOME/repository/conf/cache.xml file and change the cache configuration as follows.
Comment the following cache configuration.
<configuration> <cacheMode>local</cacheMode> </configuration>
Uncomment the following cache configuration.
<configuration> <clustering> <enabled>true</enabled> <clusterName>wso2carbon-cache</clusterName> </clustering> <cacheMode>replicated</cacheMode> <sync>true</sync> </configuration>
Make same changes to BPS master and slave nodes as well.
12. If you are running WSO2 Product Instances on same server, then you have to configure the different ports on each instance. Default ports are HTTP port – 9763 and HTTPS port - 9443 for all WSO2 products and you can easily change all ports in WSO2 product by setting an offset value in the carbon.xml file. $CARBON_HOME/repository/conf/carbon.xml by using port offset
This sample was configured in same server; therefore we need to change the ports.
| Product | Offset | HTTP | HTTPS |
|---|---|---|---|
| Greg | 0 | 9763 | 9443 |
| BPS Master | 1 | 9764 | 9444 |
| BPS Slave | 2 | 9765 | 9445 |
13. Before start the server we have to copy the MySQL jdbc driver library to all instances, So copy the MySQL jdbc driver library to the following directory.
GREG_HOME/repository/components/lib directory
Copy the MySQL jdbc driver library same location of the BPS nodes as well.
14. Now all configurations are completed and we need to start servers in following manner for the first time. All necessary tables structure will create from –Dsetup option
I. Go to GREG_HOME/bin
./wso2server.sh -Dsetup
II. Go to BPS_MASTER/bin
./wso2server.sh -Dsetup
III. Go to BPS_SLAVE/bin
./wso2server.sh