Configuring a High Availability authoring RHPAM using EAP Domain mode

Environment

• OS: Linux Fedora 32
• RHPAM 7.11.1 (jBPM)
• RHDG 7.3 (Infinispan)
• RH AMQ 7.8.1 (Based on Apache ActiveMQ and Apache Kafka)
• JBoss EAP 7.3.7 (Wildfly)

Architecture

In this tutorial we will configure the architecture as follow:

• 2 RHPAM nodes with only Business Central
• 2 RHDG nodes in a cluster
• 1 RH AMQ

The RHDG nodes and the AQM Broker will run each in their own JVM, but the Business Central will use the Domain from EAP with the following configuration:

• 1 Master node with one Server
• 1 Slave node with one Server
• 1 Server Group with the two servers described above

Environment details

As you can see in the Environment section above, to configure the RHPAM HA we need more tools than one RHPAM and EAP. This tutorial will cover the installation, configuration, and use of these tools when is needed to do the main objective of this tutorial. You will not find extensible details about each one.

All the installation and configuration will be done in only one machine, because of that we will need to configure specific ports for each tool and will avoid an important step when using this with the correct configuration, each node in a different machine, where you need to configure an NFS to each RHPAM share the same file structure.

During this tutorial we will use the following environment variables:

AMQ

To install, configure, and run the AMQ, follow the steps bellow

1. Download the AMQ Broker 7.8.1 from Red Hat

2. Extract the zip folder under $AMQ_HOME

3. Change to $AMQ_HOME/bin folder

4. Run the command bellow

   ./artemis create --host localhost --user rhpam --password amqpam --require-login pam_broker
   

About this command:

The result of this command is to create a folder with the same name used in the –require-login inside the $AMQ_HOME/bin.

To start the AMQ follow the steps below:

1. Go to $AMQ_HOME/bin/pam_broker/bin
2. Run the command: $ ./artemis run

If all is ok, you will see something like this:

RHDG

We will configure the RHDG nodes as a cluster

First of all, download the RHDG 7.3.0 from Red Hat. The same files will be used in each node.

Node 1

Install, configure, and run node 1 following the steps below

1. Extract the zip folder inside the $RHDG1

2. To simplify the startup command we will do some changes in the configuration files

   1. Go to $RHDG1/standalone/configuration

   2. Create a backup from standalone.xml file with the command: $ cp standalone.xml standalone.xml.OLD

   3. Copy the clustered.xml file content to the standalone.xml file: $ cp clustered.xml standalone.xml

   4. Open the standalone.xml in a text editor and do the following changes

      2. On line 3 change the to
      3. On line 474 change the value of ${jboss.socket.binding.port-offset:0} to ${jboss.socket.binding.port-offset:100}

3. Go to $RHDG1/bin e run the command to start the server:

   ./standalone.sh
   

About the options changed in the clustered.xml file:

Keep this node running to follow the steps of Node 2 below

Node 2

Install, configure, and run node 2 following the steps below

1. Extract the zip folder inside the $RHDG1

2. To simplify the startup command we will do some changes in the configuration files

   1. Go to $RHDG1/standalone/configuration

   2. Create a backup from standalone.xml file with the command: $ cp standalone.xml standalone.xml.OLD

   3. Copy the clustered.xml file content to the standalone.xml file: $ cp clustered.xml standalone.xml

   4. Open the standalone.xml in a text editor and do the following changes

      2. On line 3 change the to
      3. On line 474 change the value of ${jboss.socket.binding.port-offset:0} to ${jboss.socket.binding.port-offset:200}

3. Go to $RHDG1/bin e run the command to start the server:

   ./standalone.sh
   

If all is right, you will see a log, in node1, like this one:

19:06:15,787 INFO  [org.infinispan.CLUSTER] (remote-thread--p2-t21) [Context=___hotRodTopologyCache]ISPN100002: Starting rebalance with members [nodeA, nodeB], phase READ_OLD_WRITE_ALL, topology id 2
19:06:15,809 INFO  [org.infinispan.CLUSTER] (remote-thread--p2-t24) [Context=___hotRodTopologyCache]ISPN100009: Advancing to rebalance phase READ_ALL_WRITE_ALL, topology id 3
19:06:15,818 INFO  [org.infinispan.CLUSTER] (remote-thread--p2-t1) [Context=___hotRodTopologyCache]ISPN100009: Advancing to rebalance phase READ_NEW_WRITE_ALL, topology id 4
19:06:15,830 INFO  [org.infinispan.CLUSTER] (remote-thread--p2-t25) [Context=___hotRodTopologyCache]ISPN100010: Finished rebalance with members [nodeA, nodeB], topology id 5

EAP

As we will create an authoring environment, an environment focused on design projects only, not executing it, we will deploy only the business-central.war in the server group.

Before deploying the application, we need to configure the EAP domain

We will use the same EAP installation file in the Master and Slave node, so start downloading it from Red Hat and the path update version 7.3.7

Node 1

The Node 1 will be the Domain Controller, this is the node that will be the interface between us and the cluster, on it we will configure the host controllers, server groups, and so on.

Install and apply the patch

1. Extract the jboss-eap-7.3.0.zip into the $PAM1

2. Go to $PAM1/bin and start the server: $ ./standalone.sh

3. Open another terminal window and go to $PAM1/bin

4. Start the jboss cli: $ ./jboss-cli.sh -c

5. Apply the 7.3.7 path:

   [standalone@localhost:9990 /] patch apply ~/eap/jboss-eap-7.3.7-patch.zip
   *output*
   {
       "outcome" : "success",
       "response-headers" : {
           "operation-requires-restart" : true,
           "process-state" : "restart-required"
       }
   }
   
   

6. Close the CLI: [standalone@localhost:9990 /] exit and close this terminal window

7. Return to the first terminal and stop the running server with a ctrl+c

Configure the host

1. Go to $PAM1/domain/configuration
2. Create a backup of the host.xml file: $ cp host.xml host.xml.OLD
3. Copy the content of host-master.xml to the host.xml file: $ cp host-master.xml host.xml

With these steps we will have a default master controller configured, now we will do some changes to make it work as we need

1. Open the host.xml file in a text editor

2. Include the native-interface in the ** section, starting in line 57:

3. Starting in line 64, change the ** to be as follow:

4. In line 78, change the value of the XX:MaxMetaspaceSize to 1024m

5. In line 179, create a new line between ** and **

6. In this new line, include the ** section as below

Configure the domain

1. Go to $PAM1/domain/configuration

2. Create a backup of the domain.xml file: $ cp domain.xml domain.xml.OLD

3. Open the domain.xml file in a text editor

4. Go to the section ** that starts in line 1892

5. Remove the other-server-group item

6. Change the name of the main-server-group to groupA

7. The result will be as follow:

   ***output ommited***
   
   
       
           
               
           
           
       
   
   
   ***output ommited***
   

8. To test if all configuration is ok, test start the server going to $PAM1/bin running the command: $ ./domain.sh

9. The result will be as below:

Node 2

> Note: Be sure that node 1 is not running when you start this section

This will be a slave node and we will configure only the host.xml file

To install the EAP, follow the same steps as showed in the Node 1 changing the location from $PAM1 to $PAM2

Configure the host

1. Go to $PAM2/domain/configuration
2. Create a backup of the host.xml file: $ cp host.xml host.xml.OLD
3. Copy the content of host-slave.xml to the host.xml file: $ cp host-slave.xml host.xml

With these steps we will have a default master controller configured, now we will do some changes to make it work as we need

1. Open the host.xml file in a text editor

2. In the line 3, include the host name: **

3. Include the native-interface in the ** section, starting in line 57, and change the port of the http interface:

4. In line 67, include a default value to jboss.domain.master.address: jboss.domain.master.address:127.0.0.1

5. On line 85, change the value of the XX:MaxMetaspaceSize to 1024m

6. In the ** section, started in line 89, remove the server-two item, and chage the server-one to be as below

To test if all is configured correctly, start Node 1 again and after it is running, start Node 2. The output from node 1 showing the new server connection will be as this:

[Host Controller] 22:54:13,753 INFO  [org.jboss.as.domain.controller] (Host Controller Service Threads - 27) WFLYHC0019: Registered remote slave host "slave", JBoss JBoss EAP 7.3.7.GA (WildFly 10.1.20.Final-redhat-00001)

Deploy the RHPAM

The Business Central must be deployed to all servers in server group A. To reach it, follow the steps below:

We will use the business-central.war file, so first, download the business central from Red Hat

Follow the steps below to configure the RHPAM nodes and deploy the application to the server group

1. Extract the rhpam-7.11.1-business-central-eap7-deployable.zip to $TMP

2. Go to $TMP/jboss-eap-7.3/bin

3. Copy all content to $PAM1/bin and $PAM2/bin:

   $ cp * $PAM1/bin
   $ cp * $PAM2/bin
   

4. Open the $TMP/jboss-eap-7.3/domain/configuration/domain.xml in a text editor

5. Replace the ** section with the one created in Configure the Domain

6. Copy all content of this folder to $PAM1/domain/configuration and $PAM1/domain/configuration

   $ cp * $PAM1/domain/configuration
   $ cp * $PAM2/domain/configuration
   

7. Go to $TMP/jboss-eap-7.3/standalone/configuration

8. Copy all content to $PAM1/standalone/configuration and $PAM1/standalone/configuration

   $ cp * $PAM1/standalone/configuration
   $ cp * $PAM2/standalone/configuration
   

9. Go to $TMP/jboss-eap-7.3/standalone/deployments/business-central.war

10. Inside the business-central.war folder run the java command to create the business-central.war file: $

11. Go to $PAM1/bin and start the server: $ ./domain.xml

12. Go to $PAM2/bin and start the server too: $ ./domain.xml

13. Open another terminal window and go to $PAM1/bin

14. Open the CLI connected to the PAM1 server: $ ./jboss-cli.sh -c

15. Rum the command below to deploy the business central

    [domain@localhost:9990 /] deploy $TMP/jboss-eap-7.3/standalone/deployments/business-central.war/business-central.war --server-groups=groupA
    

If all is ok, now you can open the Business Central app in your browser from each node:

• Node PAM1: http://localhost:8080/business-central
• Node PAM2: http://localhost:8380/business-central

Configure the RHPAM

First of all, as you saw when did the test in the last section, we don’t have a user to we can log in to the business central. Now we will create one.

1. Go to $PAM1/bin and run the command: $ ./add-user.sh

2. This will open an interactive command-line interface to add the user. Include the information below

   • Type of user: b (Application User)
   • User name: rhpamAdmin
   • Password: adminUser#1
   • Groups: admin,rest-all,kie-server
   • About to add user? yes
   • last question: no

Run the same steps in $PAM2

Now you can log in to any RHPAM node with this user and the modification you do in one node will be reflected in the other one.

Connect the RHPAM with RH AMQ

As we are using the Domain mode, all system properties of all nodes can be included in only one configuration file, the domain.xml.

Follow the steps below to connect the RHPAM with the AMQ

1. Go to $DM1/domain/configuration and open the domain.xml file in a text editor

2. Go to ** section, starting in line 44, and include the following lines:

   
       ...
       
       
       
           
   
   

Some points about the properties:

Connect the RHPAM with the RHDG

Follow the steps below to connect the RHPAM with the RHDG

1. Go to $DM1/domain/configuration and open the domain.xml file in a text editor

2. Go to ** section, starting in line 44, and include the following lines:

   
       ...
       
       
       
       
       
       
   
   

Some points about the properties:

Configure local storage

In our environment, we are using only one machine to run the two RHPAM nodes and this allows us to avoid the NFS configuration, however, each node is installed in a different location, $DM1 and $DM2, and to RHPAM correct functionality, we need to share some folders.

In the last step we configured to use the RHDG cluster as the indexer, the other shared point between the RHPAM nodes is the .niogit folder. Follow the steps to configure a shared location between the RHPAM nodes

1. Go to $CLUSTER_HOME and create a folder named .niogit: $ mkdir .niogit

2. Go to $DM1/domain/configuration and open the domain.xml file in a text editor

3. Go to ** section, starting in line 44, and include the following line:

Conclusion

After all these steps you have a full environment configured and can start all it in the following sequence:

1. Start the AMQ
2. Start the RHDG1
3. Start the RHDG2
4. Start the RHPAM1
5. Start the RHPAM2

As this tutorial is only to show how to configure, if needed, you can see more details about how the RHPAM uses the AMQ and RHDG in the product documentation. Check each product documentation if you need details about its configuration and how to improve the performance.

In summary, you can use this tutorial to configure an environment using more than one machine without problem. The only step not aborded here is the configuration of the NFS that you will need to configure and share with all RHPAM nodes to create the .niogit folder.