Set up an Active-Active Cluster
This topic describes how to set up an active-active cluster for Deploy with multiple master and multiple external workers.
HA Active-Active Setup
- In a HA cluster configuration, there are two types of cluster setups: Active-Active and Active Hot Standby.
- Digital.ai Deploy is recommended to be set up in a multi-node Active-Active cluster, as it provides realtime load balancing capability.
- In a multi-node setup, you can have multiple masters and multiple workers, with the workers connected to each of the masters.
- A master node is a Deploy server you log on to in order to access the Deploy user interface and create configuration items, such as infrastructure, environments, applications, and so on.
- The multiple masters ensure continuous functioning in case of failover and the multiple workers ensure uninterrupted task execution and scalability.
- Deployment tasks and Control tasks are assigned to the Deploy worker that performs the task execution. For more information, see Tasks, rules, and configuration Items.
- The multiple masters are controlled by a load balancer.
Important: The Deploy masters and workers must have similar configuration (database, ActiveMQ, plugins, and so on). A worker cannot run the tasks assigned by the master if the configuration differs.
Requirements to Set Up an Active-Active Cluster
- Two or more Deploy stateless master nodes
- Two or more external Deploy worker nodes
- An external database server, for example PostgreSQL
- A load balancer that receives HTTP(S) traffic and forwards that to the Deploy master nodes. For more information, see HAProxy load balancer documentation.
- ActiveMQ
- A shared drive location to store exported CIs and reports.
Important: It is always recommended to have a multi-node setup with odd number of nodes to facilitate high fault tolerance in production environments. It is also recommended not to have a cluster with more than five nodes to prevent database latency issues. You can, however, with some database configuration tuning, have a cluster with more than five nodes. Contact Digital.ai Support for more information about setting up a cluster with more than five nodes.
About the Setup
Let us consider the following multi-node Deploy setup for illustrative purposes:
- Three Deploy masters (Master 1, Master 2, and Master 3)
- Three workers (Worker 1, Worker 2, and Worker 3)
- One load balancer
- One ActiveMQ server (JMS broker)
- One database server—PostgreSQL
Step 1: Download and extract the latest Deploy application to Master 1 Server
- Create a folder for installation from where you will execute the installation tasks. This will be the root directory.
- Download the Deploy ZIP package from the Deploy Software Distribution site (requires customer login).
Step 2: Install the Deploy license
- Download the Deploy license file from the Deploy Software Distribution site
- Copy your license file to the
conf
directory of the root directory.
Step 3: Configure the external Postgres database node
You must set up a database for the Deploy to store data. Create and configure an empty database before you start Deploy installation. During the installation, Deploy creates the database schema on the database you created. Optionally, you can have separate database for the operational and reporting databases. For more information about creating two separate databases, see Separate Databases for Reporting and Repository in Deploy.
Use a industrial-grade external database server, for example PostgreSQL, for production use. For more information, see Configure the Database and Artifacts Repository.
For more information about how to create Postgres database, see the PostgreSQL documentation.
Step 4: Configure the Postgres JDBC driver
- Download the Postgres JDBC driver. See PostgreSQL JDBC driver.
- Copy the JAR file to the
lib
folder of the root directory.
Step 5: Update the deploy-repository.yaml
file with the external database details
- In the root folder, go to
centralConfiguration
folder and open thedeploy-repository.yaml
file. - Configure the parameters to point to the database schema as shown in the following sample configuration:
xl:
repository:
database:
db-driver-class-name: "org.postgresql.Driver"
db-password: "samplepassword"
db-url: "jdbc:postgresql://<IP address of the external database server>/postgres"
db-username: "sample-user"
max-pool-size: 10
artifacts.root: "build/artifacts"
Step 6: Download the ActiveMQ client
- Download the JMS message broker—ActiveMQ client.
- Add JMS
.jar
file—org.apache.activemq:activemq-client
to thelib
folder of the root directory.
Step 7: Set up the ActiveMQ node
- Run ActiveMQ in the Docker container.
- From the
centralConfiguration
folder of the root directory, opendeploy-task.yaml
file. - Set the
in-process-worker
parameter tofalse
. -
Update the JMS details as shown in the sample configuration:
deploy: queue: external: jms-driver-classname: org.apache.ActiveMQConnectionFactory jms-url: tcp://<IP address of the ActiveMQ server node>:61616 jms-username: admin jms-password: admin
Step 8: Set up the Load Balancer node
- To install the HA proxy (load balancer), run:
yum install -y haproxy
- Enable the HA proxy by running the following command:
systemctl enable haproxy
- To start the HA proxy, run:
systemct start haproxy
- Run the following command to check the status:
systemct status haproxy
Step 9: Update the hostname of the Master 1 node
- From the
conf
folder of the root directory, open thedeployit.conf
file. - Update the
server.hostname
parameter with the hostname of the master server. - Update the
deploy.cluster.node.port
parameter with the port number. For example,deploy.cluster.node.port=25520
—default port number is 25520.
Step 10: Update the deploy-cluster.yaml
file with the cluster mode
Digital.ai Deploy 22.3 brings you the following changes related to the high availability (HA) cluster setup:
- You can no longer set
mode
todefault
in thedeploy-cluster.yaml
file and have Digital.ai Deploy run in HA active-active cluster mode. - You must set
mode
tofull
in thedeploy-cluster.yaml
file if you want to run Deploy in HA active-active cluster mode.
Here’s an example deploy-cluster.yaml
configuration file with mode
set to full
.
deploy:
cluster:
akka:
actor:
loggers:
- akka.event.slf4j.Slf4jLogger
loglevel: INFO
provider: akka.cluster.ClusterActorRefProvider
cluster:
auto-down-unreachable-after: 15s
custom-downing:
down-removal-margin: 10s
stable-after: 10s
downing-provider-class: ''
membership:
heartbeat: 10 seconds
jdbc:
connection-timeout: 30 seconds
idle-timeout: 10 minutes
leak-connection-threshold: 15 seconds
max-life-time: 30 minutes
max-pool-size: 1
minimum-idle: 1
password: '{cipher}gfdqswdksahgdksahgdkas'
pool-name: ClusterPool
url: ''
username: ''
ttl: 60 seconds
mode: full
name: xld-active-cluster
-
The
2023. Digital.ai Software Inc. All rights reserveddeploy.cluster.akka.remote
object has been removed from thedeploy-cluster.yaml
file.