Set up an Active-Active Cluster

This topic describes how to set up an active-active cluster for Deploy with multiple master and multiple external workers.

HA Active-Active Setup

  • In a HA cluster configuration, there are two types of cluster setups: Active-Active and Active Hot Standby.
  • Digital.ai Deploy is recommended to be set up in a multi-node Active-Active cluster, as it provides realtime load balancing capability.
  • In a multi-node setup, you can have multiple masters and multiple workers, with the workers connected to each of the masters.
  • A master node is a Deploy server you log on to in order to access the Deploy user interface and create configuration items, such as infrastructure, environments, applications, and so on.
  • The multiple masters ensure continuous functioning in case of failover and the multiple workers ensure uninterrupted task execution and scalability.
  • Deployment tasks and Control tasks are assigned to the Deploy worker that performs the task execution. For more information, see Tasks, rules, and configuration Items.
  • The multiple masters are controlled by a load balancer.

Important: The Deploy masters and workers must have similar configuration (database, ActiveMQ, plugins, and so on). A worker cannot run the tasks assigned by the master if the configuration differs.

Requirements to Set Up an Active-Active Cluster

  • Two or more Deploy stateless master nodes
  • Two or more external Deploy worker nodes
  • An external database server, for example PostgreSQL
  • A load balancer that receives HTTP(S) traffic and forwards that to the Deploy master nodes. For more information, see HAProxy load balancer documentation.
  • ActiveMQ
  • A shared drive location to store exported CIs and reports.

Important: It is always recommended to have a multi-node setup with odd number of nodes to facilitate high fault tolerance in production environments. It is also recommended not to have a cluster with more than five nodes to prevent database latency issues. You can, however, with some database configuration tuning, have a cluster with more than five nodes. Contact Digital.ai Support for more information about setting up a cluster with more than five nodes.

Image

About the Setup

Let us consider the following multi-node Deploy setup for illustrative purposes:

  • Three Deploy masters (Master 1, Master 2, and Master 3)
  • Three workers (Worker 1, Worker 2, and Worker 3)
  • One load balancer
  • One ActiveMQ server (JMS broker)
  • One database server—PostgreSQL

Step 1: Download and extract the latest Deploy application to Master 1 Server

  1. Create a folder for installation from where you will execute the installation tasks. This will be the root directory.
  2. Download the Deploy ZIP package from the Deploy Software Distribution site (requires customer login).

Step 2: Install the Deploy license

  1. Download the Deploy license file from the Deploy Software Distribution site
  2. Copy your license file to the conf directory of the root directory.

Step 3: Configure the external Postgres database node

You must set up a database for the Deploy to store data. Create and configure an empty database before you start Deploy installation. During the installation, Deploy creates the database schema on the database you created. Optionally, you can have separate database for the operational and reporting databases. For more information about creating two separate databases, see Separate Databases for Reporting and Repository in Deploy.

Use a industrial-grade external database server, for example PostgreSQL, for production use. For more information, see Configure the Database and Artifacts Repository.

For more information about how to create Postgres database, see the PostgreSQL documentation.

Step 4: Configure the Postgres JDBC driver

  1. Download the Postgres JDBC driver. See PostgreSQL JDBC driver.
  2. Copy the JAR file to the lib folder of the root directory.

Step 5: Update the deploy-repository.yaml file with the external database details

  1. In the root folder, go to centralConfiguration folder and open the deploy-repository.yaml file.
  2. Configure the parameters to point to the database schema as shown in the following sample configuration:
xl:
  repository:
    database:
     db-driver-class-name: "org.postgresql.Driver"
     db-password: "samplepassword"
     db-url: "jdbc:postgresql://<IP address of the external database server>/postgres"
     db-username: "sample-user"
     max-pool-size: 10
   artifacts.root: "build/artifacts"

Step 6: Download the ActiveMQ client

  1. Download the JMS message broker—ActiveMQ client.
  2. Add JMS .jar file—org.apache.activemq:activemq-client to thelib folder of the root directory.

Step 7: Set up the ActiveMQ node

  1. Run ActiveMQ in the Docker container.
  2. From the centralConfiguration folder of the root directory, open deploy-task.yaml file.
  3. Set the in-process-worker parameter to false.
  4. Update the JMS details as shown in the sample configuration:

    deploy:
    queue:
     external:
       jms-driver-classname: org.apache.ActiveMQConnectionFactory
         jms-url: tcp://<IP address of the ActiveMQ server node>:61616
       jms-username: admin
         jms-password: admin

Step 8: Set up the Load Balancer node

  1. To install the HA proxy (load balancer), run:
    yum install -y haproxy
  2. Enable the HA proxy by running the following command:
    systemctl enable haproxy
  3. To start the HA proxy, run:
    systemct start haproxy
  4. Run the following command to check the status:
    systemct status haproxy

Step 9: Update the hostname of the Master 1 node

  1. From the conf folder of the root directory, open the deployit.conf file.
  2. Update the server.hostname parameter with the hostname of the master server.
  3. Update the deploy.cluster.node.port parameter with the port number. For example, deploy.cluster.node.port=25520—default port number is 25520.

Step 10: Update the deploy-cluster.yaml file with the cluster mode

Digital.ai Deploy 22.3 brings you the following changes related to the high availability (HA) cluster setup:

  • You can no longer set mode to default in the deploy-cluster.yaml file and have Digital.ai Deploy run in HA active-active cluster mode.
  • You must set mode to full in the deploy-cluster.yaml file if you want to run Deploy in HA active-active cluster mode.

Here’s an example deploy-cluster.yaml configuration file with mode set to full.

deploy:
  cluster:
    akka:
      actor:
        loggers:
        - akka.event.slf4j.Slf4jLogger
        loglevel: INFO
        provider: akka.cluster.ClusterActorRefProvider
      cluster:
        auto-down-unreachable-after: 15s
        custom-downing:
          down-removal-margin: 10s
          stable-after: 10s
        downing-provider-class: ''
    membership:
      heartbeat: 10 seconds
      jdbc:
        connection-timeout: 30 seconds
        idle-timeout: 10 minutes
        leak-connection-threshold: 15 seconds
        max-life-time: 30 minutes
        max-pool-size: 1
        minimum-idle: 1
        password: '{cipher}gfdqswdksahgdksahgdkas'
        pool-name: ClusterPool
        url: ''
        username: ''
      ttl: 60 seconds
    mode: full
    name: xld-active-cluster
  • The deploy.cluster.akka.remote object has been removed from the deploy-cluster.yaml file.

    2022. Digital.ai Software Inc. All rights reserved