Configure task queuing

As of 9.5, Deploy uses task queuing for workers using the Java Message Service (JMS) 2.0 protocol. Deploy will queue any tasks, regardless of whether a worker is available to execute them. When a worker does become available, it will automatically pick tasks from the queue and execute them.

How it works

Understanding tasks in Deploy provides a detailed discussion of Deploy task types and behavior.

As described in High availability with master-worker setup, there are three types of workers: in-process, local and external.

  • If you are running Deploy using the in-process worker, no configuration changes are required and Deploy will fallback to an in-process Artemis queue. Note: an in-process worker setup is not recommended for production environments.
  • If you are using local and/or external workers, an external task queue is required and you can perform the configuration described in this topic to support task queuing.

Starting with version 9.5, Deploy will now hold pending tasks in the queue that are not yet assigned a worker, and wait for one to become available. In this case, you can create a task and it will transition to a queued state. If you schedule a task, it will be successfully scheduled, even if no worker is available. When the scheduled start time has passed, the task will be sent to the queue and the worker will pick it up from the queue.

From Deploy’s perspective, the queued state already existed in earlier versions, but it works differently as of Deploy 9.5:

  • In previous Deploy versions, the task was immediately assigned to a worker which had its own internal queue. Tasks that exceeded the available threads on that worker were listed as queued.
  • Now, the internal worker queue is replaced by a shared JMS queue. A task can be in the queued state without being assigned a worker to process it. Once a worker has a free execution thread, it will pick up the task and begin processing it.

Configuration overview

To configure task queuing:

  1. Determine your JMS 2.0 message broker (for example, ActiveMQ, RabbitMQ).
  2. Configure your message broker setup.
  3. Download the .jar files for your message broker and copy them to the Deploy lib directory.
  4. Update your conf/xl-deploy.conf file to include the message broker details.
  5. (Optional) Navigate to your message broker’s web console URL.
  6. Restart Deploy.

Select a JMS message broker

There are many JMS 2.0 message brokers available, and the configuration is similar for each. XebiaLabs specifically tested the following JMS 2.0 brokers:

  • ActiveMQ (currently supported version is ActiveMQ Artemis 2.15.0)
  • RabbitMQ

You can select any other JMS 2.0-based broker and use the examples in this topic to configure it.

Configure your JMS Message broker setup

To properly manage queuing for all types of Deploy tasks, your JMS message broker setup should be configured to support both the Queue and Topic approaches. For background, see the following:

Note the following:

  • The Queue approach is used for tasks such as initial deployments, upgrades and undeployments.
  • The Topic approach is used for Rollback to send the Rollback task to the specific worker that was performing the deployment to be rolled back.
  • For RabbitMQ, the Topic exchange is disabled by default and should be enabled by Enabling the Topic Selector Plug-in.

Add JMS .jar files to Deploy

This section provides details specific to ActiveMQ and RabbitMQ. Download the latest version for your preferred message broker:

ActiveMQ files

Add the org.apache.activemq:activemq-client jar to your lib directory.

RabbitMQ files

Add the following drivers to your lib folder:

  • com.rabbitmq.jms:rabbitmq-jms
  • com.rabbitmq:amqp-client

Update your xl-deploy.conf file

On all masters and workers in your environment, update your xl-deploy.conf file to configure your external queue settings.

ActiveMQ example

Edit the task section in conf/xl-deploy.conf file to include queue configuration details.

Here is an example configuration for ActiveMQ:

xl {
    task {
        in-process-worker=false
        queue {
            name="xld-tasks-queue"
            external {
                jms-driver-classname="org.apache.activemq.ActiveMQConnectionFactory"
                jms-password="my_password"
                jms-url="ip_or_hostname"
                jms-username=my_username
            }
        }
    }
}

RabbitMQ example

Edit the task section in conf/xl-deploy.conf file to include queue configuration details.

Here is an example configuration for RabbitMQ:

xl {
    task {
        in-process-worker=false
        queue {
            name="xld-tasks-queue"
            external {
                jms-driver-classname="com.rabbitmq.jms.admin.RMQConnectionFactory"
                jms-password="my_password"
                jms-url="ip_or_hostname"
                jms-username=my_username
            }
        }
    }
}

Complete the configuration

  1. Ensure that all masters and workers in your environment share the same xl-deploy.conf configuration.
  2. Restart all masters and workers.
  3. You can confirm the connection by executing a simple task and navigating to the broker’s web console URL to see that the task was processed.

Monitoring

In addition to the task monitoring you can perform using Deploy, you can also use the broker’s web console to monitor the queue load. For example, if there are many unread messages in the queue/topics, you can consider increasing a worker’s execution threads or add more workers to your Deploy environment.