Plugin management in the operator environment

Here’s what it takes to manage Digital.ai Release plugins on a Release cluster that was created using the Operator-based installer:

  1. Create a temporary pod—dai-xlr-plugin-management.
  2. Stop all the other Release Pods but the newly created pod—dai-xlr-plugin-management.
  3. Log on (SSH) to the newly create temporary pod—dai-xlr-plugin-management.
  4. Add or remove plugins using the Plugin Manager CLI.
  5. Restart all the Release pods.
  6. Delete the temporary pod—dai-xlr-plugin-management.

Note: This topic uses the default namespace, digitalai, for illustrative purposes. Use your own namespace if you have installed Release in a custom namespace.

  1. Verify the PVC name on you current namespace (it depends on the CR name):

    ❯ kubectl get pvc -n digitalai
    NAME                        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                                   AGE
    dai-xlr-digitalai-release   Bound    pvc-1ace4aaf-d5e2-4ad6-a913-a39924310ea6   1Gi        RWO            vp-azure-aks-test-cluster-file-storage-class   150m
    data-dai-xlr-postgresql-0   Bound    pvc-d761fcee-787c-4393-b346-6c3e79e06112   1Gi        RWO            vp-azure-aks-test-cluster-disk-storage-class   150m
    data-dai-xlr-rabbitmq-0     Bound    pvc-dc39f7dd-d05a-4f87-acaf-89473a791765   1Gi        RWO            vp-azure-aks-test-cluster-file-storage-class   150m

    Suppose the release PVC name is dai-xlr-digitalai-release.

  2. Create a pod-dai-xlr-plugin-management.yaml file and add the following code to the file:

    apiVersion: v1
    kind: Pod
    metadata:
      name: dai-xlr-plugin-management
    spec:
      securityContext:
        fsGroup: 10001
      containers:
        - name: sleeper
          command: ["/bin/sh"]
          args: ["-c", "cp -f /opt/xebialabs/db-libs/postgresql*.jar /opt/xebialabs/xl-release-server/lib; sleep 1d;"]
          image: xebialabs/xl-release:<release version>
          imagePullPolicy: Always
          volumeMounts:
            - mountPath: /opt/xebialabs/xl-release-server/reports
              name: reports-dir
              subPath: reports
            - mountPath: /opt/xebialabs/xl-release-server/work
              name: reports-dir
              subPath: work
            - mountPath: /opt/xebialabs/xl-release-server/conf
              name: reports-dir
              subPath: conf
            - mountPath: /opt/xebialabs/xl-release-server/ext
              name: reports-dir
              subPath: ext
            - mountPath: /opt/xebialabs/xl-release-server/hotfix
              name: reports-dir
              subPath: hotfix
            - mountPath: /opt/xebialabs/xl-release-server/hotfix/lib
              name: reports-dir
              subPath: lib
            - mountPath: /opt/xebialabs/xl-release-server/hotfix/plugins
              name: reports-dir
              subPath: plugins
      restartPolicy: Never
      volumes:
        - name: reports-dir
          persistentVolumeClaim:
            claimName: <release PVC name>

    Replace the placeholders:

    • <release version> - with your release version name
    • <release PVC name> - with the PVC name from the first step
  3. Apply the pod-dai-xlr-plugin-management.yaml file.

    ❯ kubectl apply -f pod-dai-xlr-plugin-management.yaml -n digitalai
  4. Stop the Release pods.

    1. Set the number of replicas to 0.
    ❯ kubectl get digitalaireleases.xlr.digital.ai -n digitalai
    NAME      AGE
    dai-xlr   179m
    
    ❯ kubectl patch digitalaireleases.xlr.digital.ai dai-xlr -n digitalai \
    --type=merge \
    --patch '{"spec":{"replicaCount":0}}'
    1. Restart the Release stateful set, the name depends on the CR name (or you can wait few seconds, the update will be automatic after earlier change):
    ❯ kubectl rollout restart sts dai-xlr-digitalai-release -n digitalai

    Wait until all the Release pods terminate.

  5. Log on (SSH) to the newly created pod.

    kubectl exec -it dai-xlr-plugin-management -n digitalai -- bash
  6. Add or remove the plugins using the Plugin Manager CLI. Go to bin directory after SSH login:

    cd /opt/xebialabs/xl-release-server/bin

    See Plugin Manager CLI to know more about how to add or remove plugins.

    For example, the following commands are to delete the xlr-svn-plugin plugin.

    bash-4.2$ ./plugin-manager-cli.sh -list
    ...
    
    bash-4.2$ ./plugin-manager-cli.sh -delete xlr-svn-plugin
    
    xlr-svn-plugin deleted from database
    Please verify and delete plugin file in other cluster members' plugins directory if needed

    Exit the SSH shell using the exit command.

  7. Restart the Release pods.

    1. Set the number of replicas back to the required number (2 replicas in this example)
    ❯ kubectl get digitalaireleases.xlr.digital.ai -n digitalai
    NAME      AGE
    dai-xlr   179m
    
    ❯ kubectl patch digitalaireleases.xlr.digital.ai dai-xlr -n digitalai \
    --type=merge \
    --patch '{"spec":{"replicaCount":2}}'
    1. Restart the Release stateful set, the name depends on the CR name (or you can wait few seconds, the update will be automatic after previous change):
    ❯ kubectl rollout restart sts dai-xlr-digitalai-release -n digitalai
  8. Delete the temporary pod—dai-xlr-plugin-management.

    ❯ kubectl delete pod dai-xlr-plugin-management -n digitalai

A Script to Automate All the Above Steps

Replace the environment variables with relevant values of your Kubernetes environment.

Here is an example that shows how this could be done with kubectl as the bash script.

Note: Stop all the pods before you run the script.

SOURCE_PLUGIN_DIR=/tmp
PLUGIN_NAME=xlr-hashicorp-vault-plugin
PLUGIN_VERSION=22.3.0-704.113
SOURCE_PLUGIN_FILE=$PLUGIN_NAME-$PLUGIN_VERSION.jar
RELEASE_STS=dai-xlr-digitalai-release
RELEASE_CR=dai-xlr
NAMESPACE=digitalai
REPLICA_COUNT=$(kubectl get digitalaireleases.xlr.digital.ai dai-xlr -n digitalai -o 'jsonpath={.spec.replicaCount}')

kubectl apply -f pod-dai-xlr-plugin-management.yaml -n digitalai
kubectl patch digitalaireleases.xlr.digital.ai $RELEASE_CR -n $NAMESPACE \
      --type=merge \
      --patch '{"spec":{"replicaCount":0}}'
sleep 30; kubectl rollout status sts $RELEASE_STS -n $NAMESPACE --timeout=300s
kubectl wait --for condition=Ready --timeout=60s pod dai-xlr-plugin-management -n $NAMESPACE

kubectl cp $SOURCE_PLUGIN_DIR/$SOURCE_PLUGIN_FILE $NAMESPACE/dai-xlr-plugin-management:$SOURCE_PLUGIN_DIR/  
kubectl exec dai-xlr-plugin-management -n $NAMESPACE -- /opt/xebialabs/xl-release-server/bin/plugin-manager-cli.sh -add $SOURCE_PLUGIN_DIR/$SOURCE_PLUGIN_FILE

kubectl patch digitalaireleases.xlr.digital.ai $RELEASE_CR -n $NAMESPACE \
      --type=merge \
      --patch "{\"spec\":{\"replicaCount\":$REPLICA_COUNT }}"
kubectl delete -f pod-dai-xlr-plugin-management.yaml -n $NAMESPACE
sleep 30; kubectl rollout status sts $RELEASE_STS -n $NAMESPACE --timeout=300s