Upgrade Deploy 8.0.x and higher to current

This topic describes the prerequisites, considerations and process for upgrading Deploy from version 8.0 or higher to the current version.

Prerequisites

Before you upgrade:

  • We strongly recommend that you take a backup of configuration files and database.
  • Some older database versions may not work with the latest Deploy versions. For more information, refer to System Requirements.
  • Carefully read the release manual and note any changes that may apply to your situation.
  • Review any files that you have customized, such as conf/deployit-security.xml or conf/logback.xml. You will need to redo these changes in the new files.
  • Check whether there are any hotfixes installed in the hotfix directory. If hotfixes are installed, contact the Digital.ai Support team before upgrading.

Upgrade overview

Upgrading Deploy includes the following tasks:

  1. Obtain a new version of the Deploy software and, if necessary, a new license from XebiaLabs.
  2. Read the release manual so you are aware of the new functionality and possible upgrade considerations.
  3. Stop the current version of Deploy if it is running and ensure that there are no active tasks.
  4. Extract the new Deploy release into a directory for the new version of Deploy (so the old version will still be available in case of problems).
  5. Copy data from the old installation directory to the new installation directory.
  6. Redo custom changes that you made to configuration files and startup scripts.
  7. Start the new version of Deploy so that automatic upgraders can run.

You can find release notes for each version here: Release notes.

Considerations

Review the following considerations before upgrading.

Upgrading from a version earlier than 8.0.x

If you are upgrading from a version prior to Deploy 8.0.x, you must first upgrade to version 8.5.x before you can upgrade to the latest version of Deploy.

Also, if you are upgrading from a version prior to Deploy 8.0.x, the data stored in Deploy must be converted from the JackRabbit (JCR) format to SQL format. In this scenario, upgrade using the procedure described in Migrate Deploy data storage to an SQL database.

Masters, workers and satellite

You can use the procedures in this topic to upgrade Deploy masters, workers and satellites that exist in a typical production environment. Make sure that all masters, workers and satellites are upgraded to share the same version. For more information about masters, workers, and satellite configurations refer to the following topics:

Upgrading and downgrading

After you upgrade to a new version of Deploy, you cannot downgrade to an older version. If you upgrade to a release candidate (RC), alpha, or beta version, you cannot upgrade to a newer version or downgrade to an older version. Ensure that you always create a backup of your repository before you upgrade to a new version of Deploy.

Skipping versions

When upgrading, you can skip Deploy versions. Deploy will sequentially apply upgrades for the intermediate versions. However, you may be required to take manual actions for the intermediate versions; you can find these in the release manual.

Upgrading the repository

If a repository upgrade is required, Deploy will detect that it is running against an old repository and will automatically execute an upgrade when it is first started. The server log will contain extensive logging of the repository upgrade process. Save this log for future reference.

Important: New functionality in Deploy or plugins may require database schema changes. As a result, the application requires additional privileges to perform the changes to the schema. These privileges include:

  • ALTER, CREATE, and DROP privileges for tables, views, indexes, and triggers
  • REFERENCES privilege (if applicable)
  • INSERT, SELECT, UPDATE, and DELETE privileges (like in a normal operation)

Upgrading the database

Some database versions may not be supported in the newer versions. You need to upgrade the database server. Same for the server OS, Java Runtime. In others words, they must check their environment support new version.

Upgrading plugins

A plugin version is related to the version of Deploy with which it is compatible. For example, the Kubernetes plugin version 10.1.0 requires Deploy 10.1.0 or later, unless otherwise specified in the release manual.

The new version of Deploy may not be compatible with the current version of your plugins. If this is the case, updated versions of plugins must be installed during the upgrade process. Upgrading to a new plugin version may require you to take manual actions; you can find these in the release manual.

Deploy 10.1.0 or later includes a new Plugin Manager UI for easier managing of plugins. In addition to the new UI, the directory structure has undergone several changes as well. Plugins officially supported by Digital.ai must be located in the plugins/xld-official directory and all other plugins in the plugins/__local__ directory.

When upgrading over 10.1.0 from a lower version of Deploy, plugins must be copied into correct directories from the original plugins directory (this will be done automatically if automated copy file option is chosen, or manually otherwise).

Deploy will not prevent you from downgrading a plugin to an older version, but doing so is not recommended.

Deprecations

Each new version may deprecate some functionality or features in favor of newer ways of working. If functionality is marked as deprecated for a specific version, the old functionality is still available (so you can still upgrade hassle-free), but it will be removed in the next version.

The release manual may include information about how to migrate to the new way of working. This gives you the time and opportunity to migrate to the new situation before upgrading to a still newer version that will no longer have the old functionality. Be sure to read the deprecation information for each release you’re upgrading to, so you know what will change in upcoming versions.

Upgrade procedures

From 9.5 and later versions, you can use a new setup option (-previous-installation) to automate copying of files from the existing installation to the new installation. Choose which option you want to use:

Upgrade using the automated file copy option

To upgrade from Deploy version 8.0.x or higher to a current version using the -previous-installation option:

  1. Extract the server archive for the new version of Deploy. It creates an installation directory called, for example, xl-deploy-10.2-server.
  2. Log in to your existing Deploy instance as an administrator, click Explorer, expand Monitoring, double click Deployment tasks, and select All Tasks.

    • If there are any tasks with a state of Failing or Failed, cancel them.
    • If there are any tasks with a state of Executing, wait for them to complete or cancel them.
    • To open a task from Monitoring, double-click it.
    • Ensure that no new tasks are started by enabling maintenance mode.
  3. Shut down the existing Deploy server.

    Note: If you are running the Deploy server as a service and you want to use it after the upgrade, you must uninstall the existing Deploy service. You must re-install the Deploy as a service from the new location after the upgrade.

  4. From the XL_DEPLOY_SERVER_HOME/bin location of the new installation (e.g., xl-deploy-10.2.0-server/bin), run one of the following commands, depending on your operating system:

    For Unix-based systems:

    ./run.sh -setup -previous-installation `XL_DEPLOY_SERVER_HOME_EXISTING`

    For Microsoft Windows:

    run.cmd -setup -previous-installation `XL_DEPLOY_SERVER_HOME_EXISTING`

    Note: XL_DEPLOY_SERVER_HOME_EXISTING is the path of the old installation directory. ​ For example: ​For Unix-based systems: "/home/.../xl-deploy-8.5.0-server" ​For Microsoft Windows: "C:......\xl-deploy-8.5.0-server"

    The following message displays:

    Do you want to copy files to the new installation?
    Files to be copied [conf/deployit-license.lic, conf/deployit.conf, conf/deployit-defaults.properties, conf/xl-deploy.conf, conf/central-config.version, conf/command-whitelist.json, conf/db-anonymize.json, conf/repository-keystore.jceks, centralConfiguration/xld-client-gui.yaml, centralConfiguration/xld-command-whitelist.yaml, centralConfiguration/xld-db-anonymizer.yaml, centralConfiguration/xld-secret-complexity.yaml, centralConfiguration/xld-websockets.yaml]
  5. Type yes. Your response is registered. Files listed in the conf directory will be copied the new installation. The following prompt displays:

    Do you want to copy directory [XL_DEPLOY_SERVER_HOME_EXISTING/ext] to the new installation?
  6. Type yes. Your response is registered. The contents of the ext directory will be copied to the new installation.
  7. The plugins directory is parsed. Depending on the differences between the existing plugins and the new plugins directory, the following behavior occurs:

    • If a plugin exists in the previous installation, but is not found in the new installation, you are prompted to copy it to the new installation. Type yes to copy the plugin to the new installation or no to not copy it.
    • If a plugin exists in the previous installation but a new version of it is available in the new installation, the following output is provided for each plugin: Plugin [plugin-name] has been updated [9.0.0 -> 10.2.0]. Not copying it.
    • When upgrading over 10.1.0, the automated file copy option will copy files from the plugins directory into either plugins/xld-official (if a plugin is one of the officially supported Deploy plugins) or plugins/__local__ (if a plugin was developed by a 3rd party and is not among the list of officially supported plugins). Both above points still apply during automatic copying of plugins.
  8. The following message displays:

    Going to copy configuration from previous installation [XL_DEPLOY_SERVER_HOME_EXISTING]
    User response was: [yes]. Copying files [conf/deployit-license.lic, conf/xl-deploy.conf, conf/deployit.conf, conf/deployit-defaults.properties, conf/repository-keystore.jceks].

    A status for each of the copied files in the conf directory is provided. For example:

    File [conf/deployit.conf] was copied to the new installation.
  9. The contents of the ext directory are copied to the new installation.
  10. Once all copying is successful, the following message displays:

    Copy from previous installation completed.
    Please note the following:
    - You should copy the database driver to the new installation manually.
    - You should adapt and copy [xl-deploy.conf] to the new installation manually.
  11. Copy the repository directory from the old installation directory to the new installation directory.
  12. Assuming you are using an external database for your repository, copy your database driver to the new installation. Typically, the database driver is stored in your lib directory.
  13. If you have changed other files in the conf directory, such as deployit-security.xml or logback.xml, do not copy the changed files to the new installation directory. Instead, manually reapply the changes to the files that were provided in the new version of Deploy.
  14. Copy the contents of the importablePackages directory from the old installation directory to the new installation directory.
  15. If you have changed the Deploy server startup script(s) in the bin directory, do not copy the changed script(s) to the new installation directory. Instead, manually reapply the changes to the files that were provided in the new version of Deploy. Note: There are also install-service.sh and install-service.cmd scripts for running Deploy as a service. If you customized the run or install-service scripts in the old installation, you must redo these changes in the install-service script in the new installation.
  16. Start the Deploy server interactively (instead of starting as a background process or service) to allow automatic repository upgraders to run. Note: The upgrade process may ask questions or ask for confirmation.
  17. If you normally run the Deploy server as a service, shut it down and restart it as you normally do.

Upgrade by manually copying files

To upgrade from Deploy version 8.0.x or higher to a current version, manually copying files from the old installation to the new installation:

  1. Extract the server archive for the new version of Deploy. It creates an installation directory called, for example, xl-deploy-10.2.0-server.
  2. Log in to your existing Deploy instance as an administrator, click Explorer, expand Monitoring, double click Deployment tasks, and select All Tasks.

    • If there are any tasks with a state of Failing or Failed, cancel them.
    • If there are any tasks with a state of Executing, wait for them to complete or cancel them.
    • To open a task from Monitoring, double-click it.
    • Ensure that no new tasks are started by enabling maintenance mode.
  3. Shut down the Deploy server. Note: If you are running the Deploy server as a service and you want to use it after the upgrade, you must uninstall the existing Deploy service. You must re-install the Deploy as a service from the new location after the upgrade.
  4. Copy the repository directory from the old installation directory to the new installation directory.
  5. Copy the contents of the importablePackages directory from the old installation directory to the new installation directory.
  6. Copy the contents of the plugins directory from the old installation directory to the new installation directory (unless new versions of your plugins were provided with the new Deploy version).

    Note: With Deploy 10.1.0 and higher, plugins from the plugins directory must be copied into plugins/xld-official or plugins/__local__ directory. The official supported plugins documentation contains a list of all plugins which must be copied into the plugins/xld-official directory. Any other 3rd party plugins not mentioned in the official documentation must be copied into the plugins/__local__ directory.

    Tip: Check the release manual for information about plugin incompatibility.

  7. Copy the contents of the ext directory from the old installation directory to the new installation directory.
  8. If you added libraries to Deploy’s lib directory (such as database drivers), copy the additional libraries from the old installation directory to the new installation directory.
  9. Verify that libraries in the lib directory do not also appear in the plugins directory, even if their versions are different.

    For example, if lib contains guava-27.1.jar, then the plugins directory should not contain any guava-x.x.jar file (such as guava-20.0.jar). In this case, you must remove the library from the plugins directory.

  10. Copy the conf/deployit-license.lic file from the old installation directory to the new installation directory.
  11. Copy the following files from existing conf directory to the new installation’s conf directory:

    • deployit.conf
    • deployit-defaults.properties
    • repository-keystore.jceks file (if it exists) from the old installation directory to the new installation directory.
  12. If you have changed other files in the conf directory, such as xl-deploy.conf, deployit-security.xml or logback.xml, do not copy the changed files to the new installation directory. Instead, manually reapply the changes to the files that were provided in the new version of Deploy.
  13. If you have changed the Deploy server startup script(s) in the bin directory, do not copy the changed script(s) to the new installation directory. Instead, manually reapply the changes to the files that were provided in the new version of Deploy. Note: There are also install-service.sh and install-service.cmd scripts for running Deploy as a service. If you customized the run or install-service scripts in the old installation, you must redo these changes in the install-service script in the new installation.
  14. Start the Deploy server interactively (instead of starting as a background process or service) to allow automatic repository upgraders to run. Note: The upgrade process may ask questions or ask for confirmation.
  15. If you normally run the Deploy server as a service, shut it down and restart it as you normally do.

Verify Migration of Configuration Files

In Deploy 10.1 and 10.2 versions, the Deploy configuration files are migrated to the centralConfiguration repository. For more information about the Central Configuration feature, see Central Configuration.

After copying the files (automated file copy or manually), you must verify the migration of the configuration files to the centralConfiguration repository.

The Digital.ai Deploy configuration files are copied to the Central Configuration directory as part of the Digital.ai Deploy Central Configuration feature. In Digital.ai Deploy 10.1, the Central Configuration does not support the migration of all the configuration files to the Central Configuration directory. In Deploy 10.2, all the Deploy configuration files are moved to Central Configuration directory. For a detailed list of configuration files that are moved to the centralConfiguration directory, see Deploy Configuration Files.

Configuration Files Migrated in Deploy 10.1

In Deploy 10.1, the following configuration files are migrated to the Central Configuration directory:

  • xld-client-gui
  • xld-command-whitelist
  • xld-db-anonymizer
  • xld-secret-complexity
  • xld-websockets

Configuration Files Migrated in Deploy 10.2

In Deploy 10.2, the following configuration files are migrated to the Central Configuration directory:

  • deploy-artifact-resolver
  • deploy-cluster
  • deploy-client
  • type-defaults.properties
  • deploy-satellite
  • deploy-task
  • deploy-jmx
  • deploy-metrics
  • deploy-oidc
  • deploy-plugins
  • deploy-repository
  • deploy-server

Note:

  1. In Deploy 10.2, all the configuration files migrated to the Central Configuration directory have the prefix—deploy. The configuration files that were migrated to Central Configuration directory in 10.1 are renamed to reflect this change.
  2. The deploy-client-gui.yaml file is no longer available, and is subsumed by deploy-client.yaml file.
  3. The deployit-defaults.properties file is renamed as type-defaults.properties.

For more information, see Migration of Configuration Files in the Central Configuration Overview topic.

Verify Configuration Files

If you have performed a fresh installation of Digital.ai Deploy 10.1 or Deploy 10.2, do the following:

  1. Verify whether the configuration files for the release are available in the centralConfiguration directory.
  2. Configure the configuration files in the directory.
  3. To refresh the configuration files without restarting server, invoke the refresh Spring Boot Actuator endpoint manually by forcing the client to refresh and generate the new value:

    $ curl XLD_URL/actuator/refresh -d {} -H "Content-Type: application/json"

Verify Migration of Configuration Files

This section describes the steps that you need to perform after you upgrade to the newer version of Digital.ai Deploy versions —Deploy 10.0 to Deploy 10.1 and Deploy 10.1 to Deploy 10.2—to ensure the configuration files are migrated to the centralConfiguration directory.

Deploy 10.0 to Deploy 10.1

  1. In the Deploy 10.1, verify the configuration files in the centralConfiguration directory (XL_DEPLOY_SERVER_HOME/centralConfiguration).
  2. Copy the configuration files from the older server (Deploy 10.0) to the new server (Deploy 10.1). See Upgrade Procedures.
  3. Start the Deploy server.
  4. Verify whether the centralConfiguration directory has the configuration files listed in Configuration Files Migrated in Deploy 10.1.
  5. Restart the server.

Deploy 10.1 to Deploy 10.2

  1. In Deploy 10.2, verify the centralConfiguration directory (XL_DEPLOY_SERVER_HOME/centralConfiguration).
  2. Copy the configuration files from the older server (Deploy 10.1) to the new server (Deploy 10.2). See Upgrade Procedures.
  3. Start the Deploy server.
  4. Verify whether the centralConfiguration directory has the configuration files listed in Configuration Files Migrated in Deploy 10.2.
  5. Restart the server.

Note: A restart is required after the upgrade, as some of the configuration files may not get populated immediately.

Upgrade the Deploy CLI

To upgrade a Deploy command-line interface (CLI) installation:

  1. Create a directory for the new Deploy CLI installation, including the new Deploy CLI version number in the directory name.
  2. Extract the CLI archive in this directory.
  3. Copy the contents of the plugins directory from the previous installation to the new installation directory (unless new versions of your plugins were provided with the new Deploy version).
  4. Copy the contents of the ext directory from the previous installation to the new installation directory.

    Note: Do not copy the content of the hotfix directory unless instructed to do so (because hotfixes are version-specific).

  5. If you have made any changes to the Deploy CLI startup scripts (cli.sh or cli.cmd), copy these from the bin directory in the previous installation to the new installation directory.