Database Anonymizer

This module provides an anonymizing functionality for dumping the data of Deploy database, which you can afterward import back by using the anonymizer tool.

Anonymizer mean to replace all sensitive information, mainly intended for passwords and dictionary values.

Configuration to Anonymizer database

The configuration file central-config/xld-db-anonymize.yaml contains the rules how the data from the database is going to be export. There are three section in it.

  1. For tables not to export: For example users information could be quite sensitive, so by default this table not to be exported.
deploy.db-anonymizer:
  tables-to-not-export:
    - XL_USERS
  tables-to-anonymize:
    - table: XLD_DICT_ENTRIES
      column: value
      value: placeholder
    - table: XLD_DICT_ENC_ENTRIES
      column: value
      value: enc-placeholder
    - table: XLD_DB_ARTIFACTS
      column: data
      value: file
  content-to-anonymize: []
  encrypted-fields-to-ignore:
    - password-regex: "\\{aes:v0\\}.*"
      table: XLD_CI_PROPERTIES
      column: string_value
      value: password
  1. The original content of specific column in a specific table will be replaced with the define value in “value” field
"tables-to-anonymize": [
  {
    "table": "XL_USERS",
    "column": "PASSWORD",
    "value": "password"
  },
  {
    "table": "XLD_DICT_ENTRIES",
    "column": "value",
    "value": "placeholder"
  }
]
  1. Any column containing specific content of text is going to be replaced with the provided value.
"content-to-anonymize": [
  {
    "key": "Xebialabs",
    "value": "Digital.ai"
  }
]

Note: Anonymizing the content which is same as dictionary title will change the key and dictionary title as well. Anonymizing the content which is same as dictionary type will corrupt the dictionary.

To anonymize the encrypted CI password with the local key store, edit the conf/db-anonymize.json file with the following configuration:

"encrypted-fields-to-ignore": [
    {
      "passwordRegex": "\\{aes:v0\\}.*",
      "table": "XLD_CI_PROPERTIES",
      "column": "string_value",
      "value": "password"
    }
  ]

Export Anonymizing Database:

You can export anonymized data by using the command ./bin/db-anonymizer.sh.

The data will be dumped in the server home directory with the file named xl-deploy-repository-dump.xml and its corresponding validation file xl-deploy-repository-dump.dtd.

Important: If the user uses two databases (repository and reporting), user should execute the -reports to export the reporting database data file xl-deploy-reporting-dump.xml.

Import Anonymizing Database:

You can import anonymized data by using the command ./bin/db-anonymizer.sh -import

Command specific flag options while importing

Flag
Description
-import Imports data to empty database

Note: If the file is not specified it will try to import file named xl-deploy-repository-dump.xml from the server home directory. To import a specific file from different location use -import -f <absolute-path-of-file>option. Make sure xl-deploy-repository-dump.dtd file is avaliable along with the xl-deploy-repository-dump.xml in the absolute path
-f Imports a specified data file
-refresh Refreshes data in the database

Note: Every record will be verified prior to inserting. Therefore the import time increases
-batchSize Specifies the maximum number of commands in a batch

Note: Optimal batch size is different for each specific case and DBMS. However the default value 100 provides good enough results in the most cases. If you want to disable batch processing then set the value to ‘0’”
-reports Performs import on the reporting database