The Digital.ai Continuous Testing (formally Experitest) plugin

About the plugin

The Continuous Testing plugin allows users to execute Android, iOS and web application test cases that will accelerate your Release Cycles with our SeeTest Continuous Testing Platform. SeeTest can execute hundreds of manual or automated tests in parallel on real iOS & Android devices in the cloud.

Features

  • Execute Espresso test cases by providing .apk files.
  • Execute XCUI test cases by providing .ipa files.
  • Get the run status and results of your Espresso or XCUITest execution.
  • Create test views to house result sets.
  • Get test view results from your Appium test jobs.

Setup the Continuous Testing Server

To setup a server connection, do the following:

  1. In Release, on the top menu, go to Settings > Shared configuration on Continuous Testing instance, click on “+“.
  2. Under the New Continuous Testing: instance, fill-in the required fields below:

    1. Enter a title in the Title field. (ex: Continuous Testing USCloud Server)
    2. Enter the URL where the Continuous Testing server is running. This can be a local instance you have configured on premises or one of the DAI Cloud SaaS instances. (ex: https://uscloud.experitest.com)
  3. Under the Authentication section, fill-in the fields below:

    1. Enter the Continuous Testing username in the username field. (example: admin)
    2. Enter the Continuous Testing password in the password field. (example: testing123)
  4. Under the Proxy section, fill-in the appropriate fields. Please be advised that these fields are optional and only required if dictated by your organization.
  5. Click on Test to test the availability of the Continuous Testing Instance.
  6. Click on Save.

Save Shared Configuration

Usage

Available Tasks: Trigger Espresso Test Case, Trigger XCUITest case, Get Test Run Status And Result For Unit Tests, Create Test View, Get Test View Results

Trigger Espresso Test Case

Note: This task is used to run test cases for Android applications (.apk)

To create a Trigger Expresso Test Case task, do the following:

  1. Navigate and select the respective Release Template where you want to create the Trigger Expresso Test Case task.
  2. Under the Phase, click on Add task.
  3. Enter the Title in the Task title field.
  4. Select Continuous Testing > Trigger Espresso Test Case using the dropdown.

Select Trigger Espresso test case

  1. Click on Add.
  2. Click on the created task (Trigger Espresso Test Case).
  3. Enter the required properties in the opened Trigger Espresso Test Case task.
  4. By default, the server field picks the created server. If you have multiple configured Continuous Testing servers, you can pick which one you want to use.
  5. Select the required Running Type.

    Note: Here we define the execution mode. Coverage Running Type is used to run ALL test cases on devices and fastFeedback is used to spread the test cases across devices.

  6. Enter the URL where the APK application can be retrieved.

    Example: https://sitename.s3.us-east-2.amazonaws.com/filename.apk -

    Note: you can also retrieve from a public facing repo like Github or Bitbucket, but at this time the plugin does not have a way to pass credentials to a secured repo site. This capability will be coming in a future release.

  7. Enter the URL where the APK test cases can be retrieved.

    Example: https://sitename.s3.us-east-2.amazonaws.com/test-filename.apk -

    Note: you can also retrieve from a public facing repo like Github or Bitbucket, but at this time the plugin does not have a way to pass credentials to a secured repo site. This capability will be coming in a future release.

  8. Enter the appropriate Device Query.

    Note: the details for constructing a device query can be obtained here: https://docs.experitest.com/display/TE/Device+Queries

  9. Close the Trigger Espresso Test Case task.

Close Trigger Espresso test case

Trigger XCUI Test Case

Note: This task is used to run test cases for iOS applications (.ipa)

To create a Trigger XCUI Test case task, do the following:

  1. Navigate and select the respective Release Template where you want to create the Trigger XCUI Test task.
  2. Under the Phase, click on Add task.
  3. Enter the Title in the Task title field.
  4. Select the Continuous Testing > Trigger XCUI Test Case using the dropdown.

Select Trigger XCUI test case

  1. Click on Add.
  2. Click on the created task (Trigger XCUI Test Case).
  3. Enter the required properties in the opened Trigger XCUI Test Case task.
  4. By default, the server field picks the created server. If you have multiple configured Continuous Testing servers, you can pick which one you want to use.
  5. Select the required Running Type.

    Note: Here we define the execution mode. Coverage Running Type is used to run ALL test case on devices and fastFeedback is used to spread the test cases across devices.

  6. Enter the URL where the IPA application can be retrieved.

    Example: https://sitename.s3.us-east-2.amazonaws.com/filename.ipa -

    Note: you can also retrieve from a public facing repo like Github or Bitbucket, but at this time the plugin does not have a way to pass credentials to a secured repo site. This capability will be coming in a future release.

  7. Enter the URL where the IPA test cases can be retrieved.

    Example: https://sitename.s3.us-east-2.amazonaws.com/test-filename.ipa -

    Note: you can also retrieve from a public facing repo like Github or Bitbucket, but at this time the plugin does not have a way to pass credentials to a secured repo site. This capability will be coming in a future release.

  8. Enter the appropriate Device Query.

    Note: the details for constructing a device query can be obtained here: https://docs.experitest.com/display/TE/Device+Queries

  9. Close the Trigger XCUI Test Case task.

Close Trigger XCUI test case

Get Test Run Status and Result for Unit Tests

Note: This task provides the test results for Espresso and XCUI Test tasks.

To create Get Test Run Status and Result for Unit Test task, do the following:

  1. Under the Phase, click on Add task.
  2. Enter the Title in the Task title field.
  3. Select the Continuous Testing > Get Test Run Status and Result for Unit Tests using the dropdown.

Select test run status and result for unit test

  1. Click on Add.
  2. Click on the created task (Get Test Run Status and Result for Unit Tests).
  3. Enter the required properties in the opened Get Test Run Status and Result for Unit Tests task.
  4. By default, the server field picks the created server. If you have multiple configured Continuous Testing servers, you can pick which one you want to use.
  5. Enter the Test Run ID. This value is an output property from the Trigger Test Case (Espresso / XCUI Test) tasks. Leverage variables to pass this value between the two tasks.
  6. Close the Get Test Run Status and Result for Unit Tests task.

Close the get test run status

The Output properties shown above will be populated after the tests have completed executing in your SeeTest instance.

Create Test View

To create a Create Test View task, do the following steps:

  1. Under the Phase, click on Add task.
  2. Enter the Title in the Task title field.
  3. Select the Continuous Testing > Create Test View using the dropdown.

Select the create test view

  1. Click on Add.
  2. Click on the created task (Create Test View).
  3. Enter the required properties in the opened Create Test View task.
  4. By default, the server field picks the created server. If you have multiple configured Continuous Testing servers, you can pick which one you want to use.
  5. Enter a string value in the Test View Name field.
  6. Close the Test View Name task.

Close the Test View Name task

Get Test View Results

To create Get Test View Results task, do the following steps:

  1. Under the Phase, click on Add task.
  2. Enter the Title in the Task title field.
  3. Select the Continuous Testing > Get Test View Results using the dropdown.

Select the get test view results

  1. Click on Add.
  2. Click on the created task (Get Test View Results).
  3. Enter the required properties in the opened Get Test View Results task.
  4. By default, the server field picks the created server. If you have multiple configured Continuous Testing servers, you can pick which one you want to use.
  5. Enter the Test view ID in the Test View ID field. Test View ID is an output property from the Create Test View task. Use release variables to pass this value between tasks.
  6. Enter the Jenkins Build Number in the Jenkins Build Number field. You can get this value from the Jenkins task. It is an output property of the Jenkins task. Again, use release variables to pass this value between tasks.
  7. Close the Get Test View Results task.

Close the get test view results task