Testruns: Swarm Test Integration

Get testrun details of a review version

Summary

Get the testrun details of a review version.

GET /api/v11/reviews/{reviewId}/testruns

Description

Used to get the testrun details for a review version.

Note

Any authenticated user can get test run details for a review version. The test run uuid is not included in the request response.

Parameters

Parameter Description Type Parameter Type Required

reviewId

Review ID

integer

path

Yes

version

An optional parameter that enables you to specify the review version you want to return the testrun details for. Omitting this parameter or passing an empty value shows testrun details for the latest version of the review.

string

query

No

Example of usage

Get details for all of the test runs for the latest version of review 12345

curl "https://myswarm.url/api/v11/reviews/12345/testruns"

Swarm responds with the testrun details:

HTTP/1.1 200 OK		

{
  "error": null,
  "messages": [],
  "data" : {
    "testruns" : [
      {
        "id": 706,
        "change": 12345,
        "version": 2,
        "test": "global1",
        "startTime": 1567895432, 
        "completedTime": 1567895562, 
        "status": "pass",
        "messages": [
          "Test completed successfully",
          "another message" 
        ],
        "url": "http://my.jenkins.com/projectx/main/1224"
      },
      {
        <ids for other testruns of the review, formatted as above>
      }
    ]
  }
}

Get details for all of the test runs for version 2 of review 12345

curl "https://myswarm.url/api/v11/reviews/12345/testruns?version=2"

Swarm responds with the testrun details:

HTTP/1.1 200 OK		

{
  "error": null,
  "messages": [],
  "data" : {
    "testruns" : [
      {
        "id": 706,
        "change": 12345,
        "version": 2,
        "test": "global1",
        "startTime": 1567895432, 
        "completedTime": 1567895562, 
        "status": "pass",
        "messages": [
          "Test completed successfully",
          "another message"
        ],
        "url": "http://my.jenkins.com/projectx/main/1224"
      },
      {
        <ids for other testruns for version 2 of the review, formatted as above>
      }
    ]
    "status" : "pass"
  }
}
Tip

When you make a GET request for all of the testruns in a specific review version, Swarm includes a "status" for the review version as a whole in the response. This is shown after the individual testruns. This status is calculated from the "status" values of all of the individual testruns for that review version:

  • If the test "status" for any of the testruns is "running" or if any On Demand tests have not been run, the overall result for that version is "running".
  • If no testruns are "running" and the test "status" of any of the testruns is "fail" the overall result for that version is "fail".
  • If the test "status" of all of the testruns is "pass" then the overall result for that version is "pass".

If a request fails

HTTP/1.1 <response error code>		

{
  "error": <high level description>,
  "messages": [{"code" : "<code string>", "msg" : "<error message>"}],
  "data" : null
}

Create a testrun for a review version

Summary

Create a testrun for a review

POST /api/v11/reviews/{reviewid}/testruns

Description

Create a testrun for a review version. After you have created a testrun for a version of a review, you can update the testrun details as the test progresses using the PATCH and PUT API endpoints or the {pass}, {fail} and {update} callback urls.

Important

Only admin users can create a test run for a review.

Parameters

Parameter Description Type Parameter Type Required Restriction

reviewid

Review ID.

integer

path

Yes

  

version

The version of the review the testrun is being run against.

integer

body

Yes

  

test

Specifies the test used for the testrun and checks that it is a valid name.

string

body

Yes

1 to 32 characters

startTime

The time the testrun started, expressed as an epoch value.

integer

body

Yes

> 0

completedTime

The time the testrun completed, expressed as an epoch value.

integer

body

No

> 0

status

The status of the testrun, options are: pass, fail, and running

string

body

No

Valid options are pass, fail, and running

messages

An array of one or more messages for the testrun. They should be formatted in JSON in the body of the POST request. You can pass a maximum 10 messages, if you provide more than 10 messages only the first 10 are saved. Each message can contain a maximum of 80 characters, any messages with more than 80 characters will be automatically truncated.

array

body

No

  

url

The CI system url for the testrun, this is the link to the CI that enables you to view the current testrun results.

string

body

No

1 to 1024 characters

uuid

The uuid for the testrun. Used to enable non-admin users to update the testrun. If the requested uuid does not match the review an error message is returned.

string

body

No

Minimum 32 max 64 characters

Example usage

Append a new testrun to version 2 of review 12345

curl -X POST -H "Content-Type: application/json" -u "username:ticket" -d "@mybodyfilename.txt" "https://myswarm.url/api/v11/reviews/12345/testruns"

The "mybodyfilename.txt" file contains the testrun details:

{
  "change": 12345,
  "version": 2,
  "test": "mytest",
  "startTime": 1567895432, 
  "status": "running",
  "messages": [
    "Mytest running",
    "another message"
  ]
  "url": "http://my.jenkins.com/projectx/main/1224",
  "uuid": "FAE4501C-E4BC-73E4-A11A-FF710601BC3F"
}

Swarm responds with the testrun details:

HTTP/1.1 200 OK		

{
  "error": null,
  "messages": [],
  "data" : {
    "testruns" : [
      {
        "id": 706,
        "change": 12345,
        "version": 2,
        "test": "mytest",
        "startTime": 1567895432, 
        "completedTime": null, 
        "status": "running",
        "messages": [
          "Mytest running",
          "another message"
        ],
        "url": "http://my.jenkins.com/projectx/main/1224"
      }
    ]
  }
}

If the request fails

HTTP/1.1 <response error code>		

{
  "error": <high level description>,
  "messages": [{"code" : "<code string>", "msg" : "<error message>"}],
  "data" : null
}

Create a testrun for a review version using a specified test

Summary

Create a testrun for a review using a specified test

POST /api/v11/reviews/{reviewid}/testruns/{test_id}

Description

Create a testrun for a review version using a specified test. Tests are configured from the Swarm Tests page, see Tests. After you have created a testrun for a version of a review, you can update the testrun details as the test progresses using the PATCH and PUT API endpoints or the {pass}, {fail} and {update} callback urls.

Important

Only admin users can create a test run for a review.

Note

The test_id parameter specified must exist in Swarm, if it doesn't exist an error is generated when the API request is made.

Parameters

Parameter Description Type Parameter Type Required Restriction

reviewid

Review ID.

integer

path

Yes

  

version

The version of the review the testrun is being run against.

integer

body

Yes

  

test_id

Specifies the test_id used for the testrun and checks that it is a valid test id. If there is a test name in the body of the request, Swarm will ignore it.

If the test_id parameter is omitted, Swarm uses the test name in the body of the request.

string

path

Yes

1 to 32 characters

startTime

The time the testrun started, expressed as an epoch value.

integer

body

Yes

> 0

completedTime

The time the testrun completed, expressed as an epoch value.

integer

body

No

> 0

status

The status of the testrun, options are: pass, fail, and running.

string

body

No

Valid options are pass, fail, and running

messages

An array of one or more messages for the testrun. They should be formatted in JSON in the body of the POST request. You can pass a maximum 10 messages, if you provide more than 10 messages only the first 10 are saved. Each message can contain a maximum of 80 characters, any messages with more than 80 characters will be automatically truncated.

array

body

No

  

url

The CI system url for the testrun, this is the link to the CI that enables you to view the current testrun results.

string

body

No

1 to 1024 characters

uuid

The uuid for the testrun. Used to enable non-admin users to update the testrun. If the requested uuid does not match the review an error message is returned.

string

body

No

Minimum 32 max 64 characters

Example usage

Append a testrun for a review using test id 5 for version 3 of review 12345

curl -X POST -H "Content-Type: application/json" -u "username:ticket" -d "@mybodyfilename.txt" "https://myswarm.url/api/v11/reviews/12345/testruns/5"

The "mybodyfilename.txt" file contains the testrun details:

{
  "change": 12345,
  "version": 3,
  "startTime": 1567895432, 
  "status": "running",
  "messages": [
    "Smoke-test running",
    "another message"
  ],
  "url": "http://my.jenkins.com/projecty/main/1224",
  "uuid": "FAE4501C-E4BC-73E4-A11A-FF710601BC4F"
}

Swarm responds with the testrun details:

HTTP/1.1 200 OK		

{
  "error": null,
  "messages": [],
  "data" : {
    "testruns" : [
      {
        "id": 5,
        "change": 12345,
        "version": 3,
        "test": "test01",
        "startTime": 1567895432, 
        "completedTime": null, 
        "status": "running",
        "messages": [
          "Smoke-test running",
          "another message"
        ],
        "url": "http://my.jenkins.com/projecty/main/1224"
      }
    ]
  }
}

If the request fails

HTTP/1.1 <response error code>		

{
  "error": <high level description>,
  "messages": [{"code" : "<code string>", "msg" : "<error message>"}],
  "data" : null
}

Update details for a testrun - PATCH

Important

Only admin users can update testrun details for a review.

Summary

Update details for a testrun

PATCH /api/v11/reviews/{reviewid}/testruns/{id}

Description

Update the details for a testrun.

Here are a few example scenarios related to depot permissions, private projects, and testrun API endpoint.

  • If a user is a private project member with access to the depot and has "write" permissions, they can view the review in Swarm. However, the user does not have permissions to run the testRun API.

  • If a user is not a member of a private project but can access the depot, and if the user has "write" permissions for the depot, the user will not be able to view the review in Swarm or run the testRun API.

  • If a user is not a private project member but has "admin" permissions for the Helix Core Server while using the testRun API, they can view the review in Swarm, update the test status, and test description.

  • If a user is not a private project member but has “write” permissions for the Helix Core Server, the user cannot see the review in Swarm and they cannot run the testRun API.

Parameters

Parameter Description Type Parameter Type Required Restriction

Id

Testrun ID, identifies a specific testrun for the review. Automatically generated by Swarm when the testrun is created.

integer

path

Yes

  

reviewid

Review ID.

integer

path

Yes

  

version

The version of the review the testrun is being run against.

integer

body

No

  

test

Specifies the test used for the testrun and checks that it is a valid name.

string

body

No

1 to 32 characters

startTime

The time the testrun started, expressed as an epoch value.

integer

body

No

> 0

completedTime

The time the testrun completed, expressed as an epoch value.

integer

body

No

> 0

status

The status of the testrun, options are: pass, fail, and running

string

body

No

Valid options are pass, fail, and running

messages

An array of one or more messages for the testrun. They should be formatted in JSON in the body of the POST request. You can pass a maximum 10 messages, if you provide more than 10 messages only the first 10 are saved. Each message can contain a maximum of 80 characters, any messages with more than 80 characters will be automatically truncated.

array

body

No

  

url

The CI system url for the testrun, this is the link to the CI that enables you to view the current testrun results.

string

body

No

1 to 1024 characters

uuid

The uuid for the testrun. Used to enable non-admin users to update the testrun. If the requested uuid does not match the review an error message is returned.

string

body

No

Minimum 32 max 64 characters

Example usage

Update testrun 706 for review 12345

curl -X PATCH -H "Content-Type: application/json" -u "username:ticket" -d "@mybodyfilename.txt" "https://myswarm.url/api/v11/reviews/12345/testruns/706"

The "mybodyfilename.txt" file contains the testrun details you want to update. 

{
  "completedTime": "1567895562",
  "status": "fail",
  "messages": [
    "Test has failed",
    "yet another message"
  ]
}

Swarm responds with the testrun details:

HTTP/1.1 200 OK		

{
  "error": null,
  "messages": [],
  "data" : {
    "testruns" : [
      {
        "id": 706,
        "change": 12345,
        "version": 2,
        "test": "mytest",
        "startTime": 1567895432, 
        "completedTime": 1567895562, 
        "status": "fail",
        "messages": [
          "Test has failed",
          "yet another message"
        ],
        "url": "http://my.jenkins.com/projectx/main/1224"
      }
    ]
  }
}

If the request fails

HTTP/1.1 <response error code>		

{
  "error": <high level description>,
  "messages": [{"code" : "<code string>", "msg" : "<error message>"}],
  "data" : null
}

Update details for a testrun - PUT

Summary

Update details for a testrun

PUT /api/v11/reviews/{reviewid}/testruns/{id}

Description

Update the details for a testrun. All values must be provided in the request.

Important

Only admin users can update testrun details for a review.

Parameters

Parameter Description Type Parameter Type Required Restriction

Id

Testrun ID, identifies a specific testrun for the review. Automatically generated by Swarm when the testrun is created.

integer

path

Yes

  

reviewid

Review ID.

integer

path

Yes

  

version

The version of the review the testrun is being run against.

integer

body

Yes

  

test

Specifies the test used for the testrun and checks that it is a valid name.

string

body

Yes

1 to 32 characters

startTime

The time the testrun started, expressed as an epoch value.

integer

body

Yes

> 0

completedTime

The time the testrun completed, expressed as an epoch value.

integer

body

Yes

> 0

status

The status of the testrun, options are: pass, fail, and running

string

body

Yes

Valid options are pass, fail, and running

messages

An array of one or more messages for the testrun. They should be formatted in JSON in the body of the POST request. You can pass a maximum 10 messages, if you provide more than 10 messages only the first 10 are saved. Each message can contain a maximum of 80 characters, any messages with more than 80 characters will be automatically truncated.

array

body

Yes

  

url

The CI system url for the testrun, this is the link to the CI that enables you to view the current testrun results.

string

body

Yes

1 to 1024 characters

uuid

The uuid for the testrun. Used to enable non-admin users to update the testrun. If the requested uuid does not match the review an error message is returned.

string

body

Yes

Minimum 32 max 64 characters

Example usage

Update testrun 706 for review 12345

curl -X PUT -H "Content-Type: application/json" -u "username:ticket" -d "@mybodyfilename.txt" "https://myswarm.url/api/v11/reviews/12345/testruns/706"

The "mybodyfilename.txt" file must contain all of the testrun details:

{
  "id": 706,
  "change": 12345,
  "version": 2,
  "test": "mytest",
  "startTime": 1567895432, 
  "completedTime": 1567895562, 
  "status": "fail",
  "messages": [
    "Test has failed",
    "and another message"
  ],
  "url": "http://my.jenkins.com/projectx/main/1224",
  "uuid": "FAE4501C-E4BC-73E4-A11A-FF710601BC3F"
}

Swarm responds with the testrun details:

HTTP/1.1 200 OK		

{
  "error": null,
  "messages": [],
  "data" : {
    "testruns" : [
      {
        "id": 706,
        "change": 12345,
        "version": 2,
        "test": "mytest",
        "startTime": 1567895432, 
        "completedTime": 1567895562, 
        "status": "fail",
        "messages": [
          "Test has failed",
          "and another message"
        ],
        "url": "http://my.jenkins.com/projectx/main/1224"
      }
    ]
  }
}

If the request fails

HTTP/1.1 <response error code>		

{
  "error": <high level description>,
  "messages": [{"code" : "<code string>", "msg" : "<error message>"}],
  "data" : null
}

Rerun a specific test

Summary

Rerun a specific test.

POST /api/v11/reviews/{reviewid}/testruns/{testrunid}/run

Description

Used to rerun a specific test.

Example usage

Rerun testrun 706 for review 12345

curl -X POST -H "Content-Type: application/json" -u "username:ticket" "https://myswarm.url/api/v11/reviews/12345/testruns/706/run"

Swarm responds with the testrun details:

HTTP/1.1 200 OK		

{
  "error": null,
  "messages": [],
  "data" : {
    "testruns" : [
      {
        "id": 706,
        "change": 12345,
        "version": 2,
        "test": "mytest",
        "startTime": 1567895432, 
        "completedTime": 1567895562, 
        "status": "running",
        "messages": null,
        "url": "http://my.jenkins.com/projecty/main/1224"
        "title": "Swarm-main-codeSniff"
      }
    ]
  }
}

If the request fails

HTTP/1.1 <response error code>		

{
  "error": <high level description>,
  "messages": [{"code" : "<code string>", "msg" : "<error message>"}],
  "data" : null
}