Introduction¶

This article demonstrates how to run Squish tests using GitHub Actions. We invoke the squishrunner command to execute tests and generate JUnit and web reports.

Additionally, squishrunner is called with a --exitCodeOnFail switch so it returns a custom exit code (non-zero) if any of the test cases have failed, and zero otherwise. Therefore, GitHub Actions can set the job status accordingly.

What is GitHub Actions¶

GitHub Actions is a powerful and flexible automation tool provided by GitHub. It allows you to define custom workflows and automate various tasks directly within your GitHub repository. With GitHub Actions, you can create workflows that respond to events, such as push events (when code is pushed to the repository), pull request events, issue events, and more.

Local GitHub Runner (Agent) Configuration¶

To set up our configuration we should have a repository with our sources in GitHub. Next, we must have a local runner set up ( How to set up a local runner ).

During the runner configuration process, it is required to select the way the runner is started: as a .cmd process or as a service. We need to choose the .cmd process option (Input n or Enter). The service option will NOT work because Windows services do not have access to the GUI, and, therefore, all our tests will fail.

Pipeline Configuration¶

The behavior of the CI/CD pipeline in GitHub Actions is described in the .yml file located in /.github/workflows/ folder.

In order to run Squish tests, we set the content of the configuration file as following:

name: GitHub Actions Demo
run-name: ${{ github.actor }} is running addressbook tests
on: [push]
jobs:
  Run-Squish-Addressbook-Tests:
    runs-on: self-hosted
    steps:
      - name: Checkout repository
        uses: actions/checkout@v1

      - name: Run addressbook tests
        run: D:\Squish/7.1/Squish_7.1.0_for_Qt_5.11/bin/squishrunner.exe --testsuite suite_addressbook_tests --local --exitCodeOnFail 13 --reportgen junit,reports/xml/junit_report.xml --reportgen html,reports/web_report
        shell: cmd

      - name: Publish Test Report
        uses: mikepenz/action-junit-report@v3
        if: success() || failure()
        with:
          report_paths: '**/reports/xml/*.xml'
          
      - name: Upload Artifacts
        if: success() || failure()
        uses: actions/upload-artifact@v2
        with:
          name: My Artifacts
          path: |
            ${{ github.workspace }}/reports/

This job performs the following actions:

1. Sets execution of this job to our self-hosted runner.
2. Performs checkout of source code.
3. Runs Squish tests, sets --exitCodeOnFail 13 setting and generates a JUnit report and HTML report.
4. Stores artifacts (files produced by report generation).
5. Stores test results (JUnit reports)

Result¶

Now, when the CI/CD pipeline is successfully set up, GitHub will run our tests on each push that is made to this repository. The results can be viewed in the 'Actions' tab on the GitHub web page:

JUnit Test Report and Artifacts are located here as well:

• integration