GitHub Actions makes it easy for you to run automated tests on your code. Whether you store your tests in the same repository as your application code or use a separate repo for your automation suite, you can configure GitHub Actions workflows to build your app, perform tests on it, and report the results to a test management tool like Testmo.

In this guide, we will walk through a detailed example of how you can trigger test automation workflows in GitHub Actions directly from Testmo. Launching automation jobs from Testmo gives you control over when you want to launch your test automation suite, so you don't have to waste time switching tools or waiting on other teams. Besides improving efficiency, automation launching also gives you end-to-end visibility over all of your testing activities, including your manual, exploratory, and automated tests.

If you are not yet familiar with basic test automation with GitHub Actions pipelines or how to run automated tests, we have  created separate detailed guides on this. We recommend reading these articles first if the concepts are completely new to you, as we will not cover the basics in this article:

• GitHub Actions Parallel Test Automation Jobs
• GitHub Actions Test Automation Setup & Reporting

Setting Up Your GitHub Actions Workflow

For this article, we have set up a public GitHub repository with an example project that builds on our previous GitHub Actions Parallel Test Automation Jobs example. You can use the files in this repo to follow along with the guide on your own system!

Before we can trigger GitHub Actions workflows from Testmo, we first need to configure them in GitHub. In our example, we have set up our pipeline to execute the following jobs:

1. build - a dummy code build step, for demonstration purposes
2. test-setup - creates an automation run in Testmo, ready for reporting results to
3. test - executes the tests in GitHub Actions, in parallel threads
4. test-complete - submits the test results to Testmo
5. deploy - deploys the dummy code to our specified target environment, with the requested dataset

In addition to running a basic test automation workflow, we made a few additions to the build.yml file in our GitHub Actions workflow to accommodate how we expect to use our pipeline for the following scenario:

As a tester
I want to build and deploy code to a specified environment on demand, from Testmo
And include a testing dataset
So I can perform my testing on a smoke-tested and pre-staged environment

You can see our build.yml file below, which includes addition of inputs we would like to use in when our GitHub Actions workflow runs:

The inputs tell GitHub Actions that an environment must be specified, and provides a default value of staging, and defines the other valid option as production. We also specify that a dataset value may be provided as an input, but is not required. Valid inputs for the dataset option are small or large.

Once the repository has been initialized with the workflow and the necessary testing files have been added (explained in our guide on parallel testing in GitHub), you will see that running the workflow will request you to specify values for the inputs we just defined:

Finally, we should add the TESTMO_URL & TESTMO_TOKEN secrets to our GitHub project settings, since the test-setup and test-completesteps will require those in order to create our run in Testmo and to submit the test results. 

You can configure these by going to your repository settings (or organization settings) and adding two new secrets called TESTMO_URL and TESTMO_TOKEN with the relevant values. You can then tell GitHub Actions to make these secrets available as environment variables, to be used in the workflow file:

You can now test your GitHub Actions workflow by launching it from the GitHub interface. As long as it completes successfully, you’ll be rewarded with a view like the one below. All the steps are green! Now you’re ready to launch the workflow from Testmo.

Configuring Testmo to Point to Your GitHub Actions Workflow

Now that you have set up your GitHub Actions workflow to run some demo automated tests and report the results to Testmo, let's configure our automation launching settings in Testmo.

Connect Testmo to GitHub

The next step is to connect Testmo to your GitHub account, if you haven't done so already. The Testmo GitHub integration relies on a secure OAuth-based connection, which you can enable via the Admin > Integration area in your Testmo account. To see how set up a connection from Testmo to your GitHub account, you can follow our documentation.

Once you have successfully connected Testmo and GitHub, come back to this article and continue to the next step!

Set Up a New Automation Target in Testmo

With the GitHub integration completed and connection successful, we can add our GitHub Actions workflow as an automation target in Testmo. Automation targets give you a way to specify pipelines, branches, or tags that you would like to be able to trigger ahead of time. While setting up automation targets, you can also include unique parameters, inputs, or variables to trigger pipelines for particular environments, datasets, etc.

For this example, you can set up your automation target in Testmo by following the steps below:

• In the Admin > Automation view in Testmo, click on the Add Target button.
• In the Add target dialogue:
  • Name: Give your target a suitably descriptive name. To reflect our scenario, let's enter something like “smoke test & deploy code with data”.
  • Connection: Select the GitHub connection you just configured in the previous step as the CI/CD connection.
  • Settings > Repository: Add the Repository path from GitHub. The account name and repository in which you’ve implemented the workflow example from above (for example: “yourname/example-automation-launching”).
  • Settings > Workflow: Add the Workflow ID from GitHub. Use the full name of the workflow file including the .yml suffix. For our scenario, it should be “build.yml”.
  • Settings > Reference: Add the Reference branch or tag name to be used in the pipeline. We just have a main branch in the example above, so we can just add “main” here.
  • Inputs: Add any inputs you wish to supply to GitHub as part of the workflow. In order to satisfy our environment and data requirements, we can specify the inputs to the workflow here by adding them to our target:
    • dataset = small (or large)
    • environment = staging (or production)
• Once you’ve added all the settings & inputs, then click Add target save the target.

As an additional step, you may wish to constrain the target to only a specific project(s), preventing it from being triggered from unauthorized projects. If so, select the desired projects from the Projects tab in the Add target dialogue.

Triggering the GitHub Actions Workflow from Testmo

Once you’ve added an automation target for your GitHub Actions workflow, you can launch it from Testmo using an Automation job. Simply navigate to the Automation > Jobs view in any project where you enabled your GitHub workflow as an automation target and click the Add Job button. Choose the target you just configured, verify the settings, and click Add job:

Testmo will queue your job for execution, and once it triggers your GitHub Actions worklow, let you know whether the workflow was launched successfully, or not. 

• If it the job launched successfully, you’ll see a blue cog. Clicking on the link will open the workflow page directly in GitHub, so you can see the progress and overall outcome of the jobs running in the workflow there.
• If the job wasn’t triggered successfully, you’ll see an yellow exclamation mark instead. Clicking on the details link will provide a summary of the failure reason. Often the failure will be down to a misconfiguration of the target, or a missing or unexpected input. Our FAQ’s page contains helpful advice for troubleshooting failures.

Viewing Automated Test Results in Testmo

Assuming the job was executed successfully, your tests will also have been run and the results will be reported to the specified Testmo instance (remember the TESTMO_URL & TESTMO_TOKEN secrets from earlier!) for review. You’ll be able to see what happened with them in detail in the Automation > Runs & Results view.

By drilling down to the specific run, you’ll also be able to see valuable information about the test executions like how long they took to run, which thread contained what tests, what the slowest tests were, whether there were any failures, what CI job generated the tests, and much more besides. You can also download a report of your test results and share it with interested stakeholders by simply hitting the print button. 

Conclusion

Success! We’ve set up a GitHub Actions workflow so that it builds our code, runs some smoke tests against it, sends the results to Testmo and deploys the code to our specified environment with the desired dataset. We’ve also integrated GitHub with Testmo so that the workflow can be launched on-demand from Testmo, giving you the power to launch your automated tests at any time, right from Testmo.

If you want to learn more about using Testmo with GitHub and test automation tools such as Selenium, we have lots of other articles you can read too: 

• Product: GitHub integration features
• Guide: GitHub Actions Test Automation & Reporting
• Guide: GitHub Actions Parallel Test Automation Jobs
• Guide: Complete GitHub Actions & Selenium Guide
• Code: GitHub Actions test automation example
• Code: GitHub Actions parallel test automation example
• Code: GitHub Actions Selenium test automation example
• Docs: Testmo CI/CD integration
• Docs: Testmo CLI tool
• Docs: Testmo test automation