Testing GitHub Actions...With Actions

There are plenty of useful blog posts on how to write a GitHub Action, but while writing my own Action, I didn’t find much material out there on how to help me properly test it. In this blog post, I want to talk about how I implemented automated testing of my Action, hoping that you can benefit from this as well.

Project Structure

Part of the reason you use the GitHub Actions offering is likely to perform automated testing within your project. You can use this same idea for testing GitHub Actions. Essentially, you can use test your own GitHub Actions with Actions!

Here’s the project structure of the GitHub Action that I recently built (files irrelevant to this topic are left out):

.github/workflows/
test/
Dockerfile
action.yml
entrypoint.sh

The files that compose of my Action are Dockerfile, action.yml, and entrypoint.sh. But, notice that I also have the .github/workflows/ folder for creating automated workflows. I also have a separate test/ folder for saving assets related to testing. We’ll get to each of these. First, let’s discuss the .github/workflows/ folder.

The .github/workflows/ Folder

In my Action, I only have one file under .github/workflows/ called test.yml. This file is likely all you need for testing. Inside test.yml are my different test cases. Let’s take a look at this file and one of these test cases.

The test.yml File

Here’s a short snippet of my test.yml file:

name: Test Action
on: push

jobs:
  test-action:
    runs-on: ubuntu-latest
    steps:
      ## Check out this action so we can test it
      - uses: actions/checkout@v2

      ## Close open PRs from deweya:feat/sync before continuing
      - name: Close existing PRs
        run: test/close-prs.sh deweya0/helm-charts deweya:feat/sync ${{ secrets.PAT }}

      ## Test sample PR with no change to charts
      - name: No change to charts
        uses: ./
        with:
          auth_token: ${{ secrets.PAT }}
          fork_name: deweya/helm-charts
          upstream_owner: deweya0
          committer_email: deweya964@gmail.com
          local_charts_dir: test/00-test-no-change
      - name: Verify no change
        run: test/00-verify-no-change.sh master deweya0/helm-charts deweya:feat/sync ${{ secrets.PAT }}

The test workflow for my Action begins on line 8. The first thing it does is check out my Action’s code from the branch that I pushed to. Once the Action is checked out, you can start doing some testing. I designed my Action to help with automating PRs, so the first thing my workflow does is close any existing PRs so testing can start with a clean slate.

I wrote my first real test case on lines 15-25. Don’t worry about understanding what it is that I’m testing. I’ll conceptualize it enough so that you get the gist.

First, I gave my test case a name, “No change to charts”. Next, I set uses to “./”. Usually, you’d expect to see “uses” following an “owner/repo@ref” format, but since we want to test the Action that we just checked out, you’ll want to refer to it locally as “./”.

For with, I passed a few parameters that my Action needs for this particular test case.

Then, on line 24 I added another step called “Verify no change” to verify that my Action successfully carried out its task. While the previous step told my Action to run some task, this next step verifies that it performed the task the way I expected.

Notice on line 25 that I have a run instruction to execute a shell script under the test/ folder. Let’s discuss this in detail in the next section.

The test/ Folder

The test/ folder is the folder I used to keep any assets related to testing, such as bash scripts. I wrote my scripts here and then referred to them in my workflow to verify that my Action is working correctly. I got this idea by reviewing the actions/checkout repo, where they do the same thing under a __test__/ folder. Of course, you can name this folder anything you want, as long as you refer to that same name in your workflow.

In my bash scripts, I like to make sure I leave good error or info messages depending on my test’s status. For example, here’s what my 00-verify-no-change.sh script looks like:

#!/bin/bash

export BASE=$1
export UPSTREAM_REPO=$2
export HEAD=$3
export GITHUB_TOKEN=$4

gh pr list --state open --base $BASE --repo $UPSTREAM_REPO | grep $HEAD
exit_code=$?
set -e
if [ $exit_code -eq 0 ]; then
 echo "FAIL: PR found, but no PR was expected"
 exit 1
else
 echo "PASS: No PR found"
 exit 0
fi

Don’t try to get too wrapped up in the details of what this script is doing. The key takeaway I want you to understand is on lines 11-17. Based on some condition, the script will either leave a fail message and exit 1 or leave a pass message and exit 0. These messages give other contributors and me a clear and concise message to help understand why a test case either passed or failed, and if it failed, it lets us know what should have happened.

Thanks For Reading

That’s all for this post! I hope this helps you configure automated testing in your own GitHub Actions. The Action I used for this example is located at https://github.com/deweya/chart-repo-pr-action in case you would like to explore my workflow in greater detail. Feel free to leave a comment if you have any questions. Thanks!