SWMM5 GitHub Actions Workflow

 Below is a step‐by‐step explanation of how this GitHub Actions workflow is structured and what each section does. This workflow has two jobs:

1. unit_test: Builds the project with tests enabled and then runs unit tests.
2. reg_test: Builds the project again (with a different method) and then runs regression tests using nrtest scripts.

It also demonstrates secrets usage, environment variables, artifact uploads, and different shell/working directory configurations.

---

1. High-Level Workflow Definition

name: Build and Test

on:
  push:
    branches: [ master, develop, release ]
  pull_request:
    branches: [ master, develop, release ]

• name: Build and Test: The workflow is named “Build and Test.”
• on:: The workflow triggers on:
  • push events and pull_request events targeting branches named master, develop, or release.
• This ensures that whenever someone pushes code or opens a PR against one of those branches, the workflow is kicked off.

---

2. Environment Variables

env:
  OMP_NUM_THREADS: 1
  BUILD_HOME: build
  TEST_HOME: nrtests
  PACKAGE_NAME:  vcpkg-export-20240724-151754.1.0.0
  PKG_NAME: vcpkg-export-20240724-151754

• These environment variables are global to the entire workflow, meaning all jobs can reference them. Examples:
  • OMP_NUM_THREADS: 1: Restricts OpenMP parallel to one thread.
  • BUILD_HOME: path or name for build output.
  • PACKAGE_NAME and PKG_NAME: used when installing packages from a private NuGet source.

---

3. Jobs Overview

The YAML defines two jobs: unit_test and reg_test. Each job:

1. Runs on windows-2019 (runs-on: windows-2019).
2. Has steps that check out code, build, and test in distinct ways.

3.1 unit_test Job

  unit_test:
    name: Build and unit test
    runs-on: windows-2019
    environment: testing
    defaults:
      run:
        shell: cmd
    steps: ...

• unit_test runs on a Windows 2019 runner.
• The environment: testing key can place the job into a specific environment named “testing,” if configured in your repository’s environments.
• defaults.run.shell: cmd means all steps default to a CMD shell instead of PowerShell.

3.1.1 Steps in unit_test

1. Checkout the repository:

   - name: Checkout repo
     uses: actions/checkout@v4
   

   • Grabs the code from the current repository/branch.

2. Install boost-test via NuGet:

   - name: Install boost-test
     env:
       REMOTE_STORE: "https://nuget.pkg.github.com/USEPA/index.json"
       USERNAME: michaeltryby
     run: |
       nuget sources add -Name github -Source ${{ env.REMOTE_STORE }} -Username ${{ env.USERNAME }} -Password ${{ secrets.ACCESS_TOKEN }}
       nuget install ${{env.PKG_NAME}} -Source github
   

   • Adds a custom NuGet feed named “github.”
   • Installs a package identified by $env.PKG_NAME.
   • Uses secrets.ACCESS_TOKEN for authentication.

3. Build:

   - name: Build
     env:
       TOOL_CHAIN_PATH: \scripts\buildsystems\vcpkg.cmake
     run: |
       cmake -B.\build -DBUILD_TESTS=ON -DCMAKE_TOOLCHAIN_FILE=.\${{env.PACKAGE_NAME}}${{env.TOOL_CHAIN_PATH}} .
       cmake --build .\build --config DEBUG
   

   • Sets an environment variable TOOL_CHAIN_PATH which is appended to PACKAGE_NAME.
   • Uses CMake out-of-source build:
     • cmake -B.\build ... . to generate the build system in a .\build folder, enabling tests.
     • cmake --build .\build --config DEBUG to actually compile in Debug config.

4. Unit Test:

   - name: Unit Test
     run: ctest --test-dir .\build -C Debug --output-on-failure
   

   • Runs CTest tests located in .\build.
   • --output-on-failure shows failing test output.

This job thus compiles the code, runs the unit tests, and outputs any errors directly.

---

3.2 reg_test Job

  reg_test:
    name: Build and reg test
    runs-on: windows-2019
    defaults:
      run:
        shell: cmd
        working-directory: ci-tools/windows
    steps: ...

• Also uses a Windows 2019 runner.
• Defaults for all steps to run cmd shell and use working-directory: ci-tools/windows, meaning every step’s starting directory is ci-tools/windows (unless changed).

3.2.1 Steps in reg_test

1. Checkout swmm repo:

   - name: Checkout swmm repo
     uses: actions/checkout@v4
   

   • Brings main repository code into the default GITHUB_WORKSPACE folder.
   • Because working-directory is set to ci-tools/windows, the actual location might be relative to that. Typically the top-level code is in a sibling directory.

2. Checkout ci-tools repo:

   - name: Checkout ci-tools repo
     uses: actions/checkout@v4
     with:
       repository: USEPA/swmm-ci-tools
       ref: master
       path: ci-tools
   

   • Pulls a second repository USEPA/swmm-ci-tools into a folder named ci-tools/.
   • The code uses these scripts for regression tests.

3. Setup python:

   - name: Setup python
     uses: actions/setup-python@v5
     with:
       python-version: '3.11'
   

   • Installs Python 3.11 so scripts can run.

4. Install requirements:

   - name: Install requirements
     run: |
       python -m pip install --upgrade pip
       python -m pip install -r requirements-swmm.txt
   

   • Installs packages needed for the regression test environment from a requirements-swmm.txt.

5. Build:

   - name: Build
     run: make.cmd /g "Visual Studio 16 2019"
   

   • Calls a custom script make.cmd in ci-tools/windows. Possibly calls MSBuild or runs the solution build for SWMM with Visual Studio 2019.

6. Before reg test:

   - name: Before reg test
     env:
       NRTESTS_URL: https://github.com/USEPA/swmm-nrtestsuite
       BENCHMARK_TAG: v2.5.0-dev
     run: before-nrtest.cmd ${{ env.BENCHMARK_TAG }}
   

   • Possibly clones or updates an nrtests suite from GitHub, checking out a benchmark data set labeled v2.5.0-dev. The before-nrtest.cmd script presumably sets up the environment for regression tests.

7. Run reg test:

   - name: Run reg test
     run: run-nrtests.cmd %GITHUB_RUN_ID%_%GITHUB_RUN_NUMBER%
   

   • Calls run-nrtests.cmd to run regression tests, using the run ID plus run number to label results.
   • nrtest typically compares SWMM’s output to baseline reference results for various test models.

8. Upload artifacts:

   - name: Upload artifacts
     if: ${{ always() }}
     uses: actions/upload-artifact@v4
     with:
       name: build-test-artifacts
       path: upload/*.*
   

   • If the job completes (success or fail, thanks to always()), it uploads everything matching upload/*.* as an artifact named build-test-artifacts.
   • This typically includes logs, test reports, or other results.

---

4. Execution Flow & Key Points

1. Branch triggers: Only on master, develop, or release branches for push/pull_request.
2. Two jobs run in parallel by default:
   • unit_test: Focused on building and running unit tests (CTests).
   • reg_test: Focused on building in a different manner (via make.cmd) and running regression tests (via nrtest).
3. NuGet references**: This workflow fetches a custom package from a GitHub NuGet feed, presumably containing dependencies like boost-test.
4. Under the hood: The make.cmd, before-nrtest.cmd, and run-nrtests.cmd scripts do more specialized tasks (e.g., building via MSBuild, setting up test data, comparing outputs).

---

5. Summary

• unit_test job:
  • Uses CMake + a custom vcpkg toolchain to build the code in Debug mode.
  • Runs CTest for unit tests.
• reg_test job:
  • Uses custom scripts in ci-tools/windows for building and then runs nrtest for regression comparisons.
  • Uploads test artifacts for further inspection.

This approach separates quick unit testing from heavier regression checks, ensuring thorough coverage of the SWMM codebase. By splitting them into two jobs, they can run in parallel, speeding up the overall CI process.