Post

Let's try: Github actions for Github integration

With Github Actions, our CI/CD directly connect to Github repo and able to deploy to our GCP services

Let's try: Github actions for Github integration

Speaking of CI/CD tools, I have some old blogs about Google Cloud Build. But today we come to talk about another popular CI/CD tool which comes with Github, that is “Github Actions”.


What is “Github Actions”?

Github Actions is a CI/CD tool comes with Github. When we implemented our own apps in Github repositories, we can use this tool out-of-the-box. Similar to Google Cloud Build, we can control our workflows to run, test, and deploy with simple YAML files.

This is an official page of Github Actions.


Start a first action

Github Actions just needs YAML files inside .github/workflows directory so we’re gonna create a very simple workflow in one YAML file.

1. Create a workflow file

The command below will create an empty file inside that directory.

1
2
mkdir -p .github/workflows
touch sample.yml

So we should have the file in the structure like this.

1
2
3
4
.
└── .github
    └── workflows
        └── sample.yml

2. Simple echo

We edit the content of .github/workflows/sample.yml file to just say “hello world”, like this.

1
2
3
4
5
6
7
8
9
10
11
12
13
name: "Sample workflow"

on:
  workflow_dispatch:

jobs:
  prepare:
    name: "prepare"
    runs-on: ubuntu-latest
    steps:
      - name: simple echo
        run: >
          echo "hello world"
  • name is a name of this Github workflow.
  • on is an event to run this workflow and workflow_dispatch means we can run this workflow manually.
  • jobs is a section we define each of our jobs.
    • prepare is a job name as a local reference.
    • name here is the job name to display in the Github Actions page.
    • runs-on is an OS image to run this job. We can use ubuntu-latest, windows-latest, or macos-latest runners.
    • steps is a list of steps we want to execute in this job.

3. Review the workflow

When we commit and push it to the default branch, we can find the new workflow here at the “Actions” tab of the repository.

action-page

New workflow can be found when the workflow files are ready in the default branch12 or a pull request that have it if the trigger is on pull_request.
Find out the default branch at the “Settings” tab, under “Branches” section.

As we set on event to workflow_dispatch, this means we can run this workflow manually by selecting the branch and clicking the “Run workflow” button.

run wokflow

When the job ran completely (or failed), we can see the status and the job history like this.

run complete

4. Review results

When we clicked the job name, we can see the details of the job.

This job names “prepare” and has only one step, “simple echo” so we can see only one box.

run dag

Clicking the “prepare” box will show the output of the step. There is “hello world” under the “simple echo” step.

dag output


Some more configs

Variables

We can define variables

1
2
env:
  text: "Hello world."

And reference them in the workflow like this.

1
2
3
4
5
6
7
8
9
10
11
env:
  text: "Hello world."

jobs:
  prepare:
    name: "prepare"
    runs-on: ubuntu-latest
    steps:
      - name: simple echo
        run: |
          echo "${{ env.text }}"

We can also embed variables3 in the repo at “Settings” tab → “Security” section → “Secrets and variables” → “Actions”.

Trigger events

1
2
3
4
5
6
7
8
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
  workflow_dispatch: # manual trigger

There are several events to trigger4 the workflow.

  • push event will trigger the workflow when we push to the given branch. It’s main branch in the example above.
  • pull_request event will trigger when we create a pull request to the given branch. It’s also a main branch in the example.
  • workflow_dispatch event to run manually.

Job details

1
2
3
4
5
6
7
8
9
10
11
12
jobs:
  <job_name>:
    name: "<job_name>"
    runs-on: <runner>
    needs: <job_name> # optional, to run after another job
    environment: <environment_name> # optional, to run in a specific environment
    steps:
      - name: "<step_name>"
        run: |
          <commands>
    outputs: # optional, to define outputs for the job
      <output_name>: "<output_value>"

Under jobs, we can define each job and its details5:

  • name: name of the job to display.
  • runs-on: the runner to run this job. It can be ubuntu-latest, windows-latest, or macos-latest.
  • needs: one or more job names to run before this job as dependencies.
  • environment: a specific environment to run this job.
  • steps: a list of steps to run.
    • name: name of the step.
    • run: commands to run in this step.
  • outputs: outputs of the job in case we want to parse to other jobs.

Work with Google Cloud

Now we are at the core. When it comes to CI/CD, we usually have to integrate it with some platforms such as GCP.

In this blog, we are going to implement a simple Github workflow to check Google Cloud Storage files through GCP Workload Identity Federation. We need to understand how can we authenticate and let’s see together.

Workload Identity Federation

Workload Identity Federation6 allows us to securely authenticate to Google Cloud Platform without service account keys or other risky methods if we lose them. Instead, we use short-lived tokens as an identity, including impersonating service accounts.

When the setup is ready, our Github actions can connect to the GCP as this diagram.

sequenceDiagram
  autonumber

  box Google Cloud
    participant WIP as Workload Identity Pool
    participant WIPP as Workload Identity<br/>Pool Provider
    participant SA as GCP Service Account
  end
  
  box Github
    participant GHA as Github Actions
  end

  alt setup
    WIP ->> WIPP : contains
    SA ->> WIPP : binding with principal<br/>or principal set
  end
  GHA ->> SA : uses
    SA ->> WIPP : authorize
  activate WIPP
  WIPP ->> GHA : access token
  deactivate WIPP

There are many ways to setup the GCP Workflow Identity Federation e.g. gcloud CLI7. This blog will use Terraform8 to create the resources.

Create Workflow Identity Federation resources

  1. First, we need a GCP Workload Identity Pool. This resource is currently in google-beta provider.

    1
    2
    3
    4
    5
    6
    7
    
     resource "google_iam_workload_identity_pool" "my_github_pool" {
       provider = google-beta
    
       workload_identity_pool_id = "pool-id"
       display_name              = "pool-name"
       description               = "Identity pool operates in FEDERATION_ONLY mode for testing purposes"
     }
    

    Deleting a pool has 30 days of grace period which means the deleted pool can be restored and we can’t create a new pool with the same name for the certain period9.

  2. Next is the provider in the pool. There are many ways to setup attribute_condition and this time I set the condition to be repo owner10.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    
     resource "google_iam_workload_identity_pool_provider" "my_github_provider" {
       provider = google-beta
    
       workload_identity_pool_id          = google_iam_workload_identity_pool.my_github_pool.workload_identity_pool_id
       workload_identity_pool_provider_id = "provider-id"
       display_name                       = "provider-name"
       description                        = "Provider for GitHub Actions to access Google Cloud resources"
       attribute_condition                = "assertion.repository_owner == 'owner'"
       attribute_mapping = {
         "google.subject"             = "assertion.sub"
         "attribute.actor"            = "assertion.actor"
         "attribute.repository"       = "assertion.repository"
         "attribute.repository_owner" = "assertion.repository_owner"
       }
       oidc {
         issuer_uri = "https://token.actions.githubusercontent.com"
       }
     }
    

    Deleting a provider has 30 days of grace period which means the deleted provider can be restored and we can’t create a new provider with the same name for the certain period11.

  3. Third, create a service account to connect to the provider.

    1
    2
    3
    4
    5
    
     resource "google_service_account" "my_service_account" {
       account_id   = "service-account-id"
       display_name = "service-account-name"
       description  = "Service account for GitHub Actions to access Google Cloud resources"
     }
    
  4. We are binding the service account with the role roles/iam.workloadIdentityUser as a user of the provider.
    members must be either principal or principalSet12 and assign the corresponding principal identifier13. There are some forums1415 in case of the errors from incorrect members.

    1
    2
    3
    4
    5
    
     resource "google_service_account_iam_binding" "github_actions_sa_member" {
       service_account_id = google_service_account.my_service_account.name
       role               = "roles/iam.workloadIdentityUser"
       members            = ["principalSet://iam.googleapis.com/${google_iam_workload_identity_pool.my_github_pool.name}/attribute.repository_owner/owner"]
     }
    
  5. Last, we grant Service Account Token Creator role for impersonation and other necessary roles.

    1
    2
    3
    4
    5
    6
    7
    8
    
     resource "google_project_iam_member" "github_actions_sa_iam_roles" {
       for_each = toset([
         "roles/iam.serviceAccountTokenCreator",
         "roles/storage.admin",
       ])
       role   = each.value
       member = "serviceAccount:${google_service_account.my_service_account.email}"
     }
    

Create Github Actions with GCP provider

Let’s say we have completed the setup and it’s the time to create a Github Actions to do something with GCP services.

  1. prepare credentials

    1
    2
    3
    4
    
     env:
       PROJECT_ID: <project_id>
       WLID_PROVIDER: projects/<project_number>/locations/global/workloadIdentityPools/<pool_id>/providers/<provider_id>
       SA_EMAIL: <service_account>@<project_id>.iam.gserviceaccount.com
    

    We need project id, project number, pool id, provider id, and service account email.

    The project number (e.g. projects/123456789/) is required to authenticate to the provider, or we could see an error when authenticating.

  2. checkout16 first and the workflow can access and fetch the repo.

    1
    2
    3
    
     steps:
       - name: Checkout
         uses: actions/checkout@v3
    
  3. authenticate to GCP17 with the service account.

    1
    2
    3
    4
    5
    6
    
       - name: Authenticate to Google Cloud
         uses: google-github-actions/auth@v2
         with:
           workload_identity_provider: ${{ env.WLID_PROVIDER }}
           project_id: ${{ env.PROJECT_ID }}
           service_account: ${{ env.SA_EMAIL }}
    
  4. setup gcloud18 to configure Google SDK into the Github Actions environment.

    1
    2
    3
    4
    
       - name: Setup GCloud
         uses: google-github-actions/setup-gcloud@v2
         with:
           project_id: ${{ env.PROJECT_ID }}
    

Example result

At the step “List files”, I run gcloud storage ls gs://... and it successfully returns a list of folders and files in that path.

gcp action

Debug OIDC Claims

OIDC or OpenID Connect1920 is a protocol to verify the identity of the user or service. At the step of creating provider, we set OIDC URI to be Github that means Github will send a token to GCP to verify.

However, when we have issues with OIDC claims, we can add this step of “actions-oidc-debugger”21 to debug OIDC claims in order to investigate values and other issues in authentication.

1
2
3
4
  - name: Debug OIDC Claims
    uses: github/actions-oidc-debugger@main
    with:
      audience: "${{ github.server_url }}/${{ github.repository_owner }}"

Repo

I have setup both Terraform and Github Actions in the repo below.


References

  1. Workflow is not shown so I cannot run it manually (Github Actions) - Stack Overflow ↩︎

  2. Running a GitHub Actions workflow that doesn’t exist yet on the default branch - Thomas Levesque’s .NET Blog ↩︎

  3. Store information in variables - GitHub Docs ↩︎

  4. Triggering a workflow - GitHub Docs ↩︎

  5. Workflow syntax for GitHub Actions - GitHub Docs ↩︎

  6. Workload Identity Federation  |  IAM Documentation  |  Google Cloud ↩︎

  7. Secure your use of third party tools with identity federation | Google Cloud Blog ↩︎

  8. google_iam_workload_identity_pool_provider | Resources | hashicorp/google | Terraform | Terraform Registry ↩︎

  9. Delete a pool ↩︎

  10. Define an attribute condition ↩︎

  11. Delete a provider ↩︎

  12. Principal types ↩︎

  13. Allow your external workload to access Google Cloud resources ↩︎

  14. google cloud platform - github actions to GCP OIDC error: 403 ‘Unable to acquire impersonated credentials’ [principalSet mismatch with the Subject claim] - Stack Overflow ↩︎

  15. Github actions ‘Unable to acquire impersonated credentials’ from GCP OIDC: error403 [principalSet mismatch with the Subject claim] · Issue #310 · google-github-actions/auth ↩︎

  16. actions/checkout: Action for checking out a repo ↩︎

  17. google-github-actions/auth: A GitHub Action for authenticating to Google Cloud. ↩︎

  18. google-github-actions/setup-gcloud: A GitHub Action for installing and configuring the gcloud CLI. ↩︎

  19. Configuring OpenID Connect in cloud providers - GitHub Docs ↩︎

  20. Configure Workload Identity Federation with other identity providers  |  IAM Documentation  |  Google Cloud ↩︎

  21. github/actions-oidc-debugger: An Action for printing OIDC claims in GitHub Actions. ↩︎

This post is licensed under CC BY 4.0 by the author.