In my previous post, I introduced how our new Workflows capability works under the hood, and how to get started by configuring your first CI pipeline from scratch, or importing existing CI pipelines.  In the next series of posts, I’m going to dive into how to get started with Workflows for your specific CI tools––namely GitHub Actions, GitLab CI/CD, and Azure DevOps.

In this post we will learn how to configure GitHub Actions to work with Firefly’s Workflows.  Below you will find a sample repository that is structured to manage AWS infrastructure using Terraform, specifically targeting a staging environment. 

This is the concise overview of the structure:

root ├── .github │ └── workflows │ └── deploy-aws-stage.yml ├── aws-stage │ ├── main.tf │ ├── variables.tf │ ├── outputs.tf │ ├── providers.tf │ └── modules │ └── example-module │ ├── main.tf │ ├── variables.tf │ └── outputs.tf └── README.md

.github/workflows/deploy-aws-stage.yml contains the GitHub Actions workflow configuration for deploying the Terraform workspace in the aws-stage directory.

We will walk through in the examples below how to create your workflow in GitHub Actions, and import them to your Firefly dashboard.

Workflow Trigger

We will start with the workflow trigger code as follows:

"on": push: branches: - main paths: - aws-stage/** pull_request: branches: - main paths: - aws-stage/** workflow_dispatch: inputs: apply: description: Should apply Terraform changes? type: boolean default: false

This configuration ensures that the workflow only runs when relevant changes are made to the aws-stage directory, minimizing unnecessary executions. It supports:

  1. Automatic execution on pushes and pull requests to the main branch, ensuring that changes are validated before merging.
  2. Manual execution via workflow_dispatch for flexibility, allowing team members to run the workflow on demand with specific parameters.

Jobs Configuration

Next we’ll configure the “deploy” job, see the code below for a sample of how to do so:

jobs: deploy: name: Deploy Terraform Workspace runs-on: ubuntu-latest permissions: contents: read id-token: write defaults: run: working-directory: aws-stage env: FIREFLY_ACCESS_KEY: ${{ secrets.FIREFLY_ACCESS_KEY }}

This job runs on an ubuntu-latest GitHub-hosted runner, which provides a clean, consistent environment for running CI tasks. The job requires:

  1. contents: read permission to read the repository content.
  2. id-token: write permission to obtain a web identity token for authenticating with AWS services.

Setting the default working directory to aws-stage ensures that all commands are executed in the correct directory. Environment variables for Firefly access keys are securely injected from repository secrets.


Steps - Check-out Repository

Once we have configured the “deploy” job configuration, we will set the job steps. The first step is to check-out the repository:

steps: - name: Check-out Repository uses: actions/checkout@v4 with: fetch-depth: 0

The actions/checkout@v4 action fetches the repository's code. Using fetch-depth: 0 fetches the entire history, which is useful for:

  1. Accessing commit history for commands that might need it (e.g., determining differences between commits).
  2. Ensuring that any Git operations that depend on the full history can function correctly.
  3. Firefly CI may utilize this complete commit history to fetch the changed code and run scans on it.

Steps - Set Deployment Parameters

Next, we will set the deployment parameters, as configured in the code below:

- name: Set deployment parameters id: set-params run: |- if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then echo "apply=${{ github.event.inputs.apply }}" >> $GITHUB_OUTPUT elif [ "${{ github.event_name }}" = "push" ] && [ "${{ github.ref }}" = "refs/heads/main" ]; then echo "apply=true" >> $GITHUB_OUTPUT else echo "apply=false" >> $GITHUB_OUTPUT

This step sets a parameter to determine whether Terraform should apply changes. To make this determination, it considers:

  1. The manual triggers (workflow_dispatch) and uses the apply input to decide.
  2. The automatic execution on pushes to main, setting apply to true for automatic deployment.
  3. Checks whether the default to apply is false for other scenarios, avoiding unintended deployments.


Steps - Get Web Identity Token

This next step retrieves a web identity token from GitHub's OIDC provider. This token is used to authenticate with AWS, enabling secure, short-lived access without needing static AWS credentials in the workflow. This is how to set the identity token for retrieval:

- name: Get Web Identity Token run: 'curl -s -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" "$ACTIONS_ID_TOKEN_REQUEST_URL&audience=sts.amazonaws.com" | jq -r ''.value'' > oidc'

Steps - Setup Terraform

Next we need to configure the Terraform setup:

- name: Setup Terraform uses: hashicorp/setup-terraform@v3 with: terraform_version: 1.8.1 terraform_wrapper: false

The hashicorp/setup-terraform@v3 action sets up Terraform:

  1. Where terraform_version: 1.8.1 was defined, is where you can ensure a specific version of Terraform is used, providing consistency and avoiding issues with version discrepancies.
  2. terraform_wrapper: false disables the Terraform wrapper. This is done to avoid any additional layers that might interfere with the Terraform commands and to ensure native Terraform behavior, especially important for debugging and running commands exactly as intended.

Steps - Terraform Init

This step initializes the Terraform configuration, setting up backend configurations and downloading necessary modules. The output is logged to init.log:

- name: Terraform Init id: terraform-init run: terraform init >& init.log continue-on-error: true

NOTE: The continue-on-error: true allows the workflow to capture the log even if initialization fails, which will be sent to Firefly for troubleshooting.

Steps - Terraform Plan

If initialization is successful, this step generates a Terraform plan, which outlines the changes Terraform will make. The plan is output in JSON and raw formats for detailed analysis. This step also continues on error to ensure logs are sent to Firefly for troubleshooting:

- name: Terraform Plan id: terraform-plan if: steps.terraform-init.outcome == 'success' run: terraform plan -json -out=tf.plan > plan_log.jsonl && terraform show -json tf.plan > plan_output.json && terraform show tf.plan > plan_output_raw.log continue-on-error: true

Steps - Firefly Post Plan

This step sends the Terraform plan and init outputs to Firefly CI for further analysis. Firefly CI runs scanners like KICS (which checks for security vulnerabilities) and Infracost (which estimates the cost of infrastructure changes), and visualizes the results. This integration provides additional insights into the planned changes, helping to identify potential issues and optimize costs:

- name: Firefly Post Plan uses: gofireflyio/fireflyci@v0.4.1 with: command: post-plan context: aws-stage init-log-file: init.log plan-json-log-file: plan_log.jsonl plan-output-file: plan_output.json plan-raw-log-file: plan_output_raw.log workspace: aws-stage continue-on-error: true

Steps - Terraform Apply

If the apply parameter is true and the plan was successful, this step applies the Terraform changes. The -auto-approve flag is used to bypass manual approval, and the output is logged in JSON format for detailed tracking. continue-on-error ensures that logs are captured even if the apply step encounters issues to ensure logs are sent to Firefly for troubleshooting:

- name: Terraform Apply if: steps.set-params.outputs.apply == 'true' && steps.terraform-plan.outcome == 'success' run: terraform apply -auto-approve -json apply_log.jsonl continue-on-error: true

Steps - Firefly Post Apply

This step sends the apply output to Firefly CI, allowing further analysis and visualization. This helps ensure that any issues during the apply phase are captured and analyzed, providing comprehensive insights into the deployment process:

- name: Firefly Post Apply if: steps.set-params.outputs.apply == 'true' && steps.terraform-plan.outcome == 'success' uses: gofireflyio/fireflyci@v0.4.1 with: apply-log-file: apply_log.jsonl command: post-apply context: aws-stage workspace: aws-stage continue-on-error: true

Firefly CI Integration

Firefly CI integration enhances the workflow by:

  1. Sending Terraform plan and apply outputs to the Firefly server.
  2. Running scanners like KICS for security checks and Infracost for cost management.
  3. Visualizing results to provide insights into changed assets, policy violations, compliance issues, and cost implications.

Adding Firefly Credentials as Repository Secrets

To securely add Firefly credentials:

  1. Navigate to the GitHub repository.
  2. Go to Settings > Secrets and variables > Actions.
  3. Click New repository secret.
  4. Add FIREFLY_ACCESS_KEY and FIREFLY_SECRET_KEY with the respective values.

Using repository secrets ensures that sensitive information, such as Firefly access keys, are securely stored and only accessible during workflow execution, adhering to best practices for security and automation.

By combining Terraform with GitHub Actions and Firefly CI, this workflow provides a robust, automated process for managing infrastructure as code, enhancing security, cost management, and compliance through continuous integration and detailed analysis.

Full Workflow Code

Now to piece everything together into a single cohesive block of code, below is the full end-to-end Workflows code example for importing your GitHub Actions into Firefly, that was walked through, step-by-step, above:

name: Deploy aws-stage Terraform Workspace "on": push: branches: - main paths: - aws-stage/** pull_request: branches: - main paths: - aws-stage/** workflow_dispatch: inputs: apply: description: Should apply Terraform changes? type: boolean default: false jobs: deploy: name: Deploy Terraform Workspace runs-on: ubuntu-latest permissions: contents: read id-token: write defaults: run: working-directory: aws-stage env: FIREFLY_ACCESS_KEY: ${{ secrets.FIREFLY_ACCESS_KEY }} FIREFLY_SECRET_KEY: ${{ secrets.FIREFLY_SECRET_KEY }} steps: - name: Check-out Repository uses: actions/checkout@v4 with: fetch-depth: 0 - name: Set deployment parameters id: set-params run: |- if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then echo "apply=${{ github.event.inputs.apply }}" >> $GITHUB_OUTPUT elif [ "${{ github.event_name }}" = "push" ] && [ "${{ github.ref }}" = "refs/heads/main" ]; then echo "apply=true" >> $GITHUB_OUTPUT else echo "apply=false" >> $GITHUB_OUTPUT fi - name: Get Web Identity Token run: 'curl -s -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" "$ACTIONS_ID_TOKEN_REQUEST_URL&audience=sts.amazonaws.com" | jq -r ''.value'' > oidc' - name: Setup Terraform uses: hashicorp/setup-terraform@v3 with: terraform_version: 1.8.1 terraform_wrapper: false - name: Terraform Init id: terraform-init run: terraform init >& init.log continue-on-error: true - name: Terraform Plan id: terraform-plan if: steps.terraform-init.outcome == 'success' run: terraform plan -json -out=tf.plan > plan_log.jsonl && terraform show -json tf.plan > plan_output.json && terraform show tf.plan > plan_output_raw.log continue-on-error: true - name: Firefly Post Plan uses: gofireflyio/fireflyci@v0.4.1 with: command: post-plan context: aws-stage init-log-file: init.log plan-json-log-file: plan_log.jsonl plan-output-file: plan_output.json plan-raw-log-file: plan_output_raw.log workspace: aws-stage continue-on-error: true - name: Terraform Apply if: steps.set-params.outputs.apply == 'true' && steps.terraform-plan.outcome == 'success' run: terraform apply -auto-approve -json > apply_log.jsonl continue-on-error: true - name: Firefly Post Apply if: steps.set-params.outputs.apply == 'true' && steps.terraform-plan.outcome == 'success' uses: gofireflyio/fireflyci@v0.4.1 with: apply-log-file: apply_log.jsonl command: post-apply context: aws-stage workspace: aws-stage continue-on-error: true

In our upcoming posts, we will walk through how to set up your GitLab CI/CD and Azure DevOps to manage your CI Pipelines and Workflows alongside all of your cloud assets in the Firefly platform.

Check it out and get started here.

‍