DevOps at BYU
Overview
Welcome to the home of DevOps documentation at BYU. Currently, this page is split into the following three parts:
- DevOps Process: The organizational structure for DevOps in BYU’s OIT.
- GitHub Actions: The recommended tool for running continuous integration and continuous deployment (CI/CD) operations in the BYU ecosystem.
- Terraform: The recommended infrastructure-as-code (IAC) tool in the BYU ecosystem.
Assign Secret to Repository
For those using GitHub Actions: this order form can be used to give specific GitHub repos access to specific AWS accounts.
“Hello World” Project Templates
If you are looking for examples of DevOps using IAC and CI/CD tools, start here. These templates show best practices for setting up your project repos to take advantage of DevOps practices (IAC, CI/CD, etc.). You can create your project repos directly from these templates.
Security Note: When developing custom GitHub Actions, do NOT set any sensitive secrets as environment variables from your custom action. Any following action/step has access to any and all environment variables that have been set prior. If you need to pass sensitive data from your custom action, then export them as outputs.
DevOps Processes
Application Engineering
The Application Engineering department in BYU’s Office of IT is lead by Mark Epling.
Development stack definitions and best practices can be found here.
GitHub Actions
GitHub Actions is the recommended tool for automating continuous integration and continuous deployment (CI/CD) operations and workflows. Using CI/CD workflows helps automate tasks during the development process such as building code, running tests, and deploying code changes. You can think of GitHub Actions as OIT’s successor to handel-codepipeline.
These actions simplify and standardize common CI/CD tasks. You reference the actions from .yml files inside your project repo via versioned name.
CodeDeploy Action
https://github.com/byu-oit/github-action-codedeploy/
A GitHub Action for deploying an application with AWS CodeDeploy.
Create ECR Action
https://github.com/byu-oit/github-action-create-ecr-repo-if-missing/
A GitHub Action for creating AWS ECR repositories.
Disallow Concurrent Runs Action
https://github.com/byu-oit/github-action-disallow-concurrent-runs/
A GitHub Action for disallowing concurrent workflow runs. Unfortunately, GitHub Actions does not currently have a way to queue workflows. This action works around this limitation by failing if the given workflow is already running for a particular branch.
End Standard Change Action
https://github.com/byu-oit/github-action-end-standard-change/
A GitHub Action for ending standard change RFCs in BYU’s ServiceNow system.
Setup Maven Action
https://github.com/byu-oit/github-action-setup-maven/
A GitHub Action for setting up a workflow to use BYU’s maven environment. This action will create the maven settings.xml file along with the security-settings.xml file to allow your workflow to access the BYU maven repository. This action does not install maven, nor java (both are already included in workflows by default). If you need to install a different version of java you can use the setup-java action.
Slack Action
https://github.com/byu-oit/github-action-slack/
A GitHub Action for sending Slack notifications.
Start Standard Change Action
https://github.com/byu-oit/github-action-start-standard-change/
A GitHub Action for starting standard change RFCs in BYU’s ServiceNow system.
Terraform Plan Comment Action
https://github.com/byu-oit/github-action-tf-plan-comment
A GitHub Action for making a comment on PRs with a summary of a terraform plan to be executed if merged.
Terraform Plan Analyzer Action
https://github.com/byu-oit/github-action-tf-plan-analyzer
A GitHub Action to analyze the proposed Terraform plan for security and compliance.
Terraform
About
What is Terraform?
For our purposes, Terraform is a declarative way to provision AWS resources. It can be used for infrastructure or deployable applications.
To use Hashicorp’s own description, Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
You can think of Terraform as OIT’s successor to handel. It’s more powerful and more flexible, but it requires some additional knowledge about the way AWS resources operate.
Getting Started
Requirements
Try out Terraform
Follow the steps at byu-oit/hw-lambda-api or byu-oit/hw-fargate-api to create and deploy a simple application.
See Examples
Discover Modules
Standards and Best Practices
S3 Backend
Use the S3 Backend module to deploy the Backend resources (S3 Bucket and Dynamo Table) into your account. This only needs to be done once per account.
In each project’s main.tf, include the following block at the top:
GitHub Conventions
Terraform Modules are in public GitHub repositories using the naming pattern terraform-
. They should be created from the terraform module template and have the terraform-module GitHub topic.
Deployable Terraform applications should be in private GitHub repositories. They should have the terraform GitHub topic.
Terraform Modules
These modules simplify and standardize the creation of application infrastructure. You reference these modules from .tf files inside your project repo via versioned URL.
Module Structure
Our terraform modules follow (for the most part) the module structure defined by Terraform. See the terraform module template for the basic structure of our terraform modules.
When using a terraform module you specify a source which tells Terraform where to find the source code for the module.
We have opted to use the GitHub source method using git over HTTPS to pull the source code of our modules living in public GitHub repos. This is the easiest way to pull modules from GitHub from within a CICD pipeline (aka CodeBuild) without having to do extra configuration. The downside to this is if you need to use a module residing in a private GitHub repo, you will then have to use a GitHub access token tied to a user who has access to the private repo.
BYU’s Modules
ACS Info Module
https://github.com/byu-oit/terraform-aws-acs-info/
This module retrieves some basic ACS information and exposes them via outputs. This module does not create nor update any resources.
S3 Backend Module
https://github.com/byu-oit/terraform-aws-backend-s3/
Terraform module that creates an S3 bucket and DynamoDB table for backend state files.
Bastion Module
https://github.com/byu-oit/terraform-aws-bastion/
Creates a temporary bastion in an AWS Account.
ECR Module
https://github.com/byu-oit/terraform-aws-ecr/
Terraform module to create an ECR repository with lifecycle and repository policies.
Fargate API Module
https://github.com/byu-oit/terraform-aws-fargate-api/
Terraform module pattern to build a standard Fargate API. This module creates a Fargate service with an ALB, AutoScaling, CodeDeploy configuration and a DNS record in front.
Lambda API Module
https://github.com/byu-oit/terraform-aws-lambda-api/
Terraform module pattern to build a standard Lambda API.
RDS Module
https://github.com/byu-oit/terraform-aws-rds/
This terraform deploys an RDS instance.
S3 Static Site Module
https://github.com/byu-oit/terraform-aws-s3staticsite/
This Terraform module deploys an S3-hosted static site with HTTPS enabled.
Postman Test Lambda Module
https://github.com/byu-oit/terraform-aws-postman-test-lambda/
This Terraform module creates a lambda function that runs postman tests against a collection with an environment.
Scheduled Fargate Module
https://github.com/byu-oit/terraform-aws-scheduled-fargate/
Terraform module to creates a scheduled Fargate Task that happens based off of a cron or rate schedule.
Domain Redirect Module
https://github.com/byu-oit/terraform-aws-domain-redirect/
Terraform module to manage AWS resources needed to redirect all traffic from one domain to a different URL.
Humio Logger Module
https://github.com/byu-oit/terraform-aws-humio-logger/
Terraform module to forward CloudWatch log group events to Humio.
Create a module
To create a new module, create a new repo from the the terraform module template, name it accordingly, and share the repository with the DevOps Tooling team on GitHub, which will allow you to request reviews from the DevOps Developers.