Build your own Status Page with Terraform, S3 and AWS CodeBuild.
Automatically communicate service health to your users so that you can concentrate on fixing an outage if it occurs.
Background
MailSlurp is a service for sending and receiving emails via REST. It is used by companies in their test suites and deployment pipelines so up-time is mission critical.
Recently the API went down due to heavy load and our users sent tweets and emails asking what was wrong. After redeploying and tweaking the our Kubernetes scaling parameters we decided that a status page would provide a lot of assurance to paying customers. It would allow us to fix issues instead of answering emails if things went wrong again.
It was a fun process so we decided to share what we learnt. This post will explain how we built a status page using Terraform, Codebuild, S3 and a bit of HTML and Javascript (VueJS and Bootstrap CDNs). It took us a couple of hours but saved us a whole lot more time (and money) in the long run!
What is a status page?
Status pages are typically simple websites that display the current or recent historical state of an online service or API. Many tech companies use them to communicate the health of their systems to customers. These status pages relieve users of any doubt about what might be broken (their code? their internet? or the service they rely on?) and reduce the number of incoming support requests.
Paid services
Before we start making our status page it would be prudent to evaluate the available paid services. Here are a few we evaluated. They all offer nice features and quick starts but their pricing models do add up. If you want more control and lower costs its easy to build you own.
Status Page examples (what are we building?)
Status pages differ in design but have one thing in common: they display the state of the service in a clear way. For our status page we wanted to keep it simple so we decided to display only the last known status plus some links to support and Twitter etc.
Determining service health
So how do we know whether a service is alive or dead? When should we display a green tick or red cross? For MailSlurp we have a suite of smoketests that run every ten minutes and test the key functionality of the MailSlurp API. This includes:
- Can the landing page and API be reached via HTTP request?
- Can the SDK client be installed via NPM and can it execute queries
- Can emails be sent and received with the API (a core feature of MailSlurp)
This smoketest is written in Javascript using Jest and is run periodically by AWS CodeBuild. If we can capture the results of these tests and save the status in S3 we can fetch that status in Javascript from a static HTML status page whenever a user loads it!
CodeBuild setup
I’ve already written a detailed post on how to schedule jobs in CodeBuild but when building a status page you can use any CI tool to run your tests: CircleCI, Go.cd, Jenkins etc. All you need to do is wrap the tests in a script that can interpret the results and post them to a remote publicly accessible storage location. At MailSlurp we use AWS extensively so CodeBuild and an S3 bucket make sense. Here is our CodeBuild buildspec.yml for the MailSlurp smoketest suite.
version: 0.2phases:
build:
commands:
- yarn install
- node run.js
As you can see it simply installs node dependencies and executes the run.js script. More on that below.
Handling test results
Our test suite exits with 0if all tests are successful and >0 if any test fails. So let’s write a script called run.js that execute the tests in a child process and handles the result. Later we will want to post the results to S3 so let’s install some dependencies first.
Here are the dependencies for our Jest test runner and for the AWS SDK:
{
"dependencies": {
"aws-sdk": "^2.485.0",
"jest": "^24.1.0"
}
}We don’t need any S3 credentails in this case as CodeBuild is a privileged environment. You may need to configure the client in your environment.
Now let’s write run.js so that it:
- spawns a process
- runs the tests
- intercepts the exit code
- posts the exit code as JSON to S3
const { spawnSync } = require('child_process')
const S3 = require('aws-sdk/clients/s3')
const s3Client = new S3()// run jest tests in a process and handle failures
console.log("Running tests in subprocess")
const {
status,
stdout,
stderr
} = spawnSync('npx', ['jest'], { ...process.env, CI: true })// log the outputs from process if they exist
[stdout, stderr].forEach(output => {
console.log((output || "").toString('utf8'))
})// create json representation of test results plus epoch timestamp
const json = JSON.stringify({
status,
timestamp: new Date().getTime() / 1000
})// specify s3 access and destination
const params = {
ACL: "public-read",
Body: json,
Bucket: "mailslurp-public-status",
Key: "status.json",
ContentType: "application/json"
}// upload json to s3
console.log("Uploading status to S3")
s3Client.putObject(params, (err, _) => {
// if s3 client error then exit with error
if (err) {
console.error(err)
process.exit(1)
} else {
// if successful
// exit with same exit code as the test process
console.log("Upload successful")
process.exit(status);
}
})
Setting up S3 with Terraform
So now we have our smoketest CodeBuild repository that handles the test result and posts it as JSON to a bucket. Remember this test is being executing on a schedule so will update the JSON file every ten minutes to reflect the system state.
One little problem — the bucket doesn’t exist yet! We need to create the bucket in order for the script to succeed. You could do this manually but Terraform is a really nice way to manage infrastructure. Here is how you do it (note you will need to configure a Terraform provider):
resource "aws_s3_bucket" "status" {
bucket = "mailslurp-public-status"
acl = "public-read"cors_rule {
allowed_headers = ["*"]
allowed_methods = ["GET"]
allowed_origins = ["*"]
expose_headers = ["ETag"]
max_age_seconds = 3000
}
}
Now run terraform apply to apply the resources to your AWS account. If you open up the AWS Console in the web browser you should see the bucket you created.
Once the bucket is created you can run the node script and see the results via a public URL. In this case https://mailslurp-public-status.s3-us-west-2.amazonaws.com/status.json.
{"status":0,"timestamp":1562022690.971}Woohoy! Almost there. Now for the HTML to display this data in a more user friendly way.
Building the status page HTML
To display the JSON to users we need to create a simple static website and a bit of Javascript to fetch the results. Let’s start with some basic HTML5 markup and include Bootstrap, jQuery and VueJS via CDNs.You can do all of this without any libraries or dependencies but for the sake of simplicity and speed we chose these CDN hosted helper libraries.
Remember we want to keep this status page simple — we don’t want to go overboard and build a fully-fledged PWA with tests and build-steps. We want a single file. jQuery will help with cross browser AJAX requests while VueJS will simplify templating the result. Bootstrap will make it look nice.
Here’s an excerpt of the HTML. Notice the use of CDNs to avoid having to manage and deploy these libraries.
<!DOCTYPE html>
<html lang="en"><head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>MailSlurp Status</title> <!-- jQuery -->
<script src="https://code.jquery.com/jquery-3.4.1.slim.js"></script> <!-- VueJS -->
<script src="https://cdn.jsdelivr.net/npm/vue@2.6.10/dist/vue.js"></script> <!-- bootstrap -->
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css">
</head><body>
<!-- we will put markup and javascript here -->
</body>
</html>
Fetching status data on the frontend
To fetch the test status results we just need to fetch the JSON from the bucket we created. To support older browsers we’ll use jQuery to do that Let’s see what that looks like.
$.getJSON("https://mailslurp-public-status.s3-us-west-2.amazonaws.com/status.json", function (data) { /* todo */ });So to put this AJAX call to use, let’s create a Vue element and map the results to our HTML.
<script>
new Vue({
el: '#root',
data: function () {
return {
loading: true,
status: null,
timestamp: null
}
},
created: function () {
var _this = this;
$.getJSON("https://mailslurp-public-status.s3-us-west-2.amazonaws.com/status.json", function (data) {
_this.status = data.status;
_this.timestamp = data.timestamp;
_this.loading = false;
})
}
})
</script>And here is some HTML to display the results. It uses Bootstrap classes for layout and styling. The #root ID maps to Vue’s el property and allows Vue to monitor and update the elements within it.
We added an H1 title and a loading message with a v-if="loading". We will display this message until we have fetched the result and decided what to do with it.
Notice how the other v-else and v-else-if elements have a v-cloak attribute. This is a special attribute that Vue will remove from managed elements once it has loaded itself. It is up to us to handle the time when Vue has not removed this tag. So a CSS rule like [v-cloak]{ display: none } will insure that the elements with this attribute are not visible before Vue starts up and is able to hide them (because of their v-else logic).
<div id="root" class="container py-5">
<h1 class="h1 mb-5">MailSlurp Status</h1>
<!-- loading -->
<p class="lead" v-if="loading">Loading...</p><!-- 0 exit code means tests passed -->
<div v-else-if="status === 0" v-cloak>
<div class="alert alert-success">
<h4 class="alert-heading">All clear!</h4>
<p class="mb-0">All systems are functioning. If you need help in any way please contact support</p>
</div>
</div><!-- anything else indicates an error -->
<div v-else v-cloak>
<div class="alert alert-danger"><h4 class="alert-heading">Service disruptions!</h4>
<p class="mb-0">Some MailSlurp systems are failing, please hold tight. Developers are looking into the
issue. If it persists please contact support.</p>
</div>
</div><div class="pt-3">
<a href="https://www.mailslurp.com">Home</a>
<span>|</span>
<a href="https://docs.mailslurp.com">Documentation</a>
<span>|</span>
<a href="https://twitter.com/@mailslurp">Twitter</a>
<span>|</span>
<a href="jackmahoney212@gmail.com">Support</a>
</div>
</div><style>
[v-cloak] {
display: none;
}
</style>
Lastly we include some links and contact information.
Pretty simple! If we open the HTML in a browser it should look something like this:
Deploying the HTML to a CDN and domain
Now that we can display the status in HTML we want to deploy it to a CDN so that anyone can easily access and view our system health. We also want https support and a convenient domain name for our site. status.mailslurp.com would fit nicely.
Once again we can use Terraform to create an S3 website and CloudFront CDN distribution. We can then create a Route53 Alias record for the CloudFront distribution and attach an existing SSL certificate for the MailSlurp domain.
There’s a bit of code but the steps are simple.
- Create an S3 bucket website resource for the HTML
- Create a CloudFront distribution for said bucket and attach our top level certificate
- Create a Route53 A Record for this distribution with the name
status.mailslurp.comso that the CDN is easily accessible
Note: Terraform CDN resources sometimes take up to an hour to deploy. Just let it do its thing, it will eventually finish.
# create a bucket for the status html
resource "aws_s3_bucket" "status_html" {
bucket = "status.mailslurp.com"
acl = "public-read" website {
index_document = "index.html"
error_document = "404.html"
} policy = <<EOF
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "PublicReadForGetBucketObjects",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::status.mailslurp.com/*"
}
]
}
EOF
}# enter a route 53 record for our cdn
resource "aws_route53_record" "status_cdn_alias" {
zone_id = "${var.zone_id}"
name = "status.mailslurp.com"
type = "A"alias {
name = "${aws_cloudfront_distribution.status_cdn.domain_name}"
zone_id = "${aws_cloudfront_distribution.status_cdn.hosted_zone_id}"
evaluate_target_health = false
}
}# create the cdn for caching and ssl
resource "aws_cloudfront_distribution" "status_cdn" {
origin {
custom_origin_config {
http_port = "80"
https_port = "443"
origin_protocol_policy = "http-only"
origin_ssl_protocols = ["TLSv1", "TLSv1.1", "TLSv1.2"]
}domain_name = "${aws_s3_bucket.status_html.website_endpoint}"
origin_id = "status.mailslurp.com"
}enabled = true
is_ipv6_enabled = true
comment = "Managed by Terraform"
default_root_object = "index.html"aliases = ["status.mailslurp.com"]default_cache_behavior {
allowed_methods = ["HEAD", "DELETE", "POST", "GET", "OPTIONS", "PUT", "PATCH"]cached_methods = [
"GET",
"HEAD",
]target_origin_id = "status.mailslurp.com"forwarded_values {
query_string = truecookies {
forward = "none"
}
}viewer_protocol_policy = "redirect-to-https"
min_ttl = 0
default_ttl = 3600
max_ttl = 86400
compress = true
}price_class = "PriceClass_100"restrictions {
geo_restriction {
restriction_type = "none"
}
}viewer_certificate {
# mailslurp certificate arn
acm_certificate_arn = "${var.certificate_arn}"
ssl_support_method = "sni-only"
}
}
Publishing the HTML to our bucket
The last step is to push the HTML for our status page to the status_html bucket we created with Terraform. There are many ways to do this but the AWS CLI makes it easy.
aws s3 cp index.html s3://status.mailslurp.comFinishing up and testing it out
That’s it. We have everything we need for a status page. Let’s recap how it works.
- A scheduled test-suite is executed by CodeBuild ever ten minutes
- The test-suite is wrapped in a Node script that captures results and publishes them as JSON to an S3 bucket
- We created the S3 bucket for the status storage in Terraform
- We created an HTML page that uses Bootstrap, Vue and jQuery to fetch and render the saved JSON from the bucket
- We created a Route53 alias entry, CDN, and S3 website for the status page HTML and deployed it with the AWS CLI
Now let’s make the tests fail on purpose to see if our status page updates…
And voila. After failing the tests and loading https://status.mailslurp.com we see an alert for a service disruption on MailSlurp.
Conclusion
A status page communicates service health to customers so that developers can concentrate on fixing issues when times get tough. A good way to test service/API health is with end-to-end tests. MailSlurp is an API for creating real email addresses via REST and sending and receiving from them. You can use these emails in your tests to sign up with unique accounts, verify oauth confirmation codes, parse transactional mail and much more. Check it out!
