Automating safe, hands-off deployments

3 minute read | Last updated 2-Aug-2022 | An icon depicting a retail tag with a heart for 'favourite' AWS Cloud

The Amazon Builder’s Library is a great set of deep dive papers into the challenges with modern systems. This post looks at how Amazon conducts hands-off deployments.

“Automating safe, hands-off deployments, takes a look at how Amazon does continuous deployment. Amazon has been doing this practice for a long while and has automated each process to the point where it’s hands-off.

I call out a few more details in the Twitter thread below…

Tweet 1/10 Next tweet

in today’s review, I’m looking at “Automating safe, hands-off deployments” by @clare_liguori at @awscloud

https://aws.amazon.com/builders-library/automating-safe-hands-off-deployments/

this one is all about getting your work out to your customers

☁️ #cloud #devops

marknca @marknca tweeted at 11-Nov-2021, 21:12

Tweet 2/10 Next tweet Start

this thread is available unrolled at https://t.co/PIavI51500

yesterday’s thread on “Fairness in multi-tenant systems” is up at https://markn.ca/2021/fairness-in-multi-tenant-systems/

☁️ #cloud #devops

marknca @marknca tweeted at 11-Nov-2021, 21:12

Tweet 3/10 Next tweet Start

right out of the gate, I love @clare_liguori’s callout, “continuous deployment pipelines to help us deploy quickly and safely”

two keys there:
– quickly
– safely

☁️ #cloud #devops

marknca @marknca tweeted at 11-Nov-2021, 21:12

Tweet 4/10 Next tweet Start

another key quote from early on in the paper, “We identified deployment risks and found ways to mitigate those risks through new safety automation in pipelines” ❤️❤️❤️

spot on! modern deployment reduces risk when done well. keep that in mind #infosec folks

☁️ #cloud #devops

marknca @marknca tweeted at 11-Nov-2021, 21:12

Tweet 5/10 Next tweet Start

the paper is roughly broken down in the parts of a pipeline:

– source
– build
– test
– prod

…and of course how they all fit together

☁️ #cloud #devops

marknca @marknca tweeted at 11-Nov-2021, 21:12

Tweet 6/10 Next tweet Start

one key point is that _everything_ in production at Amazon gets there via a pipeline. static assets, code, infrastructure, everything

these pipelines make it easier to keep track of the history of changes & to ensure a high bar for quality

☁️ #cloud #devops

marknca @marknca tweeted at 11-Nov-2021, 21:12

Tweet 7/10 Next tweet Start

this also make it easier to do code reviews, as they are all going through the pipelines. a nice example of a code review checklist in presented in the

☁️ #cloud #devops

marknca @marknca tweeted at 11-Nov-2021, 21:12

Tweet 8/10 Next tweet Start

testing is discussed in depth & honestly there is a lot more testing than I expected. it’s an example of how Amazon’s approach has matured & takes advantage of the @awscloud environment

☁️ #cloud #devops

marknca @marknca tweeted at 11-Nov-2021, 21:12

Tweet 9/10 Next tweet Start

deployments themselves are a bit crazy. that, I did expect.

the pipelines need to ensure that they don’t negatively impact production…especially at scale.

@clare_liguori does a great job of breaking down the concerns & techniques used here

☁️ #cloud #devops

marknca @marknca tweeted at 11-Nov-2021, 21:12

Tweet 10/10 Next tweet Start

finishes off w/idea of pipelines as code. very meta but it makes sense given the number of pipelines in use

if you’re deploying to production, you should check out this paper. that’s a nice way of saying you should go read this paper ASAP

/ ☁️ #cloud #devops

marknca @marknca tweeted at 11-Nov-2021, 21:12

Start