Peter DeSantis’ Keynote at AWS re:Invent 2021

rolling now with the keynote from Peter DeSantis, SVP Utility Computing and Apps, @awscloud

#reinvent

@marknca tweeted at 01-Dec-2021, 23:01

this used to be Monday or Tuesday Night Live and is always one of my favourite talks of the week

#reinvent

@marknca tweeted at 01-Dec-2021, 23:02

Peter giving his perspective on the start of the @awscloud

#reinvent

@marknca tweeted at 01-Dec-2021, 23:03

big focus from the team before the launch was on the key word “Elastic”

<< I’m willing to be it still is!

#reinvent

@marknca tweeted at 01-Dec-2021, 23:03

early memories from the original EC2 team. 7/10 of the team are still @awscloud

#reinvent

@marknca tweeted at 01-Dec-2021, 23:04

email from @JeffBezos to the EC2 team in the early days. love the focus on the user experience

#reinvent

@marknca tweeted at 01-Dec-2021, 23:04

key areas of focus for EVERYTHING @awscloud

#reinvent

@marknca tweeted at 01-Dec-2021, 23:05

looks like we’re starting the night by looking at storage

#reinvent

@marknca tweeted at 01-Dec-2021, 23:05

remember, this is (usually) not a feature/service launch keynote. it’s typically a look behind the scenes @awscloud. lots of amazing technology and engineering challenges

#reinvent

@marknca tweeted at 01-Dec-2021, 23:06

remember when @awscloud looked like this?

#reinvent

@marknca tweeted at 01-Dec-2021, 23:06

lots of stuff build out from S3

#reinvent

@marknca tweeted at 01-Dec-2021, 23:06

for a longer look back at @awscloud’s early days. be sure to check out @jeffbarr’s session, “15 years of AWS with Jeff Barr”

https://virtual.reinvent.awsevents.com/login/

#reinvent

@marknca tweeted at 01-Dec-2021, 23:08

we’re going right down to the fundamentals. looking at old school(ish) hard drives

#reinvent

@marknca tweeted at 01-Dec-2021, 23:08

lots of mechanical engineering packed into these types of drives. Peter used a crazy airplane analogy that I won’t even try to summarize

lots of improvements but the mechanics are pretty steady

#reinvent

@marknca tweeted at 01-Dec-2021, 23:10

“Hard drives remain the best way to store large amounts of data when you need to access it immediately”, Peter << he left out, “…even though we don’t want that to be the case”

#reinvent

@marknca tweeted at 01-Dec-2021, 23:11

“Let’s do some quick math”, Peter << a key indicator as to why this is always my favourite keynote

#reinvent

(sorry @werner)

@marknca tweeted at 01-Dec-2021, 23:11

here’s what the math says : hard drives suck for these type of big data workloads

#reinvent

@marknca tweeted at 01-Dec-2021, 23:12

the opposite view for a large but low usage workload

#reinvent

@marknca tweeted at 01-Dec-2021, 23:13

basically the individual workload doesn’t align to the hard drive mechanics. you need to aggregate these workloads in order to smooth things out

#reinvent

@marknca tweeted at 01-Dec-2021, 23:14

Peter’s currently walking through how @awscloud distributes customer S3 workloads across the physical backend of the service

it’s the only way the math adds up

#reinvent

@marknca tweeted at 01-Dec-2021, 23:15

“If you want to innovate at scale, you need to move quickly. AND you need to do it safely”, Peter DeSantis, @awscloud

#reinvent

@marknca tweeted at 01-Dec-2021, 23:16

talking about how @awscloud approaches testing S3 now

#reinvent

@marknca tweeted at 01-Dec-2021, 23:16

I love this type of deep dive. especially the contrast in the complexity behind the scenes of S3 for something as simple as a bucket/key #ux

#reinvent

@marknca tweeted at 01-Dec-2021, 23:17

for testing, @awscloud is leveraging automated reasoning more and more. it’s the only way to truly test things out

more at https://www.amazon.science/

#reinvent

@marknca tweeted at 01-Dec-2021, 23:18

excellent timing by the @AmazonScience team, “A gentle introduction to automated reasoning” << just published

https://www.amazon.science/blog/a-gentle-introduction-to-automated-reasoning

#reinvent

@marknca tweeted at 01-Dec-2021, 23:18

formal tools are great. they have a very high bar for correctness

…but at the cost of speed and reducing the available talent pool

#reinvent

@marknca tweeted at 01-Dec-2021, 23:19

this isn’t a great choice to face. at @awscloud’s scale, they deal with it all of the time

…so they developed a new approach, “Lightweight Formal Methods”

#reinvent

@marknca tweeted at 01-Dec-2021, 23:20

here’s a great related paper from @AmazonScience , “Using lightweight formal methods to validate a key-value storage node in Amazon S3”

https://www.amazon.science/publications/using-lightweight-formal-methods-to-validate-a-key-value-storage-node-in-amazon-s3

#reinvent

@marknca tweeted at 01-Dec-2021, 23:21

lots of advantages to this approach…biggest challenge would be the initial design to allow for the application of these formal methods when required

#reinvent

@marknca tweeted at 01-Dec-2021, 23:21

ohhh, I beat Peter to the punch

#reinvent

@marknca tweeted at 01-Dec-2021, 23:22

first customer story up, this one’s from @Adobe

#reinvent

@marknca tweeted at 01-Dec-2021, 23:22

. @adobe’s intro covered how they are massive scale in the cloud, not turning to storage

#reinvent

@marknca tweeted at 01-Dec-2021, 23:24

talking about photos and how we stored physical photos

#reinvent

@marknca tweeted at 01-Dec-2021, 23:24

digital is a similar challenge but at a larger scale

how many photos & videos do you have?

#reinvent

@marknca tweeted at 01-Dec-2021, 23:24

next year “we” (not sure who that is, all of us?), we’ll generate 1,500,000,000,000 photos

#reinvent

@marknca tweeted at 01-Dec-2021, 23:25

. @adobe turning to the two primary storage use cases for storing photos and videos

“I want it now"

“I want it sometime, whenever really”

#reinvent

@marknca tweeted at 01-Dec-2021, 23:26

the obvious questions about this story form @adobe is why do I only get 100GB of storage with Creative Cloud then?

<< feature request == let me use my own @awscloud account of Creative Cloud storage & sharing

#reinvent

@marknca tweeted at 01-Dec-2021, 23:27

s/of/for/

@marknca tweeted at 01-Dec-2021, 23:28

learn more about @adobe Sensei at https://www.adobe.com/ca/sensei.html

#reinvent

@marknca tweeted at 01-Dec-2021, 23:29

. @adobe is obviously the reference customer for Amazon S3 Glacier Instance Retrieval

more on the @awscloud feature at https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-s3-glacier-instant-retrieval-storage-class/

#reinvent

@marknca tweeted at 01-Dec-2021, 23:29

it snuck under the radar for most people but it could be quite a game changer

#reinvent

@marknca tweeted at 01-Dec-2021, 23:30

Peter moving on to block storage types now

#reinvent

@marknca tweeted at 01-Dec-2021, 23:30

now diving into SSD technology

#reinvent

@marknca tweeted at 01-Dec-2021, 23:31

SSD remove most of the mechanical challenges but they have their own issues…like the limitations of flash storage

#reinvent

@marknca tweeted at 01-Dec-2021, 23:31

primary the paging system & the lifetime of those pages

#reinvent

@marknca tweeted at 01-Dec-2021, 23:32

as Peter is delivering this section of the talk, I realize I know way too much of the low level functionality of both HDD and SDD.

this is the downside of nerd life. lots of obscure knowledge

…but I wouldn’t have it any other way

#reinvent

@marknca tweeted at 01-Dec-2021, 23:33

of course there are different issues at hyper scale that @awscloud has discovered with low level SSD tech

#reinvent

@marknca tweeted at 01-Dec-2021, 23:34

small inconsistencies add up fast at hyper scale. things we would never notice even with hundreds of systems, they see regularly

#reinvent

@marknca tweeted at 01-Dec-2021, 23:35

. @awscloud Nitro smooths a lot of this out

more on Nitro at https://aws.amazon.com/ec2/nitro/

#reinvent

@marknca tweeted at 01-Dec-2021, 23:35

…of course they built their own Nitro SSD

#reinvent

@marknca tweeted at 01-Dec-2021, 23:36

btw, @QuinnyPig it live tweeting this keynote as well. for his take ( ) head over to https://twitter.com/QuinnyPig/status/1466181160099139586

#reinvent

@marknca tweeted at 01-Dec-2021, 23:37

Peter citing the performance delivered by the custom SSD approach via @awscloud Nitro

#reinvent

@marknca tweeted at 01-Dec-2021, 23:37

more on io2 Block Express at https://aws.amazon.com/blogs/aws/now-in-preview-larger-faster-io2-ebs-volumes-with-higher-throughput/

#reinvent

@marknca tweeted at 01-Dec-2021, 23:38

on to @awscloud Graviton now…

#reinvent

@marknca tweeted at 01-Dec-2021, 23:38

more on Graviton at https://aws.amazon.com/ec2/graviton/

#reinvent

@marknca tweeted at 01-Dec-2021, 23:39

. @awslcoud is pushing Graviton everywhere. and why not? more bang for your buck in most cases w/cloud native designs

#reinvent

@marknca tweeted at 01-Dec-2021, 23:40

Peter touches on the Graviton3 update

#reinvent

@marknca tweeted at 01-Dec-2021, 23:41

have we seen these stats for Graviton3 yet?

…not that we should focus on them according to Peter (and I agree)

#reinvent

@marknca tweeted at 01-Dec-2021, 23:42

I like Apple’s performance per watt approach. which @awscloud has followed as well

#reinvent

@marknca tweeted at 01-Dec-2021, 23:42

more on Dennard scaling at https://en.wikipedia.org/wiki/Dennard_scaling

#reinvent

@marknca tweeted at 01-Dec-2021, 23:43

Peter talking about power consumption now. it’s a critical attribute for all of us, mind boggling at @awscloud’s hyper scale

#reinvent

@marknca tweeted at 01-Dec-2021, 23:44

How do we efficiently increase performance of a Graviton core?

Make the core WIDER

#reinvent

@marknca tweeted at 01-Dec-2021, 23:44

there have been lots of improvements in silicon design over the past decade. the easy way to think about it is for year all we worried about was the size of the engine

we finally realized that the tires, chassis, fuel, and other attributes matter as well…

#reinvent

@marknca tweeted at 01-Dec-2021, 23:46

…that means more cores, wider memory lanes, direct access to storage, etc.

this is why we’ve seen massive boosts in performance in things like @apple’s M1 and @awscloud Graviton3

#reinvent

@marknca tweeted at 01-Dec-2021, 23:47

another customer story up now, this time it’s Fannie Mae

#reinvent

@marknca tweeted at 01-Dec-2021, 23:48

Kimberly Johnson, EVP & COO of @FannieMae given the scale and scope of their operations

tl:dr = it’s big

#reinvent

@marknca tweeted at 01-Dec-2021, 23:48

ooohhh, @FannieMae built out an HPC solution @awscloud Lambda. would love to get a deep dive on that one…

#reinvent

@marknca tweeted at 01-Dec-2021, 23:52

very interesting solution from @fanniemae. they leverage some key @awscloud data services to expand credit score analysis to include rent payments…regardless of how they were made

#reinvent

@marknca tweeted at 01-Dec-2021, 23:55

now talking about evaluating risk to housing due to climate change. again another massive data problem, taking unstructured data into structure data for analysis and prediction

#reinvent

@marknca tweeted at 01-Dec-2021, 23:56

good customer story from @fanniemae. I think they struck the right depth to solve solutions to unique problems at a scale that’s only really possible in the cloud

#reinvent

@marknca tweeted at 01-Dec-2021, 23:57

on to #ml now. I swear this is slide is even more packed than @SwamiSivasubram from earlier today

#reinvent

@marknca tweeted at 01-Dec-2021, 23:58

Peter’s breaking #ml down into two big stages:

• training
• inference

you need different infrastructure & tools for both

#reinvent

@marknca tweeted at 01-Dec-2021, 23:59

diving into inference now…

#reinvent

@marknca tweeted at 02-Dec-2021, 00:00

more on @awscloud’s custom silicon, Inferentia, for #ml inference at https://aws.amazon.com/machine-learning/inferentia/

#reinvent

@marknca tweeted at 02-Dec-2021, 00:00

we’ll get there in a minute (I’m sure) but here’s more on @awscloud Trainium, the custom chip for #ml training

https://aws.amazon.com/machine-learning/trainium/

#reinvent

@marknca tweeted at 02-Dec-2021, 00:01

back to inference, @awscloud is working to improve inference work on general purpose CPUs as well. sometimes you just need to run the process where the workload is w/out the custom chips

#reinvent

@marknca tweeted at 02-Dec-2021, 00:02

…or at least _those_ custom chips

#reinvent

@marknca tweeted at 02-Dec-2021, 00:02

sticker stats for Trainium…

#reinvent

@marknca tweeted at 02-Dec-2021, 00:03

Peter says the name “Trainium” implies training #ml models. for most, it’s probably

#reinvent

@marknca tweeted at 02-Dec-2021, 00:04

“..as is normal in a math party” << f–k yeah, Peter

#reinvent

@marknca tweeted at 02-Dec-2021, 00:05

Peter diving into the math of #ml training runs. you need it to figure out how to scale out/up a workload to get the job done

#reinvent

@marknca tweeted at 02-Dec-2021, 00:06

GPT-3 is slightly larger than BERT-Large #nlp #ml models

#reinvent

@marknca tweeted at 02-Dec-2021, 00:06

these models are massive. they create their own computer science problems

#reinvent

@marknca tweeted at 02-Dec-2021, 00:07

more on how to tackle these challenges the @awscloud Well-Architected Framework, Machine Learning Lens

that’s available at https://docs.aws.amazon.com/wellarchitected/latest/machine-learning-lens/machine-learning-lens.html

#reinvent

@marknca tweeted at 02-Dec-2021, 00:08

more math party analogy/reference!

#reinvent

@marknca tweeted at 02-Dec-2021, 00:08

…lots of room left in that graph…

#reinvent

@marknca tweeted at 02-Dec-2021, 00:09

Peter talking about the biggest @awscloud EC2 instance types for #ml training

#reinvent

@marknca tweeted at 02-Dec-2021, 00:09

the network optimized version of the Trn1 instance type will have 1600 Gbps network bandwidth << fantastic!

#reinvent

@marknca tweeted at 02-Dec-2021, 00:10

sizing your training pool is a critical decision. not only will it impact how LONG you need to wait but also how MUCH you pay

read that Well-Architected Lens, please!

https://docs.aws.amazon.com/wellarchitected/latest/machine-learning-lens/machine-learning-lens.html

#reinvent

@marknca tweeted at 02-Dec-2021, 00:11

Tranium uses a technique called “Systolic Array Manipulation”

going to have to dig into this one later on: https://en.wikipedia.org/wiki/Systolic_array

#reinvent

@marknca tweeted at 02-Dec-2021, 00:12

Tranium provide 16 fully programmable inline data processors. this allows further tuning for each training workload

<< still doesn’t fix the name though

#reinvent

@marknca tweeted at 02-Dec-2021, 00:13

now we’re talking about how to round numbers. I ❤️ this stuff

super niche but fascinating

#reinvent

@marknca tweeted at 02-Dec-2021, 00:14

of course Trainium supports stochastic rounding directly in hardware. because, why not?

#reinvent

@marknca tweeted at 02-Dec-2021, 00:14

more on the @awscloud Neuron SDK at https://aws.amazon.com/machine-learning/neuron/

<< it abstracts a lot of what Peter just covered away, so you just get the benefits

#reinvent

@marknca tweeted at 02-Dec-2021, 00:15

Peter is going to wrap up with sustainability and the climate pledge

more on the pledge (now over 200 signatories) at https://sustainability.aboutamazon.com/about/the-climate-pledge

#reinvent

@marknca tweeted at 02-Dec-2021, 00:16

. @awscloud is always looking to improve efficiency in every aspect of our infrastructure << we’ve heard various achievements and efforts year over year from them. it’s nice to see

#reinvent

@marknca tweeted at 02-Dec-2021, 00:17

that’s a big commitment…and they are on a path to achieve it 5 years early (originally target was 2030)

#reinvent

@marknca tweeted at 02-Dec-2021, 00:18

#reinvent

@marknca tweeted at 02-Dec-2021, 00:18

storage isn’t the only problem. moving that power around is a big problem as well

#reinvent

@marknca tweeted at 02-Dec-2021, 00:20

here’s a great research piece from @voxdotcom

https://www.youtube.com/watch?v=s3ScJ_FwaZk&feature=youtu.be

#reinvent

@marknca tweeted at 02-Dec-2021, 00:20

. @mikegchambers called this the other night in our day one recap!

#reinvent

@marknca tweeted at 02-Dec-2021, 00:21

you can watch that recap on @mikegchambers excellent YouTube channel at https://www.youtube.com/watch?v=FiJfYTqM02M&feature=youtu.be

#reinvent

@marknca tweeted at 02-Dec-2021, 00:22

here are some other sessions at @awscloud #reinvent that focus on sustainability. be sure to check them out

#reinvent

@marknca tweeted at 02-Dec-2021, 00:22

..and that’s a wrap from Peter. not nearly as broad as previous years but still a great deep dive into things like @awscloud custom silicon

/ #reinvent

@marknca tweeted at 02-Dec-2021, 00:23

Start