Episode 74: Feeding the Matrix: AWS

Molly Wood Voice-Over: Welcome to Everybody in the Pool, the podcast where we dive deep into the innovative solutions and the brilliant minds who are tackling the climate crisis head-on. I'm Molly Wood.

This week it’s the second episode of Feeding the Matrix my special series on AI and energy use

In the last episode, I spoke with Nvidia’s head of sustainability about the company’s race to make AI more efficient with its chips and software platform while also maintaining its position as the world’s seventh largest company.

But of course running artificial intelligence takes more than just chips.

It takes a massive amount of computing power, and huge buildings full of computers data centers.
Today, we're going inside those data centers into the nitty gritty of how to make them more efficient with the world’s biggest cloud provider by market share A-W-S.

We’ll get into the real details of making a data center more efficient - from where you put the racks of computers, to the concrete in the walls, to custom chips that AWS has been quietly making since 2017.

As you’ll hear in this interview it’s good for the planet it’s good for business and so far at least AWS's biggest customers are demanding it.

Chris Walker

My name is Chris Walker. I lead AWS's sustainability organization. And we sit within the infrastructure space within AWS and can work across all of AWS, including our service teams and partner with the broader Amazon team as well.

Molly Wood

For the nine -ish people who don't know, can you tell us what AWS is?

Chris Walker - AWS

Yeah, sure. AWS is Amazon Web Services. We are a company underneath the Amazon umbrella, and we are a cloud provider that provides and builds data centers around the world, sells 300 plus services to variety of customers, and we operate, build, and design the cloud as we know it.

Molly Wood 

Okay, let's talk about the cloud and the climate impact of the cloud as we know it and the climate impact of data centers. I suspect that the kind of growing energy use is a big reason why a large organization, it sounds like, was stood up somewhat recently, relatively speaking, within Amazon.

Chris Walker - AWS

Yes.

Chris Walker - AWS 

Yeah, yeah, we realized as we were responding to the customer demand as we have been and continue to do, we needed to use our scale kind of to continue to grow and make progress within sustainability. And like I mentioned, back in 2019, we set the climate pledge, when Amazon set the climate pledge in partnership with Global Optimism,

We kind of not only saw the industry changing, but we also saw within our own company that we needed to make some changes and make some, set some ambitious goals, set some big targets for us to start thinking about what the industry and our own company was going to do in the years to come. I think net zero carbon by 2040 is an ambitious goal. It's an ambitious goal for any company, but it's an ambitious goal for a company at our size, especially.

and you mentioned kind of the rate that we're scaling.

Molly Wood 

which has like, and which we should be clear, which has not just data centers, but like delivery and packaging and sort of all of like a lot Amazon Prime video, right? Yep.

Chris Walker - AWS 

an Amazon Prime video studios, all of those underneath the umbrella. And so I think that's, that's also a misconception, not a misconception, but it's a misunderstanding at times where customers think that it's just AWS. It's not just AWS. It's all of the companies, all of the business units within Amazon that we're trying to work in partner on behalf of. And then we also set 100 % renewable energy. We're looking to match our global power use.

with renewable energy and we originally set that goal for 2030. We built and structured a team, not only renewable energy procurement professionals, but also renewable energy operations teams which sits in my organization. And we also have technical program managers that work across the regional teams to go, okay, here's the top level goal. How do we now meet that? What are the incremental steps we need to take on that journey? And so we met that goal.

back in last year in 2023. We're really excited to meet at seven years ahead of time. But it's also, you know, as you start looking at the industry and where data center and industry, but specifically kind of AI and gen AI is going, that's going to become even more important in the coming years as we start looking at different types of power generation.

Molly Wood 

Right, so let's do that. Let's start with the cloud and data centers and why they're so intensive, and then we'll move specifically to AI and why that is such a big concern.

Chris Walker - AWS 

Yeah.

Chris Walker - AWS 

Yeah, yeah, it's It's a concern. I I would say it's it's Yes, yes, sorry

Molly Wood 

Wait, data centers first. Let's set the baseline. Where does all that energy use come from for people who just sort of don't understand that the cloud is in fact like a crap ton of computers?

Chris Walker - AWS 

Yeah, yeah. So data centers are warehouse structures. You know, looking at them from the outside, they look like a four walled general logistics warehouse, which is actually where some of them were originally designed after. On the inside, though, they're very, very, very different data centers. When you think about building a data center, there's largely three kind of that always talk about the three legs of the stool, but there's now

Traditionally, it was real estate, you needed enough land, you needed the right power connectivity, and you needed the right fiber connections. Energy is largely driving some of location decisions now, but what I mean by that is when we start thinking about what's inside of a data center, you have rooms and rows of server networks and networking equipment that drive and operate both compute power and storage power for our customers.

We think about the cloud and we think about what data centers are. They're essentially large building computers that are running different models and different instances for customers based on what their need is. So everything from our Netflix storage and video response through our amazon .com shopping site and how customers interact with that, that all runs through the cloud and operates on the cloud the same way.

Molly Wood 

And so at this point, data centers, think, are consuming something like, depending on your estimate, 2 to 4 % of global electricity. And that is only increasing as we add AI capabilities,

Chris Walker - AWS 

Yeah, the percentage, the increase, I think the given is we know it's going to go up, right? We don't know where it's going to end. You can pick an article, pick a data point, but we don't entirely know where that trajectory is. Exactly. But I think it's part of the reason why we, as AWS, started going, we know that we're going to, you know, AI or whether it's large,

Molly Wood 

Right. Right.

Molly Wood 

It ends at the matrix. ends at humans as batteries.

Chris Walker - AWS 

compute models, large language models. We know that those are energy intensive things that we're going to continue to to build and need energy for. How do we continue to make progress on our sustainability journey as the increase is there? And that's when we started shifting to efficiency. We really started focusing on driving efficiency through all aspects of our infrastructure, whether that's reducing waste.

whether that's through a water positive goal by 2030, whether that's our custom chips that we're using and designing inside of our data centers. It's not just one silver bullet that's going to solve all of this. It's the different aspects that we're starting to pull into and really use within our sites to drive that work.

CUT TO

Molly Wood 

All right, so let's talk about efficiency. So the idea is not necessarily, there are certainly outfits that are working on making LLMs themselves more efficient. But the goal really is if you're going to have to do more computing to do that computing as efficiently as possible, right? And it sounds like you have a bunch of specifics within that umbrella. talk about, yeah. So I heard water positivity and custom chips. What else?

Chris Walker - AWS

Right. Right.

Chris Walker - AWS 

Yeah, let's go through them. Yep, yeah, let's start with the custom chips, because I think that that one's really exciting. So we yeah, we started a handful of years ago looking at how we can design our own machine learning chips, which obviously are powering millions of workloads daily, but to really be more energy efficient. We have AWS Graviton. That's a processor chip that

Molly Wood 

Yeah, that's fascinating. Yeah, you're making your own.

Chris Walker - AWS 

delivers high performance computing with high levels of energy efficiency. And the latest release was our Graviton 4 chip. We're seeing up to 30 % better computing performance, 75 % more memory bandwidth through our current generation of the Graviton 3. And we're looking at those technologies and, in terms of how we can release them, we want to make sure with any releases we're seeing those energy efficiency gains. And I think that's important for customers.

We're looking at using advanced modeling methods, you know, in terms of to optimize our data center design. And I think that's something that it allows our operations teams. It allows our engineering teams to understand how a data center is going to perform before it's built. We can use these technologies to not only bring higher reliability into the site, but it also allows us to bring, you know, more energy efficiency throughout the systems as well. And I think when we started

Again, kind going back to why efficiency Chips was one of those things that it seemed like a given right? We wanted to make sure that at the smallest levels inside of the data centers We were looking at it, but it doesn't stop there Data center when we start thinking about data centers, you know, mentioned it's the warehouse design Low carbon concrete and steel we started looking at the materials that we're applying into our data centers for the structure itself

how can we lower the carbon footprint in that space? And so we partnered with a handful of companies to fold in low carbon concrete and steel. And we were seeing 30 to 40 % less embodied carbon into our data centers that were standing up. And in the past, those were done as one -offs, right? We would have to go intercept individual builds. With the organization as it stands, we've now kind of embedded ourselves into

The engineering organization which has allowed us to really implement those changes at the basis of design which now means data centers as they're being built moving forward, they will take those changes into effect.

Molly Wood Voice-Over: Time for a quick break. When we come back, we’re going to talk more about the specifics of making a data center more efficient from being water positive to designing modular data centers so you don’t build more than you need to literally rearranging the furniture.

Molly Wood Voice-Over: Welcome back to Everybody in the Pool. We’re talking with Chris Walker who leads sustainability at the Amazon cloud giant A-W-S.

Molly Wood 

OK, and then let's talk about water, which is a surprisingly massive part of data centers in cloud computing.

Chris Walker - AWS 

Yeah, so water positivity is interesting. And we've set a goal to be water positive by 2030, which means return more water to our communities than what we use by 2030. And we are 41 % of the way as of this year, 41 % of the way to that goal. And we take a balanced approach to those projects. And what I mean by that is we look at both the localized level in terms of how we are

implementing those programs, right? If we building a data center in country X, what's the water landscape there? Can we partner with farming communities to ensure that the water that we're reusing or recycled water, we can push out to the farming communities to go use on their crops? And we also look at the more metropolitan areas that we're building as well to ensure that...

we're partnering with the local governments and the regionalized governments to make sure that the right efforts are being taken. So we're really excited about that progress. We also looked at our cooling facilities and how we cool our data centers, right? Water, when you think about data centers and cooling, water is mostly used in the cooling technologies.

We, years ago, we started looking at using evaporative cooling and kind of free air cooling inside of our data centers. And so in every location that we can, we look to use open air cooling and allow, bring the outside air in to cool the environment, lowers our water usage overall.

Molly Wood 

And then how, so this is a, it sounds like most of this applies to new data centers. Like is there, and does it also determine, I would assume, where you decide to build?

Chris Walker - AWS 

Yeah, I mean kind of going back to that analysis we do in terms of where we're looking. Energy, real estate, localized community engagements, water. I would actually, you know, sometimes customer demand in terms of, not even sometimes, a lot of times we look at customer demand in terms of where we're locating our data centers. Those all kind of come into play.

The team that I was on previously, our real estate team, we would take and do a large six page analysis up to our executives and say, here's kind of the lay of the landscape in certain markets. And I think water certainly plays a piece of that now.

Molly Wood 

Right. And then what about energy mix in those locations? Like, there places that you just wouldn't, I mean, I think people are probably generally familiar now with the idea that when you get to 100 % renewable energy, it's not because everything is powered by its own little solar plants. It's because you're enabling renewable energy projects in other places. But are there places that you just flat out wouldn't build because the energy mix is so unfavorable?

Chris Walker - AWS 

Yeah, I don't know if we'd go for as far as that I think it honestly it depends on the customers right and then the customers again going back to our customer obsession That's where it really starts if customers and if the demand is In a certain location we would do the analysis like we would in any other place We would look at the the energy generation we would Kind of align the the water usage the inner generation the the fiber connectivity and the real estate along with the customer demand

and make a business decision as needed. think now more than ever, we're starting to really look at our transition from, like you said, we met our 100 % renewable goal. Now we're starting to look at other carbon -free energy sources, which include nuclear, utility scale, battery storage, and continue to our investments within the renewable space as well.

Molly Wood 

What do the customers get out of it? Like, sounds like you said you were responding to customer demand. Are customers saying we want a more sustainable solution? Are they saying we want a cheaper solution? Are they saying some mix of both? I mean, I'm sure they're not saying we want a more expensive solution. I'm not new here. Yeah.

Chris Walker - AWS 

Yeah. Right. They are not saying that. So I spend probably 50 % of my time on any given week with customers. And it's a very balanced conversation, usually with tension on both sides. And what I mean by that is they are looking for us to continue to grow and they're looking for more services from us.

but they want us to continue to make progress in our sustainability journey. And right now, if you kind of look at the industry landscape, that's a little bit at odds, right? In terms of power generation, especially. And so I think

Molly Wood 

Mm

Molly Wood

Wait, why is it at odds? Because the renewable energy generation just isn't at scale?

Chris Walker - AWS 

Renewable energy generation is certainly there, but for us to have a reliable power source for data centers, it needs to be consistent and constant. Renewable energy, by its given nature, solar and wind are variable power sources. And so you need to continue to scale battery storage solutions to further enable renewables. And there needs to be a high level of reliability for any of the power sources coming into our data centers.

Molly Wood

Right.

Molly Wood 

Got it.

Chris Walker - AWS 

And I think that's part of customers are starting to realize that a lot more than they have in the past. We're regularly sitting down with them and talking them through some of those problems, not problems, but some of those kind of new solutions that we're thinking through, especially within the data center industry landscape.

Molly Wood 

So they're literally coming and saying, what, we want you to build a data center nearby so that we can have faster cloud access. Walk me through the conversation that somebody, and also it has to be powered by 100 % renewables and your solar's intermittent.

Chris Walker - AWS 

Yeah, it's,

Chris Walker - AWS 

Yeah, yeah, at least so. So on my end, and I'm certain I don't sit with our sales executives as much I do every once in a while. On my end, they're looking for transparency usually is where it starts, right? They want to know the data. They want to know their footprint on us. And they also want to know what we're doing about our overall carbon footprint. And so I, you know, I regularly talk them through the progress we're making within our CCFT, which is again, their customer facing emissions.

Molly Wood 

Mm

Molly Wood 

Hmm. Got it.

Chris Walker - AWS 

tool that they can see their footprint on AWS services. We talk about the different aspects that we're taking, whether it's the water positive journey, whether it's our energy, renewable energy story and kind of how we're thinking about the evolution of renewable energy and carbon free energy. We also talk through different efficiency conversations, whether it's circular economy, whether it's our

how we are applying the efficiency space with and through our new chips. We sit and talk through quite a bit of the like progress that we're making in addition to the transparency in the data piece.

Molly Wood 

It it's interesting to hear you describe this as a driver of new business. Like, a competitive advantage, do you feel that  that that is the case for most customers? Some subset of customers? Like, are you going to lose business over not being sufficiently transparent or sustainable?

It's a tough, tough question to answer.  Um,

no.

hope so, but probably not.

Well,  yeah, I don't know how to answer it necessarily. I

mean, we are regularly  and constantly looking at being more transparent, right? Like, I think that is something that we know we need to continue to make progress in. Um, I don't think we're, we're not going to lose customers because we're, we're not being transparent because we're, we're constantly, we're showing them we're making progress in that space.

And I think the light goes with the sustainability story. I think it'd be 1 thing if we weren't making progress and showcasing our progress and the data wasn't backing us up. Um, I think it's, it's actually the opposite. We, we have the data we are showing, you know, our carbon emissions, uh, going down despite a year of growth and, and, you know, we were talking about efficiency and I, I, Meant to mention there was a recent study that shows that Accenture.

We  partnered with Accenture to show a study around energy efficiency and by moving workloads, I. T. workloads from on premise data centers onto the AWS data centers cloud. Um, it's up to 4. 1 times more efficient than traditional on prem data centers. And so I think those data points are really starting to come clear for customers.

They're seeing that it. You know, their associated carbon footprint can be reproduced up to 99 99 percent by shifting on the AWS. Um, I think it's again, they want to know more about the journey. They're now seeing some of the data. They now want to constantly be no. And honestly, communications key there as well.

I think, uh, Gone are the years where customers want to hear about it once, once a year, they want to hear about it regularly, whether it's through customer roundtables, whether it's through our field community members, whether it's through my organization, um, they, they want to be kind of brought along for the journey, not just informed of it.

CUT TO

Molly Wood

Interesting. I want to go back to designing a data center. What would go into, like if you're designing the perfect data center from the ground up, it sounds like it's light bulbs, it's concrete, it's chips. Talk to me about many to all of the kind of pieces that would go into that.

Chris Walker - AWS 

aspects. Yeah, so there's a couple of different areas in my previous role. We were again, I were kind of in charge of network deployment. And so I'll let you know why this part of the conversation matters. When we start thinking about rack placement within a data centers, right? Rack placement and rack positions are large rectangles and optimizing for where certain racks

fall within that data center will drive energy use up or down. And so my old organization started looking at if we pulled, you know, we're the dark fiber and dark fiber is just a fiber that's not being used into the data center, right? That usually connects the data center to the outside of internet. If we have the most compute heavy networking racks closer to the dark fiber,

And we are planning specifically around some of those larger networking racks. So they're closer to that. Will it drive energy efficiency up? And so we started, whether that was applying through certain workloads or looking at different designs, we started mapping our cable trays to maximize for some of those efficient routes. And so in the past, there would be, it would be done ad hoc project by project basis. rolled.

three or four racks in the place and we would connect cables down certain pathways, but honestly it was left up to the individual operators. We pulled those services in and started looking at what if we designed the cable trays day one for the entire room, right? How can we optimize and efficiently run it? Some of it is logistics based, right? How do you design a data center to make sure that the loading dock to the interior door

to the rack placement, right, if human efficiency is the most efficient route. Some of it is cable tray efficiency and networking efficiency. But when you start to answer your question, I know I'm jumping all around, but to answer your question, I think there's different areas. There's not only does future proofing your design, because I think that's what's critical. When you think about us and as a data center, we design our data centers to day one.

Molly Wood

I know, I like it,

Chris Walker - AWS 

we scale into our data center as customers demand. And so it's not like other data center, they'll build out their entire data center day one. We will build the initial footprint of our data center and the networking base. And as customers demand increase, we will continue to scale throughout our data center. And so when you think about racks.

Molly Wood 

Does that mean like a modular building or a building that has space to grow into? Modular infrastructure, yep, okay.

Chris Walker - AWS 

Modular infrastructure. And so cooling infrastructure, electrical infrastructure, HVAC equipment, we will modularly build and design that as we continue to grow out through the data center. And so...

Molly Wood 

So you're not cooling what you don't need to cool and you're not wiring unnecessarily. Yep.

Chris Walker - AWS 

Correct. Yeah, and power isn't being pulled into racks that aren't being used, right? I think that's the other critical piece is that allows us to see the power usage grow with the demand rather than seeing it all being used day one and some of it sitting idle. I think that's critical.

Molly Wood 

It's so interesting how basic this seems just like every other efficiency thing, right? It's sort of like, ugh.

Chris Walker - AWS 

Right. And then, you know, kind of going back to some of the logistics pieces, right? We also design not just for a single building on a site, we design for multiple buildings on a site. And so when you talk about trucks being pulled into the parking lot with racks being on place, the routes and the efficiencies in terms of how we are looking at the loading docks being placed, everything from loading dock to our interior red zone and our security zones, all the way through rack placement is usually

designed in mind with future proofing down the road for additional buildings to be taken into account.

CUT TO

Molly Wood 

Right. Is it fair to say that that's, you know, it's interesting because I think it almost becomes an argument for sustainable development that it costs less. Like maybe that's not why, and I'm sure companies are coming to you for both reasons. They want cheaper services, but they also have reporting requirements around sustainability or genuinely care. And I feel like it is important to highlight that efficiency often means both things.

Chris Walker - AWS 

Yeah, I think depending on the type of customer we're talking to, right, and you're kind of engaging with level of maturities is honestly all over the place. And that's okay. I think that's what's important, right? Is like at our scale, what we're delivering for our customers, know, democratizing the cloud and we're democratizing sustainability in general, in terms of making it available for customers large and small. I think

We get certain customers that are looking to continue to lower their costs and while continuing to increase their demands. Some customers are talking about constantly lowering their carbon emissions and their footprint. Others are just looking for just the right level of data for us to provide to them so they can in turn hand that to regulators. I think it's a little bit all over the place. I think what's consistent is they're looking for us to continue to make progress. They see us as again, kind of the light bearer within this industry. they know at our scale, if we can make the progress, it kind of showcases to not only our competitors, but the industry, the broader industry that you can make progress because, you know, we're doing it at arguably the largest scale.

Molly Wood 

Right. And this sort of, I think this touches on a question that a lot of people have, which is like, can you trust companies to make change? And, you know.

Chris Walker - AWS 

Hmm.

Molly Wood 

To what extent do you find that people believe you?

Chris Walker - AWS 

Yeah, I, I, I'm not gonna, I don't want to sound like a broken record, but I think it would be different if customer obsession wasn't our, our go -to leadership principle, right? Like I think for us, we, customers have seen us, whether it's through our website, whether it's through prime video, whether it's through AWS and the services we've offered, they've seen us respond to customer demand one way or the other. they've seen us take feedback into account. they've seen us respond to

industry changes or market changes. And we're doing the same within the sustainability space. I think as the energy consumption that you and I have talked about is continuing to grow, they're seeing us shift to efficiency because we know the energy use is going to continue to increase. And think on their behalf, the more that we can drive efficiency through our organization, only helps them on their journey.

CUT TO

Molly Wood 

Chris Walker, thanks so much for the time today. I appreciate it.

Chris Walker - AWS 

Thank you very much, Molly.

Molly Wood Voice-Over:

That's it for this episode of Everybody in the Pool. Thank you so much for listening.

Now much like changing all your light bulbs to LEDs can actually SERIOUSLY lower your electricity bill and your carbon footprint I get it. It’s not as satisfying as some big cool new invention or technology breakthrough that could change the whole game, right?

I mean still do it still rearrange those racks and re-route the dark fiber

But in the second half of this series we’re going to reimagine everything from how data centers get power at all hello little islands of fusion energy to how to design them completely differently so that cities states and entire countries don’t ban their construction completely.

Email me your thoughts and suggestions to in at everybody in the pool dot com and find all the latest episodes and more at everybody in the pool dot com, the website. And if you want to become a subscriber and get an ad free version of the show, hit the link in the description in your podcast app of choice.

Thank you to those of you who already have. See you next week.