By: Craig Kennedy
Amazon Web Services (AWS) wrapped up its 11th annual re:Invent conference in Las Vegas last week (11/28/22 – 12/02/22). Amazon.com CTO Dr. Werner Vogels gave the final keynote on Thursday morning, making the final series of new announcements for the week.
Each announcement was interesting, however the overarching theme of the announcements throughout the conference really resonated with me.
Customers Start Doing Things You Never Expected Them to Do
Dr Vogels described a particular customer use case where vast amounts of data was being generated on a daily basis and needed to be analyzed to provide a few specific parameters within this vast data source.
To provide some scale, around 37GB of data was created daily and was distributed globally into 500,000 files.
As a developer, the reasonable approach was to use AWS Kafka and AWS Elastic Map Reduce (EMR).
Kafka could easily handle the volume and distributed nature of the data, and EMR could consume the data from the Kafka pipelines, analyze all the data, and get the responses they wanted. Seems simple, right?
This Is Too Complex—It Needs to be Simpler
The response from the customer was a resounding NO, they wanted something way simpler. They wanted to do this using Amazon Step functions with just a couple of AWS Lambda calls, that’s it. Seems simple, right?
The problem was, when AWS built Step Functions, this was not a use case they had anticipated.
Listen, Learn, Improve—The Customer is Always Right
Dr. Vogels spoke to the importance of listening to the customer and understanding not only the problems they want to solve, but how they want a solution that can solve it as easy as possible.
This use case drove the development supporting the announcement in the keynote of the general availability of “AWS Step Functions Distributed Map” which enables the orchestration of large-scale parallel workloads in serverless applications, meaning a couple of Lambda functions using Amazon Step functions, just what the customer was looking for.
And, to make this seemingly great solution even better, AWS will only charge for the filtered results meaning that regardless of the incoming data pipeline size. In this case 37GB, it’s the result set that you will be paying for.
Usability Is the New Technology
Many of the announcements in this year’s AWS re:Invent followed this theme of “simplification” and “ease of use”. This is noticeably different from prior years where the majority of AWS announcements were very feature and function based, focusing heavily on technology.
Announcement after announcement this year was less focused on wowing the participants with new leading-edge technology and more focused on improving ease-of-use for their customers by leveraging existing AWS technologies with simplified user interfaces to abstract the details of the technology stack.
The Power of Composing = The Power of Linux = Pipes
One additional announcement that resonated with me, probably due to my deep background with Linux, was the announcement of Amazon EventBridge Pipes. This new offering allows you to stitch together AWS microservices by ‘piping’ the output of one microservice to the input of another, much in the way you ‘pipe’ standard output from one Linux command to the standard input of another command. It’s really that simple.
Amazon EventBridge Pipes supports data flow between microservices in a direct point-to-point or producer-to-consumer method, or you can filter the data stream through lambda functions, much in the same way you’d parse Linux output with pipes.
Again, adhering to the theme of making it easier on the user to consume technology.
Bottom Line
Amazon has had a history of focusing on technology toolkits and leaving it up to professional developers to assemble the pieces together to make a functioning business application.
The age of low-code and no-code development is rapidly gaining traction and AWS is getting on the bus and offering solutions that appeal to both developers and non-developers to easily consume AWS functionality, and in this analyst’s opinion it’s about time.
SEE CRAIG LIVE TOMORROW AT TRANSFORM 2022!
Aragon Research’s analysts will be discussing, ‘The Top Ten Technologies for 2023 and 2028? on Thursday, December 8th at 10 AM PT / 1 PM ET.
To see the full list LIVE, register here!
SEE CRAIG LIVE AT HIS UPCOMING WEBINAR
Cloud Computing—Why Are Costs So Out of Control
Next week, Sr. Director of Research, Craig Kennedy, will be hosting a live webinar, titled, Cloud Computing—Why Are Costs Out of Control?.
Technology tools are pricey these days. This means careful consideration of the technology tools that are worth it for your enterprise is needed. Craig will be covering why Cloud Computing costs are astronomically high and when it makes the most sense for the digital enterprise.
Don’t miss out on our live Q&A session to get your questions answered by Craig.