Rehashing re:Invent

The holiday season is a very merry time for everyone. You have Christmas to look forward to. And in addition to this, when it comes to late November, there is something else I have been looking forward to since 2012 (coincidently, I started my journey as a techie around the same time.) Yes, you guessed it right. I am talking about AWS re:Invent. Since its first edition, re:Invent has been an exciting conference for the computing community around the globe. Not only the actual attendees but millions around the world are excited to see the launch of new services that are going to directly or indirectly influence the way we compute. And the best thing is you don't need to attend the event in person. There are things like keynotes and re:Invent app that keeps you posted with all the exciting stuff.

More than 450 sessions, chalk talks, and lots of new service announcements were the highlights this time. There were several major announcements around machine learning, databases, hybrid cloud, and account management. It will be impossible to write about all of them in a single blog. So, I will restrict myself to some of the highlighting areas I found interesting.

Compute Instances

One of the reasons to love AWS re:Invent is its affection for compute instances. You must have observed that AWS always comes up with new instances at re:Invent. This time we saw the launch of Amazon EC2 A1, the first AWS Graviton powered instance that will be best suited for ARM-based processors. And C5n Instances that will allow up to 100Gbps of network bandwidth.

Another interesting update is now you can hibernate your EC2 instance. This is welcoming news for those EC2 users who want to scale quickly without warming up new VMs, as they can now hibernate a running (Amazon Linux) instance and wake it up when needed. Of course, users will only pay for storage and Elastic IPs when the VM is in a hibernated state.

Storage and File Systems

AWS announced a new storage class for S3 called "S3 Intelligent Tiering" in which S3 contents will be moved automatically to a lower-priced storage when it is not accessed frequently. It will move the object that has not been accessed for 30 consecutive days to an infrequent tier. In addition to this, now you can lock an object during a customer-defined period to support data retention requirements.

There was one more announcement in the file system area - Amazon FSx for Lustre, a fully managed file system optimized for compute-intensive workloads, such as high-performance computing and machine learning. You can leverage the scale and performance of FSx for Lustre to process your file-based datasets from Amazon S3 or other durable data stores. Plus, you have Amazon FSx for Windows File Server, a fully managed Windows file system built on native Windows file servers.

Database (SQL/NoSQL)

Database is another area of interest for most of us. Here are some of the highlighting releases:

• Amazon DynamoDB Transactions: DynamodB transaction offers ACID (atomicity, consistency, isolation, and durability) properties across one or more tables within a single AWS account and region. It will elevate the scalability, performance, and other benefits of DynamoDB to another extent.
• Amazon DynamoDB On-Demand (Pay per request pricing and no need for capacity planning): On-demand pricing for DynamoDB is a billing option for DynamoDB. It allows you to pay only for what you have used for each read-and-write request. It doesn't require up-front capacity planning, but it is capable of serving thousands of requests per second.
• Amazon Aurora Global Database: A new feature of Amazon Aurora that will allow a single Aurora database to span across multiple AWS regions. This includes faster replication, enables the low-latency global read and helps in disaster recovery across regions. It has a typical latency of less than 1 second with dedicated infrastructure fully available to serve the application.
• Amazon Timestream: A time series database similar to InfluxDB used to collect, store, and process data such as server and network logs, sensor data and data from IoT. It can process trillions of events per day at 1/10th cost of a relational database.
• Amazon Quantum Ledger Database: It is a fully managed ledger database that provides transparent, immutable transaction logs and helps you to secure your application's data.

Serverless

Serverless was one of the hot topics in this edition of AWS re:Invent. Releases and updates in this area included:

• Lambda layers: To centrally manage code and data that is shared across multiple functions. This will promote component sharing and will keep the core function package as small as possible.
• Application load balancer: Now supports invoking Lambda function to serve HTTP/S request.
• AWS Step function: A fully managed workflow service and a reliable, repeatable way to connect and coordinate activities that will enable you to think and work at a high level. And all this while keeping your business logic separate from your workflow logic. Once you are done with the design and test part for your workflows (which we call state machines), you can deploy them at scale with tens or even hundreds of thousands running independently and concurrently.
• AWS Firecracker: A new virtualization open source project that makes use of KVM. It will help you to launch lightweight micro-virtual machines in a non-virtualized environment, and it is used by lambda for sandboxing the functions. It might turn out to be a good one for running multi-tenant Container workloads, but how it will stand with Docker will be a thing to see.

Application/Container based services

Use AWS CodeDeploy to implement blue/green deployments for AWS Fargate and Amazon ECS: So you can now reduce your downtime during application updates using this service. This is basically for services hosted on AWS ECS or Fargate containerized applications. It will allow you to launch a new version of your application along with the old version and test the new version before rerouting the traffic to it.

Analytics and Data Processing

Amazon Managed Streaming for Kafka (in preview mode) Amazon managing stream for Kafka which provides a highly available, secure service, making it easy for developers to run their applications on Apache Kafka. This saves you from installing external Kafka services.

Migration and Transfer

Migrating your data and logs can be a tiresome process. So to help you out, here are some releases from AWS:

• AWS DataSync: Allows you to automate/transfer the data from on-premises to S3 or EFS. It automatically handles the data sync and allows transfer of data to AWS at speeds up to 10 times faster than open-source tools available.
• AWS SFTP for Amazon S3: A fully managed service that enables the transfer of files directly into and out of Amazon S3 using the Secure File Transfer Protocol (SFTP), also known as Secure Shell (SSH) File Transfer Protocol. It helps you migrate your file transfer workflows to S3 for SFTP.

Machine Learning/Artificial Intelligence

Machine learning was one of the focus areas for this year’s re:Invent. AWS announced that their own Machine Learning University will be available to all developers. There were multiple announcements related to Machine Learning and Artificial Intelligence, particularly Amazon SageMaker, SageMaker RL (for data scientists with limited data sets for training and an interest in reinforced learning), and AWS Marketplace for Machine Learning (more than 150 algorithms and models that can be deployed directly to Amazon SageMaker.)

Apart from these, AWS launched an exhilarating line of services in areas like IoT, Robotics, and Blockchain. Some of them are still in preview mode, but it will be interesting to see AWS taking on some of the big names in these areas. I have covered services that I am looking forward to getting my hands on. To know more about all the releases, you can visit the AWS re:Invent product announcements page.