In Cloud Computing This Week [Jan 20th 2023]

This week, Amazon Elastic Container Service (Amazon ECS) launched a new console experience for all users. The new Amazon ECS console makes it easier to deploy containerized applications, and configure load balancing, networking, and monitoring. The new Amazon ECS console also provides users with new workflows for effective operations and troubleshooting.

With the new Amazon ECS console, users can now use multi-step wizards and guided experiences to deploy and operate their applications and services without context switching to other tools. For example, users can now create a new load balancer or select an existing load balancer, when they create a new service.

The new console introduces new Amazon ECS features, like Amazon ECS Service Connect for simplified networking, and more. Users can leverage intuitive defaults or configure advanced capabilities with a built-in JSON editor. Customers can toggle back to the Classic Amazon ECS console, using a toggle button in the Amazon ECS console.

Amazon S3 File Gateway increases the number of supported file shares per gateway from 10 to 50 file shares. Previously, a single gateway could only support up to 10 file shares. With this release, gateway admins can optimize resources, and save time and money by creating and managing up to 50 file shares from a single gateway.

Amazon S3 File Gateway users who have underutilized gateway resources, can add more shares and support more end users with fewer gateways and resources. By having fewer resources dedicated to gateways, customers will also save on infrastructure costs. Having fewer gateways to manage will give gateway administrators more time to devote to other activities.

Amazon S3 File Gateway enables your on-premises applications to seamlessly store files as objects in Amazon S3, and access them using industry standard file access protocols such as NFS and SMB. You can use Amazon S3 File Gateway for migrating on-premises file data to AWS while maintaining fast on-premises access to recently accessed data, backing up on-premises file data to AWS, and providing on-premises applications low latency access to data stored in Amazon S3.

Amazon Relational Database Service (Amazon RDS) for MariaDB supports encrypted SSL/TLS connections to the database instances. Starting today, you can enforce SSL/TLS client connections to your Amazon RDS for MariaDB database instance for enhanced transport layer security.

To enforce SSL/TLS, enable the require_secure_transport parameter (disabled by default) through the Amazon RDS Management Console, the AWS CLI or the API. When the require_secure_transport parameter is enabled, a database client will be able to connect to the RDS for MariaDB instance only if it can establish an encrypted connection. 

require_secure_transport parameter is supported on RDS for MariaDB versions 10.5 and higher. To learn more about enforcing encrypted client connections using require_secure_transport parameter, please refer to the Amazon RDS User Guide.

Customers can now create file systems using Amazon Elastic File System (Amazon EFS) in the AWS Europe (Spain) Region.

Amazon EFS is designed to provide serverless, fully elastic file storage that lets you share file data without provisioning or managing storage capacity and performance. It is built to scale on demand to petabytes without disrupting applications, growing and shrinking automatically as you add and remove files.

Because Amazon EFS has a simple web services interface, you can create and configure file systems quickly and easily. The service is designed to manage file storage infrastructure for you, meaning that you can avoid the complexity of deploying, patching, and maintaining complex file system configurations.

Amazon CloudWatch announces support for cross-account Metric Streams. With Metric Streams, you can create a continuous, near real-time stream of metrics to a destination of your choice. With this new capability you can include metrics that span across multiple AWS accounts within an AWS region in a single Metric Stream. This helps to reduce the number of streams needed to collect metrics for a common destination. 

Metric Streams enables you to keep your Observability tools updated with the most recent metric data from your applications running on Amazon Web Services (AWS). You can use Metric Streams to send metrics to AWS Partner solutions including Datadog, New Relic, Splunk, Dynatrace and Sumo Logic, or to your data lake on AWS, such as Amazon Simple Storage Service (Amazon S3).

Now, using cross-account Metric Streams, you can consolidate metrics from many different AWS accounts into a common destination using a single Metric Stream. With this feature, you can simplify your configuration by having a single Metric Stream in your monitoring account that sends metrics from many source accounts to a single destination and further simplifies the management experience.

Cross-Account Metric Streams is now available in commercial AWS Regions, excluding the Amazon Web Services China (Beijing) Region, operated by Sinnet, and the Amazon Web Services China (Ningxia) Region, operated by NWCD. 

To get started, you first need to setup CloudWatch cross-account observability by connecting your source accounts with your monitoring account. Please refer to CloudWatch cross-account setup for the instructions. Once configuration is complete, you can login to the console using the monitoring account and navigate to the Metric Streams screen. You will see a new toggle option to enable a cross-account Metric Streams.

Alternatively, you can use the CloudWatch API, AWS SDK, AWS CLI, or AWS CloudFormation to provision and configure cross-account Metric Streams. Standard CloudWatch Metric Stream pricing applies, see Amazon CloudWatch pricing page for details.

Amazon OpenSearch Service now lets you validate configuration changes before applying them to your clusters. With the enhanced dry run option, Amazon OpenSearch Service checks for validation errors that might occur when deploying your configuration changes and provides a summary of these errors, if any. The dry run feature will also indicate whether a blue/green deployment will be required to apply a change, so that you can plan for these changes accordingly. 

Previously, when you submitted a configuration change, the domain status would change to ‘Processing’ and the validation errors identified while applying the changes would subsequently be displayed for your action while the domain continued to remain in the processing state, preventing further changes to your cluster until you took corrective actions. However, with the new enhanced dry run, you can check for possible validation errors before you actually apply a change to your clusters, and take corrective action as necessary.

You can use the dry run feature when applying a configuration change using the OpenSearch Service console, or by using the new ‘dryRun’ with ‘verbose’ parameter when calling an UpdateDomainConfig API. 

Enhanced dry run for configuration changes is now available in all AWS Regions where Amazon OpenSearch Service is available. Please refer to the AWS Region Table for more information about Amazon OpenSearch Service availability.

AWS Fault Injection Simulator (FIS) now supports higher resource quotas and quota adjustment using AWS Service Quotas. Quotas, also referred to as limits, are the maximum values for the resources, actions, and items in an AWS account. Previously, a maximum of 5 resources could be targeted by an FIS action. Now, you can adjust the maximum number of resources that some FIS fault actions can target using AWS Service Quotas. Quotas are now more granular so you can define quotas for specific combinations of FIS fault action and target resource type.

For example, you can increase the number of instances which terminate from 5 to 200. Actions with increased quotas include EC2 reboot instances, EC2 stop instances, EC2 terminate instances, ECS stop task, Spot instance interruptions, and Systems Manager send command, which supports CPU stress, memory stress, IO stress, kill process, network blackhole, network latency, and network packet loss.

FIS is a fully managed service for running fault injection experiments to improve an application’s performance, observability, and resilience. FIS simplifies the process of setting up and running controlled fault injection experiments across a range of AWS services, so teams can build confidence in their application behavior.

Amazon Kendra is an intelligent search service powered by machine learning, enabling organizations to provide relevant information to customers and employees, when they need it. Starting today, AWS customers can use the Amazon Kendra Microsoft (MS) Yammer Connector to index and search messages from MS Yammer.

Critical information can be scattered across multiple data sources in an enterprise, including messaging platforms like MS Yammer. Yammer is an enterprise social networking service that is part of the Microsoft 365 family of products. It is used mainly for private communication within organizations but is also used for networks spanning various organizations. Amazon Kendra customers can now use the Kendra MS Yammer Connector to index messages and search for information across this content using Kendra Intelligent Search.

Amazon S3 File Gateway now supports capturing, retaining, and enforcing DOS attributes listed in file metadata. Previously, files copied to the gateway were copied without this common metadata (Archive, Hidden, Read Only, and System). Support for DOS attributes is intended for users who want to hide files or folders, enforce read only access at the file or folder level, or mark files as archived once moved to Amazon S3.

Amazon S3 File Gateway users who archive data often need the data and its corresponding metadata to remain intact for consistency and compliance purposes. Other gateway users want to denote that files moved to Amazon S3 are now “archived”, when viewing the files on the gateway. In both cases, the gateway will now capture, retain, and enforce the metadata flags for A = Archive, H = Hidden, R = Read Only, and S = System.

AWS Customers can preserve attributes that their file system will enforce, even if the data moves to Amazon S3, is removed from the gateway, and then recalled back to the gateway from Amazon S3. Customers maintain complete control over these metadata flags even as they persist into Amazon S3.

Amazon S3 File Gateway enables your on-premises applications to seamlessly store files as objects in Amazon S3, and access them using industry standard file access protocols such as NFS and SMB. You can use Amazon S3 File Gateway for migrating on-premises file data to AWS while maintaining fast on-premises access to recently accessed data, backing up on-premises file data to AWS, and providing on-premises applications low latency access to data stored in Amazon S3.

AWS CloudFormation announces support for CloudFormation Linter (cfn-lint) in AWS Serverless Application Model Command Line Interface (AWS SAM CLI). The cfn-lint tool validates your SAM JSON/YAML template against CloudFormation-based rules, and returns diagnostic error messages. With this launch, you can use an optional parameter —lint in sam validate command to run cfn-lint validations on your SAM JSON/YAML templates.

You can use --lint to validate for CloudFormation-based rules. These rules checks for template attributes such as template size, template description limits, Fn::GetAtt parameters, Fn::If syntax structure, and others. For a complete list of default rules, refer to cfn-lint rules guideline.

This feature is available in US East (Ohio and N. Virginia), US West (N. California and Oregon), Africa (Cape Town), Asia Pacific (Hong Kong, Hyderabad, Jakarta, Mumbai, Osaka, Seoul, Singapore, Sydney, and Tokyo), Canada (Central), China (Beijing) operated by Sinnet, and China (Ningxia), operated by NWCD, Europe (Frankfurt, Ireland, London, Milan, Paris, Spain, Stockholm, and Zurich), Middle East (Bahrain, and UAE), South America (São Paulo), and AWS GovCloud (US-East) and (US-West) Regions.

Amazon EMR Serverless is a serverless option in Amazon EMR that makes it simple for data engineers and data scientists to run open-source big data analytics frameworks without configuring, managing, and scaling clusters or servers. Today we are introducing a new service quota called Max concurrent vCPUs per account. This vCPU-based quota allows you to set the maximum number of aggregate vCPUs your applications are able to scale up to within a Region.

With this feature, Amazon EMR Serverless provides you with two controls to help manage costs. You can view and increase this vCPU-based quota in the AWS Service Quotas Management console to set the maximum concurrent active vCPUs for all applications in your account. You can also configure the application property maximum capacity in terms of vCPU, memory (GB), and disk (GB) to define how much an individual application can scale up to.

Note that the application-level quota (Maximum active workers) that allowed you to set maximum capacity at an application level is deprecated as the application property maximum capacity allows you to do the same. For e.g. if you have 5 applications and each application can scale up to 1000 vCPUs, set the maximum capacity property to 1000 vCPUs for each application and request the account-level vCPU-based quota to be set to 5 * 1000 = 5000 vCPUs.

Amazon ElastiCache for Memcached has added support for Memcached version 1.6.17. This version is a cumulative update and contains all changes and improvements from version 1.6.12 to 1.6.17.

For the full list of improvements and bug fixes in Amazon ElastiCache for Memcached 1.6.17, see the release notes. You can create an Amazon ElastiCache cluster with Memcached 1.6.17 using the AWS Management Console, AWS CLI or the AWS SDK.

Memcached version 1.6.17 is now available in all AWS regions.

AWS Elemental MediaTailor is now available in the Africa (Cape Town), Asia Pacific (Mumbai), and US East (Ohio) regions. You may now configure and operate MediaTailor using the console or API endpoints within these new regions.

AWS Elemental MediaTailor is a channel assembly and personalized ad-insertion service for video providers to create linear over-the-top (OTT) channels using existing video content. The service then lets you monetize those channels—or other live streams—with personalized advertising across the broadest range of devices with a seamless viewer experience. MediaTailor functions independently or as part of AWS Media Services, a family of services that form the foundation of cloud-based workflows.

Starting this week, customers can use Amazon Elastic Block Store (EBS) direct APIs in the AWS Europe (Spain), Europe (Zurich) and Asia Pacific (Hyderabad) Regions to create EBS snapshots of their block storage data regardless of where it resides, including on-premises. 

Customers can use EBS direct APIs to backup their on-premises workloads to EBS snapshots that can be quickly and efficiently recovered into EBS volumes for use cases like disaster recovery. EBS direct APIs can also be used to write data directly to your snapshots, read data on your snapshots, and identify the differences or changes between two snapshots.

Amazon EC2 Auto Scaling now helps you easily determine whether adding a predictive scaling policy can further optimize the capacity of your Auto Scaling groups. Predictive scaling policies proactively add EC2 instances to your Auto Scaling group in anticipation of demand spikes. This results in better availability and performance for your applications that have predictable demand patterns and long initialization times.

It may also help alleviate the need for costly capacity buffers that you otherwise maintain to account for demand spikes. This new recommendation feature provides prescriptive indications about the potential impact of predictive scaling on your application availability and EC2 costs, making it simpler for you to get started with predictive scaling. 

Until now, customers had to manually determine the accuracy and the potential impact of predictive scaling for their various Auto Scaling groups. With this new feature, customers can get recommendation on whether predictive scaling does a better job at optimizing their EC2 capacity than their existing scaling configuration, based on up to 8 weeks of past data.

A non-mutative Forecast Only mode allows customers to better understand the impact of a predictive scaling policy without having to actually apply it to their Auto Scaling groups. With this information, customers can now make quick and informed decisions about using predictive scaling and the best configurations for it.

This week AWS were excited to announce that Amazon EMR has made it 30% faster to launch an EMR on EC2 cluster in a private subnet. Customers can get the faster cluster start-up times by simply relaunching their EMR on EC2 private subnet clusters. No further action is needed.

Amazon EMR is a cloud big data platform for data processing, interactive analysis, and machine learning using open-source frameworks such as Apache Spark, Apache Hive, and Presto. Additionally, with EMR, you can launch your EMR on EC2 clusters in a private subnet which makes your cluster inaccessible from the public internet.

With this launch, your EMR on EC2 clusters in private subnet will start-up 30% faster with no impact to performance. Faster start-up time are particularly helpful for customers running short-running jobs on transient clusters.

The Amazon Chime SDK now supports up to 250 webcam video streams per WebRTC session. The Amazon Chime SDK lets developers add intelligent real-time audio, video, and screen share to their web and mobile applications. Every WebRTC attendee can enable webcam video and view any combination of up to 25 webcam video streams from other attendees.

Developers can create tailored views for each session participant based on their role. For example, in an online learning application, the teacher’s view may include a panel of 10 students’ webcam videos that automatically scrolls through the entire class. The student’s view may focus on the shared content but also has dedicated spaces to display the webcam video from the teacher, any active talking student, and a few of their friends, as selected by the student.

To enable up to 250 webcam video streams, developers must first request an increase to the service quota, “maximum concurrent video streams published per meeting.” To learn more about the Amazon Chime SDK and its video capabilities, review the following resources:

• Amazon Chime SDK website
• Amazon Chime SDK for JavaScript, iOS or Android on GitHub
• Service Quotas in the Amazon Chime SDK Developer Guide

Amazon EC2 network performance metrics now supports a new metric to monitor available EC2 instance tracked connections, the ConnTrack Utilization metric. EC2 instance Security Groups act as stateful virtual firewalls to control incoming and outgoing traffic. These stateful firewalls track network connection information to enable return traffic to and from an instance to pass through. With this new metric customers have visibility into the number of ConnTrack entries remaining, which will allow them to proactively manage capacity and select the right instance size to meet emergent demand.

Prior to this announcement, customers had the capability to monitor dropped packets once the instance exceeded its tracked connection allowance via the EC2 instance network performance metrics. With this metric, customers could scale up their EC2 instances after they started seeing packet drops. With this launch, customers can now monitor their EC2 instance ConnTrack Utilization to proactively manage EC2 instance capacity with scale up or out actions to help meet network connections demand before dropping packets.

Customers can save on EC2 instance costs by scaling down or scaling in the instance fleet once the demand for tracked connections subsides. This new metric can also help customers benchmark a given workload for ConnTracks in pre-production environments to accurately assess EC2 instance production capacity needs.

The ConnTrack Utilization Metric (Conntrack_allowance_available) is available on Nitro based EC2 instances using the Linux driver for Elastic Network Adapter (ENA) starting from version 2.8.1, and it can be accessed from within the instance like other network performance metrics via the ethtool at no extra cost using simple command line tools.

Customers can also export this metric to AWS Cloud Watch using Cloud Watch agent or 3rd party observability tools. Customers can download the required Linux driver for ENA from the Amazon github repository. EC2 instance ConnTrack Utilization metric is available in all AWS Commercial Regions and AWS GovCloud (US) Regions.

Amazon CloudWatch now provides enhanced visibility into errors in Embedded Metric Format (EMF), with two new error metrics (EMFValidationErrors & EMFParsingErrors). 

Structured Log Format within CloudWatch Logs allows customers to emit metrics within their logs which are extracted and published to CloudWatch via EMF. With today’s launch, as customers are instrumenting their logs to leverage EMF, they will have visibility into errors in their instrumentation via the newly added error metrics. This enhanced visibility can help them quickly identify and remediate such errors, thereby simplifying the instrumentation process. 

Now customers can use EC2 Image Builder to create custom Amazon Machine Images (AMIs) that are hardened using Center for Internet Security (CIS) Benchmarks. EC2 Image Builder hosts CIS Benchmarks Level 1 for Amazon Linux 2, Red Hat Enterprise Linux (RHEL) 7, Microsoft Windows Server 2019, and Microsoft Windows Server 2022.

You no longer have to manage your own custom scripts for CIS Level 1 hardening of images with these operating systems. With this feature, you can also choose to automatically update AMIs to the latest version of the CIS standards as they become available.

To add security hardening, you simply subscribe to the required CIS AMI in the AWS Marketplace from the EC2 Image Builder Console and use that CIS AMI as your base AMI for the image customization process. Your AWS Marketplace AMI subscription for the CIS AMI will unlock access to the CIS hardening components in EC2 Image Builder.

You can use these components to security harden your AMIs to the recommended CIS Benchmarks, and can view CloudWatch build logs for the hardening process in EC2 Image Builder.

This feature is available in all AWS Regions, including the AWS GovCloud (US) Regions, but excluding AWS China regions (Beijing, operated by Sinnet), and China (Ningxia, operated by NWCD). Get started on this feature from the EC2 Image Builder Console, CLI, API, CloudFormation, or CDK, and learn more about the service in the EC2 Image Builder documentation.

You can find specific information about CIS and compliance features in EC2 Image Builder on the feature documentation page. Also, learn more about upcoming EC2 Image Builder features on our public roadmap.

This week, AWS released two new AWS managed policies for Amazon Detective. AWS managed policies make it easier for users to gain the proper level of permissions to leverage the service for security investigations. AWS managed policies are maintained by AWS to reduce work for customers in managing access permissions for users in specific job roles. For more information on AWS managed policies, you can read AWS managed policies in the IAM User Guide.

AmazonDetectiveMemberAccess is a new AWS managed policy that allows users to view invitations to Detective’s behavior graph, accept or reject invitations, and view how usage contributes to cost. AmazonDetectiveInvestigatorAccess is a new AWS managed policy designed for security analysts who need to conduct full security investigations, archive Amazon GuardDuty findings, but not manage member accounts. 

We also updated permissions to the current AWS managed policy AmazonDetectiveFullAccess to ensure assigned users can see the full details of GuardDuty findings in the Detective console. To learn more about the new AWS managed policies and permissions, visit the Detective documentation page.

There is no additional charge for these new AWS managed policies, and they are available today for existing and new Detective customers

Now AWS customers can search AWS Marketplace Amazon Machine Images (AMIs) directly in the EC2 Image Builder Console and use those AMIs as base images in their image build workflows. Those AMIs can be accessed and used via the EC2 Image Builder Console, CLI, API, CloudFormation and CDK interfaces. This feature makes it easier for you to seamlessly track and integrate your AWS Marketplace AMI subscriptions in your image customization workflows.

With this feature, you no longer have to navigate away from the EC2 Image Builder Console to the AWS Marketplace Console and search for your compatible AMI products for your image pipeline. You can directly look up required Marketplace AMI from the EC2 Image Builder Console, subscribe to the AMI on AWS Marketplace, and use that AMI as a base image for your image build pipeline. Your subscribed AWS Marketplace AMIs are available in the subscriptions section in the EC2 Image Builder Console.

On January 17, 2023 Amazon announced quarterly security and critical updates for Amazon Corretto Long-Term Supported (LTS) versions of OpenJDK. Corretto 19.0.2, 17.0.6, 11.0.18, 8u362 are now available for download. Amazon Corretto is a no-cost, multi-platform, production-ready distribution of OpenJDK.

Click on the Corretto home page to download Corretto 8, Corretto 11, Corretto 17, or Corretto 19. You can also get the updates on your Linux system by configuring a Corretto Apt or Yum repo.

AWS Network Firewall now supports IPv6 for dual stack subnets so you can filter IPv4 and IPv6 traffic flows to and from the public internet, on-premises network, or any endpoint in your Amazon Virtual Private Cloud (VPC). Now, you can use AWS Network Firewall to protect your IPv6 workloads on AWS.

AWS Network Firewall is a managed firewall service that makes it easy to deploy essential network protections for all your Amazon VPCs. With this capability, you can enable AWS Network Firewall endpoints to filter both IPv4 and IPv6 traffic in dual stack subnets. A dual stack subnet is a subnet with both an IPv4 CIDR block and an IPv6 CIDR block. Resources in a dual-stack subnet can communicate over IPv4 and IPv6.

There is no additional cost to enable dual stack AWS Network Firewall endpoints. You can configure dual stack firewall endpoints using the AWS Management Console, AWS CLI, AWS SDK, or the AWS Network Firewall API. IPv6 is supported in all AWS Regions where AWS Network Firewall is available today, including the AWS GovCloud (US) Regions.

Amazon Elastic File System (Amazon EFS) has increased the maximum number of Access Points per file system from 120 to 1,000, enabling you to control file system access permissions across a larger number of applications in multi-tenant environments.

Amazon EFS Access Points are application-specific entry points into your file system which are designed to enforce a specified POSIX identity and root directory for connected clients. You can use EFS Access Points to more easily isolate data between applications sharing a single EFS file system. With this launch, you can now utilize up to 1,000 EFS Access Points per EFS file system for higher scale multi-tenant environments.

Now deploy patch policies across AWS accounts and AWS Regions using AWS Systems Manager Patch Manager and AWS Organizations.

This week, AWS introduced Patch Policies, a new capability of AWS Systems Manager Patch Manager. Patch Policies provide a user experience in a single console to easily define and enforce patch compliance across accounts and Regions with a few clicks.

With this launch, you can now make sure all instances in your AWS Organization scan and install patches based on centrally defined patch rules from an AWS Organization management account. You can create and manage multiple Patch Policies at once, enabling you to control patching operations for different sets of instances across accounts and Regions. 

To get started, navigate to Patch Manager in the Systems Manager console and create a Patch Policy. Patch Policies automate the process of scanning and installing patches and can be deployed across the entire AWS organization or for specific organizational units (OUs) and nodes.

You can define Patch Policies based on AWS recommended configuration and patch rules or customize them to suit your requirements. You can view instance patch compliance by navigating to AWS Systems Manager Explorer.

Dialogflow

Vertex AI  

Public Preview: Azure App Health Extension - Rich Health States

Application Health Extension, Rich Health States allows for more detailed health reporting on your VM applications.

Azure Machine Learning - Generally availability updates for January 2023

New features in GA include the ability to place customized tags, search for machine learning assets, isolate network for managed online endpoints, build custom metrics views, and simplify data security. 

Azure Machine Learning - Public preview announcements for January 2023

New features now available in Public Preview include the ability to build an end-to-end training pipeline with no-code, recover a deleted workspace, and identify root causes of pipeline failure.

General Availability: Active Directory Connector for Arc-enabled SQL MI

Active Directory Connector (ADC) for Arc-enabled SQL Managed Instance is now available to perform Active Directory authentication.

Public preview: Tribal Group events available to Azure Event Grid customers

You can now subscribe to events Tribal Group applications and automate your processes based on Tribal Group application state changes.

Generally available: Azure Active Directory authentication for SQL Server 2022

Customers with SQL Server 2022 can enable Azure Active Directory authentication, single sign-on, unify SQL Server authentication and leverage multi-factor authentication.

Public Preview of Viewing SQL Server Databases via Azure Arc

Surfaces all the active databases and their configurations for each of the Arc-enabled SQL Servers in Azure.

Public Preview: Microsoft Purview access policies for SQL Server 2022

Author policies that govern the access to Active Directory users directly from Azure to all your SQL Server 2022 data sources at scale.