This week's roundup of all the cloud news.
Here's a round up of all things GCP, Azure and AWS for the week ending Friday 2nd July 2021.
To stay in the loop, make sure you subscribe on the right - There's a new Newsletter series starting soon that will keep you up to date with all our new releases, enhancements and capabilities and will also showcase lesser known but powerful features that you may not be aware of.
Of course we'd love to keep in touch at the usual places. Come and say hello on:
AWS Updates and Releases
AWS AMPLIFY CLI ADDS SUPPORT FOR STORING ENVIRONMENT VARIABLES AND SECRETS ACCESSED BY AWS LAMBDA FUNCTIONS
Amplify CLI now supports storing environment variables and secrets to be used in AWS Lambda functions to help separate environment-specific configurations from business logic. The AWS Amplify CLI is a command line toolchain that helps frontend developers create app backends in the cloud that often include business logic powered by AWS Lambda functions. Customers use environment variables to store environment-specific values, such as API endpoints, and secrets to securely store sensitive information, such as API keys.
AWS ParallelCluster is a fully supported and maintained open source cluster management tool that makes it easy for scientists, researchers, and IT administrators to deploy and manage High Performance Computing (HPC) clusters in the AWS cloud. HPC clusters are collections of tightly coupled compute, storage, and networking resources that enable customers to run large scale scientific and engineering workloads.
AWS are excited to announce that Kendra Enterprise Edition is now offered in smaller more granular units to enable smaller workloads. The base Kendra Enterprise Edition (KEE) now starts at $1.40/h ($1,008/mo), offering the same functionality and availability at a smaller scale and cost. The base KEE now supports up to 100,000 documents and 8,000 searches per day, with adaptive bursting capability to better handle unpredictable query spikes. Similarly, the Virtual Storage Capacity units now offer scaling in increments of 100,000 documents (up to 30GB) per unit at $0.7/h. Our Virtual Query Units offer scaling increments of 8,000 queries per day at $0.7/h.
With this launch, AWS Amplify now supports conditional backend builds, automatic build-time aws-exports.js generation, and a simpler console workflow to re-use backends across multiple frontend branches. AWS Amplify’s continuous deployment service offers frontend developers the fastest way to build and deploy full-stack apps (both the frontend and backend) on every code commit.
You can now deploy Amazon AppStream 2.0 in the AWS Europe (London) Region. Deploying AppStream 2.0 in your local region provides users with a more responsive experience and helps support your local data residency obligations. With this launch, you can deploy General Purpose, Compute Optimized, Memory Optimized, Graphics Design, Graphics Pro and Graphics G4 instances to meet the needs of your users.
You can now share files directly with specific experts in AWS IQ chat interface. Experts can also share files in the chat, attach files to a proposal, or to a payment request after completing work.
We are delighted to announce the addition of Tag Tamer to the AWS Solutions Implementations portfolio. AWS Solutions Implementations help you solve common problems and build faster using the AWS platform.
AWS now supports three additional Amazon Elastic Block Store (Amazon EBS) volume types in the Los Angeles AWS Local Zones: General Purpose SSD (gp3), Throughput Optimized HDD (st1), and Cold HDD (sc1) volumes. With this expansion, you can now use gp3, st1, and sc1 volume types in addition to your existing gp2 and io1 volumes in the Los Angeles AWS Local Zones.
AWS Storage Gateway is now available in the AWS Asia Pacific (Osaka) Region
When using AWS DataSync to copy data between Amazon FSx for Windows File Server file systems and Server Message Block (SMB) shares, you can now copy object metadata describing NTFS system access control lists typically used by administrators to define the operations on files and folders that generate audit logs. With this launch, DataSync can now copy all access control entries (ACEs) for files and folders between your SMB shares and FSx for Windows File Server file systems as well as between FSx for Windows File Server file systems.
AWS Glue DataBrew now supports writing datasets from job runs directly into the AWS Glue Data Catalog
AWS Glue DataBrew now supports the ability to write datasets created from jobs that run your data preparation recipes directly to the AWS Glue Data Catalog. You can choose to store datasets in Amazon S3, Amazon Redshift, and Amazon RDS (Aurora, Oracle, SQL Server, MySQL, and PostgreSQL) tables in the Data Catalog.
We are excited to announce an update to Amazon GameLift, an AWS managed service. Trusted by game companies from all over the world like Wargaming, Metalhead Software, and Illfonic, GameLift FlexMatch launched in 2017 as a feature that uses a powerful matchmaking algorithm and flexible developer-defined rules to create high-quality matches at AWS scale.
Self-service features and trusted device support are now available on the Amazon WorkSpaces Android Client App
Self-service features and trusted device support are now available on the WorkSpaces Android Client app on Google Play that runs on Android and Android-compatible Chrome OS devices . The new features are intended for customers who use Android or Android-compatible Chrome OS devices as primary WorkSpaces client endpoint devices.
AWS Glue Schema Registry now supports defining schemas in JSON Schema format in addition to Apache Avro, allowing customers who choose JSON Schema as the format for their streaming data to centrally control the evolution of data streams and avoid having to manage their own registries. Through Apache-licensed serializers and deserializers, Glue Schema Registry integrates with Java applications developed for Apache Kafka/Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon Kinesis Data Streams, Apache Flink/Amazon Kinesis Data Analytics for Apache Flink, and AWS Lambda.
This week, AWS announced the General Availability of the Amazon ECS-optimized Bottlerocket Amazon Machine Image (AMI). Bottlerocket is an open source Linux-based Operating System (OS) that is purpose-built to run containers. Bottlerocket includes only the software needed to run containers and comes with a single step update mechanism. This enables you to improve security posture and reduce maintenance overhead for your Amazon ECS clusters. With this release, Amazon ECS also helps you automate OS updates for Bottlerocket, helping you improve application availability and reduce disruption during updates.
AWS are excited to announce that they now offer a new framework-specific Deep Learning AMI (DLAMI). Today we launched framework-specific DLAMIs for PyTorch 1.9.0 and TensorFlow 2.5.0, with support for Amazon Linux 2, Ubuntu 18.04, and Ubuntu 20.04. The AWS DLAMIs provide machine learning practitioners and researchers with the infrastructure and tools to accelerate deep learning in the cloud, at any scale. These optimized new images are up to 60% smaller in size, accelerating the time for machine learning practitioners to launch DLAMIs on AWS. We will continue to release and support DLAMI in the previous format that includes multiple frameworks and operating systems within a single AMI.
As you prepare your data, AWS Glue DataBrew adds support to automatically identify and mark advanced data types for columns, making it easy to normalize columns containing data of types: Social Security Number (SSN), Email Address, Phone Number, Gender, Credit Card, URL, IP Address, Date and Time, Currency, Zip Code, Country, Region, State, and City. Additionally, DataBrew visually marks columns containing Personally Identifiable Information (PII), allowing you to easily scan for all PII columns in your dataset and apply transformations. Learn more about all supported advanced datatypes .
Amazon Relational Database Service (Amazon RDS) for MariaDB now supports MariaDB minor versions 10.2.37 , 10.3.28 , 10.4.18 , and 10.5.9 . We recommend that customers upgrade to the latest minor versions to fix known security vulnerabilities in prior versions of MariaDB, and to benefit from the numerous bug fixes, performance improvements, and new functionality added by the MariaDB community. Amazon RDS customers can leverage automatic minor version upgrades to automatically upgrade their databases to the latest minor version during scheduled maintenance windows. Learn more about upgrading your database instances, including automatic minor version upgrades, in the Amazon RDS User Guide .
AWS Lambda now supports SASL/PLAIN authentication for functions triggered from self-managed Apache Kafka
AWS Lambda functions that are triggered from self-managed Apache Kafka topics can now access usernames and passwords secured by AWS Secrets Manager using SASL/PLAIN, a simple username/password authentication mechanism that is typically used with TLS for encryption to implement secure authentication. This is in addition to SASL/SCRAM, which is already supported on Lambda. To get started, customers who select Apache Kafka as the event source for their Lambda function can choose SASL/PLAIN as their authentication mechanism, and select their credentials from Secrets Manager on the AWS Management Console, AWS CLI or AWS SDK for Lambda. This feature requires no additional charge to use, and is available in all AWS Regions where self-managed Apache Kafka is supported as an event source for AWS Lambda. To learn more about using SASL/PLAIN authentication for your Lambda functions triggered from Amazon MSK topics, read the Lambda Developer Guide .
AWS Elemental MediaConvert now supports creating AVC-Intra, VC3, and XAVC mezzanine formats carried in the MXF container. Often referred to as “intermediate” or “editing” file formats, these lightly compressed video codecs balance quality and performance by providing visually lossless compression, lightweight decoding requirements, and native editing and playback support in non-linear editing tools.
Starting this week, AWS Firewall Manager is available in Asia Pacific (Osaka).
AWS Identity and Access Management (IAM) Access Analyzer makes it easier for customers to author secure and functional permissions by providing over 100 policy checks with actionable recommendations during policy authoring. Now, IAM Access Analyzer extended policy validation by adding new policy checks that validate conditions included in IAM policies. These checks analyze the condition block in your policy statement and report security warnings, errors, and suggestions along with actionable recommendations. These checks help you set fine-grained permissions by guiding you to apply conditions in a secure and functional way. For example, IAM Access Analyzer validates that policies that specify tagging conditions include the required tag information in the condition.
AWS Elemental MediaConvert has added support for HDR10+ metadata analysis and insertion. HDR10+ is a technology that further enhances HDR video with dynamic color metadata for viewing devices to adjust automatically on a scene-by-scene basis. Using MediaConvert you now have the option to perform a HDR10+ analysis on your HDR10 content and then to insert the resulting metadata into the encoded outputs for consumption on HDR10+ enabled devices. This metadata is treated like an enhancement layer to the base HDR10 metadata and allows for backwards compatibility with HDR10 compliant devices.
AWS CloudFormation Modules are now available in AWS GovCloud (US). A Module encapsulates one or more AWS or Non-AWS resources and their configurations for reuse across your organization.
Session Manager, a capability of AWS Systems Manager, now supports free text search for nodes in the search bar of the Session Manager console. Customers can easily locate the node to connect to without knowing the exact value of a property such as Name or Instance ID. The free text search is supported for multiple properties of a managed node, including Instance ID, Name, Agent Version, Platform, Status, and many more.
With the Apple Business Chat integration in Amazon Connect, your customers can interact with you using the Apple Messages application on their iPhone, iPad, or Mac. Your customers can now have an experience that is as familiar and convenient as chatting with a friend, while using rich customer service features like interactive messages to do things like schedule appointments. Apple Business Chat makes it easy for your customers to chat with you anytime they click on your registered phone number on an Apple device. The Apple Business Chat integration allows you to use the same configuration, analytics, routing, and agent UI that you're already using for Amazon Connect Voice and Chat.
AWS WAF now supports 15 additional text transformations, allowing you to reformat web requests to remove any unusual formatting, or sanitize input before rule evaluation. It can be used to identify threats that may be obscured by attackers in an effort to bypass detection. You can use these new text transformations with WAF rule statements, such as SQLi detection, string match, and regex pattern set. You can chain up to 10 text transformations together in a single rule statement. Once configured, AWS WAF will apply the transformations first before evaluating the rule statement.
Google Cloud Releases and Updates
Anthos Clusters on Azure
The preview release of Anthos clusters on Azure is now available. With this release, you can create, use, and tear down Anthos clusters on Azure, as well as load balancers, and storage volumes.
Anthos clusters on Azure is available for customers with an existing support relationship with Google Cloud. Contact your account representative for access.
Anthos clusters on Azure supports Kubernetes version 1.19.10-gke.1000.
To create a cluster, see the Installation overview.
New features include:
- Private clusters with private IPs
gcloud alpha container azure clustersand
- Application-layer secrets encryption
- Choice of volume type, size, and customer-managed encryption keys
- Cluster Autoscaler
Anthos clusters on VMware
Anthos clusters on VMware 1.8.0-gke.21 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.8.0-gke.21 runs on Kubernetes v1.20.5-gke.1301.
The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.8, 1.7, and 1.6.
Preview: Cluster autoscaling is now available in preview. With cluster autoscaling, you can horizontally scale node pools in proportion to workload demand. When demand is high, the cluster autoscaler adds nodes to the node pool. When demand is low, the cluster autoscaler removes nodes from the node pool, scaling back down to a minimum size that you designate. Cluster autoscaling can increase the availability of your workloads while controlling costs.
Preview: User cluster control-plane node and admin cluster add-on node auto sizing are now available in preview. The features can be enabled separately in user cluster or admin cluster configurations. When you enable user cluster control-plane node auto sizing, user cluster control-plane nodes are automatically resized in proportion to the number of node pool nodes in the given user cluster. When you enable admin cluster add-on node auto sizing, admin cluster add-on nodes are automatically resized in proportion to the number nodes in the admin cluster.
Preview: Windows Server container support for Anthos clusters on VMware is now available in preview. This allows you to modernize and run your Windows-based apps more efficiently in your data centers without having to go through risky application rewrites. You can use Windows containers alongside Linux containers for your container workloads. The same experience and benefits that you have come to enjoy with Anthos clusters on VMware using Linux--application portability, consolidation, cost savings, and agility--can now be applied to Windows Server applications also.
Preview: Admin cluster backup is now available in preview. With this feature enabled, admin cluster backups are automatically performed before and after user and admin cluster creation, update, and upgrade. A new
gkectl backup admincommand performs manual backup. Upon admin cluster storage failure, you can restore the admin cluster from a backup with the
gkectl repair admin-cluster --restore-from-backupcommand.
The Ubuntu node image is qualified with the CIS (Center for Internet Security) L1/L2 Server Benchmark.
Generally available: Workload identity support is now generally available. For more information, see Fleet workload identity. The connect-agent service account key is no longer required during installation. The connect agent uses workload identity to authenticate to Google Cloud instead of an exported Google Cloud service account key.
You can now use
gkectlto rotate system root CA certificates for user clusters.
You can now use
gkectlto update vCenter CA certificates for both admin clusters and user clusters.
Preview: You can enable Secrets encryption with internally generated keys instead of a hardware security model (HSM). This feature will be enabled by default in a future release.
Anthos Config Management
Config Sync now supports accessing Cloud Source Repositories through a Google service account when Workload Identity is enabled in your cluster. To learn more, see Granting Config Sync read-only access to Git.
Config Management is now available on GKE. Config Management enables you to use Policy Controller. GKE users can also now install Config Sync using the Cloud Console or by using the gcloud command-line tool. To learn more, see Installing Config Sync.
You can now configure your cluster with the same settings used by another cluster by using
gcloud fetch-for-apply. To learn more, see Configuring Config Sync.
Config Sync cluster selectors now support CustomResourceDefinitions.
Anthos Service Mesh
Anthos Service Mesh user authentication is now generally available (GA). This feature lets you use existing Identity Providers (IDP) for user authentication and access control to your workloads. For more information, see Configuring Anthos Service Mesh user authentication.
Is now available in asia-south2
BigQuery now supports multi-statement transactions. These allow you to perform mutating operations, such as inserting or deleting rows, on one or more tables, and either commit or roll back the changes atomically.
BigQuery now supports access management data control language (DCL) statements and corresponding views:
BigQuery now supports the following casting features:
These features are generally available (GA).
BigQuery BI Engine / Datatransfer / ML
Now available in Delhi asia-south2
Now available in Delhi asia-south2
Is available in asia-east1 (Taiwan)
Cloud Logging now lets you copy logs from a Cloud Logging bucket to a Cloud Storage bucket. To learn more, see Copying log entries.
The Monitoring dashboards page in the Cloud Console now includes a collection of sample dashboards. The sample dashboards provide support for many common applications. You can preview, install, and then customize these dashboards. For more information, see Installing sample dashboards.
The Ops Agent is now Generally Available as version 2.0.0. This agent combines logging and metrics into a single agent. The Ops Agent is targeted toward specialized logging workloads that require higher throughput and improved resource efficiency.
The GA version of the Ops Agent can be configured to collect specific sets of metrics, as described in Configuring the Ops Agent. There is a small number of metrics that the GA version of the Ops Agent handles differently from the Preview versions of the Ops Agent and the Monitoring agent; see Differences in metric collection.
Cloud Monitoring is launching a new Observability tab on Compute Engine's VM instance details page. This tab replaces the Monitoring tab. The enhanced Observability tab provides access to logs and greater visibility into CPU, disk, and network metrics.
Cloud run is now available in asia-south2 (Delhi)
Cloud SQL for SQL Server
Integration of SQL Server with Managed Service for Microsoft Active Directory is generally available.
This provides capabilities for authentication, authorization, and more.
Joining an instance to a managed Active Directory domain enables you to log in to your SQL Server instances using Windows Authentication. Additionally, you can integrate with your on-premises AD domains by establishing a trust with the Managed Service for Microsoft Active Directory.
Cloud SQL for MySQL
Cloud SQL for MySQL now offers stored procedures that you can execute on your instances. You can use stored procedures to add or drop secondary indexes on read replicas. See Cloud SQL stored procedures.
Cloud Spanner now supports Cloud External Key Manager (Cloud EKM) when using customer-managed encryption keys. Cloud EKM also provides Key Access Justification to give you more visibility into key access requests.
- Enforcing public access prevention prevents data in your organization or project from being accidentally exposed to the public.
Preview: You can now configure N2D VMs with up to 100 Gbps of network bandwidth.
This feature is ideal for network-intensive distributed workloads.
The Machine types documentation has been renamed to Machine families. The URL remains the same.
New pages have been added to reflect the expansion of our machine fleet.
You can learn about Virtio memory balloon devices at the Dynamic resource management page.
Preview: You can now autoscale both regional and zonal managed instance groups based on a Cloud Monitoring metric that provides an aggregated value for the group. You can also apply filters to group metrics to further scope the scaling signal. For more information, see Scaling based on Cloud Monitoring metrics.
Generally available: Compute Engine's VM instance details page has a new Observability tab, which replaces the Monitoring tab. The enhanced Observability tab provides access to logs and greater visibility into CPU, disk, and network metrics.
GPU support on Dataflow is now in General Availability.
Dataflow is now able to use workers, Dataflow Shuffle, Streaming Engine, FlexRS, and regional endpoints in zones in
Dataproc is now available in the
asia-south2 region (Delhi).
Filestore is available in the europe-central2 (Warsaw) region. See Regions and zones.
Google Cloud Armor
Google Cloud Armor now supports parsing of the JSON content of POST bodies when preconfigured WAF rules are evaluated. JSON parsing must be enabled on a per-security-policy basis. In addition, you can enable verbose request logging to provide more details about why a particular rule was triggered. These features are Generally Available.
Google Kubernetes Engine
In GKE node version 1.21.1-gke.2200 and later, Containerd is available as a runtime for Windows Server LTSC and SAC node images. Containerd is the recommended container runtime for GKE. For more information, see Node images.
Migrate for Anthos
Enhanced runtime support added which lets you deploy containers to GKE Autopilot clusters and to Cloud Run, and simplifies the process of deploying containers to Anthos clusters on AWS that use workload identity. This feature is in preview.
See Enhanced runtime for more.
Added support for the preview release of the fit assessment tool that is intended to eventually replace the existing Linux discovery tool. The new fit assessment tool provides you with:
- Ability to get the inventory information about VMware VMs through direct connection to vCenter.
- Enhanced HTML output that makes it easier to view the assessment results.
- New collection script,
mfit_linux_collect.sh, and new assessment tool,
See Using the fit assessment tool for more.
Secret Manager now offers a limited number of free resources as part of the Google Cloud Free program.
For more details on free resources, see Secret Manager pricing.
Secret Manager now has a guide for rotating secrets and binding a secret version to your application.
To learn more, see Rotation of secrets.
Virtual Private Cloud
Deleting a private services access connection now also removes configurations created by the service producer, if Google is the service producer (for example, Cloud SQL). The improved deletion process simplifies administration if you delete a private services access connection, but later want to recreate it. This feature is now available in General Availability.
Microsoft Azure Releases And Updates
If you manage your Azure resources from PowerShell, update versions 7.0 and 7.1 of PowerShell as soon as possible.
Participate in the retail evaluation now to ensure full compatibility. The 21.07 OS evaluation period provides 21 days for backward compatibility testing.
New enhancements and updates released in Azure Security Center in June 2021.
To make it easier to get started with X.509 authentication with IoT Hub and Device Provisioning Services (DPS), we're removing the requirement to prove possession of uploaded certificates.
If you have processes or tools to detect accidental deletion of files and directories for your Azure Data Lake Storage accounts, you can now also restore the deleted objects using soft delete for blobs capability. During the retention period that you specify, you can now restore a soft-deleted object to its state at the time it was deleted.
Get started with Android map control using vector map libraries, now generally available with Azure Maps Android SDK.
Use Azure Image Builder to streamline your image building pipeline and integrate your DevOps application life-cycle with the cloud native tools.
Azure VPN Gateway NAT (Network Address Translation) can be used to connect on-premises networks or branch offices to an Azure virtual network with overlapping IP addresses.
New DCsv3 and DCdsv3-series Azure Virtual Machines transform the state-of-the-art for confidential workloads
Now you can automatically onboard your Arc enabled servers to best practice Azure management services using Azure Automanage, saving time and manual management effort.
The preview of on-demand disk bursting for Premium SSDs P30 and greater (lager than 512 GiB) is now expanded to all production regions.
Ingestion Client in conjunction with Azure Speech allows customers to transcribe audio files without any development effort. Whether you have few or millions of file Azure will transcribe those securely and quickly.
Have you tried Hava automated diagrams for AWS, Azure and GCP. Get back your precious time and possibly your sanity and rid yourself of manual drag and drop diagram builders forever.
Hava automatically generates accurate fully interactive cloud infrastructure and security diagrams when connected to your AWS, Azure or GCP accounts. Once diagrams are created, they are kept up to date, hands free. When changes are detected, new diagrams are auto-generated and the superseded documentation is moved to a version history. Older diagrams are also interactive, so can be opened and individual resources inspected interactively, just like the live diagrams.
Check it out for free here: