35 min read

In Cloud Computing This Week [Oct 1st 2021]

October 1, 2021

This week's roundup of all the cloud news.



Here's a cloud round up of all things GCP, Azure and AWS for the week ending Friday 1st October 2021 

To stay in the loop, make sure you subscribe on the right - There's a new newsletter series starting later this year that will keep you up to date with all our new releases, enhancements and capabilities and will also showcase lesser known but powerful features that you may not be aware of. 

Of course we'd love to keep in touch at the usual places. Come and say hello on:

Facebook.      Linkedin.     Twitter.


AWS Updates and Releases

Source: aws.amazon.com

AWS announces the general availability of AWS Cloud Control API

AWS announces the general availability of AWS Cloud Control API, a set of common application programming interfaces (APIs) that is designed to make it easy for developers to manage their cloud infrastructure in a consistent manner and leverage the latest AWS capabilities faster. Using Cloud Control API, developers can manage the lifecycle of hundreds of AWS resources and over a dozen third-party resources with five consistent APIs instead of using distinct service-specific APIs. With this launch, AWS Partner Network (APN) Partners can now automate how their solutions integrate with existing and future AWS features and services through a one-time integration, instead of spending weeks of custom development work as new resources become available. Terraform by HashiCorp and Pulumi have integrated their solutions as part of this launch.

AWS Step Functions adds support for over 200 AWS Services with AWS SDK Integration

AWS Step Functions now integrates with the AWS SDK, expanding the number of supported AWS Services from 17 to over 200 and AWS API Actions from 46 to over 9,000.

AWS IoT Core now makes it optional for customers to send the entire trust chain when provisioning devices using Just-in-Time Provisioning and Just-in-Time Registration

You can now provision devices using AWS IoT Core Just-in-Time Provisioning and Just-in-Time Registration features without having to send the entire trust chain on devices’ first connection to IoT Core. Until now, customers were required to configure their devices to present both the registered CA certificate and the client certificate signed by the registered CA certificate as part of the TLS handshake on devices’ first connection to IoT Core. Effective today, AWS IoT core makes it optional for customers to present the CA certificate on devices’ first connection to IoT Core when using Just-in-Time Provisioning and Just-in-Time Registration. This enhancement makes it easy for customers to migrate brownfield devices to AWS IoT Core, example, from customers’ self-managed cloud solutions.

Amazon ECS Service Discovery Now Available in AWS GovCloud (US) Regions

This week, Amazon Elastic Container Service (ECS) launches integrated service discovery in the AWS GovCloud (US) Regions.

AWS Data Exchange now supports automatic exports of third-party data updates

AWS Data Exchange subscribers can now use auto-export to automatically copy newly published revisions from their 3rd party data subscriptions to an Amazon S3 bucket of their choice in just a few clicks. With auto-export, subscribers no longer have to manually export new revisions or dedicate engineering resources to build ingestion pipelines that export new revisions as soon as they are published. For data subscribers that manage frequent updates to their file-based 3rd party data, auto-export saves significant time and effort.

Amazon Monitron launches iOS app

This week AWS announced the launch of Amazon Monitron iOS app . The iOS app joins the existing Android app, giving customers more options for using Amazon Monitron. iPhone users can now use the Amazon Monitron iOS app to set up their sensors and gateway devices, and receive reports on operating behavior and alerts to potential failures in their equipment.

Amazon SageMaker JumpStart introduces new multimodal (long-form text, tabular) financial analysis tools

Amazon SageMaker JumpStart helps you quickly and easily get started with machine learning. SageMaker JumpStart provides a set of solutions for the most common use cases that can be deployed readily with just a few clicks and one-click deployment and fine-tuning of popular open source models. Starting today, you can now access a collection of multimodal financial text analysis tools, including example notebooks, text models, and a solution. 

AQUA for Amazon Redshift launches in three additional AWS regions

AQUA (Advanced Query Accelerator) for Amazon Redshift is now generally available in three additional AWS regions: Europe (Stockholm), Asia Pacific (Seoul), and US West (N. California).

AWS Lambda now supports triggering Lambda functions from an Amazon SQS queue in a different account

AWS Lambda now allows customers to trigger functions from Amazon Simple Queue Service (Amazon SQS) queues that are in a different AWS account. Previously, customers could trigger Lambda functions from SQS queues in the same account only. Starting today, customers can create Lambda functions in multiple AWS accounts without needing to replicate the event source in each account.

Amazon SES now supports 2048-bit DKIM keys

Amazon Simple Email Service (Amazon SES) customers can now use 2048-bit DomainKeys Identified Mail (DKIM) keys to enhance their email security. DKIM is an email security standard designed to make sure that an email that claims to have come from a specific domain was indeed authorized by the owner of that domain. It uses public-key cryptography to sign an email with a private key. Recipient servers can then use a public key published to a domain's DNS to verify that parts of the email have not been modified during the transit.

Amazon Comprehend adds two Trusted Advisor checks

Amazon Comprehend now supports two new AWS Trusted Advisor checks to help customers optimize the cost and security of Amazon Comprehend endpoints.

AWS announces AWS Snowcone SSD

AWS Snowcone is now available in solid state drives (SSD) with 14TB storage capacity. AWS Snowcone is the smallest AWS Snow Family device equipped to handle edge computing, edge storage, and data transfers. With this launch, AWS Snowcone is now available in both hard disk drive (HDD) and solid state drive (SSD). Snowcone SSD has the same motherboard (4 vCPU and 4GB RAM) and industrial design as Snowcone, but Snowcone SSD will enable new data transfer and edge computing use cases that require 1) higher throughput performance 2) stronger vibration resistance operation 3) expanded durability and, 4) increased storage capacity (14TB Snowcone SSD vs. 8TB in Snowcone).

Monitoring clock accuracy on AWS Fargate with Amazon ECS

You can now monitor the system time accuracy for your Amazon ECS tasks running on AWS Fargate. For time-sensitive workloads running on Fargate, this gives customers the ability to monitor the clock error bound, which is used as a proxy for clock error, to know if the difference between reference time and system time exceeds a threshold. This capability leverages Amazon Time Sync Service  to measure clock accuracy and provide the clock error bound for containers.

Announcing General Availability of Amplify Geo for AWS Amplify

This week AWS announced that Amplify Geo for JavaScript is generally available, following our initial Developer Preview release in August. Amplify Geo enables frontend developers to quickly add location-aware features to their web applications. Extending existing Amplify use case categories like Auth, DataStore and Storage, Amplify Geo includes a set of abstracted client libraries built on top of Amazon Location Service, and includes ready-to-use map UI components based on the popular MapLibre open-source library. Amplify Geo also updates the Amplify Command Line Interface (CLI) tool to make it simple for people who aren’t familiar with AWS to achieve common mapping use cases by provisioning all required cloud services

Amazon Redshift launches RA3.xlplus in AWS GovCloud (US) Regions

Amazon Redshift RA3.xlplus nodes are now available in the AWS GovCloud (US) Regions. Amazon Redshift RA3 instances with managed storage allow you to scale compute and storage independently for fast query performance and lower costs. RA3 is available in three different node types to allow you to balance price and performance depending upon your workload requirements. RA3.xlplus nodes offer one-third compute (4 vCPU) and memory (32 GiB) compared to RA3.4xlarge at one-third of the price. RA3 nodes are built on the AWS Nitro System and feature high bandwidth networking and large high-performance SSDs as local caches.

Achieve up to 34% better price/performance with AWS Lambda Functions powered by AWS Graviton2 processor

AWS Lambda functions powered by next-generation AWS Graviton2 processors are now generally available. Graviton2 functions, using an Arm-based processor architecture, are designed to deliver up to 19% better performance at 20% lower cost for a variety of Serverless workloads, such as web and mobile backends, data, and media processing. With lower latency and better performance, functions powered by AWS Graviton2 processors are ideal for powering mission critical Serverless applications.

Amazon Managed Service for Prometheus is now Generally Available with support for alert manager and rules

Amazon Managed Service for Prometheus is now generally available. Amazon Managed Service for Prometheus is a fully managed Prometheus-compatible monitoring service that makes it easy to monitor and alarm on operational metrics at scale. Prometheus is a popular Cloud Native Computing Foundation open-source project for monitoring and alerting that is optimized for container environments.

Amazon ECR Public adds the ability to launch containers directly to AWS App Runner

This week, Amazon Elastic Container Registry Public (Amazon ECR Public) announced the ability to launch containers directly from the ECR Public Gallery  to AWS App Runner to quickly test popular web application container images. AWS App Runner is a fully managed service that makes it easier for developers to quickly deploy web applications and APIs, at scale with no prior infrastructure experience required.

Announcing general availability of Amazon RDS for MySQL and Amazon Aurora MySQL databases as new data sources for federated querying

With Amazon Redshift federated query capability, many customers have been able to combine live data from operational databases with the data in Amazon Redshift data warehouse and the data in Amazon S3 data lake environment in order to get unified analytics view across all the data in the enterprise. Now Amazon Redshift federated query support is generally available for Amazon Aurora MySQL and Amazon RDS for MySQL databases in addition to the existing Amazon Aurora PostgreSQL and Amazon RDS for PostgreSQL databases.

Amazon Redshift announces the next generation of Amazon Redshift Query Editor

Amazon Redshift Query Editor V2 makes data in your Amazon Redshift data warehouse and data lake more accessible with a web-based tool for SQL users such as data analysts, data scientists, and database developers. With Query Editor V2, users can explore, analyze, and collaborate on data. It reduces the operational costs of managing query tools by providing a web-based application that allows you to focus on exploring your data without managing your infrastructure.

AWS Snowcone is now available in the US East (Ohio), US West (San Francisco) and South America (Sao Paulo) regions

The AWS Snowcone service is now available for customer orders in the US East (Ohio), US West (San Francisco) and South America (Sao Paulo). With this launch, Snowcone is now available for order in US East (Ohio), US West (San Francisco) and South America (Sao Paulo), AWS Asia Pacific (Singapore), Asia Pacific (Tokyo), Canada (Central), Asia Pacific (Sydney), EU (Frankfurt), EU (Ireland), US East (N. Virginia), and US West (Oregon) Regions. AWS Snowcone is the smallest member of the AWS Snow Family of edge computing, edge storage, and data transfer devices. Snowcone is portable, rugged, and secure – small and light enough to fit in a backpack, and able to withstand harsh environments. Customers use Snowcone to deploy applications at the edge, and to collect data, process it locally, and move it to AWS either offline (by shipping the device to AWS) or online (by using AWS DataSync on Snowcone to send the data to AWS over the network).

AWS IoT Events is available in the AWS GovCloud (US-West) Region

AWS IoT Events is now available in the AWS GovCloud (US-West) Region.

AWS IoT SiteWise is now available in the AWS GovCloud (US-West) Region

AWS IoT SiteWise is now available in the AWS GovCloud (US-West) Region, extending the footprint to 8 AWS Regions.

AWS App Mesh is now available in the AWS China (Beijing) Region and AWS China (Ningxia) Region

AWS App Mesh is now available in the Amazon Web Services China (Beijing) Region, operated by Sinnet, and Amazon Web Services China (Ningxia) Region, operated by NWCD. AWS App Mesh is a service mesh that provides application-level networking to make it easy for your services to communicate with each other across multiple types of compute infrastructure. AWS App Mesh standardizes how your services communicate, giving you end-to-end visibility and options to tune for high-availability of your applications.

AWS Device Farm announces support for testing web apps on Microsoft Edge browser

AWS Device Farm’s Desktop Browser Testing feature lets you test your web applications on different versions of Chrome, Firefox, and Internet Explorer browsers. With today’s launch, we are adding support for the Microsoft Edge browser.

Amazon EC2 now offers Global View on the console to view all resources across regions together

You can now view your AWS resources such as Instances, VPCs, Subnets, Security Groups, Volumes across AWS Regions. Previously, finding specific resources, monitoring for their status or taking inventory in the console was manual and time consuming. You had to know which region a particular instance resided in, or had to manually switch across multiple different regions to look for it. Global View provides visibility to all your resources in a single pane of glass across AWS regions. It helps monitor resource counts, notice abnormalities sooner rather than later, and find stray resources.

Now auto-terminate idle EMR clusters to lower cost

Amazon EMR is the industry-leading cloud big data platform for processing vast amounts of data using open source tools such as Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi, and Presto. Today, we are excited to announce that Amazon EMR now supports auto-terminating idle EMR clusters, a new feature that automatically terminates your EMR cluster if it has been idle, to reduce the cost without the need to manually monitor cluster activity. You can specify the idle timeout value when enabling auto-termination for both existing and new clusters and EMR will automatically terminate the cluster when it has been idle for specified time.

Amazon Connect Voice ID is now generally available

Amazon Connect Voice ID is a Machine Learning (ML) powered voice authentication feature for Amazon Connect that makes voice interactions in contact centers more secure and efficient. Historically, contact centers have used a time-consuming knowledge-based authentication process where callers have to answer multiple questions based on personal details, such as social security number or date of birth. Amazon Connect Voice ID analyzes caller's unique voice characteristics using machine learning to verify identity in real-time without changing the natural flow of conversation. This helps improve agent productivity and contact center operating costs. Amazon Connect Voice ID also detects fraudsters in real-time from a custom watch-list for a contact center instance, improving security of contact center operations.

AWS Backup simplifies recovery point deletions

AWS Backup now makes it easier to delete recovery points that customers no longer need. Customers can use the new asynchronous delete operation from the console, CLI or APIs, to clean up existing recovery points in bulk and manage their backups more cost-effectively.

Application Load Balancer now enables AWS PrivateLink and static IP addresses by direct integration with Network Load Balancer

Elastic Load Balancing now supports forwarding traffic directly from Network Load Balancer (NLB) to Application Load Balancer (ALB). With this feature, you can now use AWS PrivateLink and expose static IP addresses for applications built on ALB.

Amazon RDS for Oracle now supports Oracle Application Express (APEX) Version 21.1

Amazon Relational Database Service (RDS) for Oracle now supports version 21.1 of Oracle Application Express  (APEX) for 12.1, 12.2 and 19c versions of Oracle Database. Using APEX, developers can build applications entirely within their web browser. To learn more about the latest features of APEX 21.1, please refer to Oracle’s blog post .

AWS Launch Wizard now supports Microsoft SQL Server Always On deployments on Red Hat Enterprise Linux

Following the launch of Red Hat Enterprise Linux with Microsoft SQL Server for Amazon EC2, you can now easily deploy RHEL SQL Server Always On availability groups using AWS Launch Wizard.

Introducing AWS WAF Security Automations v3.2

The AWS Solutions team recently updated AWS WAF Security Automations, a solution that automatically deploys a set of AWS WAF (web application firewall) rules that filter common web-based attacks. Users can select from preconfigured protective features that define the rules included in an AWS WAF web access control list (web ACL). Once deployed, AWS WAF protects your Amazon CloudFront distributions or Application Load Balancers by inspecting web requests.

Amazon RDS for Oracle now supports sqlnet.ora client parameters for the Oracle Native Network Encryption (NNE)option

Amazon Relational Database Service (Amazon RDS) for Oracle now supports four new customer modifiable sqlnet.ora client parameters for the Oracle Native Network Encryption (NNE) option. Amazon RDS for Oracle already supports server parameters which define encryption properties for incoming sessions. These client parameters apply to outgoing connections such as those used by database links.

Amazon Genomics CLI is now Generally Available

This week, AWS announced the general availability of Amazon Genomics CLI, an open-source tool for genomics and life science customers to process genomics data at petabyte scale on AWS.


Google Cloud Releases and Updates
Source: cloud.google.com


Anthos Clusters on VMware

Anthos clusters on VMware 1.9.0-gke.8 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.9.0-gke.8 runs on Kubernetes v1.21.4-gke.200.

The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.9, 1.8, and 1.7.


Cluster lifecycle Improvements:

    • GA: You can register an admin cluster during its creation by filling in the gkeConnect section in the admin cluster configuration file, similar to user cluster registration.

Platform enhancements:

    • Preview: User clusters can now be in a different vSphere datacenter from the admin cluster, resulting in datacenter isolation between the admin cluster and user clusters. This provides greater resiliency in the case of vSphere environment failures.

    • GA: Support for Windows node pools is generally available.This release adds:

      • Preview: Windows DataplaneV2 support, which allows for using Windows Network Policy
      • Node Problem Detector (NPD) support on Windows
      • Streamlined process for preparing Windows images in a private registry
      • Enhanced Flannel CNI support on Windows

      The upstream fixes for the "Windows Pod stuck at terminating status" error are also applied to this release, which improves the stability of running Windows workloads.

    • GA: Support for Container-Optimized OS (COS) node pools is generally available.

    • GA: CoreDNS is now the cluster DNS provider.

      • Clusters that are upgraded to 1.9 will have their KubeDNS provider replaced with CoreDNS. During the upgrade, CoreDNS is first deployed and then KubeDNS is removed, so applications should not observe DNS unavailability. However before upgrading, ensure that your cluster has enough additional resources to deploy CoreDNS. CoreDNS requires 100 millicpu and 170 MiB of memory per instance, all clusters require a minimum of 2 instances, and there is an additional instance deployed for every 16 nodes in the cluster.
      • You can configure cluster DNS options such as upstream name servers by using the new ClusterDNS custom resource.

Security enhancements:

    • GA: Always-on secrets encryption: You can enable secrets encryption with internally generated keys instead of a hardware security module (HSM). Use the gkectl update command to rotate these keys or to enable or disable secrets encryption after cluster creation.
    • Preview: Windows network policy support. This release introduces a new network plugin, Antrea, for Windows nodes. In addition to network connectivity and services support, it provides network policy support. When creating a user cluster, you can set enableWindowsDataplaneV2 to true to enable this feature. Enabling this feature replaces Flannel with Antrea on Windows nodes.
    • Preview: Azure AD group support for Authentication: This feature allows cluster admins to configure RBAC policies based on Azure AD groups for authorization in clusters. This supports retrieval of groups information for users belonging to more than 200 groups, thus overcoming a limitation of regular OIDC configured with Azure AD as the identity provider.

Simplify day-2 operations:

    • Preview: When creating a user cluster, you can set enableVMTracking in the configuration file to true to enable vSphere tag creation and attachment to the VMs in the user cluster. This allows easy mapping of VMs to clusters and node pools. See Enable VM tracking.
    • GA: New metrics agents based on open telemetry are introduced to improve reliability, scalability and resource usage.
    • Preview: You can enable or disable Stackdriver with gkectl update on existing user clusters. You can enable or disable cloud audit logging and monitoring with gkectl update on both admin and user clusters.


  • There is now a checkpoint file for the admin cluster, located in the same datastore folder as the admin cluster data disk, with the name DATA_DISK_NAME-checkpoint.yaml, or DATA_DISK_NAME.yaml if the length of DATA_DISK_NAME is greater than the filename length limit. This file is required for future upgrades and should be considered as important as the admin cluster data disk.

    Note: If you have enabled VM encryption in vCenter, you must grant Cryptographer.Access permission to the vCenter credentials specified in your admin cluster configuration file, before trying to create or upgrade your admin cluster.

  • The admin cluster backup with gkectl preview feature introduced in 1.8 now allows updates to clusterBackup.datastore. This datastore may be different from vCenter.datastore so long as it is in the same datacenter as the cluster.

  • The k8s 1.21 release includes the following metrics changes:

    • Add new field status for storage_operation_duration_seconds, so that you can know about all status storage operation latency.
    • The storage metrics storage_operation_errors_total and storage_operation_status_count are marked deprecated. In both cases, the storage_operation_duration_seconds metric can be used to recover equivalent counts (using status=fail-unknown in the case of storage_operations_errors_total).

    • Rename the metric etcd_object_counts to apiserver_storage_object_counts and mark it as stable. The original etcd_object_counts metrics name is marked as "Deprecated" and will be removed in the future.

  • A new GKE on-prem control plane uptime dashboard is introduced with a new metric, kubernetes.io/anthos/container/uptime, for component availability. The old GKE on-prem control plane status dashboard and old kubernetes.io/anthos/up metric are deprecated. New alerts for admin cluster control plane components availability and user cluster control plane components availability are introduced with a new kubernetes.io/anthos/container/uptime metric to replace deprecated alerts and the old kubernetes.io/anthos/up metric.

  • You can now skip certain health checks performed by gkectl diagnose cluster with the –skip-validation-xxx flag.

Anthos on Bare Metal

Anthos clusters on bare metal 1.9.0 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.0 runs on Kubernetes 1.21.

Improved cluster lifecycle functionalities:

    • Preview: Added ability to reset individual nodes with the bmctl reset node command. To give access to the needed cluster configuration file, use the command with the -c flag.

    • Preview: Added ability to recover from HA control plane quorum loss withbmctl restore --control-plane-node command.

    • Added bmctl create ksa command to create a Kubernetes Service Account (KSA) and generate a bearer token. To log in to the registered cluster, you can use the token in Cloud Console Kubernetes Engine > Clusters.

    • Preview: Added bmctl backup cluster and bmctl restore cluster commands to facilitate disaster recovery for clusters.

Introduced new troubleshooting capabilities:

    • Updated the bmctl check cluster --snapshot command to support uploading cluster diagnostic snapshots to a Cloud Storage bucket for review by Cloud Customer Care.

    • Provided access to bootstrap cluster logs to help troubleshoot cluster creation or upgrade problems.

    • Preview: Added support for Node Problem Detector service on nodes for quick detection of common node problems.

Enhanced monitoring and logging:

    • GA: Cloud Audit Logs capability is now generally available and enabled by default. Audit logs are useful for investigating suspicious API requests and for collecting statistics. For more information, see Use Audit Logging.

    • Switched to new open telemetry-based metrics agents to improve reliability, ability to scale, and resource usage.

Improved networking capabilities:

    • GA: The multi-NIC capability to provide additional interfaces to your pods is now generally available.

    • Preview: Added the single root I/O virtualization (SR-IOV) container network interface (CNI) plugin for multi-NIC.

    • Added support to configure cluster Domain Name System (DNS) provider options, such as upstream nameservers, with the new ClusterDNS custom resource definition.

Enhanced security:

    • SELinux is now always enabled in the container runtime for CentOS and RHEL.

    • Preview: Enhanced the capability to rotate cluster certificate authorities (CAs). Updates include support for all cluster types, rotation of front-proxy and etcd CAs, and changes to the bmctl command syntax.

    • Preview: Added Okta group support for authentication in Anthos Identity Service.

Functionality changes:

    • Changed default container runtime to containerd, containerRuntime: containerd for new clusters. Customers can still choose Docker as the container runtime.
    • Preview: Updated bmctl command, bmctl reset nodes --force, to support force removal of control plane nodes with etcd membership cleanup.

    • Added checks for cluster updates to verify access to cluster machines if changes to loginUser or sshKeyPrivatePath are detected. If the checks pass, Anthos clusters on bare metal saves the secret in the cluster.

    • Added new Anthos cluster control plane uptime dashboard in Cloud Monitoring with new metric kubernetes.io/anthos/container/uptime for component availability.

    • Added new alerts for control plane components availability with new metric kubernetes.io/anthos/container/uptime to replace deprecated alerts with metric kubernetes.io/anthos/up.

App Engine standard environment Go / Java / PHP / Python

Many legacy App Engine APIs are now available to select second-generation runtimes. These APIs are available for Go 1.12+ in preview, through language-idiomatic libraries. Calls to these APIs are billed according to the standard rates.

Big Query

Table functions are now generally available (GA). With the GA release, authorized table functions are now supported.

BigQuery now supports the following geospatial data functions:

  • ST_BOUNDINGBOX: Returns a STRUCT that represents the bounding box for a geography.

  • ST_EXTENT: Returns a STRUCT that represents the bounding box for a set of geographies.

  • S2_COVERINGCELLIDS: Returns an array of S2 cell IDs that cover a geography.

  • S2_CELLIDFROMPOINT: Returns the S2 cell ID covering a point geography.


Uppercase has been rebranded as Google Cloud Threat Intelligence (GCTI).

Cloud Bigtable

Storage limits for Cloud Bigtable nodes have been doubled. Each node now supports twice as much storage, with no increase in per-node costs. This feature is generally available

Cloud Composer

Cloud Composer 1.17.2 release started on September 29, 2021. Get ready for upcoming changes and features as we roll out the new release to all regions. This release is in progress at the moment. Listed changes and features might not be available in some regions yet.

Cloud Composer supports the IP Masquerade agent in Preview. This feature is available in new Cloud Composer 1 environments.

Changes in the preinstalled apache-airflow-backport-providers-google package for Airflow 1.10.15:

  • Dataflow job operators can be run in async mode.
  • Dataflow Hook handles no Job Type.

New versions of Cloud Composer images:

  • composer-1.17.2-airflow-2.1.2
  • composer-1.17.2-airflow-2.0.2
  • composer-1.17.2-airflow-1.10.15 (default)
  • composer-1.17.2-airflow-1.10.14
  • composer-1.17.2-airflow-1.10.12
  • composer-2.0.0-preview.3-airflow-2.1.2 (default)
  • composer-2.0.0-preview.3-airflow-2.0.2

Cloud Load Balancing

External HTTP(S) Load Balancing is now available in a regional mode. The new regional external HTTP(S) load balancer contains many of the features of our existing global load balancer, but with an ever-growing list of advanced traffic management capabilities. You can use this load balancer for workloads with jurisdictional compliance requirements or to access the Standard Network Tier.

For details, see:

This load balancer is available in Public Preview.

Cloud Monitoring

Cloud Monitoring dashboards now support displays of data in tabular form. For information about this feature, see Configure tables with the Cloud Console and Configure tables by using the API.

You can now install the Ops Agent on one or more Compute Engine VMs from the Inventory tab of the Monitoring VM Instances dashboard. The dashboard generates Cloud Shell commands you can use to install the Ops Agent (recommended) or the legacy agents (if needed) on the selected VMs.

Cloud Run

Customer managed encryption keys are now at general availability (GA).

Cloud SQL for MySQL / PostgreSQL / SQL Server

Cloud SQL supports the preview version of two recommenders that help you optimize your database costs:

When a database instance is nearly out of storage capacity, it's automatically stopped to prevent the loss of information. For more information, see Stopping an instance.

Compute Engine

Preview: Enable automatic renewal on your resource commitments. For more information, see Renew commitments automatically.


You can now use Customer-Managed Encryption Keys (CMEK) to protect all data at rest in Filestore's Enterprise tier instances. CMEK in Filestore is a preview feature. For more information, see Encrypt data with customer-managed encryption keys.

Filestore's Enterprise tier now supports snapshots. A snapshot is a preserved state of your file share data that can be used to restore data. For more information, see the snapshots documentation page.


Now you can see how effectively your GKE clusters and workloads are utilizing your available compute resources. The new Cost Optimization tab lets you view, filter, and learn more about the CPU and memory usage, requests, allocation, and limit amounts of each of your clusters and workloads. This information can help you identify opportunities to optimize your clusters or workloads for more cost effective resource utilization. This feature is now available in Preview. For more information, see View cost-related optimization metrics.

Identity and Access Management


IAM role recommendations for folder- and organization-level roles are now generally available.

Network Connectivity Center

Previously, if you used a Router appliance spoke to connect more than 1,000 VMs, you might have experienced problems establishing BGP sessions between the router appliance instance and the Cloud Router. This issue has been resolved.

Network Connectivity Center includes new limits on the number of underlying resources that can be associated with a spoke. For information about the new limits, see Network Connectivity Center quotas and limits.

Transcoder API

Transcoder API is GA: The Transcoder API has graduated out of beta and has reached v1. All API endpoints are updated to use https://transcoder.googleapis.com/v1/.

Added Troubleshooting guide.

VPC Service Controls

General availability for the following integration:


Microsoft Azure Releases And Updates
Source: azure.microsoft.com

Object replication in public preview for Premium Block Blob Storage 

Object replication allows you to replicate your premium block blob data at the blob level from one storage account to another anywhere in the Azure.

General availability: Azure Data Factory managed virtual network


Azure Data Factory managed virtual network provides you with a more secure and manageable data integration solution.

Azure Site Recovery update rollup 58 is now generally available - September 2021

The latest Azure Site Recovery update provides fixes and download links for Site Recovery components.

Azure Automation Hybrid Worker Extension for Azure and Arc-enabled servers now in public preview

Onboard Azure Automation User Hybrid Runbook Worker based on VM extension platform for Azure Virtual machines and Arc-enabled servers. The extension-based platform leverages Azure Active Directory authentication.

Az module support in Azure Automation is now available

Azure Automation now supports Az-module by default for all new Automation Accounts.

Azure Purview is now generally available

The Azure Purview Data Map and Data Catalog are generally available.

Azure IoT middleware for FreeRTOS in general availability

Announcing the release of a solution to simplify the connection of devices running FreeRTOS to Azure IoT services.

Norway East Availability Zones now generally available

Azure Availability Zones are now generally available in the Norway East region. These three new zones provide customers with options for additional resiliency and tolerance to infrastructure impact.

Azure Site Recovery: Upgrade to TLS 1.2 or later by November 15, 2021 - for improved security

Azure Site Recovery: Upgrade to TLS 1.2 or later by November 15, 2021 for improved security of replication data.

Have you tried Hava automated diagrams for AWS, Azure and GCP.  Get back your precious time and sanity and rid yourself of manual drag and drop diagram builders forever.
Hava automatically generates accurate fully interactive cloud infrastructure and security diagrams when connected to your AWS, Azure or GCP accounts. Once diagrams are created, they are kept up to date, hands free. 

When changes are detected, new diagrams are auto-generated and the superseded documentation is moved to a version history. Older diagrams are also interactive, so can be opened and individual resources inspected interactively, just like the live diagrams.
Check it out for free here:
Try Hava For Free Today!

Topics: aws azure gcp news
Team Hava

Written by Team Hava

The Hava content team