Here's a cloud round up of all things GCP, Azure and AWS for the week ending Friday 1st October 2021
To stay in the loop, make sure you subscribe on the right - There's a new newsletter series starting later this year that will keep you up to date with all our new releases, enhancements and capabilities and will also showcase lesser known but powerful features that you may not be aware of.
Of course we'd love to keep in touch at the usual places. Come and say hello on:
Source: aws.amazon.com
AWS announces the general availability of AWS Cloud Control API, a set of common application programming interfaces (APIs) that is designed to make it easy for developers to manage their cloud infrastructure in a consistent manner and leverage the latest AWS capabilities faster. Using Cloud Control API, developers can manage the lifecycle of hundreds of AWS resources and over a dozen third-party resources with five consistent APIs instead of using distinct service-specific APIs. With this launch, AWS Partner Network (APN) Partners can now automate how their solutions integrate with existing and future AWS features and services through a one-time integration, instead of spending weeks of custom development work as new resources become available. Terraform by HashiCorp and Pulumi have integrated their solutions as part of this launch.
AWS Step Functions now integrates with the AWS SDK, expanding the number of supported AWS Services from 17 to over 200 and AWS API Actions from 46 to over 9,000.
You can now provision devices using AWS IoT Core Just-in-Time Provisioning and Just-in-Time Registration features without having to send the entire trust chain on devices’ first connection to IoT Core. Until now, customers were required to configure their devices to present both the registered CA certificate and the client certificate signed by the registered CA certificate as part of the TLS handshake on devices’ first connection to IoT Core. Effective today, AWS IoT core makes it optional for customers to present the CA certificate on devices’ first connection to IoT Core when using Just-in-Time Provisioning and Just-in-Time Registration. This enhancement makes it easy for customers to migrate brownfield devices to AWS IoT Core, example, from customers’ self-managed cloud solutions.
This week, Amazon Elastic Container Service (ECS) launches integrated service discovery in the AWS GovCloud (US) Regions.
AWS Data Exchange subscribers can now use auto-export to automatically copy newly published revisions from their 3rd party data subscriptions to an Amazon S3 bucket of their choice in just a few clicks. With auto-export, subscribers no longer have to manually export new revisions or dedicate engineering resources to build ingestion pipelines that export new revisions as soon as they are published. For data subscribers that manage frequent updates to their file-based 3rd party data, auto-export saves significant time and effort.
This week AWS announced the launch of Amazon Monitron iOS app . The iOS app joins the existing Android app, giving customers more options for using Amazon Monitron. iPhone users can now use the Amazon Monitron iOS app to set up their sensors and gateway devices, and receive reports on operating behavior and alerts to potential failures in their equipment.
Amazon SageMaker JumpStart helps you quickly and easily get started with machine learning. SageMaker JumpStart provides a set of solutions for the most common use cases that can be deployed readily with just a few clicks and one-click deployment and fine-tuning of popular open source models. Starting today, you can now access a collection of multimodal financial text analysis tools, including example notebooks, text models, and a solution.
AQUA (Advanced Query Accelerator) for Amazon Redshift is now generally available in three additional AWS regions: Europe (Stockholm), Asia Pacific (Seoul), and US West (N. California).
AWS Lambda now allows customers to trigger functions from Amazon Simple Queue Service (Amazon SQS) queues that are in a different AWS account. Previously, customers could trigger Lambda functions from SQS queues in the same account only. Starting today, customers can create Lambda functions in multiple AWS accounts without needing to replicate the event source in each account.
Amazon Simple Email Service (Amazon SES) customers can now use 2048-bit DomainKeys Identified Mail (DKIM) keys to enhance their email security. DKIM is an email security standard designed to make sure that an email that claims to have come from a specific domain was indeed authorized by the owner of that domain. It uses public-key cryptography to sign an email with a private key. Recipient servers can then use a public key published to a domain's DNS to verify that parts of the email have not been modified during the transit.
Amazon Comprehend now supports two new AWS Trusted Advisor checks to help customers optimize the cost and security of Amazon Comprehend endpoints.
AWS Snowcone is now available in solid state drives (SSD) with 14TB storage capacity. AWS Snowcone is the smallest AWS Snow Family device equipped to handle edge computing, edge storage, and data transfers. With this launch, AWS Snowcone is now available in both hard disk drive (HDD) and solid state drive (SSD). Snowcone SSD has the same motherboard (4 vCPU and 4GB RAM) and industrial design as Snowcone, but Snowcone SSD will enable new data transfer and edge computing use cases that require 1) higher throughput performance 2) stronger vibration resistance operation 3) expanded durability and, 4) increased storage capacity (14TB Snowcone SSD vs. 8TB in Snowcone).
You can now monitor the system time accuracy for your Amazon ECS tasks running on AWS Fargate. For time-sensitive workloads running on Fargate, this gives customers the ability to monitor the clock error bound, which is used as a proxy for clock error, to know if the difference between reference time and system time exceeds a threshold. This capability leverages Amazon Time Sync Service to measure clock accuracy and provide the clock error bound for containers.
This week AWS announced that Amplify Geo for JavaScript is generally available, following our initial Developer Preview release in August. Amplify Geo enables frontend developers to quickly add location-aware features to their web applications. Extending existing Amplify use case categories like Auth, DataStore and Storage, Amplify Geo includes a set of abstracted client libraries built on top of Amazon Location Service, and includes ready-to-use map UI components based on the popular MapLibre open-source library. Amplify Geo also updates the Amplify Command Line Interface (CLI) tool to make it simple for people who aren’t familiar with AWS to achieve common mapping use cases by provisioning all required cloud services
Amazon Redshift RA3.xlplus nodes are now available in the AWS GovCloud (US) Regions. Amazon Redshift RA3 instances with managed storage allow you to scale compute and storage independently for fast query performance and lower costs. RA3 is available in three different node types to allow you to balance price and performance depending upon your workload requirements. RA3.xlplus nodes offer one-third compute (4 vCPU) and memory (32 GiB) compared to RA3.4xlarge at one-third of the price. RA3 nodes are built on the AWS Nitro System and feature high bandwidth networking and large high-performance SSDs as local caches.
AWS Lambda functions powered by next-generation AWS Graviton2 processors are now generally available. Graviton2 functions, using an Arm-based processor architecture, are designed to deliver up to 19% better performance at 20% lower cost for a variety of Serverless workloads, such as web and mobile backends, data, and media processing. With lower latency and better performance, functions powered by AWS Graviton2 processors are ideal for powering mission critical Serverless applications.
Amazon Managed Service for Prometheus is now generally available. Amazon Managed Service for Prometheus is a fully managed Prometheus-compatible monitoring service that makes it easy to monitor and alarm on operational metrics at scale. Prometheus is a popular Cloud Native Computing Foundation open-source project for monitoring and alerting that is optimized for container environments.
This week, Amazon Elastic Container Registry Public (Amazon ECR Public) announced the ability to launch containers directly from the ECR Public Gallery to AWS App Runner to quickly test popular web application container images. AWS App Runner is a fully managed service that makes it easier for developers to quickly deploy web applications and APIs, at scale with no prior infrastructure experience required.
With Amazon Redshift federated query capability, many customers have been able to combine live data from operational databases with the data in Amazon Redshift data warehouse and the data in Amazon S3 data lake environment in order to get unified analytics view across all the data in the enterprise. Now Amazon Redshift federated query support is generally available for Amazon Aurora MySQL and Amazon RDS for MySQL databases in addition to the existing Amazon Aurora PostgreSQL and Amazon RDS for PostgreSQL databases.
Amazon Redshift Query Editor V2 makes data in your Amazon Redshift data warehouse and data lake more accessible with a web-based tool for SQL users such as data analysts, data scientists, and database developers. With Query Editor V2, users can explore, analyze, and collaborate on data. It reduces the operational costs of managing query tools by providing a web-based application that allows you to focus on exploring your data without managing your infrastructure.
The AWS Snowcone service is now available for customer orders in the US East (Ohio), US West (San Francisco) and South America (Sao Paulo). With this launch, Snowcone is now available for order in US East (Ohio), US West (San Francisco) and South America (Sao Paulo), AWS Asia Pacific (Singapore), Asia Pacific (Tokyo), Canada (Central), Asia Pacific (Sydney), EU (Frankfurt), EU (Ireland), US East (N. Virginia), and US West (Oregon) Regions. AWS Snowcone is the smallest member of the AWS Snow Family of edge computing, edge storage, and data transfer devices. Snowcone is portable, rugged, and secure – small and light enough to fit in a backpack, and able to withstand harsh environments. Customers use Snowcone to deploy applications at the edge, and to collect data, process it locally, and move it to AWS either offline (by shipping the device to AWS) or online (by using AWS DataSync on Snowcone to send the data to AWS over the network).
AWS IoT Events is now available in the AWS GovCloud (US-West) Region.
AWS IoT SiteWise is now available in the AWS GovCloud (US-West) Region, extending the footprint to 8 AWS Regions.
AWS App Mesh is now available in the Amazon Web Services China (Beijing) Region, operated by Sinnet, and Amazon Web Services China (Ningxia) Region, operated by NWCD. AWS App Mesh is a service mesh that provides application-level networking to make it easy for your services to communicate with each other across multiple types of compute infrastructure. AWS App Mesh standardizes how your services communicate, giving you end-to-end visibility and options to tune for high-availability of your applications.
AWS Device Farm’s Desktop Browser Testing feature lets you test your web applications on different versions of Chrome, Firefox, and Internet Explorer browsers. With today’s launch, we are adding support for the Microsoft Edge browser.
You can now view your AWS resources such as Instances, VPCs, Subnets, Security Groups, Volumes across AWS Regions. Previously, finding specific resources, monitoring for their status or taking inventory in the console was manual and time consuming. You had to know which region a particular instance resided in, or had to manually switch across multiple different regions to look for it. Global View provides visibility to all your resources in a single pane of glass across AWS regions. It helps monitor resource counts, notice abnormalities sooner rather than later, and find stray resources.
Amazon EMR is the industry-leading cloud big data platform for processing vast amounts of data using open source tools such as Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi, and Presto. Today, we are excited to announce that Amazon EMR now supports auto-terminating idle EMR clusters, a new feature that automatically terminates your EMR cluster if it has been idle, to reduce the cost without the need to manually monitor cluster activity. You can specify the idle timeout value when enabling auto-termination for both existing and new clusters and EMR will automatically terminate the cluster when it has been idle for specified time.
Amazon Connect Voice ID is a Machine Learning (ML) powered voice authentication feature for Amazon Connect that makes voice interactions in contact centers more secure and efficient. Historically, contact centers have used a time-consuming knowledge-based authentication process where callers have to answer multiple questions based on personal details, such as social security number or date of birth. Amazon Connect Voice ID analyzes caller's unique voice characteristics using machine learning to verify identity in real-time without changing the natural flow of conversation. This helps improve agent productivity and contact center operating costs. Amazon Connect Voice ID also detects fraudsters in real-time from a custom watch-list for a contact center instance, improving security of contact center operations.
AWS Backup now makes it easier to delete recovery points that customers no longer need. Customers can use the new asynchronous delete operation from the console, CLI or APIs, to clean up existing recovery points in bulk and manage their backups more cost-effectively.
Elastic Load Balancing now supports forwarding traffic directly from Network Load Balancer (NLB) to Application Load Balancer (ALB). With this feature, you can now use AWS PrivateLink and expose static IP addresses for applications built on ALB.
Amazon Relational Database Service (RDS) for Oracle now supports version 21.1 of Oracle Application Express (APEX) for 12.1, 12.2 and 19c versions of Oracle Database. Using APEX, developers can build applications entirely within their web browser. To learn more about the latest features of APEX 21.1, please refer to Oracle’s blog post .
Following the launch of Red Hat Enterprise Linux with Microsoft SQL Server for Amazon EC2, you can now easily deploy RHEL SQL Server Always On availability groups using AWS Launch Wizard.
The AWS Solutions team recently updated AWS WAF Security Automations, a solution that automatically deploys a set of AWS WAF (web application firewall) rules that filter common web-based attacks. Users can select from preconfigured protective features that define the rules included in an AWS WAF web access control list (web ACL). Once deployed, AWS WAF protects your Amazon CloudFront distributions or Application Load Balancers by inspecting web requests.
Amazon Relational Database Service (Amazon RDS) for Oracle now supports four new customer modifiable sqlnet.ora client parameters for the Oracle Native Network Encryption (NNE) option. Amazon RDS for Oracle already supports server parameters which define encryption properties for incoming sessions. These client parameters apply to outgoing connections such as those used by database links.
This week, AWS announced the general availability of Amazon Genomics CLI, an open-source tool for genomics and life science customers to process genomics data at petabyte scale on AWS.
Anthos Clusters on VMware
Anthos clusters on VMware 1.9.0-gke.8 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.9.0-gke.8 runs on Kubernetes v1.21.4-gke.200.
The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.9, 1.8, and 1.7.
Features:
Cluster lifecycle Improvements:
gkeConnect section in the admin cluster configuration file, similar to user cluster registration.Platform enhancements:
Preview: User clusters can now be in a different vSphere datacenter from the admin cluster, resulting in datacenter isolation between the admin cluster and user clusters. This provides greater resiliency in the case of vSphere environment failures.
GA: Support for Windows node pools is generally available.This release adds:
The upstream fixes for the "Windows Pod stuck at terminating status" error are also applied to this release, which improves the stability of running Windows workloads.
GA: Support for Container-Optimized OS (COS) node pools is generally available.
GA: CoreDNS is now the cluster DNS provider.
Security enhancements:
gkectl update command to rotate these keys or to enable or disable secrets encryption after cluster creation.enableWindowsDataplaneV2 to true to enable this feature. Enabling this feature replaces Flannel with Antrea on Windows nodes.Simplify day-2 operations:
enableVMTracking in the configuration file to true to enable vSphere tag creation and attachment to the VMs in the user cluster. This allows easy mapping of VMs to clusters and node pools. See Enable VM tracking.gkectl update on existing user clusters. You can enable or disable cloud audit logging and monitoring with gkectl update on both admin and user clusters.Changes:
There is now a checkpoint file for the admin cluster, located in the same datastore folder as the admin cluster data disk, with the name DATA_DISK_NAME-checkpoint.yaml, or DATA_DISK_NAME.yaml if the length of DATA_DISK_NAME is greater than the filename length limit. This file is required for future upgrades and should be considered as important as the admin cluster data disk.
Note: If you have enabled VM encryption in vCenter, you must grant Cryptographer.Access permission to the vCenter credentials specified in your admin cluster configuration file, before trying to create or upgrade your admin cluster.
The admin cluster backup with gkectl preview feature introduced in 1.8 now allows updates to clusterBackup.datastore. This datastore may be different from vCenter.datastore so long as it is in the same datacenter as the cluster.
The k8s 1.21 release includes the following metrics changes:
storage_operation_duration_seconds, so that you can know about all status storage operation latency.The storage metrics storage_operation_errors_total and storage_operation_status_count are marked deprecated. In both cases, the storage_operation_duration_seconds metric can be used to recover equivalent counts (using status=fail-unknown in the case of storage_operations_errors_total).
Rename the metric etcd_object_counts to apiserver_storage_object_counts and mark it as stable. The original etcd_object_counts metrics name is marked as "Deprecated" and will be removed in the future.
A new GKE on-prem control plane uptime dashboard is introduced with a new metric, kubernetes.io/anthos/container/uptime, for component availability. The old GKE on-prem control plane status dashboard and old kubernetes.io/anthos/up metric are deprecated. New alerts for admin cluster control plane components availability and user cluster control plane components availability are introduced with a new kubernetes.io/anthos/container/uptime metric to replace deprecated alerts and the old kubernetes.io/anthos/up metric.
You can now skip certain health checks performed by gkectl diagnose cluster with the –skip-validation-xxx flag.
Anthos on Bare Metal
Anthos clusters on bare metal 1.9.0 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.0 runs on Kubernetes 1.21.
Improved cluster lifecycle functionalities:
Preview: Added ability to reset individual nodes with the bmctl reset node command. To give access to the needed cluster configuration file, use the command with the -c flag.
Preview: Added ability to recover from HA control plane quorum loss withbmctl restore --control-plane-node command.
Added bmctl create ksa command to create a Kubernetes Service Account (KSA) and generate a bearer token. To log in to the registered cluster, you can use the token in Cloud Console Kubernetes Engine > Clusters.
Preview: Added bmctl backup cluster and bmctl restore cluster commands to facilitate disaster recovery for clusters.
Introduced new troubleshooting capabilities:
Updated the bmctl check cluster --snapshot command to support uploading cluster diagnostic snapshots to a Cloud Storage bucket for review by Cloud Customer Care.
Provided access to bootstrap cluster logs to help troubleshoot cluster creation or upgrade problems.
Preview: Added support for Node Problem Detector service on nodes for quick detection of common node problems.
Enhanced monitoring and logging:
GA: Cloud Audit Logs capability is now generally available and enabled by default. Audit logs are useful for investigating suspicious API requests and for collecting statistics. For more information, see Use Audit Logging.
Switched to new open telemetry-based metrics agents to improve reliability, ability to scale, and resource usage.
Improved networking capabilities:
GA: The multi-NIC capability to provide additional interfaces to your pods is now generally available.
Preview: Added the single root I/O virtualization (SR-IOV) container network interface (CNI) plugin for multi-NIC.
Added support to configure cluster Domain Name System (DNS) provider options, such as upstream nameservers, with the new ClusterDNS custom resource definition.
Enhanced security:
SELinux is now always enabled in the container runtime for CentOS and RHEL.
Preview: Enhanced the capability to rotate cluster certificate authorities (CAs). Updates include support for all cluster types, rotation of front-proxy and etcd CAs, and changes to the bmctl command syntax.
Preview: Added Okta group support for authentication in Anthos Identity Service.
Functionality changes:
containerRuntime: containerd for new clusters. Customers can still choose Docker as the container runtime.Preview: Updated bmctl command, bmctl reset nodes --force, to support force removal of control plane nodes with etcd membership cleanup.
Added checks for cluster updates to verify access to cluster machines if changes to loginUser or sshKeyPrivatePath are detected. If the checks pass, Anthos clusters on bare metal saves the secret in the cluster.
Added new Anthos cluster control plane uptime dashboard in Cloud Monitoring with new metric kubernetes.io/anthos/container/uptime for component availability.
Added new alerts for control plane components availability with new metric kubernetes.io/anthos/container/uptime to replace deprecated alerts with metric kubernetes.io/anthos/up.
App Engine standard environment Go / Java / PHP / Python
Many legacy App Engine APIs are now available to select second-generation runtimes. These APIs are available for Go 1.12+ in preview, through language-idiomatic libraries. Calls to these APIs are billed according to the standard rates.
Big Query
Table functions are now generally available (GA). With the GA release, authorized table functions are now supported.
BigQuery now supports the following geospatial data functions:
ST_BOUNDINGBOX: Returns a STRUCT that represents the bounding box for a geography.
ST_EXTENT: Returns a STRUCT that represents the bounding box for a set of geographies.
S2_COVERINGCELLIDS: Returns an array of S2 cell IDs that cover a geography.
S2_CELLIDFROMPOINT: Returns the S2 cell ID covering a point geography.
Chronicle
Uppercase has been rebranded as Google Cloud Threat Intelligence (GCTI).
Cloud Bigtable
Storage limits for Cloud Bigtable nodes have been doubled. Each node now supports twice as much storage, with no increase in per-node costs. This feature is generally available
Cloud Composer
Cloud Composer 1.17.2 release started on September 29, 2021. Get ready for upcoming changes and features as we roll out the new release to all regions. This release is in progress at the moment. Listed changes and features might not be available in some regions yet.
Cloud Composer supports the IP Masquerade agent in Preview. This feature is available in new Cloud Composer 1 environments.
Changes in the preinstalled apache-airflow-backport-providers-google package for Airflow 1.10.15:
New versions of Cloud Composer images:
Cloud Load Balancing
External HTTP(S) Load Balancing is now available in a regional mode. The new regional external HTTP(S) load balancer contains many of the features of our existing global load balancer, but with an ever-growing list of advanced traffic management capabilities. You can use this load balancer for workloads with jurisdictional compliance requirements or to access the Standard Network Tier.
For details, see:
This load balancer is available in Public Preview.
Cloud Monitoring
Cloud Monitoring dashboards now support displays of data in tabular form. For information about this feature, see Configure tables with the Cloud Console and Configure tables by using the API.
You can now install the Ops Agent on one or more Compute Engine VMs from the Inventory tab of the Monitoring VM Instances dashboard. The dashboard generates Cloud Shell commands you can use to install the Ops Agent (recommended) or the legacy agents (if needed) on the selected VMs.
Cloud Run
Customer managed encryption keys are now at general availability (GA).
Cloud SQL for MySQL / PostgreSQL / SQL Server
Cloud SQL supports the preview version of two recommenders that help you optimize your database costs:
Idle database instance recommender: Identifies idle database instances in your project and provides recommendations about the savings that you can make by shutting them down.
Overprovisioned database instance recommender: Identifies overprovisioned database instances in your project and provides recommendations about the savings that you can make by rightsizing these instances.
When a database instance is nearly out of storage capacity, it's automatically stopped to prevent the loss of information. For more information, see Stopping an instance.
Compute Engine
Preview: Enable automatic renewal on your resource commitments. For more information, see Renew commitments automatically.
Filestore
You can now use Customer-Managed Encryption Keys (CMEK) to protect all data at rest in Filestore's Enterprise tier instances. CMEK in Filestore is a preview feature. For more information, see Encrypt data with customer-managed encryption keys.
Filestore's Enterprise tier now supports snapshots. A snapshot is a preserved state of your file share data that can be used to restore data. For more information, see the snapshots documentation page.
GKE
Now you can see how effectively your GKE clusters and workloads are utilizing your available compute resources. The new Cost Optimization tab lets you view, filter, and learn more about the CPU and memory usage, requests, allocation, and limit amounts of each of your clusters and workloads. This information can help you identify opportunities to optimize your clusters or workloads for more cost effective resource utilization. This feature is now available in Preview. For more information, see View cost-related optimization metrics.
Identity and Access Management
IAM role recommendations for folder- and organization-level roles are now generally available.
Network Connectivity Center
Previously, if you used a Router appliance spoke to connect more than 1,000 VMs, you might have experienced problems establishing BGP sessions between the router appliance instance and the Cloud Router. This issue has been resolved.
Network Connectivity Center includes new limits on the number of underlying resources that can be associated with a spoke. For information about the new limits, see Network Connectivity Center quotas and limits.
Transcoder API
Transcoder API is GA: The Transcoder API has graduated out of beta and has reached v1. All API endpoints are updated to use https://transcoder.googleapis.com/v1/.
Added Troubleshooting guide.
Added guidance on job limits.
VPC Service Controls
General availability for the following integration:
Microsoft Azure Releases And Updates
Source: azure.microsoft.com
Object replication allows you to replicate your premium block blob data at the blob level from one storage account to another anywhere in the Azure.
Azure Data Factory managed virtual network provides you with a more secure and manageable data integration solution.
The latest Azure Site Recovery update provides fixes and download links for Site Recovery components.
Onboard Azure Automation User Hybrid Runbook Worker based on VM extension platform for Azure Virtual machines and Arc-enabled servers. The extension-based platform leverages Azure Active Directory authentication.
Azure Automation now supports Az-module by default for all new Automation Accounts.
The Azure Purview Data Map and Data Catalog are generally available.
Announcing the release of a solution to simplify the connection of devices running FreeRTOS to Azure IoT services.
Azure Availability Zones are now generally available in the Norway East region. These three new zones provide customers with options for additional resiliency and tolerance to infrastructure impact.
Azure Site Recovery: Upgrade to TLS 1.2 or later by November 15, 2021 for improved security of replication data.
Have you tried Hava automated diagrams for AWS, Azure and GCP. Get back your precious time and sanity and rid yourself of manual drag and drop diagram builders forever.
Hava automatically generates accurate fully interactive cloud infrastructure and security diagrams when connected to your AWS, Azure or GCP accounts. Once diagrams are created, they are kept up to date, hands free.
When changes are detected, new diagrams are auto-generated and the superseded documentation is moved to a version history. Older diagrams are also interactive, so can be opened and individual resources inspected interactively, just like the live diagrams.
Check it out for free here: