This week's roundup of all the cloud news.
Here's a round up of all things GCP, Azure and AWS for the week ending Friday 26th March 2021.
Here at Hava we've been doing even more cool website and platform design planning and you should start seeing the benefits soon.
To stay in the loop, make sure you subscribe on the right - There's a new Newsletter series starting soon that will keep you up to date with all our new releases, enhancements and capabilities and will also showcase lesser known but powerful features that you may not be aware of.
Of course we'd love to keep in touch at the usual places. Come and say hello on:
AWS Updates and Releases
Amazon ElastiCache for Redis now supports running clusters with high availability across multiple AWS Local Zones. AWS Local Zones are an extension of an AWS Region where you can run your latency-sensitive applications using AWS services in geographic proximity to end-users. Previously, Amazon ElastiCache for Redis only supported launching a cluster in a single AWS Local Zone.
AWS Security Hub integrates with Amazon Macie to automatically ingest sensitive data findings for improved centralized security posture management
AWS Security Hub is now integrated with Amazon Macie to automatically ingest sensitive data findings from Macie. Security Hub previously ingested policy findings from Macie, and this integration adds sensitive data findings. All of Security Hub’s findings are automatically normalized using the AWS Security Finding Format (ASFF), enabling you to more easily search, correlate, and operationalize them. To get started, visit the Settings page in the Macie console and select Security Hub as a publish destination for sensitive data findings.
AWS Backup, a cost-effective, fully managed, policy-based service that further simplifies data protection at scale, is now available in the AWS Asia Pacific (Osaka) Region. With AWS Backup, you can centralize and automate data protection across AWS services and accounts, to support your regulatory compliance obligations and meet your business continuity goals.
DETECT ANOMALIES IN YOUR METRICS, AND DIAGNOSE ISSUES QUICKLY WITH AMAZON LOOKOUT FOR METRICS – NOW GENERALLY AVAILABLE
Today Amazon Lookout for Metrics is generally available to all AWS customers. Lookout for Metrics is a machine learning (ML) service that detects anomalies or unexpected changes in your metrics, helping you proactively monitor the health of your business, diagnose issues and find opportunities quickly – with no ML experience required.
AWS License Manager automated discovery now allows customers to exclude instances that do not need license payment per a customer’s licensing agreements. AWS License Manager automatically tracks instances across AWS and on-premises environments based on customer defined rules. While configuring automated discovery, administrators can now define exclusion rules using custom defined tags or AWS Account IDs to identify instances that should not count towards license usage. It helps automated discovery exclude instances that matches these rules and thus further simplifies the management of software licenses.
Amazon Elastic File System (Amazon EFS) One Zone is now available in the Asia (Osaka) Region, reducing storage costs by 47% compared to Amazon EFS Standard storage classes, while maintaining the Amazon EFS capabilities that customers love
Amazon Kinesis Data Analytics for Apache Flink now supports streaming applications built using Python version 3.7. This enables you to write streaming applications in the Python language and run them using Apache Flink v1.11 on Amazon Kinesis Data Analytics. Apache Flink v1.11 offers support for Python through the Table API, which is a unified, relational API for data processing
AWS IoT Core now enables customers to customize the behavior of their data endpoints, making it simpler to onboard IoT applications with existing devices in the field. Customers can now configure their data endpoints with custom domain names and associated server certificates stored in AWS Certificate Manager. They can also attach custom authorizers , and create multiple data endpoints for their accoun
The AWS Snowcone service is now available for order for customers in the AWS Asia Pacific (Sydney) Region. With this launch, Snowcone is now available for order in Asia Pacific (Sydney), EU (Frankfurt), EU (Ireland), US East (N. Virginia), and US West (Oregon) Regions. AWS Snowcone is the smallest member of the AWS Snow Family of edge computing, edge storage, and data transfer devices. Snowcone is portable, rugged, and secure – small and light enough to fit in a backpack, and able to withstand harsh environments. Customers use Snowcone to deploy applications at the edge, and to collect data, process it locally, and move it to AWS either offline (by shipping the device to AWS) or online (by using AWS DataSync on Snowcone to send the data to AWS over the network).
Containers have proven popular with AWS and Red Hat customers because they increase developer velocity, improve application portability, and enable faster application development. Currently, AWS offers a broad range of containers technology in the cloud, including Amazon Elastic Containers Service (ECS), Amazon Elastic Kubernetes Service (EKS), Amazon ECS and Amazon EKS with AWS Fargate. Now customers have an additional option with the launch of Red Hat OpenShift Service on AWS (ROSA), which provides a new managed service that makes it easier for Red Hat OpenShift customers to build, scale, and manage containerized applications on AWS.
Amazon AppFlow, a fully managed integration service that helps customers to securely transfer data between AWS services and software-as-a-service (SaaS) applications in just a few clicks, now supports Zendesk as a destination.
With this new release of the AWS Toolkit for VS Code, customers can use federated credentials, MFA and AWS Single Sign-On (AWS SSO) to connect Visual Studio Code to AWS. Using AWS SSO, users can sign in to their organization’s Active Directory, a built-in AWS SSO directory, or another external identity provider (IdP) connected to AWS SSO.
You can now access Amazon Timestream APIs from your Amazon Virtual Private Cloud (Amazon VPC) using VPC endpoints. Amazon VPC endpoints are easy to configure and provide reliable connectivity to Amazon Timestream APIs without requiring an internet gateway or a Network Address Translation (NAT) instance.
You can now manage non-IP based cloud resources in AWS Cloud Map’s public and private DNS namespaces. AWS Cloud Map is a cloud resource discovery service. Using AWS Cloud Map, you can define custom names for your application resources, such as Amazon ECS tasks, Amazon EC2 instances, Amazon S3 buckets, Amazon DynamoDB tables, or any other cloud resource. Your application can then discover the location and metadata of cloud resources associated with these custom names via AWS SDK or by making authenticated API calls.
We are excited to announce that you can now enable notifications for workflow status changes while using Amazon Forecast, allowing you to work seamlessly without the disruption of having to check if a particular workflow has completed. Additionally, you can now automate workflows through the notifications to increase work efficiency. Amazon Forecast uses machine learning (ML) to generate more accurate demand forecasts, without requiring any prior ML experience. Amazon Forecast brings the same technology used at Amazon.com to developers as a fully managed service, removing the need to manage resources or rebuild your systems.
This week, AWS announced the general availability of ML Detect, a new feature of AWS IoT Device Defender that automatically detects device-level operational and security anomalies across your IoT fleet by learning from your past data. Now, in addition to being able to manually set static alarms with AWS IoT Device Defender Rules Detect, you can use machine learning to automatically learn your fleet's expected behaviors so that you don’t need an in-depth understanding of how your devices act across a range of metrics to get started.
AWS Backup adds support for bulk deletion of recovery points from backup vaults in the AWS Backup console. You can now select multiple recovery points for deletion at one time.
Amazon Elasticsearch Service announces Auto-Tune feature for improved performance and application availability
Amazon Elasticsearch Service now supports automated memory management of Elasticsearch clusters with the new Auto-Tune feature. Auto-Tune is an adaptive resource management system that automatically adjusts Elasticsearch internal settings to handle dynamic workloads, optimizing cluster resources to improve efficiency and performance. With Auto-Tune, you can achieve performance boost in ingestion throughput for log analytics workloads, and reduced tail latencies for search queries.
NOW YOU CAN USE AWS CLOUDTRAIL TO LOG DATA-PLANE API ACTIVITY TO MONITOR, ALARM, AND ARCHIVE ITEM-LEVEL ACTIVITY IN YOUR AMAZON DYNAMODB TABLES
Now you can use AWS CloudTrail to log data-plane API activity to monitor, alarm, and archive item-level activity in your Amazon DynamoDB tables. You can use this information about item-level activity as part of an audit, to help address compliance requirements, and monitor which AWS Identity and Access Management (IAM) users, roles, and permissions are being used to access your table data.
The Amazon Elastic File System (EFS) Container Storage Interface (CSI) driver now supports dynamic provisioning of Kubernetes persistent volumes. Kubernetes applications requiring access to a shared file system on AWS can now have storage provisioned on demand, eliminating the need for cluster administrators to pre-provision volumes.
Amazon Redshift Spectrum is now available in four additional AWS Regions: Africa (Cape Town), Asia Pacific (Osaka), Europe (Milan), and AWS GovCloud (US-East)
AWS Glue Studio now provides the option to define transforms using SQL queries, allowing you to perform aggregations, easily apply filter logic to your data, add calculated fields, and more. This feature makes it easy to seamlessly mix SQL queries with AWS Glue Studio’s visual transforms while authoring ETL jobs.
AWS has expanded the availability of Amazon EC2 Inf1 instances to Europe (Milan), Europe (Stockholm), and AWS GovCloud (US) Regions. Inf1 instances are powered by AWS Inferentia chips, which AWS custom-designed to provide high performance and lowest cost machine learning inference in the cloud.
AWS IoT Core’s Apache Kafka action now supports username and password authentication for Amazon Managed Streaming for Apache Kafka (Amazon MSK)
Customers using the AWS IoT Core Apache Kafka action to connect their IoT applications to Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters can now use usernames and passwords to authenticate their rules. The Apache Kafka action for AWS IoT Core evaluates inbound messages and delivers them to your Amazon MSK or self managed Apache Kafka cluster.
AWS Client VPN has expanded the number of Availability Zones supported inside the US East (Northern Virginia), US West (Oregon), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Sydney), and Canada (Central) regions. This expansion gives you more zones to choose from and additional flexibility when associating subnets. This change also increases the number of concurrent connections you can make per endpoint in these Regions.
Amazon EC2 now supports Unified Extensible Firmware Interface (UEFI) boot when migrating virtual machines from on-premise to EC2 without having to convert them to Legacy BIOS. UEFI is a modern firmware that initializes your operating system.
Google Cloud Releases and Updates
Access Transparency logs contain a new field called
accessApprovals. This field lists the approvals that granted access to a resource that is enrolled in Access Approval. Access Transparency logs published before March 24, 2021 will not have this field populated. This feature is subject to Access Approval exclusions and only available for the services supported by Access Approval.
Anthos clusters on VMware
Anthos clusters on VMware 1.7.0-gke.16 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.7.0-gke.16 runs on Kubernetes 1.19.7-gke.2400.
The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting GKE On-Prem are 1.6, 1.5, and 1.4.
Cluster lifecycle improvements
The cluster upgrade process has changed. Instead of upgrading the admin cluster first, you can upgrade user clusters to the newer version without upgrading the admin cluster. The new flow, which requires upgrading
gkeadm, allows you to preview new features before performing a full upgrade with the admin cluster. In addition, the 1.7.0 version of
gkectlcan perform operations on both 1.6.X and 1.7.0 clusters.
Starting with version 1.7.0, you can deploy Anthos clusters on vSphere 7.0 environments in addition to vSphere 6.5 and 6.7. Note that Anthos clusters on VMware will phase out vSphere 6.5 support following VMware end of general support timelines.
GA: Node auto repair is now generally available and enabled by default for newly created clusters. When the feature is enabled,
cluster-health-controllerperforms periodic health checks, surfaces problems as events on cluster objects, and automatically repairs unhealthy nodes.
GA: vSphere resource metrics is now generally available and enabled by default for newly created clusters. When the feature is enabled, VM level resource contention metrics are collected and displayed in the VM health dashboards automatically created through out-of-the-box monitoring. You can use these dashboards to track VM resource contention issues.
GA: Dataplane V2 is now generally available and can be enabled in newly created clusters.
GA: Network Policy Logging is now generally available. Network policy logging is available only for clusters running Dataplane V2.
You can attach vSphere tags to user cluster node pools during cluster creation and update. You can use tags to organize and select VMs in vCenter.
BigQuery, BI Engine, Big Query Data Transfer and ML are now available in the Warsaw (europe-central2) region.
The API for updating
ResourceRecordSets in Cloud DNS is now available in GA.
Cloud Load Balancing
Subsetting for internal TCP/UDP load balancers lets you scale your internal TCP/UDP load balancer to support a larger number of backend VM instances per internal backend service.
This feature is in Preview.
Cloud Spanner regional instances can now be created in Warsaw (
Cloud SQL for PostgreSQL
Support available in europe-central2 (Warsaw)
Generally available: Start and stop virtual machine (VM) instances automatically using instance schedules. By automating the deployment of your VMs, instance schedules can help you optimize costs and manage VMs more efficiently.
General-purpose E2 and N1 machines are available in Warsaw, Poland
europe-central2 in all three zones. See VM instance pricing for details.
Disks, snapshots, and images are available in Warsaw, Poland
europe-central2 in all three zones. See Disks and image pricing for details.
Support for OS Login in VPC Service Controls is now Generally Available.
Config Connector version 1.44.0 is now available.
Added support for the
ContainerAnalysisNote resource (no config-connector CLI support)
mtu field to
nodeConfig.ephemeralStorageConfig field to
settings.backupConfiguration.transactionLogRetentionDays fields to
materializedView.query field in
nicType field in
Added support for acquisitions of
Fixed incorrect file extension for Terraform files output by the config-connector CLI.
config-connector CLI now supports a flag to filter out deleted
Added support for
IAPBrand (no config-connector CLI support)
Added support for
IAPIdentityAwareProxyClient (no config-connector CLI support)
Conflict Prevention is now turned off by default. The current implementation results in the
Ready condition destabilizing despite the resource reflecting user-desired state.
Work is enqueued to improve this behavior, but the functionality is turned off for new resources in the interim.
Webhook certificates that do not contain a SAN are now re-created on upgrade of the Config Connector operator.
Added support for
Dataflow SQL now supports user-defined functions (UDFs) written using SQL. For more information, see Dataflow SQL user-defined functions. This feature is in Preview.
The default Dataproc image is now image version 2.0.
Announcing the General Availability (GA) release of Dataproc cluster Stop/Start.
New sub-minor versions of Dataproc images:
- Updated Iceberg to version 0.11.0.
- Updated Flink to version 1.12.2.
The feed URL will also change from https://cloud.google.com/feeds/kubernetes-engine-release-notes.xml to https://cloud.google.com/feeds/gke-main-release-notes.xml. Google will automatically redirect from the old URL to the new one.
Workload Identity for Windows Server nodes is now available in GKE versions 1.18.16-gke.1200, 1.19.8-gke.1300, 1.20.4-gke.1500, and later.
Windows Server, version 1909 is reaching end of support on May 11, 2021. Newer Windows Server image versions are available in GKE versions 1.19.8-gke.1600+ and 1.20.4-gke.500+.
The Resource Manager v3 API has been released into public preview. For more information, see the API reference documentation
Virtual Private Cloud
For auto mode VPC networks, added a new subnet
10.186.0.0/20 for the Warsaw
europe-central2 region. For more information, see Auto mode IP ranges.
The ability to connect VM interfaces other than nic0 to a Shared VPC is now available in General Availability for instance templates and managed instance groups. This feature is available in the
gcloud command-line tool and the API.
Microsoft Azure Releases And Updates
Take advantage of new VM sizes that are now available when creating your Batch pools, including the latest HPC and GPU sizes.
If you're using IP filter, add your computer's IP address to the allow list to keep using IoT Hub in Azure portal.
Check out the new Power BI embedded analytics playground (public preview). You can find in the new playground experiences for developer and non-developers to get first-hand experience of Power BI embedded analytics.
AzCopy v10.9.0 now in preview supports Google Cloud Platform (GCP) to Microsoft Azure Storage Block Blob imports. This version also includes scanning logs which can help in debugging.
This support will unlock hybrid connectivity for you as you expand into new, IPv6-dependent markets or transition to IPv6 in your own networks.
Automate tasks across Azure & Non-Azure environment using PowerShell and Python based scripts.
Azure Container Registry is now generally available in Norway East region. Azure Container Registry handles private Docker container images as well as related content formats, such as Helm charts, OCI artifacts, and images built to the OCI image format specification.
ISVs can capture their images directly into Shared Image Gallery and select those images for publishing with an inline experience in Partner Center in just two steps.
Log Analytics enhances pinned parts Azure Dashboards experience.
UPComing Training & Events:
AWS announced the launch of the updated Advanced Architecting on AWS course this week. This instructor-led training course is designed for cloud architects who want to extend their baseline knowledge of AWS services. An expert AWS instructor will help you learn advanced architecting topics such as hybrid connectivity and hybrid AWS devices, networking with a focus on AWS Transit Gateway connectivity, AWS Container services, automation tools for CI/CD, security options, and much more.
AWS Training and Certification launched a new course entitled AWS Cloud Technical Essentials. Available for free on Coursera and edX, this course uses video lectures and demonstrations to teach the technical fundamentals of AWS. Upon course completion, learners will be able to make an informed decision about when and how to apply core AWS services for compute, storage, and database to different use cases.
New digital course: Amazon S3 Cost Optimization
AWS are excited to announce a free digital course: Amazon Simple Storage Service Cost Optimization. This advanced 60-minute course explores techniques and tools you can use to optimize your Amazon S3 costs. Designed for cloud architects, storage architects, developers, and operations engineers, it includes interactive lessons and video demonstrations.
Four new AWS digital training offerings for AWS End User Computing
AWS introduced four new digital training offerings that help you learn how to plan, deploy, secure, and manage cloud-based desktops and applications. The offerings are designed for desktop or virtual desktop infrastructure managers, IT administrators, and technical professionals interested in cloud-based virtualization. These free self-paced courses and curriculums include presentations, interactive e-learning modules, videos, demonstrations, and quizzes.
Azure Virtual Events
Microsoft have a full schedule of Virtual Events
A full list including session times and details are here : Azure Events
AWS events are pretty fluid at the moment, with most in-person events being cancelled or postponed. There are a number that have been taken online and full details can be found here: AWS Events
Thanks for reading again this week, we hope you found something useful.
hava.io allows users to visualise their AWS, GCP and Azure cloud environments in interactive diagram form including unique infrastructure, security and container views. hava.io continuously polls your cloud configuration and logs changes in a version history for later inspection which helps with issue resolution and provides history of all configs for audit and compliance purposes.
If you haven't opened a free hava.io account to see what the GCP, Azure and AWS automated diagram generator can do for your workflow, security and compliance needs, you can below - if you have questions, please get in touch.
You can reach us on chat, email email@example.com to book a callback or demo.
Read more about Hava Azure Infrastructure Diagrams here.
Read more about Hava AWS Architecture Diagrams here.
Read more about Hava GCP Architecture Diagrams here