Here's a round up of all things GCP, Azure and AWS for the week ending Friday 28th May 2021.
To stay in the loop, make sure you subscribe on the right - There's a new Newsletter series starting soon that will keep you up to date with all our new releases, enhancements and capabilities and will also showcase lesser known but powerful features that you may not be aware of.
Of course we'd love to keep in touch at the usual places. Come and say hello on:
Source: aws.amazon.com
We're excited to announce that Redshift ML is now generally available. Amazon Redshift ML enables you to create, train, and deploy machine learning (ML) models using familiar SQL commands. With Amazon Redshift ML, you can now leverage Amazon SageMaker, a fully managed machine learning service, without moving your data or learning new skills
AWS Glue DataBrew now supports nest and unnest transformations to help users pack or unpack data into columns to manipulate their datasets. With these transformations, users can now easily extract data from nested json string fields or combine data without writing any code.
All AWS System and Organization Controls (SOC) (1,2,3) compliant services deployed in AWS Wavelength are now in scope. You can download the SOC Report in AWS Artifact. AWS maintains certifications through extensive audits of its controls to ensure that information security risks that affect the confidentiality, integrity, and availability of company and customer information are appropriately managed.
Amazon FSx for Lustre, a service that provides cost-effective, high-performance, scalable file storage for compute workloads, now supports data compression, enabling you to reduce the cost of storage for machine learning, high performance computing (HPC), and analytics data sets.
Amazon Kendra is an intelligent search service powered by machine learning, that enables organizations to provide more relevant information to customers and employees, when they need it. Starting today, AWS customers can use Amazon Kendra’s Query Suggestions feature to guide end users towards more commonly asked questions and save them time in their search for the information they need.
Today, we are excited to announce the general availability of Amazon Elastic Container Service (ECS) Anywhere, a new capability of Amazon ECS. Amazon ECS Anywhere provides a fully managed container orchestration service that enables customers to run and manage containerized applications on-premises using the same APIs, cluster management, workload scheduling, monitoring, and deployment pipelines they use with Amazon ECS today. Customers can use Amazon ECS Anywhere on their own infrastructure by connecting their servers or instances to the managed Amazon ECS control plane. Enterprises who need to keep their data on-premises for latency or regulatory reasons can now use a single and consistent cloud-managed container orchestrator across their hybrid footprint.
You can now create Amazon API Gateway REST APIs to trigger a Synchronous Express Workflow from AWS Step Functions and respond with the output of that workflow.
You can now route traffic from your custom root domains (such as example.com) to your Lightsail container services using Lightsail DNS. Lightsail DNS already supported creating CNAME records mapping to your container services. With this launch, Lightsail DNS supports aliasing your custom root domain to your container service’s default domain which was not possible before. Thus, you can create an A/AAAA record in your Lightsail hosted DNS zone pointing the zone apex to a container service.
The Alarms feature in AWS IoT Events allows you to set up, visualize and manage rule-based alerts for devices, equipment, and processes. You can receive alerts via SMS or email in near-real time when equipment data breaches thresholds, allowing operations teams to take timely actions to reduce unplanned equipment downtime.
AWS Training and Certification has launched four, new self-paced digital courses, Building Modern .NET Applications on AWS, How to Buy Cloud - Strategies for Cloud Procurement, Improve your Java Code using Amazon CodeGuru, and Improve your Python Code using Amazon CodeGuru on Coursera, edX, and Udemy.
AWS Certificate Manager (ACM) Private Certificate Authority (CA) now supports using S3 Block Public Access when storing certificate revocation links (CRL) in S3 buckets.
Starting today, Amazon EC2 High Memory Instances with 6TB, 9TB, and 12TB of memory (u-6tb1.56xlarge, u-6tb1.112xlarge, u-9tb1.112xlarge, and u-12tb1.112xlarge) are available in US West (Oregon) and AWS GovCloud (US-West) regions. The global availability of these new instances now includes US East (N. Virginia), US West (Oregon), AWS GovCloud (US-West), Europe (Ireland, Frankfurt), and Asia Pacific (Singapore) AWS regions. Customers can start using these new High Memory instances with OD and Savings Plan purchase options. For more information, visit the Amazon EC2 High Memory page.
Amazon Aurora with MySQL compatibility now supports T3.large instances, in addition to the T3.medium, T3.small, and R5 class instances already available. Using T3.large instances with Amazon Aurora MySQL is a cost-effective option for smaller workloads such as test, dev, and QA, while still giving you the option to use larger class instances for production deployments.
AWS Security Hub now supports a bidirectional integration with Atlassian Jira Service Management (JSM) . You can now automatically create and update issues in Atlassian JSM from AWS Security Hub findings. Updates to those issues in Atlassian JSM will be synced with the findings in AWS Security Hub. This integration is available via the AWS Service Management Connector for JSM app. After downloading the app, you can decide if you want to send all findings, or only certain findings with specific severity levels to JSM. When you make an update to various fields in an issue, such as state or priority, those changes are automatically sent to AWS Security Hub so that AWS Security Hub always has the latest and correct information about that issue.
You can now launch Apache Airflow 2.0 environments on Amazon Managed Workflows for Apache Airflow (MWAA). Apache Airflow 2.0 is the latest version of the popular open-source tool that helps customers author, schedule, and monitor workflows.
Amazon Aurora MySQL-Compatible Edition now maintains read availability through writer node restarts. Previously, when a writer node restarted, all nodes in an Aurora MySQL cluster restarted as well. With today’s launch, reader nodes will continue to serve read requests during a writer node restart, improving read availability in the cluster.
This week, AWS IoT is announcing the general availability of Fleet Hub for AWS IoT Device Management. The feature enables customers to easily create a fully managed web application to view and interact with their device fleets to monitor fleet and device health, respond to alarms, take remote actions, and reduce time for troubleshooting.
Amazon SageMaker Pipelines, the first purpose-built continuous integration and continuous delivery (CI/CD) service for machine learning (ML), is now integrated with SageMaker Experiments, a capability that lets customers organize, track, compare, and evaluate their ML experiments. Customers can now compare metrics such as model training accuracy across multiple executions of their SageMaker Pipelines just as easily as they compare such metrics across multiple trials of a ML model training experiment. SageMaker Pipelines automatically creates an Experiment with the pipeline name and an Experiment trial for every execution of the pipeline. The creation of an experiment for a pipeline and a trial for every pipeline execution is turned on by default. You can choose to opt-out of the auto-creation.
AWS IoT is excited to announce the public preview of “Job Templates” for AWS IoT Device Management Jobs. Job Templates make deploying remote actions faster, easier, and more secure. IoT developers and fleet administrators can pre-define the remote operations to be executed on their IoT devices, and specify key deployment parameters like rollout rates, abort thresholds, and timeout criteria. Fleet operators and technicians who focus on fleet monitoring and troubleshooting can identify and select specific groups of devices as targets, and use these pre-defined Job Templates to securely deploy remote actions to their targets.
AWS Transfer Family customers can now use AWS Managed Microsoft Active Directory (AD), on-premises and self-managed AD in AWS to authenticate their file transfer end users, enabling seamless migration of file transfer workflows that rely on AD, without changing end users’ credentials or needing a custom authorizer.
Amazon Managed Streaming for Apache Kafka (Amazon MSK) now supports Apache Kafka version 2.7.1 for new and existing clusters. Apache Kafka 2.7.1 includes several bug fixes. For a complete list of fixes, see the Apache Kafka release notes for 2.7.1.
Amazon Braket now offers a fully managed, density matrix simulator, DM1, to simulate quantum circuits with noise. DM1 helps you investigate the effects of realistic noise on your quantum algorithms to inform error mitigation strategies to get more accurate results from today’s quantum computing devices. With DM1, you can simulate circuits with up to 17 qubits and run up to 35 simulations in parallel, to speed up your experiments. For rapid prototyping and debugging, you can now also use the local noise simulator in the Amazon Braket SDK.
AWS Database Migration Service (AWS DMS) has expanded functionality by adding support for PostgreSQL version 13 in preview mode. Using AWS DMS, you can now perform live migrations from any AWS DMS supported sources to PostgreSQL 13 databases, and from PostgreSQL 13 databases to any AWS DMS supported targets .
Data subscribers on AWS Data Exchange can now pay for their data products in installments with custom payment schedules created by their data provider. Before this launch, subscribers paid the total price of their private offer at the start of the subscription. With the introduction of payment schedules, providers can now create private offers for subscribers that generate up to 36 invoices on specific dates throughout a subscription.
Starting today Amazon Relational Database Service (RDS) for Oracle supports the creation of a DB instance with a single pluggable database (PDB) using the Oracle multitenant architecture, which enables the DB instance to operate as a multitenant container database (CDB). A PDB is a set of schemas, schema objects, and non-schema objects that logically appears to a client as a non-CDB.
AWS Lambda Extensions are a new way to integrate your favorite operational tools for monitoring, observability, security, and governance with AWS Lambda. Starting today, extensions are generally available with new performance improvements and an expanded set of partners including Imperva, Instana, Sentry, Site24x7, and the AWS Distro for OpenTelemetry.
You can now view and manage all Amazon CloudWatch Logs transactional API service quotas with Service Quotas. Service Quotas consolidates the default values and your account specific quotas for CloudWatch Logs in one single view with the Service Quotas console. With the CloudWatch Logs and Service Quotas integration you can now easily view and adjust your quotas.
AWS Compute Optimizer is a service that recommends optimal AWS resources for your workloads to reduce costs and improve performance by using machine learning to analyze historical utilization metrics. Today, Compute Optimizer is launching several updates to its EC2 instance type recommendations. Through these updates, Compute Optimizer has improved recommendation quality, doubled the number of supported EC2 instance types, offered deeper insights into the recommendations, and provided customers a way to identify OS configuration changes needed to apply recommendations.
Starting today, you can create and manage reserved capacity on Amazon EC2 instances on your Outposts. You can also share the Capacity Reservations on Outposts with other AWS accounts within your organization. With Capacity Reservations on Outposts, you can ensure that you always have access to EC2 capacity when you need it, for as long as you need it.
Amazon Lumberyard 1.28 Beta is now available for download, bringing you a range of updates including updated Twitch Gem, an improved setup and installation experience, plus various smaller improvements and over 40 bug fixes.
You can now use Amazon EC2 On-Demand Capacity Reservations (ODCRs) while creating clusters in Amazon EMR. When rotating a long running cluster, you might have strict requirements on the instance types and Availability Zones for the new instances you provision. Using Capacity Reservations, you can get capacity assurance to complete the cluster rotation without interruptions. You can also use Capacity Reservations to provision a group of successive short-lived clusters for individual workloads such that when you terminate a cluster, the next cluster can use the Capacity Reservations.
Cloud Data Fusion
Cloud Data Fusion version 6.4.1 is now available. To upgrade, see Upgrading instances and pipelines. This release is in parallel with the CDAP 6.4.1 release.
In Cloud Data Fusion version 6.4.1, Replication supports the Datetime data type in BigQuery targets. You can now read and write to tables that contain Datetime fields.
Compute Engine
Preview: Disable simultaneous multithreading (SMT) on VMs. For more information, see Disabling simultaneous multithreading.
Generally Available: Enable nested virtualization directly when creating a VM. For more information, see Nested virtualization overview.
Datastream
Datastream is a serverless and easy-to-use change data capture (CDC) and replication service. It allows you to synchronize data across heterogeneous databases and applications reliably, and with minimal latency and downtime.
Datastream supports streaming from Oracle and MySQL databases into Cloud Storage. The service offers streamlined integration with Dataflow templates to power up-to-date materialized views in BigQuery for analytics, replicate your databases into Cloud SQL or Spanner for database synchronization, or leverage the event stream directly from Cloud Storage to realize event-driven architectures.
Benefits of Datastream include:
Documentation for Datastream includes a quickstart, conceptual content, how to use this service through the user interface, REST API calls, and gcloud, an API tutorial, and reference, support, and resource-related information. Click here to access the documentation.
GKE
1.21 Features
The following features are introduced in version 1.21:
CronJob (GA)
The CronJob API has graduated to General Availability (GA), bringing performance improvements and allowing scheduled jobs to be run using a stable API.
batch/v1
group/version.batch/v1beta1
group/version is deprecated, and will be removed in version 1.25. See the migration guide for details.PodDisruptionBudget (GA)
The PodDisruptionBudget has graduated to GA, allowing pod evictions to be controlled using a stable API.
policy/v1
group/version.policy/v1beta1
group/version is deprecated, and will be removed in version 1.25. See the migration guide for details.EndpointSlice (GA)
The EndpointSlice API has graduated to GA, bringing performance improvements over the v1 Endpoints API.
discovery.k8s.io/v1
.discovery.k8s.io/v1beta1
group/version is deprecated, and will be removed in version 1.25. See the migration guide for details.Default namespace label (Beta)
Namespace API objects now have a kubernetes.io/metadata.name
label matching their metadata.name
field to allow selecting any namespace by its name using a label selector. This can be used for objects which select namespaces by label, such as admission webhooks and network policies.
Bound service account token volumes (Beta)
The API credentials injected into containers at/var/run/secrets/kubernetes.io/serviceaccount/token
are now time-limited, auto-refreshed, and invalidated when the containing pod is deleted.serviceaccount_stale_tokens_total
and the audit annotation authentication.k8s.io/stale-token
can be used to monitor for workloads that depend on the extended lifetime and are continuing to use tokens even after a refreshed token is provided to the container.k8s.io/client-go
version 11.0.0+ and 0.15.0+ reload tokens automatically.GKE clusters running version 1.18 or later now support container native Cloud DNS (available in Preview). Cloud DNS can be used as the in-cluster DNS provider instead of kube-dns.
Memorystore for Memcached
Added support for the Reserved Memory configuration for Memorystore for Memcached. For more information, see Memory management best practices.
Memorystore for Redis
Added support for specifying an IP address range for the private service access connection mode. For more information, see Custom ranges with private services access.
Security Command Center
Security Command Center Premium has launched project- and folder-level roles in general availability. The feature lets you grant users Identity and Access Management (IAM) roles for specific folders and projects. You have more granular control over who can access what resources throughout your organization. For more information, see Access control.
You must be a Security Command Center Premium customer to use this feature. Security Command Center Standard continues to support granting roles only at the organization level. To subscribe to Security Command Center Premium, contact your sales representative or fill out our inquiry form.
Use T-SQL in Azure Synapse to read Delta Lake format.
Support for 73 languages, prebuilt ID model, invoice line item extraction, table header extraction tagging as table, and more generally available in Azure Form Recognizer, part of Applied AI Services.
Log Analytics, a feature of Azure Monitor, is now generally available for customers to start collecting telemetry and analyzing their services for health and usage in South India.
Application Insight, a part of Azure Monitor, is now generally available in South India region for customers to collect telemetry and analyze their service health in production environments.
Blob inventory allows you to get a full listing of all blobs within a storage account or container. This can be used in conjunction with Synapse to calculate per container metrics such as capacity and count.
PowerShell users can use Durable Functions create serverless workflows to orchestrate complex, long-running automation processes.
Durable Functions, an extension to the Azure Functions runtime, adds two new storage provider options in public preview.
GitHub’s integration with Azure Security Center is now in public preview—offering new ways for you to integrate security and compliance into early stages of the software development lifecycle.
Azure Purview now supports automated scanning from a Hive Metastore Database to extract schema and lineage between data.
Access unused Azure compute capacity at discounted rates using Spot VMs on Azure Databricks
Event Hubs Premium is a new product SKU that is tailor-made for high-end event streaming scenarios that require elastic, superior performance with predictable latency.
Beginning in June 2021, enjoy the Azure Database for MySQL - Flexible Server B1ms compute SKU and 32GB storage free offer for the first 12 months.
Customers and developers can use the Microsoft Build of OpenJDK to build their Java applications, and commercial support is available for applications running on Azure and Azure Stack.
Use Azure SQL Database ledger (preview) to eliminate the additional cost, complexity, and performance overhead of decentralized blockchain technology and enable tamper-evidence through cryptographic verification of your centralized data store.
You now have the ability to perform path-level updates on specific fields/properties in a document with partial document update for Azure Cosmos DB.
Azure Marketplace offers to run JBoss EAP on Azure Virtual Machines and Virtual Machine Scale Sets are now available. Customers can quickly move existing Java EE/Jakarta EE workloads to the cloud with limited code changes.
IBM WebSphere Application Server (Traditional) Network Deployment on Azure Linux Virtual Machines is now available, jointly developed and supported by IBM and Microsoft.
Azure API Management announces native support for WebSocket APIs.
Create a managed service built on top of the Confidential Consortium Framework that spins up a permissioned blockchain network of nodes within Azure confidential computing.
New feature: Prebuilt Docker images for Inferencing, now in public preview
Azure IoT Verified Telemetry provides data reliability and is available as part of Dependable IoT SDK and can be seamlessly integrated into current Azure IoT offerings such as Azure RTOS and IoT PnP.
The new, standard Question Answering feature provides users the capability to answer questions over a passage of text without having to create knowledgebases and manage additional storage
Now generally available, use Text Analytics for Health to detect words and phrases mentioned in unstructured text as entities in the healthcare and biomedical domain.
Logic Apps has been updated with new hosting options, improved performance, and developer workflows.
Now generally available, use Document Translation to translate documents at scale.
The latest release of the Python extension for VS Code comes with a much improved experience for Python developers thanks to the new Pylance language server in general availability.
Developer Velocity Lab (DVL) is a new research initiative that includes contributions across Microsoft, its research labs and GitHub.
You can secure your custom domains at no additional cost.
AKS on Azure Stack HCI is a Kubernetes based platform that developers and administrators can use to build, deploy and manage containerized applications in their datacenters and edge locations.
Azure AI is introducing a new category of services called Azure Applied AI Services. These services accelerate time-to-value in building AI solutions for common business processes. Within this category, Azure Video Analyzer is now in public preview. The service brings Live Video Analytics and Video Indexer together so developers can quickly build AI-powered video analytics solutions.
Azure AI is introducing a new category of services called Azure Applied AI Services. These services accelerate time-to-value in building AI solutions for common business processes. Within this category, there are important enhancements and status updates for Azure Bot Service and Azure Metrics Advisor.
New features now generally available including Data Labeling – Image Instance Segmentation and MLflow support
Managed Online and Batch Endpoints, in public preview, allow for seamless deployment and operationalization
A new workbooks-based insights experience for all log analytics workspaces available per resource, and at scale.
Logs export can be separated between multiple destinations to overcome destination rate limit and to maintain access control in destinations.
Query packs are a new capability within log analytics intended to make creating, saving, managing, and discovering log analytics queries simpler and more efficient.
Customers writing Java apps on Azure App Services can now onboard to application insights for application monitoring, without making any code changes. App Service Linux, both code-based and custom containers, and App Service Windows code-based are supported.
This enables real-time communication when users are behind Network Address Translation (NAT) or firewalls.
Add voice and video calling capabilities to applications that run on Microsoft Windows.
Developers can simplify the cross-channel user experiences within their applications with Azure Communication Services UI Library.
Use cluster auto-upgrade feature while minimizing interruptions.
You can now run Event Grid on Kubernetes clusters anywhere to manage routing of all your events from any source to any destination.
Benefit from enhanced pod creation speed and stability when creating Windows server containers in AKS.
AKS customers can now benefit from new features in Kubernetes 1.21
Azure Kubernetes Service makes it easier for you to achieve FedRAMP compliance by supporting FIPS compliant nodes
Azure Kubernetes Service (AKS) makes it easier for regulated industries, such as healthcare & finance, to adopt Kubernetes for their regulated workloads
You can now enable host-based encryption in AKS for end-to-end security.
Encrypt data inside the client application and never reveal the plain data or encryption keys to the database engine with Always Encrypted for Azure Cosmos DB.
Achieve unified management and access control across Azure and AKS resources.
Improve ingestion latency, query performance, and discoverability by setting up resource-specific diagnostics in log analytics with Azure Cosmos DB cost-optimized diagnostics.
Enable fine-grained access control to data and leverage Azure Active Directory (AAD) authentication with Azure Cosmos DB role-based access control (RBAC), now generally available.
Develop and test locally at no cost on Linux and macOS with our new Azure Cosmos DB Linux emulator.
Optimize read costs in Azure Cosmos DB by enabling an integrated cache, an in-memory cache that is easy to configure and requires only minimal application code changes.
Optimize costs for spiky or intermittent workloads with Azure Cosmos DB serverless, a pure consumption offer where you are only billed for the Request Units and storage capacity consumed.
Get the first 1000 RU/s provisioned throughput and 25 GB storage free each month for the lifetime of one Azure Cosmos DB account with the Azure Cosmos DB free tier.
With the support of Citus 10, columnar storage, and more are now included in Azure Database for PostgreSQL – Hyperscale (Citus), a managed service running the open source Postgres database on Azure.
Utilize the B1ms compute SKU and 32GB storage free offer (coming June 2021) on Azure Database for PostgreSQL – Flexible Server, a managed service running the open source Postgres database.
You can now run Web Apps against a range of fully managed App Service plans or choose to deploy against Azure Kubernetes Service or your own Kubernetes clusters connected through Azure Arc that are running on-premises, at the edge, or in other clouds.
Included with Citus 10, columnar compression is now supported in the Hyperscale (Citus) option in Azure Database for PostgreSQL, a managed service running the open source Postgres database on Azure.
Create Azure Database for PostgreSQL - Hyperscale (Citus) server groups with Postgres 12 and Postgres 13, in addition to previously supported Postgres 11.
Azure Stack HCI built-in free software trial has been extended from 30 to 60 days giving more time to evaluate the solution before making a purchasing decision.