All the news from AWS, Azure and GCP for the week ending October 16th 2020.
AWS Glue crawlers now support Amazon DocumentDB (with MongoDB compatibility) and MongoDB collections. You can now use AWS Glue crawlers to infer schema of Amazon DocumentDB (with MongoDB compatibility) and MongoDB collections and create or update a table in the Glue Data Catalog. A configuration option allows you to specify if you want the crawler to crawl the entire data set or select a sample of the data to reduce crawl time.
The storage space allocated to your Amazon Aurora database cluster will now dynamically decrease when you delete data from the cluster. The storage space already automatically increases up to a maximum size of 128 tebibytes (TiB), and will now automatically decrease when data is deleted. You only pay for the storage you use. Starting September 21, 2020, dynamic resizing for storage space is being enabled region by region for Aurora MySQL versions 1.23 and 2.09 , and Aurora PostgreSQL versions 10.13 and 11.8 , and is expected to be enabled across all Aurora regions by end of November 2020.
Starting today, Amazon RDS for PostgreSQL will concurrently upgrade all of your in-region read replicas along with the upgrade of your primary database instance during major version upgrades of PostgreSQL.
AWS Purchase Order Management gives you the ability to define and manage your purchase orders (“POs”) for AWS services in a way that meets your business needs. You can manage your PO information from your AWS Billing Console, configure multiple POs, and define the rules of how POs map to your AWS invoices. You can also track your PO status and balance, and set up email notifications to proactively validate your POs.
Streaming extract, transform, and load (ETL) jobs in AWS Glue can now read data encoded in the Apache Avro format. Previously, streaming ETL jobs could read data in the JSON, CSV, Parquet, and XML formats. With the addition of Avro, streaming ETL jobs now support all the same formats as batch AWS Glue jobs.
AWS Graviton2-based database instances are now generally available for Amazon Relational Database Service (RDS). Graviton2 instances provide up to 35% performance improvement and up to 52% price/performance improvement for RDS open source databases depending on database engine, version, and workload. You can launch these database instances when using Amazon RDS for MySQL, Amazon RDS for PostgreSQL, and Amazon RDS for MariaDB. Support for Amazon Aurora is coming soon.
Starting today, you can define the action you want to take in your account when a budget exceeds its threshold (actual or forecasted amounts). This level of control will allow you to reduce unintentional overspending in your account. You can choose among three action types: Identity and Access Management (IAM) policy, Service Control policy (SCPs), or target running instances (EC2 or RDS).
AWS End of Support Migration Program (EMP) for Windows Server provides the tooling to migrate your legacy applications from Windows Server 2003, 2008, and 2008 R2 to newer, supported versions on AWS, without any refactoring.
AWS Identity and Access Management (IAM) Access Analyzer now supports archive rules that allow you to retroactively mark existing findings as intended. Archive rules automatically archive new findings for public and cross-account access that meet the criteria you define. Now, you can apply the rules retroactively to mark existing findings as intended. For example, you can create a rule to archive all findings for a specific S3 bucket that you regularly grant read access to. This lets you focus on remediating findings that help you reduce broad access.
To coincide with AWS's new AWS Budgets Actions launch, they are reducing the price for budget-days from $.02 to free. Before this reduction, AWS customers received 62 free budget-days per month before paying $.02 per budget-day for all active budgets within a regular account or consolidated billing family. AWS customers who simply want to monitor their cost, usage, RI or Savings Plans coverage and utilization can now do so free of charge.
Amazon launches CloudWatch Synthetics Recorder, a Chrome browser extension, to help you create canaries more easily. The Recorder records your click and type actions on a website and automatically generates a script that you can use to create a canary that follows the same user click and type actions.
Customers using container services including Amazon Elastic Container Service (ECS), Amazon Elastic Kubernetes Services (EKS), or self-managed Kubernetes can now send their container logs to S3 using the Fluent Bit log router. Fluent Bit allows customers to route container logs to various AWS and partner monitoring solutions including Amazon CloudWatch, Amazon Kinesis, Datadog, Splunk, and now Amazon S3.
Streaming extract, transform, and load (ETL) jobs in AWS Glue can now ingest data from Apache Kafka clusters that you manage yourself. Previously, AWS Glue supported reading specifically from Amazon Managed Streaming for Apache Kafka (Amazon MSK). With this update, AWS Glue allows you to perform streaming ETL on data from Apache Kafka whether it is deployed on-premises or in the cloud.
Amazon Aurora now supports Database Activity Streams in the South America (Sao Paulo), Middle East (Bahrain), Africa (Cape Town), and Europe (Milan) regions. Database Activity Streams for Amazon Aurora with MySQL compatibility and Amazon Aurora with PostgreSQL compatibility provides a near real-time stream of database activities in your relational database. When integrated with third party database activity monitoring tools, Database Activity Streams can monitor and audit database activity to provide safeguards for your database and help you meet compliance and regulatory requirements.
This free new digital training curriculum contains modules that explain the benefits of and process for moving from self-managed databases to fully-managed database solutions in the cloud. The four-hour fundamental curriculum includes eight self-paced courses with video demonstrations and is designed for data platform engineers, database developers, and solutions architects.
Kubernetes is rapidly evolving, with frequent feature releases and bug fixes. Highlights of the Kubernetes 1.18 release include Topology Manager reaching beta status, a new beta of Server-side Apply , and a new IngressClass resource for the Ingress specification which makes it simpler to customize Ingress configuration. Additionally, you can now configure the behavior of horizontal pod autoscaling . Learn more about Kubernetes version 1.18 in the Kubernetes project release notes .
Amazon Rekognition content moderation is a deep learning-based service that can detect inappropriate, unwanted, or offensive images and videos, making it easier to find and remove such content at scale. Amazon Rekognition provides a detailed taxonomy of moderation categories such as 'Explicit Nudity', 'Suggestive', ‘Violence’, and 'Visually Disturbing'. Starting today, customers can detect 6 new categories - 'Drugs', ‘Tobacco’, ‘Alcohol’, ‘Gambling’, ‘Rude Gestures’, and ‘Hate Symbols’. In addition, customers also get improved detection rates for already supported categories. Using Amazon Rekognition moderation APIs, social media, broadcast media, advertising, and e-commerce customers can create a better user experience, provide brand safety assurances to advertisers, or comply with local and global regulations.
Amazon CloudWatch Application Insights, working closely with customers, has introduced an enhanced user interface to make setting up and managing your enterprise application monitoring even more straight forward. CloudWatch Application Insights is a capability that helps enterprise customers easily setup application monitoring and enhanced observability for AWS resources. The user interface improvements streamline the steps for doing this in a more intuitive and consistent approach.
Starting this week AWS announced you can use easily restore a new Amazon RDS for MySQL database instance from a backup of your existing MySQL 8.0 database, whether it’s running on Amazon EC2 or outside of AWS. This is done by using Percona XtraBackup to create a backup of your existing MySQL database, uploading the resulting files to an Amazon S3 bucket, and then creating a new Amazon RDS DB instance through the RDS Console or AWS Command Line Interface (CLI).
Amazon EMR now allows you to leverage AWS Lake Formation for defining and enforcing fine-grained access control policies for Apache Spark applications. Previously, this feature was in beta.
You can now discover endpoints registered in AWS Cloud Map with optional parameters that filter the returned results only when there is a matching custom attribute. AWS Cloud Map is a cloud resource discovery service. Using AWS Cloud Map, you can define custom names for your application resources, such as Amazon EC2 instances, Amazon ECS tasks, Amazon S3 buckets, or any other cloud resource. Your application can then discover the location and metadata of cloud resources associated with these custom names via AWS SDK or by making authenticated API calls.
Docker is a widely used container technology. Since it’s now a GCP Dataproc optional component, Docker daemons can now be installed on every node of the Dataproc cluster. This will give you the ability to install containerized applications and interact with Hadoop clusters easily on the cluster.
In addition, Docker is also critical to supporting these features:
Running containers with YARN
Portable Apache Beam job
Running containers on YARN allows you to manage dependencies of your YARN application separately, and also allows you to create containerized services on YARN.
Organizations all over the world rely on Cloud CDN for fast, reliable web and video content delivery. Now, GCP are making it even easier for you to take advantage of their global network and cache infrastructure by reducing the cost of Cloud CDN for your content delivery going forward.
First, they are reducing the price of cache fill (content fetched from your origin) charges across the board, by up to 80%. You still get the benefit of GCP's global private backbone for cache fill though—ensuring continued high performance, at a reduced cost. They’ve also removed cache-to-cache fill charges and cache invalidation charges for all customers going forward.
This price reduction, along with GCP's recent introduction of a new set of flexible caching capabilities, makes it even easier to use Google Cloud CDN to optimize the performance of your applications.
This month, BigQuery users have a lot of new reasons to smile about with the release of new user-friendly SQL capabilities now generally available.
New features include add column and truncate table operations, new external storage capabilities plus some scripting and SQL improvements.
As a developer building a new application, you want to focus on writing code, not containerizing it. And if you already use containers, you know that creating a good, secure container image can be complicated and time-consuming. Today we’re launching broad support across Google Cloud for buildpacks—an open-source technology that makes it fast and easy for you to create secure, production-ready container images from source code and without a Dockerfile.
Availability zones are one of the best ways to mitigate risks from outages and disasters. With that in mind, we are announcing the preview for zone redundancy in Azure Cache for Redis.
Availability Zones on Azure
Azure Availability Zones are geographically isolated datacenter locations within an Azure region, providing redundant power, cooling, and networking. By maintaining a physically separate set of resources with the low latency from remaining in the same region, Azure Availability Zones provide a high availability solution that is crucial for businesses requiring resiliency and business continuity.
Redundancy options in Azure Cache for Redis
Azure Cache for Redis is increasingly becoming critical to Azure customers’ data infrastructure. As a fully managed service, Azure Cache for Redis provides various high availability options. By default, caches in the standard or premium tier have built-in replication with a two-node configuration—a primary and a replica hosting two identical copies of your data.
Google Cloud Virtual Events
To support the unique needs of GCP customers in Europe, Middle East, and Africa, on 29 September GCP are kicking off a brand-new Next OnAir event exclusively for EMEA.
Google Cloud Next OnAir EMEA offers a full roster of curated content, including more than 30 new sessions specially tailored to the region. Join Google experts and local customers to learn how organizations are already transforming in the cloud, and connect and collaborate with industry experts to solve your toughest challenges.
Each week Google will be highlighting a different focus:
20 Oct: Data analytics, data management, databases, and Cloud AI—Learn about how to migrate and do more with your data on a serverless, fully-managed platform and with artificial intelligence.
27 Oct: Application modernization and business application platform—Explore how to develop and modernise applications with open source and other software, and how APIs give you better visibility and control.
Register today, for free, on the Next OnAir EMEA website. You’ll get full access to all 30+ sessions being presented throughout the five-week event alongside the more than 250 sessions created for the global Google Cloud Next ’20: OnAir program.
Azure Virtual Events
Microsoft have a full schedule of Virtual Events
A full list including session times and details are here : https://azure.microsoft.com/en-us/community/events/
AWS events are pretty fluid at the moment, with most in-person events being cancelled or postponed. There are a number that have been taken online and full details can be found here: https://aws.amazon.com/events/
Thanks for reading again this week, we hope you found something useful.
hava.io allows users to visualise their AWS, GCP and Azure cloud environments in interactive diagram form including unique infrastructure, security and container views. hava.io continuously polls your cloud configuration and logs changes in a version history for later inspection which helps with issue resolution and provides history of all configs for audit and compliance purposes.
If you haven't taken a hava.io free trial to see what the GCP, Azure and AWS automated diagram generator can do for your workflow, security and compliance needs - please get in touch.
You can reach us on chat, email firstname.lastname@example.org to book a callback or demo.