Here's a round up of all things GCP, Azure and AWS for the week ending Friday 9th April 2021.
Here at Hava our dev teams have been busy refining our self-hosted deployment and adding more security options around the sign on and team management process.
To stay in the loop, make sure you subscribe on the right - There's a new Newsletter series starting soon that will keep you up to date with all our new releases, enhancements and capabilities and will also showcase lesser known but powerful features that you may not be aware of.
Of course we'd love to keep in touch at the usual places. Come and say hello on:
Amazon RDS for SQL Server now support SQL Server extended events, a performance monitoring system that can be used to monitor and troubleshoot performance problems in SQL Server.
Amazon Athena users can now view the execution plan for their queries. When querying large, complex datasets, users are often unaware of how their query will be executed or how optimizations will impact performance. Now generally available for Amazon Athena, the EXPLAIN statement helps users understand and improve the efficiency of their queries.
Amazon EC2 Auto Scaling Warm Pools help applications scale out faster and save money by requiring fewer continuously running instances. With Warm Pools, customers can improve the elasticity of their applications by creating a pool of pre-initialized EC2 instances that are ready to quickly serve application traffic. Additionally, Warm Pools offer a way to save compute costs by placing pre-initialized instances in a stopped state.
You can now use AWS CloudFormation templates to create Amazon Connect Customer Profiles resources. This feature helps you to use CloudFormation to deploy Amazon Connect Customer Profiles resources — along with the rest of your AWS infrastructure — in a secure, efficient, and repeatable way. You can use CloudFormation templates to specify Amazon Connect Customer Profiles domains and pre-built connectors to bring in customer data from your applications to create customer profiles.
You can now use AWS PrivateLink to access Amazon Connect Customer Profiles directly as a private endpoint within your secure, virtual network using a new interface VPC endpoint in your Virtual Private Cloud. This extends the functionality of existing gateway endpoints by enabling you to access Amazon Connect Customer Profiles using private IP addresses. API requests and HTTPS requests to Amazon Connect Customer Profiles from your on-premises applications are automatically directed through interface endpoints, which connect to Customer Profiles securely and privately through PrivateLink.
This week, Amazon Lookout for Equipment is generally available to all AWS customers. Amazon Lookout for Equipment is a machine learning (ML) industrial equipment monitoring service that detects abnormal equipment behavior so customers can take action and avoid unplanned downtime.
eksctl , the official CLI for Amazon Elastic Kubernetes Service (Amazon EKS), now lets you create node groups inside an Amazon EKS cluster by simply providing node requirements in vCPUs, memory, or GPUs. Additionally, eksctl now lets you preview the cluster configuration before creating a new cluster or a new node group by using the new dry-run mode
AWS Control Tower is releasing four new, less restrictive, mandatory preventative S3 Log Archive guardrails and changing the guidance of the four previous, more restrictive, preventative S3 Log Archive guardrails from mandatory to elective. With these guardrail changes you can now separate S3 Log Archive governance for resources created by AWS Control Tower from governance for the S3 resources you create.
AWS IoT Analytics is a fully managed service that makes it easy to collect, pre-process, enrich, store and analyze IoT data at scale to run sophisticated analytics on massive volumes of IoT data and gain insights into how IoT devices are operating without having to worry about the complexity typically required to build an analytics platform.
AWS customers can use ServiceNow as a single place to track operational items from AWS Systems Manager OpsCenter. ServiceNow users can now view, investigate, and resolve operational items related to their AWS resources, while using their existing workflows in ServiceNow. Additionally, they can use AWS Systems Manager Automation runbooks from ServiceNow to remediate known issues. AWS Systems Manager OpsCenter enables operators to track and resolve operational items related to AWS resources in a central place, helping reduce time to issue resolution.
AWS RoboMaker now supports the ability to configure simulation tools to diagnose or interact with RoboMaker simulation jobs. This simulation tool configuration feature provides developers flexibility to use custom simulation tools in place of, or in addition to the default simulation tools (Gazebo, rqt, rviz and terminal access tools) provided by default in RoboMaker.
AWS Step Functions now features a data flow simulator in the Step Functions console, making it easier to evaluate the input and output processing for your state machines, allowing you to build workflows faster.
Amazon GuardDuty is now available in the AWS Asia Pacific (Osaka) Region. You can now continuously monitor and detect security threats in the region to help protect your AWS accounts, workloads, and data stored in Amazon S3.
Amazon Keyspaces (for Apache Cassandra), a scalable, highly available, and fully managed Cassandra-compatible database service, now offers Federal Information Processing Standards (FIPS) 140-2 compliant endpoints to help you run highly regulated workloads more easily.
AWS CloudFormation StackSets announces the ability to deploy CloudFormation Stacks to multiple AWS regions in parallel. This reduces the overall stack set provisioning times and provides a performance improvement when using StackSets. StackSets extend the functionality of stacks by enabling you to create, update, or delete stacks across multiple AWS accounts and regions with a single operation. Currently, using StackSets you can specify the region order preference to deploy stacks across AWS regions in a sequential manner. StackSets now expands on that by providing the additional ability to specify deploying across AWS regions in parallel as part of the deployment preferences for the stack set.
Amazon Fraud Detector is a fully managed service that makes it easy to identify potentially fraudulent online activities, such as the creation of fake accounts or online payment fraud, using customized machine learning (ML) models. To train an ML model, customers provide a dataset that contains examples of legitimate and fraudulent events related to the business activity they want to evaluate for fraud risk. These fraud datasets are often highly imbalanced. For example, a dataset containing one million past transactions may only include 5,000 fraudulent ones, corresponding to a fraud rate of 0.5%. This imbalance in the training data can lead to lower model performance, which results in the customer capturing less fraud. There are a number of common techniques used to treat imbalanced datasets, but applying them requires ML expertise and the best technique often depends on the characteristics of the particular dataset.
AWS Glue now offers missing value imputation on incomplete datasets. You can use the Fill Missing Values transform to get predicted values for blank entries in a column of your data. This feature makes it easy to clean datasets that have null or empty values so that they don’t need to be accounted for at query time.
You can now launch RabbitMQ 3.8.11 brokers on Amazon MQ. This patch update to RabbitMQ contains several fixes and new features compared to the previously supported version, RabbitMQ 3.8.6.
When AWS launched IAM Access Analyzer, they started by helping you remove unintended public and cross account access by analyzing your existing permissions. Recently, IAM Access Analyzer added policy validation to help you set secure and functional permissions during policy authoring. Now, we are taking that a step further and generating policies for you. You can now use IAM Access Analyzer to generate fine-grained policies based on your access activity found in your CloudTrail. When you request a policy, IAM Access Analyzer gets to work and generates a policy by analyzing your CloudTrail logs to identify your activity. The generated policy makes it easier to grant only the required permissions for your workloads.
Amazon ElastiCache now supports tag-based access control and adding tags to additional cluster resources. By using tags for access control, your AWS Identity and Access Management (IAM) users, groups, and roles get access only to the Amazon ElastiCache resources with matching tags. This provides you the capability to scale by reducing the number of distinct permissions you need to create and manage in your AWS account. You can define AWS IAM policies that grant or deny access to a resource based on its tags. Furthermore, you can use specific condition context keys to customize your AWS IAM policies to limit specific behaviors on Amazon ElastiCache resources. For a complete list of condition context keys for Amazon ElastiCache, visit the Amazon ElastiCache documentation .
Last week, AWS announced the Amazon Route 53 Resolver DNS Firewall, a managed firewall that enables customers to block DNS queries made for known malicious domains and to allow queries for trusted domains. DNS Firewall provides more granular control over the DNS querying behavior of resources within your Amazon Virtual Private Clouds (VPCs).
With thirteen more regions, EBS Multi-Attach for Provisioned IOPS io2 is now available in US East (N. Virginia), US East (Ohio), US West (Oregon), US West (N. California), EU (Ireland), EU (Frankfurt), Asia Pacific (Singapore), Asia Pacific (Tokyo), Asia Pacific (Mumbai), Asia Pacific (Sydney), Asia Pacific (Seoul), Canada (Central), EU (London), EU (Stockholm), Middle East (Bahrain), and Asia Pacific (Hong Kong).
With Amazon Interactive Video Service (Amazon IVS) you can now save your live video content to Amazon Simple Storage Service (Amazon S3). Saved video is available for actions like editing or replaying as a video on-demand (VOD).
AWS announces a new feature of AWS Systems Manager Parameter Store that supports the removal of a label associated with a parameter, to enable customers to reorganize Parameter Store parameters with new labels.
Amazon EC2 G4ad instances which provide the best price performance for graphics intensive applications in the cloud are now available in US East (Ohio), Europe (Frankfurt and London), Asia Pacific (Tokyo), and Canada (Central). G4ad instances are powered by AMD Radeon Pro V520 GPUs and second-generation AMD EPYC processors, and provide up to 45% better price performance over G4dn instances for graphics intensive applications such as virtual workstations, game streaming, and graphics rendering.
Amazon CodeGuru Reviewer is a developer tool that leverages automated reasoning and machine learning to detect potential defects that are difficult to find in your code and offers suggestions for improvements. Today, we are excited to announce, a new repository size-based pricing model with a price reduction of up to 90%, making it easier for customers to predictably scale their automated code reviews across their software development processes.
The data management experience in the AWS Amplify Admin UI is now generally available with new capabilities for seed data generation, data sorting, and data filtering. Launched at re:Invent 2020, the Amplify Admin UI is an externally hosted console for frontend teams to visually create an app backend and manage the app content and users.
Streaming ETL jobs in AWS Glue can now read from Amazon Kinesis Data Streams in a different AWS account than the one running the AWS Glue job. This feature allows you to run your ETL jobs from the consumer account rather than the data producer account, keeping all ETL activity in one location and simplifying data-integration management.
Run Command, a capability of AWS Systems Manager, now displays up to 24,000 characters of the output log of a Run Command invocation from the console. This is an increase from the 2,500 characters available in the console before this enhancement. Additionally, you can now view the output and error logs separately, and you can copy the logs or download them as text files directly from the console. This feature reduces the need to navigate to the Amazon Simple Storage Service (Amazon S3) or Amazon CloudWatch consoles to view and download logs for Run Command invocations, which helps you troubleshoot invocation issues more quickly.
Amazon SageMaker Pipelines, the first purpose-built, continuous integration and continuous deployment (CI/CD) service for machine learning (ML), is now supported as a target for routing events in Amazon EventBridge. This enables customers to trigger the execution of the Amazon SageMaker model building pipeline based on any event in their event bus or on a schedule by selecting the pipeline as the target in Amazon EventBridge. For example, customers can set up EventBridge to trigger the execution of the SageMaker model building pipeline when a new file with the training data set is uploaded to an Amazon S3 bucket or when the SageMaker Model Monitor indicates a deviation in model quality through alarms in Amazon CloudWatch metrics. Customers can also create rules in Amazon EventBridge that trigger the pipeline execution on an automated schedule.
Amazon MQ is now available in a total of 20 regions, with the addition of the Japan (Osaka) region.
Starting today, you can use the Amazon WorkSpaces macOS client application with Common Access Card (CAC) and Personal Identity Verification (PIV) smart cards. Smart card support is available on WorkSpaces using the WorkSpaces Streaming Protocol (WSP). WSP is a cloud-native streaming protocol that enables a consistent user experience when your end users are accessing their WorkSpaces across global distances and unreliable networks. To learn more, see Amazon WorkSpaces Streaming Protocol.
Amazon Macie is now available in AWS Asia Pacific (Osaka) Region. You can now discover sensitive data stored in this region to help protect your AWS workloads and data in Amazon S3.
Patches 1.9.1 / 2.7.1 / 3.4.1 / 4.0.1 now available for customers using Amazon Aurora PostgreSQL. For detailed release notes visit our version documentation . You can apply the new patch version in the AWS Management Console, via the AWS CLI, or via the RDS API. For detailed instructions, please see our technical documentation .
You can now configure budgets actions to enforce budget limits for your AWS Budgets resources in your AWS CloudFormation templates, stacks, and StackSets. With budget actions, you can define the action you want to take in your account when a budget exceeds the threshold you defined (actual or forecasted amounts). This level of control will allow you to reduce unintentional overspending in your account. You can choose among three action types: Identity and Access Management (IAM) policy, Service Control policy (SCPs), or target running instances (EC2 or RDS). For example, you can choose to apply a custom “Deny EC2 Run Instances” IAM policy to a user, group, or role in your account, once your monthly budget for EC2 has been exceeded. With the same budget threshold, you can configure a second action that shuts down specific EC2 instances within a particular AWS Region.
AWS WAF now lets you generate labels and customize your WAF rules based on those labels. With this feature, you can configure WAF to add descriptive labels to web requests when a WAF rule matches the request, regardless of the action associated with the rule. You can also check for the presence of those labels in subsequent WAF rules and combine with other WAF rules to take action on web requests that include the label. Creating a label also generates a corresponding CloudWatch metric and adds the label to your WAF logs for improved visibility.
You can now clean and transform data with an interactive, point-and-click visual interface from Amazon Redshift, Snowflake, Microsoft SQL Server, MySQL, Oracle Database, and PostgreSQL through native JDBC connectors in AWS Glue DataBrew. In just a few clicks, you can configure these JDBC connections from the AWS Management Console to directly explore and experiment with datasets from AWS data lakes, data warehouses, and databases without writing code.
Amazon Virtual Private Cloud (Amazon VPC) flow logs now makes it easier to query VPC flow logs using Amazon Athena. With a few clicks, you can now automate the integration between Athena and your VPC flow logs delivered to Amazon Simple Storage Service (S3). You can also choose from a set of predefined Athena queries to derive actionable insight such as the total egress traffic from your VPCs, identify the top talkers in your subnets, or troubleshoot anomalies in your VPC traffic.
Starting today, Bring Your Own IP (BYOIP) is available in four additional AWS Regions: GovCloud (US-East), GovCloud (US-West), Asia Pacific (Hong-Kong), and US West (Northern California).
Amazon Elastic Kubernetes Service (EKS) clusters running in the AWS GovCloud (US) Regions are now compliant with the Federal Risk and Authorization Management Program (FedRAMP) High baseline.
AWS Firewall Manager now enables security administrators to deploy the recently launched AWS WAF Bot Control across accounts in their organization, from a central administrator account. AWS WAF Bot Control is a new managed rule group that gives you visibility and control over common and pervasive bot traffic to your applications. You can use Bot Control to protect your web applications from automated bots that consume excess resources, skew metrics, cause downtime, or perform other malicious activities.
AWS Batch customers can now specify EFS file systems in their AWS Batch job definitions. AWS Batch jobs using EFS will automatically mount the file systems specified by the customer in the job definition and make them available to the jobs, across Availability Zones. This enables persistent, shared storage to be defined and used at the job level.
Amazon Redshift now supports managed VPC endpoints (powered by AWS PrivateLink) to connect to your Amazon Redshift cluster in a Virtual Private Cloud (VPC). With an Amazon Redshift-managed endpoint, you can now privately access your Amazon Redshift data warehouse within your (VPC) from your client applications in another VPC within the same or another AWS account and running on-premises without using public IPs or requiring traffic to traverse the Internet.
AWS Firewall Manager now supports Amazon Route 53 Resolver DNS Firewall, making it easy for security administrators to identify the set of DNS Firewall rules they wish to use and deploy across their organization, from a central place. AWS recently launched Amazon Route 53 Resolver DNS Firewall, a managed firewall feature that enables customers to block DNS queries made for known malicious domains and to allow queries for trusted domains. DNS Firewall provides more granular control over the DNS querying behavior of resources within your Amazon Virtual Private Clouds (VPCs). Now that Firewall Manager supports DNS Firewall, you can identify the set of DNS Firewall rules you wish to use and deploy them across multiple accounts, organizational units (OUs), and VPCs, all from a single central security administrator account.
AWS announces a new feature of AWS Systems Manager Parameter Store that helps you discover public parameters more easily.
You can now use the AWS Glue Schema Registry , a serverless and free feature of AWS Glue, in the Europe (Milan), Middle East (Bahrain), Africa (Cape Town), and Asia Pacific (Hong Kong) regions to validate and control the evolution of streaming data using registered Apache Avro schemas.
AWS Backint Agent version 1.03 is now available in all commercial regions, including AWS GovCloud (US) Regions and the recently announced AWS Asia Pacific (Osaka) Region.
AWS WAF announces the launch of AWS WAF Bot Control, which gives you visibility and control over common and pervasive bots that consume excess resources, skew metrics, cause downtime, or perform other undesired activities. With Bot Control, you can easily monitor, block, or rate-limit pervasive bots, such as scrapers, scanners, and crawlers, or you can allow common bots, such as status monitors and search engines. You can add the Bot Control managed rule group alongside other Managed Rules for WAF or your own custom WAF rules to protect your applications.
AWS Snowball Edge Compute Optimized is now available in the AWS Asia Pacific (Singapore) Region. Snowball Edge Compute Optimized is a secure, rugged device that brings AWS computing and storage capabilities, such as Amazon EC2, Amazon EBS, Amazon S3, AWS IoT Greengrass, AWS Lambda functions, and AWS IAM to your edge environments for machine learning, data analytics, processing, and local storage. You can use Snowball Edge devices in environments with intermittent connectivity (like manufacturing, industrial, and transportation) or in extremely remote locations (like military or maritime operations). These devices may also be rack mounted and clustered together to build larger installations.
The new AWS Storage Gateway management console makes it easier for you to create, manage, and monitor resources such as file shares, tapes, and volumes. In addition to a refreshed look and feel, you can now connect your gateway with a simple activation key to help speed up deployment, and create new file shares using a streamlined process. You can easily configure your gateway to use Amazon Virtual Private Cloud (VPC) endpoints and leverage enhanced troubleshooting information for the endpoint configuration. Throughout the console, contextual information about your Storage Gateway resources is provided as part of the help panel, allowing you to easily find answers to common questions and browse related User guide content.
Following the announcement of updates in MySQL database versions 5.6 , 5.7 , and 8.0 , AWS have updated Amazon Relational Database Service (Amazon RDS) for MySQL to support MySQL minor versions 5.6.51, 5.7.33, and 8.0.23. We recommend that customers upgrade to any of the latest minor versions to fix known security vulnerabilities in prior versions of MySQL, and to benefit from the numerous bug fixes, performance improvements, and new functionality added by the MySQL community. Learn more about upgrading your database instances in the Amazon RDS User Guide , including automatic minor version upgrades.
We are excited to announce that Amazon Transcribe Custom Language Models (CLM) now support Australian English, British English, Hindi and US Spanish. Amazon Transcribe is an automatic speech recognition (ASR) service that makes it easy for you to add speech-to-text capabilities to your applications. CLM allows you to use pre-existing data to build a custom speech engine for your specific batch transcription use cases. No prior machine learning experience is required to create your CLM.
AWS announced the launch of Amazon Route 53 Resolver DNS Firewall, a managed firewall that enables customers to block DNS queries made for known malicious domains and to allow queries for trusted domains. DNS Firewall provides more granular control over the DNS querying behavior of resources within your Amazon Virtual Private Clouds (VPCs).
Announcing Amazon CloudWatch Metric Streams - a new feature that enables customers to create a continuous, near real-time stream of metrics to a destination of their choice.
Starting today, the interactive EC2 Serial Console is now generally available. EC2 Serial Console provides a simple and secure way to troubleshoot boot and network connectivity issues by establishing a connection to the serial port of an instance. It provides a one-click, text-based access to an instances’ serial port as though a monitor and keyboard were attached to it. This access can be used for interactive troubleshooting.
AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development. Using AWS Glue Workflows , you can orchestrate and execute a complex multi-job, multi-crawler data-integration workflow. AWS Glue custom blueprints make it easy for data engineers to create repeatable AWS Glue workflows.
When creating datasets in AWS Glue DataBrew from the Amazon S3 data lake, you can now create dynamic datasets to schedule data preparation on new incoming Amazon S3 files or apply transformations on filtered or conditionally chosen files or folders in S3. You can create a dynamic S3 path to choose files based on a time-window or time of last file update, and defining custom parameters to replace string, number, or date-based values in your S3 file path with filter conditions such as begins with, ends with, contains, does not contain, less than, greater than, before, and others. Custom parameter names can be included as columns in your datasets and the revised schema will be used for jobs running on dynamic datasets. With parameterized S3 paths and/or files, users can schedule to apply existing recipes to run on selected dynamic datasets.
AWS Config now supports the ability to capture and view the compliance history of AWS Config conformance packs. You can see how the overall compliance status of a conformance pack changed over time, and which rules within a conformance pack impacted the status change. You can aggregate conformance pack compliance data from multiple accounts and AWS Regions using AWS Config aggregators to get a centralized view of your compliance regimes and operational best practices. You can maintain up to 7 years of history. You can also run AWS Config advanced queries on this data for more details about your conformance pack compliance.
AWS announced the launch of Building Data Lakes on AWS. Data lakes enable organizations to generate business value by identifying and acting upon opportunities for business growth. This new one-day classroom course will help you practice building a data lake in a hands-on environment. An expert instructor will teach you how to build and secure a data lake using AWS Lake Formation while also showing you how to optimize for cost and performance.
AWS Data Exchange now enables providers to copy descriptions, data sets, and public offers from their existing products to a new product. With this launch, providers can select any existing product, published or unpublished, and copy all of its details to a new draft product, which they can then modify as needed and publish within a few seconds. For providers that frequently create new products that share similar metadata, this will save a significant amount of time and effort.
You can now configure each path segment of an API Gateway custom domain name to route requests to different APIs. Using multi-level base path mappings, you can implement path-based API versioning and migrate API traffic between APIs according to request paths with many segments.
AWS Site-to-Site VPN service today increased the default service quotas for the dynamic routes advertised to and from a Site-to-Site VPN connection on a Transit Gateway. You can now advertise up to 1,000 dynamic routes (an increase from 100) from a customer gateway device to a Site-to-Site VPN connection on a Transit Gateway. Similarly, you can now advertise up to 5,000 routes (an increase from 1,000) from a Site-to-Site VPN connection on a Transit Gateway to a customer gateway device. Advertised routes come from the route table that's associated with the VPN attachment.
AWS Transit Gateway Connect today increased the default service quotas for the dynamic routes advertised to and from a Transit Gateway Connect peer. You can now advertise up to 1,000 dynamic routes (an increase from 100) from a customer gateway appliance in an on-premises network or a virtual router appliance in a VPC to a Transit Gateway Connect peer. Similarly, you can now advertise up to 5,000 routes (an increase from 1,000) from a Transit Gateway Connect peer to a customer gateway appliance or a virtual router appliance. Advertised routes come from the route table that's associated with the AWS Transit Gateway Connect attachment.
Amazon Fraud Detector now supports batch fraud predictions. Amazon Fraud Detector is a fully managed service that makes it easy to identify potentially fraudulent online activities, such as the creation of fake accounts or online payment fraud. Until today, Fraud Detector supported real-time fraud predictions via the GetEventPrediction API, which is ideal for low-latency synchronous fraud prediction use cases but requires customers to integrate an API and make an API call for every event they want to evaluate. Now, customers who have non real-time fraud prediction use cases can get fraud predictions for a large number of events in one go, or on an hourly or daily basis, using the new batch prediction feature without needing to write any code.
AWS Config advanced queries feature now supports pagination for queries that contain aggregate functions, such as COUNT and SUM. For example, getting the total number of resources in each AWS account requires the COUNT aggregate function. You can now use advanced queries to get complete results for your aggregate queries through pagination, which were previously limited to 500 rows. Pagination is a technique that is used to divide large results into “pages,” where each page contains a subset of results. You can process the first page of results, then the second page, and so on.
AWS Step Functions is now integrated with Amazon EMR on Amazon Elastic Kubernetes Service (Amazon EKS), making it easier to integrate Apache Spark based jobs into your analytics pipeline. You can now build workflows including steps to manage EMR on EKS virtual clusters and submit jobs without writing code to manage the state of the job.
AWS WAF now supports inserting HTTP headers to the user request when WAF allows the request to reach your application. You can use the Request Header Insertion feature to help validate that requests made to your application were evaluated by WAF and configure your application to only allow requests that contain the custom header values that you specify. You can also insert headers so your application can process the request differently based on the presence of the header, or simply log the header in your application logs for reporting and analytics.
AWS WAF now supports configuring the HTTP status code and the response body returned to the user when a request is blocked. Until today, AWS WAF could only return HTTP status code 403 (forbidden) when the user request was blocked by WAF. With Custom Response, you can now configure AWS WAF to send out a different HTTP status code, such as 3xx (redirects), 4xx (client errors), or 5xx (server errors). These codes can be used to redirect users to different parts of your application or provide users a specific response code based on the reason they were blocked by WAF. In addition, you can use Custom Response to include a response body to present a customized error message back to the user.
Source: cloud.google.com
Access Approval
Anthos clusters on VMware
Anthos Config Management
Anthos Config Management images are no longer included in Anthos on VMWare clusters. To learn more, see Changes to Anthos Config Management updates.
The ability to sync from multiple Git repositories is now a generally-available feature. To learn more, see Syncing from multiple repositories.
A memory leak in the Anthos Config Management Operator Pod that led to high memory utilization or Pod restarts due to out-of-memory errors has been corrected.
Preview versions of multi-repo occasionally used excessive CPU usage and sent unnecessary queries to the apiserver master node, resulting in an unhealthy cluster. This issue has been corrected.
Config Sync configured with sourceFormat: unstructured will have errors during syncing if the Git repository includes a "Repo" resource.
Config Sync configured with sourceFormat: unstructured will have errors during syncing if the Git repository specifies a ClusterSelector with an invalid metadata.name field.
Customers using Anthos Policy Controller who have upgraded since Anthos Config Management 1.5.1 need to update the timeoutSeconds in their ValidatingWebhookConfigurations from "5" to "3" to avoid issues with Kubernetes leader elections.
Anthos GKE on AWS
Anthos clusters on AWS 1.7.0-gke.12 is now available.
Anthos clusters on AWS 1.7.0-gke.12 clusters run the following Kubernetes versions:
To upgrade your clusters, perform the following steps:
Anthos Service Mesh
1.9.2-asm.1 is now available.
This patch release contains the same bug fixes that are in Istio 1.9.2. For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:
install_asm scriptAnthos Service Mesh user authentication is now available as a public preview feature on installations of 1.9. This feature lets you use existing Identity Providers (IDP) for user authentication and access control to your workloads. For more information, see Configuring Anthos Service Mesh user authentication.
The Anthos Service Mesh Topology (beta) page in Cloud Console won't display properly if unsupported versions, including versions earlier than Anthos Service Mesh 1.6.8, are installed on your clusters or if you have disabled the Canonical Service controller in clusters in your project.
Note that the Canonical Service controller is enabled by default on version 1.6.8 and higher. If you did not disable the Canonical Service controller on a supported version, no action is required.
Big Query
Beginning in early Q3 2021, BigQuery Storage Read API will start charging for network egress. In addition, BigQuery Storage Read API will become available in all locations, with appropriate pricing. Another release note will be issued when these changes take effect.
The BigQuery Storage Write API is now in Preview. The Storage Write API is a stream-based API for ingesting data into BigQuery at low cost and high throughput. It provides exactly-once delivery semantics with real-time latency. For more information, see Using the BigQuery Storage Write API.
BigQuery standard SQL now supports the ALTER TABLE DROP COLUMN. This feature is in Preview.
The maximum length has been increased from 128 characters to 300 characters for the following BigQuery fields: table column names, column alias names, and user-defined function names.
Cloud Bigtable
Cloud Bigtable support for customer-managed encryption keys (CMEK) is now generally available.
Data Access audit logging for Cloud Bigtable is now generally available.
If you previously enabled Data Access audit logs for all Google Cloud services in the Cloud Audit Logs default configuration, you might need to take additional steps to enable Data Access audit logging for Cloud Bigtable. Affected customers will see a notification at the top of the Cloud Bigtable page of the Cloud Console.
Cloud Build
Cloud CDN
Serve stale, bypassing cache, and negative caching are now Generally Available.
These features are available when configuring Cloud CDN enabled backend services and backend buckets in the Cloud Console, in addition to the gcloud SDK and REST API.
Cloud CDN now supports configuring negative caching for HTTP 302 (Found) and HTTP 307 (Temporary Redirect) status codes.
To learn how to enable negative caching for these status codes, visit the documentation.
Cloud CDN now treats HTTP responses with a valid, future date in the Expires header as cacheable, even if those responses do not have a Cache-Control: public directive.
This will allow Cloud CDN to cache additional responses and better align with HTTP standards.
Review the caching documentation for details on what content Cloud CDN considers cacheable vs. uncacheable.
Cloud CDN now treats the no-cache Cache-Control directive in a response as per RFC 7234 and allows these responses to be cached, provided that they are validated every time before being reused.
Visit the caching documentation to review how Cloud CDN handles the full set of HTTP caching directives.
Cloud Composer
New versions of Cloud Composer images:
Irrelevant warnings about asynchronous DAG loading parameters no longer show up in the Airflow logs.
Corrected the validation of custom Cloud SQL and Airflow web server IP ranges that are specified during the environment creation. Changed the error code and the message that are returned when a specified CIDR range is not valid.
Fixed an Airflow web UI bug that caused the DAG Tree View page to crash in rare cases.
Cloud Database Migration Service
Database Migration Service makes it easier for you to "lift and shift" your MySQL and PostgreSQL workloads into Cloud SQL. This service streamlines your networking workflows, manages one-time and continuous migrations between your source and destination databases, and provides you with statuses of the migration operations.
The documentation now contains information for using Database Migration Service with PostgreSQL. This information includes:
In addition, for this release, updates include: * Use the Cloud SDK: A guide to get started with the Cloud SDK so you can use it to manage Database Migration Service connection profiles and migration jobs. * Use the Database Migration Service API: This guide provides information about how to enable and use the REST API to administer connection profiles and migration jobs programmatically. * Providing gcloud information for managing connection profiles and migration jobs for MySQL and PostgreSQL.
Click here to access the documentation.
Cloud DNS
Cloud Interconnect
Cloud Load Balancing
External TCP/UDP Network Load Balancing is now supported with backend services. Compared to the target pool backend, a backend service gives you more fine-grained control over your load balancer, including access to features such as connection draining, failover policies, and support for managed instance groups as backends.
Network load balancers with a backend service can also use health checks that match the traffic (TCP, SSL, HTTP, HTTPS, or HTTP/2) they are distributing.
To get started, see:
This feature is available in General Availability.
Cloud Logging
Cloud Logging now supports 22 regions in which you can create a log bucket so that you can meet compliance and audit requirements when storing your logs.
Logs Views are now Generally Available (GA). Using Logs Views, you can control who has access to the logs within your Logs Buckets. For more information on this feature, refer to the Managing Logs Views guide.
Cloud Run
Restricting ingress on Cloud Run is now at general availability (GA).
Cloud Scheduler
Cloud Spanner
You can now use Customer-Managed Encryption Keys (CMEK) to protect databases in Cloud Spanner. CMEK in Cloud Spanner is now generally available. For more information, see CMEK.
You can now optionally specify the priority of data requests. For more information, see CPU utilization and task priority.
Cloud SQL for PostgreSQL
Cloud SQL for PostgreSQL now lets you use IAM database authentication with the Cloud SQL Auth proxy. The Cloud SQL Auth proxy is able to request and refresh OAuth 2.0 access tokens, ensuring that long-lived processes or applications that rely on connection pooling can have stable connections. To learn more, see Using IAM database authentication with the Cloud SQL Auth proxy.
Cloud SQL for PostgreSQL flags are now generally available. See supported PostgreSQL flags for more information.
Cloud SQL for SQL Server
Cloud SQL for SQL Server enables you to perform change data capture (CDC) operations for your Cloud SQL instances. General information about CDC in SQL Server is here.
CDC is available for the following Cloud SQL for SQL Server database versions:
You can integrate Cloud SQL for SQL Server with Managed Service for Microsoft Active Directory.
Authentication, authorization, and more are available. For example, joining an instance to a managed Active Directory domain enables you to log in using Windows Authentication. Additionally, you can integrate with your on-premises AD domains by establishing a trust.
Compute Engine
Preview: You can now configure your VM to shutdown automatically when you revoke the Cloud KMS key protecting a persistent disk attached to the VM. For more information, see Configuring VM shutdown on Cloud KMS key revocation.
N2D machines are now available in the following regions and zones:
us-central1-b - Iowaasia-northeast1-a,b - TokyoGenerally available: You can now use instance schedules from the Google Cloud Console.
Memory-optimized machines are now available in the following regions and zones:
M1 ultramem (Jakarta ) asia-southeast2-a,cM1 ultramem (Osaka) asia-northeast2-aM1 ultramem, M2 ultramem and M2 megamem (Osaka) asia-northeast2-bM2 ultramem and M2 megamem (Osaka) asia-northeast2-cConfig Connector
Config Connector version 1.45.0 is now available.
Added support for OSConfigGuestPolicy, IdentityPlatformTenant, IdentityPlatformOAuthIDPConfig and IdentityPlatformTenantOauthIDPConfig.
Added proxyBind field to ComputeTargetHTTPProxy, ComputeTargeHTTPSProxy, and ComputeTargetTCPProxy.
Added enableStreamingEngine field to DataflowJob.
Fixed issue where folderRef/organizationRef could not be defaulted from folder-id/organization-id annotations when creating Project/Folder resources with server-side apply. (More details can be found here).
Supported a viewer cluster role so that resources can be referenced cross namespaces in namespaced mode. (Issue #407)
Updated the structs' name of any field FooBar to be KindFooBar in Go Client resources. This ensures that the struct names are unique within a Go package.
DataFlow
Dataproc
Dataproc support of Dataproc Metastore services is now available in GA.
Dialogflow
(2021-R11) Version updates
GKE cluster versions have been updated.
New versions available for upgrades and new clusters
gcloud command-line tool and REST API. This feature is available in Preview.Policy Simulator is now generally available. You can use Policy Simulator to simulate policy changes before you apply them.
Secret Manager Event Notifications is generally available.
Secret Manager Event Notifications lets you configure secrets to send messages to Pub/Sub topics whenever a change is made to the secret or one of its versions.
Learn more at enabling event notifications.
Secret Manager Expiration is generally available.
Learn more at creating and managing expiring secrets.
Secret Manager Rotation is generally available.
Secret Manager Rotation sends messages to Pub/Sub topics based on the provided rotation frequency and rotation time.
Learn more at creating and managing rotation policies.
Security Command Center Legacy, previously known as Cloud Security Command Center, and Event Threat Detection Legacy are being permanently disabled for all customers on June 7, 2021.
If you onboarded to Security Command Center before May 2020, or Event Threat Detection before June 2020, and never upgraded to Security Command Center's Standard tier or Premium tier, you are using a legacy product.
To continue benefiting from Security Command Center and Event Threat Detection without an interruption in service, customers using legacy products must migrate their organizations to Security Command Center Standard or Premium. Event Threat Detection, a built-in service of Security Command Center, is available only in the Premium tier.
For details on upgrading legacy products, see Migrate from legacy Security Command Center products.
Virtual Private Cloud
Automate tasks across Azure & Non-Azure environment using PowerShell and Python based scripts.
Azure Purview is now available in public preview in the Central India region. You can now provision Azure Purview accounts in these regions as a public preview offering.
You can now keep your VMs deallocated to save cost and have it automatically start up when a user connects.
Azure Monitor for Containers is updating to use the Arc Extension Model for Kubernetes clusters hosted on Azure Arc. This support is currently in public preview.
Use Azure Cloud Services (extended support), now generally available, to increase regional resiliency and gain access to new capabilities that the Azure Resource Manager-based deployment model provides.
Seamless, fully orchestrated, no downtime for most scenarios and minimal effort migration path for your existing Cloud Services (classic) deployments.
Azure Data Factory is now available in two new regions: Norway East and UAE North
The new Authentication portal experience for App Service and Azure Functions, improving usability, initial setup, and increased security, is now available.”
Public preview enhancements and updates released for Azure Security Center in March 2021.
New enhancements and updates released for general availability in Azure Security Center in March 2021.
Azure Monitor for Windows Virtual Desktop provides a centralized view of the health of your Windows Virtual Desktop environment and enables you to optimize your deployment and quickly troubleshoot issues.
Reduce development costs and time using the new public preview features in Microsoft Power Fx.
Azure virtual machine scale sets provide the management capabilities for applications that run across many VMs, automatic scaling of resources, and load balancing of traffic. Scale set orchestration modes allow you to have greater control over how virtual machine instances are managed by the scale set.
To protect your valuable resources, Azure Backup now supports backup & restore of Azure Virtual Machines with dedicated host.
This quality release of the Azure Sphere OS includes bug fixes and updates to mitigate against the Common Vulnerabilities and Exposures (CVEs) mentioned in the article.
Upcoming changes to Azure SQL Managed Instance will improve workload performance and reduce unnecessary resource utilization.
Use Azure Private Link to connect to an Azure Cache for Redis instance from your virtual network via a private endpoint to eliminate data exposure to the public internet.
Customers can now leverage guidance jointly developed with IBM to run WebSphere Liberty and Open Liberty on Azure Kubernetes Service (AKS).
Network insights in Azure Monitor now supports monitoring Azure ExpressRoute resources out of the box without any additional configuration or setup.
Azure Static Web Apps, currently in public preview, now supports building and deploying apps with Azure DevOps.
Public preview: Open Service Mesh (OSM) add-on for AK
Service Mesh capabilities to be natively integrated with AKS via the Open Service Mesh add-on.
Benefit from the new features in Kubernetes 1.20, now supported by AKS in general availability.
With new regions announced for Azure Database for PostgreSQL – Flexible Server, you can control multiple configuration parameters for fine-grained database tuning with a simpler developer experience to accelerate end-to-end deployment.
Make your serverless Azure SQL databases resilient to a much larger set of failures, including catastrophic datacenter outages, without any changes of the application logic by selecting zone redundancy.
Now available in preview support, Azure Monitor insights and Azure Monitor workbooks makes it easier to visualize and customize metrics for dynamic monitoring of your Azure Cosmos DB resource.
You can now opt-in to have your node image auto-upgraded when a new AKS node image becomes available
You can now invoke commands in an AKS cluster for just-in-time access when you are not on the cluster private network.
Support for 73 languages, prebuilt ID model, invoice line item extraction, tagging as table, and lots more - Now in Azure Form Recognizer public preview, part of Azure Cognitive Services
Virtual machine (VM) level disk bursting enables your workloads to handle unforeseen disk traffic spikes smoothly without the need to overprovision your virtual machine. The feature is now enabled on all Dsv3-series and Esv3-series virtual machines.
Now in public preview, use the 'connect' command for Azure CLI for Azure Database for MySQL - Flexible Server to test connections and run basic queries.
Now in public preview, use the 'connect' command for Azure CLI for Azure Database for PostgreSQL - Flexible Server to test connections and run basic queries.
Key Vault references in App Service and Azure Functions now support accessing vaults with network access controls from Windows apps with virtual network integrations.
Azure Event Grid now supports additional advanced filters, setting customer headers on requests delivering events, configuring TTL on messages delivered to Azure Storage Queues, and setting system-assigned managed identities on regional System Topics.
With this update, we are including temporary changes that will impact the process to acquire Azure Pipelines free grants. New users will have to send an email and provide additional information to get free CI/CD.
TARGET RETIREMENT DATE: MARCH 31, 2022
Because Azure Monitor Virtual Machine insights and Service Map both provide network connection data sets in Logs, utilize the Dependency agent, and have connections with Azure Sentinel, we will be retiring Wire Data 2.0 (preview) on 31 March 2022. Please transition to using Azure Monitor Virtual Machine insights or Service Map by that date.
Create an external table located in Azure Blob Storage, Azure Data Lake Store Gen1, or Azure Data Lake Store Gen2 to analyze and query your data.
Encryption scopes enable you to provision multiple encryption keys to manage encryption at the container or blob level. Customers and ISVs can now use a single storage account for multi-tenancy scenarios by provisioning separate encryption keys for each customer. The key that protects an encryption scope may be either a Microsoft-managed key or a customer-managed key in Azure Key Vault.
Now you can use Azure Communication Services to add video, voice, SMS, chat, and telephony capabilities into almost any web, mobile, or desktop application
Azure Disk Backup offers snapshot lifecycle management to Azure Managed Disk by automating periodic creation of snapshot and retain it for configured duration using Backup policy
UPComing Training & Events:
AWS Training and Certification was excited to announce the launch our updated one-day classroom course, AWS Cloud Practitioner Essentials. Learn from an accredited AWS expert instructor with deep technical knowledge to explore the AWS Cloud via a mix of presentations, peer discussion, and interactive activities. Whether you’re in a technical or non-technical role, this course helps develop the knowledge and skills necessary to demonstrate an overall understanding of the AWS Cloud.
AWS announced the launch of the updated Advanced Architecting on AWS course this week. This instructor-led training course is designed for cloud architects who want to extend their baseline knowledge of AWS services. An expert AWS instructor will help you learn advanced architecting topics such as hybrid connectivity and hybrid AWS devices, networking with a focus on AWS Transit Gateway connectivity, AWS Container services, automation tools for CI/CD, security options, and much more.
AWS Training and Certification launched a new course entitled AWS Cloud Technical Essentials. Available for free on Coursera and edX, this course uses video lectures and demonstrations to teach the technical fundamentals of AWS. Upon course completion, learners will be able to make an informed decision about when and how to apply core AWS services for compute, storage, and database to different use cases.
AWS are excited to announce a free digital course: Amazon Simple Storage Service Cost Optimization. This advanced 60-minute course explores techniques and tools you can use to optimize your Amazon S3 costs. Designed for cloud architects, storage architects, developers, and operations engineers, it includes interactive lessons and video demonstrations.
AWS introduced four new digital training offerings that help you learn how to plan, deploy, secure, and manage cloud-based desktops and applications. The offerings are designed for desktop or virtual desktop infrastructure managers, IT administrators, and technical professionals interested in cloud-based virtualization. These free self-paced courses and curriculums include presentations, interactive e-learning modules, videos, demonstrations, and quizzes.
Azure Virtual Events
Microsoft have a full schedule of Virtual Events
A full list including session times and details are here : Azure Events
AWS Events:
AWS events are pretty fluid at the moment, with most in-person events being cancelled or postponed. There are a number that have been taken online and full details can be found here: AWS Events
Thanks for reading again this week, we hope you found something useful.
hava.io allows users to visualise their AWS, GCP and Azure cloud environments in interactive diagram form including unique infrastructure, security and container views. hava.io continuously polls your cloud configuration and logs changes in a version history for later inspection which helps with issue resolution and provides history of all configs for audit and compliance purposes.
If you haven't opened a free hava.io account to see what the GCP, Azure and AWS automated diagram generator can do for your workflow, security and compliance needs, you can below - if you have questions, please get in touch.
You can reach us on chat, email sales@hava.io to book a callback or demo.
Read more about Hava Azure Infrastructure Diagrams here.
Read more about Hava AWS Architecture Diagrams here.
Read more about Hava GCP Architecture Diagrams here