menu
arrow_back

Build a BigQuery Processing Pipeline with Events for Cloud Run for Anthos

—/100

Checkpoints

arrow_forward

Create a GKE cluster and set up cloud run events

Create a storage bucket

Query Runner

Chart creator

Test the pipeline

Build a BigQuery Processing Pipeline with Events for Cloud Run for Anthos

1 hora 15 minutos 7 créditos

GSP760

Google Cloud Self-Paced Labs

Overview

As serverless architecture is adopted, "events" become an integral part of how individual micro-services are connected yet de-coupled. Events for Cloud Run for Anthos makes "events" a first-class citizen of the Cloud Run for Anthos offering, so that it is easy to build event-driven serverless applications.

Events enable Cloud Run services to produce and/or react to events from a variety of on-cluster as well as off-cluster event sources. Events are delivered over HTTP in the portable CloudEvents format, and application developers do not have to worry about implementing a messaging solutions. Rely on the system for event ingestion, delivery, authentication, authorization, observability and error-handling, while you focus on developing an application code to solve a business problem.

In this lab you will build a BigQuery processing pipeline to query a public COVID-19 dataset on a schedule, create charts from that data, then notify users about the new charts via SendGrid with Events for Cloud Run for Anthos. To use SendGrid account, set up a trial account with your own email account if you don't already have one. Follow this doc for more details on how to set up SendGrid.

e837d024a9b296c3.png

How does this work:

  1. Two CloudSchedulerSources are set up to call the QueryRunner service once a day for two countries.

  2. QueryRunner receives the scheduler event for both countries, queries Covid-19 cases for the country using BigQuery's public Covid-19 dataset, and saves the result in a separate BigQuery table. Once done, QueryRunner returns a custom CloudEvent of type dev.knative.samples.querycompleted.

  3. ChartCreator receives the querycompleted event, creates a chart from BigQuery data using mathplotlib and saves it to a Cloud Storage bucket.

  4. Notifier receives the com.google.cloud.storage.object.finalize event from the bucket via a CloudStorageSource and sends an email notification to users using SendGrid.

Participe do Qwiklabs para ler o restante deste laboratório e muito mais!

  • Receber acesso temporário a Console do Google Cloud.
  • Mais de 200 laboratórios, do nível iniciante ao avançado.
  • Tamanho compacto para que você possa aprender no seu próprio ritmo.
Participe para iniciar este laboratório