Cloud Composer: Qwik Start - Command Line
Workflows are a common theme in data analytics - they involve ingesting, transforming, and analyzing data to figure out the meaningful information within. In Google Cloud Platform (GCP), the tool for hosting workflows is Cloud Composer which is a hosted version of the popular open source workflow tool Apache Airflow.
In this lab, you use the Cloud Shell command line to set up a Cloud Composer environment. You then use Cloud Composer to go through a simple workflow that verifies the existence of a data file, creates a Cloud Dataproc cluster, runs an Apache Hadoop wordcount job on the Cloud Dataproc cluster, and deletes the Cloud Dataproc cluster afterwards.
This lab also shows you how to access your Cloud Composer environment through the GCP Console and the Airflow web interface.
What you'll do
Use Cloud Shell command line to create the Cloud Composer environment and set up the Composer environment variables
Verify the Environment configuration in the GCP Console
Run an Apache Airflow workflow in Cloud Composer that runs an Apache Hadoop wordcount job on the cluster
View and run the DAG (Directed Acyclic Graph) in the Airflow web interface
View the results of the wordcount job in storage
The following experience can help maximize your learning:
Basic CLI knowledge
Basic understanding of Python
이 실습의 나머지 부분과 기타 사항에 대해 알아보려면 Qwiklabs에 가입하세요.
- Google Cloud Console에 대한 임시 액세스 권한을 얻습니다.
- 초급부터 고급 수준까지 200여 개의 실습이 준비되어 있습니다.
- 자신의 학습 속도에 맞춰 학습할 수 있도록 적은 분량으로 나누어져 있습니다.