Create a Firestore Database
Deploy the Autoscaler
Build YCSB container image
Launch the load generator
Leverage the Autoscaler Tool for Cloud Spanner to Achieve Workload Elasticity
The Autoscaler tool for Cloud Spanner is a companion tool to Cloud Spanner that allows you to automatically increase or reduce the number of nodes in one or more Spanner instances, based on their utilization.
Cloud Spanner is a fully managed relational database with unlimited scale, strong consistency, and up to 99.999% availability.
When you create a Cloud Spanner instance, you choose the number of nodes that provide compute resources for the instance. As the instance's workload changes, Cloud Spanner does not automatically adjust the number of nodes in the instance.
The Autoscaler monitors your instances and automatically adds or removes nodes to ensure that they stay within the recommended maximums for CPU utilization and the recommended limit for storage per node. Note that the recommended thresholds are different depending if a Cloud Spanner instance is regional or multi-region.
In this lab you'll deploy the Autoscaler tool for Cloud Spanner in the per-project configuration where the afutoscaler tools are located in the same project as the Cloud Spanner instance being autoscaled.
The diagram above shows the components of the Cloud Spanner Autoscaler and the interaction flow:
- Using Cloud Scheduler you define how often one or more Spanner instances should be verified. You can define separate Cloud Scheduler jobs to check several Spanner instances with different schedules, or you can group many instances under a single schedule.
- At the specified time and frequency, Cloud Scheduler pushes a message into the Polling Cloud Pub/Sub topic. The message contains a JSON payload with the Autoscaler configuration parameters that you defined for each Spanner instance.
- When Cloud Scheduler pushes a message into the Poller topic, an instance of the Poller Cloud Function is created to handle the message.
- The Poller function reads the message payload and queries the Cloud Monitoring API to retrieve the utilization metrics for each Spanner instance.
- For each instance, the Poller function pushes one message into the Scaling Pub/Sub topic. The message payload contains the utilization metrics for the specific Spanner instance, and some of its corresponding configuration parameters.
- For each message pushed into the Scaler topic, an instance of the Scaler Cloud Function is created to handle it. Using the chosen scaling method, the Scaler function compares the Spanner instance metrics against the recommended thresholds and determines if the instance should be scaled, and the number of nodes that it should be scaled to.
- The Scaler function retrieves the time when the instance was last scaled from the state data stored in Cloud Firestore and compares it with the current database time.
- If the configured cooldown period has passed, then the Scaler function requests the Spanner Instance to scale out or in.
Throughout the flow, the Spanner Autoscaler writes a step by step summary of its recommendations and actions to Cloud Logging for tracking and auditing.
Setup and Requirements
Before you click the Start Lab button
Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.
This Qwiklabs hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.
What you need
To complete this lab, you need:
- Access to a standard internet browser (Chrome browser recommended).
- Time to complete the lab.
Note: If you already have your own personal Google Cloud account or project, do not use it for this lab.
Note: If you are using a Pixelbook, open an Incognito window to run this lab.
How to start your lab and sign in to the Google Cloud Console
Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is a panel populated with the temporary credentials that you must use for this lab.
Copy the username, and then click Open Google Console. The lab spins up resources, and then opens another tab that shows the Sign in page.
Tip: Open the tabs in separate windows, side-by-side.
In the Sign in page, paste the username that you copied from the Connection Details panel. Then copy and paste the password.
Important: You must use the credentials from the Connection Details panel. Do not use your Qwiklabs credentials. If you have your own Google Cloud account, do not use it for this lab (avoids incurring charges).
Click through the subsequent pages:
- Accept the terms and conditions.
- Do not add recovery options or two-factor authentication (because this is a temporary account).
- Do not sign up for free trials.
After a few moments, the Cloud Console opens in this tab.
Activate Cloud Shell
Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.
In the Cloud Console, in the top right toolbar, click the Activate Cloud Shell button.
It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. For example:
gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.
You can list the active account name with this command:
gcloud auth list
Credentialed accounts: - <myaccount>@<mydomain>.com (active)
Credentialed accounts: - email@example.com
You can list the project ID with this command:
gcloud config list project
[core] project = <project_ID>
[core] project = qwiklabs-gcp-44776a13dea667a6
Using Cloud Shell, set the default zone and project configuration:
gcloud config set compute/zone europe-west1-c gcloud config set compute/region europe-west1
You can pick and choose different zones, too. Learn more about zones in Regions & Zones documentation.
Note: When you run
gcloud on your own machine, the config settings would've been persisted across sessions. But in Cloud Shell, you will need to set this for every new session / reconnection.
Join Qwiklabs to read the rest of this lab...and more!
- Get temporary access to the Google Cloud Console.
- Over 200 labs from beginner to advanced levels.
- Bite-sized so you can learn at your own pace.