Module 8: What to learn next · Lesson 8.4 · about 15 min
Free sandboxes to practise in
Practise everything in this course for free: what runs on your laptop, what runs in free cloud sandboxes, and four steps that stop a surprise cloud bill.
Module 8 · about 15 minutes to read, an evening to set up. You’ll be able to install everything this course uses on your own laptop, use the free browser sandboxes, make a free query allowance last, and protect yourself from a surprise bill when you try real cloud services.
1. On your laptop (free, works offline)
You need a recent Python 3, Java 17 (for Spark; check the Spark docs for the versions your Spark release supports) and Docker Desktop (or Podman). Then:
| Tool | How |
|---|---|
| PySpark | pip install pyspark. The Spark UI is at http://localhost:4040 while a job runs. |
| DuckDB and Polars | pip install duckdb polars: for comparisons and small marts. |
| Kafka | docker run -d --name kafka -p 9092:9092 apache/kafka:latest, then pip install confluent-kafka (Lesson 5.2). |
| Airflow | The official Docker Compose quick start in the Airflow docs, or pip install apache-airflow with the constraints file they give. |
| ClickHouse | docker run -d -p 8123:8123 clickhouse/clickhouse-server. |
| Druid | The official single-machine quick start or its Docker Compose file. It needs several GB of free memory. |
| OpenSearch | The single-node Docker example from the OpenSearch docs (it asks you to set an admin password), plus OpenSearch Dashboards. |
| Redis | docker run -d -p 6379:6379 redis. |
| Bigtable emulator | gcloud beta emulators bigtable start (part of the Google Cloud CLI), then point the client at localhost:8086. |
| Feast and MLflow | pip install feast mlflow (Lesson 6.2). |
Low on memory? Run one Docker service at a time, stop it when done (docker stop kafka), and practise on small samples. Everything you learn on 100,000 rows works the same on 100 million.
2. In the browser (free)
- BigQuery sandbox: sign in to the Google Cloud console and open BigQuery. You get a sandbox with no credit card and a monthly free allowance for queries and storage (check the BigQuery docs for the current amounts). Tables expire after 60 days, and some features are limited. Public datasets such as
bigquery-public-data.thelook_ecommerceare ideal practice. - Google Colab: free Python notebooks;
!pip install pysparkworks for Spark practice. - Databricks Free Edition: a free, limited Databricks workspace for Spark and Delta Lake practice.
A free query allowance is measured in bytes scanned, so the habits from BigQuery (name your columns, filter on the partition) decide how far it goes:
3. Free trial credit (use with care)
Google Cloud offers new accounts trial credit for a limited time (check the current offer). Use it for a few real runs: a Cloud Storage bucket, a Serverless Spark batch, a short-lived Dataproc cluster, a Pub/Sub topic. Cloud Composer environments cost money for as long as they exist, even when idle, so learn Airflow locally and create Composer only to see it, then delete it.
Picture a hotel minibar. Everything you take is billed, whether you meant to or not, and the bill only arrives when you check out. A budget alert is the friend who texts you when the tab passes $10.
- Sandbox
- A free, limited environment with no card, where nothing can bill you.
- Trial credit
- Free money in a real paid account: real services, real bills once it runs out or expires.
- Budget alert
- An email (or message) when spending passes a share of a budget you set. It warns; it does not stop spending by itself.
- Idle shutdown
- A setting that deletes a cluster after it has done nothing for a set time.
- Physical meaning
- Learners who get a surprise cloud bill almost always left one thing running: a cluster, a Composer environment, a big scheduled query. Four settings, done in ten minutes before the first run, turn "I hope I remembered" into "the system will tell me".
“A budget in the cloud console caps my spending.” A budget sends alerts; on its own it does not switch anything off. Spending stops only when you delete the resources (or set up automation to do it). Treat the alert as a fire alarm, not a sprinkler.
4. Before you create anything paid
- Set a budget with alerts (Billing, then Budgets and alerts) at a small amount.
- Use
--max-idlewhen you create Dataproc clusters (for example--max-idle=30m), and delete them after use. - Set maximum bytes billed on BigQuery queries you schedule, so a mistake fails instead of scanning terabytes.
- When you are done, shut down the project. That stops everything in it.
5. Try it
Set up your laptop for Module 5: start Kafka in Docker, create the pos.sales.v1 topic, and run the producer and consumer from Lesson 5.2. Then open the BigQuery sandbox and run one query on a public dataset with only the columns you need.
Check: the consumer prints partition and offset numbers; in BigQuery, the “bytes processed” shown before you run the query drops when you remove columns from the SELECT.
New words: sandbox, trial credit, budget alert, idle shutdown, maximum bytes billed.
- Spark, Kafka, Airflow, ClickHouse, Druid, OpenSearch, Redis and the Bigtable emulator all run free on a laptop with Python, Java and Docker.
- The BigQuery sandbox, Colab and the Databricks Free Edition give free cloud practice with no card.
- Free query allowances are counted in bytes scanned: select only the columns you need and filter on partitions.
- Before any paid run: budget alerts, idle shutdown on clusters, maximum bytes billed on scheduled queries, and shut down the project when done.
Check yourself
Sources
- Google Cloud documentation: BigQuery sandbox (cloud.google.com/bigquery/docs/sandbox)
- Google Cloud documentation: Create, edit or delete budgets and budget alerts (cloud.google.com/billing/docs)
- Apache Spark documentation: Quick start (spark.apache.org/docs)
- Apache Kafka quick start https://kafka.apache.org/quickstart
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. All company data in the examples is synthetic. Tool versions and prices change: check the official docs before you rely on them.