Module 6: Data platforms for AI and ML · Lesson 6.2 · about 25 min
Feature stores, pipelines and MLOps
Offline and online feature stores, materialisation, ML pipelines, experiment tracking, model registries and serving, with Vertex AI, Feast and MLflow.
Module 6 · about 25 minutes · read first: 6.1. You’ll be able to explain offline and online feature stores and materialisation, work out how stale an online feature can be, sketch an ML pipeline from data to deployed model, and name the MLOps tools on Google Cloud and in open source.
1. The feature store
A feature store is a catalogue plus two storage layers that make features reusable and consistent:
- Feature definitions: named, owned and documented, such as
shopper.buys_30d, computed fromfct_sales. - Offline store: the full history, used to build training sets with point-in-time joins (Lesson 6.1). Usually BigQuery or lake tables.
- Online store: only the latest value per entity, for fast lookups by live models. Usually Bigtable, Redis or DynamoDB.
- Materialisation: jobs (batch or streaming) that compute feature values and copy the newest ones into the online store.
Think of a pharmacy. The offline store is the warehouse that keeps every batch ever made, with dates, for audits and research. The online store is the counter: only what is current, ready to hand over in seconds. Materialisation is the van that restocks the counter from the warehouse. The list of feature definitions is the formulary: what each medicine is and who is responsible for it.
Options include Vertex AI Feature Store (built on BigQuery, with online serving), Feast (open source), Databricks feature engineering, Tecton and SageMaker Feature Store. Many teams simply use well-modelled BigQuery tables plus a Bigtable or Redis copy: a feature store in all but name.
fct_sales in BigQuery at 10:12.buys_7d for shoppers with new sales. For s77 it goes from 3 to 4.s77 now holds buys_7d = 4. Only the latest value is kept.s77 from the online store in a few milliseconds and sees 4: the same number, computed by the same code, that training rows use.- Feature definition
- The one agreed recipe for a feature: its source, its formula, its owner.
- Offline store
- Every value over time, for building training sets.
- Online store
- The latest value per entity, for live predictions in milliseconds.
- Materialisation
- The scheduled or streaming job that refreshes the online store.
- Physical meaning
- When a pharmacy app shows a member a vitamin offer the moment they log in, a model read that member's features from a fast online store, filled minutes earlier by a materialisation job, using the same feature recipe the model was trained with months before.
The online store holds just one row per entity, but for millions of shoppers that still adds up:
Freshness is the other number to know. A batch materialisation job can only be as fresh as its schedule allows:
2. ML pipelines
An ML pipeline is a DAG like the ones you built with Airflow, with machine learning steps:
- Extract a training snapshot (point-in-time features plus labels) from BigQuery.
- Validate the data: schema, ranges, and drift compared with last time.
- Train (scikit-learn, XGBoost, PyTorch or BigQuery ML) and tune.
- Evaluate against the current live model, including fairness checks.
- Register the model with its metrics and lineage.
- Deploy (a batch scoring job or an online endpoint) after approval.
- Monitor, and trigger retraining when needed.
Tools: Vertex AI Pipelines (runs Kubeflow Pipelines definitions), Kubeflow, Airflow (common for batch retraining), Metaflow, ZenML and Databricks workflows.
3. Tracking, registry and serving
| Need | Google Cloud | Open source and others |
|---|---|---|
| Experiment tracking | Vertex AI Experiments | MLflow, Weights & Biases |
| Model registry | Vertex AI Model Registry | MLflow Model Registry |
| Batch prediction | Vertex AI batch prediction, BigQuery ML ML.PREDICT |
Spark or Airflow jobs |
| Online serving | Vertex AI endpoints | KServe, BentoML, Ray Serve |
| Monitoring | Vertex AI Model Monitoring | Evidently, custom checks |
| Distributed training | Vertex AI training on GPUs and TPUs | Ray, PyTorch distributed, Spark MLlib |
MLOps is the name for running all of this reliably: the machine learning version of the testing, deployment and monitoring habits from software engineering.
“We need to buy a feature store before we can do machine learning.” Many teams run well for years on tested, documented BigQuery tables for training and a small Redis or Bigtable copy for serving. Buy or adopt a feature store when several teams reuse the same features and the hand-made copies start to disagree.
4. What “a platform supporting ML” looks like
- Clean, documented, time-stamped tables in the warehouse that data scientists trust.
- Raw history kept in the lake (Parquet or Iceberg), so features can be rebuilt with new definitions.
- Reusable feature pipelines (Spark, BigQuery SQL or dbt, Flink for streaming) with tests.
- An online store fed by those pipelines, with freshness monitoring.
- Reproducibility: each model linked to its code version, data snapshot and features.
- Cost controls on GPU and training jobs, and access controls on personal data.
If you have built tested star schemas and reconciled numbers with finance (as in Pipeline to Dashboard), you already have the most valuable skill here: trustworthy, well-modelled data. Add point-in-time correctness, feature freshness and model lineage to your vocabulary and you can talk with machine learning teams as an equal.
5. Try it (optional code, about 30 minutes)
Install Feast and follow its quickstart, which builds a small demo repository with sample data:
pip install feast
feast init my_project
cd my_project/feature_repo
feast apply # register the feature definitions
feast materialize-incremental $(date -u +"%Y-%m-%dT%H:%M:%S") # fill the online store
Then run the quickstart’s script that fetches online features. Finally, write three sentences that map each Feast piece (feature view, offline store, online store, materialize) to its Vertex AI equivalent.
Check: the online lookup returns values for the demo’s driver IDs, and running materialize-incremental again with a later time changes them only for rows with newer data.
New words: feature store, feature definition, offline store, online store, materialisation, ML pipeline, experiment tracking, model registry, endpoint, MLOps.
- A feature store = feature definitions + an offline store (full history, for training) + an online store (latest values, for serving).
- Materialisation refreshes the online store; worst-case staleness = data landing delay + time between runs + run time.
- An ML pipeline is a DAG: snapshot, validate, train, evaluate, register, deploy, monitor.
- Experiment tracking and a model registry make every model reproducible and every deployment reversible.
Check yourself
Sources
- Feast documentation: Quickstart and concepts https://docs.feast.dev/
- MLflow documentation: Tracking and Model Registry https://mlflow.org/docs/latest/index.html
- Google Cloud documentation: Vertex AI Feature Store and Vertex AI Pipelines (cloud.google.com/vertex-ai/docs)
- Sculley, D. et al. (2015), Hidden Technical Debt in Machine Learning Systems, Advances in Neural Information Processing Systems 28
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. All company data in the examples is synthetic. Tool versions and prices change: check the official docs before you rely on them.