Module 7: Run it in production · Lesson 7.7 · about 35 min
Ship it and tell the story
Finish the course: a portfolio README and demo, honest resume lines, a structured answer to the platform design question, and your next 90 days.
Module 7 · the last lesson · about 35 minutes · read first: 7.6. You’ll publish the capstone so a stranger understands it in two minutes, turn seven projects into honest resume lines and one good article, answer “design a data platform” with a clear structure, and plan what to learn next.
1. Why this matters
A busy reviewer often gives a portfolio repository only a couple of minutes before deciding whether to read more. Think of an estate agent’s listing: the best house in town does not sell if the photos are dark and the address is wrong. Today is about the listing. The house is already built.
2. Look back: what you built
At the start of this course you had a folder of messy files. Here is what you can now show, project by project:
| Module | Project | What it proves |
|---|---|---|
| 1 | A multi-source landing service | files from email, drives, SFTP and APIs land once, with a manifest |
| 2 | A trustworthy cleaning pipeline | contracts, quarantine and a data quality report |
| 3 | A self-running hotel pipeline | Dagster assets, partitions, sensors and checks |
| 4 | One pipeline, any warehouse | upserts that rerun for free, audit, security, cost |
| 5 | A tested star schema | Kimball modelling, SCD2 and dbt tests |
| 6 | BI people can trust | fast models, correct DAX, reconciliation |
| 7 | The daily data platform | all of it together, observable, tested and costed |
3. The capstone README
- Title and one sentence, with the health board or architecture picture right under it.
- What it does in 5 bullets with numbers: sources, rows a day, run time, cost a month.
- Demo video, 2 to 3 minutes: the demo morning.
- Architecture diagram and a stack table with why for each choice (link the ADRs).
- Run it in 5 minutes:
just demo. - Design decisions: idempotency, business dates, quarantine, partitions, upserts, SCD2, the metric dictionary, missing is not zero.
- Reliability: SLOs, checks, triage reports, the incident review.
- Cost per location per month for three stacks.
- What I would do next.
- Links to the six project repos.
4. Resume lines and one good article
Fill in your own measured numbers. Interviewers ask “how did you measure that?”, so every number must be one you can show.
- Built an end-to-end daily data platform (Python, Dagster, dbt, Azure SQL or Postgres, Power BI) that ingests email, SFTP, shared-drive and API sources for four synthetic businesses; reruns are idempotent and every report figure traces back to source files.
- Designed Kimball star schemas with SCD2 dimensions and [N] dbt tests, including a revenue reconciliation check on every build.
- Cut full-pipeline run time from [A] to [B] by making every layer incremental and compacting Parquet files.
- Costed three platform options at 20 to 500 locations; the open-source stack came to about $[X] per location per month.
Then write one article about one hard problem, not the whole course: “Missing is not zero: how a completeness matrix caught a silent store”, or “Why my revenue doubled: fan-out and the grain sentence”. Short, specific, one chart. People share posts like that; nobody shares “I finished a course”.
“Say it was production work; everyone exaggerates.” It is easy to check and it ends interviews. Say exactly what is true, with confidence: “I built the full stack from scratch on synthetic data to understand every layer, from email and SFTP ingestion through Dagster, a warehouse and dbt to Power BI, with open-source options costed. The repos show every decision and test.” Keep any current or former employer, its systems and its data out of it.
5. The system-design question
“Design a data platform for a 200-restaurant chain that wants daily sales dashboards by 8 a.m.” Answer in this order, in about ten minutes:
- Portfolio README
- The front page of your repo: what it does, why, how to run it in 5 minutes, and links to proof.
- Measured number
- A figure you can reproduce from the repo (run time, rows, cost), never a guess.
- Structured answer
- Clarify, size, flow and tools, correctness, reliability, security and cost, evolution: in that order.
- Breadth first
- Touch every area at a basic level, then go deep where the interviewer asks.
- Physical meaning
- A hiring manager at a restaurant group opens your repo between meetings, watches the demo morning, sees "store 107 not received" in the banner and thinks "this person already knows our Monday problems". That is the whole point of today.
Maths and principle
Principle: structured answers beat brilliant fragments, because interviewers score against a rubric with many areas.
Equation: with areas scored 0 to 2, going deep on areas scores ; touching all at 1 and deepening of them scores
so breadth wins whenever , which it nearly always does in ten minutes.
Analogy: painting a room: one coat on every wall first, then a second coat where it shows. Never three coats on one wall and bare plaster on the rest.
Where it breaks: if the interviewer says “go deep on ingestion”, go deep on ingestion.
Try it: 8 areas. Deep on 3 and skip 5? (6 of 16.) All 8 at 1 point and 2 of them at 2? (10 of 16.)
6. Your next 90 days
You can now run a daily platform on one or two servers. The natural next step is what happens when the data, the team or the speed outgrows that: Data at Scale covers Spark, cloud warehouses such as BigQuery, lakehouse table formats and streaming with Kafka. It starts exactly where Lesson 7.5’s “when to change gear” ends.
Other fibub courses that fit next to this one: Power BI to go deeper on DAX and models, Statistics for the numbers behind forecasting and tests, pandas, ML foundations when the business asks for predictions, and Offline AI for local language models. If a certificate matters for the jobs you want, look at Microsoft’s Power BI data analyst and Fabric data engineer certifications (PL-300 and DP-700 at the time of writing; check the current names on Microsoft Learn).
7. Try it
- Polish the capstone README to the 10-point list and record the video.
- Update your GitHub profile README with the seven projects, one line each, linked.
- Write your resume lines with your real numbers.
- Draft the article and a short post that links to it.
- Answer the system-design question aloud twice, recorded. Watch it back once, and note which of the six steps you skipped.
- Write
docs/next-90-days.md: one deeper topic a month (for example the first three modules of Data at Scale, advanced DAX, ML in production) and one certification to consider.
What to check: a friend who has never seen the repo can run just demo from the README alone; every number in your resume lines has a link to where it was measured; your recorded answer touches all six steps inside ten minutes. Commit as docs: capstone README, demo video, next 90 days. That is the course. Well done.
- Seven projects are proof only if a stranger can understand and run them in minutes: README, demo, links.
- Every resume number must be one you measured and can show.
- Describe the work honestly: built end to end on synthetic data, with the repos as proof.
- Answer design questions in order: clarify, size, flow and tools, correctness and reliability, security and cost, evolution.
- Next: Data at Scale, for when the data, the team or the speed outgrows one server.
Check yourself
Sources
- Kleppmann, M. (2017), Designing Data-Intensive Applications, O'Reilly, Chapter 1 (reliable, scalable and maintainable applications)
- Reis, J. and Housley, M. (2022), Fundamentals of Data Engineering, O'Reilly, Chapter 3 (designing good data architecture)
- Microsoft Learn: Credentials and certifications for data and AI roles https://learn.microsoft.com/en-us/credentials/
Your ticks stay in this browser. Save them as a file on the course home.
Updated 2026-11-22. All company data in the examples is synthetic. Tool versions and prices change: check the official docs before you rely on them.