Interactive lab · Data engineering
Twelve messy records go in, a trustworthy dashboard comes out. Follow each record through every stage — then break the pipeline on purpose.
Follow one record
Before you run the full pipeline below, meet record #102 — " JOHN@EMAIL.COM " — spaces, capitals and all. Follow it through every stage, then watch what happens to a record with no email at all.
Follow one messy record as it gets cleaned up.
Follow record #102 as it travels the seven stages.
" JOHN@EMAIL.COM " — spaces, capitals and all, exactly as the signup form sent it.
The row is parsed into fields — name, email, zip, date. Celebration: “parsed!”
Email is present, the zip is five digits, the date is real. “checks passed!”
Trim the spaces — “trimmed!” — then lowercase — “lowercased!”. The record itself changes: "JOHN@EMAIL.COM" → "john@email.com".
Pulls "email.com" out of the address so the dashboard can group by domain. “domain added!”
The finished record is saved. “stored!”
The clean record lands on the chart. “on the chart!”
Record #103 has no email at all, so it never gets past step 3: no email address → rejected. The reject bin isn't a failure — it's where broken records wait for a human to fix the source. Pipelines never guess.
The simulator
This is ETL — with the two steps pipelines spend most of their time on: cleaning and validation. Press Run pipeline: each dot is one signup record. Watch what each stage does to it.
Simulated dataset of twelve made-up records. Everything runs in your browser — nothing is uploaded anywhere.
Repair challenge: the CLEAN step's “normalize email” rule is off, so VALIDATE will reject the messy emails. Run the pipeline, read the rejected-records log, then click the CLEAN stage to find the switch.
record in flightrejectedClick any stage to inspect its rules. Records move through concurrently, like an assembly line.
Output
0 signups loaded
Simulated dashboard, drawn from the records that survived the pipeline. A dashboard is only as honest as the rules behind it.
| # | name | zip | signup_date | |
|---|---|---|---|---|
| 101 | Ava Martinez | ava.m@example.com | 33901 | 2026-08-12 |
| 102 | John Reyes | " JOHN@EMAIL.COM " | 33902 | 2026-08-19 |
| 103 | Sofia Patel | (missing) | 33903 | 2026-08-21 |
| 104 | Noah Kim | noah.kim@example.com | ABCDE | 2026-08-25 |
| 105 | Emma Liu | emma.liu@example.com | 33904 | 2026-13-45 |
| 106 | Ava Martinez | ava.m@example.com (duplicate of #101) | 33901 | 2026-08-12 |
| 107 | " olivia brown " | olivia.b@sample.org | 33909 | 2026-09-02 |
| 108 | Ethan Wright | ETHAN.W@MAIL.IO | 33912 | 2026-09-15 |
| 109 | Mia Garcia | mia.garcia@example | 33913 | 2026-09-18 |
| 110 | Lucas Moore | lucas.m@example.com | 3390 | 2026-09-21 |
| 111 | Isabella Ross | isabella.r@test.net | 33990 | 09/24/2026 |
| 112 | James Park | james.p@example.com | 33914 | 2026-09-29 |
Made-up data. Messy cells are highlighted; quotes show where stray spaces hide.
Metrics
Throughput per stage
Records entering each stage. The bars shrink wherever a stage rejects records — that shrinkage is the pipeline doing its job.
Inspector
Click any stage above to see what it does and the exact rules it runs. Tip: the repair challenge involves the CLEAN stage.
Event log
Rejected records
Three jobs
Every data pipeline does these three jobs in the middle, and mixing them up is where most data bugs come from.
The bouncer. It checks each record against rules — a zip must be five digits, a date must be a real calendar day — and never edits the data. Record #104's zip ABCDE is rejected with a reason. One question: is this record acceptable as it stands?
The tidy-up. It repairs formatting, not substance: " JOHN@EMAIL.COM " becomes "john@email.com". It can't rescue a missing email or turn letters into a zip code — what it can't fix moves on to be judged.
The reshape. It derives new fields the dashboard needs: the email domain, the signup month. It assumes the data is already clean and valid — which is exactly why it runs last.
A design decision
A guess that looks like data is worse than a rejection you can see.
Record #104's zip is ABCDE — no cleaning rule can turn letters into a zip code, and inventing one would corrupt every chart downstream. So the pipeline refuses to guess: it stamps the record, drops it in the log with a reason, and a human fixes the source. Loud beats clever, every time.
Try it
Press Run pipeline in challenge mode. Watch VALIDATE reject the messy emails while the log explains why.
Click the CLEAN node: “Normalize email (trim + lowercase)” is switched off. This pipeline cleans before it validates — formatting fixes first, judgment after.
Flip the switch and run again. The two messy emails are normalized and sail through; the genuinely broken records are still rejected, correctly.
Switch to free-run mode for a healthy end-to-end run — or click any stage to inspect its real rules. Toggling one genuinely changes the run.
Behind the build
Questions
ETL stands for Extract, Transform, Load: pulling data out of where it is born, reshaping it, and loading it where it will be used. This lab shows the fuller modern version — extract, clean, validate, transform, load — because real pipelines spend most of their effort on the cleaning and checking in the middle.
Cleaning fixes formatting: trimming spaces, lowercasing an email, reformatting a date. Validation checks truth: is this a real calendar date, is the zip five digits, is the email actually present? Cleaning changes how a value looks; validation judges whether the record is acceptable. A record that cleaning cannot fix is rejected, never guessed.
A guess that looks like data is worse than a rejection you can see. If the pipeline invented a zip code for “ABCDE”, every report downstream would be quietly wrong. A rejected record lands in a log with a plain-language reason, so a human can fix the source. Rejection is a feature, not a failure.
No. The dataset is twelve made-up signup records, and the entire pipeline runs in your browser tab. Nothing you do here leaves your device.
Yes. The Data Pipeline Lab is free to use as often as you like — no account, no sign-up. PlainLogic may introduce optional accounts or paid features in the future, but the core labs are free to use today.
Keep exploring
Real pipelines carry real data. Watch live weather, alerts, earthquakes, and radar flow through an operations console.
Another simulator from the bench: send packets through switches and routers, inject faults, and learn to diagnose them.
Games, experiments, tools, and the blog — the full bench.