What is a data pipeline? A plain guide to ETL and ELT
A data pipeline is software that moves data from where it is created to where it is used, on a schedule and without anyone copying and pasting. If your month-end report depends on exporting spreadsheets from three systems and joining them by hand, you are the pipeline today.
By the FocalID team in Colombo, , 2 min read

What a pipeline does
- Collects data from its sources: your ERP, accounting system, website, machines or spreadsheets.
- Cleans and reshapes it: fixes formats, removes duplicates, joins related records and applies business rules.
- Delivers it to one place, usually a data warehouse, where reports, dashboards and AI systems can read it.
- Runs on its own, at set times or as data arrives, and raises an alert when something fails.
ETL and ELT
The two common designs use the same three steps, extract, transform and load, in a different order.
| ETL | ELT | |
|---|---|---|
| Order | Extract, transform, then load | Extract, load, then transform |
| Where data is reshaped | In the pipeline, before it reaches the warehouse | Inside the warehouse, after the raw data has landed |
| Raw data kept? | Usually not | Yes, so it can be reprocessed later |
| Suits | Sensitive data that must be filtered before storage, and older warehouses | Modern cloud warehouses with plenty of computing power |
Neither is better in general. ELT has become common because modern warehouses make it cheap to keep raw data and reshape it later. ETL is still the right choice when some data must never be stored at all.
What a reliable pipeline needs
- Checks on the data at each step, so a broken source is caught before it reaches a report.
- The ability to rerun a failed step without creating duplicates.
- Monitoring and alerts that reach a named person.
- A record of where each figure came from, so numbers can be traced and trusted.
Signs you are ready for one
- The same report is rebuilt by hand every week or month.
- Two departments quote different figures for the same thing.
- Reports are days old by the time they are read.
- An AI or analytics project is stuck because nobody can get clean data to it.
Start small
You do not need a data platform on day one. Choose the one report that costs the most effort or causes the most arguments, and build the pipeline for that. Its sources, rules and checks become the base for the next one.
Common questions
Is a data pipeline the same as a data warehouse?
No. The warehouse is where the data is stored. The pipeline is what brings data into it and keeps it up to date.
Do small businesses need data pipelines?
If people spend hours each week exporting and combining data by hand, a small pipeline will usually repay its cost quickly.
How often does a pipeline run?
As often as the business needs: nightly for most reporting, every few minutes or continuously where decisions depend on fresh data.

