Services

Resources

Company

ETL (extract, transform, load)

ETL stands for extract, transform, load. An ETL job extracts data from source systems such as application databases, files or APIs, transforms it into the shape the destination needs, and loads it into a warehouse or database. It is the classic way to feed reporting and analytics.

How it works

Extraction pulls either a full copy or only the rows changed since the last run. Transformation cleans and reshapes: converting types and units, removing duplicates and test data, joining tables and computing aggregates. Loading writes the result into target tables, often replacing one day's partition at a time.

For example, at 01:00 a job extracts 1.2 million order rows, converts amounts from cents to dollars, drops test accounts, joins each order to the customer's region, and loads daily_sales by 01:20.

Why it matters in production

ELT flips the last two steps: load raw data into the warehouse first, then transform it there with SQL. Modern columnar warehouses make ELT common, because raw data stays available for reprocessing. Either way, jobs need to be idempotent so a rerun after a failure does not double-count.

See also

On the One2N blog

References

  1. Extract, transform, load (Wikipedia)

Four-panel comic about ETL: a 07:00 sales report needs data from three sources; an ETL job extracts, transforms and loads into the warehouse; at 01:00 it extracts 1,200,000 order rows, converts cents to dollars, drops 8,000 test rows, joins customer region and loads 1,192,000 rows into daily_sales by 01:20; ELT loads raw data first and transforms inside the warehouse.