dataonaws
— Stack · Data

AWS Glue

Three things under one name: a catalog every engine reads, crawlers that fill it, and serverless Spark jobs. Compute is billed by the second, the catalog by the object.

How it fits

Glue writes the tables. Every engine reads the same catalog.

SourcesS3, databases, SaaSreadAWS GLUECrawlersinfer schemasData Catalogtables, partitions, locationsETL jobsserverless Spark or Python shellJob bookmarks · Data Quality · FlexS3 tablesParquet, IcebergAthena, Redshift, EMRread the same catalogjob logs + metrics → CloudWatch
One name, three bills

Each part is priced on its own.

Metastore

Data Catalog

First million objects and first million requests free each month, then about $1 per 100,000 objects.

Discovery

Crawlers

About $0.44 per DPU-hour while they run. Scheduled crawls add up quickly.

Compute

ETL jobs

About $0.44 per DPU-hour, billed by the second. One DPU is 4 vCPUs and 16 GB of memory.

Prices for US East (N. Virginia), check the current price for your region.

The three choices that matter

Most of the bill comes from here.

01

How big a worker

A small file transform does not need Spark. Size workers for the data, and turn on auto scaling so idle workers stop billing.

02

Standard or Flex

Flex runs on spare capacity at a lower price per DPU-hour, with no guaranteed start time. Fits nightly batch, not jobs with a deadline.

03

How much it reads

Most job time is spent reading. Filter on partition keys and use bookmarks so each run only reads what changed.

What you pay

A Glue bill, in one line.

ETL jobs
DPU-hours
Workers × run time, by the second
+
Crawlers
DPU-hours
Every scheduled run
+
Catalog
objects, requests
Beyond the free tier
+
Notebooks
interactive sessions
5 DPUs by default, while open
What we see in production
Crawlers scheduled hourly on data that rarely changes.Crawl on change, or register partitions without a crawler.
Jobs that write thousands of tiny files.Every later read gets slower and more expensive; coalesce before writing.
Notebook sessions left open overnight.Set an idle timeout on interactive sessions.
How we help

Industrial-grade data pipelines

Use cases in production, not POCs: tests, CI/CD, lineage, on-call that sleeps at night.

Read the approach →
PDF · coming soon

The one-page AWS Glue field guide.

Best practices, cost traps and performance checks, in the order you should run them. Printable, no fluff.

One email when the PDF is ready. Nothing else unless you tick the box.