— Data
Lake Formation
Governed data lakes & permissions.
At a glance
Download cheat sheet (PDF)Access control for your data lake, expressed as grants on the Glue Data Catalog instead of IAM policies on S3 paths. Lake Formation holds the permissions on databases, tables, columns and rows, and hands query engines the credentials to match.
01 — When to use
Reach for it when
- —Several teams or accounts consume the same lake and per-role IAM policies stop scaling
- —You need column, row or cell-level security on catalog tables
- —You are building data mesh or sharing data across accounts
- —You want federated sources like Redshift datashares, S3 Tables or Snowflake governed by the same grant model
Think twice when
- —One account, one team and a handful of shared roles still express your access rules fine
- —You need row or column filters on an external engine that has not done the trusted-application integration
- —You are after a business catalog experience, which is SageMaker Catalog territory rather than Lake Formation itself
02 — How a query gets authorized
01A principal sends a query to an engine such as Athena. On LF-registered resources, every catalog and data call is authorized by Lake Formation automatically.
02Lake Formation evaluates the principal's grants. Without a grant, the query fails before it ever touches S3.
03Lake Formation hands back temporary, scoped-down S3 credentials, plus a filter policy describing the authorized columns and rows when fine-grained access is in play.
04The engine reads S3 with those credentials and applies the filter policy, so only what the grant covers comes back.
01
Query submitted
A principal sends a query to an engine such as Athena. On LF-registered resources, every catalog and data call is authorized by Lake Formation automatically.
02
Grant checked
Lake Formation evaluates the principal's grants. Without a grant, the query fails before it ever touches S3.
03
Credentials vended
Lake Formation hands back temporary, scoped-down S3 credentials, plus a filter policy describing the authorized columns and rows when fine-grained access is in play.
04
Filtered results
The engine reads S3 with those credentials and applies the filter policy, so only what the grant covers comes back.
03 — What it gives you
Grant-based permissions
SELECT, DESCRIBE, ALTER and friends on databases, tables and columns. Readable by humans, auditable by design.
LF-Tags
Tag resources once (domain=finance) and grant on the tag expression. New tables with the right tag are already permissioned. The model that scales.
Data filters
Row, column and cell-level security as named filter definitions on a table. No duplicated views per audience.
Credential vending
Users never hold S3 access. Engines receive short-lived credentials scoped to each query, vended through the location's registration role.
Cross-account sharing
The same grant call targeting another account or Organization, delivered through AWS RAM with no data copy. Consumers query through resource links.
Beyond native S3 tables
Redshift datashares, S3 Tables and federated catalogs such as Snowflake Horizon or Unity Catalog enter the same grant model, with no data duplicated.
04 — Rollout & pricing
Rollout checklist
- 01Designate one or two data lake administrators, typically an IaC pipeline role plus a human one
- 02Register S3 locations along ownership boundaries, one per domain or data product with its own registration role
- 03Start in hybrid access mode and opt principals and resources into enforcement team by team
- 04Revoke the default IAMAllowedPrincipals grant on resources where Lake Formation should actually enforce
- 05Check the engine support matrix for the granularity you need before promising row or cell filters everywhere
Pricing
Lake Formation itself is free. Permissions, LF-Tags and cross-account sharing carry no charge, and there is no LF-side charge when Athena, Redshift, Glue or EMR enforce them. The one billable piece is the Storage API, the server-side filtering engine used when building a custom engine or connector, charged per byte scanned.
Works with
AthenaRedshift SpectrumEMRGlue ETLQuickSightS3 TablesStarburstDremio
Articles
Glossary
- Data lake administrator
- An IAM principal registered in Lake Formation with the authority to grant and revoke Lake Formation permissions, register S3 locations, and create databases. Distinct from an AWS account administrator.Related: Lake Formation permissions · Registered location
- Registered location
- An S3 path placed under Lake Formation management. Once registered, access to the data goes through Lake Formation credential vending instead of the caller's own IAM S3 permissions.Related: Credential vending · Hybrid access mode
- Lake Formation permissions
- The grant/revoke permission model layered on top of the Glue Data Catalog: database, table, column, row and cell-level permissions, evaluated together with IAM.Related: LF-Tags · Data filter
- LF-Tags (LF-TBAC)
- Lake Formation's tag-based access control. Attributes (key/value tags) are attached to catalog resources and permissions are granted on tag expressions instead of resource names — the scalable alternative to named-resource grants.Related: Lake Formation permissions
- Data filter
- A named combination of column-level (include/exclude) and row-level (filter expression) restrictions on a table, used to implement row- and cell-level security.Related: Lake Formation permissions
- Credential vending
- The mechanism by which Lake Formation hands short-lived, scoped-down S3 credentials to integrated engines (Athena, Redshift Spectrum, EMR, Glue) so they can read exactly the data a principal is entitled to.Related: Registered location
- Hybrid access mode
- A per-location mode where both IAM-based S3 access and Lake Formation permissions can coexist, enabling incremental migration of an existing data lake to Lake Formation without a big-bang cutover.Related: Registered location · IAMAllowedPrincipals
- IAMAllowedPrincipals
- A virtual group holding the default "Super" grant on catalog resources for backward compatibility. As long as it is in place, Lake Formation effectively defers to IAM — revoking it is the real switch to enforcement.Related: Hybrid access mode · Lake Formation permissions
- Cross-account sharing
- Granting Lake Formation permissions on databases and tables to other AWS accounts or organizations, delivered through AWS RAM. The consuming account creates resource links to query shared resources.Related: Resource link
- Resource link
- A Data Catalog entry in the consumer account that points to a shared database or table in another account, making it addressable by local engines like Athena and Redshift.Related: Cross-account sharing
Resources
AI skills
Skills to make AI assistants use Lake Formation the dataonaws way — coming soon.