dataonawsGet in touch
← Lake Formation

Lake Formation

dataonaws.com — cheat sheet

Access control for your data lake, expressed as grants on the Glue Data Catalog instead of IAM policies on S3 paths. Lake Formation holds the permissions on databases, tables, columns and rows, and hands query engines the credentials to match.

Reach for it when

  • —Several teams or accounts consume the same lake and per-role IAM policies stop scaling
  • —You need column, row or cell-level security on catalog tables
  • —You are building data mesh or sharing data across accounts
  • —You want federated sources like Redshift datashares, S3 Tables or Snowflake governed by the same grant model

Think twice when

  • —One account, one team and a handful of shared roles still express your access rules fine
  • —You need row or column filters on an external engine that has not done the trusted-application integration
  • —You are after a business catalog experience, which is SageMaker Catalog territory rather than Lake Formation itself

How a query gets authorized

Flow step 1 diagramFlow step 2 diagramFlow step 3 diagramFlow step 4 diagram
01
Query submitted
A principal sends a query to an engine such as Athena. On LF-registered resources, every catalog and data call is authorized by Lake Formation automatically.
02
Grant checked
Lake Formation evaluates the principal's grants. Without a grant, the query fails before it ever touches S3.
03
Credentials vended
Lake Formation hands back temporary, scoped-down S3 credentials, plus a filter policy describing the authorized columns and rows when fine-grained access is in play.
04
Filtered results
The engine reads S3 with those credentials and applies the filter policy, so only what the grant covers comes back.

What it gives you

Grant-based permissions
SELECT, DESCRIBE, ALTER and friends on databases, tables and columns. Readable by humans, auditable by design.
LF-Tags
Tag resources once (domain=finance) and grant on the tag expression. New tables with the right tag are already permissioned. The model that scales.
Data filters
Row, column and cell-level security as named filter definitions on a table. No duplicated views per audience.
Credential vending
Users never hold S3 access. Engines receive short-lived credentials scoped to each query, vended through the location's registration role.
Cross-account sharing
The same grant call targeting another account or Organization, delivered through AWS RAM with no data copy. Consumers query through resource links.
Beyond native S3 tables
Redshift datashares, S3 Tables and federated catalogs such as Snowflake Horizon or Unity Catalog enter the same grant model, with no data duplicated.

Rollout checklist

  1. 01Designate one or two data lake administrators, typically an IaC pipeline role plus a human one
  2. 02Register S3 locations along ownership boundaries, one per domain or data product with its own registration role
  3. 03Start in hybrid access mode and opt principals and resources into enforcement team by team
  4. 04Revoke the default IAMAllowedPrincipals grant on resources where Lake Formation should actually enforce
  5. 05Check the engine support matrix for the granularity you need before promising row or cell filters everywhere

Pricing

Lake Formation itself is free. Permissions, LF-Tags and cross-account sharing carry no charge, and there is no LF-side charge when Athena, Redshift, Glue or EMR enforce them. The one billable piece is the Storage API, the server-side filtering engine used when building a custom engine or connector, charged per byte scanned.

Glossary

Data lake administrator
An IAM principal registered in Lake Formation with the authority to grant and revoke Lake Formation permissions, register S3 locations, and create databases. Distinct from an AWS account administrator.
Registered location
An S3 path placed under Lake Formation management. Once registered, access to the data goes through Lake Formation credential vending instead of the caller's own IAM S3 permissions.
Lake Formation permissions
The grant/revoke permission model layered on top of the Glue Data Catalog: database, table, column, row and cell-level permissions, evaluated together with IAM.
LF-Tags (LF-TBAC)
Lake Formation's tag-based access control. Attributes (key/value tags) are attached to catalog resources and permissions are granted on tag expressions instead of resource names — the scalable alternative to named-resource grants.
Data filter
A named combination of column-level (include/exclude) and row-level (filter expression) restrictions on a table, used to implement row- and cell-level security.
Credential vending
The mechanism by which Lake Formation hands short-lived, scoped-down S3 credentials to integrated engines (Athena, Redshift Spectrum, EMR, Glue) so they can read exactly the data a principal is entitled to.
Hybrid access mode
A per-location mode where both IAM-based S3 access and Lake Formation permissions can coexist, enabling incremental migration of an existing data lake to Lake Formation without a big-bang cutover.
IAMAllowedPrincipals
A virtual group holding the default "Super" grant on catalog resources for backward compatibility. As long as it is in place, Lake Formation effectively defers to IAM — revoking it is the real switch to enforcement.
Cross-account sharing
Granting Lake Formation permissions on databases and tables to other AWS accounts or organizations, delivered through AWS RAM. The consuming account creates resource links to query shared resources.
Resource link
A Data Catalog entry in the consumer account that points to a shared database or table in another account, making it addressable by local engines like Athena and Redshift.