Skip to content
Datablast Docs

Materialization Strategies

Materialization strategies determine how SQL task results are stored and managed in the target database.

Table Materialization

Section titled “Table Materialization”

Create + Replace

Section titled “Create + Replace”

Replaces the entire table with new data on each run.

materialization:
  type: "table"
  strategy: "create+replace"
  partition_by: "dt"
  cluster_by: ["user_id", "event_type"]

Delete + Insert (Incremental)

Section titled “Delete + Insert (Incremental)”

Deletes existing data for a specific key and inserts new data.

materialization:
  type: "table"
  strategy: "delete+insert"
  incremental_key: "dt"

Append

Section titled “Append”

Adds new data to the existing table without removing old data.

materialization:
  type: "table"
  strategy: "append"

View Materialization

Section titled “View Materialization”

Standard View

Section titled “Standard View”

Creates a virtual table that queries underlying data.

materialization:
  type: "view"

Partitioning Configuration

Section titled “Partitioning Configuration”

Date Partitioning

Section titled “Date Partitioning”
materialization:
  type: "table"
  strategy: "create+replace"
  partition_by: "dt"

Integer Partitioning

Section titled “Integer Partitioning”
materialization:
  type: "table"
  strategy: "create+replace"
  partition_by: "user_id"

Clustering Configuration

Section titled “Clustering Configuration”

Single Column Clustering

Section titled “Single Column Clustering”
materialization:
  type: "table"
  strategy: "create+replace"
  cluster_by: ["user_id"]

Multi-Column Clustering

Section titled “Multi-Column Clustering”
materialization:
  type: "table"
  strategy: "create+replace"
  cluster_by: ["user_id", "event_type", "created_at"]

Complete Configuration Examples

Section titled “Complete Configuration Examples”

Full Table Refresh

Section titled “Full Table Refresh”
name: "analytics.daily_metrics"
type: "bq.sql"
description: "Calculate daily metrics"
run: "daily_metrics.sql"
materialization:
  type: "table"
  strategy: "create+replace"
  partition_by: "dt"
  cluster_by: ["user_id", "event_type"]

Incremental Processing

Section titled “Incremental Processing”
name: "analytics.user_events"
type: "bq.sql"
description: "Process user events incrementally"
run: "user_events.sql"
materialization:
  type: "table"
  strategy: "delete+insert"
  incremental_key: "dt"
  partition_by: "dt"

Append-Only Data

Section titled “Append-Only Data”
name: "logs.event_logs"
type: "bq.sql"
description: "Append event logs"
run: "event_logs.sql"
materialization:
  type: "table"
  strategy: "append"
  partition_by: "dt"

View Creation

Section titled “View Creation”
name: "analytics.user_summary"
type: "bq.sql"
description: "Create user summary view"
run: "user_summary.sql"
materialization:
  type: "view"

Configuration Parameters

Section titled “Configuration Parameters”

Required Fields

Section titled “Required Fields”
  • type: Materialization type (table or view)
  • strategy: Materialization strategy (for tables)

Optional Fields

Section titled “Optional Fields”
  • partition_by: Partition column
  • cluster_by: Clustering columns
  • incremental_key: Key for incremental processing
Section titled “Related Documentation”