博客 Engineering

Catching Invalid Kafka Formats at Plan Time in the Materialize Terraform Provider

英文文章. materialize_source_kafka accepted format combinations that failed at apply. The merged fix rejects them during terraform plan.

GoMaterializeInc/terraform-provider-materializeTerraformKafkaschema validation

Moving an apply-time SQL error to a plan-time validation error, in a two-year-old issue.

简短回答

The materialize_source_kafka resource let you set format together with key_format, which Materialize does not accept, so terraform apply failed with a SQL syntax error. The fix adds ConflictsWith and RequiredWith rules to the schema, so Terraform rejects invalid combinations during plan. It was merged as PR #947 and closed issue #654, which had been open since September 2024.

The problem

Issue #654 showed a Kafka source with both format { json = true } and key_format { json = true }. Applying it failed with:

Error: ERROR: Expected end of statement, found KEY (SQLSTATE 42601)

The issue already proposed the fix: add a ConflictsWith so format can't be combined with key_format and value_format.

The fix

"format": func() *schema.Schema {
    ...
    s.ConflictsWith = []string{"key_format", "value_format"}
    return s
}(),
"key_format": func() *schema.Schema {
    ...
    s.ConflictsWith = []string{"format"}
    s.RequiredWith = []string{"value_format"}
    return s
}(),
"value_format": func() *schema.Schema {
    ...
    s.ConflictsWith = []string{"format"}
    s.RequiredWith = []string{"key_format"}
    return s
}(),

According to the PR description, these constraints match the ones already used by materialize_source_table_kafka. The attribute descriptions and generated docs now spell the rules out.

By the numbers

  • Repository: MaterializeInc/terraform-provider-materialize, 14 stars (Nov 3, 2026), Go, MPL-2.0
  • Files changed: 4
  • Lines: +70 / -6
  • Commits: 2 (1 from the PR author, 1 from a maintainer)
  • Opened: Oct 6, 2026. Merged: Oct 10, 2026. Time to merge: 3 days, 20 hours
  • Review: approved and merged by a maintainer

How the review went

The maintainer who filed the original issue approved the PR and added a commit of their own: "Test the Kafka source format rules through validation, regenerate docs, and add a changelog entry." That commit replaced the PR's original 14-line test with a table-driven test that runs the resource's Validate and checks for errors on each combination. The changelog entry reads, in part, "these failed at apply with a syntax error; they are now rejected at plan time."

常见问题解答

These format attributes are marked deprecated in the provider, which points users to materialize_source_table_kafka. The fix makes the deprecated path fail early and clearly.

Where can I read more?

See the pull request and the original issue.

目标

Infrastructure tools should fail at plan, not halfway through apply. Here the schema fix was small, and the maintainer reshaped the test to match how the project tests validation.

运行闭环

把生产信号转化为已审查的修复。

开始免费试用,了解 Prilog 如何把真实事故映射到代码级 Pull Request。