블로그 Engineering

Catching Invalid Kafka Formats at Plan Time in the Materialize Terraform Provider

영문 기사. materialize_source_kafka accepted format combinations that failed at apply. The merged fix rejects them during terraform plan.

GoMaterializeInc/terraform-provider-materializeTerraformKafkaschema validation

Moving an apply-time SQL error to a plan-time validation error, in a two-year-old issue.

짧은 답변

The materialize_source_kafka resource let you set format together with key_format, which Materialize does not accept, so terraform apply failed with a SQL syntax error. The fix adds ConflictsWith and RequiredWith rules to the schema, so Terraform rejects invalid combinations during plan. It was merged as PR #947 and closed issue #654, which had been open since September 2024.

The problem

Issue #654 showed a Kafka source with both format { json = true } and key_format { json = true }. Applying it failed with:

Error: ERROR: Expected end of statement, found KEY (SQLSTATE 42601)

The issue already proposed the fix: add a ConflictsWith so format can't be combined with key_format and value_format.

The fix

"format": func() *schema.Schema {
    ...
    s.ConflictsWith = []string{"key_format", "value_format"}
    return s
}(),
"key_format": func() *schema.Schema {
    ...
    s.ConflictsWith = []string{"format"}
    s.RequiredWith = []string{"value_format"}
    return s
}(),
"value_format": func() *schema.Schema {
    ...
    s.ConflictsWith = []string{"format"}
    s.RequiredWith = []string{"key_format"}
    return s
}(),

According to the PR description, these constraints match the ones already used by materialize_source_table_kafka. The attribute descriptions and generated docs now spell the rules out.

By the numbers

  • Repository: MaterializeInc/terraform-provider-materialize, 14 stars (Nov 3, 2026), Go, MPL-2.0
  • Files changed: 4
  • Lines: +70 / -6
  • Commits: 2 (1 from the PR author, 1 from a maintainer)
  • Opened: Oct 6, 2026. Merged: Oct 10, 2026. Time to merge: 3 days, 20 hours
  • Review: approved and merged by a maintainer

How the review went

The maintainer who filed the original issue approved the PR and added a commit of their own: "Test the Kafka source format rules through validation, regenerate docs, and add a changelog entry." That commit replaced the PR's original 14-line test with a table-driven test that runs the resource's Validate and checks for errors on each combination. The changelog entry reads, in part, "these failed at apply with a syntax error; they are now rejected at plan time."

FAQ

These format attributes are marked deprecated in the provider, which points users to materialize_source_table_kafka. The fix makes the deprecated path fail early and clearly.

Where can I read more?

See the pull request and the original issue.

목표

Infrastructure tools should fail at plan, not halfway through apply. Here the schema fix was small, and the maintainer reshaped the test to match how the project tests validation.

루프 실행

프로덕션 신호를 리뷰된 수정으로 바꾸세요.

무료 체험으로 Prilog가 실제 인시던트를 코드 수준 Pull Request로 연결하는 방식을 확인하세요.