Liu created FLINK-39050:
---------------------------
Summary: Support configurable handling strategy for null rowtime
field in watermark generation
Key: FLINK-39050
URL: https://issues.apache.org/jira/browse/FLINK-39050
Project: Flink
Issue Type: Improvement
Components: Table SQL / Planner
Reporter: Liu
h1. Background
When using event time processing with Flink SQL, if the rowtime field is null,
WatermarkAssignerOperator throws a RuntimeException, causing the entire job to
fail.
In production environments, data quality issues may occasionally produce
records with
null timestamp fields. The current fail-fast behavior lacks flexibility for
users who
prefer to handle such dirty data gracefully.
h1. Proposal
Introduce a new configuration `table.exec.source.rowtime-null-handling` with
the
following options:
- FAIL (default): Current behavior, throw exception
- DROP: Drop the record silently
- SKIP_WATERMARK: Forward the record without advancing watermark
Additionally, add metrics to track the number of records affected by null
rowtime.
h1. Motivation
1. Improve fault tolerance for data quality issues
2. Align with existing patterns like `json.ignore-parse-errors`
3. Provide observability through metrics
4. Maintain backward compatibility with default FAIL behavior
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
