[
https://issues.apache.org/jira/browse/FLINK-39050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-39050:
-----------------------------------
Labels: pull-request-available (was: )
> Support configurable handling strategy for null rowtime field in watermark
> generation
> -------------------------------------------------------------------------------------
>
> Key: FLINK-39050
> URL: https://issues.apache.org/jira/browse/FLINK-39050
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Liu
> Priority: Major
> Labels: pull-request-available
>
> h1. Background
> When using event time processing with Flink SQL, if the rowtime field is
> null,
> WatermarkAssignerOperator throws a RuntimeException, causing the entire job
> to fail.
> In production environments, data quality issues may occasionally produce
> records with
> null timestamp fields. The current fail-fast behavior lacks flexibility for
> users who
> prefer to handle such dirty data gracefully.
> h1. Proposal
> Introduce a new configuration `table.exec.source.rowtime-null-handling` with
> the
> following options:
> - FAIL (default): Current behavior, throw exception
> - DROP: Drop the record silently
> - SKIP_WATERMARK: Forward the record without advancing watermark
> Additionally, add metrics to track the number of records affected by null
> rowtime.
> h1. Motivation
> 1. Improve fault tolerance for data quality issues
> 2. Align with existing patterns like `json.ignore-parse-errors`
> 3. Provide observability through metrics
> 4. Maintain backward compatibility with default FAIL behavior
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
