[
https://issues.apache.org/jira/browse/FLINK-39421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timo Walther closed FLINK-39421.
--------------------------------
Fix Version/s: 2.4.0
Resolution: Fixed
Fixed in master: 92e67cfc79f79b76e5fa67dc7596554175b57e7e
> Metadata filter push-down for table sources
> -------------------------------------------
>
> Key: FLINK-39421
> URL: https://issues.apache.org/jira/browse/FLINK-39421
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Jim Hughes
> Assignee: Jim Hughes
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
>
> Predicates on metadata columns (e.g., Kafka offset, timestamp, partition) are
> excluded from the existing SupportsFilterPushDown mechanism.
> FilterPushDownSpec serializes RexInputRef indices relative to the scan's
> produced row type, but metadata columns are appended after physical columns.
> When ProjectPushDownSpec narrows the row type during compiled plan
> restoration, metadata indices become out-of-bounds, causing
> ArrayIndexOutOfBoundsException.
> This ticket introduces a dedicated metadata filter push-down path:
> - New supportsMetadataFilterPushDown() and applyMetadataFilters() default
> methods on SupportsReadingMetadata (opt-in; existing sources unaffected)
> - New MetadataFilterPushDownSpec that snapshots the row type at optimization
> time, solving the index mismatch
> - Extension to PushFilterIntoTableSourceScanRule to classify predicates as
> physical, metadata, or mixed
> See FLIP-574
> (https://cwiki.apache.org/confluence/display/FLINK/FLIP-574+Metadata+Filter+Push-Down+for+Table+Sources)
> for full design.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
