[
https://issues.apache.org/jira/browse/FLINK-23917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17403783#comment-17403783
]
Ingo Bürk commented on FLINK-23917:
-----------------------------------
[~lzljs3620320] [~jark] I'll move the discussion from FLINK-23911 here since it
belongs to this issue.
My plan is now to extend SupportsReadingMetadata by a method which lets the
source decide whether metadata should be projected in case it doesn't support
projection pushdown. It is default-implemented to preserve current behavior.
I'm happy to further discuss if we should omit that part and just always do it
as well, of course, if you feel that would be better.
> Project metadata columns to apply to sources which do not implement
> projection pushdown
> ---------------------------------------------------------------------------------------
>
> Key: FLINK-23917
> URL: https://issues.apache.org/jira/browse/FLINK-23917
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Ingo Bürk
> Assignee: Ingo Bürk
> Priority: Major
>
> If a source implements SupportsReadingMetadata, but not
> SupportsProjectionPushDown, #applyReadableMetadata is always called with all
> metadata keys declared in the table's schema. If SupportsProjectionPushDown
> is implemented, this is filtered down to only the required metadata keys.
> This can be a problem since metadata could be expensive to calculate.
> The root cause for this is that the projection happens in
> PushProjectIntoTableSourceScanRule which doesn't run for sources which do not
> implement projection pushdown. We should add a new separate rule which
> projects required metadata keys for the source in this case.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
