[
https://issues.apache.org/jira/browse/FLINK-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15722529#comment-15722529
]
ASF GitHub Bot commented on FLINK-3848:
---------------------------------------
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/2810
This is a good question @tonycox. First of all, we need to agree on the
semantics of the `ProjectableTableSource.setProjection()` method. IMO, the
table source must return a `DataSet` (or `DataStream`) with the fields in the
order requested by `setProjection()`. If the `InputFormat` (or
`SourceFunction`) is not able to control the order of fields, the table source
needs to add a `MapFunction` to fix the order. This would happen in
`BatchTableSource.getDataSet()` (or `StreamTableSource.getDataStream()` and
hence be opaque to Calcite and not involve a `LogicalCalc`.
From a performance point of view I agree with @wuchong that it would be
good to have correct records produced by the `RowCsvInputFormat`. However, I
would see this as a "nice to have" feature. If you do not want to include it,
it would be nice if you could open a JIRA to address this issue at some later
point in time.
> Add ProjectableTableSource interface and translation rule
> ---------------------------------------------------------
>
> Key: FLINK-3848
> URL: https://issues.apache.org/jira/browse/FLINK-3848
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
> Reporter: Fabian Hueske
> Assignee: Anton Solovev
>
> Add a {{ProjectableTableSource}} interface for {{TableSource}} implementation
> that support projection push-down.
> The interface could look as follows
> {code}
> def trait ProjectableTableSource {
> def setProjection(fields: Array[String]): Unit
> }
> {code}
> In addition we need Calcite rules to push a projection into a TableScan that
> refers to a {{ProjectableTableSource}}. We might need to tweak the cost model
> as well to push the optimizer in the right direction.
> Moreover, the {{CsvTableSource}} could be extended to implement
> {{ProjectableTableSource}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
