[
https://issues.apache.org/jira/browse/CALCITE-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Wang updated CALCITE-3582:
------------------------------
Description:
There is a use case where users want to mix Beam programming model with Beam
SQL together to process a dataset. The following is an example of the use case:
dataset.apply(something user defined)
.apply(SELECT row_a, row_b)
.apply(something user defined)
As you can see, after the SQL statement is applied, the data structure should
be preserved for further processing(e.g. users expect to see there are two
rows, than flattened fields of rows)
So far as RelStructuredTypeFlattener is enabled by default in PlannerImpl,
Row/Struct will be "unnest" such that the output schema of the second SQL step
will change even users want to SELECT a whole row and works on that row.
Make calling RelStructuredTypeFlattener configurable in PlannerImpl to allow
disabling it when not needed can be a solution.
was:
There is a use case where users want to mix Beam programming model with Beam
SQL together to process a dataset. The following is an example of the use case:
dataset.apply(something user defined)
.apply(SELECT ...)
.apply(something user defined)
As you can see, after the SQL statement is applied, the data structure should
be preserved for further processing.
So far as RelStructuredTypeFlattener is enabled by default in PlannerImpl,
Row/Struct will be "unnest" such that the output schema of the second SQL step
will change even users want to SELECT a whole row and works on that row.
Make calling RelStructuredTypeFlattener configurable in PlannerImpl to allow
disabling it when not needed can be a solution.
> Make RelStructuredTypeFlattener configurable in PlannerImpl
> -----------------------------------------------------------
>
> Key: CALCITE-3582
> URL: https://issues.apache.org/jira/browse/CALCITE-3582
> Project: Calcite
> Issue Type: Task
> Reporter: Rui Wang
> Assignee: Rui Wang
> Priority: Major
>
> There is a use case where users want to mix Beam programming model with Beam
> SQL together to process a dataset. The following is an example of the use
> case:
> dataset.apply(something user defined)
> .apply(SELECT row_a, row_b)
> .apply(something user defined)
> As you can see, after the SQL statement is applied, the data structure should
> be preserved for further processing(e.g. users expect to see there are two
> rows, than flattened fields of rows)
> So far as RelStructuredTypeFlattener is enabled by default in PlannerImpl,
> Row/Struct will be "unnest" such that the output schema of the second SQL
> step will change even users want to SELECT a whole row and works on that row.
> Make calling RelStructuredTypeFlattener configurable in PlannerImpl to allow
> disabling it when not needed can be a solution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
