[
https://issues.apache.org/jira/browse/FLINK-5752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated FLINK-5752:
------------------------------------------
Description:
This is after the discussion to create NestedProjectableTableSource.
Currently we support nested schema for the non-relational type of DBs like
HBase.
But this does not allow push down projection. This JIRA is to implement that.
Once FLINK-5698 is implemented then we should be making use of the feature to
push down the projections for a nested table. So in case of HBase if we have
{f1:{a, b}, f2:{c, d}} as the nested structure then if we have a scan query
that needs to select f2.c - then we should be specifically able to project only
that column 'c' under 'f2'. FLINK-5698 plans to add a new API for such
projections and HBaseTableSource should make use of that API to do the
projection.
[~fhueske], [~tonycox], [~jark]
was:
This is after the discussion to create NestedProjectableTableSource.
Currently we support nested schema for the non-relational type of DBs like
HBase.
But this does not allow push down projection. This JIRA is to implement that.
I just did a POC on the existing PR https://github.com/apache/flink/pull/3149.
The idea was to allow the family#quallifier to be specified in the query
directly and parse the family and qualifier info and directly push down for
projections.
I tried using fam$qual (with $) as separator but that had SQL parse error.
Hence went with '_' as the separator and it worked.
So now in order to maintain the order in which the projection has to be done we
need to have a LinkedHashMap which has the qualifier to typeinfo mapping. This
is because with f1_q1, f1_q2 as the schema, we could always project f1_q2 and
the selection could be f1_q1. In that case the order of how the projection is
done is very important so that there is no type mismatch.
[~fhueske], [~tonycox], [~jark]
> Support push down projections for HBaseTableSource
> --------------------------------------------------
>
> Key: FLINK-5752
> URL: https://issues.apache.org/jira/browse/FLINK-5752
> Project: Flink
> Issue Type: Sub-task
> Components: Table API & SQL
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
>
> This is after the discussion to create NestedProjectableTableSource.
> Currently we support nested schema for the non-relational type of DBs like
> HBase.
> But this does not allow push down projection. This JIRA is to implement that.
> Once FLINK-5698 is implemented then we should be making use of the feature to
> push down the projections for a nested table. So in case of HBase if we have
> {f1:{a, b}, f2:{c, d}} as the nested structure then if we have a scan query
> that needs to select f2.c - then we should be specifically able to project
> only that column 'c' under 'f2'. FLINK-5698 plans to add a new API for such
> projections and HBaseTableSource should make use of that API to do the
> projection.
> [~fhueske], [~tonycox], [~jark]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
