[
https://issues.apache.org/jira/browse/DRILL-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16800618#comment-16800618
]
ASF GitHub Bot commented on DRILL-7011:
---------------------------------------
arina-ielchiieva commented on issue #1711: DRILL-7011: Support schema in scan
framework
URL: https://github.com/apache/drill/pull/1711#issuecomment-476166889
@paul-rogers
Actually when I was presenting the schema provisioning design, there were a
proposal to add schema property `drill.is_full_schema`. By default it’s
`false`, thus we assume that schema is partial.
If user wants to indicate that schema is strict and to ignore all columns
except of those indicated in schema, he needs to create schema the following
way:
`create schema (col int) for table dfs.tmp.t. properties
('drill.is_full_schema' = 'true')`
Since most of the `default` property problems are related to star queries,
we can state the following:
1. For queries with defined list of columns (aka projection queries: `select
id, name from t`), we apply schema consistently.
2. For star queries and when schema property `drill.is_full_schema` is set
to `false`, we might get inconsistent results with default values but it's ok
since we discover schema on the read.
3. For star queries and when schema property `drill.is_full_schema` is set
to `true`, we project only those columns indicated in schema.
What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Allow hybrid model in the Row set-based scan framework
> ------------------------------------------------------
>
> Key: DRILL-7011
> URL: https://issues.apache.org/jira/browse/DRILL-7011
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.15.0
> Reporter: Arina Ielchiieva
> Assignee: Paul Rogers
> Priority: Major
> Fix For: 1.16.0
>
>
> As part of schema provisioning project we want to allow hybrid model for Row
> set-based scan framework, namely to allow to pass custom schema metadata
> which can be partial.
> Currently schema provisioning has SchemaContainer class that contains the
> following information (can be obtained from metastore, schema file, table
> function):
> 1. schema represented by org.apache.drill.exec.record.metadata.TupleMetadata
> 2. properties represented by Map<String, String>, can contain information if
> schema is strict or partial (default is partial) etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
