[
https://issues.apache.org/jira/browse/IMPALA-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607675#comment-16607675
]
Tim Armstrong commented on IMPALA-7547:
---------------------------------------
I think this is kind of similar to IMPALA-4986 - it requires a similar
transformation. Would have to be careful parsing the Parquet spec - it might
technically allow values in the dictionary that aren't present in the data.
> For distinct queries use dictionary encoded page instead of reading all data
> ----------------------------------------------------------------------------
>
> Key: IMPALA-7547
> URL: https://issues.apache.org/jira/browse/IMPALA-7547
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Peter Ebert
> Priority: Major
>
> When dictionary encoding is in use the lookup table should contain a distinct
> list of all values in the data, can skip reading the values and just read the
> header to get distinct values.
>
> Realize this would be a big change to the read/scanner threads but could
> greatly speed up distinct queries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
