[
https://issues.apache.org/jira/browse/HUDI-9581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lin Liu updated HUDI-9581:
--------------------------
Description:
FG reader does not support point lookup by default. So after integration with
MDT, there is a behavior difference caused.
1. LogScanner allows on-demand read for Hfile data block only (not for Avro,
Parquet); in which case the keys are pushed down into native hfile reader that
filters records during the internal scan. FG reader uses full-scan by default,
which means hfile reader iterates all records, and does filtering outside of
the native hfile reader.
2. LogScanner enables point lookup for all partitoins but Files partition. FG
reader does not allow on-demand for all partitions.
Therefore, we should fix this behavior change.
was:
FG reader does not support point lookup by default. So after integration with
MDT, there is a behavior difference caused.
Therefore, we should fix this behavior change.
> Allow point lookup in MDT read
> ------------------------------
>
> Key: HUDI-9581
> URL: https://issues.apache.org/jira/browse/HUDI-9581
> Project: Apache Hudi
> Issue Type: New Feature
> Reporter: Lin Liu
> Priority: Major
> Fix For: 1.1.0
>
>
> FG reader does not support point lookup by default. So after integration with
> MDT, there is a behavior difference caused.
> 1. LogScanner allows on-demand read for Hfile data block only (not for Avro,
> Parquet); in which case the keys are pushed down into native hfile reader
> that filters records during the internal scan. FG reader uses full-scan by
> default, which means hfile reader iterates all records, and does filtering
> outside of the native hfile reader.
> 2. LogScanner enables point lookup for all partitoins but Files partition. FG
> reader does not allow on-demand for all partitions.
> Therefore, we should fix this behavior change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
