[
https://issues.apache.org/jira/browse/DRILL-6454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522841#comment-16522841
]
Bridget Bevens edited comment on DRILL-6454 at 6/28/18 12:21 AM:
-----------------------------------------------------------------
Hi [~vitalii],
Thank you for adding the doc notes to the description.
I'll add the following description to the MapR Drill release notes:
A new option, store.hive.maprdb_json.optimize_scan_with_native_reader, enables
Drill to use the native Drill reader to read Hive external tables that were
created from MapR-DB JSON tables. When you enable this option, Drill performs
faster reads of the data and applies filter pushdown optimizations.
I'll add the following info to the MapR MapR-DB format plugin doc page:
Starting in Drill 1.14 (MEP 6.0), Drill can use the native Drill reader to read
Hive external tables that were created from MapR-DB JSON tables. When using the
native reader, Drill performs faster reads of the data and can apply filter
pushdown optimizations. Use the SET command with the
store.hive.maprdb_json.optimize_scan_with_native_reader option to enable this
functionality:
SET `store.hive.maprdb_json.optimize_scan_with_native_reader` = true;
Regarding the store.hive.parquet.optimize_scan_with_native_reader: false
option, I've updated the following pages with the new option and included a
note on the options page about the old option being deprecated in Drill 1.15:
https://drill.apache.org/docs/querying-hive/
https://drill.apache.org/docs/configuration-options-introduction/
I'm going to set doc status to doc-complete, but please let me know if you see
any issues with the doc updates.
Thanks,
Bridget
was (Author: bbevens):
Hi [~vitalii],
Thank you for adding the doc notes to the description.
Can you please review the following description for the
store.hive.maprdb_json.optimize_scan_with_native_reader option?
Description:
When you enable the store.hive.maprdb_json.optimize_scan_with_native_reader
option, Drill can use the native Drill reader to read [Hive external tables
that were created from MapR-DB JSON
tables|https://maprdocs.mapr.com/home/Hive/ConnectingToMapR-DB.html]. The
native Drill reader enables Drill to perform faster reads of data and apply
filter pushdown optimizations.
Thanks,
Bridget
> Native MapR DB plugin support for Hive MapR-DB json table
> ---------------------------------------------------------
>
> Key: DRILL-6454
> URL: https://issues.apache.org/jira/browse/DRILL-6454
> Project: Apache Drill
> Issue Type: New Feature
> Components: Storage - Hive, Storage - MapRDB
> Affects Versions: 1.13.0
> Reporter: Vitalii Diravka
> Assignee: Vitalii Diravka
> Priority: Major
> Labels: doc-complete, ready-to-commit
> Fix For: 1.14.0
>
>
> Hive can create and query MapR-DB tables via maprdb-json-handler:
> [https://maprdocs.mapr.com/home/Hive/ConnectingToMapR-DB.html]
> The aim of this Jira to implement Drill native reader for Hive MapR-DB tables
> (similar to parquet).
> Design proposal is:
> - to use JsonTableGroupScan instead of HiveScan;
> - to add storage planning rule to convert HiveScan to MapRDBGroupScan;
> - to add system/session option to enable using of this native reader;
> - native reader can be used only for Drill build with mapr profile (there is
> no reason to leverage it for default profile);
>
> *For documentation:*
> two new options were added:
> store.hive.parquet.optimize_scan_with_native_reader: false,
> store.hive.maprdb_json.optimize_scan_with_native_reader: false,
> store.hive.parquet.optimize_scan_with_native_reader is new option used
> instead of store.hive.optimize_scan_with_native_readers. The latter is
> deprecated and will be removed in 1.15.
> (https://issues.apache.org/jira/browse/DRILL-6527).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
