[
https://issues.apache.org/jira/browse/ARROW-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17661736#comment-17661736
]
Rok Mihevc commented on ARROW-4714:
-----------------------------------
This issue has been migrated to [issue
#15998|https://github.com/apache/arrow/issues/15998] on GitHub. Please see the
[migration documentation|https://github.com/apache/arrow/issues/14542] for
further details.
> [C++][Java] Providing JNI interface to Read ORC file via Arrow C++
> ------------------------------------------------------------------
>
> Key: ARROW-4714
> URL: https://issues.apache.org/jira/browse/ARROW-4714
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, Java
> Reporter: Yurui Zhou
> Assignee: Yurui Zhou
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.14.0
>
> Time Spent: 19h 50m
> Remaining Estimate: 0h
>
> Currently if we want to read data from ORC data into Arrow Record Batch in
> Java runtime, we needs to first use the ORC Java reader to load data into
> memory then convert it to Arrow RecordBatch.
> However, since ORC Java Reader only read orc data into on heap memory, while
> Arrow Record only support off heap memory on Java, memory copy in unavoidable
> in conversion process. In our internal benchmark, the conversion time can
> take up to 25% E2E latency when running tpch q1.
> To workaround this overhead, a method is to enable the Java runtime directly
> reading data from native ORC c++ reader, which will load data directly into
> off heap memory and only pointer manipulation and schema ser/de would be
> involved in the conversion process.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
