[
https://issues.apache.org/jira/browse/PARQUET-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17111782#comment-17111782
]
ASF GitHub Bot commented on PARQUET-1229:
-----------------------------------------
ggershinsky commented on a change in pull request #776:
URL: https://github.com/apache/parquet-mr/pull/776#discussion_r427745576
##########
File path:
parquet-column/src/main/java/org/apache/parquet/internal/column/columnindex/OffsetIndex.java
##########
@@ -49,6 +49,13 @@
* @return the index of the first row in the page
*/
public long getFirstRowIndex(int pageIndex);
+
+ /**
+ * @param pageIndex
+ * the index of the page
+ * @return the original ordinal of the page in the column chunk
+ */
+ public short getPageOrdinal(int pageIndex);
Review comment:
The background discussion is here,
https://github.com/apache/parquet-mr/pull/776#discussion_r427743861
In the case of pages, encryption becomes an order (or two orders) of
magnitude slower if the pages are small. Basically, the hardware acceleration
does not kick in with small pages (and there are additional problems). This is
another reason not to allow more than 32K pages in a chunk.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> parquet-mr code changes for encryption support
> ----------------------------------------------
>
> Key: PARQUET-1229
> URL: https://issues.apache.org/jira/browse/PARQUET-1229
> Project: Parquet
> Issue Type: Sub-task
> Components: parquet-mr
> Reporter: Gidon Gershinsky
> Assignee: Gidon Gershinsky
> Priority: Major
> Labels: pull-request-available
>
> Addition of encryption/decryption support to the existing Parquet classes and
> APIs
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
