[jira] [Updated] (PARQUET-2136) File writer construction with encryptor

[ https://issues.apache.org/jira/browse/PARQUET-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2136: -- Fix Version/s: 1.12.3 > File writer construction with encryptor >

[jira] [Updated] (PARQUET-2144) Fix ColumnIndexBuilder for notIn predicate

[ https://issues.apache.org/jira/browse/PARQUET-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2144: -- Fix Version/s: 1.12.3 > Fix ColumnIndexBuilder for notIn predicate >

[jira] [Updated] (PARQUET-2127) Security risk in latest parquet-jackson-1.12.2.jar

[ https://issues.apache.org/jira/browse/PARQUET-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2127: -- Fix Version/s: 1.12.3 > Security risk in latest parquet-jackson-1.12.2.jar >

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539121#comment-17539121 ] ASF GitHub Bot commented on PARQUET-2149: - parthchandra commented on PR #968: URL:

[GitHub] [parquet-mr] parthchandra commented on pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

parthchandra commented on PR #968: URL: https://github.com/apache/parquet-mr/pull/968#issuecomment-1130698923 @theosib-amazon I applied my PR on top of your PR, ran thru some tests using Spark, and hit no issues. (All unit tests passed as well). -- This is an automated message from the

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539119#comment-17539119 ] ASF GitHub Bot commented on PARQUET-2149: - parthchandra commented on code in PR #968: URL:

[GitHub] [parquet-mr] parthchandra commented on a diff in pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

parthchandra commented on code in PR #968: URL: https://github.com/apache/parquet-mr/pull/968#discussion_r876197602 ## parquet-column/src/main/java/org/apache/parquet/column/page/PageReader.java: ## @@ -37,4 +39,9 @@ public interface PageReader { * @return the next page in

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538991#comment-17538991 ] ASF GitHub Bot commented on PARQUET-2149: - parthchandra commented on PR #968: URL:

[GitHub] [parquet-mr] parthchandra commented on pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

parthchandra commented on PR #968: URL: https://github.com/apache/parquet-mr/pull/968#issuecomment-1130327546 > @parthchandra One thing that confuses me a bit is that these buffers have only ByteBuffer inside them. There's no actual I/O, so it's not possible to block. Do you have

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538983#comment-17538983 ] ASF GitHub Bot commented on PARQUET-2149: - kbendick commented on code in PR #968: URL:

[GitHub] [parquet-mr] kbendick commented on a diff in pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

kbendick commented on code in PR #968: URL: https://github.com/apache/parquet-mr/pull/968#discussion_r876172260 ## parquet-common/src/main/java/org/apache/parquet/bytes/AsyncMultiBufferInputStream.java: ## @@ -0,0 +1,173 @@ +/* + * Licensed to the Apache Software Foundation

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538979#comment-17538979 ] ASF GitHub Bot commented on PARQUET-2149: - kbendick commented on code in PR #968: URL:

[GitHub] [parquet-mr] kbendick commented on a diff in pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

kbendick commented on code in PR #968: URL: https://github.com/apache/parquet-mr/pull/968#discussion_r876165378 ## parquet-column/src/main/java/org/apache/parquet/column/page/PageReader.java: ## @@ -37,4 +39,9 @@ public interface PageReader { * @return the next page in that

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538969#comment-17538969 ] ASF GitHub Bot commented on PARQUET-2149: - theosib-amazon commented on PR #968: URL:

[GitHub] [parquet-mr] theosib-amazon commented on pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

theosib-amazon commented on PR #968: URL: https://github.com/apache/parquet-mr/pull/968#issuecomment-1130275378 @parthchandra One thing that confuses me a bit is that these buffers have only ByteBuffer inside them. There's no actual I/O, so it's not possible to block. Do you have

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538967#comment-17538967 ] ASF GitHub Bot commented on PARQUET-2149: - parthchandra commented on PR #968: URL:

[GitHub] [parquet-mr] parthchandra commented on pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

parthchandra commented on PR #968: URL: https://github.com/apache/parquet-mr/pull/968#issuecomment-1130270383 > @parthchandra Would you mind having a look at my I/O performance optimization plan for ParquetMR? I think we should coordinate, since we have some ideas that might overlap

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538945#comment-17538945 ] ASF GitHub Bot commented on PARQUET-2149: - parthchandra commented on PR #968: URL:

[GitHub] [parquet-mr] parthchandra commented on pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

parthchandra commented on PR #968: URL: https://github.com/apache/parquet-mr/pull/968#issuecomment-1130229884 > Great effort! WIll have a look after the build succeed. @shangxinli I have no idea how to get the failed CI to pass. These failures appear to be in unrelated areas caused by

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538934#comment-17538934 ] ASF GitHub Bot commented on PARQUET-2149: - dbtsai commented on PR #968: URL:

[GitHub] [parquet-mr] dbtsai commented on pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

dbtsai commented on PR #968: URL: https://github.com/apache/parquet-mr/pull/968#issuecomment-1130214186 cc @rdblue @gszadovszky @ggershinsky -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (PARQUET-2148) Enable uniform decryption with plaintext footer

[ https://issues.apache.org/jira/browse/PARQUET-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538925#comment-17538925 ] ASF GitHub Bot commented on PARQUET-2148: - shangxinli merged PR #969: URL:

[GitHub] [parquet-mr] shangxinli merged pull request #969: PARQUET-2148: Enable uniform decryption with plaintext footer

shangxinli merged PR #969: URL: https://github.com/apache/parquet-mr/pull/969 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538912#comment-17538912 ] ASF GitHub Bot commented on PARQUET-2149: - theosib-amazon commented on PR #968: URL:

[GitHub] [parquet-mr] theosib-amazon commented on pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

theosib-amazon commented on PR #968: URL: https://github.com/apache/parquet-mr/pull/968#issuecomment-1130176799 @parthchandra Would you mind having a look at my I/O performance optimization plan for ParquetMR? I think we should coordinate, since we have some ideas that might overlap what

[jira] [Commented] (PARQUET-2148) Enable uniform decryption with plaintext footer

[ https://issues.apache.org/jira/browse/PARQUET-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538906#comment-17538906 ] ASF GitHub Bot commented on PARQUET-2148: - ggershinsky commented on code in PR #969: URL:

[GitHub] [parquet-mr] ggershinsky commented on a diff in pull request #969: PARQUET-2148: Enable uniform decryption with plaintext footer

ggershinsky commented on code in PR #969: URL: https://github.com/apache/parquet-mr/pull/969#discussion_r876028764 ## parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java: ## @@ -1556,25 +1558,32 @@ public ParquetMetadata

[GitHub] [parquet-mr] ggershinsky commented on a diff in pull request #969: PARQUET-2148: Enable uniform decryption with plaintext footer

ggershinsky commented on code in PR #969: URL: https://github.com/apache/parquet-mr/pull/969#discussion_r876022965 ## parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java: ## @@ -1556,25 +1558,32 @@ public ParquetMetadata

[jira] [Commented] (PARQUET-2148) Enable uniform decryption with plaintext footer

[ https://issues.apache.org/jira/browse/PARQUET-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538900#comment-17538900 ] ASF GitHub Bot commented on PARQUET-2148: - ggershinsky commented on code in PR #969: URL:

[GitHub] [parquet-mr] shangxinli commented on pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

shangxinli commented on PR #968: URL: https://github.com/apache/parquet-mr/pull/968#issuecomment-1130130378 Great effort! WIll have a look after the build succeed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [parquet-mr] shangxinli commented on a diff in pull request #969: PARQUET-2148: Enable uniform decryption with plaintext footer

shangxinli commented on code in PR #969: URL: https://github.com/apache/parquet-mr/pull/969#discussion_r876012014 ## parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java: ## @@ -1556,25 +1558,32 @@ public ParquetMetadata

[jira] [Commented] (PARQUET-2148) Enable uniform decryption with plaintext footer

[ https://issues.apache.org/jira/browse/PARQUET-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538893#comment-17538893 ] ASF GitHub Bot commented on PARQUET-2148: - shangxinli commented on code in PR #969: URL:

[jira] [Commented] (PARQUET-2148) Enable uniform decryption with plaintext footer

[ https://issues.apache.org/jira/browse/PARQUET-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538757#comment-17538757 ] ASF GitHub Bot commented on PARQUET-2148: - ggershinsky opened a new pull request, #969: URL:

[GitHub] [parquet-mr] ggershinsky opened a new pull request, #969: PARQUET-2148: Enable uniform decryption with plaintext footer

ggershinsky opened a new pull request, #969: URL: https://github.com/apache/parquet-mr/pull/969 Currently, uniform decryption is not enabled in the plaintext footer mode - for no good reason. Column metadata is available, we just need to decrypt and use it. -- This is an automated