[GitHub] [parquet-mr] LuciferYang commented on pull request #972: PARQUET-2154: `ParquetFileReader` should close its input stream when `filterRowGroups` throw Exception in constructor

LuciferYang commented on PR #972: URL: https://github.com/apache/parquet-mr/pull/972#issuecomment-1151743122 @ggershinsky yeah～ all passed now～ thanks～ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Commented] (PARQUET-2150) parquet-protobuf to compile on mac M1

[ https://issues.apache.org/jira/browse/PARQUET-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552481#comment-17552481 ] ASF GitHub Bot commented on PARQUET-2150: - sunchao commented on PR #970: URL:

[GitHub] [parquet-mr] sunchao commented on pull request #970: PARQUET-2150: parquet-protobuf to compile on Mac M1

sunchao commented on PR #970: URL: https://github.com/apache/parquet-mr/pull/970#issuecomment-1151738855 Oops @steveloughran just found out this PR. I opened #973 for the same purpose -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Created] (PARQUET-2155) Upgrade protobuf version to 3.20.1

Chao Sun created PARQUET-2155: - Summary: Upgrade protobuf version to 3.20.1 Key: PARQUET-2155 URL: https://issues.apache.org/jira/browse/PARQUET-2155 Project: Parquet Issue Type: Improvement

[GitHub] [parquet-mr] sunchao opened a new pull request, #973: Upgrade protobuf version to 3.20.1

sunchao opened a new pull request, #973: URL: https://github.com/apache/parquet-mr/pull/973 Currently the repo can't be compiled with M1 Mac: ``` com.google.protobuf:protoc:exe:osx-aarch_64:3.16.1 was not found in https://jitpack.io/ during a previous attempt. ``` since the

[GitHub] [parquet-mr] jinyius commented on pull request #445: Proposed change in AvroSchema to handle circular references.

jinyius commented on PR #445: URL: https://github.com/apache/parquet-mr/pull/445#issuecomment-1151527049 any updates here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[jira] [Comment Edited] (PARQUET-1711) [parquet-protobuf] stack overflow when work with well known json type

[ https://issues.apache.org/jira/browse/PARQUET-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552396#comment-17552396 ] J Y edited comment on PARQUET-1711 at 6/9/22 7:16 PM: -- i'd be ok of that approach:

[jira] [Commented] (PARQUET-1711) [parquet-protobuf] stack overflow when work with well known json type

[ https://issues.apache.org/jira/browse/PARQUET-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552396#comment-17552396 ] J Y commented on PARQUET-1711: -- i'd be ok of that approach: a proto option annotation to limit the

[jira] [Updated] (PARQUET-1020) Add support for Dynamic Messages in parquet-protobuf

[ https://issues.apache.org/jira/browse/PARQUET-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Blake Niskode-Dossett updated PARQUET-1020: - Component/s: parquet-protobuf > Add support for Dynamic

[jira] [Commented] (PARQUET-1020) Add support for Dynamic Messages in parquet-protobuf

[ https://issues.apache.org/jira/browse/PARQUET-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552383#comment-17552383 ] ASF GitHub Bot commented on PARQUET-1020: - dossett commented on PR #963: URL:

[jira] [Commented] (PARQUET-1020) Add support for Dynamic Messages in parquet-protobuf

[ https://issues.apache.org/jira/browse/PARQUET-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552382#comment-17552382 ] ASF GitHub Bot commented on PARQUET-1020: - dossett commented on PR #963: URL:

[GitHub] [parquet-mr] dossett commented on pull request #963: PARQUET-1020 Add DynamicMessage writing support

dossett commented on PR #963: URL: https://github.com/apache/parquet-mr/pull/963#issuecomment-1151481732 +1 (non-binding) for this change. `DynamicMessage` is quite useful in protobuf and support here would be great, I ran into a need for it just today. cc @belugabehr in case they have

[GitHub] [parquet-mr] dossett commented on pull request #963: PARQUET-1020 Add DynamicMessage writing support

dossett commented on PR #963: URL: https://github.com/apache/parquet-mr/pull/963#issuecomment-1151481731 +1 (non-binding) for this change. `DynamicMessage` is quite useful in protobuf and support here would be great, I ran into a need for it just today. cc @belugabehr in case they have

[jira] [Commented] (PARQUET-2126) Thread safety bug in CodecFactory

[ https://issues.apache.org/jira/browse/PARQUET-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552371#comment-17552371 ] ASF GitHub Bot commented on PARQUET-2126: - dossett commented on PR #959: URL:

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552370#comment-17552370 ] ASF GitHub Bot commented on PARQUET-2149: - parthchandra commented on PR #968: URL:

[GitHub] [parquet-mr] dossett commented on pull request #959: PARQUET-2126: Make cached (de)compressors thread-safe

dossett commented on PR #959: URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1151467318 @shangxinli I do not feel strongly about it. I think historical context is better kept in JIRAs and PR discussion than in code comments, but that is just a style choice if there's no

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552350#comment-17552350 ] ASF GitHub Bot commented on PARQUET-2149: - steveloughran commented on PR #968: URL:

[GitHub] [parquet-mr] steveloughran commented on pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

steveloughran commented on PR #968: URL: https://github.com/apache/parquet-mr/pull/968#issuecomment-1151417126 I've started work on a fs-api-shim library, with the goal of "apps compile against hadoop 3.2.0 can get access to the 3.3 and 3.4 APIs when available either with transparent

[jira] [Commented] (PARQUET-2149) Implement async IO for Parquet file reader

[ https://issues.apache.org/jira/browse/PARQUET-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552331#comment-17552331 ] ASF GitHub Bot commented on PARQUET-2149: - steveloughran commented on code in PR #968: URL:

[GitHub] [parquet-mr] steveloughran commented on a diff in pull request #968: PARQUET-2149: Async IO implementation for ParquetFileReader

steveloughran commented on code in PR #968: URL: https://github.com/apache/parquet-mr/pull/968#discussion_r893760667 ## parquet-common/src/main/java/org/apache/parquet/bytes/SequenceByteBufferInputStream.java: ## @@ -0,0 +1,269 @@ +/* + * Licensed to the Apache Software

[jira] [Commented] (PARQUET-2154) ParquetFileReader should close its input stream when `filterRowGroups` throw Exception in constructor

[ https://issues.apache.org/jira/browse/PARQUET-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552305#comment-17552305 ] ASF GitHub Bot commented on PARQUET-2154: - LuciferYang opened a new pull request, #972: URL:

[jira] [Commented] (PARQUET-2154) ParquetFileReader should close its input stream when `filterRowGroups` throw Exception in constructor

[ https://issues.apache.org/jira/browse/PARQUET-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552306#comment-17552306 ] ASF GitHub Bot commented on PARQUET-2154: - LuciferYang commented on PR #972: URL:

[jira] [Commented] (PARQUET-2154) ParquetFileReader should close its input stream when `filterRowGroups` throw Exception in constructor

[ https://issues.apache.org/jira/browse/PARQUET-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552304#comment-17552304 ] ASF GitHub Bot commented on PARQUET-2154: - LuciferYang closed pull request #972: PARQUET-2154:

[GitHub] [parquet-mr] LuciferYang commented on pull request #972: PARQUET-2154: `ParquetFileReader` should close its input stream when `filterRowGroups` throw Exception in constructor

LuciferYang commented on PR #972: URL: https://github.com/apache/parquet-mr/pull/972#issuecomment-1151345710 > can you re-run the CI? (eg via re-opening the PR) OK -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [parquet-mr] LuciferYang closed pull request #972: PARQUET-2154: `ParquetFileReader` should close its input stream when `filterRowGroups` throw Exception in constructor

LuciferYang closed pull request #972: PARQUET-2154: `ParquetFileReader` should close its input stream when `filterRowGroups` throw Exception in constructor URL: https://github.com/apache/parquet-mr/pull/972 -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (PARQUET-2154) ParquetFileReader should close its input stream when `filterRowGroups` throw Exception in constructor

[ https://issues.apache.org/jira/browse/PARQUET-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552303#comment-17552303 ] ASF GitHub Bot commented on PARQUET-2154: - ggershinsky commented on PR #972: URL:

[GitHub] [parquet-mr] ggershinsky commented on pull request #972: PARQUET-2154: `ParquetFileReader` should close its input stream when `filterRowGroups` throw Exception in constructor

ggershinsky commented on PR #972: URL: https://github.com/apache/parquet-mr/pull/972#issuecomment-1151344885 can you re-run the CI? (eg via re-opening the PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Commented] (PARQUET-2126) Thread safety bug in CodecFactory

[ https://issues.apache.org/jira/browse/PARQUET-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552295#comment-17552295 ] ASF GitHub Bot commented on PARQUET-2126: - shangxinli commented on PR #959: URL:

[GitHub] [parquet-mr] shangxinli commented on pull request #959: PARQUET-2126: Make cached (de)compressors thread-safe

shangxinli commented on PR #959: URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1151340364 Thanks for addressing the feedback! What I meant was that ideally when 'Threads terminate' happens, it should clean up the compressor/decompressor immediately. I understand we won't

[jira] [Commented] (PARQUET-2154) ParquetFileReader should close its input stream when `filterRowGroups` throw Exception in constructor

[ https://issues.apache.org/jira/browse/PARQUET-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552286#comment-17552286 ] ASF GitHub Bot commented on PARQUET-2154: - LuciferYang commented on PR #972: URL:

[jira] [Commented] (PARQUET-2134) Incorrect type checking in HadoopStreams.wrap

[ https://issues.apache.org/jira/browse/PARQUET-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552281#comment-17552281 ] ASF GitHub Bot commented on PARQUET-2134: - shangxinli commented on PR #951: URL:

[GitHub] [parquet-mr] shangxinli commented on pull request #951: PARQUET-2134: Fix type checking in HadoopStreams.wrap

shangxinli commented on PR #951: URL: https://github.com/apache/parquet-mr/pull/951#issuecomment-1151315837 > I've taken this PR and added the changes I was suggesting, plus tests. see #971. If you take that extra commit and merge it in here, it should complete this PR @7c00 Are you