Interest in adding the float16 logical type to the Parquet spec

2022-08-23 Thread Anja
Hello! Is there interest in having the float16 logical type standardised in the Parquet spec? I am proposing a PR for Arrow that will write float16 to Parquet as FixedSizeBinary:https://issues.apache.org/jira/browse/ARROW-17464 but for the sake of portability between data analysis tools, it would

[jira] [Created] (PARQUET-2175) Skip method skips levels and not rows for repeated fields

2022-08-23 Thread fatemah (Jira)
fatemah created PARQUET-2175: Summary: Skip method skips levels and not rows for repeated fields Key: PARQUET-2175 URL: https://issues.apache.org/jira/browse/PARQUET-2175 Project: Parquet Issue

[jira] [Commented] (PARQUET-1711) [parquet-protobuf] stack overflow when work with well known json type

2022-08-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583724#comment-17583724 ] ASF GitHub Bot commented on PARQUET-1711: - matthieun commented on code in PR #988: URL:

[GitHub] [parquet-mr] matthieun commented on a diff in pull request #988: PARQUET-1711: Break circular dependencies in proto definitions

2022-08-23 Thread GitBox
matthieun commented on code in PR #988: URL: https://github.com/apache/parquet-mr/pull/988#discussion_r952835674 ## parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java: ## @@ -79,12 +80,20 @@ public MessageType convert(Class protobufClass) { }

[GitHub] [parquet-mr] shangxinli merged pull request #986: Prevent IntelliJ from making unsolicited whitespace changes

2022-08-23 Thread GitBox
shangxinli merged PR #986: URL: https://github.com/apache/parquet-mr/pull/986 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (PARQUET-1711) [parquet-protobuf] stack overflow when work with well known json type

2022-08-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583686#comment-17583686 ] ASF GitHub Bot commented on PARQUET-1711: - shangxinli commented on code in PR #988: URL:

[jira] [Commented] (PARQUET-1711) [parquet-protobuf] stack overflow when work with well known json type

2022-08-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583688#comment-17583688 ] ASF GitHub Bot commented on PARQUET-1711: - shangxinli commented on code in PR #988: URL:

[GitHub] [parquet-mr] shangxinli commented on a diff in pull request #988: PARQUET-1711: Break circular dependencies in proto definitions

2022-08-23 Thread GitBox
shangxinli commented on code in PR #988: URL: https://github.com/apache/parquet-mr/pull/988#discussion_r952778132 ## parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java: ## @@ -79,12 +80,20 @@ public MessageType convert(Class protobufClass) { }

[GitHub] [parquet-mr] shangxinli commented on a diff in pull request #988: PARQUET-1711: Break circular dependencies in proto definitions

2022-08-23 Thread GitBox
shangxinli commented on code in PR #988: URL: https://github.com/apache/parquet-mr/pull/988#discussion_r952778132 ## parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java: ## @@ -79,12 +80,20 @@ public MessageType convert(Class protobufClass) { }

Re: Fail to read back written large parquet file

2022-08-23 Thread Chao Sun
Jozef, feel free to open a Parquet JIRA to give the issue more details. Ideally the writer should recover itself and produce the correct result, but I don't have enough context yet so not sure whether it's doable. On Mon, Aug 8, 2022 at 1:53 AM Jozef Vilcek wrote: > > Found it. Problem in my

[jira] [Commented] (PARQUET-1711) [parquet-protobuf] stack overflow when work with well known json type

2022-08-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583672#comment-17583672 ] ASF GitHub Bot commented on PARQUET-1711: - shangxinli commented on code in PR #988: URL:

[GitHub] [parquet-mr] shangxinli commented on a diff in pull request #988: PARQUET-1711: Break circular dependencies in proto definitions

2022-08-23 Thread GitBox
shangxinli commented on code in PR #988: URL: https://github.com/apache/parquet-mr/pull/988#discussion_r952764513 ## parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java: ## @@ -79,12 +80,20 @@ public MessageType convert(Class protobufClass) { }