[jira] [Commented] (PARQUET-1647) [Java] support for Arrow's float16

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792698#comment-17792698
 ] 

ASF GitHub Bot commented on PARQUET-1647:
-

wgtmac commented on PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#issuecomment-1838002383

   BTW, it would be good to add an interoperability test to read parquet files 
from here: 
https://github.com/apache/parquet-testing/commit/da467dac2f095b979af37bcf40fa0d1dee5ff652.
 You may want to take a look at this example: 
https://github.com/apache/parquet-mr/blob/44b56225be6fe7b74667f4f2430326ef1f076cc5/parquet-hadoop/src/test/java/org/apache/parquet/hadoop/codec/TestInteropReadLz4RawCodec.java#L40
 




> [Java] support for Arrow's float16
> --
>
> Key: PARQUET-1647
> URL: https://issues.apache.org/jira/browse/PARQUET-1647
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-format, parquet-thrift
>Reporter: The Alchemist
>Priority: Minor
>
> h2. DESCRIPTION
>  
> I'm wondering if there's any interest in supporting Arrow's {{float16}} type 
> in Parquet.
> There seem to be one or two {{float16}} / {{halffloat}} tickets here (e.g., 
> PARQUET-1403) but nothing that speaks to adding half-float support to Parquet 
> in-general.
>  
> h2. PLANS
> I'm able to spend some time on this, if someone points me  in the right 
> direction.
>  
>  # Add the {{HALFFLOAT}} or {{FLOAT16}} enum (any preferred naming 
> convention?) to 
> [https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L32]
>  # Add {{HALFFLOAT}} to {{org.apache.parquet.schema.PrimitiveType}}
>  # Add {{HALFFLOAT}} support to 
> {{org.apache.parquet.arrow.schema.SchemaConverter}}
>  # Add encoding for new type at {{org.apache.parquet.column.Encoding}}
>  # ??
> If anyone has any interest in this, pointers, or comments, they would be 
> greatly appreciated!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-1647: [Java][Parquet] Implement FLOAT16 logical type [parquet-mr]

2023-12-03 Thread via GitHub


wgtmac commented on PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#issuecomment-1838002383

   BTW, it would be good to add an interoperability test to read parquet files 
from here: 
https://github.com/apache/parquet-testing/commit/da467dac2f095b979af37bcf40fa0d1dee5ff652.
 You may want to take a look at this example: 
https://github.com/apache/parquet-mr/blob/44b56225be6fe7b74667f4f2430326ef1f076cc5/parquet-hadoop/src/test/java/org/apache/parquet/hadoop/codec/TestInteropReadLz4RawCodec.java#L40
 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (PARQUET-2395) Prefer `singletonList` over `asList`

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792695#comment-17792695
 ] 

ASF GitHub Bot commented on PARQUET-2395:
-

wgtmac commented on PR #1218:
URL: https://github.com/apache/parquet-mr/pull/1218#issuecomment-1837995639

   Thanks for the explanation! 




> Prefer `singletonList` over `asList`
> 
>
> Key: PARQUET-2395
> URL: https://issues.apache.org/jira/browse/PARQUET-2395
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-2395: Prefer `singletonList` over `asList` [parquet-mr]

2023-12-03 Thread via GitHub


wgtmac commented on PR #1218:
URL: https://github.com/apache/parquet-mr/pull/1218#issuecomment-1837995639

   Thanks for the explanation! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (PARQUET-2395) Prefer `singletonList` over `asList`

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792693#comment-17792693
 ] 

ASF GitHub Bot commented on PARQUET-2395:
-

Fokko merged PR #1218:
URL: https://github.com/apache/parquet-mr/pull/1218




> Prefer `singletonList` over `asList`
> 
>
> Key: PARQUET-2395
> URL: https://issues.apache.org/jira/browse/PARQUET-2395
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (PARQUET-2395) Prefer `singletonList` over `asList`

2023-12-03 Thread Fokko Driesprong (Jira)


 [ 
https://issues.apache.org/jira/browse/PARQUET-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong resolved PARQUET-2395.
---
Resolution: Fixed

> Prefer `singletonList` over `asList`
> 
>
> Key: PARQUET-2395
> URL: https://issues.apache.org/jira/browse/PARQUET-2395
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-2395) Prefer `singletonList` over `asList`

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792694#comment-17792694
 ] 

ASF GitHub Bot commented on PARQUET-2395:
-

Fokko commented on PR #1218:
URL: https://github.com/apache/parquet-mr/pull/1218#issuecomment-1837993837

   Thanks for the review @wgtmac, @zhangjiashen and @amousavigourabi 🙌 




> Prefer `singletonList` over `asList`
> 
>
> Key: PARQUET-2395
> URL: https://issues.apache.org/jira/browse/PARQUET-2395
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-2395) Prefer `singletonList` over `asList`

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792692#comment-17792692
 ] 

ASF GitHub Bot commented on PARQUET-2395:
-

Fokko commented on PR #1218:
URL: https://github.com/apache/parquet-mr/pull/1218#issuecomment-1837993326

   @wgtmac Two things:
   
   - `singletonList` is completely immutable, while with `asList` you can still 
mutate the reference.
   - `singletonList` is not backed by an array, reducing the memory footprint.




> Prefer `singletonList` over `asList`
> 
>
> Key: PARQUET-2395
> URL: https://issues.apache.org/jira/browse/PARQUET-2395
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-2344) Bump to Thirft 0.19.0

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792691#comment-17792691
 ] 

ASF GitHub Bot commented on PARQUET-2344:
-

wgtmac commented on code in PR #1192:
URL: https://github.com/apache/parquet-mr/pull/1192#discussion_r141345


##
pom.xml:
##
@@ -619,6 +622,9 @@
 
   true
   true
+  
+
javax.annotation:javax.annotation-api:jar:1.3.2

Review Comment:
   Why do we need to ignore this?



##
parquet-thrift/src/main/java/org/apache/parquet/thrift/ThriftSchemaConverter.java:
##
@@ -225,14 +225,18 @@ private static ThriftField toThriftField(String name, 
Field field, ThriftField.R
 final Field listElemField = field.getListElemField();
 type = new ThriftType.ListType(toThriftField(listElemField.getName(), 
listElemField, requirement));
 break;
+  case UUID:
   case ENUM:
-Collection enumValues = field.getEnumValues();
-List values = new ArrayList();
-for (TEnum tEnum : enumValues) {
-  values.add(new EnumValue(tEnum.getValue(), tEnum.toString()));
+if (field.isEnum()) {

Review Comment:
   Why mixing UUID and ENUM in this case?



##
parquet-format-structures/pom.xml:
##
@@ -156,6 +156,11 @@
   libthrift
   ${format.thrift.version}
 
+
+  javax.annotation
+  javax.annotation-api

Review Comment:
   Where do we need this?



##
parquet-thrift/src/main/java/org/apache/parquet/thrift/struct/ThriftTypeID.java:
##
@@ -51,10 +51,15 @@ public enum ThriftTypeID {
   LIST (TType.LIST, true, ListType.class),
   ENUM (TType.ENUM, TType.I32, EnumType.class);
 
-  private static ThriftTypeID[] types = new ThriftTypeID[17];
+  private static final ThriftTypeID[] types;
   static {
+types = new ThriftTypeID[18];
 for (ThriftTypeID t : ThriftTypeID.values()) {
-  types[t.thriftType] = t;
+  if (t.thriftType == -1) {

Review Comment:
   It would be good to add the link to the comment as well. Or at least we need 
to explain why -1 is used here.





> Bump to Thirft 0.19.0
> -
>
> Key: PARQUET-2344
> URL: https://issues.apache.org/jira/browse/PARQUET-2344
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-format, parquet-mr
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: format-2.10.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-2395: Prefer `singletonList` over `asList` [parquet-mr]

2023-12-03 Thread via GitHub


Fokko commented on PR #1218:
URL: https://github.com/apache/parquet-mr/pull/1218#issuecomment-1837993837

   Thanks for the review @wgtmac, @zhangjiashen and @amousavigourabi 🙌 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



Re: [PR] PARQUET-2395: Prefer `singletonList` over `asList` [parquet-mr]

2023-12-03 Thread via GitHub


Fokko merged PR #1218:
URL: https://github.com/apache/parquet-mr/pull/1218


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



Re: [PR] PARQUET-2395: Prefer `singletonList` over `asList` [parquet-mr]

2023-12-03 Thread via GitHub


Fokko commented on PR #1218:
URL: https://github.com/apache/parquet-mr/pull/1218#issuecomment-1837993326

   @wgtmac Two things:
   
   - `singletonList` is completely immutable, while with `asList` you can still 
mutate the reference.
   - `singletonList` is not backed by an array, reducing the memory footprint.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



Re: [PR] PARQUET-2344: Bump to Thrift 0.19.0 [parquet-mr]

2023-12-03 Thread via GitHub


wgtmac commented on code in PR #1192:
URL: https://github.com/apache/parquet-mr/pull/1192#discussion_r141345


##
pom.xml:
##
@@ -619,6 +622,9 @@
 
   true
   true
+  
+
javax.annotation:javax.annotation-api:jar:1.3.2

Review Comment:
   Why do we need to ignore this?



##
parquet-thrift/src/main/java/org/apache/parquet/thrift/ThriftSchemaConverter.java:
##
@@ -225,14 +225,18 @@ private static ThriftField toThriftField(String name, 
Field field, ThriftField.R
 final Field listElemField = field.getListElemField();
 type = new ThriftType.ListType(toThriftField(listElemField.getName(), 
listElemField, requirement));
 break;
+  case UUID:
   case ENUM:
-Collection enumValues = field.getEnumValues();
-List values = new ArrayList();
-for (TEnum tEnum : enumValues) {
-  values.add(new EnumValue(tEnum.getValue(), tEnum.toString()));
+if (field.isEnum()) {

Review Comment:
   Why mixing UUID and ENUM in this case?



##
parquet-format-structures/pom.xml:
##
@@ -156,6 +156,11 @@
   libthrift
   ${format.thrift.version}
 
+
+  javax.annotation
+  javax.annotation-api

Review Comment:
   Where do we need this?



##
parquet-thrift/src/main/java/org/apache/parquet/thrift/struct/ThriftTypeID.java:
##
@@ -51,10 +51,15 @@ public enum ThriftTypeID {
   LIST (TType.LIST, true, ListType.class),
   ENUM (TType.ENUM, TType.I32, EnumType.class);
 
-  private static ThriftTypeID[] types = new ThriftTypeID[17];
+  private static final ThriftTypeID[] types;
   static {
+types = new ThriftTypeID[18];
 for (ThriftTypeID t : ThriftTypeID.values()) {
-  types[t.thriftType] = t;
+  if (t.thriftType == -1) {

Review Comment:
   It would be good to add the link to the comment as well. Or at least we need 
to explain why -1 is used here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Resolved] (PARQUET-2396) Refactor `ColumnIndexBuilder`

2023-12-03 Thread Fokko Driesprong (Jira)


 [ 
https://issues.apache.org/jira/browse/PARQUET-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong resolved PARQUET-2396.
---
Resolution: Fixed

> Refactor `ColumnIndexBuilder`
> -
>
> Key: PARQUET-2396
> URL: https://issues.apache.org/jira/browse/PARQUET-2396
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-2396) Refactor `ColumnIndexBuilder`

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792690#comment-17792690
 ] 

ASF GitHub Bot commented on PARQUET-2396:
-

Fokko merged PR #1219:
URL: https://github.com/apache/parquet-mr/pull/1219




> Refactor `ColumnIndexBuilder`
> -
>
> Key: PARQUET-2396
> URL: https://issues.apache.org/jira/browse/PARQUET-2396
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-2396: Refactor `ColumnIndexBuilder` [parquet-mr]

2023-12-03 Thread via GitHub


Fokko merged PR #1219:
URL: https://github.com/apache/parquet-mr/pull/1219


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (PARQUET-2391) Remove unnecessary unboxing

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792668#comment-17792668
 ] 

ASF GitHub Bot commented on PARQUET-2391:
-

Fokko commented on PR #1214:
URL: https://github.com/apache/parquet-mr/pull/1214#issuecomment-1837969587

   Thanks for the review @wgtmac & @amousavigourabi 🙌 




> Remove unnecessary unboxing
> ---
>
> Key: PARQUET-2391
> URL: https://issues.apache.org/jira/browse/PARQUET-2391
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-1647) [Java] support for Arrow's float16

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792667#comment-17792667
 ] 

ASF GitHub Bot commented on PARQUET-1647:
-

zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1413455235


##
pom.xml:
##
@@ -596,6 +597,9 @@
 
[Java] support for Arrow's float16
> --
>
> Key: PARQUET-1647
> URL: https://issues.apache.org/jira/browse/PARQUET-1647
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-format, parquet-thrift
>Reporter: The Alchemist
>Priority: Minor
>
> h2. DESCRIPTION
>  
> I'm wondering if there's any interest in supporting Arrow's {{float16}} type 
> in Parquet.
> There seem to be one or two {{float16}} / {{halffloat}} tickets here (e.g., 
> PARQUET-1403) but nothing that speaks to adding half-float support to Parquet 
> in-general.
>  
> h2. PLANS
> I'm able to spend some time on this, if someone points me  in the right 
> direction.
>  
>  # Add the {{HALFFLOAT}} or {{FLOAT16}} enum (any preferred naming 
> convention?) to 
> [https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L32]
>  # Add {{HALFFLOAT}} to {{org.apache.parquet.schema.PrimitiveType}}
>  # Add {{HALFFLOAT}} support to 
> {{org.apache.parquet.arrow.schema.SchemaConverter}}
>  # Add encoding for new type at {{org.apache.parquet.column.Encoding}}
>  # ??
> If anyone has any interest in this, pointers, or comments, they would be 
> greatly appreciated!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-2391: Remove unnecessary unboxing [parquet-mr]

2023-12-03 Thread via GitHub


Fokko commented on PR #1214:
URL: https://github.com/apache/parquet-mr/pull/1214#issuecomment-1837969587

   Thanks for the review @wgtmac & @amousavigourabi 🙌 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (PARQUET-2391) Remove unnecessary unboxing

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792666#comment-17792666
 ] 

ASF GitHub Bot commented on PARQUET-2391:
-

Fokko merged PR #1214:
URL: https://github.com/apache/parquet-mr/pull/1214




> Remove unnecessary unboxing
> ---
>
> Key: PARQUET-2391
> URL: https://issues.apache.org/jira/browse/PARQUET-2391
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-1647: [Java][Parquet] Implement FLOAT16 logical type [parquet-mr]

2023-12-03 Thread via GitHub


zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1413455235


##
pom.xml:
##
@@ -596,6 +597,9 @@
 
   
   org.apache.parquet.arrow.schema.SchemaMapping
+
+  org.apache.parquet.io.api.Binary

Review Comment:
   @wgtmac Just updated pom.xml to only exclude methods 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Resolved] (PARQUET-2391) Remove unnecessary unboxing

2023-12-03 Thread Fokko Driesprong (Jira)


 [ 
https://issues.apache.org/jira/browse/PARQUET-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong resolved PARQUET-2391.
---
Resolution: Fixed

> Remove unnecessary unboxing
> ---
>
> Key: PARQUET-2391
> URL: https://issues.apache.org/jira/browse/PARQUET-2391
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-2391: Remove unnecessary unboxing [parquet-mr]

2023-12-03 Thread via GitHub


Fokko merged PR #1214:
URL: https://github.com/apache/parquet-mr/pull/1214


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Resolved] (PARQUET-2385) Don't initialize CodecFactory in ParquetWriter

2023-12-03 Thread Gang Wu (Jira)


 [ 
https://issues.apache.org/jira/browse/PARQUET-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Wu resolved PARQUET-2385.
--
Fix Version/s: 1.14.0
 Assignee: Atour Mousavi Gourabi
   Resolution: Fixed

> Don't initialize CodecFactory in ParquetWriter
> --
>
> Key: PARQUET-2385
> URL: https://issues.apache.org/jira/browse/PARQUET-2385
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-mr
>Reporter: Atour Mousavi Gourabi
>Assignee: Atour Mousavi Gourabi
>Priority: Major
> Fix For: 1.14.0
>
>
> In ParquetWriter we initialize a CodecFactory, instead we should allow users 
> to set their own via the builder as to provide a little more flexibility 
> (analogous to PARQUET-2282).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-2385) Don't initialize CodecFactory in ParquetWriter

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792645#comment-17792645
 ] 

ASF GitHub Bot commented on PARQUET-2385:
-

wgtmac merged PR #1203:
URL: https://github.com/apache/parquet-mr/pull/1203




> Don't initialize CodecFactory in ParquetWriter
> --
>
> Key: PARQUET-2385
> URL: https://issues.apache.org/jira/browse/PARQUET-2385
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-mr
>Reporter: Atour Mousavi Gourabi
>Priority: Major
>
> In ParquetWriter we initialize a CodecFactory, instead we should allow users 
> to set their own via the builder as to provide a little more flexibility 
> (analogous to PARQUET-2282).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-2385: Allow user to specify CodecFactory for ParquetWriter [parquet-mr]

2023-12-03 Thread via GitHub


wgtmac merged PR #1203:
URL: https://github.com/apache/parquet-mr/pull/1203


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (PARQUET-2390) Replace anonymouse functions with lambda's

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792641#comment-17792641
 ] 

ASF GitHub Bot commented on PARQUET-2390:
-

Fokko merged PR #1213:
URL: https://github.com/apache/parquet-mr/pull/1213




> Replace anonymouse functions with lambda's
> --
>
> Key: PARQUET-2390
> URL: https://issues.apache.org/jira/browse/PARQUET-2390
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (PARQUET-2390) Replace anonymouse functions with lambda's

2023-12-03 Thread Fokko Driesprong (Jira)


 [ 
https://issues.apache.org/jira/browse/PARQUET-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong resolved PARQUET-2390.
---
Resolution: Fixed

> Replace anonymouse functions with lambda's
> --
>
> Key: PARQUET-2390
> URL: https://issues.apache.org/jira/browse/PARQUET-2390
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-2390) Replace anonymouse functions with lambda's

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792642#comment-17792642
 ] 

ASF GitHub Bot commented on PARQUET-2390:
-

Fokko commented on PR #1213:
URL: https://github.com/apache/parquet-mr/pull/1213#issuecomment-1837879063

   Thanks for the review @wgtmac 




> Replace anonymouse functions with lambda's
> --
>
> Key: PARQUET-2390
> URL: https://issues.apache.org/jira/browse/PARQUET-2390
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-2390: Replace anonymous functions with lambdas [parquet-mr]

2023-12-03 Thread via GitHub


Fokko commented on PR #1213:
URL: https://github.com/apache/parquet-mr/pull/1213#issuecomment-1837879063

   Thanks for the review @wgtmac 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



Re: [PR] PARQUET-2390: Replace anonymous functions with lambdas [parquet-mr]

2023-12-03 Thread via GitHub


Fokko merged PR #1213:
URL: https://github.com/apache/parquet-mr/pull/1213


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (PARQUET-2400) Update Spotless command in PR prompt to include vector plugins

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792640#comment-17792640
 ] 

ASF GitHub Bot commented on PARQUET-2400:
-

wgtmac merged PR #1223:
URL: https://github.com/apache/parquet-mr/pull/1223




> Update Spotless command in PR prompt to include vector plugins
> --
>
> Key: PARQUET-2400
> URL: https://issues.apache.org/jira/browse/PARQUET-2400
> Project: Parquet
>  Issue Type: Improvement
>Reporter: Atour Mousavi Gourabi
>Priority: Minor
>
> The Maven command to apply Spotless referenced in the PR prompt does not 
> include applying it to the parquet-plugins. This needs to be updated in those 
> docs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (PARQUET-2400) Update Spotless command in PR prompt to include vector plugins

2023-12-03 Thread Gang Wu (Jira)


 [ 
https://issues.apache.org/jira/browse/PARQUET-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Wu resolved PARQUET-2400.
--
Fix Version/s: 1.14.0
 Assignee: Atour Mousavi Gourabi
   Resolution: Fixed

> Update Spotless command in PR prompt to include vector plugins
> --
>
> Key: PARQUET-2400
> URL: https://issues.apache.org/jira/browse/PARQUET-2400
> Project: Parquet
>  Issue Type: Improvement
>Reporter: Atour Mousavi Gourabi
>Assignee: Atour Mousavi Gourabi
>Priority: Minor
> Fix For: 1.14.0
>
>
> The Maven command to apply Spotless referenced in the PR prompt does not 
> include applying it to the parquet-plugins. This needs to be updated in those 
> docs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-2400: Update Spotless command in PR prompt to include plugins [parquet-mr]

2023-12-03 Thread via GitHub


wgtmac merged PR #1223:
URL: https://github.com/apache/parquet-mr/pull/1223


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (PARQUET-1647) [Java] support for Arrow's float16

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792628#comment-17792628
 ] 

ASF GitHub Bot commented on PARQUET-1647:
-

wgtmac commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1413353347


##
pom.xml:
##
@@ -596,6 +597,9 @@
 
[Java] support for Arrow's float16
> --
>
> Key: PARQUET-1647
> URL: https://issues.apache.org/jira/browse/PARQUET-1647
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-format, parquet-thrift
>Reporter: The Alchemist
>Priority: Minor
>
> h2. DESCRIPTION
>  
> I'm wondering if there's any interest in supporting Arrow's {{float16}} type 
> in Parquet.
> There seem to be one or two {{float16}} / {{halffloat}} tickets here (e.g., 
> PARQUET-1403) but nothing that speaks to adding half-float support to Parquet 
> in-general.
>  
> h2. PLANS
> I'm able to spend some time on this, if someone points me  in the right 
> direction.
>  
>  # Add the {{HALFFLOAT}} or {{FLOAT16}} enum (any preferred naming 
> convention?) to 
> [https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L32]
>  # Add {{HALFFLOAT}} to {{org.apache.parquet.schema.PrimitiveType}}
>  # Add {{HALFFLOAT}} support to 
> {{org.apache.parquet.arrow.schema.SchemaConverter}}
>  # Add encoding for new type at {{org.apache.parquet.column.Encoding}}
>  # ??
> If anyone has any interest in this, pointers, or comments, they would be 
> greatly appreciated!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-1647: [Java][Parquet] Implement FLOAT16 logical type [parquet-mr]

2023-12-03 Thread via GitHub


wgtmac commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1413353347


##
pom.xml:
##
@@ -596,6 +597,9 @@
 
   
   org.apache.parquet.arrow.schema.SchemaMapping
+
+  org.apache.parquet.io.api.Binary

Review Comment:
   Thanks for rebasing! Could you change the exclusion to the level of a 
specific method? Class level seems too wide to me.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (PARQUET-1647) [Java] support for Arrow's float16

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792621#comment-17792621
 ] 

ASF GitHub Bot commented on PARQUET-1647:
-

zhangjiashen commented on PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#issuecomment-1837800275

   > Could you please rebase it?
   
   Rebased, can you help merge this PR?




> [Java] support for Arrow's float16
> --
>
> Key: PARQUET-1647
> URL: https://issues.apache.org/jira/browse/PARQUET-1647
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-format, parquet-thrift
>Reporter: The Alchemist
>Priority: Minor
>
> h2. DESCRIPTION
>  
> I'm wondering if there's any interest in supporting Arrow's {{float16}} type 
> in Parquet.
> There seem to be one or two {{float16}} / {{halffloat}} tickets here (e.g., 
> PARQUET-1403) but nothing that speaks to adding half-float support to Parquet 
> in-general.
>  
> h2. PLANS
> I'm able to spend some time on this, if someone points me  in the right 
> direction.
>  
>  # Add the {{HALFFLOAT}} or {{FLOAT16}} enum (any preferred naming 
> convention?) to 
> [https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L32]
>  # Add {{HALFFLOAT}} to {{org.apache.parquet.schema.PrimitiveType}}
>  # Add {{HALFFLOAT}} support to 
> {{org.apache.parquet.arrow.schema.SchemaConverter}}
>  # Add encoding for new type at {{org.apache.parquet.column.Encoding}}
>  # ??
> If anyone has any interest in this, pointers, or comments, they would be 
> greatly appreciated!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-1647: [Java][Parquet] Implement FLOAT16 logical type [parquet-mr]

2023-12-03 Thread via GitHub


zhangjiashen commented on PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#issuecomment-1837800275

   > Could you please rebase it?
   
   Rebased, can you help merge this PR?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Resolved] (PARQUET-2388) Deprecate `CHARSETS` on `PlainValuesWriter`

2023-12-03 Thread Fokko Driesprong (Jira)


 [ 
https://issues.apache.org/jira/browse/PARQUET-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong resolved PARQUET-2388.
---
Resolution: Fixed

> Deprecate `CHARSETS` on `PlainValuesWriter`
> ---
>
> Key: PARQUET-2388
> URL: https://issues.apache.org/jira/browse/PARQUET-2388
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-2388) Deprecate `CHARSETS` on `PlainValuesWriter`

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792618#comment-17792618
 ] 

ASF GitHub Bot commented on PARQUET-2388:
-

Fokko merged PR #1211:
URL: https://github.com/apache/parquet-mr/pull/1211




> Deprecate `CHARSETS` on `PlainValuesWriter`
> ---
>
> Key: PARQUET-2388
> URL: https://issues.apache.org/jira/browse/PARQUET-2388
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-2389) Remove redundant initializers

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792617#comment-17792617
 ] 

ASF GitHub Bot commented on PARQUET-2389:
-

Fokko commented on PR #1212:
URL: https://github.com/apache/parquet-mr/pull/1212#issuecomment-1837791938

   Thanks for the review @wgtmac and @amousavigourabi 🙌 




> Remove redundant initializers
> -
>
> Key: PARQUET-2389
> URL: https://issues.apache.org/jira/browse/PARQUET-2389
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (PARQUET-2389) Remove redundant initializers

2023-12-03 Thread Fokko Driesprong (Jira)


 [ 
https://issues.apache.org/jira/browse/PARQUET-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong resolved PARQUET-2389.
---
Resolution: Fixed

> Remove redundant initializers
> -
>
> Key: PARQUET-2389
> URL: https://issues.apache.org/jira/browse/PARQUET-2389
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-2389: Remove redundant initializers [parquet-mr]

2023-12-03 Thread via GitHub


Fokko commented on PR #1212:
URL: https://github.com/apache/parquet-mr/pull/1212#issuecomment-1837791938

   Thanks for the review @wgtmac and @amousavigourabi 🙌 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



Re: [PR] PARQUET-2389: Remove redundant initializers [parquet-mr]

2023-12-03 Thread via GitHub


Fokko merged PR #1212:
URL: https://github.com/apache/parquet-mr/pull/1212


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



Re: [PR] PARQUET-2388: Deprecate `CHARSETS` on `PlainValuesWriter` [parquet-mr]

2023-12-03 Thread via GitHub


Fokko merged PR #1211:
URL: https://github.com/apache/parquet-mr/pull/1211


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (PARQUET-2389) Remove redundant initializers

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792616#comment-17792616
 ] 

ASF GitHub Bot commented on PARQUET-2389:
-

Fokko merged PR #1212:
URL: https://github.com/apache/parquet-mr/pull/1212




> Remove redundant initializers
> -
>
> Key: PARQUET-2389
> URL: https://issues.apache.org/jira/browse/PARQUET-2389
> Project: Parquet
>  Issue Type: Improvement
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] Bump actions/setup-java from 3 to 4 [parquet-mr]

2023-12-03 Thread via GitHub


Fokko merged PR #1225:
URL: https://github.com/apache/parquet-mr/pull/1225


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Commented] (PARQUET-2387) Simplify `hasFieldsIgnored` expression

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792615#comment-17792615
 ] 

ASF GitHub Bot commented on PARQUET-2387:
-

Fokko commented on PR #1210:
URL: https://github.com/apache/parquet-mr/pull/1210#issuecomment-1837786670

   Thanks @wgtmac 🙌 




> Simplify `hasFieldsIgnored` expression
> --
>
> Key: PARQUET-2387
> URL: https://issues.apache.org/jira/browse/PARQUET-2387
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-thrift
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (PARQUET-2387) Simplify `hasFieldsIgnored` expression

2023-12-03 Thread Fokko Driesprong (Jira)


 [ 
https://issues.apache.org/jira/browse/PARQUET-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fokko Driesprong resolved PARQUET-2387.
---
Resolution: Fixed

> Simplify `hasFieldsIgnored` expression
> --
>
> Key: PARQUET-2387
> URL: https://issues.apache.org/jira/browse/PARQUET-2387
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-thrift
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-2387) Simplify `hasFieldsIgnored` expression

2023-12-03 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792614#comment-17792614
 ] 

ASF GitHub Bot commented on PARQUET-2387:
-

Fokko merged PR #1210:
URL: https://github.com/apache/parquet-mr/pull/1210




> Simplify `hasFieldsIgnored` expression
> --
>
> Key: PARQUET-2387
> URL: https://issues.apache.org/jira/browse/PARQUET-2387
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-thrift
>Affects Versions: 1.13.1
>Reporter: Fokko Driesprong
>Assignee: Fokko Driesprong
>Priority: Major
> Fix For: 1.14.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: [PR] PARQUET-2387: Simplify `hasFieldsIgnored` expression [parquet-mr]

2023-12-03 Thread via GitHub


Fokko commented on PR #1210:
URL: https://github.com/apache/parquet-mr/pull/1210#issuecomment-1837786670

   Thanks @wgtmac 🙌 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



Re: [PR] PARQUET-2387: Simplify `hasFieldsIgnored` expression [parquet-mr]

2023-12-03 Thread via GitHub


Fokko merged PR #1210:
URL: https://github.com/apache/parquet-mr/pull/1210


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[jira] [Created] (PARQUET-2403) Create specialized HadoopParquetReader

2023-12-03 Thread Atour Mousavi Gourabi (Jira)
Atour Mousavi Gourabi created PARQUET-2403:
--

 Summary: Create specialized HadoopParquetReader
 Key: PARQUET-2403
 URL: https://issues.apache.org/jira/browse/PARQUET-2403
 Project: Parquet
  Issue Type: Improvement
  Components: parquet-hadoop, parquet-mr
Reporter: Atour Mousavi Gourabi


Setting up a specialized HadoopParquetReader allows for deprecating the Hadoop 
specific methods in ParquetReader as in PARQUET-2381. This could allow for an 
easier and more gentle transition away from the current Hadoop-coupled Parquet 
reader API by encouraging transitioning to this specialized class already, 
before completely phasing out the Hadoop methods from ParquetReader in the 
future.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (PARQUET-2402) Create specialized HadoopParquetWriter

2023-12-03 Thread Atour Mousavi Gourabi (Jira)
Atour Mousavi Gourabi created PARQUET-2402:
--

 Summary: Create specialized HadoopParquetWriter
 Key: PARQUET-2402
 URL: https://issues.apache.org/jira/browse/PARQUET-2402
 Project: Parquet
  Issue Type: Improvement
  Components: parquet-hadoop, parquet-mr
Reporter: Atour Mousavi Gourabi


Setting up a specialized HadoopParquetWriter allows for deprecating the Hadoop 
specific methods in ParquetWriter as in PARQUET-2381. This could allow for an 
easier and more gentle transition away from the current Hadoop-coupled Parquet 
writer API by encouraging transitioning to this specialized class already, 
before completely phasing out the Hadoop methods from ParquetWriter in the 
future.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (PARQUET-1776) Add Java NIO Avro OutputFile InputFile

2023-12-03 Thread Atour Mousavi Gourabi (Jira)


[ 
https://issues.apache.org/jira/browse/PARQUET-1776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792539#comment-17792539
 ] 

Atour Mousavi Gourabi commented on PARQUET-1776:


As far as I can tell this has been resolved by 
[https://github.com/apache/parquet-mr/pull/,] so I think we could close 
this ticket.

> Add Java NIO Avro OutputFile InputFile
> --
>
> Key: PARQUET-1776
> URL: https://issues.apache.org/jira/browse/PARQUET-1776
> Project: Parquet
>  Issue Type: Improvement
>  Components: parquet-avro
>Reporter: David Mollitor
>Priority: Minor
>
> Add a wrapper around Java NIO Path for {{org.apache.parquet.io.OutputFile}} 
> and {{org.apache.parquet.io.InputFile}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[PR] Bump actions/setup-java from 3 to 4 [parquet-format]

2023-12-03 Thread via GitHub


dependabot[bot] opened a new pull request, #223:
URL: https://github.com/apache/parquet-format/pull/223

   Bumps [actions/setup-java](https://github.com/actions/setup-java) from 3 to 
4.
   
   Release notes
   Sourced from https://github.com/actions/setup-java/releases";>actions/setup-java's 
releases.
   
   v4.0.0
   What's Changed
   In the scope of this release, the version of the Node.js runtime was 
updated to 20. The majority of dependencies were updated to the latest 
versions. From now on, the code for the setup-java will run on Node.js 20 
instead of Node.js 16.
   Breaking changes
   
   Update Node.js runtime to version 20 by https://github.com/aparnajyothi-y";>@​aparnajyothi-y in 
https://redirect.github.com/actions/setup-java/pull/558";>actions/setup-java#558
   
   Non-breaking changes
   
   Adding support for microsoft openjdk 21.0.0 by https://github.com/ralfstuckert";>@​ralfstuckert in https://redirect.github.com/actions/setup-java/pull/546";>actions/setup-java#546
   Update @​actions/cache dependency and documentation by https://github.com/IvanZosimov";>@​IvanZosimov in https://redirect.github.com/actions/setup-java/pull/549";>actions/setup-java#549
   Implementation of the cache-dependency-path option to control caching 
dependency by https://github.com/itchyny";>@​itchyny 
in https://redirect.github.com/actions/setup-java/pull/499";>actions/setup-java#499
   
   New Contributors
   
   https://github.com/ralfstuckert";>@​ralfstuckert made 
their first contribution in https://redirect.github.com/actions/setup-java/pull/546";>actions/setup-java#546
   https://github.com/itchyny";>@​itchyny made 
their first contribution in https://redirect.github.com/actions/setup-java/pull/499";>actions/setup-java#499
   
   Full Changelog: https://github.com/actions/setup-java/compare/v3...v4.0.0";>https://github.com/actions/setup-java/compare/v3...v4.0.0
   v3.13.0
   What's changed
   In the scope of this release, support for Dragonwell JDK was added by https://github.com/Accelerator1996";>@​Accelerator1996 in 
https://redirect.github.com/actions/setup-java/pull/532";>actions/setup-java#532
   steps:
- name: Checkout
  uses: actions/checkout@v3
- name: Setup-java
  uses: actions/setup-java@v3
  with:
distribution: 'dragonwell'
java-version: '17'
   
   Several inaccuracies were also fixed:
   
   Fix XML namespaces wrongly using https by https://github.com/gnodet";>@​gnodet in https://redirect.github.com/actions/setup-java/pull/503";>actions/setup-java#503
   Fix typo and remove unintentional(?) word by https://github.com/CyberFlameGO";>@​CyberFlameGO in https://redirect.github.com/actions/setup-java/pull/518";>actions/setup-java#518
   Fix usage link within the README.md file by https://github.com/dassiorleando";>@​dassiorleando in https://redirect.github.com/actions/setup-java/pull/525";>actions/setup-java#525
   
   New Contributors
   
   https://github.com/CyberFlameGO";>@​CyberFlameGO made 
their first contribution in https://redirect.github.com/actions/setup-java/pull/518";>actions/setup-java#518
   https://github.com/dassiorleando";>@​dassiorleando made 
their first contribution in https://redirect.github.com/actions/setup-java/pull/525";>actions/setup-java#525
   https://github.com/gnodet";>@​gnodet made their 
first contribution in https://redirect.github.com/actions/setup-java/pull/503";>actions/setup-java#503
   https://github.com/Accelerator1996";>@​Accelerator1996 
made their first contribution in https://redirect.github.com/actions/setup-java/pull/532";>actions/setup-java#532
   
   Full Changelog: https://github.com/actions/setup-java/compare/v3...v3.13.0";>https://github.com/actions/setup-java/compare/v3...v3.13.0
   v3.12.0
   
   
   ... (truncated)
   
   
   Commits
   
   https://github.com/actions/setup-java/commit/387ac29b308b003ca37ba93a6cab5eb57c8f5f93";>387ac29
 Upgrade Node to v20 (https://redirect.github.com/actions/setup-java/issues/558";>#558)
   https://github.com/actions/setup-java/commit/9eda6b51cc4f6ee99be3dd5537b85e389e47bda9";>9eda6b5
 feat: implement cache-dependency-path option to control caching dependency 
(#...
   https://github.com/actions/setup-java/commit/78078da0cd035d0d177cc2cb696e05d96fba7d11";>78078da
 Update @​actions/cache dependency and documentation (https://redirect.github.com/actions/setup-java/issues/549";>#549)
   https://github.com/actions/setup-java/commit/5caaba646e214abb5c4c808eb8fe13db519ab757";>5caaba6
 add support for microsoft openjdk 21.0.0 (https://redirect.github.com/actions/setup-java/issues/546";>#546)
   See full diff in https://github.com/actions/setup-java/compare/v3...v4";>compare 
view
   
   
   
   
   
   [![Dependabot compatibility 
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/setup-java&package-manager=github_actions&previous-version=3&new-version=4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-depen