zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1335132367
##
parquet-column/src/main/java/org/apache/parquet/schema/PrimitiveStringifier.java:
##
@@ -448,4 +449,16 @@ private void appendHex(byte[] array, int offset,
zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1335132272
##
parquet-common/src/main/java/org/apache/parquet/util/Float16.java:
##
@@ -0,0 +1,192 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1335132034
##
parquet-common/src/test/java/org/apache/parquet/util/TestFloat16.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1335131998
##
parquet-common/src/main/java/org/apache/parquet/util/Float16.java:
##
@@ -0,0 +1,192 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1335131904
##
parquet-hadoop/src/test/java/org/apache/parquet/format/converter/TestParquetMetadataConverter.java:
##
@@ -990,6 +990,30 @@ private void
zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1335131904
##
parquet-hadoop/src/test/java/org/apache/parquet/format/converter/TestParquetMetadataConverter.java:
##
@@ -990,6 +990,30 @@ private void
zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1335131781
##
parquet-column/src/main/java/org/apache/parquet/schema/PrimitiveComparator.java:
##
@@ -276,4 +279,24 @@ public String toString() {
return
zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1335131705
##
parquet-common/src/test/java/org/apache/parquet/util/TestFloat16.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
dependabot[bot] opened a new pull request, #1152:
URL: https://github.com/apache/parquet-mr/pull/1152
Bumps [com.h2database:h2](https://github.com/h2database/h2database) from
2.2.220 to 2.2.224.
Release notes
Sourced from
dependabot[bot] opened a new pull request, #1151:
URL: https://github.com/apache/parquet-mr/pull/1151
Bumps [io.airlift:aircompressor](https://github.com/airlift/aircompressor)
from 0.21 to 0.25.
Commits
MaheshGPai commented on PR #1121:
URL: https://github.com/apache/parquet-mr/pull/1121#issuecomment-1732238952
> This is a great initiative. Do you still have plan to address the feedback
@MaheshGPai ?
@shangxinli I do plan to work on it. But I have not had time to get to this.
--
zhangjiashen commented on PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#issuecomment-1732203105
> CI failures are likely due to the fact that the addition of the logical
type to parquet-format is unmerged, so the specific [PR
Fokko merged PR #1150:
URL: https://github.com/apache/parquet-mr/pull/1150
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
dependabot[bot] opened a new pull request, #1150:
URL: https://github.com/apache/parquet-mr/pull/1150
Bumps [com.squareup.okhttp3:okhttp](https://github.com/square/okhttp) from
4.6.0 to 4.11.0.
Commits
dependabot[bot] opened a new pull request, #1149:
URL: https://github.com/apache/parquet-mr/pull/1149
Bumps
[com.twitter.elephantbird:elephant-bird-core](https://github.com/twitter/elephant-bird)
from 4.4 to 4.17.
Release notes
Sourced from
Fokko merged PR #1146:
URL: https://github.com/apache/parquet-mr/pull/1146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
Fokko merged PR #1148:
URL: https://github.com/apache/parquet-mr/pull/1148
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
Fokko merged PR #1145:
URL: https://github.com/apache/parquet-mr/pull/1145
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
findepi commented on code in PR #216:
URL: https://github.com/apache/parquet-format/pull/216#discussion_r1333952085
##
src/main/thrift/parquet.thrift:
##
@@ -216,7 +216,12 @@ struct Statistics {
/** count of distinct values occurring */
4: optional i64 distinct_count;
shangxinli commented on PR #1121:
URL: https://github.com/apache/parquet-mr/pull/1121#issuecomment-1730734276
This is a great initiative. Do you still have plan to address the feedback
@MaheshGPai ?
--
This is an automated message from the Apache Git Service.
To respond to the message,
mapleFU commented on code in PR #216:
URL: https://github.com/apache/parquet-format/pull/216#discussion_r1333823691
##
src/main/thrift/parquet.thrift:
##
@@ -216,7 +216,12 @@ struct Statistics {
/** count of distinct values occurring */
4: optional i64 distinct_count;
wgtmac commented on code in PR #216:
URL: https://github.com/apache/parquet-format/pull/216#discussion_r1333778393
##
src/main/thrift/parquet.thrift:
##
@@ -216,7 +216,12 @@ struct Statistics {
/** count of distinct values occurring */
4: optional i64 distinct_count;
findepi commented on PR #216:
URL: https://github.com/apache/parquet-format/pull/216#issuecomment-1730285274
Thanks @raunaqmorarka, this lgtm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
raunaqmorarka commented on code in PR #216:
URL: https://github.com/apache/parquet-format/pull/216#discussion_r128796
##
src/main/thrift/parquet.thrift:
##
@@ -216,7 +216,12 @@ struct Statistics {
/** count of distinct values occurring */
4: optional i64
wgtmac commented on code in PR #216:
URL: https://github.com/apache/parquet-format/pull/216#discussion_r1333286445
##
src/main/thrift/parquet.thrift:
##
@@ -216,7 +216,12 @@ struct Statistics {
/** count of distinct values occurring */
4: optional i64 distinct_count;
Fokko merged PR #1147:
URL: https://github.com/apache/parquet-mr/pull/1147
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
Fokko merged PR #203:
URL: https://github.com/apache/parquet-format/pull/203
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
Fokko closed pull request #1054: PARQUET-2268: Bump Thrift to 0.18.1
URL: https://github.com/apache/parquet-mr/pull/1054
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
Fokko opened a new pull request, #1148:
URL: https://github.com/apache/parquet-mr/pull/1148
Dependabot is effectively not working because
5 PRs have been open for ages.
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Parquet
dependabot[bot] commented on PR #1110:
URL: https://github.com/apache/parquet-mr/pull/1110#issuecomment-1729435128
OK, I won't notify you about version 3.3.x again, unless you re-open this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please
dependabot[bot] closed pull request #1110: Bump scalatest_2.12 from 3.0.1 to
3.3.0-SNAP4
URL: https://github.com/apache/parquet-mr/pull/1110
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
Fokko commented on PR #1110:
URL: https://github.com/apache/parquet-mr/pull/1110#issuecomment-1729435056
@dependabot ignore this minor version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
Fokko commented on PR #1068:
URL: https://github.com/apache/parquet-mr/pull/1068#issuecomment-1729434254
@lukasnalezenec what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
dependabot[bot] closed pull request #1062: Bump jmh.version from 1.21 to 1.36
URL: https://github.com/apache/parquet-mr/pull/1062
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
dependabot[bot] commented on PR #1062:
URL: https://github.com/apache/parquet-mr/pull/1062#issuecomment-1729427261
Superseded by #1147.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
dependabot[bot] opened a new pull request, #1147:
URL: https://github.com/apache/parquet-mr/pull/1147
Bumps `jmh.version` from 1.21 to 1.37.
Updates `org.openjdk.jmh:jmh-core` from 1.21 to 1.37
Commits
Fokko commented on PR #1062:
URL: https://github.com/apache/parquet-mr/pull/1062#issuecomment-1729426235
@dependabot rebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
Fokko opened a new pull request, #1146:
URL: https://github.com/apache/parquet-mr/pull/1146
IDEA was lighting up as a christmas tree :)
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Parquet
Fokko opened a new pull request, #1145:
URL: https://github.com/apache/parquet-mr/pull/1145
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references
them in
Fokko opened a new pull request, #1144:
URL: https://github.com/apache/parquet-mr/pull/1144
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references
them in
raunaqmorarka opened a new pull request, #216:
URL: https://github.com/apache/parquet-format/pull/216
### Jira
- https://issues.apache.org/jira/browse/PARQUET-2352
This updates the spec to allow truncation of row group min_values/max_value
statistics so that readers can take
benibus commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1331975632
##
parquet-common/src/main/java/org/apache/parquet/util/Float16.java:
##
@@ -0,0 +1,192 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ *
clairemcginty commented on code in PR #1140:
URL: https://github.com/apache/parquet-mr/pull/1140#discussion_r1331817509
##
parquet-avro/src/main/java/org/apache/parquet/avro/AvroReadSupport.java:
##
@@ -129,10 +129,10 @@ public RecordMaterializer prepareForRead(
wgtmac commented on PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#issuecomment-1726797426
> CI failures are likely due to the fact that the addition of the logical
type to parquet-format is unmerged, so the specific [PR
wgtmac commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1330863760
##
parquet-common/src/test/java/org/apache/parquet/util/TestFloat16.java:
##
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ *
wgtmac commented on PR #1141:
URL: https://github.com/apache/parquet-mr/pull/1141#issuecomment-1726754834
Thanks for working on this! I will take a look by the end of this week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
wgtmac commented on code in PR #1140:
URL: https://github.com/apache/parquet-mr/pull/1140#discussion_r1330857433
##
parquet-avro/src/main/java/org/apache/parquet/avro/AvroReadSupport.java:
##
@@ -129,10 +129,10 @@ public RecordMaterializer prepareForRead(
avroSchema =
wgtmac commented on PR #1140:
URL: https://github.com/apache/parquet-mr/pull/1140#issuecomment-1726750224
Thanks @wwang-talend for opening the PR! Could you create a JIRA issue for
this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
benibus commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1330570985
##
parquet-hadoop/src/test/java/org/apache/parquet/format/converter/TestParquetMetadataConverter.java:
##
@@ -990,6 +990,30 @@ private void
wgtmac commented on code in PR #1143:
URL: https://github.com/apache/parquet-mr/pull/1143#discussion_r1330272179
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/rewrite/ParquetRewriter.java:
##
@@ -366,6 +366,10 @@ private void processChunk(ColumnChunkMetaData chunk,
gszadovszky commented on code in PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#discussion_r1329973062
##
parquet-hadoop/src/test/java/org/apache/parquet/hadoop/TestParquetFileWriter.java:
##
@@ -89,10 +89,13 @@
import
gszadovszky commented on code in PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#discussion_r1329967395
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/vectorio/BindingUtils.java:
##
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software
steveloughran commented on PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#issuecomment-1725306371
@gszadovszky thanks for your comments, will update the PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
steveloughran commented on code in PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#discussion_r1329958167
##
parquet-hadoop/src/test/java/org/apache/parquet/hadoop/TestParquetFileWriter.java:
##
@@ -89,10 +89,13 @@
import
steveloughran commented on code in PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#discussion_r1329955534
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/vectorio/BindingUtils.java:
##
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software
steveloughran commented on code in PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#discussion_r1329955142
##
parquet-hadoop/pom.xml:
##
@@ -79,6 +79,11 @@
${hadoop.version}
provided
+
Review Comment:
no, will cut. that was how I migrated
steveloughran commented on code in PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#discussion_r1329954899
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/vectorio/BindingUtils.java:
##
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software
gszadovszky commented on code in PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#discussion_r1329695018
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/vectorio/BindingUtils.java:
##
@@ -0,0 +1,211 @@
+/*
+ * Licensed to the Apache Software
Fokko merged PR #1134:
URL: https://github.com/apache/parquet-mr/pull/1134
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
benibus commented on PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#issuecomment-1724083429
CI failures are likely due to the fact that the addition of the logical type
to parquet-format is unmerged, so the specific [PR
steveloughran commented on PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#issuecomment-1724053099
@danielcweeks that's a good point about pluggability.
1. an interface/implementation split in parquet would line you up later to
choose the back end, maybe?
2. I've
steveloughran commented on PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#issuecomment-1724037037
@shangxinli looking forward to your comments -anything you can do to test
will be wonderful too!
--
This is an automated message from the Apache Git Service.
To respond to the
ConeyLiu commented on PR #1143:
URL: https://github.com/apache/parquet-mr/pull/1143#issuecomment-1722961986
Hi @wgtmac, could you help to review this? Thanks in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
ConeyLiu opened a new pull request, #1143:
URL: https://github.com/apache/parquet-mr/pull/1143
The bloomfilter data is lost after rewriting with recompression or
re-encrypt. We should rewrite the bloomfilter data as well.
Make sure you have checked _all_ steps below.
### Jira
shangxinli commented on PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#issuecomment-1722528243
@steveloughran Thanks a lot for creating this PR! This is an important
feature that we improve the reading performance of Parquet. I just took a brief
look and they look great! I
zhangjiashen opened a new pull request, #1142:
URL: https://github.com/apache/parquet-mr/pull/1142
### Jira
- [ ] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references
them in the PR title. For example, "PARQUET-1234: My
amousavigourabi opened a new pull request, #1141:
URL: https://github.com/apache/parquet-mr/pull/1141
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references
danielcweeks commented on code in PR #1139:
URL: https://github.com/apache/parquet-mr/pull/1139#discussion_r1326633841
##
parquet-common/src/main/java/org/apache/parquet/io/SeekableInputStream.java:
##
@@ -105,4 +107,21 @@ public abstract class SeekableInputStream extends
wwang-talend opened a new pull request, #1140:
URL: https://github.com/apache/parquet-mr/pull/1140
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references
dependabot[bot] commented on PR #215:
URL: https://github.com/apache/parquet-format/pull/215#issuecomment-1718760575
OK, I won't notify you about org.slf4j:slf4j-api again, unless you re-open
this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message,
dependabot[bot] closed pull request #215: PARQUET-2346: Bump
org.slf4j:slf4j-api from 1.7.12 to 2.0.9
URL: https://github.com/apache/parquet-format/pull/215
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
wgtmac commented on PR #215:
URL: https://github.com/apache/parquet-format/pull/215#issuecomment-1718760542
@dependabot ignore this dependency
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
wgtmac commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1324650964
##
src/main/thrift/parquet.thrift:
##
@@ -764,6 +810,14 @@ struct ColumnMetaData {
* in a single I/O.
*/
15: optional i32 bloom_filter_length;
+
+ /**
steveloughran opened a new pull request, #1139:
URL: https://github.com/apache/parquet-mr/pull/1139
Make sure you have checked _all_ steps below.
### Jira
- [X] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references
etseidl commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1323524900
##
src/main/thrift/parquet.thrift:
##
@@ -977,6 +1038,25 @@ struct ColumnIndex {
/** A list containing the number of null values for each page **/
5:
emkornfield commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1323506069
##
src/main/thrift/parquet.thrift:
##
@@ -977,6 +1038,25 @@ struct ColumnIndex {
/** A list containing the number of null values for each page **/
5:
emkornfield commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1323505565
##
src/main/thrift/parquet.thrift:
##
@@ -764,6 +810,14 @@ struct ColumnMetaData {
* in a single I/O.
*/
15: optional i32 bloom_filter_length;
+
+
JFinis commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1323069846
##
src/main/thrift/parquet.thrift:
##
@@ -764,6 +810,14 @@ struct ColumnMetaData {
* in a single I/O.
*/
15: optional i32 bloom_filter_length;
+
+ /**
JFinis commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1323069846
##
src/main/thrift/parquet.thrift:
##
@@ -764,6 +810,14 @@ struct ColumnMetaData {
* in a single I/O.
*/
15: optional i32 bloom_filter_length;
+
+ /**
JFinis commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1323059489
##
src/main/thrift/parquet.thrift:
##
@@ -977,6 +1038,25 @@ struct ColumnIndex {
/** A list containing the number of null values for each page **/
5:
JFinis commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1323028211
##
src/main/thrift/parquet.thrift:
##
@@ -764,6 +810,14 @@ struct ColumnMetaData {
* in a single I/O.
*/
15: optional i32 bloom_filter_length;
+
+ /**
JFinis commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1323028211
##
src/main/thrift/parquet.thrift:
##
@@ -764,6 +810,14 @@ struct ColumnMetaData {
* in a single I/O.
*/
15: optional i32 bloom_filter_length;
+
+ /**
emkornfield commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1322319582
##
src/main/thrift/parquet.thrift:
##
@@ -977,6 +1038,25 @@ struct ColumnIndex {
/** A list containing the number of null values for each page **/
5:
dependabot[bot] closed pull request #204: Bump slf4j-api from 1.7.12 to 2.0.7
URL: https://github.com/apache/parquet-format/pull/204
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
dependabot[bot] commented on PR #204:
URL: https://github.com/apache/parquet-format/pull/204#issuecomment-1712817820
Superseded by #215.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
dependabot[bot] opened a new pull request, #215:
URL: https://github.com/apache/parquet-format/pull/215
Bumps org.slf4j:slf4j-api from 1.7.12 to 2.0.9.
[