jinyius commented on PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#issuecomment-1232470935
hmm... what timing. i actually have a pr for what i think is a more robust
approach that truncates at an arbitrary recursion depth by putting the
remaining recursion levels into a binar
emkornfield commented on PR #184:
URL: https://github.com/apache/parquet-format/pull/184#issuecomment-1232420353
> t is not that trivial. For the half-precision floating point numbers we do
not have native support for either cpp or java so we can define the total
ordering as we want. But we
gszadovszky commented on PR #184:
URL: https://github.com/apache/parquet-format/pull/184#issuecomment-1231323733
> > It would not be too easy to implement the half-precision floating point
comparison logic since java does not have such a primitive type.
>
> While not effortless, it sh
pitrou commented on PR #184:
URL: https://github.com/apache/parquet-format/pull/184#issuecomment-1231300374
> It would not be too easy to implement the half-precision floating point
comparison logic since java does not have such a primitive type.
While not effortless, it should be rel
gszadovszky commented on PR #184:
URL: https://github.com/apache/parquet-format/pull/184#issuecomment-1231284535
> It isn't clear to me if this should be a logical type or a physical type.
We would need understand if there is different handling for forward
compatibility purposes (what do we
emkornfield commented on code in PR #184:
URL: https://github.com/apache/parquet-format/pull/184#discussion_r958042831
##
src/main/thrift/parquet.thrift:
##
@@ -232,6 +232,7 @@ struct MapType {} // see LogicalTypes.md
struct ListType {}// see LogicalTypes.md
struct Enu
emkornfield commented on PR #184:
URL: https://github.com/apache/parquet-format/pull/184#issuecomment-1231187345
It isn't clear to me if this should be a logical type or a physical type. We
would need understand if there is different handling for forward compatibility
purposes (what do we w
emkornfield commented on PR #184:
URL: https://github.com/apache/parquet-format/pull/184#issuecomment-1231185256
We should probably specify that using the [Byte Split
Encodings](https://github.com/apache/parquet-format/blob/master/Encodings.md#byte-stream-split-byte_stream_split--9)
can be
anjakefala commented on code in PR #184:
URL: https://github.com/apache/parquet-format/pull/184#discussion_r957822642
##
src/main/thrift/parquet.thrift:
##
@@ -342,6 +343,7 @@ union LogicalType {
12: JsonType JSON // use ConvertedType JSON
13: BsonType BSON
pitrou commented on PR #184:
URL: https://github.com/apache/parquet-format/pull/184#issuecomment-1229983463
@anjakefala You need to add to the `LogicalType` union, not to the `Type`
enum (which is for physical types).
Also cc @emkornfield
--
This is an automated message from the A
pitrou commented on code in PR #184:
URL: https://github.com/apache/parquet-format/pull/184#discussion_r957065065
##
src/main/thrift/parquet.thrift:
##
@@ -889,6 +891,7 @@ union ColumnOrder {
* INT32 - signed comparison
* INT64 - signed comparison
* INT96 (only
pitrou commented on code in PR #184:
URL: https://github.com/apache/parquet-format/pull/184#discussion_r957064862
##
src/main/thrift/parquet.thrift:
##
@@ -416,6 +417,7 @@ enum Encoding {
* BOOLEAN - 1 bit per value. 0 is false; 1 is true.
* INT32 - 4 bytes per value. S
pitrou commented on code in PR #184:
URL: https://github.com/apache/parquet-format/pull/184#discussion_r957064370
##
src/main/thrift/parquet.thrift:
##
@@ -34,6 +34,7 @@ enum Type {
INT32 = 1;
INT64 = 2;
INT96 = 3; // deprecated, only used by legacy implementations.
+
pitrou commented on code in PR #184:
URL: https://github.com/apache/parquet-format/pull/184#discussion_r957063147
##
LogicalTypes.md:
##
@@ -245,6 +245,18 @@ comparison.
To support compatibility with older readers, implementations of parquet-format
should
write `DecimalType`
pitrou commented on code in PR #184:
URL: https://github.com/apache/parquet-format/pull/184#discussion_r957060962
##
LogicalTypes.md:
##
@@ -245,6 +245,18 @@ comparison.
To support compatibility with older readers, implementations of parquet-format
should
write `DecimalType`
pitrou commented on code in PR #184:
URL: https://github.com/apache/parquet-format/pull/184#discussion_r957060962
##
LogicalTypes.md:
##
@@ -245,6 +245,18 @@ comparison.
To support compatibility with older readers, implementations of parquet-format
should
write `DecimalType`
matthieun commented on PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#issuecomment-1229042610
@shangxinli Let me know if this is good to merge!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
anjakefala opened a new pull request, #184:
URL: https://github.com/apache/parquet-format/pull/184
Make sure you have checked _all_ steps below.
### Jira
- [X] My PR addresses the following [Parquet Jira
1](https://issues.apache.org/jira/browse/PARQUET-758) and
[2](https://iss
sekikn opened a new pull request, #991:
URL: https://github.com/apache/parquet-mr/pull/991
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references
them in th
sekikn opened a new pull request, #990:
URL: https://github.com/apache/parquet-mr/pull/990
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references
them in th
NickCrews commented on PR #433:
URL: https://github.com/apache/parquet-mr/pull/433#issuecomment-1226667307
It might be nice if we actually suggested an alternative instead of just
saying "don't do this."
You can see my solution at
https://gist.github.com/NickCrews/7a47ef4083160011e8e
zhongyujiang commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r953866331
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##
@@ -109,7 +110,17 @@ public BytesInput decompress(BytesInput bytes, int
uncompress
patchwork01 opened a new pull request, #989:
URL: https://github.com/apache/parquet-mr/pull/989
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references
them
matthieun commented on code in PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#discussion_r952835674
##
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java:
##
@@ -79,12 +80,20 @@ public MessageType convert(Class
protobufClass) {
}
shangxinli merged PR #986:
URL: https://github.com/apache/parquet-mr/pull/986
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@parquet.ap
shangxinli commented on code in PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#discussion_r952778132
##
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java:
##
@@ -79,12 +80,20 @@ public MessageType convert(Class
protobufClass) {
}
shangxinli commented on code in PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#discussion_r952778132
##
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java:
##
@@ -79,12 +80,20 @@ public MessageType convert(Class
protobufClass) {
}
shangxinli commented on code in PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#discussion_r952764513
##
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java:
##
@@ -79,12 +80,20 @@ public MessageType convert(Class
protobufClass) {
}
matthieun commented on code in PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#discussion_r951648555
##
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java:
##
@@ -79,12 +80,20 @@ public MessageType convert(Class
protobufClass) {
}
matthieun commented on code in PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#discussion_r951644453
##
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java:
##
@@ -79,12 +80,20 @@ public MessageType convert(Class
protobufClass) {
}
steveloughran commented on code in PR #985:
URL: https://github.com/apache/parquet-mr/pull/985#discussion_r951589888
##
pom.xml:
##
@@ -160,7 +160,11 @@
org.slf4j
-slf4j-log4j12
+*
Review Comment:
it means that
1
shangxinli commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r950908609
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -88,6 +136,21 @@ public long skip(long n) {
return bytesToSkip;
shangxinli commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r950908296
##
parquet-common/src/main/java/org/apache/parquet/bytes/MultiBufferInputStream.java:
##
@@ -379,4 +427,120 @@ public void remove() {
second.remove();
}
shangxinli commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r950908215
##
parquet-common/src/main/java/org/apache/parquet/bytes/MultiBufferInputStream.java:
##
@@ -238,8 +257,31 @@ public int read(byte[] bytes, int off, int len) {
}
shangxinli commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r950908127
##
parquet-common/src/main/java/org/apache/parquet/bytes/ByteBufferInputStream.java:
##
@@ -138,6 +134,18 @@ public int read(byte[] b, int off, int len) throws
IOEx
shangxinli commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r950907839
##
parquet-common/src/main/java/org/apache/parquet/bytes/MultiBufferInputStream.java:
##
@@ -238,8 +257,31 @@ public int read(byte[] bytes, int off, int len) {
}
shangxinli commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r950906824
##
parquet-common/src/main/java/org/apache/parquet/bytes/ByteBufferInputStream.java:
##
@@ -138,6 +134,18 @@ public int read(byte[] b, int off, int len) throws
IOEx
shangxinli commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r950903406
##
parquet-common/src/main/java/org/apache/parquet/bytes/ByteBufferInputStream.java:
##
@@ -157,4 +165,80 @@ public void reset() throws IOException {
public boole
shangxinli commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r950903342
##
parquet-common/src/main/java/org/apache/parquet/bytes/ByteBufferInputStream.java:
##
@@ -157,4 +165,80 @@ public void reset() throws IOException {
public boole
shangxinli commented on code in PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#discussion_r950888122
##
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java:
##
@@ -79,12 +80,20 @@ public MessageType convert(Class
protobufClass) {
}
shangxinli commented on code in PR #988:
URL: https://github.com/apache/parquet-mr/pull/988#discussion_r950887905
##
parquet-protobuf/src/main/java/org/apache/parquet/proto/ProtoSchemaConverter.java:
##
@@ -79,12 +80,20 @@ public MessageType convert(Class
protobufClass) {
}
shangxinli commented on PR #986:
URL: https://github.com/apache/parquet-mr/pull/986#issuecomment-1221602126
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscr
shangxinli commented on code in PR #985:
URL: https://github.com/apache/parquet-mr/pull/985#discussion_r950886634
##
pom.xml:
##
@@ -160,7 +160,11 @@
org.slf4j
-slf4j-log4j12
+*
Review Comment:
'*' might be too broad
shangxinli commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r950883464
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##
@@ -109,7 +110,17 @@ public BytesInput decompress(BytesInput bytes, int
uncompressed
shangxinli commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r950883464
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##
@@ -109,7 +110,17 @@ public BytesInput decompress(BytesInput bytes, int
uncompressed
ggershinsky commented on PR #987:
URL: https://github.com/apache/parquet-mr/pull/987#issuecomment-1220261015
This breaks the parquet columnar encryption mode. We use the parquet
"uniform" encryption mode instead for file encryption in Iceberg. Please have a
look at https://github.com/apache
matthieun opened a new pull request, #988:
URL: https://github.com/apache/parquet-mr/pull/988
In case some proto definitions have circular dependencies, the proto schema
converter breaks those and logs a warning, instead of a
`StackOverflowException`.
### Jira
- [x] My PR addr
renshangtao commented on PR #987:
URL: https://github.com/apache/parquet-mr/pull/987#issuecomment-1219439769
@ggershinsky please review it. Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
renshangtao opened a new pull request, #987:
URL: https://github.com/apache/parquet-mr/pull/987
I use iceberg to encryption parquet fileļ¼Then i find It will return "No
encryption setup found for column [c1]".
Looking at the code, I can see that this parameter is fixed to false, which
gszadovszky merged PR #981:
URL: https://github.com/apache/parquet-mr/pull/981
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@parquet.a
parthchandra commented on PR #986:
URL: https://github.com/apache/parquet-mr/pull/986#issuecomment-1218501682
I think one needs this also -
https://www.jetbrains.com/help/idea/reformat-and-rearrange-code.html#keep_existing_formatting
--
This is an automated message from the Apache Git S
theosib-amazon commented on PR #986:
URL: https://github.com/apache/parquet-mr/pull/986#issuecomment-1218488123
@parthchandra This might be the cause of the problem we were encountering
with the whitespace changes.
--
This is an automated message from the Apache Git Service.
To respond to
theosib-amazon opened a new pull request, #986:
URL: https://github.com/apache/parquet-mr/pull/986
Every time I make a PR on this project, I get a whole bunch of complaints
about superfluous whitespace changes that I have to manually revert. Those
changes are caused by a flag in .editorconf
iemejia commented on PR #981:
URL: https://github.com/apache/parquet-mr/pull/981#issuecomment-1218433182
Ah oups sorry for the confusion @sunchao :)
@gszadovszky maybe?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r948043918
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -38,6 +39,34 @@ class SingleBufferInputStream extends ByteBufferInpu
steveloughran commented on PR #985:
URL: https://github.com/apache/parquet-mr/pull/985#issuecomment-1217104595
i've also built against the next release of hadoop, and of 3.4.0-SNAPSHOT.
the parquet build fails there as jackson 1 is purged from the hadoop
classpath, breaking the japicm
steveloughran opened a new pull request, #985:
URL: https://github.com/apache/parquet-mr/pull/985
Hadoop 3.3.3 moved to reload4j for logging to stop
shipping a version of log4j with known (albeit unused)
CVEs.
This bypasses the existing exclusion code used to
keep hadoop's
ggershinsky commented on code in PR #968:
URL: https://github.com/apache/parquet-mr/pull/968#discussion_r946662874
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java:
##
@@ -126,6 +127,42 @@ public class ParquetFileReader implements Closeable {
steveloughran commented on code in PR #983:
URL: https://github.com/apache/parquet-mr/pull/983#discussion_r945594211
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/HadoopInputFile.java:
##
@@ -66,7 +68,13 @@ public long getLength() {
@Override
public Seek
zhongyujiang commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r945428783
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##
@@ -109,7 +110,12 @@ public BytesInput decompress(BytesInput bytes, int
uncompress
sunchao commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r944925723
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##
@@ -109,7 +110,12 @@ public BytesInput decompress(BytesInput bytes, int
uncompressedSiz
sunchao commented on PR #981:
URL: https://github.com/apache/parquet-mr/pull/981#issuecomment-1212528245
I'm not a committer. I think @nandorKollar can do it since he gave +1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub an
dependabot[bot] opened a new pull request, #984:
URL: https://github.com/apache/parquet-mr/pull/984
Bumps hadoop-common from 3.2.3 to 3.2.4.
[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.apache.hadoop:hado
iemejia commented on PR #981:
URL: https://github.com/apache/parquet-mr/pull/981#issuecomment-1212462489
@sunchao maybe?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
iemejia commented on PR #981:
URL: https://github.com/apache/parquet-mr/pull/981#issuecomment-1210998275
Can somebody please merge this one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the sp
sunchao commented on code in PR #983:
URL: https://github.com/apache/parquet-mr/pull/983#discussion_r941644415
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/HadoopInputFile.java:
##
@@ -66,7 +68,13 @@ public long getLength() {
@Override
public SeekableIn
steveloughran commented on code in PR #983:
URL: https://github.com/apache/parquet-mr/pull/983#discussion_r941640533
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/HadoopInputFile.java:
##
@@ -66,7 +68,13 @@ public long getLength() {
@Override
public Seek
sunchao commented on code in PR #983:
URL: https://github.com/apache/parquet-mr/pull/983#discussion_r940452465
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/HadoopInputFile.java:
##
@@ -66,7 +68,13 @@ public long getLength() {
@Override
public SeekableIn
steveloughran opened a new pull request, #983:
URL: https://github.com/apache/parquet-mr/pull/983
This is me looking at what minimal changes could be made to
boost IO performance working with the cloud stores.
Compiles against hadoop 3.3.3; will need hadoop 3.3.5 for some
of the
zhongyujiang commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r939789492
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##
@@ -109,7 +110,12 @@ public BytesInput decompress(BytesInput bytes, int
uncompress
zhongyujiang commented on code in PR #982:
URL: https://github.com/apache/parquet-mr/pull/982#discussion_r939787073
##
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/CodecFactory.java:
##
@@ -109,7 +110,12 @@ public BytesInput decompress(BytesInput bytes, int
uncompress
zhongyujiang opened a new pull request, #982:
URL: https://github.com/apache/parquet-mr/pull/982
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET-2160) issues and references
parthchandra commented on code in PR #968:
URL: https://github.com/apache/parquet-mr/pull/968#discussion_r927128065
##
parquet-hadoop/src/main/java/org/apache/parquet/HadoopReadOptions.java:
##
@@ -61,9 +65,10 @@ private HadoopReadOptions(boolean useSignedStringMinMax,
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934673092
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -88,6 +136,21 @@ public long skip(long n) {
return bytesToSkip;
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934670047
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -38,6 +39,34 @@ class SingleBufferInputStream extends ByteBufferInpu
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934669363
##
parquet-common/src/main/java/org/apache/parquet/bytes/MultiBufferInputStream.java:
##
@@ -379,4 +427,120 @@ public void remove() {
second.remove();
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934668939
##
parquet-common/src/main/java/org/apache/parquet/bytes/MultiBufferInputStream.java:
##
@@ -89,6 +91,15 @@ public long skip(long n) {
return bytesSkipped;
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934649460
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -174,4 +254,64 @@ public boolean markSupported() {
public int ava
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934648806
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -88,6 +136,21 @@ public long skip(long n) {
return bytesToSkip;
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934644315
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -70,9 +105,22 @@ public int read(byte[] bytes, int offset, int lengt
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934641756
##
parquet-common/src/main/java/org/apache/parquet/bytes/MultiBufferInputStream.java:
##
@@ -379,4 +427,120 @@ public void remove() {
second.remove();
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934629584
##
parquet-common/src/main/java/org/apache/parquet/bytes/MultiBufferInputStream.java:
##
@@ -238,8 +257,31 @@ public int read(byte[] bytes, int off, int len) {
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934627078
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -38,6 +39,34 @@ class SingleBufferInputStream extends ByteBufferInpu
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934623419
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -38,6 +39,34 @@ class SingleBufferInputStream extends ByteBufferInpu
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r934619931
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -88,6 +136,21 @@ public long skip(long n) {
return bytesToSkip;
iemejia opened a new pull request, #981:
URL: https://github.com/apache/parquet-mr/pull/981
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Parquet
Jira](https://issues.apache.org/jira/browse/PARQUET-2169) issues and references
them
steveloughran commented on PR #959:
URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1199891512
ypu might want to look at WeakReferences...we've been using them recently to
implement threadlocal-like storage where GCs will trigger cleanup of instances
which aren't being used
sunchao commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r932674870
##
parquet-common/src/main/java/org/apache/parquet/bytes/SingleBufferInputStream.java:
##
@@ -88,6 +136,21 @@ public long skip(long n) {
return bytesToSkip;
}
theosib-amazon commented on PR #959:
URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1195676003
I just thought of something that makes me nervous about this PR that
requires further investigation. Consider the following scenario:
- Thread A allocates a codec
- Thread A
theosib-amazon commented on PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#issuecomment-1195568967
> Is this mostly a refactoring PR? I also don't see
`LittleEndianDataInputStream` being marked as deprecated.
I initially marked `LittleEndianDataInputStream` as deprecated.
ggershinsky commented on PR #978:
URL: https://github.com/apache/parquet-mr/pull/978#issuecomment-1195083014
cc @shangxinli
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
shangxinli merged PR #980:
URL: https://github.com/apache/parquet-mr/pull/980
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@parquet.ap
sunchao commented on PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#issuecomment-1194739358
Is this mostly a refactoring PR? I also don't see
`LittleEndianDataInputStream` being marked as deprecated.
--
This is an automated message from the Apache Git Service.
To respond to
theosib-amazon commented on PR #959:
URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1194259084
I did some poking around. It looks like if you call release() on a codec, it
(a) resets the codec (freeing resources, I think) and (b) returns it to a pool
of codecs without actua
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r929020852
##
parquet-common/src/main/java/org/apache/parquet/bytes/MultiBufferInputStream.java:
##
@@ -379,4 +427,120 @@ public void remove() {
second.remove();
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r929018213
##
parquet-common/src/main/java/org/apache/parquet/bytes/MultiBufferInputStream.java:
##
@@ -238,8 +257,31 @@ public int read(byte[] bytes, int off, int len) {
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r929011710
##
parquet-common/src/main/java/org/apache/parquet/bytes/MultiBufferInputStream.java:
##
@@ -238,8 +257,31 @@ public int read(byte[] bytes, int off, int len) {
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r929010259
##
parquet-common/src/main/java/org/apache/parquet/bytes/ByteBufferInputStream.java:
##
@@ -157,4 +165,80 @@ public void reset() throws IOException {
public b
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r929008981
##
parquet-common/src/main/java/org/apache/parquet/bytes/ByteBufferInputStream.java:
##
@@ -157,4 +165,80 @@ public void reset() throws IOException {
public b
theosib-amazon commented on code in PR #960:
URL: https://github.com/apache/parquet-mr/pull/960#discussion_r929007342
##
parquet-common/src/main/java/org/apache/parquet/bytes/ByteBufferInputStream.java:
##
@@ -157,4 +165,80 @@ public void reset() throws IOException {
public b
501 - 600 of 1570 matches
Mail list logo