wgtmac commented on code in PR #1292:
URL: https://github.com/apache/parquet-mr/pull/1292#discussion_r1514562564
##########
parquet-hadoop/.pom.xml.swp:
##########
Review Comment:
What is this?
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnChunkPageReadStore.java:
##########
@@ -168,12 +168,7 @@ public DataPage visit(DataPageV1 dataPageV1) {
decompressedBuffer,
dataPageV1.getUncompressedSize());
setDecompressMetrics(bytes, start);
-
- // HACKY: sometimes we need to do `flip` because the position of
output bytebuffer is
Review Comment:
Should we keep the comment?
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/DirectCodecFactory.java:
##########
@@ -68,7 +68,8 @@ class DirectCodecFactory extends CodecFactory implements
AutoCloseable {
try {
tempClass =
Class.forName("org.apache.hadoop.io.compress.DirectDecompressionCodec");
tempCreateMethod = tempClass.getMethod("createDirectDecompressor");
- tempDecompressMethod = tempClass.getMethod("decompress",
ByteBuffer.class, ByteBuffer.class);
+ Class<?> tempClass2 =
Class.forName("org.apache.hadoop.io.compress.DirectDecompressor");
+ tempDecompressMethod = tempClass2.getMethod("decompress",
ByteBuffer.class, ByteBuffer.class);
Review Comment:
Is the old behavior a bug?
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/DirectCodecFactory.java:
##########
@@ -150,27 +151,25 @@ private IndirectDecompressor(Decompressor decompressor) {
}
@Override
- public BytesInput decompress(BytesInput bytes, int uncompressedSize)
throws IOException {
+ public BytesInput decompress(BytesInput bytes, int decompressedSize)
throws IOException {
decompressor.reset();
byte[] inputBytes = bytes.toByteArray();
decompressor.setInput(inputBytes, 0, inputBytes.length);
- byte[] output = new byte[uncompressedSize];
- decompressor.decompress(output, 0, uncompressedSize);
+ byte[] output = new byte[decompressedSize];
+ decompressor.decompress(output, 0, decompressedSize);
return BytesInput.from(output);
}
@Override
- public void decompress(ByteBuffer input, int compressedSize, ByteBuffer
output, int uncompressedSize)
+ public void decompress(ByteBuffer input, int compressedSize, ByteBuffer
output, int decompressedSize)
throws IOException {
decompressor.reset();
byte[] inputBytes = new byte[compressedSize];
- input.position(0);
Review Comment:
It seems that the new contract consumes the input from current position.
Will it break the legacy code depending on it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
