Fokko commented on code in PR #3359:
URL: https://github.com/apache/parquet-java/pull/3359#discussion_r2555202645
##########
parquet-format-structures/src/main/java/org/apache/parquet/format/Util.java:
##########
@@ -358,26 +391,56 @@ public void consume(RowGroup rowGroup) {
byte[] plainText = decryptor.decrypt(input, AAD);
from = new ByteArrayInputStream(plainText);
}
- new EventBasedThriftReader(protocol(from)).readStruct(eventConsumer);
+ new EventBasedThriftReader(protocol(from,
maxMessageSize)).readStruct(eventConsumer);
} catch (TException e) {
throw new IOException("can not read FileMetaData: " + e.getMessage(), e);
}
}
private static TProtocol protocol(OutputStream to) throws
TTransportException {
- return protocol(new TIOStreamTransport(to));
+ return protocol(new TIOStreamTransport(to), -1);
}
private static TProtocol protocol(InputStream from) throws
TTransportException {
- return protocol(new TIOStreamTransport(from));
+ return protocol(new TIOStreamTransport(from), -1);
+ }
+
+ private static TProtocol protocol(InputStream from, int maxMessageSize)
throws TTransportException {
+ return protocol(new TIOStreamTransport(from), maxMessageSize);
}
- private static InterningProtocol protocol(TIOStreamTransport t) {
+ private static InterningProtocol protocol(TIOStreamTransport t, int
configuredMaxMessageSize)
+ throws TTransportException, NumberFormatException {
+
+ int maxMessageSize = 104857600; // Default to 100 MB
+ if (configuredMaxMessageSize > 0) {
+ maxMessageSize = configuredMaxMessageSize;
+ } else if (configuredMaxMessageSize == -1) {
+ // Set to default 100 MB
+ maxMessageSize = 104857600;
Review Comment:
I think we set the default here twice. Can we break out the default to a
static constant?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
