pitrou commented on a change in pull request #11123:
URL: https://github.com/apache/arrow/pull/11123#discussion_r710897414
##########
File path: cpp/src/parquet/thrift_internal.h
##########
@@ -363,12 +364,26 @@ static inline format::EncryptionAlgorithm
ToThrift(EncryptionAlgorithm encryptio
using ThriftBuffer = apache::thrift::transport::TMemoryBuffer;
+// On Thrift 0.14.0+, we want to use TConfiguration to raise the max message
size
+// limit (ARROW-13655). If we wanted to protect against huge messages, we
could
+// do it ourselves since we know the message size up front.
+
+inline std::shared_ptr<ThriftBuffer> CreateReadOnlyMemoryBuffer(uint8_t* buf,
+ uint32_t len) {
+#if PARQUET_THRIFT_VERSION_MAJOR > 0 || PARQUET_THRIFT_VERSION_MINOR >= 14
+ auto conf = std::make_shared<apache::thrift::TConfiguration>();
+ conf->setMaxMessageSize(std::numeric_limits<int>::max());
+ return std::make_shared<ThriftBuffer>(buf, len, ThriftBuffer::OBSERVE, conf);
Review comment:
See a bit below. `setStringSizeLimit` and `setContainerSizeLimit` should
protect us against Thrift memory bombs, IIRC. However, we don't have any
protection against compression memory bombs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
