pitrou commented on code in PR #37792:
URL: https://github.com/apache/arrow/pull/37792#discussion_r1338141293
##########
cpp/src/arrow/array/builder_binary.h:
##########
@@ -463,6 +464,238 @@ class ARROW_EXPORT LargeStringBuilder : public
LargeBinaryBuilder {
std::shared_ptr<DataType> type() const override { return large_utf8(); }
};
+// ----------------------------------------------------------------------
+// BinaryViewBuilder, StringViewBuilder
+//
+// These builders do not support building raw pointer view arrays.
+
+namespace internal {
+
+// We allocate medium-sized memory chunks and accumulate data in those, which
+// may result in some waste if there are many large-ish strings. If a string
+// comes along that does not fit into a block, we allocate a new block and
+// write into that.
+//
+// Later we can implement optimizations to continuing filling underfull blocks
+// after encountering a large string that required allocating a new block.
+class ARROW_EXPORT StringHeapBuilder {
+ public:
+ static constexpr int64_t kDefaultBlocksize = 32 << 10; // 32KB
Review Comment:
Hmm, I see. Perhaps we can keep chunking but with a default very large chunk
size, then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
