gortiz commented on code in PR #10528:
URL: https://github.com/apache/pinot/pull/10528#discussion_r1211291187
##########
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/memory/PinotDataBuffer.java:
##########
@@ -99,6 +104,101 @@ public String toString() {
private static final AtomicLong ALLOCATION_FAILURE_COUNT = new AtomicLong();
private static final Map<PinotDataBuffer, BufferContext> BUFFER_CONTEXT_MAP
= new WeakHashMap<>();
+ /**
+ * Configuration key used to change the offheap buffer factory used by Pinot.
+ * Value should be the qualified path of a class that extends {@link
PinotBufferFactory} and has empty
+ * constructor.
+ */
+ private static final String OFFHEAP_BUFFER_FACTORY_CONFIG =
"pinot.offheap.buffer.factory";
+ /**
+ * Boolean configuration that decides whether to allocate using {@link
ByteBufferPinotBufferFactory} when the buffer
+ * to allocate fits in a {@link ByteBuffer}.
+ *
+ * Defaults to true.
+ */
+ private static final String OFFHEAP_BUFFER_PRIORITIZE_BYTE_BUFFER_CONFIG =
"pinot.offheap.prioritize.bytebuffer";
Review Comment:
Yes, although it can be used with the older implementation as well.
I don't expect this config to be changed from their default value in the
near future. Benchmarks show that the new implementation is also slower than
the byte buffers inlining when dealing with batches or loops.
In other words, I don't expect this property to be used in production until
the Foreign Memory API is ready to use. But I think it is useful to be there
just in case we want to do end to end benchmarks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
