mcvsubbu commented on a change in pull request #5400:
URL: https://github.com/apache/incubator-pinot/pull/5400#discussion_r426973388
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/io/reader/impl/v1/BaseChunkSingleValueReader.java
##########
@@ -75,7 +67,7 @@ public BaseChunkSingleValueReader(PinotDataBuffer
pinotDataBuffer) {
headerOffset += Integer.BYTES;
int dataHeaderStart = headerOffset;
- if (_version > 1) {
+ if (version > 1) {
Review comment:
The version was made into a member variable so that when we add or bump
the version it is available to derived classes. Not sure why you removed it.
Recently we made a version bump and it was painful and re-factoring had to
be done. Some background in PR #5285
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/io/readerwriter/impl/FixedByteSingleColumnSingleValueReaderWriter.java
##########
@@ -176,18 +173,15 @@ private int getBufferId(int row) {
private void addBuffer() {
LOGGER.info("Allocating {} bytes for: {}", _chunkSizeInBytes,
_allocationContext);
+ // NOTE: PinotDataBuffer is tracked in the PinotDataBufferMemoryManager.
No need to track it inside the class.
PinotDataBuffer buffer = _memoryManager.allocate(_chunkSizeInBytes,
_allocationContext);
- _dataBuffers.add(buffer);
+ _writers.add(
Review comment:
when are these closed?
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/io/readerwriter/impl/FixedByteSingleColumnMultiValueReaderWriter.java
##########
@@ -132,47 +129,40 @@ public FixedByteSingleColumnMultiValueReaderWriter(int
maxNumberOfMultiValuesPer
_maxNumberOfMultiValuesPerRow = maxNumberOfMultiValuesPerRow;
_headerSize = rowCountPerChunk * SIZE_OF_INT * NUM_COLS_IN_HEADER;
_rowCountPerChunk = rowCountPerChunk;
- addHeaderBuffers();
+ addHeaderBuffer();
//at least create space for million entries, which for INT translates into
4mb buffer
_incrementalCapacity = incrementalCapacity;
- addDataBuffers(initialCapacity);
+ addDataBuffer(initialCapacity);
//init(_rowCountPerChunk, _columnSizeInBytes,
_maxNumberOfMultiValuesPerRow, initialCapacity, _incrementalCapacity);
}
- private void addHeaderBuffers() {
+ private void addHeaderBuffer() {
LOGGER.info("Allocating header buffer of size {} for: {}", _headerSize,
_context);
- _headerBuffer = _memoryManager.allocate(_headerSize, _context);
+ // NOTE: PinotDataBuffer is tracked in the PinotDataBufferMemoryManager.
No need to track it inside the class.
+ PinotDataBuffer headerBuffer = _memoryManager.allocate(_headerSize,
_context);
Review comment:
Not sure I understand this. is headerBuffer(s) alllocated here closed
anywhere? Same q with databBuffers . The reader and writer classes we construct
here do not close these buffers, right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
