richardstartin commented on a change in pull request #7629:
URL: https://github.com/apache/pinot/pull/7629#discussion_r735872146
##########
File path:
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/fwd/MultiValueVarByteRawIndexCreator.java
##########
@@ -67,13 +67,15 @@ public MultiValueVarByteRawIndexCreator(File baseIndexDir,
ChunkCompressionType
* @param totalDocs Total number of documents to index
* @param valueType Type of the values
* @param maxRowLengthInBytes the size in bytes of the largest row, the
chunk size cannot be smaller than this
+ * @param maxNumberOfElements the maximum number of elements in a row
* @param writerVersion writer format version
*/
- public MultiValueVarByteRawIndexCreator(File baseIndexDir,
ChunkCompressionType compressionType,
- String column, int totalDocs, DataType valueType, int writerVersion, int
maxRowLengthInBytes)
+ public MultiValueVarByteRawIndexCreator(File baseIndexDir,
ChunkCompressionType compressionType, String column,
+ int totalDocs, DataType valueType, int writerVersion, int
maxRowLengthInBytes, int maxNumberOfElements)
throws IOException {
//we will prepend the actual content with numElements and length array
containing length of each element
- int totalMaxLength = Integer.BYTES + Math.max(maxRowLengthInBytes,
TARGET_MAX_CHUNK_SIZE);
+ int maxLengthPrefixes = Integer.BYTES * maxNumberOfElements;
+ int totalMaxLength = Integer.BYTES + Math.max(maxRowLengthInBytes +
maxLengthPrefixes, TARGET_MAX_CHUNK_SIZE);
Review comment:
I think I see what you're picking up on, `totalMaxLength` should not be
used down on line 82, which is the actual bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
