Jackie-Jiang commented on a change in pull request #7629:
URL: https://github.com/apache/pinot/pull/7629#discussion_r735894917
##########
File path:
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/fwd/MultiValueFixedByteRawIndexCreator.java
##########
@@ -71,33 +67,23 @@ public MultiValueFixedByteRawIndexCreator(File
baseIndexDir, ChunkCompressionTyp
* @param column Name of column to index
* @param totalDocs Total number of documents to index
* @param valueType Type of the values
- * @param maxLengthOfEachEntry length of longest entry (in bytes)
* @param deriveNumDocsPerChunk true if writer should auto-derive the number
of rows per chunk
* @param writerVersion writer format version
*/
public MultiValueFixedByteRawIndexCreator(File baseIndexDir,
ChunkCompressionType compressionType,
- String column, int totalDocs, DataType valueType, final int
maxLengthOfEachEntry,
- final int maxNumberOfMultiValueElements, boolean deriveNumDocsPerChunk,
- int writerVersion)
+ String column, int totalDocs, DataType valueType, int
maxNumberOfMultiValueElements,
+ boolean deriveNumDocsPerChunk, int writerVersion)
throws IOException {
- File file = new File(baseIndexDir,
- column + Indexes.RAW_MV_FORWARD_INDEX_FILE_EXTENSION);
- FileUtils.deleteQuietly(file);
- int totalMaxLength = maxNumberOfMultiValueElements * maxLengthOfEachEntry;
+ File file = new File(baseIndexDir, column +
Indexes.RAW_MV_FORWARD_INDEX_FILE_EXTENSION);
+ int totalMaxLength =
VarByteChunkSVForwardIndexWriter.CHUNK_HEADER_ENTRY_ROW_OFFSET_SIZE
Review comment:
The `VarByteChunkSVForwardIndexWriter` expects `lengthOfLongestEntry`
which does not include the chunk header offset. `lengthOfLongestEntry =
maxNumberOfMultiValueElements * valueType.size()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
