mike-tr-adamson commented on code in PR #2989:
URL: https://github.com/apache/cassandra/pull/2989#discussion_r1430134005
##########
src/java/org/apache/cassandra/index/sai/disk/v1/segment/SegmentBuilder.java:
##########
@@ -112,54 +114,31 @@ protected SegmentMetadata.ComponentMetadataMap
flushInternal(IndexDescriptor ind
indexIdentifier,
indexTermType.fixedSizeOf(),
maxSegmentRowId);
- return writer.writeCompleteSegment(trieBuffer.iterator());
+ return writer.writeCompleteSegment(segmentRamBuffer.iterator());
}
}
- public static class RAMStringSegmentBuilder extends SegmentBuilder
+ public static class LiteralSegmentBuilder extends RamBufferSegmentBuilder
{
- final RAMStringIndexer ramIndexer;
-
- final BytesRefBuilder stringBuffer = new BytesRefBuilder();
-
- public RAMStringSegmentBuilder(IndexTermType indexTermType,
NamedMemoryLimiter limiter)
+ public LiteralSegmentBuilder(IndexTermType indexTermType,
NamedMemoryLimiter limiter)
{
super(indexTermType, limiter);
-
- ramIndexer = new RAMStringIndexer();
- totalBytesAllocated = ramIndexer.estimatedBytesUsed();
- }
-
- @Override
- public boolean isEmpty()
- {
- return ramIndexer.isEmpty();
}
@Override
protected long addInternal(ByteBuffer term, int segmentRowId)
{
- copyBufferToBytesRef(term, stringBuffer);
- return ramIndexer.add(stringBuffer.get(), segmentRowId);
+ return segmentRamBuffer.add(v -> ByteSource.fixedLength(term),
term.limit(), segmentRowId);
Review Comment:
Some incorrect assumptions were going on here. The `isLiteral` flag was
being used as a catch-all for all types here. After looking at it, it is only
frozen types that need any special handling, and they only need to use
`ByteSource.of`. I have pushed this down to `IndexTermType.asComparableBytes`
to handle.
This greatly simplifies things, in the `TrieMemoryIndex` and in the
`SegmentBuilder`, since we don't need to handle custom translations of the
bytes in multiple places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
