Re: PackedInts functionalities

2023-10-17 Thread Dongyu Xu
Thank you both for your ideas. I sensed that there could be potential confusions. Let me try to clear them first -- I'm not changing how postings are encoded. Rather I'm changing how the postings's metadata (in code, this will be the IntBlockTermState

ByteBufferIndexInput.alreadyClosed creates an exception that doesn't track its cause

2023-10-17 Thread Michael Sokolov
I was messing around with something that was resulting in AlreadyClosedException being thrown and I noticed that we weren't tracking the exception that caused it. I found this in ByteBufferIndexInput: // the unused parameter is just to silence javac about unused variables AlreadyClosedExcept

Re: PackedInts functionalities

2023-10-17 Thread Adrien Grand
+1 to what Mikhail wrote, this is e.g. how postings work: instead of interleaving doc IDs and frequencies, they always store a block of 128 doc IDs followed by a block of 128 frequencies. For reference, bit packing feels space-inefficient for this kind of data. I would expect docFreqs to have a zi