buchireddy commented on issue #4318: #4317 Feature/variable length bytes offline dictionary URL: https://github.com/apache/incubator-pinot/pull/4318#issuecomment-502547203 @mcvsubbu I did look at `MutableOffheapByteArrayStore` and took some concepts from it while implementing `VarLengthBytesValueReaderWriter` too. However, I haven't used the former fully because of the following reasons: * In order to support a seamless transition from fixed length to variable length offline dictionary, we need a special magic header to be added at the beginning, which needs some changes to `MutableOffheapByteArrayStore`. * `MutableOffheapByteArrayStore` is handling the buffer expansions, multiple buffers logic because it needs to be mutable but the immutable one doesn't need all those. So, it's a trade-off b/w simple code vs avoiding the code duplication (though we anyways need a new class for variable length dictionary class implementation). I'm open for debate and for further brainstorming. P.S: This is my first time going through Pinot code so please let me know if I'm missing any aspects here.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
