clintropolis opened a new pull request #6045: Fix 'auto' encoded longs + compression serializer URL: https://github.com/apache/incubator-druid/pull/6045 Fixes #6044 by changing `VSizeLongSerde` serializers to treat `close` as a 'flush' when used with `BlockLayoutColumnarLongsSerializer`, allowing unwritten values to get added to the buffer when the block is compressed to the write out medium. This seemed less heavy handed than creating new serializers for each block that is flushed, but I'm open to other ideas. To help catch further issues like this, the only thing I could think of that would make me feel reasonably confident about stuff was to add a rather exhaustive test that tries a variety of 'bits per value', number of rows, and compression strategies, but it unfortunately adds a minute or two to overall test time. ``` Running io.druid.segment.data.CompressedLongsAutoEncodingSerdeTest Tests run: 104, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 58.116 sec - in io.druid.segment.data.CompressedLongsAutoEncodingSerdeTest ``` It isn't even completely exhaustive, as the behavior of `IntermediateColumnarLongsSerializer` isn't directly controllable, but both table and delta encodings are exercised. I'm considering marking it `@Ignore`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
