apurtell commented on a change in pull request #3244:
URL: https://github.com/apache/hbase/pull/3244#discussion_r629763212
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java
##########
@@ -302,14 +343,28 @@ protected Cell parseCell() throws IOException {
compression.getDictionary(CompressionContext.DictionaryIndex.QUALIFIER));
pos += elemLen;
- // timestamp, type and value
- int tsTypeValLen = length - pos;
+ // timestamp
+ long ts = StreamUtils.readLong(in);
+ pos = Bytes.putLong(backingArray, pos, ts);
+ // type and value
+ int typeValLen = length - pos;
if (tagsLength > 0) {
- tsTypeValLen = tsTypeValLen - tagsLength - KeyValue.TAGS_LENGTH_SIZE;
+ typeValLen = typeValLen - tagsLength - KeyValue.TAGS_LENGTH_SIZE;
+ }
+ // high bit of type byte is 1 if value is compressed
+ byte type = (byte)in.read();
+ if ((type & 0x80) == 0x80) {
Review comment:
We decide at each value whether to store it compressed or not. First,
there is a threshold. Too small and there is no point. Even so, sometimes
values will not compress, even though value compression is enabled and by size
it is eligible. If this happens we throw the compressed version away and just
store the original.
> Isn't it better to just zlib compress all the values? WDYT.
Some value data will not compress but we won't know ahead of time.
I'm not strongly wedded to this idea but it seems good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
