[
https://issues.apache.org/jira/browse/HBASE-8497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13664813#comment-13664813
]
Himanshu Vashishtha commented on HBASE-8497:
--------------------------------------------
bq. I responded to your response, it seems if the trailer size is 0 with
magic/size present it would point to one byte beyond.
No it will not; with the latest patch, lastPositionToRead points to the last
byte of the WALEdit.
I ran a patched rs and read a hlog:
{code}
2013-05-22 17:11:23,400 DEBUG wal.ProtobufLogReader
(ProtobufLogReader.java:initInternal(118)) - After reading the trailer:
lastPositionToRead: 372, fileLength: 380, trailerPresent: true
{code}
Note, magic is 4 bytes [397-380], trailer size is 4 bytes [393-396]. The
trailer size is 0; so last byte of the WALEdit is at 392, and
lastPositionToRead is pointing to 392 only.
Re making it configurable vs constant: I set the default value to 1MB.
Currently, the one use case I have is adding Region level stats at the end. It
shouldn't be more than 150 bytes per region (region name + long value). If we
store CF level flush sequence Ids also, then it would be about 500-600 bytes.
This means a scope of about 2000 regions on one server without showing the
warning message. I think it is pretty decent number (usually we tend to advice
not more than 100 or so regions).
Left some comments on rb too. Thanks for the reviews.
> Protobuf WAL also needs a trailer
> ----------------------------------
>
> Key: HBASE-8497
> URL: https://issues.apache.org/jira/browse/HBASE-8497
> Project: HBase
> Issue Type: Sub-task
> Components: Protobufs, wal
> Affects Versions: 0.95.1
> Reporter: Enis Soztutar
> Assignee: Himanshu Vashishtha
> Fix For: 0.98.0, 0.95.1
>
> Attachments: HBASE-8497-v0.patch, HBASE-8497-v2.patch,
> HBASE-8497-v3.patch, HBASE-8497-v4.patch, HBASE-8497-v5.patch
>
>
> New Protobuf WAL has a header, but we will probably need a trailer as well,
> reserved for later usage.
> Right now, we can we just serialize an empty trailer, but putting more
> metadata there, like range of sequence_id's, region names, table names etc
> might be needed in the future.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
