[
https://issues.apache.org/jira/browse/HBASE-7413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633332#comment-13633332
]
Sergey Shelukhin commented on HBASE-7413:
-----------------------------------------
The comments above add to almost a page ;)
bq. Do we have to make our own format? Can we not reuse?
We use protobufs. The only additions on top of that is: we don't want to put
KVs in protobuf, so we write them using writables as before; and we need magic
at start to tell the new files apart from old (old ones have different magic).
What do you have in mind to reuse?
bq. Ideally, can take a protobuf multi message off rpc w/ cellBlocks as is and
just put it into the WAL w/o unmarshalling/marshalling making new types
WAL RPC message currently has byte buffer for KVs. That would result in extra
copy of KVs... I did lift most of the Admin.proto RPC stuff, with few changes.
bq. (cellBlocks have codecs now and they can be compressed – would be sweet if
they retained their compression/codec when in WAL – would mean though that the
compression/codec used would have to be associated w/ the WAL entry since it is
currently only in the Connection header for the particular client server-side)
Compression is retained.
bq. If we could do away with the HLog entry key, that'd be cool but it has info
such as target region, replication...
I actually moved replication scopes from WALEdit to HLogKey, so WALEdit is just
KVs. I'd say if we have some reasonable way to put KVs into PB w/o overhead
there should just be one structure because WALEdit and HLogKey are used
together everywhere for practical purposes.
Another obstacle to uniting them is all the code that uses them of course :) So
this should be a separate JIRA, I did leave a TODO
> Convert WAL to pb
> -----------------
>
> Key: HBASE-7413
> URL: https://issues.apache.org/jira/browse/HBASE-7413
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Reporter: stack
> Assignee: Sergey Shelukhin
> Priority: Critical
> Fix For: 0.95.1
>
>
> From HBASE-7201
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
