[
https://issues.apache.org/jira/browse/HBASE-14623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205096#comment-15205096
]
stack commented on HBASE-14623:
-------------------------------
bq. For system table to have its own WAL, we would recover system table faster
(fast log split, fast log replay).
Says above in the description. Why would we recover the system tables 'faster'?
Why would log split and replay be 'faster'?
bq. It would probably benefit AssignmentManager on system table region
assignment.
How would it 'probably' benefit AM...
And reading further down, I see you are actually copy/pasting someone else's
words (Stephen's...) up into the description. Is it because you cannot explain
why we should do this issue yourself?
One concernt of mine from above I don't see answered:
"....Should we compound all system tables so assignment is easier rather than
have a small system table per domain each with their own heavyweight assignment
and a complex required ordering of assign each with their own split to run?
Should we be propagating an exception in our WAL system? And so on. Showing up
with a patch that complicates an already difficult startup without
discussion/justification other than someone suggested it is going to get some
pushback...."
In particular, how will this not slow down assign (if each system table has to
wait on its own log to finish split.... we are not serializing something that
was previously done in parallel....) Any testing done?
bq. I think most system table is one region table (currently, only ACL table
could become big).
That is currently the case, yes, but a much-discussed option is that system
tables may have many regions (e.g. a splittalble hbase:meta).
There is no commentary explaining how this patch is supposed to work.
So, the meta WAL handling goes untouched? We in effect copy/paste the
hbase:meta handling? There is a logroller for user-space WALs, one for meta
and then another for system tables? Why not one logroller? We have enough
threads running in the system already.
So, hbase:meta file gets a .meta ending but 'system' WALs -- namespace, ACL --
get a '.sys' ending? Does that mean .meta is not a sys table?
No tests?
> Implement dedicated WAL for system tables
> -----------------------------------------
>
> Key: HBASE-14623
> URL: https://issues.apache.org/jira/browse/HBASE-14623
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Reporter: Ted Yu
> Assignee: Ted Yu
> Labels: wal
> Fix For: 2.0.0
>
> Attachments: 14623-v1.txt, 14623-v2.txt, 14623-v2.txt, 14623-v2.txt,
> 14623-v2.txt, 14623-v3.txt, 14623-v4.txt
>
>
> As Stephen suggested in parent JIRA, dedicating separate WAL for system
> tables (other than hbase:meta) should be done in new JIRA.
> This task is to fulfill the system WAL separation.
> Below is summary of discussion:
> For system table to have its own WAL, we would recover system table faster
> (fast log split, fast log replay). It would probably benefit
> AssignmentManager on system table region assignment. At this time, the new
> AssignmentManager is not planned to change WAL. So the existence of this JIRA
> is good for overall system, not specific to AssignmentManager.
> There are 3 strategies for implementing system table WAL:
> 1. one WAL for all non-meta system tables
> 2. one WAL for each non-meta system table
> 3. one WAL for each region of non-meta system table
> Currently most system tables are one region table (only ACL table may become
> big). Choices 2 and 3 basically are the same.
> From implementation point of view, choices 2 and 3 are cleaner than choice 1
> (as we have already had 1 WAL for META table and we can reuse the logic).
> With choice 2 or 3, assignment manager performance should not be impacted and
> it would be easier for assignment manager to assign system table region (eg.
> without waiting for user table log split to complete for assigning system
> table region).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
