[
https://issues.apache.org/jira/browse/HBASE-23326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997897#comment-16997897
]
Duo Zhang commented on HBASE-23326:
-----------------------------------
{quote}
I'll try to get further comments into the context of the doc, but we absolutely
must have archiving for this to go into a branches-2. I've had too many issues
with things that thought they could skip archiving in the last 6 months already.
{quote}
What's the problem? Need get the WAL back for recovery or debugging?
{quote}
Sure. Was suggesting you use a more generic name, do should Master need to
persist other info, we could group it under a master dir
(s/MasterProcs/master/).
{quote}
This is what I want to avoid... Just store the data in system tables. Why we
have to use a local region is because that hbase:meta has not been intialized
when we loading procedures, so it should not use hbase:meta or other system
tables to store the data. It is fine to change the namespace to master, no
problem, but I still do not want to make it generic. Do you have any ideas
which data will be stored in the 'local' HRegion, besides procedure?
{quote}
RSS is an Interface. Was suggesting you pass an impl that only did the subset
needed to run local Region. I suggest this so local Region looks like any other
Region in the system.
{quote}
This does not work. Usually we just test whether the RegionServerServices is
null and then do something, so a non-null RegionServerServices actually means
we will enable lots of features. I think this can be done in the future, but we
need to refactor HRegion first if we want to do this.
{quote}
The WALs are differently named?
IIRC, just moving the WALs into the WALs dir gets them considered by the
general archiving system; no extra work to be done beyond the move.
{quote}
They are in the same name pattern so it will be a problem if master carrying
table, as it may overwrite the WAL file for master. Since we want to isolate
the procedure store data, let's implement the archive logic by our own, it is
easy and can use different configs comparing to normal wals.
{quote}
I suppose if out-of-order, once read into member, the procedures will be
ordered... .and we won't have odd stuff like seeing a subprocedure before its
parent (shouldn't matter).
{quote}
No, no problem. We will load all the procedures to memory first, and then build
the ProcedureTree, and then pass the procedures to the ProcedureLoader. You can
see the code in ProcedureTree for more details.
{quote}
Would be sweet if we could just move the WALs into recovered.edits dir (it is
named something else as flag to trigger different behavior? OK....).
{quote}
Could be done in the future when we refactor HRegion? This needs a more high
level design to unify the different scenarios.
Thanks.
> Implement a ProcedureStore which stores procedures in a HRegion
> ---------------------------------------------------------------
>
> Key: HBASE-23326
> URL: https://issues.apache.org/jira/browse/HBASE-23326
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Priority: Critical
> Fix For: 3.0.0, 2.3.0
>
>
> So we can resue the code in HRegion for persisting the procedures, and also
> the optimized WAL implementation for better performance.
> This requires we merge the hbase-procedure module to hbase-server, which is
> an anti-pattern as we make the hbase-server module more overloaded. But I
> think later we can first try to move the WAL stuff out.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
