[
https://issues.apache.org/jira/browse/HBASE-23326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16998146#comment-16998146
]
Duo Zhang commented on HBASE-23326:
-----------------------------------
{quote}
If master later wants to use filesystem for data that doesn't fit system tables
perhaps because it needed at startup, this would be natural place for it to put
its data beside the local region and WALs.
{quote}
This is exactly the thing I want to avoid... I do not think there are other
data which needs to store like this way, please use system tables?
{quote}
My suggestion is s/MasterProcs/master/. A dir named MasterProcs with a region
and WALs in it is kinda wonky
{quote}
In general, name it as MasterProcs means it contains the master procedure data,
you are free to store them in any format and now, the suitable way is to store
them with the region format. At the first place I even wanted to reuse the old
directory but then I found out that the name is 'MasterProcWALs', which is not
correct as now not only WALs are stored in it, so I named it 'MasterProcs'.
And I need to say again, in my mind there are no other data have to be stored
in this way, so I do not think we should make it general. Correct me if I'm
wrong.
{quote}
Suggestion is just get the WALs into existing archive and use its existing
archiving system (rather than have one for data WALs and another for procedure
WALs). We can write another if we have to because someone wants Master to carry
regions.
{quote}
In general I think this is fine. But after checking the code carefully, this is
not easy in the currect code base. The wal directory and archive directory must
be under the same wal root directory. And now since we changed the wal root
directory to /hbase/MasterProcs, there is no way for us to set the archive
directory to /hbase/oldWALs(unless we set the directory to ../oldWALs but I do
no think this is the correct way, it is like a CVE...). Another way is to write
the WAL to the normal /hbase/WALs directory and then we could use the
/hbase/oldWALs to archive the WAL files but this breaks my assumption above
that all the procedure data should be stored in the 'MasterProcs' so I do not
think this is the correct way.
Anyway, let me see if I can reuse the log cleaner code here.
Thanks.
> Implement a ProcedureStore which stores procedures in a HRegion
> ---------------------------------------------------------------
>
> Key: HBASE-23326
> URL: https://issues.apache.org/jira/browse/HBASE-23326
> Project: HBase
> Issue Type: Improvement
> Components: proc-v2
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Priority: Critical
> Fix For: 3.0.0, 2.3.0
>
>
> So we can resue the code in HRegion for persisting the procedures, and also
> the optimized WAL implementation for better performance.
> This requires we merge the hbase-procedure module to hbase-server, which is
> an anti-pattern as we make the hbase-server module more overloaded. But I
> think later we can first try to move the WAL stuff out.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
