[
https://issues.apache.org/jira/browse/HBASE-13260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516303#comment-14516303
]
Enis Soztutar commented on HBASE-13260:
---------------------------------------
[~mbertozzi] am I missing anything? I've ran the test, and here are the
results. It is very little difference between the approaches:
{code}
java.lang.AssertionError: Wrote 1000000 procedures with 5 threads with
useProcV2Wal=true hsync=false in 50.4700sec (50.470sec)
java.lang.AssertionError: Wrote 1000000 procedures with 5 threads with
useProcV2Wal=false hsync=false in 14.9200sec (14.920sec)
java.lang.AssertionError: Wrote 1000000 procedures with 10 threads with
useProcV2Wal=true hsync=false in 34.0240sec (34.024sec)
java.lang.AssertionError: Wrote 1000000 procedures with 10 threads with
useProcV2Wal=false hsync=false in 15.4410sec (15.441sec)
java.lang.AssertionError: Wrote 1000000 procedures with 30 threads with
useProcV2Wal=true hsync=false in 20.8070sec (20.807sec)
java.lang.AssertionError: Wrote 1000000 procedures with 30 threads with
useProcV2Wal=false hsync=false in 24.7590sec (24.759sec)
java.lang.AssertionError: Wrote 1000000 procedures with 50 threads with
useProcV2Wal=false hsync=false in 18.2410sec (18.241sec)
java.lang.AssertionError: Wrote 1000000 procedures with 50 threads with
useProcV2Wal=true hsync=false in 20.1100sec (20.110sec)
{code}
> Bootstrap Tables for fun and profit
> ------------------------------------
>
> Key: HBASE-13260
> URL: https://issues.apache.org/jira/browse/HBASE-13260
> Project: HBase
> Issue Type: Bug
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.1.0
>
> Attachments: hbase-13260_bench.patch, hbase-13260_prototype.patch
>
>
> Over at the ProcV2 discussions(HBASE-12439) and elsewhere I was mentioning an
> idea where we may want to use regular old regions to store/persist some data
> needed for HBase master to operate.
> We regularly use system tables for storing system data. acl, meta, namespace,
> quota are some examples. We also store the table state in meta now. Some data
> is persisted in zk only (replication peers and replication state, etc). We
> are moving away from zk as a permanent storage. As any self-respecting
> database does, we should store almost all of our data in HBase itself.
> However, we have an "availability" dependency between different kinds of
> data. For example all system tables need meta to be assigned first. All
> master operations need ns table to be assigned, etc.
> For at least two types of data, (1) procedure v2 states, (2) RS groups in
> HBASE-6721 we cannot depend on meta being assigned since "assignment" itself
> will depend on accessing this data. The solution in (1) is to implement a
> custom WAL format, and custom recover lease and WAL recovery. The solution in
> (2) is to have the table to store this data, but also cache it in zk for
> bootrapping initial assignments.
> For solving both of the above (and possible future use cases if any), I
> propose we add a "boostrap table" concept, which is:
> - A set of predefined tables hosted in a separate dir in HDFS.
> - A table is only 1 region, not splittable
> - Not assigned through regular assignment
> - Hosted only on 1 server (typically master)
> - Has a dedicated WAL.
> - A service does WAL recovery + fencing for these tables.
> This has the benefit of using a region to keep the data, but frees us to
> re-implement caching and we can use the same WAL / Memstore / Recovery
> mechanisms that are battle-tested.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
