[
https://issues.apache.org/jira/browse/HBASE-11842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-11842:
----------------------------------
Attachment: hbase-11842_v1.patch
Attaching a patch for an integration test for the async wal replicas feature.
This single test covers most of the testing HBASE-10070 Phase 2 items including
Async WAL replication, memory pressure handling, and failover handling. Using
this test allowed us to uncover a couple of fixes that are already fixed in the
patches, or upcoming in the follow up patches (HBASE-12561, HBASE-11332,
HBASE-11908, etc).
The test runs LoadTestTool with N writes and M reader threads. The logic for
writer and readers (key distribution, verification, etc) is used as it is. The
test initializes a table with replicas and also assumes that async wal
replication is enabled in the cluster (via configuration).
The writer threads pick up keys to write and they perform the writes to the
table via primary regions. The expectation is that async wal replication will
replicate the modified keys shortly after to the region replicas. There is a
queue between writers and readers in LTT where the writers pick up keys to
write, and when the write is complete, they put the key to the queue so that
readers pick it up from there. This test simply changes the queue to be a delay
queue, which introduces a configurable amount of delay (for example 30 sec)
between a writer thread putting the key to the queue, and making it available
to a reader thread. This means that the readers are reading keys written 30 sec
or more before. The readers in the test read the keys from a region replica an
verify. If the async wal replication is broken, or there is data loss between
replicas, the test will fail. The test can also run chaos monkey.
> Integration test for async wal replication to secondary regions
> ---------------------------------------------------------------
>
> Key: HBASE-11842
> URL: https://issues.apache.org/jira/browse/HBASE-11842
> Project: HBase
> Issue Type: Sub-task
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 2.0.0
>
> Attachments: hbase-11842_v1.patch
>
>
> We should add an integration test to cover async wal replication to secondary
> regions.
> Async wal replication to region replicas has been committed (HBASE-11568) and
> flush /compaction handling (HBASE-11569) and failover handling for secondary
> region replicas (HBASE-11580) is in the works.
> The test can use LoadTestTool over a table with region replicas. We can write
> to the primary regions, and read from the secondary region replicas in reader
> threads to do the verification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
