[
https://issues.apache.org/jira/browse/HADOOP-15446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Marquardt updated HADOOP-15446:
--------------------------------------
Release Note: WASB: Bug fix to support non-sequential page blob reads.
Required for HBASE replication.
Status: Patch Available (was: Open)
The patch is attached as HADOOP-15446-001.patch. The patch adds 11 new tests.
In particular it adds tests that use PageBlobInputStream.skip and
PageBlobInputStream.seek to perform random reads and validate that page blobs
now support random reads. The patch was also validated against an HBASE
replication scenario.
All WASB test cases are passing against my Azure storage account. Running in
IntelliJ, there were 814 tests run and they all passed. I don't fully grep the
new parallel test execution options, which seem endless, but all the tests
passed when I ran the following:
*$ mvn -T 1C -Dparallel-tests clean verify*
Tests run: 252, Failures: 0, Errors: 0, Skipped: 11
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
Tests run: 418, Failures: 0, Errors: 0, Skipped: 55
Tests run: 126, Failures: 0, Errors: 0, Skipped: 10
> WASB: PageBlobInputStream.skip breaks HBASE replication
> -------------------------------------------------------
>
> Key: HADOOP-15446
> URL: https://issues.apache.org/jira/browse/HADOOP-15446
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Affects Versions: 3.0.2, 2.9.0
> Reporter: Thomas Marquardt
> Assignee: Thomas Marquardt
> Priority: Major
> Attachments: HADOOP-15446-001.patch
>
>
> Page Blobs are primarily used by HBASE. HBASE replication, which apparently
> has not been used with WASB until recently, performs non-sequential reads on
> log files using PageBlobInputStream. There are bugs in this stream
> implementation which prevent skip and seek from working properly, and
> eventually the stream state becomes corrupt and unusable.
> I believe this bug affects all releases of WASB/HADOOP. It appears to be a
> day-0 bug in PageBlobInputStream. There were similar bugs opened in the past
> (HADOOP-15042) but the issue was not properly fixed, and no test coverage was
> added.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
