[jira] [Commented] (HBASE-11280) Document distributed log replay and distributed log splitting
[ https://issues.apache.org/jira/browse/HBASE-11280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034752#comment-14034752 ] Hudson commented on HBASE-11280: FAILURE: Integrated in HBase-TRUNK #5215 (See [https://builds.apache.org/job/HBase-TRUNK/5215/]) HBASE-11280 Document distributed log replay and distributed log splitting (Misty Stanley-Jones) (stack: rev 1b9242259704d5c6a653a7f56cb1121b981ae3da) * src/main/docbkx/book.xml Document distributed log replay and distributed log splitting - Key: HBASE-11280 URL: https://issues.apache.org/jira/browse/HBASE-11280 Project: HBase Issue Type: Sub-task Components: documentation, MTTR, wal Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Fix For: 0.99.0 Attachments: HBASE-11280-1.patch, HBASE-11280.patch Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11280) Document distributed log replay and distributed log splitting
[ https://issues.apache.org/jira/browse/HBASE-11280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028559#comment-14028559 ] Misty Stanley-Jones commented on HBASE-11280: - Any further progress in the review, [~stack]? Document distributed log replay and distributed log splitting - Key: HBASE-11280 URL: https://issues.apache.org/jira/browse/HBASE-11280 Project: HBase Issue Type: Sub-task Components: documentation, MTTR, wal Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Fix For: 0.99.0 Attachments: HBASE-11280-1.patch, HBASE-11280.patch Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11280) Document distributed log replay and distributed log splitting
[
https://issues.apache.org/jira/browse/HBASE-11280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14017517#comment-14017517
]
Hadoop QA commented on HBASE-11280:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12648295/HBASE-11280.patch
against trunk revision .
ATTACHMENT ID: 12648295
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+
xlink:href=http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/regionserver/wal/HLog.html;HLog/link.
+ paraThe WAL resides in HDFS in the
filename/hbase/.logs//filename directory, with subdirectories per
+ paraLog splitting is done by the HMaster during cluster start-up
or by the ServerShutdownHandler
+file. The temporary edit file is stored to disk with the
following naming pattern:/para
+ para When the region is opened, the
filenamerecovered.edits/filename folder is checked for recovered
+(link
xlink:href=https://issues.apache.org/jira/browse/HBASE-1364;HBASE-1364/link)
+[hdfs%3A%2F%2Fhost2.sample.com%3A56020%2Fhbase%2F.logs%2Fhost8.sample.com%2C57020%2C1340474893275-splitting%2Fhost8.sample.com%253A57020.1340474893900,
+hdfs%3A%2F%2Fhost2.sample.com%3A56020%2Fhbase%2F.logs%2Fhost3.sample.com%2C57020%2C1340474893299-splitting%2Fhost3.sample.com%253A57020.1340474893931,
+hdfs%3A%2F%2Fhost2.sample.com%3A56020%2Fhbase%2F.logs%2Fhost4.sample.com%2C57020%2C1340474893287-splitting%2Fhost4.sample.com%253A57020.1340474893946]
+userinputget
/hbase/splitlog/hdfs%3A%2F%2Fhost2.sample.com%3A56020%2Fhbase%2F.logs%2Fhost6.sample.com%2C57020%2C1340474893287-splitting%2Fhost6.sample.com%253A57020.1340474893945
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.regionserver.TestHRegion
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9688//console
This message is automatically generated.
Document distributed log replay and distributed log splitting
-
Key: HBASE-11280
URL: https://issues.apache.org/jira/browse/HBASE-11280
Project: HBase
Issue Type: Sub-task
Components: documentation, MTTR, wal
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
Fix For: 0.99.0
Attachments: HBASE-11280.patch
Enable 'distributed log replay' by default. Depends on hfilev3 being enabled.
--
This
[jira] [Commented] (HBASE-11280) Document distributed log replay and distributed log splitting
[ https://issues.apache.org/jira/browse/HBASE-11280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14017752#comment-14017752 ] stack commented on HBASE-11280: --- This is great (I like this kind of stuff 'firsttermWrite Ahead Log (WAL)/firstterm records) Location looks correct to me. Change this The WAL implementation used by HBase is because the HLog you point to is only an Interface. Implementations implement this Interface. Currently there is one only but attempts have been made at doing others and we may be doing a new one soon to do multiwal. Change this One HLog instance exists ... to be instead Usually there is one instance of a WAL only per RegionServer... (there is one HLog implementatoin true but I think you are trying to say one WAL here). We intend there to be many in the hopefully not too distant future. WALs used to be kept under filename/hbase/.logs//filename in and before 0.94. Now they are in filename/hbase/WALs/filename The description of '+titleWAL Splitting, Step by Step/title' applies to Distributed Log Splitting. You might want to call it out as so? Maybe mention that the HMaster enrolls all regionservers in the log splitting process farming log splitting process to the regionservers. That is what I have so far (about half way through review) Document distributed log replay and distributed log splitting - Key: HBASE-11280 URL: https://issues.apache.org/jira/browse/HBASE-11280 Project: HBase Issue Type: Sub-task Components: documentation, MTTR, wal Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Fix For: 0.99.0 Attachments: HBASE-11280.patch Enable 'distributed log replay' by default. Depends on hfilev3 being enabled. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11280) Document distributed log replay and distributed log splitting
[
https://issues.apache.org/jira/browse/HBASE-11280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14018361#comment-14018361
]
Hadoop QA commented on HBASE-11280:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12648395/HBASE-11280-1.patch
against trunk revision .
ATTACHMENT ID: 12648395
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+Usually, there is only one instance of a WAL per RegionServer.
The RegionServer records Puts and Deletes to
+HBase 0.94, they were stored in
filename/hbase/.logs//filename), with subdirectories per
+ paraLog splitting is done by the HMaster during cluster start-up
or by the ServerShutdownHandler
+file. The temporary edit file is stored to disk with the
following naming pattern:/para
+ para When the region is opened, the
filenamerecovered.edits/filename folder is checked for recovered
+(link
xlink:href=https://issues.apache.org/jira/browse/HBASE-1364;HBASE-1364/link)
+[hdfs%3A%2F%2Fhost2.sample.com%3A56020%2Fhbase%2F.logs%2Fhost8.sample.com%2C57020%2C1340474893275-splitting%2Fhost8.sample.com%253A57020.1340474893900,
+hdfs%3A%2F%2Fhost2.sample.com%3A56020%2Fhbase%2F.logs%2Fhost3.sample.com%2C57020%2C1340474893299-splitting%2Fhost3.sample.com%253A57020.1340474893931,
+hdfs%3A%2F%2Fhost2.sample.com%3A56020%2Fhbase%2F.logs%2Fhost4.sample.com%2C57020%2C1340474893287-splitting%2Fhost4.sample.com%253A57020.1340474893946]
+userinputget
/hbase/splitlog/hdfs%3A%2F%2Fhost2.sample.com%3A56020%2Fhbase%2F.logs%2Fhost6.sample.com%2C57020%2C1340474893287-splitting%2Fhost6.sample.com%253A57020.1340474893945
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9692//console
This message is automatically generated.
Document distributed log replay and distributed log splitting
-
Key: HBASE-11280
URL: https://issues.apache.org/jira/browse/HBASE-11280
Project: HBase
Issue Type: Sub-task
Components: documentation, MTTR, wal
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
Fix For: 0.99.0
Attachments: HBASE-11280-1.patch, HBASE-11280.patch
Enable 'distributed log replay' by default. Depends on hfilev3 being enabled.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
