[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015524#comment-14015524 ] Sergey Soldatov commented on HBASE-10985: - Fixed lineLengths errors Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Assignee: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch, HBASE_10985-v2.patch, HBASE_10985-v3.patch, HBASE_10985-v4.patch, HBASE_10985-v5.patch, HBASE_10985-v6.patch, HBASE_10985-v7.patch, HBASE_10985-v8.patch, HBASE_10985-v9.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015697#comment-14015697
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12647924/HBASE_10985-v9.patch
against trunk revision .
ATTACHMENT ID: 12647924
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ || (currentType != EventType.RS_ZK_REGION_SPLIT
currentType != EventType.RS_ZK_REGION_SPLITTING)) {
+ splitTransactionCoordination = new
MockedSplitTransactionCoordination(this, watcher, region);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9664//console
This message is automatically generated.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Assignee: Sergey Soldatov
Fix For: 0.99.0
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch,
HBASE_10985-v2.patch, HBASE_10985-v3.patch, HBASE_10985-v4.patch,
HBASE_10985-v5.patch, HBASE_10985-v6.patch, HBASE_10985-v7.patch,
HBASE_10985-v8.patch, HBASE_10985-v9.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015734#comment-14015734 ] Hudson commented on HBASE-10985: FAILURE: Integrated in HBase-TRUNK #5166 (See [https://builds.apache.org/job/HBase-TRUNK/5166/]) HBASE-10985 Decouple Split Transaction from Zookeeper (Sergey Soldatov) (stack: rev 43be19794ab4d46d57e0609ad6631901740d94f9) * hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/ZKSplitTransactionCoordination.java * hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/ZkCoordinatedStateManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/BaseCoordinatedStateManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java * hbase-common/src/main/resources/hbase-default.xml * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/consensus/ZkCoordinatedStateManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/CoordinatedStateManagerFactory.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/SplitTransactionCoordination.java * hbase-server/src/main/java/org/apache/hadoop/hbase/consensus/BaseCoordinatedStateManager.java Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Assignee: Sergey Soldatov Fix For: 0.99.0 Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch, HBASE_10985-v2.patch, HBASE_10985-v3.patch, HBASE_10985-v4.patch, HBASE_10985-v5.patch, HBASE_10985-v6.patch, HBASE_10985-v7.patch, HBASE_10985-v8.patch, HBASE_10985-v9.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14014092#comment-14014092 ] stack commented on HBASE-10985: --- I added review up on rb Sergey. Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Assignee: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch, HBASE_10985-v2.patch, HBASE_10985-v3.patch, HBASE_10985-v4.patch, HBASE_10985-v5.patch, HBASE_10985-v6.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14014493#comment-14014493
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12647739/HBASE_10985-v8.patch
against trunk revision .
ATTACHMENT ID: 12647739
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ public ZKSplitTransactionCoordination(CoordinatedStateManager
coordinationProvider, ZooKeeperWatcher watcher) {
+ b.getRegionInfo(), coordinationManager.getServer().getServerName(),
zstd, RS_ZK_REGION_SPLITTING,
+b.getRegionInfo(),
coordinationManager.getServer().getServerName(), zstd, RS_ZK_REGION_SPLIT,
+ || (currentType != EventType.RS_ZK_REGION_SPLIT currentType
!= EventType.RS_ZK_REGION_SPLITTING)) {
+ splitTransactionCoordination = new
MockedSplitTransactionCoordination(this, watcher, region);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9655//console
This message is automatically generated.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Assignee: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch,
HBASE_10985-v2.patch, HBASE_10985-v3.patch, HBASE_10985-v4.patch,
HBASE_10985-v5.patch, HBASE_10985-v6.patch, HBASE_10985-v7.patch,
HBASE_10985-v8.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14013187#comment-14013187 ] Sergey Soldatov commented on HBASE-10985: - I'm unable to reproduce audit and javadoc failures on my box. The patch is available for review at https://reviews.apache.org/r/21919/ Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Assignee: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch, HBASE_10985-v2.patch, HBASE_10985-v3.patch, HBASE_10985-v4.patch, HBASE_10985-v5.patch, HBASE_10985-v6.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009180#comment-14009180
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12646824/HBASE_10985-v6.patch
against trunk revision .
ATTACHMENT ID: 12646824
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 31 release
audit warnings (more than the trunk's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9592//console
This message is automatically generated.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Assignee: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch,
HBASE_10985-v2.patch, HBASE_10985-v3.patch, HBASE_10985-v4.patch,
HBASE_10985-v5.patch, HBASE_10985-v6.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13999574#comment-13999574
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12644662/HBASE_10985-v5.patch
against trunk revision .
ATTACHMENT ID: 12644662
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
60 warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ .getSplitTransactionConsensus().waitForSplitTransaction(server,
services, parent, hri_a, hri_b, std);
+ .completeSplitTransaction(server, services, regions.getFirst(),
regions.getSecond(), std, parent);
+ public ZKSplitTransactionConsensus(ConsensusProvider consensusProvider,
ZooKeeperWatcher watcher) {
+ public void completeSplitTransaction(final Server server, final
RegionServerServices services, HRegion a,
+.completeSplitTransaction(server, server, regions.getFirst(),
regions.getSecond(), split.std, region);
+public void completeSplitTransaction(Server server, RegionServerServices
services, HRegion a, HRegion b,
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.master.TestTableLockManager
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.rest.TestTableResource
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
org.apache.hadoop.hbase.util.TestHBaseFsck
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at
org.apache.hadoop.hbase.master.TestMaster.testMasterOpsWhileSplitting(TestMaster.java:89)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:361)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:361)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9519//console
This message is automatically generated.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL:
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995720#comment-13995720
]
Mikhail Antonov commented on HBASE-10985:
-
Some small comments:
- unused import of ZkSplitTransactionConsensus in AssignmentManager,
znodeVersion in SplitTransaction isn't used (and shouldn't be needed?)
- Why do we need to pass in Server instance to methods of
SplitTransactionConsensus? We can retrieve it from the consensusProvider
reference kept in ZkSplitTransactionConsensus instance (also the split
consensus could better be field in SplitTransaction class I guess, shouldn't
need to retrieve it from ConsensusProvider each time?)
- fields consensus and watcher in ZkSplitTransactionConsensus are unused
- can we have the methods on split consensus to not use zk-related terms like
transition node, or it's hard because of the zk is too tightly integrated
here? Ideally guys glancing thru code other than consensus impl shall not need
to know that. E.g. we could have it renamed like: (transitionSplittingNode -
preCommitByRegionServer; transitionZkNode - commitByRegionServer,
createNodeSplitting - startTxnOnRegionServer, getNode -
heartbeatRegionSplitting - do you think this naming accurately reflects the
actual meaning?), also - could we not return values like -1 coming from ZK
world from methods of consensus?
- to eliminate KeeperException from SplitTransaction, could we move some
portion of post splitting steps in openDaughers() like this:
{code}
ry {
// add 2nd daughter first (see HBASE-4335)
services.postOpenDeployTasks(b, server.getCatalogTracker());
// Should add it to OnlineRegions
services.addToOnlineRegions(b);
services.postOpenDeployTasks(a, server.getCatalogTracker());
services.addToOnlineRegions(a);
{code}
to separate consensus method? This way we can remove KeeperException from
imports.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch,
HBASE_10985-v2.patch, HBASE_10985-v3.patch, HBASE_10985-v4.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993498#comment-13993498 ] Mikhail Antonov commented on HBASE-10985: - Not quite sure the failure is related to the changes in patch. Jenkins report doesn't show any failed tests.. Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch, HBASE_10985-v2.patch, HBASE_10985-v3.patch, HBASE_10985-v4.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13993453#comment-13993453
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12644038/HBASE_10985-v4.patch
against trunk revision .
ATTACHMENT ID: 12644038
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnDatanodeDeath(TestLogRolling.java:371)
at
org.apache.hadoop.hbase.mapreduce.TestImportExport.testImport94Table(TestImportExport.java:230)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9489//console
This message is automatically generated.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch,
HBASE_10985-v2.patch, HBASE_10985-v3.patch, HBASE_10985-v4.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992604#comment-13992604 ] Mikhail Antonov commented on HBASE-10985: - With HBASE-11092 committed to trunk yesterday the patch could be simplified a bit. Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch, HBASE_10985-v2.patch, HBASE_10985-v3.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990350#comment-13990350
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12643475/HBASE_10985-v2.patch
against trunk revision .
ATTACHMENT ID: 12643475
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 48 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 8
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestTableMapReduceBase.testMultiRegionTable(TestTableMapReduceBase.java:96)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9465//console
This message is automatically generated.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch,
HBASE_10985-v2.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991076#comment-13991076
]
Hadoop QA commented on HBASE-10985:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12643534/HBASE_10985-v3.patch
against trunk revision .
ATTACHMENT ID: 12643534
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 48 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9468//console
This message is automatically generated.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch,
HBASE_10985-v2.patch, HBASE_10985-v3.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987250#comment-13987250 ] Mikhail Antonov commented on HBASE-10985: - Linked jira about getting ConsensusProvider instance from Server. Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980729#comment-13980729 ] Mikhail Antonov commented on HBASE-10985: - Looking at the patch, a few things needs some thinking. Actually, those are things similar in design to closing/opening region, unless I'm missing something? So I would try to unify some requirements to consensus classes, I guess. May be add to design doc too. That said, some of the items below may be a bit tricky to achieve at first cut.. 1) ConsensusProvider should only be instantiated / initialize()-d / start()-ed / stop()-ped once in Server impl (HMaster or HRegionServer) 2) SplitTransactionConsensus shouldn't use ZK api, i.e. shouldn't do any of this: - import ZK classes - take params like znodeVersion or ZK watcher (znode version and stuff like that we can encapsulate in classes like SplitTransactionProposal, or SplitTransactionDetails, and this SplitTransactionDetails class needs to be constructed from..protobuf request or something like that, while ZKWatcher should be retrieved from ZkConsensusProvider internally in consensus class), - throw KeeperException (we may need to introduce ConsensusException? didn't see the need so far, but sure we may need to) - have methods named after ZK concepts (like transitionNode(), should be..I don't know..finishTransaction() or something like that?) 3) I guess we shouldn't need to pass Server or RegionServerServices into consensus class methods, we should be able to retrieve them from consensusProvider. 4) SplitTransaction which we abstract shouldn't reference ZK API like KeeperException (I guess it was easier for closing regions somewhat, but here we need to either handle all KeeperException-s in ZkSplitTransactionConsensus, or add ConsensusException). Let me know if I missed something. Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980730#comment-13980730 ] Mikhail Antonov commented on HBASE-10985: - I just meant if we have like.. a checklist for abstracting-zk-patches, reviews would be easier for the folks, I guess. Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13979341#comment-13979341
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641654/HBASE-10985.patch
against trunk revision .
ATTACHMENT ID: 12641654
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9387//console
This message is automatically generated.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13979619#comment-13979619
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641703/HBASE-10985.patch
against trunk revision .
ATTACHMENT ID: 12641703
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9389//console
This message is automatically generated.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13979964#comment-13979964
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641704/HBASE-10985.patch
against trunk revision .
ATTACHMENT ID: 12641704
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+hri_b, sn, -1, EventType.RS_ZK_REQUEST_REGION_SPLIT,
EventType.RS_ZK_REGION_SPLITTING) == -1) {
+ znodeVersion =
consensus.getSplitTransactionConsensus().getZKNode(server, services, parent,
hri_a, hri_b);
+consensus.getSplitTransactionConsensus().transitionNode(server, services,
regions.getFirst(), regions.getSecond(),znodeVersion, parent);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testSplitRegionWithNoStoreFiles(TestSplitTransactionOnCluster.java:915)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9391//console
This message is automatically generated.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980050#comment-13980050
]
Ted Yu commented on HBASE-10985:
The test failure was due to NullPointerEx from this line in
SplitTransaction.java :
{code}
consensus.getSplitTransactionConsensus().createNodeSplitting(server,
parent, server.getServerName(), hri_a, hri_b);
{code}
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980056#comment-13980056 ] Sergey Soldatov commented on HBASE-10985: - Yep, I'm fixing those failures right at the moment. Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980263#comment-13980263
]
Ted Yu commented on HBASE-10985:
{code}
+public interface SplitTransactionConsensus {
{code}
Please add annotation for audience.
{code}
+ * Finish off split transaction, transition the zknode
{code}
Consensus shouldn't be concerned with zk node, right ?
Same with znodeVersion below:
{code}
+ HRegion first, HRegion second, int znodeVersion, HRegion parent)
{code}
SplitTransactionConsensus shows, in various places, the dependence on zookeeper.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980370#comment-13980370
]
stack commented on HBASE-10985:
---
Do we have to do this:
+import
org.apache.hadoop.hbase.regionserver.consensus.SplitTransactionConsensus;
... and this:
+import
org.apache.hadoop.hbase.regionserver.consensus.SplitTransactionConsensus;
+import
org.apache.hadoop.hbase.regionserver.consensus.ZKSplitTransactionConsensus;
i.e. have a higher level package reach over into a package that is at a level
below? We'd like to not build tangles across packages w/ dependencies going
both up and down the package hierarchy. If under consensus we have a
regionserver subpackage, is that a mess? It needs to get its hands into the
regionserver guts?
Tough problem.
Are these to be used by a subclass?
- private Server server;
- private ZooKeeperWatcher watcher;
+ protected Server server;
+ protected ZooKeeperWatcher watcher;
+ protected SplitTransactionConsensus splitTransactionConsensus;
Is that why access is opened up?
Is this right? We have to pass in the watcher. When Mikhail did his close, I
don't think he passed in a zk instance. Having to pass in a zk instance even
though the consensus might not be using zk at all is implementation leaking out
of the Interface/Implementation?
+ConsensusProvider consensus =
ConsensusProviderFactory.getConsensusProvider(server
+.getConfiguration());
+consensus.initialize(server);
+consensus.start();
+if
(consensus.getSplitTransactionConsensus().transitionSplittingNode(watcher, p,
hri_a,
+hri_b, sn, -1, EventType.RS_ZK_REQUEST_REGION_SPLIT,
EventType.RS_ZK_REGION_SPLITTING) == -1) {
byte[] data = ZKAssign.getData(watcher, encodedName);
We have to initialize and then start the consensus? (A start w/o a stop is a
little jarring -- when I see 'start', I always expect to see a 'stop')
When I see this in SplitTransaction which is in the regionserver subpackage:
+import org.apache.hadoop.hbase.consensus.ConsensusProvider;
+import org.apache.hadoop.hbase.consensus.ConsensusProviderFactory;
+import
org.apache.hadoop.hbase.regionserver.consensus.SplitTransactionConsensus;
+import
org.apache.hadoop.hbase.regionserver.consensus.ZKSplitTransactionConsensus;
I wonder if we can do better? Do we have to have this many different packages?
When we do they are leaking into each other. Not good package encapsulation.
When is this called?
+ public void setConsensusProvider(ConsensusProvider consensus) {
Should it be called in the constructor for SplitTransaction?
(Is this 'consensus' we are doing here? Or rather is it, cluster state
transition? we need consensus to do this?)
Be careful about tabs in hbase they are two spaces:
- this.parent.getRegionNameAsString(), e);
- }
+ consensus.getSplitTransactionConsensus().createNodeSplitting(server,
+ parent, server.getServerName(), hri_a, hri_b);
Again, the implementation is leaking here?
- znodeVersion = getZKNode(server, services);
+ znodeVersion =
consensus.getSplitTransactionConsensus().getZKNode(server, services, parent,
hri_a, hri_b);
We are asking the consensus provider to do a zk op. Would think it would be an
op that is not named for the implementation type.
I did about 1/3rd of the patch.
Thanks.
Decouple Split Transaction from Zookeeper
-
Key: HBASE-10985
URL: https://issues.apache.org/jira/browse/HBASE-10985
Project: HBase
Issue Type: Sub-task
Components: Consensus, Zookeeper
Reporter: Sergey Soldatov
Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch
As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper.
This is an initial patch for review. At the moment the consensus provider
placed directly to SplitTransaction to minimize affected code. In the ideal
world it should be done in HServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13976678#comment-13976678
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641214/HBASE-10985.patch
against trunk revision .
ATTACHMENT ID: 12641214
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the trunk's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+if
(ConsensusProviderFactory.getConsensusProvider(server.getConfiguration()).getSplitTransactionConsensus().transitionSplittingNode(watcher,
p,
+ znodeVersion =
consensus.getSplitTransactionConsensus().getZKNode(server, services, parent,
hri_a, hri_b);
+consensus.getSplitTransactionConsensus().transitionNode(server, services,
regions.getFirst(), regions.getSecond(),znodeVersion, parent);
+ * @see
org.apache.hadoop.hbase.regionserver.consensus.SplitTransactionConsensus#transitionSplittingNode(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.hbase.HRegionInfo,
org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.hbase.ServerName, int,
org.apache.hadoop.hbase.executor.EventType,
org.apache.hadoop.hbase.executor.EventType)
+void transitionNode(Server server, RegionServerServices services, HRegion
a, HRegion b,int znodeVersion, HRegion parent)
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence
org.apache.hadoop.hbase.regionserver.TestHRegion
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait
org.apache.hadoop.hbase.master.TestTableLockManager
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.rest.TestTableResource
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
org.apache.hadoop.hbase.util.TestHBaseFsck
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:361)
at
org.apache.hadoop.hbase.master.TestMaster.testMasterOpsWhileSplitting(TestMaster.java:89)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:361)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9358//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9358//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9358//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9358//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9358//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9358//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9358//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9358//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9358//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[
https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969801#comment-13969801
]
Hadoop QA commented on HBASE-10985:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12640297/HBASE-10985.patch
against trunk revision .
ATTACHMENT ID: 12640297
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+if
(ConsensusProviderFactory.getConsensusProvider(server.getConfiguration()).getSplitTransactionConsensus().transitionSplittingNode(watcher,
p,
+ znodeVersion =
consensus.getSplitTransactionConsensus().getZKNode(server, services, parent,
hri_a, hri_b);
+consensus.getSplitTransactionConsensus().transitionNode(server, services,
regions.getFirst(), regions.getSecond(),znodeVersion, parent);
+ * @see
org.apache.hadoop.hbase.regionserver.consensus.SplitTransactionConsensus#createNodeSplitting(org.apache.hadoop.hbase.Server,
org.apache.hadoop.hbase.regionserver.HRegion,
org.apache.hadoop.hbase.ServerName, org.apache.hadoop.hbase.HRegionInfo,
org.apache.hadoop.hbase.HRegionInfo)
+ * @see
org.apache.hadoop.hbase.regionserver.consensus.SplitTransactionConsensus#transitionSplittingNode(org.apache.hadoop.hbase.Server,
org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.hbase.HRegionInfo,
org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.hbase.ServerName, int,
org.apache.hadoop.hbase.executor.EventType,
org.apache.hadoop.hbase.executor.EventType)
+ * @see
org.apache.hadoop.hbase.regionserver.consensus.SplitTransactionConsensus#transitionSplittingNode(org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher,
org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.hbase.HRegionInfo,
org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.hbase.ServerName, int,
org.apache.hadoop.hbase.executor.EventType,
org.apache.hadoop.hbase.executor.EventType)
+ * @see
org.apache.hadoop.hbase.regionserver.consensus.SplitTransactionConsensus#getZKNode(org.apache.hadoop.hbase.Server,
org.apache.hadoop.hbase.regionserver.RegionServerServices,
org.apache.hadoop.hbase.regionserver.HRegion,
org.apache.hadoop.hbase.HRegionInfo, org.apache.hadoop.hbase.HRegionInfo)
+ * @see
org.apache.hadoop.hbase.regionserver.consensus.SplitTransactionConsensus#transitionNode(org.apache.hadoop.hbase.Server,
org.apache.hadoop.hbase.regionserver.RegionServerServices,
org.apache.hadoop.hbase.regionserver.HRegion,
org.apache.hadoop.hbase.regionserver.HRegion, int,
org.apache.hadoop.hbase.regionserver.HRegion)
+ * @see
org.apache.hadoop.hbase.regionserver.consensus.SplitTransactionConsensus#cleanZK(org.apache.hadoop.hbase.Server,
org.apache.hadoop.hbase.HRegionInfo)
+void transitionNode(Server server, RegionServerServices services, HRegion
a, HRegion b,int znodeVersion, HRegion parent)
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestSplitTransaction
org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9293//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9293//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9293//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9293//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9293//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9293//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9293//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13969887#comment-13969887 ] Mikhail Antonov commented on HBASE-10985: - This patch also somewhat depends on HBASE-10915 (consensus infra). Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: regionserver, Zookeeper Reporter: Sergey Soldatov Attachments: HBASE-10985.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
