[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258779#comment-13258779 ] ramkrishna.s.vasudevan commented on HBASE-5848: --- @Lars Ohwe were using these options in one of our scenarios in 0.94. We found no issues, may be this some other case which is causing the problem. Thanks for bringing in this Lars. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258780#comment-13258780 ] ramkrishna.s.vasudevan commented on HBASE-5809: --- @Ted Thanks for your follow up. But the same had passed in Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1578//testReport/. The QA build that ran on this patch. I will check it up and commit the addendum. Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: client Fix For: 0.96.0 Attachments: 5809.addendum, HBASE-5809.patch, HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5635) If getTaskList() returns null, splitlogWorker would go down and it won't serve any requests
[
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258782#comment-13258782
]
ramkrishna.s.vasudevan commented on HBASE-5635:
---
The above failure in TestSplitLogManager QA does not repeat. Prev run on QA
has passed.
Will look into it more before committing.
If getTaskList() returns null, splitlogWorker would go down and it won't
serve any requests
---
Key: HBASE-5635
URL: https://issues.apache.org/jira/browse/HBASE-5635
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch,
HBASE-5635._trunk.patch, HBASE-5635.patch, HBASE-5635_0.94.patch
During the hlog split operation if all the zookeepers are down ,then the
paths will be returned as null and the splitworker thread wil be exited
Now this regionserver wil not be able to acquire any other tasks since the
splitworker thread is exited
Please find the attached code for more details
{code}
private ListString getTaskList() {
for (int i = 0; i zkretries; i++) {
try {
return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
this.watcher.splitLogZNode));
} catch (KeeperException e) {
LOG.warn(Could not get children of znode +
this.watcher.splitLogZNode, e);
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
LOG.warn(Interrupted while trying to get task list ..., e1);
Thread.currentThread().interrupt();
return null;
}
}
}
{code}
in the org.apache.hadoop.hbase.regionserver.SplitLogWorker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258784#comment-13258784
]
Lars Hofhansl commented on HBASE-5848:
--
@Ram: Are you sure?
This one liner caused the HMaster to abort in every cluster I tried:
{code}
new HBaseAdmin(HBaseConfiguration.create()).createTable(new
HTableDescriptor(x), new byte[][] {HConstants.EMPTY_START_ROW});
{code}
Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to
abort
Key: HBASE-5848
URL: https://issues.apache.org/jira/browse/HBASE-5848
Project: HBase
Issue Type: Bug
Components: client
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
A coworker of mine just had this scenario. It does not make sense the
EMPTY_START_ROW as splitKey (since the region with the empty start key is
implicit), but it should not cause the HMaster to abort.
The abort happens because it tries to bulk assign the same region twice and
then runs into race conditions with ZK.
The same would (presumably) happen when two identical split keys are passed,
but the client blocks that. The simplest solution here is to also block
passed null or EMPTY_START_ROW as split key by the client.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5652) [findbugs] Fix lock release on all paths
[ https://issues.apache.org/jira/browse/HBASE-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258786#comment-13258786 ] ramkrishna.s.vasudevan commented on HBASE-5652: --- @Greg I think the lock paths here are unlocked in finally block. And the try catch block added may not be needed. If Jon accepts we can add it into exclude file i think. @Jon Your thoughts please. [findbugs] Fix lock release on all paths - Key: HBASE-5652 URL: https://issues.apache.org/jira/browse/HBASE-5652 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: Gregory Chanan Attachments: HBASE-5652-v0.patch See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS Category UL -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5850) Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized
[ https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xufeng updated HBASE-5850: -- Fix Version/s: 0.94.0 0.92.2 0.90.7 Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized --- Key: HBASE-5850 URL: https://issues.apache.org/jira/browse/HBASE-5850 Project: HBase Issue Type: Bug Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2, 0.94.0 Attachments: backport-5454(createTable)-to-94.patch, backport-5454(createTable)-to-94_surefire-report.html, backport-5454(createTable)-to-trunk.patch, backport-5454(createTable)-to-trunk_surefire-report.html, backport-5454-to-90.patch, backport-5454-to-92.patch, backport-5454-to-92_surefire-report.html This issue is needed in 0.90 0.92 also. And update the hbase-5454 patch that add the checkInitialized() into HMaster#createTable(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258787#comment-13258787 ] ramkrishna.s.vasudevan commented on HBASE-5848: --- @Lars No Lars, what i meant was we were trying different other options while creating tables but may not be this way. Once i go to office on monday i can tell you what are the other options that we have tried. (including duplicate keys, empty keys, etc.) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5844) Delete the region servers znode after a regions server crash
[ https://issues.apache.org/jira/browse/HBASE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258788#comment-13258788 ] nkeywal commented on HBASE-5844: I didn't know this parameter. It's interesting, because with ZK the default timeout is 30 seconds, but with HBase it's now 180s (from hbase-default.xml). It was increased to 60s a first time in HBASE-1772. It seems it was increased because of the GC. But it means that deleting immediately the ZK represents a huge mttr improvement for the regions server crash case. Delete the region servers znode after a regions server crash Key: HBASE-5844 URL: https://issues.apache.org/jira/browse/HBASE-5844 Project: HBase Issue Type: Improvement Components: regionserver, scripts Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 5844.v1.patch today, if the regions server crashes, its znode is not deleted in ZooKeeper. So the recovery process will stop only after a timeout, usually 30s. By deleting the znode in start script, we remove this delay and the recovery starts immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5652) [findbugs] Fix lock release on all paths
[
https://issues.apache.org/jira/browse/HBASE-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258791#comment-13258791
]
Uma Maheswara Rao G commented on HBASE-5652:
Agreed with Ram, variable assignment in finally block before unlocking would
not cause any exception here. But the standard pattern for read/write locks I
have seen is, after acquiring the lock, every line should be in try and in
finally block we will release the lock. That might be the findbugs worry here.
But in this case, there is no way of throwing exception from variable
assignments. So, we can just skip I feel. Let's see Jon opinion on this.
Here try/finally almost no use.
{code}
try {
+this.logRollRunning = false;
+ } finally {
+this.cacheFlushLock.unlock();
+ }
{code}
Other problem I see in adding into exclude list is, we are not able to pin
point exact lication of the code. We may give just
package/class/method/feilds..and bug pattern,type ...etc. Unfortunately if same
bug introduces but this is valid to fix in the same area of code, then it may
get skipped due to other exclude entry presents in the file which is almost
matching to the same. So, we have to reduce exclude filter entries also as less
as possible.
[findbugs] Fix lock release on all paths
-
Key: HBASE-5652
URL: https://issues.apache.org/jira/browse/HBASE-5652
Project: HBase
Issue Type: Sub-task
Components: scripts
Reporter: Jonathan Hsieh
Assignee: Gregory Chanan
Attachments: HBASE-5652-v0.patch
See
https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS
Category UL
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258792#comment-13258792 ] ramkrishna.s.vasudevan commented on HBASE-5809: --- +1 on addendum Ted. Thanks for the patch. I will commit it. Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: client Fix For: 0.96.0 Attachments: 5809.addendum, HBASE-5809.patch, HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258793#comment-13258793 ] ramkrishna.s.vasudevan commented on HBASE-5809: --- Committed to trunk. Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: client Fix For: 0.96.0 Attachments: 5809.addendum, HBASE-5809.patch, HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5652) [findbugs] Fix lock release on all paths
[ https://issues.apache.org/jira/browse/HBASE-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258794#comment-13258794 ] ramkrishna.s.vasudevan commented on HBASE-5652: --- Yes Uma you are right. Jon also is strict in moving things to exclude file. [findbugs] Fix lock release on all paths - Key: HBASE-5652 URL: https://issues.apache.org/jira/browse/HBASE-5652 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: Gregory Chanan Attachments: HBASE-5652-v0.patch See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS Category UL -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5844) Delete the region servers znode after a regions server crash
[
https://issues.apache.org/jira/browse/HBASE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258796#comment-13258796
]
stack commented on HBASE-5844:
--
If we go to a count 100 we just continue the startup? Is that what you want?
{code}
+while (!tracker.checkIfBaseNodeAvailable() ++count100) {
+ Thread.sleep(100);
+}
{code}
Be like the rest of the code regards spaces; i.e. spaces around operators...
+
+if (fileName==null){
Maybe you don't need deleteMyEphemeralNodeOnDisk if you instead use
http://docs.oracle.com/javase/6/docs/api/java/io/File.html#deleteOnExit()
inside in writeMyEphemeralNodeOnDisk?
Patch looks good N.
We upped the timeout because noobs would install hbase then run big mapreduce
jobs w/o turning jvm and so big GCs. We figured they'd rather have their
regionserver ride over the big pauses than have them be 'sensitive' out of the
box.
Delete the region servers znode after a regions server crash
Key: HBASE-5844
URL: https://issues.apache.org/jira/browse/HBASE-5844
Project: HBase
Issue Type: Improvement
Components: regionserver, scripts
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Attachments: 5844.v1.patch
today, if the regions server crashes, its znode is not deleted in ZooKeeper.
So the recovery process will stop only after a timeout, usually 30s.
By deleting the znode in start script, we remove this delay and the recovery
starts immediately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5844) Delete the region servers znode after a regions server crash
[ https://issues.apache.org/jira/browse/HBASE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258803#comment-13258803 ] nkeywal commented on HBASE-5844: For the tracker, it's my private workaround for HBASE-5666, it should not have been included in this patch. Sorry about this. I think it's better to delete the file explicitly, just after the znode deletion. HRegionServer#deleteMyEphemeralNode is called only once, and I added deleteMyEphemeralNodeOnDisk just after this call. If we rely on #deleteOnExit, I fear we could have the file deleted with a still alive znode. I'm not sure and I have not tried it, but I think it's too easy to enter into the jvm-specific-behavior space here. I will fix the java code and try the whole fix on a real cluster for the v2. Thanks you for the review. Delete the region servers znode after a regions server crash Key: HBASE-5844 URL: https://issues.apache.org/jira/browse/HBASE-5844 Project: HBase Issue Type: Improvement Components: regionserver, scripts Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 5844.v1.patch today, if the regions server crashes, its znode is not deleted in ZooKeeper. So the recovery process will stop only after a timeout, usually 30s. By deleting the znode in start script, we remove this delay and the recovery starts immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258805#comment-13258805 ] Hudson commented on HBASE-5809: --- Integrated in HBase-TRUNK #2793 (See [https://builds.apache.org/job/HBase-TRUNK/2793/]) HBASE-5809 Avoid move api to take the destination server same as the source server. Addendum to correct tet case. (Ted) (Revision 1328594) Result = FAILURE ramkrishna : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: client Fix For: 0.96.0 Attachments: 5809.addendum, HBASE-5809.patch, HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine
[ https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258819#comment-13258819 ] Andrew Purtell commented on HBASE-5732: --- bq. I'll revert User.java and have it only support hadoop-1.0++. Sounds reasonable? Sounds good, thanks Devaraj. Remove the SecureRPCEngine and merge the security-related logic in the core engine -- Key: HBASE-5732 URL: https://issues.apache.org/jira/browse/HBASE-5732 Project: HBase Issue Type: Improvement Reporter: Devaraj Das Assignee: Devaraj Das Attachments: rpcengine-merge.patch Remove the SecureRPCEngine and merge the security-related logic in the core engine. Follow up to HBASE-5727. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu resolved HBASE-5809. --- Resolution: Fixed Hadoop Flags: Reviewed Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: client Fix For: 0.96.0 Attachments: 5809.addendum, HBASE-5809.patch, HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258860#comment-13258860 ] Hudson commented on HBASE-5809: --- Integrated in HBase-TRUNK-security #178 (See [https://builds.apache.org/job/HBase-TRUNK-security/178/]) HBASE-5809 Avoid move api to take the destination server same as the source server. Addendum to correct tet case. (Ted) (Revision 1328594) Result = FAILURE ramkrishna : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: client Fix For: 0.96.0 Attachments: 5809.addendum, HBASE-5809.patch, HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5851) TestProcessBasedCluster sometimes fails
Zhihong Yu created HBASE-5851:
-
Summary: TestProcessBasedCluster sometimes fails
Key: HBASE-5851
URL: https://issues.apache.org/jira/browse/HBASE-5851
Project: HBase
Issue Type: Test
Reporter: Zhihong Yu
TestProcessBasedCluster failed in
https://builds.apache.org/job/HBase-TRUNK-security/178
Looks like cluster failed to start:
{code}
2012-04-21 14:22:32,666 INFO [Thread-1]
util.ProcessBasedLocalHBaseCluster(176): Waiting for HBase to startup. Retries
left: 2
java.io.IOException: Giving up trying to location region in meta: thread is
interrupted.
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1173)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:956)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:917)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:252)
at org.apache.hadoop.hbase.client.HTable.init(HTable.java:192)
at
org.apache.hadoop.hbase.util.ProcessBasedLocalHBaseCluster.startHBase(ProcessBasedLocalHBaseCluster.java:174)
at
org.apache.hadoop.hbase.util.TestProcessBasedCluster.testProcessBasedCluster(TestProcessBasedCluster.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
java.lang.InterruptedException: sleep interrupted at
java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:134)
at
org.apache.hadoop.hbase.util.ProcessBasedLocalHBaseCluster.startHBase(ProcessBasedLocalHBaseCluster.java:178)
at
org.apache.hadoop.hbase.util.TestProcessBasedCluster.testProcessBasedCluster(TestProcessBasedCluster.java:56)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5621: --- Status: Open (was: Patch Available) Convert admin protocol of HRegionInterface to PB Key: HBASE-5621 URL: https://issues.apache.org/jira/browse/HBASE-5621 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5621_v3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5621: --- Attachment: hbase_5621_v4.patch Convert admin protocol of HRegionInterface to PB Key: HBASE-5621 URL: https://issues.apache.org/jira/browse/HBASE-5621 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5621_v3.patch, hbase_5621_v4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5621: --- Status: Patch Available (was: Open) Minor changes to fix TestHBaseFsck failure and some security test failures. Convert admin protocol of HRegionInterface to PB Key: HBASE-5621 URL: https://issues.apache.org/jira/browse/HBASE-5621 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5621_v3.patch, hbase_5621_v4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258963#comment-13258963 ] Hadoop QA commented on HBASE-5621: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523657/hbase_5621_v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 42 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 9 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestAdmin Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1598//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1598//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1598//console This message is automatically generated. Convert admin protocol of HRegionInterface to PB Key: HBASE-5621 URL: https://issues.apache.org/jira/browse/HBASE-5621 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5621_v3.patch, hbase_5621_v4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5621: - Attachment: hbase_5621_v4.patch Retrying Convert admin protocol of HRegionInterface to PB Key: HBASE-5621 URL: https://issues.apache.org/jira/browse/HBASE-5621 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5621_v3.patch, hbase_5621_v4.patch, hbase_5621_v4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5621: - Status: Patch Available (was: Open) Convert admin protocol of HRegionInterface to PB Key: HBASE-5621 URL: https://issues.apache.org/jira/browse/HBASE-5621 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5621_v3.patch, hbase_5621_v4.patch, hbase_5621_v4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5621: - Status: Open (was: Patch Available) Convert admin protocol of HRegionInterface to PB Key: HBASE-5621 URL: https://issues.apache.org/jira/browse/HBASE-5621 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5621_v3.patch, hbase_5621_v4.patch, hbase_5621_v4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5844) Delete the region servers znode after a regions server crash
[ https://issues.apache.org/jira/browse/HBASE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258988#comment-13258988 ] stack commented on HBASE-5844: -- Ok on your reasoning for not using deleteOnExit. Try and have the two methods share more code like getting the name of the file w/ the znode name in it. Otherwise, sounds good. Delete the region servers znode after a regions server crash Key: HBASE-5844 URL: https://issues.apache.org/jira/browse/HBASE-5844 Project: HBase Issue Type: Improvement Components: regionserver, scripts Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 5844.v1.patch today, if the regions server crashes, its znode is not deleted in ZooKeeper. So the recovery process will stop only after a timeout, usually 30s. By deleting the znode in start script, we remove this delay and the recovery starts immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258993#comment-13258993 ] Hadoop QA commented on HBASE-5621: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523663/hbase_5621_v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 42 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 9 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1599//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1599//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1599//console This message is automatically generated. Convert admin protocol of HRegionInterface to PB Key: HBASE-5621 URL: https://issues.apache.org/jira/browse/HBASE-5621 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5621_v3.patch, hbase_5621_v4.patch, hbase_5621_v4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258994#comment-13258994 ] stack commented on HBASE-5621: -- We seem to have run a good few less tests, ~880 vs 906 or so Does it pass all tests for you Jimmy? Thanks. Convert admin protocol of HRegionInterface to PB Key: HBASE-5621 URL: https://issues.apache.org/jira/browse/HBASE-5621 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5621_v3.patch, hbase_5621_v4.patch, hbase_5621_v4.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5850) Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized
[ https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5850: - Fix Version/s: (was: 0.94.0) HBASE-5454 is in 0.94 already. Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized --- Key: HBASE-5850 URL: https://issues.apache.org/jira/browse/HBASE-5850 Project: HBase Issue Type: Bug Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2 Attachments: backport-5454(createTable)-to-94.patch, backport-5454(createTable)-to-94_surefire-report.html, backport-5454(createTable)-to-trunk.patch, backport-5454(createTable)-to-trunk_surefire-report.html, backport-5454-to-90.patch, backport-5454-to-92.patch, backport-5454-to-92_surefire-report.html This issue is needed in 0.90 0.92 also. And update the hbase-5454 patch that add the checkInitialized() into HMaster#createTable(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5833) 0.92 build has been failing pretty consistently on TestMasterFailover....
[
https://issues.apache.org/jira/browse/HBASE-5833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258998#comment-13258998
]
stack commented on HBASE-5833:
--
More digging. The newest test added here,
testShouldCheckMasterFailOverWhenMETAIsInOpenedState, is a little interesting.
It was added by this commit:
{code}
r1172063 | tedyu | 2011-09-17 13:27:00 -0700 (Sat, 17 Sep 2011) | 3 lines
HBASE-4400 .META. getting stuck if RS hosting it is dead and znode state is in
RS_ZK_REGION_OPENED (Ramkrishna)
{code}
The test is a bunch of copy/paste confirming stuff its not using. It then does
a cluster shutdown but does it explicitly on a cluster object and not via
HBaseTestingUtility though it then starts a cluster subsequently with
HBaseTestingUtility. Not using HTU to do both the shutodwn and the startup can
make he HTU state confused on whether there a master available so we just wait
for ever. This seems to be responsible for case where test would timeout after
15 minutes and say no tests run and none failed.
I added a timeout for this test of 3 minutes.
Other interesting stuff is that this TestMasterFailover starts clusters per
method but shutdown leaves around some threads. I dug in some and was able to
clean up an LruBlockCache eviction thread but others persist and would take a
little more work to undo. They seem harmless but I'll list them anyways:
{code}
TestMasterFailover [JUnit]
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner at
localhost:54811
Thread [main] (Running)
Thread [ReaderThread] (Running)
Thread [Thread-2] (Suspended (breakpoint at line 587 in
HBaseTestingUtility))
HBaseTestingUtility.shutdownMiniCluster() line: 587
TestMasterFailover.testSimpleMasterFailover() line: 178
NativeMethodAccessorImpl.invoke0(Method, Object,
Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line:
39
DelegatingMethodAccessorImpl.invoke(Object, Object[])
line: 25
Method.invoke(Object, Object...) line: 597
FrameworkMethod$1.runReflectiveCall() line: 45
FrameworkMethod$1(ReflectiveCallable).run() line: 15
FrameworkMethod.invokeExplosively(Object, Object...)
line: 42
InvokeMethod.evaluate() line: 20
FailOnTimeout$StatementThread.run() line: 62
Daemon Thread [Poller SunPKCS11-Darwin] (Running)
Thread [pool-1-thread-1] (Running)
Thread [pool-2-thread-1] (Running)
Thread [pool-3-thread-1] (Running)
Thread [pool-4-thread-1] (Running)
Daemon Thread [LeaseChecker] (Running)
Daemon Thread
[RegionServer:2;192.168.1.74,54842,1335066804457.decayingSampleTick.1]
(Running)
Daemon Thread
[Master:2;192.168.1.74,54838,1335066803952-SendThread(fe80:0:0:0:0:0:0:1%1:21818)]
(Running)
Daemon Thread
[Master:2;192.168.1.74,54838,1335066803952-EventThread] (Running)
Daemon Thread
[Master:1;192.168.1.74,54836,1335066798880-EventThread] (Running)
Daemon Thread
[Master:1;192.168.1.74,54836,1335066798880-SendThread(localhost:21818)]
(Running)
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java (Apr
21, 2012 8:53:07 PM)
{code}
The thread names are enhanced -- v2 of this patch -- but things like
decayingSampleTick are set in a static so hard to get rid of in test setup.
The SendThread/EventThread are zk client hangouts. Not sure what
pool-4-thread-1 are (I've enhanced the HTable executor to include htable in
name so these are identifiable going forward but above executor does not seem
to be HTable).
0.92 build has been failing pretty consistently on TestMasterFailover
-
Key: HBASE-5833
URL: https://issues.apache.org/jira/browse/HBASE-5833
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.92.2
Attachments: 5833.txt, closehregions.txt
Trunk seems fine but 0.92 fails on this test pretty regularly. Running it
local it seems to hang for me.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
