[jira] [Created] (HBASE-12943) Set sun.net.inetaddr.ttl in HBase
Liu Shaohui created HBASE-12943: --- Summary: Set sun.net.inetaddr.ttl in HBase Key: HBASE-12943 URL: https://issues.apache.org/jira/browse/HBASE-12943 Project: HBase Issue Type: Bug Reporter: Liu Shaohui Assignee: Liu Shaohui The default value of config: sun.net.inetaddr.ttl is -1 and the java processes will cache the mapping of hostname to ip address forever, See: http://docs.oracle.com/javase/7/docs/technotes/guides/net/properties.html But things go wrong when a regionserver with same hostname and different ip address rejoins the hbase cluster. The HMaster will get wrong ip address of the regionserver from this cache and every region assignment to this regionserver will be blocked for a time because the HMaster can't communicate with the regionserver. A tradeoff is to set the sun.net.inetaddr.ttl to 10m or 1h and make the wrong cache expired. Suggestions are welcomed. Thanks~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12667) Deadlock in AssignmentManager
[ https://issues.apache.org/jira/browse/HBASE-12667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla resolved HBASE-12667. - Resolution: Duplicate Already fixed as part of HBASE-12901. Marking as duplicate. Deadlock in AssignmentManager - Key: HBASE-12667 URL: https://issues.apache.org/jira/browse/HBASE-12667 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: zhaoyunjiong No order between regionPlans and regionStates caused dead lock. Trunk don't have the problem since it's already got refactor. master:phxhshdc11en0004:6: at org.apache.hadoop.hbase.master.AssignmentManager.clearRegionPlan(AssignmentManager.java:2898) - waiting to lock 0x00048cefe520 (a java.util.TreeMap) at org.apache.hadoop.hbase.master.AssignmentManager.regionOnline(AssignmentManager.java:1286) at org.apache.hadoop.hbase.master.AssignmentManager.handleRegionSplitting(AssignmentManager.java:3552) - locked 0x00048cf6fc10 (a org.apache.hadoop.hbase.master.RegionStates) at org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:732) at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:601) at org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRecoverLostRegions(AssignmentManager.java:2851) at org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:519) at org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:459) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:900) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:609) at java.lang.Thread.run(Thread.java:744) AM.-pool1-t10: at org.apache.hadoop.hbase.master.RegionStates.getRegionAssignments(RegionStates.java:154) - waiting to lock 0x00048cf6fc10 (a org.apache.hadoop.hbase.master.RegionStates) at org.apache.hadoop.hbase.master.AssignmentManager.getSnapShotOfAssignment(AssignmentManager.java:3610) at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.getRegionAssignmentsByServer(BaseLoadBalancer.java:1146) at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.createCluster(BaseLoadBalancer.java:959) at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.randomAssignment(BaseLoadBalancer.java:1010) at org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:2209) - locked 0x00048cefe520 (a java.util.TreeMap) at org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:2166) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1886) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1445) at org.apache.hadoop.hbase.master.AssignCallable.call(AssignCallable.java:45) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12859) New master API to track major compaction completion
[ https://issues.apache.org/jira/browse/HBASE-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-12859. --- Resolution: Fixed Hadoop Flags: Reviewed Pushed to 2.0 and 1.1 New master API to track major compaction completion --- Key: HBASE-12859 URL: https://issues.apache.org/jira/browse/HBASE-12859 Project: HBase Issue Type: Brainstorming Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 2.0.0, 1.1.0 Attachments: 12859-v1.txt, 12859-v2.txt, 12859-v3.txt, 12859-v4.txt, 12859-v5.txt, 12859-v6.txt, 12859-v7-1.1.txt, 12859-v7.txt, 12859-wip-UNFINISHED.txt In various scenarios it is helpful to know a guaranteed timestamp up to which all data in a table was major compacted. We can do that keeping a major compaction timestamp in META. A client then can iterate all region of a table and find a definite timestamp, which is the oldest compaction timestamp of any of the regions. [~apurtell], [~ghelmling], [~giacomotaylor]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12945) Port: New master API to track major compaction completion to 0.8
Lars Hofhansl created HBASE-12945: - Summary: Port: New master API to track major compaction completion to 0.8 Key: HBASE-12945 URL: https://issues.apache.org/jira/browse/HBASE-12945 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Fix For: 0.98.11 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12944) Support patches to branches in precommit jenkins build
Enis Soztutar created HBASE-12944: - Summary: Support patches to branches in precommit jenkins build Key: HBASE-12944 URL: https://issues.apache.org/jira/browse/HBASE-12944 Project: HBase Issue Type: New Feature Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0 We have a quite a few active branches now, which makes backporting a full time job. I was thinking about whether we can get hadoopqa to test the patches specific to branches with the code from that branch. I think we can grab the branch name from the patch file name and check out that branch prior to running the tests. I have a patch, but not sure whether it will work. Let me experiment a bit. If hadoopqa gets broken, it is probably this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Precommit build (hadoopqa) can now test the patch on branches
Nice feature !! On Fri, Jan 30, 2015 at 12:06 PM, lars hofhansl la...@apache.org wrote: This is absolutely awesome. Thanks Enis! From: Enis Söztutar e...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Thursday, January 29, 2015 10:31 PM Subject: Precommit build (hadoopqa) can now test the patch on branches Devs, Over at HBASE-12944, we've made some changes so that the Precommit jenkins build (commonly known and loved as hadoopqa) can now test patches on branches. For testing a patch on a branch, you should use the name of the branch in name of the patch file. For example a patch file named hbase-123-0.98.patch will test the hadoopqa build on top of the 0.98 branch. Right now, all active branches, master, branch-1.0, branch-1, 0.98 and 0.94 are supported. If no branch name is found in the patch file, master will be used. Also keep in mind that hadoopqa only picks up the latest patch from an issue in Patch Available state. This means that you cannot attach 3 patches at the same time for different branches and expect hadoopqa to report on all 3. The way to test a patch for master and branch-1 for example would be to attach the master patch, wait for the precommit build [1] to start for your patch, and then after it started attach the branch-1 patch with branch-1 in the patch file name. There is another job called PreCommit-Admin [2] who looks at the jira issues and kicks the precommit build every 10 minutes. It means you have to wait at least 10 minutes in between (or kick [2] manually if you are a committer). [1] https://builds.apache.org/job/PreCommit-HBASE-Build/ [2] https://builds.apache.org/view/All/job/PreCommit-Admin/ Cheers, Enis
Re: Precommit build (hadoopqa) can now test the patch on branches
This is absolutely awesome. Thanks Enis! From: Enis Söztutar e...@apache.org To: dev@hbase.apache.org dev@hbase.apache.org Sent: Thursday, January 29, 2015 10:31 PM Subject: Precommit build (hadoopqa) can now test the patch on branches Devs, Over at HBASE-12944, we've made some changes so that the Precommit jenkins build (commonly known and loved as hadoopqa) can now test patches on branches. For testing a patch on a branch, you should use the name of the branch in name of the patch file. For example a patch file named hbase-123-0.98.patch will test the hadoopqa build on top of the 0.98 branch. Right now, all active branches, master, branch-1.0, branch-1, 0.98 and 0.94 are supported. If no branch name is found in the patch file, master will be used. Also keep in mind that hadoopqa only picks up the latest patch from an issue in Patch Available state. This means that you cannot attach 3 patches at the same time for different branches and expect hadoopqa to report on all 3. The way to test a patch for master and branch-1 for example would be to attach the master patch, wait for the precommit build [1] to start for your patch, and then after it started attach the branch-1 patch with branch-1 in the patch file name. There is another job called PreCommit-Admin [2] who looks at the jira issues and kicks the precommit build every 10 minutes. It means you have to wait at least 10 minutes in between (or kick [2] manually if you are a committer). [1] https://builds.apache.org/job/PreCommit-HBASE-Build/ [2] https://builds.apache.org/view/All/job/PreCommit-Admin/ Cheers, Enis
Precommit build (hadoopqa) can now test the patch on branches
Devs, Over at HBASE-12944, we've made some changes so that the Precommit jenkins build (commonly known and loved as hadoopqa) can now test patches on branches. For testing a patch on a branch, you should use the name of the branch in name of the patch file. For example a patch file named hbase-123-0.98.patch will test the hadoopqa build on top of the 0.98 branch. Right now, all active branches, master, branch-1.0, branch-1, 0.98 and 0.94 are supported. If no branch name is found in the patch file, master will be used. Also keep in mind that hadoopqa only picks up the latest patch from an issue in Patch Available state. This means that you cannot attach 3 patches at the same time for different branches and expect hadoopqa to report on all 3. The way to test a patch for master and branch-1 for example would be to attach the master patch, wait for the precommit build [1] to start for your patch, and then after it started attach the branch-1 patch with branch-1 in the patch file name. There is another job called PreCommit-Admin [2] who looks at the jira issues and kicks the precommit build every 10 minutes. It means you have to wait at least 10 minutes in between (or kick [2] manually if you are a committer). [1] https://builds.apache.org/job/PreCommit-HBASE-Build/ [2] https://builds.apache.org/view/All/job/PreCommit-Admin/ Cheers, Enis
