[jira] [Commented] (HBASE-6844) upgrade 0.23 version dependency in 0.94
[ https://issues.apache.org/jira/browse/HBASE-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459369#comment-13459369 ] Hudson commented on HBASE-6844: --- Integrated in HBase-0.94 #474 (See [https://builds.apache.org/job/HBase-0.94/474/]) HBASE-6844 upgrade 0.23 version dependency in 0.94 (Revision 1387856) Result = SUCCESS stack : Files : * /hbase/branches/0.94/pom.xml upgrade 0.23 version dependency in 0.94 --- Key: HBASE-6844 URL: https://issues.apache.org/jira/browse/HBASE-6844 Project: HBase Issue Type: Bug Reporter: Francis Liu Assignee: Francis Liu Fix For: 0.92.3, 0.94.2 Attachments: 6844-092.txt, 6844.txt hadoop 0.23 has been promoted to stable. The snapshot jar no longer exists in maven. https://repository.apache.org/content/repositories/releases/org/apache/hadoop/hadoop-common/0.23.3/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6746) Impacts of HBASE-6435 vs. HDFS 2.0 trunk
[ https://issues.apache.org/jira/browse/HBASE-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6746: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) This was committed to trunk a while back. Thanks Nicolas Impacts of HBASE-6435 vs. HDFS 2.0 trunk Key: HBASE-6746 URL: https://issues.apache.org/jira/browse/HBASE-6746 Project: HBase Issue Type: Bug Components: master, regionserver, test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.96.0 Attachments: 6746.v1.patch When using the trunk of HDFS branch 2, I had two errors linked to HBASE-6435: - a missing test to null - a method removed. This fixes it: - add the test - make the test case less dependant on HDFS internal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6730: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks for the patch Elliott. Should this be the default balancer? If so, put up a patch to make it so (I think we could at least try it; if we get it in early now it'll be part of the general 0.96 testing and if issues, they will surface...) Enable rolling averages in StochasticLoadBalancer -- Key: HBASE-6730 URL: https://issues.apache.org/jira/browse/HBASE-6730 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, HBASE-6730-3.patch, HBASE-6730-4.patch, HBASE-6730-5.patch Now that all of the ServerLoad transitions into pb are done. the load balancer should get more updates about the current state of the cluster. This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6670) Untangle mixture of protobuf and Writable reference / usage
[ https://issues.apache.org/jira/browse/HBASE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6670: - Priority: Critical (was: Major) Lets try and do this in 0.96. It'd be a milestone. Making critical so gets some attention. Untangle mixture of protobuf and Writable reference / usage --- Key: HBASE-6670 URL: https://issues.apache.org/jira/browse/HBASE-6670 Project: HBase Issue Type: Bug Reporter: Ted Yu Priority: Critical Fix For: 0.96.0 Currently HbaseObjectWritable uses ProtobufUtil to perform serialization of Scan objects, ProtobufUtil.toParameter() calls HbaseObjectWritable.writeObject(). We should untangle such mixture and ultimately remove HbaseObjectWritable -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6702: - Priority: Critical (was: Major) Making critical so this gets some consideration before we cut 0.96. Seems like a useful facility that we are letting rot. I like Jesse suggestion of common base class, etc. ResourceChecker refinement -- Key: HBASE-6702 URL: https://issues.apache.org/jira/browse/HBASE-6702 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.96.0 Reporter: Jesse Yates Priority: Critical Fix For: 0.96.0 This was based on some discussion from HBASE-6234. The ResourceChecker was added by N. Keywal to help resolve some hadoop qa issues, but has since not be widely utilized. Further, with modularization we have had to drop the ResourceChecker from the tests that are moved into the hbase-common module because bringing the ResourceChecker up to hbase-common would involved bringing all its dependencies (which are quite far reaching). The question then is, what should we do with it? Get rid of it? Refactor and resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6649) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1]
[ https://issues.apache.org/jira/browse/HBASE-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6649: - Priority: Blocker (was: Major) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1] --- Key: HBASE-6649 URL: https://issues.apache.org/jira/browse/HBASE-6649 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Priority: Blocker Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: 6649-0.92.patch, 6649-1.patch, 6649-2.txt, 6649-fix-io-exception-handling-1.patch, 6649-fix-io-exception-handling-1-trunk.patch, 6649-fix-io-exception-handling.patch, 6649-trunk.patch, 6649-trunk.patch, 6649.txt, HBase-0.92 #495 test - queueFailover [Jenkins].html, HBase-0.92 #502 test - queueFailover [Jenkins].html Have seen it twice in the recent past: http://bit.ly/MPCykB http://bit.ly/O79Dq7 .. Looking briefly at the logs hints at a pattern - in both the failed test instances, there was an RS crash while the test was running. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6441) MasterFS doesn't set scheme for internal FileSystem
[ https://issues.apache.org/jira/browse/HBASE-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6441: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) This has been committed to trunk. Open new issue if to commit to 0.94. MasterFS doesn't set scheme for internal FileSystem --- Key: HBASE-6441 URL: https://issues.apache.org/jira/browse/HBASE-6441 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.96.0, 0.94.2 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Attachments: java_HBASE-6441_v0.patch FSUtils.getRootDir() just takes a configuration object, which is used to: 1) Get the name of the root directory 2) Create a filesystem (based on the configured scheme) 3) Qualify the root onto the filesystem However, the FileSystem from the master filesystem won't generate the correctly qualified root directory under hadoop-2.0 (though it works fine on hadoop-1.0). Seems to be an issue with the configuration parameters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4050) Update HBase metrics framework to metrics2 framework
[ https://issues.apache.org/jira/browse/HBASE-4050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459386#comment-13459386 ] Elliott Clark commented on HBASE-4050: -- One of the patches above was the one that added the compat jars and the first versions of some of the classes that were used. A couple patches have since been committed with hbase-4050 in the commit message even though the code was from sub-issues. This really has one issue left to solve and then the metrics2 move will be complete. Now that all testing is possible and all the helper classes are in (MetricsAssertHelper, MetricsHistogram, and MetricQuantile) just the region server is left. I'll start work on that. However it will be the most detail oriented so it might take a little longer than some of the other sub-issues. Update HBase metrics framework to metrics2 framework Key: HBASE-4050 URL: https://issues.apache.org/jira/browse/HBASE-4050 Project: HBase Issue Type: New Feature Components: metrics Affects Versions: 0.90.4 Environment: Java 6 Reporter: Eric Yang Assignee: Elliott Clark Priority: Critical Fix For: 0.96.0 Attachments: 4050-metrics-v2.patch, 4050-metrics-v3.patch, HBASE-4050-0.patch, HBASE-4050-1.patch, HBASE-4050-2.patch, HBASE-4050-3.patch, HBASE-4050-5.patch, HBASE-4050-6.patch, HBASE-4050-7.patch, HBASE-4050-8_1.patch, HBASE-4050-8.patch, HBASE-4050.patch Metrics Framework has been marked deprecated in Hadoop 0.20.203+ and 0.22+, and it might get removed in future Hadoop release. Hence, HBase needs to revise the dependency of MetricsContext to use Metrics2 framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6842) the jar used in coprocessor is not deleted in local which will exhaust the space of /tmp
[ https://issues.apache.org/jira/browse/HBASE-6842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459393#comment-13459393 ] Hudson commented on HBASE-6842: --- Integrated in HBase-TRUNK #3358 (See [https://builds.apache.org/job/HBase-TRUNK/3358/]) HBASE-6842 the jar used in coprocessor is not deleted in local which will exhaust the space of /tmp (Revision 1387861) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java the jar used in coprocessor is not deleted in local which will exhaust the space of /tmp --- Key: HBASE-6842 URL: https://issues.apache.org/jira/browse/HBASE-6842 Project: HBase Issue Type: Bug Components: coprocessors Affects Versions: 0.94.1 Reporter: Zhou wenjian Assignee: Zhou wenjian Priority: Critical Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6842-trunk.patch FileSystem fs = path.getFileSystem(HBaseConfiguration.create()); Path dst = new Path(System.getProperty(java.io.tmpdir) + java.io.File.separator +. + pathPrefix + . + className + . + System.currentTimeMillis() + .jar); fs.copyToLocalFile(path, dst); fs.deleteOnExit(dst); change to File tmpLocal = new File(dst.toString()); tmpLocal.deleteOnExit(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459394#comment-13459394 ] Hudson commented on HBASE-6730: --- Integrated in HBase-TRUNK #3358 (See [https://builds.apache.org/job/HBase-TRUNK/3358/]) HBASE-6730 Enable rolling averages in StochasticLoadBalancer (Revision 1387865) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerChore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/ClusterStatusChore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java Enable rolling averages in StochasticLoadBalancer -- Key: HBASE-6730 URL: https://issues.apache.org/jira/browse/HBASE-6730 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, HBASE-6730-3.patch, HBASE-6730-4.patch, HBASE-6730-5.patch Now that all of the ServerLoad transitions into pb are done. the load balancer should get more updates about the current state of the cluster. This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6178) LoadTest tool no longer packaged after the modularization
[ https://issues.apache.org/jira/browse/HBASE-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459395#comment-13459395 ] Hudson commented on HBASE-6178: --- Integrated in HBase-TRUNK #3358 (See [https://builds.apache.org/job/HBase-TRUNK/3358/]) HBASE-6178 LoadTest tool no longer packaged after the modularization (Revision 1387860) Result = FAILURE stack : Files : * /hbase/trunk/pom.xml * /hbase/trunk/src/assembly/components.xml * /hbase/trunk/src/assembly/hadoop-one-compat.xml * /hbase/trunk/src/assembly/hadoop-two-compat.xml LoadTest tool no longer packaged after the modularization - Key: HBASE-6178 URL: https://issues.apache.org/jira/browse/HBASE-6178 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jesse Yates Fix For: 0.96.0 Attachments: hbase-6178-v0.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6839) Operations may be executed without rowLock
[ https://issues.apache.org/jira/browse/HBASE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459396#comment-13459396 ] Hudson commented on HBASE-6839: --- Integrated in HBase-TRUNK #3358 (See [https://builds.apache.org/job/HBase-TRUNK/3358/]) HBASE-6839 Operations may be executed without rowLock (Revision 1387863) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Operations may be executed without rowLock -- Key: HBASE-6839 URL: https://issues.apache.org/jira/browse/HBASE-6839 Project: HBase Issue Type: Bug Components: regionserver Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6839.patch HRegion#internalObtainRowLock will return null if timed out, but many place which call this method don't handle this case The bad result is operation will be executed even if it havn't obtained the row lock. Such as put、delete、increment。。。 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6806) HBASE-4658 breaks backward compatibility / example scripts
[ https://issues.apache.org/jira/browse/HBASE-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459408#comment-13459408 ] Lukas commented on HBASE-6806: -- python, c++ and java: worksforme™ I had no chance to test php, perl and ruby, they might even have syntactic errors in it... HBASE-4658 breaks backward compatibility / example scripts -- Key: HBASE-6806 URL: https://issues.apache.org/jira/browse/HBASE-6806 Project: HBase Issue Type: Bug Components: thrift Affects Versions: 0.94.0 Reporter: Lukas Attachments: HBASE-6806-fix-examples.diff HBASE-4658 introduces the new 'attributes' argument as a non optional parameter. This is not backward compatible and also breaks the code in the example section. Resolution: Mark as 'optional' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6844) upgrade 0.23 version dependency in 0.94
[ https://issues.apache.org/jira/browse/HBASE-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459412#comment-13459412 ] Hudson commented on HBASE-6844: --- Integrated in HBase-0.92 #581 (See [https://builds.apache.org/job/HBase-0.92/581/]) HBASE-6844 upgrade 0.23 version dependency in 0.94 (Revision 1387858) Result = FAILURE stack : Files : * /hbase/branches/0.92/pom.xml upgrade 0.23 version dependency in 0.94 --- Key: HBASE-6844 URL: https://issues.apache.org/jira/browse/HBASE-6844 Project: HBase Issue Type: Bug Reporter: Francis Liu Assignee: Francis Liu Fix For: 0.92.3, 0.94.2 Attachments: 6844-092.txt, 6844.txt hadoop 0.23 has been promoted to stable. The snapshot jar no longer exists in maven. https://repository.apache.org/content/repositories/releases/org/apache/hadoop/hadoop-common/0.23.3/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6842) the jar used in coprocessor is not deleted in local which will exhaust the space of /tmp
[ https://issues.apache.org/jira/browse/HBASE-6842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459418#comment-13459418 ] Hudson commented on HBASE-6842: --- Integrated in HBase-0.94 #475 (See [https://builds.apache.org/job/HBase-0.94/475/]) HBASE-6842 the jar used in coprocessor is not deleted in local which will exhaust the space of /tmp (Revision 1387862) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java the jar used in coprocessor is not deleted in local which will exhaust the space of /tmp --- Key: HBASE-6842 URL: https://issues.apache.org/jira/browse/HBASE-6842 Project: HBase Issue Type: Bug Components: coprocessors Affects Versions: 0.94.1 Reporter: Zhou wenjian Assignee: Zhou wenjian Priority: Critical Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6842-trunk.patch FileSystem fs = path.getFileSystem(HBaseConfiguration.create()); Path dst = new Path(System.getProperty(java.io.tmpdir) + java.io.File.separator +. + pathPrefix + . + className + . + System.currentTimeMillis() + .jar); fs.copyToLocalFile(path, dst); fs.deleteOnExit(dst); change to File tmpLocal = new File(dst.toString()); tmpLocal.deleteOnExit(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6839) Operations may be executed without rowLock
[ https://issues.apache.org/jira/browse/HBASE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459419#comment-13459419 ] Hudson commented on HBASE-6839: --- Integrated in HBase-0.94 #475 (See [https://builds.apache.org/job/HBase-0.94/475/]) HBASE-6839 Operations may be executed without rowLock (Revision 1387864) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Operations may be executed without rowLock -- Key: HBASE-6839 URL: https://issues.apache.org/jira/browse/HBASE-6839 Project: HBase Issue Type: Bug Components: regionserver Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6839.patch HRegion#internalObtainRowLock will return null if timed out, but many place which call this method don't handle this case The bad result is operation will be executed even if it havn't obtained the row lock. Such as put、delete、increment。。。 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6381) AssignmentManager should use the same logic for clean startup and failover
[
https://issues.apache.org/jira/browse/HBASE-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459440#comment-13459440
]
rajeshbabu commented on HBASE-6381:
---
@Jimmy,
In the following scenario region assignment may not happen with the latest
patch:
1)lets suppose a region R1 is moving from RS1 to RS2
2)if the master and RS1 restarted before update server info for R1 with RS2 in
META.(during region open in RS)
3)in rebuild user regions we will select R1 as dead region on dead server RS1.
4)Now server info updated in META with RS2.
5)In processDeadServersAndRecoverLostRegions we will expiry server and delete
znode of the region.
{code}
if (!serverManager.isServerDead(serverName)) {
serverManager.expireServer(serverName); // Let SSH do region re-assign
}
if (!nodes.isEmpty()) {
for (HRegionInfo deadRegion : server.getValue()) {
String encodedName = deadRegion.getEncodedName();
if (nodes.remove(encodedName)) {
ZKAssign.deleteNodeFailSilent(watcher, deadRegion);
}
}
}
{code}
6)if the znode deletion happened before transitioning to opened,then the region
wont be online.
{code}
if (!transitionToOpened(region)) {
// If we fail to transition to opened, it's because of one of two cases:
//(a) we lost our ZK lease
// OR (b) someone else opened the region before us
// In either case, we don't need to transition to FAILED_OPEN state.
// In case (a), the Master will process us as a dead server. In case
// (b) the region is already being handled elsewhere anyway.
cleanupFailedOpen(region);
return;
}
{code}
Even while processing SSH of RS1 also we wont assign it because in META server
info got changed to RS2.
Please correct me if wrong.
AssignmentManager should use the same logic for clean startup and failover
--
Key: HBASE-6381
URL: https://issues.apache.org/jira/browse/HBASE-6381
Project: HBase
Issue Type: Bug
Components: master
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: hbase-6381-notes.pdf, hbase-6381.pdf,
trunk-6381_v5.patch, trunk-6381_v7.patch, trunk-6381_v8.patch
Currently AssignmentManager handles clean startup and failover very
differently.
Different logic is mingled together so it is hard to find out which is for
which.
We should clean it up and share the same logic so that AssignmentManager
handles
both cases the same way. This way, the code will much easier to understand
and
maintain.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5843) Improve HBase MTTR - Mean Time To Recover
[ https://issues.apache.org/jira/browse/HBASE-5843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459444#comment-13459444 ] nkeywal commented on HBASE-5843: bq. Interesting. The sad part is we often find ourselves having to increase the ZK timeout in order to deal with Juliet GC pauses. Given that detection time dominates, perhaps we should put some effort into correcting that (multiple RS on a single box?) Imho, multiple RS on the same box would put us in a dead end: it increases the number of tcp connections, add workload on ZooKeeper, makes the balancer more complicated, ... We can also have operational issues (rolling upgrade, fixed tcp ports, ...). The possible options I know are: - improving ZooKeeper to have an algorithm that takes variance into account: it's a common solution to have a good failure detection while minimizing wrong positive. 20 years ago, it saved TCP from dying by congestion. There is ZOOKEEPER-702 about this. That's medium term (hopes...), but would be useful for HDFS HA also. - Using the new gc options available in JDK 1.7 (see http://www.oracle.com/technetwork/java/javase/tech/g1-intro-jsp-135488.html). That's short term, simple. Only issue, it has been tried a few month ago (by Andrew Purtell IIRC), and crashed the JVM. Still, it's something to look at, and may be we should raise the bugs to Oracle if we find some. - The offheap mentioned by Stack. I don't think it's one or another, we're likely to need all of them :-). Still, knowing where we stand in regards of JDK 1.7 is important imho. bq. Yes, CPU definitely need a diet. Probably start with eliminating a bunch of threads. It's not directly MTTR, but I agree, we have far too many threads, and far too many thread pools. Not only it's bad for performance, it makes analysing the performances complicated. bq. Right, I think HBASE-6752 is a great idea, but it doesn't address serving reads more quickly. I'm wondering if there is more we can do to address that. There is HBASE-6774 for the special case of empty hlog regions. It would be interesting to see how many regions are in this situation on different production clusters. There are so many ways to be in this situation... I would love to have a stat on at a given point of time, what's the proportion of the regions with a non empty memstore. And improving memstore flush policy would lead us to improvement here as well I think. With HBASE-6752 we serve as well timeranged reads (if they're lucky on the range). But yep, we don't cover all cases. Ideas welcome :-) bq. Why do you say this with respect to locking? Is the performance not as good as you would expect? Or just haven't looked at it yet? I was expecting much better performances, but I haven't looked enough at it. bq. I've wondered why we don't do this. Do you see any implementation challenges with doing this? Maybe I'll look into it. Well, it's closed to the assignment part, so... :-) But it would be great if you can have a look at this, because with all the discussions around assignment, it's important to take these new use cases into account as well.. Improve HBase MTTR - Mean Time To Recover - Key: HBASE-5843 URL: https://issues.apache.org/jira/browse/HBASE-5843 Project: HBase Issue Type: Umbrella Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal A part of the approach is described here: https://docs.google.com/document/d/1z03xRoZrIJmg7jsWuyKYl6zNournF_7ZHzdi0qz_B4c/edit The ideal target is: - failure impact client applications only by an added delay to execute a query, whatever the failure. - this delay is always inferior to 1 second. We're not going to achieve that immediately... Priority will be given to the most frequent issues. Short term: - software crash - standard administrative tasks as stop/start of a cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459459#comment-13459459 ] nkeywal commented on HBASE-6702: Yeah. I don't have any opinion in the field vs. common base class approach. Both are ok to me. What I don't remember is if I was not constrained by the JUnit annotation approach. I just remember I had to try multiple options, before finding a suitable one. Something to consider as well is to declare the runlistener in the pom. I know I didn't try it at this time, but it could be simpler actually (it would be nothing to add at all :-) ). I will give it a try. ResourceChecker refinement -- Key: HBASE-6702 URL: https://issues.apache.org/jira/browse/HBASE-6702 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.96.0 Reporter: Jesse Yates Priority: Critical Fix For: 0.96.0 This was based on some discussion from HBASE-6234. The ResourceChecker was added by N. Keywal to help resolve some hadoop qa issues, but has since not be widely utilized. Further, with modularization we have had to drop the ResourceChecker from the tests that are moved into the hbase-common module because bringing the ResourceChecker up to hbase-common would involved bringing all its dependencies (which are quite far reaching). The question then is, what should we do with it? Get rid of it? Refactor and resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6783) Make read short circuit the default
[ https://issues.apache.org/jira/browse/HBASE-6783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6783: --- Attachment: 6783.v2.patch Make read short circuit the default --- Key: HBASE-6783 URL: https://issues.apache.org/jira/browse/HBASE-6783 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.96.0 Attachments: 6783.v2.patch, HBASE-6783.v1.patch Per mailing discussion, read short circuit has little or no drawback, hence should used by default. As a consequence, we activate it on the default tests. It's possible to launch the test with -Ddfs.client.read.shortcircuit=false to execute the tests without the shortcircuit, it will be used for some builds on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6783) Make read short circuit the default
[ https://issues.apache.org/jira/browse/HBASE-6783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6783: --- Status: Open (was: Patch Available) Make read short circuit the default --- Key: HBASE-6783 URL: https://issues.apache.org/jira/browse/HBASE-6783 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.96.0 Attachments: 6783.v2.patch, 6783.v2.patch, HBASE-6783.v1.patch Per mailing discussion, read short circuit has little or no drawback, hence should used by default. As a consequence, we activate it on the default tests. It's possible to launch the test with -Ddfs.client.read.shortcircuit=false to execute the tests without the shortcircuit, it will be used for some builds on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6783) Make read short circuit the default
[ https://issues.apache.org/jira/browse/HBASE-6783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6783: --- Attachment: 6783.v2.patch Make read short circuit the default --- Key: HBASE-6783 URL: https://issues.apache.org/jira/browse/HBASE-6783 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.96.0 Attachments: 6783.v2.patch, 6783.v2.patch, HBASE-6783.v1.patch Per mailing discussion, read short circuit has little or no drawback, hence should used by default. As a consequence, we activate it on the default tests. It's possible to launch the test with -Ddfs.client.read.shortcircuit=false to execute the tests without the shortcircuit, it will be used for some builds on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6783) Make read short circuit the default
[ https://issues.apache.org/jira/browse/HBASE-6783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6783: --- Status: Patch Available (was: Open) Make read short circuit the default --- Key: HBASE-6783 URL: https://issues.apache.org/jira/browse/HBASE-6783 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.96.0 Attachments: 6783.v2.patch, 6783.v2.patch, HBASE-6783.v1.patch Per mailing discussion, read short circuit has little or no drawback, hence should used by default. As a consequence, we activate it on the default tests. It's possible to launch the test with -Ddfs.client.read.shortcircuit=false to execute the tests without the shortcircuit, it will be used for some builds on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6783) Make read short circuit the default
[ https://issues.apache.org/jira/browse/HBASE-6783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459520#comment-13459520 ] Hadoop QA commented on HBASE-6783: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12545879/6783.v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. -1 javadoc. The javadoc tool appears to have generated 139 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2904//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2904//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2904//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2904//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2904//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2904//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2904//console This message is automatically generated. Make read short circuit the default --- Key: HBASE-6783 URL: https://issues.apache.org/jira/browse/HBASE-6783 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.96.0 Attachments: 6783.v2.patch, 6783.v2.patch, HBASE-6783.v1.patch Per mailing discussion, read short circuit has little or no drawback, hence should used by default. As a consequence, we activate it on the default tests. It's possible to launch the test with -Ddfs.client.read.shortcircuit=false to execute the tests without the shortcircuit, it will be used for some builds on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6677) Random ZooKeeper port in test can overrun max port
[
https://issues.apache.org/jira/browse/HBASE-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liang xie updated HBASE-6677:
-
Attachment: HBASE-6677.patch
one line change, it should be safe, i didn't run any test case...
Random ZooKeeper port in test can overrun max port
--
Key: HBASE-6677
URL: https://issues.apache.org/jira/browse/HBASE-6677
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.96.0
Reporter: Gregory Chanan
Priority: Trivial
Labels: noob
Attachments: HBASE-6677.patch
{code}
while (true) {
try {
standaloneServerFactory = new NIOServerCnxnFactory();
standaloneServerFactory.configure(
new InetSocketAddress(tentativePort),
configuration.getInt(HConstants.ZOOKEEPER_MAX_CLIENT_CNXNS,
1000));
} catch (BindException e) {
LOG.debug(Failed binding ZK Server to client port: +
tentativePort);
// This port is already in use, try to use another.
tentativePort++;
continue;
}
break;
}
{code}
In the case of failure and all the above ports have already been binded, you
can extend past the max port. Need to check against a max value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6677) Random ZooKeeper port in test can overrun max port
[
https://issues.apache.org/jira/browse/HBASE-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liang xie updated HBASE-6677:
-
Status: Patch Available (was: Open)
Random ZooKeeper port in test can overrun max port
--
Key: HBASE-6677
URL: https://issues.apache.org/jira/browse/HBASE-6677
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.96.0
Reporter: Gregory Chanan
Priority: Trivial
Labels: noob
Attachments: HBASE-6677.patch
{code}
while (true) {
try {
standaloneServerFactory = new NIOServerCnxnFactory();
standaloneServerFactory.configure(
new InetSocketAddress(tentativePort),
configuration.getInt(HConstants.ZOOKEEPER_MAX_CLIENT_CNXNS,
1000));
} catch (BindException e) {
LOG.debug(Failed binding ZK Server to client port: +
tentativePort);
// This port is already in use, try to use another.
tentativePort++;
continue;
}
break;
}
{code}
In the case of failure and all the above ports have already been binded, you
can extend past the max port. Need to check against a max value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6178) LoadTest tool no longer packaged after the modularization
[ https://issues.apache.org/jira/browse/HBASE-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459529#comment-13459529 ] Hudson commented on HBASE-6178: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #183 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/183/]) HBASE-6178 LoadTest tool no longer packaged after the modularization (Revision 1387860) Result = FAILURE stack : Files : * /hbase/trunk/pom.xml * /hbase/trunk/src/assembly/components.xml * /hbase/trunk/src/assembly/hadoop-one-compat.xml * /hbase/trunk/src/assembly/hadoop-two-compat.xml LoadTest tool no longer packaged after the modularization - Key: HBASE-6178 URL: https://issues.apache.org/jira/browse/HBASE-6178 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jesse Yates Fix For: 0.96.0 Attachments: hbase-6178-v0.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6842) the jar used in coprocessor is not deleted in local which will exhaust the space of /tmp
[ https://issues.apache.org/jira/browse/HBASE-6842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459527#comment-13459527 ] Hudson commented on HBASE-6842: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #183 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/183/]) HBASE-6842 the jar used in coprocessor is not deleted in local which will exhaust the space of /tmp (Revision 1387861) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java the jar used in coprocessor is not deleted in local which will exhaust the space of /tmp --- Key: HBASE-6842 URL: https://issues.apache.org/jira/browse/HBASE-6842 Project: HBase Issue Type: Bug Components: coprocessors Affects Versions: 0.94.1 Reporter: Zhou wenjian Assignee: Zhou wenjian Priority: Critical Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6842-trunk.patch FileSystem fs = path.getFileSystem(HBaseConfiguration.create()); Path dst = new Path(System.getProperty(java.io.tmpdir) + java.io.File.separator +. + pathPrefix + . + className + . + System.currentTimeMillis() + .jar); fs.copyToLocalFile(path, dst); fs.deleteOnExit(dst); change to File tmpLocal = new File(dst.toString()); tmpLocal.deleteOnExit(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6730) Enable rolling averages in StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459528#comment-13459528 ] Hudson commented on HBASE-6730: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #183 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/183/]) HBASE-6730 Enable rolling averages in StochasticLoadBalancer (Revision 1387865) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerChore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/ClusterStatusChore.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java Enable rolling averages in StochasticLoadBalancer -- Key: HBASE-6730 URL: https://issues.apache.org/jira/browse/HBASE-6730 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 Attachments: HBASE-6730-1.patch, HBASE-6730-2.patch, HBASE-6730-3.patch, HBASE-6730-4.patch, HBASE-6730-5.patch Now that all of the ServerLoad transitions into pb are done. the load balancer should get more updates about the current state of the cluster. This should be used to allow StochasticLoadBalancer to get rolling averages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6839) Operations may be executed without rowLock
[ https://issues.apache.org/jira/browse/HBASE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459530#comment-13459530 ] Hudson commented on HBASE-6839: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #183 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/183/]) HBASE-6839 Operations may be executed without rowLock (Revision 1387863) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Operations may be executed without rowLock -- Key: HBASE-6839 URL: https://issues.apache.org/jira/browse/HBASE-6839 Project: HBase Issue Type: Bug Components: regionserver Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6839.patch HRegion#internalObtainRowLock will return null if timed out, but many place which call this method don't handle this case The bad result is operation will be executed even if it havn't obtained the row lock. Such as put、delete、increment。。。 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6501) Integrate with unit-testing tools of hadoop's metrics2 framework
[ https://issues.apache.org/jira/browse/HBASE-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459526#comment-13459526 ] Hudson commented on HBASE-6501: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #183 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/183/]) HBASE-6501 Integrate with unit-testing tools of hadoop's metrics2 framework (Revision 1387820) Result = FAILURE stack : Files : * /hbase/trunk/hbase-hadoop-compat/src/test/java/org/apache/hadoop/hbase/test/MetricsAssertHelper.java * /hbase/trunk/hbase-hadoop1-compat/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterMetricsSourceImpl.java * /hbase/trunk/hbase-hadoop1-compat/src/test/java/org/apache/hadoop/hbase/test/MetricsAssertHelperImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterMetricsSourceImpl.java * /hbase/trunk/hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/test/MetricsAssertHelperImpl.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterStatistics.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetrics.java Integrate with unit-testing tools of hadoop's metrics2 framework Key: HBASE-6501 URL: https://issues.apache.org/jira/browse/HBASE-6501 Project: HBase Issue Type: Sub-task Components: metrics Reporter: Alex Baranau Assignee: Elliott Clark Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-6501-0.patch, HBASE-6501_2.patch, HBASE-6501_3.patch, HBASE-6501.patch Hadoop's metrics2 framework provides handy tools to write unit-tests for metrics sources. E.g. MetricsAsserts class. We want to use that too in HBase unit-tests. Integration seems straightforward, wowever when integrating this piece we faced maven bug: http://jira.codehaus.org/browse/MRRESOURCES-53. Hence we decided to extract this into separate issue (originally was done in one of the patches of HBASE-6411). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459540#comment-13459540 ] ramkrishna.s.vasudevan commented on HBASE-6698: --- Oops, yes HBASE-6769 is the reason. I will update the patch.. Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6839) Operations may be executed without rowLock
[ https://issues.apache.org/jira/browse/HBASE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459546#comment-13459546 ] ramkrishna.s.vasudevan commented on HBASE-6839: --- Nice one Chunhui. Operations may be executed without rowLock -- Key: HBASE-6839 URL: https://issues.apache.org/jira/browse/HBASE-6839 Project: HBase Issue Type: Bug Components: regionserver Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6839.patch HRegion#internalObtainRowLock will return null if timed out, but many place which call this method don't handle this case The bad result is operation will be executed even if it havn't obtained the row lock. Such as put、delete、increment。。。 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6698: -- Attachment: HBASE-6698_8.patch Hope fully this should be fine. I have handled FailedSanitycheckException and NoSuchColumnFamilyException seperately. Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6698: -- Status: Open (was: Patch Available) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6698: -- Status: Patch Available (was: Open) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6698: -- Status: Open (was: Patch Available) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6698: -- Attachment: HBASE-6698_8.patch Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6698: -- Status: Patch Available (was: Open) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6677) Random ZooKeeper port in test can overrun max port
[
https://issues.apache.org/jira/browse/HBASE-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459563#comment-13459563
]
Hadoop QA commented on HBASE-6677:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12545886/HBASE-6677.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
-1 javadoc. The javadoc tool appears to have generated 139 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 14 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2905//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2905//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2905//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2905//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2905//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2905//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2905//console
This message is automatically generated.
Random ZooKeeper port in test can overrun max port
--
Key: HBASE-6677
URL: https://issues.apache.org/jira/browse/HBASE-6677
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.96.0
Reporter: Gregory Chanan
Priority: Trivial
Labels: noob
Attachments: HBASE-6677.patch
{code}
while (true) {
try {
standaloneServerFactory = new NIOServerCnxnFactory();
standaloneServerFactory.configure(
new InetSocketAddress(tentativePort),
configuration.getInt(HConstants.ZOOKEEPER_MAX_CLIENT_CNXNS,
1000));
} catch (BindException e) {
LOG.debug(Failed binding ZK Server to client port: +
tentativePort);
// This port is already in use, try to use another.
tentativePort++;
continue;
}
break;
}
{code}
In the case of failure and all the above ports have already been binded, you
can extend past the max port. Need to check against a max value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6839) Operations may be executed without holding rowLock
[ https://issues.apache.org/jira/browse/HBASE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6839: -- Summary: Operations may be executed without holding rowLock (was: Operations may be executed without rowLock) Operations may be executed without holding rowLock -- Key: HBASE-6839 URL: https://issues.apache.org/jira/browse/HBASE-6839 Project: HBase Issue Type: Bug Components: regionserver Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6839.patch HRegion#internalObtainRowLock will return null if timed out, but many place which call this method don't handle this case The bad result is operation will be executed even if it havn't obtained the row lock. Such as put、delete、increment。。。 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6839) Operations may be executed without holding rowLock
[ https://issues.apache.org/jira/browse/HBASE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459578#comment-13459578 ] Ted Yu commented on HBASE-6839: --- Integrated to 0.92 as well. Thanks, Chunhui. Operations may be executed without holding rowLock -- Key: HBASE-6839 URL: https://issues.apache.org/jira/browse/HBASE-6839 Project: HBase Issue Type: Bug Components: regionserver Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6839.patch HRegion#internalObtainRowLock will return null if timed out, but many place which call this method don't handle this case The bad result is operation will be executed even if it havn't obtained the row lock. Such as put、delete、increment。。。 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6845) Enable exponential weighted average for StochasticLoadBalancer
Ted Yu created HBASE-6845: - Summary: Enable exponential weighted average for StochasticLoadBalancer Key: HBASE-6845 URL: https://issues.apache.org/jira/browse/HBASE-6845 Project: HBase Issue Type: Bug Reporter: Ted Yu HBASE-6730 introduced rolling average for StochasticLoadBalancer http://tdunning.blogspot.com/2011/03/exponential-weighted-averages-with.html provided a better approach. We should implement it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3546) XSS in the WebUI
[
https://issues.apache.org/jira/browse/HBASE-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459586#comment-13459586
]
liang xie commented on HBASE-3546:
--
I tried to repro on 0.94 but failed, seems jamon(RegionListTmpl.jamon) did the
HTML escape. I saw the scriptalert('js')/script text in Start/End Key
columns from WebUI, w/o JS code running.
On 0.90, there's no escape in regionserver.jsp, this issue only exists on
0.90(or before), seems didn't worth to fix it?
Please correct me if i am wrong:)
XSS in the WebUI
Key: HBASE-3546
URL: https://issues.apache.org/jira/browse/HBASE-3546
Project: HBase
Issue Type: Bug
Reporter: Takuya Ueshin
Priority: Minor
There are possibilities of XSS in the WebUI.
If ColumnFamily or Region splitting keys are like
Bytes.toBytes(scriptalert('js')/script)
then browsers run the JavaScript code.
I tested on HBase-0.90.0 .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459597#comment-13459597 ] Hadoop QA commented on HBASE-6698: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12545896/HBASE-6698_8.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. -1 javadoc. The javadoc tool appears to have generated 139 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2906//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2906//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2906//console This message is automatically generated. Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6846) BitComparator bug - ArrayIndexOutOfBoundsException
Lucian George Iordache created HBASE-6846:
-
Summary: BitComparator bug - ArrayIndexOutOfBoundsException
Key: HBASE-6846
URL: https://issues.apache.org/jira/browse/HBASE-6846
Project: HBase
Issue Type: Bug
Components: filters
Affects Versions: 0.94.1
Environment: HBase 0.94.1 + Hadoop 2.0.0-cdh4.0.1
Reporter: Lucian George Iordache
The HBase 0.94.1 BitComparator introduced a bug in the method compareTo:
@Override
public int compareTo(byte[] value, int offset, int length) {
if (length != this.value.length) {
return 1;
}
int b = 0;
//Iterating backwards is faster because we can quit after one non-zero byte.
for (int i = value.length - 1; i = 0 b == 0; i--) {
switch (bitOperator) {
case AND:
b = (this.value[i] value[i+offset]) 0xff;
break;
case OR:
b = (this.value[i] | value[i+offset]) 0xff;
break;
case XOR:
b = (this.value[i] ^ value[i+offset]) 0xff;
break;
}
}
return b == 0 ? 1 : 0;
}
I've encountered this problem when using a BitComparator with a configured
this.value.length=8, and in the HBase table there were KeyValues with
keyValue.getBuffer().length=207911 bytes. In this case:
for (int i = 207910; i = 0 b == 0; i--) {
switch (bitOperator) {
case AND:
b = (this.value[207910] ... == ArrayIndexOutOfBoundsException
break;
That loop should use:
for (int i = length - 1; i = 0 b == 0; i--) { (or this.value.length.)
Should I provide a patch for correcting the problem?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6669) Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459648#comment-13459648 ] Julian Wissmann commented on HBASE-6669: I've written a Test for BDColumnInterpreter, however, all Medium Tests requiring MiniDFSCluster fail on my machine: --- Test set: org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol --- Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.239 sec FAILURE! org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol Time elapsed: 0.001 sec ERROR! java.lang.NullPointerException at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422) at org.apache.hadoop.hdfs.MiniDFSCluster.init(MiniDFSCluster.java:280) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:433) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:653) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:603) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:590) at org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol.setupBeforeClass(TestAggregateProtocol.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:24) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Therefor I have not been able to do a run of my test, so far. I can however attach it here, if someone who does not run into this problem is willing to give it a try. Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient -- Key: HBASE-6669 URL: https://issues.apache.org/jira/browse/HBASE-6669 Project: HBase Issue Type: New Feature Components: client, coprocessors Reporter: Anil Gupta Priority: Minor Labels: client, coprocessors Attachments: BigDecimalColumnInterpreter.java, BigDecimalColumnInterpreter.patch, BigDecimalColumnInterpreter.patch I recently created a Class for doing aggregations(sum,min,max,std) on values stored as BigDecimal in HBase. I would like to commit the BigDecimalColumnInterpreter into HBase. In my opinion this class can be used by a wide variety of users. Please let me know if its not appropriate to add this class in HBase. Thanks, Anil Gupta Software Engineer II, Intuit, Inc -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6839) Operations may be executed without holding rowLock
[ https://issues.apache.org/jira/browse/HBASE-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459650#comment-13459650 ] Hudson commented on HBASE-6839: --- Integrated in HBase-0.92 #582 (See [https://builds.apache.org/job/HBase-0.92/582/]) HBASE-6839 correct variable name : mutation should be put (Revision 1388009) HBASE-6839 Operations may be executed without holding rowLock (Chunhui) (Revision 1388004) Result = SUCCESS tedyu : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java tedyu : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Operations may be executed without holding rowLock -- Key: HBASE-6839 URL: https://issues.apache.org/jira/browse/HBASE-6839 Project: HBase Issue Type: Bug Components: regionserver Reporter: chunhui shen Assignee: chunhui shen Priority: Critical Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6839.patch HRegion#internalObtainRowLock will return null if timed out, but many place which call this method don't handle this case The bad result is operation will be executed even if it havn't obtained the row lock. Such as put、delete、increment。。。 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6501) Integrate with unit-testing tools of hadoop's metrics2 framework
[ https://issues.apache.org/jira/browse/HBASE-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459658#comment-13459658 ] Alex Baranau commented on HBASE-6501: - Thanx Elliott! So what was the solution for this maven problem? Did you adjust the build commands/script in Jenkins? Integrate with unit-testing tools of hadoop's metrics2 framework Key: HBASE-6501 URL: https://issues.apache.org/jira/browse/HBASE-6501 Project: HBase Issue Type: Sub-task Components: metrics Reporter: Alex Baranau Assignee: Elliott Clark Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-6501-0.patch, HBASE-6501_2.patch, HBASE-6501_3.patch, HBASE-6501.patch Hadoop's metrics2 framework provides handy tools to write unit-tests for metrics sources. E.g. MetricsAsserts class. We want to use that too in HBase unit-tests. Integration seems straightforward, wowever when integrating this piece we faced maven bug: http://jira.codehaus.org/browse/MRRESOURCES-53. Hence we decided to extract this into separate issue (originally was done in one of the patches of HBASE-6411). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6071) getRegionServerWithRetires, should log unsuccessful attempts and exceptions.
[ https://issues.apache.org/jira/browse/HBASE-6071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459660#comment-13459660 ] Harsh J commented on HBASE-6071: Nice patch. I'd prefer if we logged this at INFO itself though. This sorta retry is also known to kill remote bulkloads with tons of re-requests presently and we wouldn't know that if we didn't catch it via logs. Secondly, it is tough on many older versions of Hadoop to enable DEBUG level logs in MR (where this (LE) is often reported from). getRegionServerWithRetires, should log unsuccessful attempts and exceptions. Key: HBASE-6071 URL: https://issues.apache.org/jira/browse/HBASE-6071 Project: HBase Issue Type: Improvement Components: client, ipc Affects Versions: 0.92.0, 0.94.0 Reporter: Igal Shilman Priority: Minor Labels: client, ipc Attachments: HBASE-6071.patch, HBASE-6071.v2.patch, HBASE-6071.v3.patch, HConnectionManager_HBASE-6071-0.90.0.patch, lease-exception.txt HConnectionImplementation.getRegionServerWithRetries might terminate w/ an exception different then a DoNotRetryIOException, thus silently drops exceptions from previous attempts. [~ted_yu] suggested ([here|http://mail-archives.apache.org/mod_mbox/hbase-user/201205.mbox/%3CCAFebPXBq9V9BVdzRTNr-MB3a1Lz78SZj6gvP6On0b%2Bajt9StAg%40mail.gmail.com%3E]) adding a log message inside the catch block describing the exception type and details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6501) Integrate with unit-testing tools of hadoop's metrics2 framework
[ https://issues.apache.org/jira/browse/HBASE-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459675#comment-13459675 ] Elliott Clark commented on HBASE-6501: -- Yeah the dev-support scripts now do package, and I added some documentation that explains that package or install is needed to compile. Integrate with unit-testing tools of hadoop's metrics2 framework Key: HBASE-6501 URL: https://issues.apache.org/jira/browse/HBASE-6501 Project: HBase Issue Type: Sub-task Components: metrics Reporter: Alex Baranau Assignee: Elliott Clark Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-6501-0.patch, HBASE-6501_2.patch, HBASE-6501_3.patch, HBASE-6501.patch Hadoop's metrics2 framework provides handy tools to write unit-tests for metrics sources. E.g. MetricsAsserts class. We want to use that too in HBase unit-tests. Integration seems straightforward, wowever when integrating this piece we faced maven bug: http://jira.codehaus.org/browse/MRRESOURCES-53. Hence we decided to extract this into separate issue (originally was done in one of the patches of HBASE-6411). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[
https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459694#comment-13459694
]
ramkrishna.s.vasudevan commented on HBASE-6698:
---
org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics. This test cases
passes locally.
The error in the QA build seems to be different
{code}
java.io.IOException: Shutting down
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:227)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:89)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:693)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:666)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:661)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:603)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:572)
at
org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics.setUp(TestRegionServerMetrics.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{code}
Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
--
Key: HBASE-6698
URL: https://issues.apache.org/jira/browse/HBASE-6698
Project: HBase
Issue Type: Improvement
Reporter: ramkrishna.s.vasudevan
Fix For: 0.96.0
Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch,
HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch,
HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch,
HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch
Currently the checkAndPut and checkAndDelete api internally calls the
internalPut and internalDelete. May be we can just call doMiniBatchMutation
only. This will help in future like if we have some hooks and the CP
handles certain cases in the doMiniBatchMutation the same can be done while
doing a put thro checkAndPut or while doing a delete thro checkAndDelete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6071) getRegionServerWithRetires, should log unsuccessful attempts and exceptions.
[ https://issues.apache.org/jira/browse/HBASE-6071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-6071: - Fix Version/s: 0.94.2 0.96.0 getRegionServerWithRetires, should log unsuccessful attempts and exceptions. Key: HBASE-6071 URL: https://issues.apache.org/jira/browse/HBASE-6071 Project: HBase Issue Type: Improvement Components: client, ipc Affects Versions: 0.92.0, 0.94.0 Reporter: Igal Shilman Priority: Minor Labels: client, ipc Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6071.patch, HBASE-6071.v2.patch, HBASE-6071.v3.patch, HConnectionManager_HBASE-6071-0.90.0.patch, lease-exception.txt HConnectionImplementation.getRegionServerWithRetries might terminate w/ an exception different then a DoNotRetryIOException, thus silently drops exceptions from previous attempts. [~ted_yu] suggested ([here|http://mail-archives.apache.org/mod_mbox/hbase-user/201205.mbox/%3CCAFebPXBq9V9BVdzRTNr-MB3a1Lz78SZj6gvP6On0b%2Bajt9StAg%40mail.gmail.com%3E]) adding a log message inside the catch block describing the exception type and details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[
https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459730#comment-13459730
]
ramkrishna.s.vasudevan commented on HBASE-6698:
---
As per my analysis, before the HBaseRPCServer on the RS could start and set the
volatile variable 'started' the master started the assignment because the RS
registration got completed successfully.
{code}
Caused by: org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running
yet
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1798)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1300)
at
org.apache.hadoop.hbase.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:178)
... 11 more
{code}
This shows that the assignment failed due to Servernotrunningexcep.
Connected to master logs comes even before this
{code}
2012-09-20 13:10:31,810 INFO
[RegionServer:0;asf011.sp2.ygridcore.net,34620,1348146629931]
regionserver.HRegionServer(1943): Attempting connect to Master server at
asf011.sp2.ygridcore.net,49804,1348146629519
2012-09-20 13:10:31,841 INFO
[RegionServer:0;asf011.sp2.ygridcore.net,34620,1348146629931]
regionserver.HRegionServer(1952): Connected to master at
asf011.sp2.ygridcore.net/67.195.138.20:49804
2012-09-20 13:10:31,841 INFO
[RegionServer:0;asf011.sp2.ygridcore.net,34620,1348146629931]
regionserver.HRegionServer(1998): Telling master at
asf011.sp2.ygridcore.net,49804,1348146629519 that we are up with port=34620,
startcode=1348146629931
2012-09-20 13:10:31,882 INFO [IPC Server handler 0 on 49804]
master.ServerManager(307): Registering
server=asf011.sp2.ygridcore.net,34620,1348146629931
2012-09-20 13:10:31,893 DEBUG
[RegionServer:0;asf011.sp2.ygridcore.net,34620,1348146629931]
regionserver.HRegionServer(1171): Config from master:
hbase.rootdir=hdfs://localhost:52552/user/jenkins/hbase
2012-09-20 13:10:31,894 DEBUG
[RegionServer:0;asf011.sp2.ygridcore.net,34620,1348146629931]
regionserver.HRegionServer(1171): Config from master:
fs.default.name=hdfs://localhost:52552
2012-09-20 13:10:31,894 INFO
[RegionServer:0;asf011.sp2.ygridcore.net,34620,1348146629931]
regionserver.HRegionServer(1164): Master passed us hostname to use.
Was=asf011.sp2.ygridcore.net, Now=asf011.sp2.ygridcore.net
{code}
And the ROOT assignment also started
{code}
012-09-20 13:10:33,811 INFO
[Master:0;asf011.sp2.ygridcore.net,49804,1348146629519]
master.AssignmentManager(1579): Assigning region -ROOT-,,0.70236052 to
asf011.sp2.ygridcore.net,34620,1348146629931
2012-09-20 13:10:33,811 INFO
[Master:0;asf011.sp2.ygridcore.net,49804,1348146629519]
master.RegionStates(250): Region {NAME = '-ROOT-,,0', STARTKEY = '', ENDKEY
= '', ENCODED = 70236052,} transitioned from {-ROOT-,,0.70236052
state=OFFLINE, ts=1348146633581, server=null} to {-ROOT-,,0.70236052
state=PENDING_OPEN, ts=1348146633811,
server=asf011.sp2.ygridcore.net,34620,1348146629931}
{code}
I think this should be addressed in a seperate JIRA. Let me know if i can
commit the patch?
Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
--
Key: HBASE-6698
URL: https://issues.apache.org/jira/browse/HBASE-6698
Project: HBase
Issue Type: Improvement
Reporter: ramkrishna.s.vasudevan
Fix For: 0.96.0
Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch,
HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch,
HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch,
HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch
Currently the checkAndPut and checkAndDelete api internally calls the
internalPut and internalDelete. May be we can just call doMiniBatchMutation
only. This will help in future like if we have some hooks and the CP
handles certain cases in the doMiniBatchMutation the same can be done while
doing a put thro checkAndPut or while doing a delete thro checkAndDelete.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6071) getRegionServerWithRetires, should log unsuccessful attempts and exceptions.
[ https://issues.apache.org/jira/browse/HBASE-6071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-6071: - Fix Version/s: (was: 0.94.2) (was: 0.96.0) getRegionServerWithRetires, should log unsuccessful attempts and exceptions. Key: HBASE-6071 URL: https://issues.apache.org/jira/browse/HBASE-6071 Project: HBase Issue Type: Improvement Components: client, ipc Affects Versions: 0.92.0, 0.94.0 Reporter: Igal Shilman Priority: Minor Labels: client, ipc Attachments: HBASE-6071.patch, HBASE-6071.v2.patch, HBASE-6071.v3.patch, HConnectionManager_HBASE-6071-0.90.0.patch, lease-exception.txt HConnectionImplementation.getRegionServerWithRetries might terminate w/ an exception different then a DoNotRetryIOException, thus silently drops exceptions from previous attempts. [~ted_yu] suggested ([here|http://mail-archives.apache.org/mod_mbox/hbase-user/201205.mbox/%3CCAFebPXBq9V9BVdzRTNr-MB3a1Lz78SZj6gvP6On0b%2Bajt9StAg%40mail.gmail.com%3E]) adding a log message inside the catch block describing the exception type and details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6649) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1]
[ https://issues.apache.org/jira/browse/HBASE-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459733#comment-13459733 ] Lars Hofhansl commented on HBASE-6649: -- +1 on last patch. [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1] --- Key: HBASE-6649 URL: https://issues.apache.org/jira/browse/HBASE-6649 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Priority: Blocker Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: 6649-0.92.patch, 6649-1.patch, 6649-2.txt, 6649-fix-io-exception-handling-1.patch, 6649-fix-io-exception-handling-1-trunk.patch, 6649-fix-io-exception-handling.patch, 6649-trunk.patch, 6649-trunk.patch, 6649.txt, HBase-0.92 #495 test - queueFailover [Jenkins].html, HBase-0.92 #502 test - queueFailover [Jenkins].html Have seen it twice in the recent past: http://bit.ly/MPCykB http://bit.ly/O79Dq7 .. Looking briefly at the logs hints at a pattern - in both the failed test instances, there was an RS crash while the test was running. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6381) AssignmentManager should use the same logic for clean startup and failover
[
https://issues.apache.org/jira/browse/HBASE-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459737#comment-13459737
]
Jimmy Xiang commented on HBASE-6381:
@Rajesh, good catch! I changed the corresponding code as below:
{noformat}
if (deadServers != null) {
for (Map.EntryServerName, ListHRegionInfo server:
deadServers.entrySet()) {
ServerName serverName = server.getKey();
if (!serverManager.isServerDead(serverName)) {
serverManager.expireServer(serverName); // Let SSH do region re-assign
}
}
}
nodes = ZKUtil.listChildrenAndWatchForNewChildren(
this.watcher, this.watcher.assignmentZNode);
if (!nodes.isEmpty()) {
for (String encodedRegionName : nodes) {
processRegionInTransition(encodedRegionName, null, deadServers);
}
}
failoverCleanupDone();
{noformat}
Any other issues? Any place we can enhance more? We can use RB if you prefer.
Thanks.
AssignmentManager should use the same logic for clean startup and failover
--
Key: HBASE-6381
URL: https://issues.apache.org/jira/browse/HBASE-6381
Project: HBase
Issue Type: Bug
Components: master
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: hbase-6381-notes.pdf, hbase-6381.pdf,
trunk-6381_v5.patch, trunk-6381_v7.patch, trunk-6381_v8.patch
Currently AssignmentManager handles clean startup and failover very
differently.
Different logic is mingled together so it is hard to find out which is for
which.
We should clean it up and share the same logic so that AssignmentManager
handles
both cases the same way. This way, the code will much easier to understand
and
maintain.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6698: -- Status: Open (was: Patch Available) Retrying the patch. Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6698: -- Attachment: HBASE-6698_8.patch Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6698: -- Status: Patch Available (was: Open) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459748#comment-13459748 ] stack commented on HBASE-6698: -- Nice analysis Ram. Commit your patch and open a new one for the failed test especially since you've done the work to figure why it failed. Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6841) Meta prefetching is slower than doing multiple meta lookups
[ https://issues.apache.org/jira/browse/HBASE-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459753#comment-13459753 ] Lars Hofhansl commented on HBASE-6841: -- ensureZookeeperTrackers only does any work when ZK was not setup already. Previously this was sprinkled all over, but did the same amount of work. That part is OK, I think (in 0.94 HConnection needs a working ZK connection to function). Just checked, HConnectionManager.execute is called all over the place (12 different places in HConnectionManager and HTable). We do some weird stuff. For example setting the prefetching is also done through execute (i.e. get a - potentially - new HConntionImplementation, call setPrefetching on it, then close it). On the face of it, this does not look like it's specific to region prefetching, but the fact that repeatedly do something with a new connection. @J-D: Did you disable this path and found faster? Meta prefetching is slower than doing multiple meta lookups --- Key: HBASE-6841 URL: https://issues.apache.org/jira/browse/HBASE-6841 Project: HBase Issue Type: Improvement Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.94.2 I got myself into a situation where I needed to truncate a massive table while it was getting hits and surprisingly the clients were not recovering. What I see in the logs is that every time we prefetch .META. we setup a new HConnection because we close it on the way out. It's awfully slow. We should just turn it off or make it useful. jstacks coming up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6649) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1]
[ https://issues.apache.org/jira/browse/HBASE-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459754#comment-13459754 ] Lars Hofhansl commented on HBASE-6649: -- J-D, any objections to committing this? [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1] --- Key: HBASE-6649 URL: https://issues.apache.org/jira/browse/HBASE-6649 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Priority: Blocker Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: 6649-0.92.patch, 6649-1.patch, 6649-2.txt, 6649-fix-io-exception-handling-1.patch, 6649-fix-io-exception-handling-1-trunk.patch, 6649-fix-io-exception-handling.patch, 6649-trunk.patch, 6649-trunk.patch, 6649.txt, HBase-0.92 #495 test - queueFailover [Jenkins].html, HBase-0.92 #502 test - queueFailover [Jenkins].html Have seen it twice in the recent past: http://bit.ly/MPCykB http://bit.ly/O79Dq7 .. Looking briefly at the logs hints at a pattern - in both the failed test instances, there was an RS crash while the test was running. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5937) Refactor HLog into an interface.
[
https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459755#comment-13459755
]
Flavio Junqueira commented on HBASE-5937:
-
I've been able to fix a few bugs, and I'm getting fewer failures/errors now:
{noformat}
Failed tests:
testExceptionFromCoprocessorDuringPut(org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort):
The put should have failed, as the coprocessor is buggy
Tests in error:
testZKClosingNodeVersionMismatch(org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler):
KeeperErrorCode = NodeExists for
/hbase/unassigned/740ffc6356202cd16e98dbe3ddf302cd
testCloseRegion(org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler):
KeeperErrorCode = NodeExists for
/hbase/unassigned/98327c03377cb4f0bf4a366bff09e68c
testLocalHBaseCluster(org.apache.hadoop.hbase.TestLocalHBaseCluster): Master
not initialized after 200 seconds
{noformat}
I'll check those more closely. I was wondering of anyone has more thoughts on
the getReader/getWriter point I asked above. Thanks!
Refactor HLog into an interface.
Key: HBASE-5937
URL: https://issues.apache.org/jira/browse/HBASE-5937
Project: HBase
Issue Type: Sub-task
Reporter: Li Pi
Assignee: Flavio Junqueira
Priority: Minor
Attachments:
org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
What the summary says. Create HLog interface. Make current implementation use
it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6841) Meta prefetching is slower than doing multiple meta lookups
[ https://issues.apache.org/jira/browse/HBASE-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459757#comment-13459757 ] Jean-Daniel Cryans commented on HBASE-6841: --- bq. ensureZookeeperTrackers only does any work when ZK was not setup already. Right, but since we always close the connection at the end of execute() we always end up re-creating it. bq. Did you disable this path and found faster? No, currently it's just an educated observation that's things are broken. Meta prefetching is slower than doing multiple meta lookups --- Key: HBASE-6841 URL: https://issues.apache.org/jira/browse/HBASE-6841 Project: HBase Issue Type: Improvement Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.94.2 I got myself into a situation where I needed to truncate a massive table while it was getting hits and surprisingly the clients were not recovering. What I see in the logs is that every time we prefetch .META. we setup a new HConnection because we close it on the way out. It's awfully slow. We should just turn it off or make it useful. jstacks coming up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6649) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1]
[ https://issues.apache.org/jira/browse/HBASE-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459758#comment-13459758 ] Jean-Daniel Cryans commented on HBASE-6649: --- I'm going to create a new jira first (should have done that when I found that problem) and post the patches there with a small nit fixed. [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1] --- Key: HBASE-6649 URL: https://issues.apache.org/jira/browse/HBASE-6649 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Priority: Blocker Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: 6649-0.92.patch, 6649-1.patch, 6649-2.txt, 6649-fix-io-exception-handling-1.patch, 6649-fix-io-exception-handling-1-trunk.patch, 6649-fix-io-exception-handling.patch, 6649-trunk.patch, 6649-trunk.patch, 6649.txt, HBase-0.92 #495 test - queueFailover [Jenkins].html, HBase-0.92 #502 test - queueFailover [Jenkins].html Have seen it twice in the recent past: http://bit.ly/MPCykB http://bit.ly/O79Dq7 .. Looking briefly at the logs hints at a pattern - in both the failed test instances, there was an RS crash while the test was running. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6788) Convert AuthenticationProtocol to protocol buffer service
[ https://issues.apache.org/jira/browse/HBASE-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459762#comment-13459762 ] Gary Helmling commented on HBASE-6788: -- Looking at this, we also need to convert the serialization of AuthenticationKey to be PB-based. AuthenticationKeys are serialized into ZK znodes to coordinate secret key rolling for authentication token signing and validation. Convert AuthenticationProtocol to protocol buffer service - Key: HBASE-6788 URL: https://issues.apache.org/jira/browse/HBASE-6788 Project: HBase Issue Type: Sub-task Components: coprocessors Reporter: Gary Helmling Priority: Blocker Fix For: 0.96.0 With coprocessor endpoints now exposed as protobuf defined services, we should convert over all of our built-in endpoints to PB services. AccessControllerProtocol was converted as part of HBASE-5448, but the authentication token provider still needs to be changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6847) HBASE-6649 broke replication
Jean-Daniel Cryans created HBASE-6847:
-
Summary: HBASE-6649 broke replication
Key: HBASE-6847
URL: https://issues.apache.org/jira/browse/HBASE-6847
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Priority: Blocker
Fix For: 0.96.0, 0.92.3, 0.94.2
After running with HBASE-6646 and replication enabled I encountered this:
{noformat}
2012-09-17 20:04:08,111 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening log
for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78617132
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Break on
IOE:
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318,
entryStart=78641557, pos=78771200, end=78771200, edit=84
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
currentNbOperations:164529 and seenEntries:84 and size: 154068
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Replicating
84
2012-09-17 20:04:08,146 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Going to report log #va1r3s24%2C10304%2C1347911704238.1347911706318 for
position 78771200 in
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
2012-09-17 20:04:08,158 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Removing 0 logs in the list: []
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Replicated
in total: 93234
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening log
for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78771200
2012-09-17 20:04:08,163 ERROR
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Unexpected
exception in ReplicationSource,
currentPath=hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
java.lang.IndexOutOfBoundsException
at java.io.DataInputStream.readFully(DataInputStream.java:175)
at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2001)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1901)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1947)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:235)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:394)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:307)
{noformat}
There's something weird at the end of the file and it's killing replication. We
used to just retry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6649) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1]
[ https://issues.apache.org/jira/browse/HBASE-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans resolved HBASE-6649. --- Resolution: Fixed Re-closing, I opened HBASE-6847. [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-1] --- Key: HBASE-6649 URL: https://issues.apache.org/jira/browse/HBASE-6649 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Priority: Blocker Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: 6649-0.92.patch, 6649-1.patch, 6649-2.txt, 6649-fix-io-exception-handling-1.patch, 6649-fix-io-exception-handling-1-trunk.patch, 6649-fix-io-exception-handling.patch, 6649-trunk.patch, 6649-trunk.patch, 6649.txt, HBase-0.92 #495 test - queueFailover [Jenkins].html, HBase-0.92 #502 test - queueFailover [Jenkins].html Have seen it twice in the recent past: http://bit.ly/MPCykB http://bit.ly/O79Dq7 .. Looking briefly at the logs hints at a pattern - in both the failed test instances, there was an RS crash while the test was running. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6846) BitComparator bug - ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459767#comment-13459767
]
stack commented on HBASE-6846:
--
bq. Should I provide a patch for correcting the problem?
That'd be excellent.
BitComparator bug - ArrayIndexOutOfBoundsException
--
Key: HBASE-6846
URL: https://issues.apache.org/jira/browse/HBASE-6846
Project: HBase
Issue Type: Bug
Components: filters
Affects Versions: 0.94.1
Environment: HBase 0.94.1 + Hadoop 2.0.0-cdh4.0.1
Reporter: Lucian George Iordache
The HBase 0.94.1 BitComparator introduced a bug in the method compareTo:
@Override
public int compareTo(byte[] value, int offset, int length) {
if (length != this.value.length) {
return 1;
}
int b = 0;
//Iterating backwards is faster because we can quit after one non-zero
byte.
for (int i = value.length - 1; i = 0 b == 0; i--) {
switch (bitOperator) {
case AND:
b = (this.value[i] value[i+offset]) 0xff;
break;
case OR:
b = (this.value[i] | value[i+offset]) 0xff;
break;
case XOR:
b = (this.value[i] ^ value[i+offset]) 0xff;
break;
}
}
return b == 0 ? 1 : 0;
}
I've encountered this problem when using a BitComparator with a configured
this.value.length=8, and in the HBase table there were KeyValues with
keyValue.getBuffer().length=207911 bytes. In this case:
for (int i = 207910; i = 0 b == 0; i--) {
switch (bitOperator) {
case AND:
b = (this.value[207910] ... == ArrayIndexOutOfBoundsException
break;
That loop should use:
for (int i = length - 1; i = 0 b == 0; i--) { (or this.value.length.)
Should I provide a patch for correcting the problem?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-3546) XSS in the WebUI
[
https://issues.apache.org/jira/browse/HBASE-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-3546.
--
Resolution: Won't Fix
Assignee: liang xie
Closing since a 0.90 minor bug (0.90 is old stuff). Assigning Liang since he
did the investigation.
XSS in the WebUI
Key: HBASE-3546
URL: https://issues.apache.org/jira/browse/HBASE-3546
Project: HBase
Issue Type: Bug
Reporter: Takuya Ueshin
Assignee: liang xie
Priority: Minor
There are possibilities of XSS in the WebUI.
If ColumnFamily or Region splitting keys are like
Bytes.toBytes(scriptalert('js')/script)
then browsers run the JavaScript code.
I tested on HBase-0.90.0 .
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6847) HBASE-6649 broke replication
[
https://issues.apache.org/jira/browse/HBASE-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Daniel Cryans updated HBASE-6847:
--
Attachment: HBASE-6847-0.94.patch
HBASE-6847.patch
Attaching the patches that Devaraj posted in HBASE-6649 except that I changed
the position into startPosition since it's more relevant and it's not the
same name as a class member.
[~devaraj] I saw you also fixed an issue with the position giving the wrong
size for the batch when trying to decide when to stop. Very good!
HBASE-6649 broke replication
Key: HBASE-6847
URL: https://issues.apache.org/jira/browse/HBASE-6847
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Priority: Blocker
Fix For: 0.96.0, 0.92.3, 0.94.2
Attachments: HBASE-6847-0.94.patch, HBASE-6847.patch
After running with HBASE-6646 and replication enabled I encountered this:
{noformat}
2012-09-17 20:04:08,111 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78617132
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Break on
IOE:
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318,
entryStart=78641557, pos=78771200, end=78771200, edit=84
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
currentNbOperations:164529 and seenEntries:84 and size: 154068
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Replicating 84
2012-09-17 20:04:08,146 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Going to report log #va1r3s24%2C10304%2C1347911704238.1347911706318 for
position 78771200 in
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
2012-09-17 20:04:08,158 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Removing 0 logs in the list: []
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Replicated in total: 93234
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78771200
2012-09-17 20:04:08,163 ERROR
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Unexpected exception in ReplicationSource,
currentPath=hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
java.lang.IndexOutOfBoundsException
at java.io.DataInputStream.readFully(DataInputStream.java:175)
at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2001)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1901)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1947)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:235)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:394)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:307)
{noformat}
There's something weird at the end of the file and it's killing replication.
We used to just retry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6847) HBASE-6649 broke replication
[
https://issues.apache.org/jira/browse/HBASE-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Daniel Cryans reassigned HBASE-6847:
-
Assignee: Devaraj Das
HBASE-6649 broke replication
Key: HBASE-6847
URL: https://issues.apache.org/jira/browse/HBASE-6847
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Assignee: Devaraj Das
Priority: Blocker
Fix For: 0.96.0, 0.92.3, 0.94.2
Attachments: HBASE-6847-0.94.patch, HBASE-6847.patch
After running with HBASE-6646 and replication enabled I encountered this:
{noformat}
2012-09-17 20:04:08,111 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78617132
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Break on
IOE:
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318,
entryStart=78641557, pos=78771200, end=78771200, edit=84
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
currentNbOperations:164529 and seenEntries:84 and size: 154068
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Replicating 84
2012-09-17 20:04:08,146 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Going to report log #va1r3s24%2C10304%2C1347911704238.1347911706318 for
position 78771200 in
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
2012-09-17 20:04:08,158 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Removing 0 logs in the list: []
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Replicated in total: 93234
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78771200
2012-09-17 20:04:08,163 ERROR
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Unexpected exception in ReplicationSource,
currentPath=hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
java.lang.IndexOutOfBoundsException
at java.io.DataInputStream.readFully(DataInputStream.java:175)
at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2001)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1901)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1947)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:235)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:394)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:307)
{noformat}
There's something weird at the end of the file and it's killing replication.
We used to just retry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6677) Random ZooKeeper port in test can overrun max port
[
https://issues.apache.org/jira/browse/HBASE-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-6677:
-
Resolution: Fixed
Fix Version/s: 0.96.0
Assignee: liang xie
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk. Looks good. Keeps us in the dynamic port range. Thanks
for the patch Liang.
Random ZooKeeper port in test can overrun max port
--
Key: HBASE-6677
URL: https://issues.apache.org/jira/browse/HBASE-6677
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.96.0
Reporter: Gregory Chanan
Assignee: liang xie
Priority: Trivial
Labels: noob
Fix For: 0.96.0
Attachments: HBASE-6677.patch
{code}
while (true) {
try {
standaloneServerFactory = new NIOServerCnxnFactory();
standaloneServerFactory.configure(
new InetSocketAddress(tentativePort),
configuration.getInt(HConstants.ZOOKEEPER_MAX_CLIENT_CNXNS,
1000));
} catch (BindException e) {
LOG.debug(Failed binding ZK Server to client port: +
tentativePort);
// This port is already in use, try to use another.
tentativePort++;
continue;
}
break;
}
{code}
In the case of failure and all the above ports have already been binded, you
can extend past the max port. Need to check against a max value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459785#comment-13459785 ] Jonathan Leavitt commented on HBASE-6524: - bq. Doc is great. (it would not be very difficult to remove this requirement). What would this take? Well to remove the client addendum we would have to instrument the entry points into hbase. I will try to find one good class to instrument as the entry point in to the system. HTable.java might work, but it doesn't include all operations, and importantly does not include all of the steps of a 'get', bx getting the HTable is a big part, right? Maybe HTablePool.java would work. bq. Can we do this for hdfs too yet? Not yet. I think Matt or Todd might be working on it. I have been looking into, but it does require all of the additional instrumentation to HDFS. bq. What about an example that enables trace over hdfs too while the Get is going on? I'm not sure what you mean by this. bq. On doc in general, I came across this recent quote Documentation is like sex: when it is good, it is very very good; and when it is bad, it is better than nothing. Great quote. Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Assignee: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, 6524-v2.txt, 6524v3.txt, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6669) Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459787#comment-13459787 ] Anil Gupta commented on HBASE-6669: --- Hi Julian, I am curious to know whether you got the opportunity to test BDCI utility i sent(on hbase user list) last week along with some suggestions on using it? Did it run successfully? I will try to have a look at your unit test over weekend. Thanks, Anil Gupta Software Engineer II, Intuit, Inc Add BigDecimalColumnInterpreter for doing aggregations using AggregationClient -- Key: HBASE-6669 URL: https://issues.apache.org/jira/browse/HBASE-6669 Project: HBase Issue Type: New Feature Components: client, coprocessors Reporter: Anil Gupta Priority: Minor Labels: client, coprocessors Attachments: BigDecimalColumnInterpreter.java, BigDecimalColumnInterpreter.patch, BigDecimalColumnInterpreter.patch I recently created a Class for doing aggregations(sum,min,max,std) on values stored as BigDecimal in HBase. I would like to commit the BigDecimalColumnInterpreter into HBase. In my opinion this class can be used by a wide variety of users. Please let me know if its not appropriate to add this class in HBase. Thanks, Anil Gupta Software Engineer II, Intuit, Inc -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6847) HBASE-6649 broke replication
[
https://issues.apache.org/jira/browse/HBASE-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459791#comment-13459791
]
Devaraj Das commented on HBASE-6847:
Thanks, JD for posting the patches (and yes, noticed that other place where
position was used and fixed that as well). The variable name change looks good.
+1.
HBASE-6649 broke replication
Key: HBASE-6847
URL: https://issues.apache.org/jira/browse/HBASE-6847
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Assignee: Devaraj Das
Priority: Blocker
Fix For: 0.96.0, 0.92.3, 0.94.2
Attachments: HBASE-6847-0.94.patch, HBASE-6847.patch
After running with HBASE-6646 and replication enabled I encountered this:
{noformat}
2012-09-17 20:04:08,111 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78617132
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Break on
IOE:
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318,
entryStart=78641557, pos=78771200, end=78771200, edit=84
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
currentNbOperations:164529 and seenEntries:84 and size: 154068
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Replicating 84
2012-09-17 20:04:08,146 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Going to report log #va1r3s24%2C10304%2C1347911704238.1347911706318 for
position 78771200 in
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
2012-09-17 20:04:08,158 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Removing 0 logs in the list: []
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Replicated in total: 93234
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78771200
2012-09-17 20:04:08,163 ERROR
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Unexpected exception in ReplicationSource,
currentPath=hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
java.lang.IndexOutOfBoundsException
at java.io.DataInputStream.readFully(DataInputStream.java:175)
at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2001)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1901)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1947)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:235)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:394)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:307)
{noformat}
There's something weird at the end of the file and it's killing replication.
We used to just retry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6788) Convert AuthenticationProtocol to protocol buffer service
[ https://issues.apache.org/jira/browse/HBASE-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459795#comment-13459795 ] Andrew Purtell commented on HBASE-6788: --- I'd be +1 with dropping CoprocessorProtocol from 0.96+, given all of the other (deliberate) incompatibilities posed with RPC going from 0.94 to 0.96+. Convert AuthenticationProtocol to protocol buffer service - Key: HBASE-6788 URL: https://issues.apache.org/jira/browse/HBASE-6788 Project: HBase Issue Type: Sub-task Components: coprocessors Reporter: Gary Helmling Priority: Blocker Fix For: 0.96.0 With coprocessor endpoints now exposed as protobuf defined services, we should convert over all of our built-in endpoints to PB services. AccessControllerProtocol was converted as part of HBASE-5448, but the authentication token provider still needs to be changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6788) Convert AuthenticationProtocol to protocol buffer service
[ https://issues.apache.org/jira/browse/HBASE-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459795#comment-13459795 ] Andrew Purtell edited comment on HBASE-6788 at 9/21/12 4:55 AM: I'd be +1 with dropping CoprocessorProtocol from 0.96 and up, given all of the other (deliberate) incompatibilities posed with RPC going from 0.94 to 0.96 and up. Edit: Removed accidental markup. was (Author: apurtell): I'd be +1 with dropping CoprocessorProtocol from 0.96+, given all of the other (deliberate) incompatibilities posed with RPC going from 0.94 to 0.96+. Convert AuthenticationProtocol to protocol buffer service - Key: HBASE-6788 URL: https://issues.apache.org/jira/browse/HBASE-6788 Project: HBase Issue Type: Sub-task Components: coprocessors Reporter: Gary Helmling Priority: Blocker Fix For: 0.96.0 With coprocessor endpoints now exposed as protobuf defined services, we should convert over all of our built-in endpoints to PB services. AccessControllerProtocol was converted as part of HBASE-5448, but the authentication token provider still needs to be changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459798#comment-13459798 ] stack commented on HBASE-6524: -- bq. What about an example that enables trace over hdfs too while the Get is going on? Prerequisite is tracing in hdfs which is not done yet so ignore the above. Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Assignee: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, 6524-v2.txt, 6524v3.txt, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6696) Add CP hooks pre and post split transaction point-of-no-return
[
https://issues.apache.org/jira/browse/HBASE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459801#comment-13459801
]
Andrew Purtell commented on HBASE-6696:
---
[~stack] Thoughts on when the cut point for 0.96 might be?
Add CP hooks pre and post split transaction point-of-no-return
--
Key: HBASE-6696
URL: https://issues.apache.org/jira/browse/HBASE-6696
Project: HBase
Issue Type: Improvement
Components: coprocessors, regionserver
Affects Versions: 0.96.0, 0.94.2
Reporter: Andrew Purtell
Assignee: Andrew Purtell
In the discussion Improving Coprocessor postSplit/postOpen synchronization
on user@hbase, a user implementing a secondary indexing scheme writes in:
{quote}
The goal with splits is to create two new daughter regions with the
corresponding splits of the secondary indexes and lock these regions such
that Puts/Deletes that occur while postSplit is in progress will be queued
up so we don't run into consistency issues. [...] As of right now, the HBase
coprocessors do not easily support a way to achieve this level of consistency
in that there is no way to distinguish a region being opened from a split or
a regular open.
{quote}
Setting aside the particulars of the use case, the issue is the {{preSplit}}
hook does not provide the identities of the daughter regions (they don't
exist yet) and the {{postSplit}} hook, while providing the identities of the
daughter regions, runs after the master has processed the split and the
daughters are already opened or opening. A CP implementer has no interception
point available where the daughters exist but are not yet open.
This JIRA proposes to add two new CP hooks to just before and after the
point-of-no-return (PONR) in the split transaction: {{preSplitPONR}} and
{{postSplitPONR}}. Perhaps the naming can be improved. This will address the
above use case but additionally support overloading of the split transaction
itself. We also need to address observer notification if the split
transaction fails. This is like HBASE-5827 but scoped to this specific
consideration only.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6698: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) thanks for the patch Priya. Thanks for the review Ted, Lars, Stack Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6698) Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation
[ https://issues.apache.org/jira/browse/HBASE-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459808#comment-13459808 ] Hadoop QA commented on HBASE-6698: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12545931/HBASE-6698_8.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. -1 javadoc. The javadoc tool appears to have generated 139 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2907//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2907//console This message is automatically generated. Refactor checkAndPut and checkAndDelete to use doMiniBatchMutation -- Key: HBASE-6698 URL: https://issues.apache.org/jira/browse/HBASE-6698 Project: HBase Issue Type: Improvement Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0 Attachments: HBASE-6698_1.patch, HBASE-6698_2.patch, HBASE-6698_3.patch, HBASE-6698_5.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_6.patch, HBASE-6698_7.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698_8.patch, HBASE-6698.patch Currently the checkAndPut and checkAndDelete api internally calls the internalPut and internalDelete. May be we can just call doMiniBatchMutation only. This will help in future like if we have some hooks and the CP handles certain cases in the doMiniBatchMutation the same can be done while doing a put thro checkAndPut or while doing a delete thro checkAndDelete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6805) Extend co-processor framework to provide observers for filter operations
[ https://issues.apache.org/jira/browse/HBASE-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459810#comment-13459810 ] Andrew Purtell commented on HBASE-6805: --- [~jason.dai] Regarding your point 1, that sounds reasonable. For point 2, filter wrapping from scanner creation is something the current API supports, so you should be good there. For point 3, I'm not sure I understand, if you have wrapped a scanner, why you then need to call out to (or receive upcalls on) filter observer hooks, but perhaps your patch will make it clear. To assist with this understanding, please consider providing a small example of filter+scan wrapping? See (on trunk) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/example/ for other such examples. Extend co-processor framework to provide observers for filter operations Key: HBASE-6805 URL: https://issues.apache.org/jira/browse/HBASE-6805 Project: HBase Issue Type: Sub-task Components: coprocessors Affects Versions: 0.96.0 Reporter: Jason Dai There are several filter operations (e.g., filterKeyValue, filterRow, transform, etc.) at the region server side that either exclude KVs from the returned results, or transform the returned KV. We need to provide observers (e.g., preFilterKeyValue and postFilterKeyValue) for these operations in the same way as the observers for other data access operations (e.g., preGet and postGet). This extension is needed to support DOT (e.g., extracting individual fields from the document in the observers before passing them to the related filter operations) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6799) Store more metadata in HFiles
[ https://issues.apache.org/jira/browse/HBASE-6799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459821#comment-13459821 ] Andrew Purtell commented on HBASE-6799: --- A generic/custom tags facility would be great, then we can try out a number of things without requiring core patching. I would like to see CF access statistics. Could do a snapshot of current CF metrics when the HFile is written, as a first cut. Then dynamic per-CF metrics could be reinitialized after region migration or cold boot from the most recent HFile - a recent flush, presumably. Perhaps we might want to differentiate between online measurements (= 15 minutes) and a longer historical view, and initialize only the latter. Anyway, then we have a local memory of the per-CF metrics, for such things as HBASE-6572. Store more metadata in HFiles - Key: HBASE-6799 URL: https://issues.apache.org/jira/browse/HBASE-6799 Project: HBase Issue Type: Brainstorming Reporter: Lars Hofhansl Current we store metadata in HFile: * the timerange of KVs * the earliest PUT ts * max sequence id * whether or not this file was created from a major compaction. I would like to brainstorm what extra data we need to store to make an HFile self describing. I.e. it could be backed up to somewhere with external tools (without invoking an HBase server) can gleam enough information from it to make use of the data inside. Ideally it would also be nice to be able to recreate .META. from a bunch of HFiles to standup a temporary HBase instance to process a bunch of HFiles. What I can think of: * min/max key * table * column family (or families to be future proof) * custom tags (set by a backup tools for example) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6572) Tiered HFile storage
[ https://issues.apache.org/jira/browse/HBASE-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459824#comment-13459824 ] Andrew Purtell commented on HBASE-6572: --- [~sureshms] Agreed, HDFS-2832 is it. Tiered HFile storage Key: HBASE-6572 URL: https://issues.apache.org/jira/browse/HBASE-6572 Project: HBase Issue Type: Brainstorming Reporter: Andrew Purtell Assignee: Andrew Purtell Consider how we might enable tiered HFile storage. If HDFS has the capability, we could create certain files on solid state devices where they might be frequently accessed, especially for random reads; and others (and by default) on spinning media as before. We could support the move of frequently read HFiles from spinning media to solid state. We already have CF statistics for this, would only need to add requisite admin interface; could even consider an autotiering option. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6799) Store more metadata in HFiles
[ https://issues.apache.org/jira/browse/HBASE-6799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459821#comment-13459821 ] Andrew Purtell edited comment on HBASE-6799 at 9/21/12 5:21 AM: A generic/custom tags facility would be great, then we can try out a number of things without requiring core patching. I would like to see CF access statistics. Could do a snapshot of current CF metrics when the HFile is written. Then we would have a local memory of dynamic per-CF metrics, for such things as HBASE-6572. And compaction could perhaps merge such CF statistics snapshots in HFiles with time based exponential weighting. Further, we might differentiate between online measurements (= 15 minutes) and a longer historical view of per-CF metrics, and initialize the latter after region migration or cold boot from the most recent HFile. was (Author: apurtell): A generic/custom tags facility would be great, then we can try out a number of things without requiring core patching. I would like to see CF access statistics. Could do a snapshot of current CF metrics when the HFile is written, as a first cut. Then dynamic per-CF metrics could be reinitialized after region migration or cold boot from the most recent HFile - a recent flush, presumably. Perhaps we might want to differentiate between online measurements (= 15 minutes) and a longer historical view, and initialize only the latter. Anyway, then we have a local memory of the per-CF metrics, for such things as HBASE-6572. Store more metadata in HFiles - Key: HBASE-6799 URL: https://issues.apache.org/jira/browse/HBASE-6799 Project: HBase Issue Type: Brainstorming Reporter: Lars Hofhansl Current we store metadata in HFile: * the timerange of KVs * the earliest PUT ts * max sequence id * whether or not this file was created from a major compaction. I would like to brainstorm what extra data we need to store to make an HFile self describing. I.e. it could be backed up to somewhere with external tools (without invoking an HBase server) can gleam enough information from it to make use of the data inside. Ideally it would also be nice to be able to recreate .META. from a bunch of HFiles to standup a temporary HBase instance to process a bunch of HFiles. What I can think of: * min/max key * table * column family (or families to be future proof) * custom tags (set by a backup tools for example) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6847) HBASE-6649 broke replication
[
https://issues.apache.org/jira/browse/HBASE-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459831#comment-13459831
]
Himanshu Vashishtha commented on HBASE-6847:
I wonder about the IndexOutOfBound exception. Is this hlog part of a failover
regionserver?
HBASE-6649 broke replication
Key: HBASE-6847
URL: https://issues.apache.org/jira/browse/HBASE-6847
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Assignee: Devaraj Das
Priority: Blocker
Fix For: 0.96.0, 0.92.3, 0.94.2
Attachments: HBASE-6847-0.94.patch, HBASE-6847.patch
After running with HBASE-6646 and replication enabled I encountered this:
{noformat}
2012-09-17 20:04:08,111 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78617132
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Break on
IOE:
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318,
entryStart=78641557, pos=78771200, end=78771200, edit=84
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
currentNbOperations:164529 and seenEntries:84 and size: 154068
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Replicating 84
2012-09-17 20:04:08,146 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Going to report log #va1r3s24%2C10304%2C1347911704238.1347911706318 for
position 78771200 in
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
2012-09-17 20:04:08,158 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Removing 0 logs in the list: []
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Replicated in total: 93234
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78771200
2012-09-17 20:04:08,163 ERROR
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Unexpected exception in ReplicationSource,
currentPath=hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
java.lang.IndexOutOfBoundsException
at java.io.DataInputStream.readFully(DataInputStream.java:175)
at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2001)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1901)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1947)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:235)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:394)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:307)
{noformat}
There's something weird at the end of the file and it's killing replication.
We used to just retry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6848) Make hbase-hadoop-compat findbugs clean
Elliott Clark created HBASE-6848: Summary: Make hbase-hadoop-compat findbugs clean Key: HBASE-6848 URL: https://issues.apache.org/jira/browse/HBASE-6848 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark There are a few findbugs errors in hbase-hadoop-compat, hbase-hadoop1-compat, and hbase-hadoop2-compat. Lets fix these up; since these are new modules it would be nice to keep them with 0 findbugs errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6848) Make hbase-hadoop-compat findbugs clean
[ https://issues.apache.org/jira/browse/HBASE-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6848: - Priority: Minor (was: Major) Make hbase-hadoop-compat findbugs clean --- Key: HBASE-6848 URL: https://issues.apache.org/jira/browse/HBASE-6848 Project: HBase Issue Type: Improvement Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 There are a few findbugs errors in hbase-hadoop-compat, hbase-hadoop1-compat, and hbase-hadoop2-compat. Lets fix these up; since these are new modules it would be nice to keep them with 0 findbugs errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6848) Make hbase-hadoop-compat findbugs clean
[ https://issues.apache.org/jira/browse/HBASE-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6848: - Affects Version/s: 0.96.0 Make hbase-hadoop-compat findbugs clean --- Key: HBASE-6848 URL: https://issues.apache.org/jira/browse/HBASE-6848 Project: HBase Issue Type: Improvement Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 There are a few findbugs errors in hbase-hadoop-compat, hbase-hadoop1-compat, and hbase-hadoop2-compat. Lets fix these up; since these are new modules it would be nice to keep them with 0 findbugs errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6848) Make hbase-hadoop-compat findbugs clean
[ https://issues.apache.org/jira/browse/HBASE-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6848: - Fix Version/s: 0.96.0 Make hbase-hadoop-compat findbugs clean --- Key: HBASE-6848 URL: https://issues.apache.org/jira/browse/HBASE-6848 Project: HBase Issue Type: Improvement Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 There are a few findbugs errors in hbase-hadoop-compat, hbase-hadoop1-compat, and hbase-hadoop2-compat. Lets fix these up; since these are new modules it would be nice to keep them with 0 findbugs errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6847) HBASE-6649 broke replication
[
https://issues.apache.org/jira/browse/HBASE-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459836#comment-13459836
]
Jean-Daniel Cryans commented on HBASE-6847:
---
bq. I wonder about the IndexOutOfBound exception. Is this hlog part of a
failover regionserver?
No.
HBASE-6649 broke replication
Key: HBASE-6847
URL: https://issues.apache.org/jira/browse/HBASE-6847
Project: HBase
Issue Type: Bug
Reporter: Jean-Daniel Cryans
Assignee: Devaraj Das
Priority: Blocker
Fix For: 0.96.0, 0.92.3, 0.94.2
Attachments: HBASE-6847-0.94.patch, HBASE-6847.patch
After running with HBASE-6646 and replication enabled I encountered this:
{noformat}
2012-09-17 20:04:08,111 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78617132
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Break on
IOE:
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318,
entryStart=78641557, pos=78771200, end=78771200, edit=84
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
currentNbOperations:164529 and seenEntries:84 and size: 154068
2012-09-17 20:04:08,120 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Replicating 84
2012-09-17 20:04:08,146 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Going to report log #va1r3s24%2C10304%2C1347911704238.1347911706318 for
position 78771200 in
hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
2012-09-17 20:04:08,158 INFO
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
Removing 0 logs in the list: []
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Replicated in total: 93234
2012-09-17 20:04:08,158 DEBUG
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening
log for replication va1r3s24%2C10304%2C1347911704238.1347911706318 at 78771200
2012-09-17 20:04:08,163 ERROR
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Unexpected exception in ReplicationSource,
currentPath=hdfs://va1r5s41:10101/va1-backup/.logs/va1r3s24,10304,1347911704238/va1r3s24%2C10304%2C1347911704238.1347911706318
java.lang.IndexOutOfBoundsException
at java.io.DataInputStream.readFully(DataInputStream.java:175)
at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2001)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1901)
at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1947)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:235)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:394)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:307)
{noformat}
There's something weird at the end of the file and it's killing replication.
We used to just retry.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6696) Add CP hooks pre and post split transaction point-of-no-return
[
https://issues.apache.org/jira/browse/HBASE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459837#comment-13459837
]
stack commented on HBASE-6696:
--
[~andrew.purt...@gmail.com] I did a review of issues last night. There are
like 10 blockers and 20 criticals that need fixing. We could pick up some
majors. I'll do a more thorough pass today. Its looking like its going to be
a month at least to branch (thinking mostly passing jenkins and most of the
blockers and criticals done). Going to focus on it from here on.
Add CP hooks pre and post split transaction point-of-no-return
--
Key: HBASE-6696
URL: https://issues.apache.org/jira/browse/HBASE-6696
Project: HBase
Issue Type: Improvement
Components: coprocessors, regionserver
Affects Versions: 0.96.0, 0.94.2
Reporter: Andrew Purtell
Assignee: Andrew Purtell
In the discussion Improving Coprocessor postSplit/postOpen synchronization
on user@hbase, a user implementing a secondary indexing scheme writes in:
{quote}
The goal with splits is to create two new daughter regions with the
corresponding splits of the secondary indexes and lock these regions such
that Puts/Deletes that occur while postSplit is in progress will be queued
up so we don't run into consistency issues. [...] As of right now, the HBase
coprocessors do not easily support a way to achieve this level of consistency
in that there is no way to distinguish a region being opened from a split or
a regular open.
{quote}
Setting aside the particulars of the use case, the issue is the {{preSplit}}
hook does not provide the identities of the daughter regions (they don't
exist yet) and the {{postSplit}} hook, while providing the identities of the
daughter regions, runs after the master has processed the split and the
daughters are already opened or opening. A CP implementer has no interception
point available where the daughters exist but are not yet open.
This JIRA proposes to add two new CP hooks to just before and after the
point-of-no-return (PONR) in the split transaction: {{preSplitPONR}} and
{{postSplitPONR}}. Perhaps the naming can be improved. This will address the
above use case but additionally support overloading of the split transaction
itself. We also need to address observer notification if the split
transaction fails. This is like HBASE-5827 but scoped to this specific
consideration only.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6848) Make hbase-hadoop-compat findbugs clean
[ https://issues.apache.org/jira/browse/HBASE-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6848: - Attachment: HBASE-6848-0.patch I don't think that any of the issues were very big, but it's always nice to keep things clean. Make hbase-hadoop-compat findbugs clean --- Key: HBASE-6848 URL: https://issues.apache.org/jira/browse/HBASE-6848 Project: HBase Issue Type: Improvement Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-6848-0.patch There are a few findbugs errors in hbase-hadoop-compat, hbase-hadoop1-compat, and hbase-hadoop2-compat. Lets fix these up; since these are new modules it would be nice to keep them with 0 findbugs errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6848) Make hbase-hadoop-compat findbugs clean
[ https://issues.apache.org/jira/browse/HBASE-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6848: - Status: Patch Available (was: Open) Make hbase-hadoop-compat findbugs clean --- Key: HBASE-6848 URL: https://issues.apache.org/jira/browse/HBASE-6848 Project: HBase Issue Type: Improvement Affects Versions: 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-6848-0.patch There are a few findbugs errors in hbase-hadoop-compat, hbase-hadoop1-compat, and hbase-hadoop2-compat. Lets fix these up; since these are new modules it would be nice to keep them with 0 findbugs errors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6849) Make StochasticLoadBalancer the default
Elliott Clark created HBASE-6849: Summary: Make StochasticLoadBalancer the default Key: HBASE-6849 URL: https://issues.apache.org/jira/browse/HBASE-6849 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459841#comment-13459841 ] stack commented on HBASE-6702: -- Thanks [~nkeywal] ResourceChecker refinement -- Key: HBASE-6702 URL: https://issues.apache.org/jira/browse/HBASE-6702 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.96.0 Reporter: Jesse Yates Priority: Critical Fix For: 0.96.0 This was based on some discussion from HBASE-6234. The ResourceChecker was added by N. Keywal to help resolve some hadoop qa issues, but has since not be widely utilized. Further, with modularization we have had to drop the ResourceChecker from the tests that are moved into the hbase-common module because bringing the ResourceChecker up to hbase-common would involved bringing all its dependencies (which are quite far reaching). The question then is, what should we do with it? Get rid of it? Refactor and resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459838#comment-13459838 ] stack commented on HBASE-6524: -- Ok I integrate your doc into the book? Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Assignee: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, 6524-v2.txt, 6524v3.txt, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6849) Make StochasticLoadBalancer the default
[ https://issues.apache.org/jira/browse/HBASE-6849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6849: - Fix Version/s: 0.96.0 Make StochasticLoadBalancer the default --- Key: HBASE-6849 URL: https://issues.apache.org/jira/browse/HBASE-6849 Project: HBase Issue Type: Improvement Components: master Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.96.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
