[jira] [Updated] (HBASE-10251) Restore API Compat for PerformanceEvaluation.generateValue()
[ https://issues.apache.org/jira/browse/HBASE-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-10251: Attachment: HBASE_10251_v3.patch Sorry, JD; let's try again with a file that's not malformed... :-) Restore API Compat for PerformanceEvaluation.generateValue() Key: HBASE-10251 URL: https://issues.apache.org/jira/browse/HBASE-10251 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.98.1, 0.99.0, 0.98.2 Reporter: Aleksandr Shulman Assignee: Dima Spivak Labels: api_compatibility Fix For: 0.98.0, 0.98.1, 0.99.0, 0.98.2 Attachments: HBASE-10251-v2.patch, HBASE_10251.patch, HBASE_10251_v3.patch Observed: A couple of my client tests fail to compile against trunk because the method PerformanceEvaluation.generateValue was removed as part of HBASE-8496. This is an issue because it was used in a number of places, including unit tests. Since we did not explicitly label this API as private, it's ambiguous as to whether this could/should have been used by people writing apps against 0.96. If they used it, then they would be broken upon upgrade to 0.98 and trunk. Potential Solution: The method was renamed to generateData, but the logic is still the same. We can reintroduce it as deprecated in 0.98, as compat shim over generateData. The patch should be a few lines. We may also consider doing so in trunk, but I'd be just as fine with leaving it out. More generally, this raises the question about what other code is in this grey-area, where it is public, is used outside of the package, but is not explicitly labeled with an AudienceInterface. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10251) Restore API Compat for PerformanceEvaluation.generateValue()
[ https://issues.apache.org/jira/browse/HBASE-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-10251: Attachment: HBASE-10251-v2.patch Passes VALUE_LENGTH into #generateData(Random, int). Restore API Compat for PerformanceEvaluation.generateValue() Key: HBASE-10251 URL: https://issues.apache.org/jira/browse/HBASE-10251 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.98.0, 0.98.1, 0.99.0, 0.98.2 Reporter: Aleksandr Shulman Assignee: Dima Spivak Labels: api_compatibility Fix For: 0.98.0, 0.98.1, 0.99.0, 0.98.2 Attachments: HBASE-10251-v2.patch, HBASE_10251.patch Observed: A couple of my client tests fail to compile against trunk because the method PerformanceEvaluation.generateValue was removed as part of HBASE-8496. This is an issue because it was used in a number of places, including unit tests. Since we did not explicitly label this API as private, it's ambiguous as to whether this could/should have been used by people writing apps against 0.96. If they used it, then they would be broken upon upgrade to 0.98 and trunk. Potential Solution: The method was renamed to generateData, but the logic is still the same. We can reintroduce it as deprecated in 0.98, as compat shim over generateData. The patch should be a few lines. We may also consider doing so in trunk, but I'd be just as fine with leaving it out. More generally, this raises the question about what other code is in this grey-area, where it is public, is used outside of the package, but is not explicitly labeled with an AudienceInterface. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4920) We need a mascot, a totem
[ https://issues.apache.org/jira/browse/HBASE-4920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13997932#comment-13997932 ] Dima Spivak commented on HBASE-4920: I agree with Enis, the mean orca looks like it's going to do something nefarious with all your data. That said, the overall design makes it much clearer that it's a whale (versus a few of the more abstract proofs that, as Lars pointed out two years ago, look embryonic). Perhaps the design team can muster an orca in a better mood, Stack? :-) We need a mascot, a totem - Key: HBASE-4920 URL: https://issues.apache.org/jira/browse/HBASE-4920 Project: HBase Issue Type: Task Reporter: stack Attachments: Apache_HBase_Orca_Logo_1.jpg, Apache_HBase_Orca_Logo_Mean_version-3.pdf, Apache_HBase_Orca_Logo_Mean_version-4.pdf, HBase Orca Logo.jpg, Orca_479990801.jpg, Screen shot 2011-11-30 at 4.06.17 PM.png, apache hbase orca logo_Proof 3.pdf, apache logo_Proof 8.pdf, krake.zip, more_orcas.png, more_orcas2.png, photo (2).JPG, plus_orca.png We need a totem for our t-shirt that is yet to be printed. O'Reilly owns the Clyesdale. We need something else. We could have a fluffy little duck that quacks 'hbase!' when you squeeze it and we could order boxes of them from some off-shore sweatshop that subcontracts to a contractor who employs child labor only. Or we could have an Orca (Big!, Fast!, Killer!, and in a poem that Marcy from Salesforce showed me, that was a bit too spiritual for me to be seen quoting here, it had the Orca as the 'Guardian of the Cosmic Memory': i.e. in translation, bigdata). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11196) Update description of -ROOT- in ref guide
[ https://issues.apache.org/jira/browse/HBASE-11196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11196: Description: Since the resolution of HBASE-3171, \-ROOT- is no longer used to store the location(s) of .META. . Unfortunately, not all of [our documentation|http://hbase.apache.org/book/arch.catalog.html] has been updated to reflect this change in architecture. (was: Since the resolution of [HBASE-3171|https://issues.apache.org/jira/browse/HBASE-3171], \-ROOT- is no longer used to store the location(s) of .META. . Unfortunately, not all of [our documentation|http://hbase.apache.org/book/arch.catalog.html] has been updated to reflect this change in architecture.) Update description of -ROOT- in ref guide - Key: HBASE-11196 URL: https://issues.apache.org/jira/browse/HBASE-11196 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Since the resolution of HBASE-3171, \-ROOT- is no longer used to store the location(s) of .META. . Unfortunately, not all of [our documentation|http://hbase.apache.org/book/arch.catalog.html] has been updated to reflect this change in architecture. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11196) Update description of -ROOT- in ref guide
[ https://issues.apache.org/jira/browse/HBASE-11196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11196: Description: Since the resolution of [HBASE-3171|https://issues.apache.org/jira/browse/HBASE-3171], \-ROOT- is no longer used to store the location(s) of .META. . Unfortunately, not all of [our documentation|http://hbase.apache.org/book/arch.catalog.html] has been updated to reflect this change in architecture. (was: Since the resolution of [HBASE-3171|https://issues.apache.org/jira/browse/HBASE-3171], -ROOT- is no longer used to store the location(s) of .META. . Unfortunately, not all of [our documentation|http://hbase.apache.org/book/arch.catalog.html] has been updated to reflect this change in architecture.) Update description of -ROOT- in ref guide - Key: HBASE-11196 URL: https://issues.apache.org/jira/browse/HBASE-11196 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Since the resolution of [HBASE-3171|https://issues.apache.org/jira/browse/HBASE-3171], \-ROOT- is no longer used to store the location(s) of .META. . Unfortunately, not all of [our documentation|http://hbase.apache.org/book/arch.catalog.html] has been updated to reflect this change in architecture. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11196) Update description of -ROOT- in ref guide
Dima Spivak created HBASE-11196: --- Summary: Update description of -ROOT- in ref guide Key: HBASE-11196 URL: https://issues.apache.org/jira/browse/HBASE-11196 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Since the resolution of [HBASE-3171|https://issues.apache.org/jira/browse/HBASE-3171], -ROOT- is no longer used to store the location(s) of .META. . Unfortunately, not all of [our documentation|http://hbase.apache.org/book/arch.catalog.html] has been updated to reflect this change in architecture. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11237) Bulk load initiated by user other than hbase fails
Dima Spivak created HBASE-11237: --- Summary: Bulk load initiated by user other than hbase fails Key: HBASE-11237 URL: https://issues.apache.org/jira/browse/HBASE-11237 Project: HBase Issue Type: Bug Affects Versions: 0.98.1 Reporter: Dima Spivak Assignee: Jimmy Xiang Priority: Critical Running TestLoadIncrementalHFiles and TestHFileOutputFormat as a properly kinit'd HBase superuser who isn't hbase began to fail last month after a patch to fix HBASE-10902 was committed to trunk. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-10217) SlabCache size logging on initialization is wrong
[
https://issues.apache.org/jira/browse/HBASE-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak resolved HBASE-10217.
-
Resolution: Fixed
Assignee: Dima Spivak (was: bharath v)
I couldn't reproduce this problem on my cluster. Doing a bit of digging, it
looks like this problem was fixed in 0.96, 0.98, and master by a commit Nick
made on 2013.12.12. Please reopen if I'm mistaken.
SlabCache size logging on initialization is wrong
-
Key: HBASE-10217
URL: https://issues.apache.org/jira/browse/HBASE-10217
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.99.0
Reporter: Nick Dimiduk
Assignee: Dima Spivak
Priority: Trivial
Labels: noob
From the logs:
{noformat}
2013-12-17 23:57:40,060 INFO [main] slab.SlabCache: Creating a slab of
blockSize 72089 with 140233 blocks, 1.4 Gbytes.
2013-12-17 23:57:43,622 INFO [main] slab.SlabCache: Creating a slab of
blockSize 137625 with 18363 blocks, -1.6 Gbytes.
{noformat}
By my math, these values should be 9.4G and 2.4G respectively.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11271) Chaos Monkey documentation is out of date
Dima Spivak created HBASE-11271: --- Summary: Chaos Monkey documentation is out of date Key: HBASE-11271 URL: https://issues.apache.org/jira/browse/HBASE-11271 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak The [ref guide chapter on testing discusses|http://hbase.apache.org/book/hbase.tests.html#integration.tests] Chaos Monkey and speaks extensively of running it standalone on a deployed cluster. This functionality hasn't been in HBase since 0.94 and so should probably be taken out of the documentation or updated with some sort deprecation notice. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10217) SlabCache size logging on initialization is wrong
[
https://issues.apache.org/jira/browse/HBASE-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14012204#comment-14012204
]
Dima Spivak commented on HBASE-10217:
-
It looks like this one:
https://github.com/apache/hbase/commit/2124f4054d7684ad36b0a33ff37dd39d153cff2b
SlabCache size logging on initialization is wrong
-
Key: HBASE-10217
URL: https://issues.apache.org/jira/browse/HBASE-10217
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.99.0
Reporter: Nick Dimiduk
Assignee: Dima Spivak
Priority: Trivial
Labels: noob
From the logs:
{noformat}
2013-12-17 23:57:40,060 INFO [main] slab.SlabCache: Creating a slab of
blockSize 72089 with 140233 blocks, 1.4 Gbytes.
2013-12-17 23:57:43,622 INFO [main] slab.SlabCache: Creating a slab of
blockSize 137625 with 18363 blocks, -1.6 Gbytes.
{noformat}
By my math, these values should be 9.4G and 2.4G respectively.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11271) Chaos Monkey documentation is out of date
[ https://issues.apache.org/jira/browse/HBASE-11271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14013987#comment-14013987 ] Dima Spivak commented on HBASE-11271: - Can do, ~stack! I'll add it to my list for next week, so will close this one for now. Chaos Monkey documentation is out of date - Key: HBASE-11271 URL: https://issues.apache.org/jira/browse/HBASE-11271 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak The [ref guide chapter on testing discusses|http://hbase.apache.org/book/hbase.tests.html#integration.tests] Chaos Monkey and speaks extensively of running it standalone on a deployed cluster. This functionality hasn't been in HBase since 0.94 and so should probably be taken out of the documentation or updated with some sort deprecation notice. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11276) Add back support for running ChaosMonkey as standalone tool
Dima Spivak created HBASE-11276: --- Summary: Add back support for running ChaosMonkey as standalone tool Key: HBASE-11276 URL: https://issues.apache.org/jira/browse/HBASE-11276 Project: HBase Issue Type: Task Affects Versions: 0.96.0, 0.98.0, 0.99.0 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor [According to the ref guide|http://hbase.apache.org/book/hbase.tests.html#integration.tests], it was once possible to run ChaosMonkey as a standalone tool against a deployed cluster. After 0.94, this is no longer possible. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11271) Chaos Monkey documentation is out of date
[ https://issues.apache.org/jira/browse/HBASE-11271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak resolved HBASE-11271. - Resolution: Invalid Will look at re-adding this functionality. Chaos Monkey documentation is out of date - Key: HBASE-11271 URL: https://issues.apache.org/jira/browse/HBASE-11271 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak The [ref guide chapter on testing discusses|http://hbase.apache.org/book/hbase.tests.html#integration.tests] Chaos Monkey and speaks extensively of running it standalone on a deployed cluster. This functionality hasn't been in HBase since 0.94 and so should probably be taken out of the documentation or updated with some sort deprecation notice. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11238) Add info about SlabCache and BucketCache to Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015169#comment-14015169 ] Dima Spivak commented on HBASE-11238: - Just as a thought: it might be worth noting on which machines particular files need to be modified (e.g. those running RegionServers) for things like this. Playing around with setting up BucketCache earlier today, I inadvertently modified hbase-env.sh on my gateway before understanding why my RS' couldn't restart without throwing OutOfMemoryError. Add info about SlabCache and BucketCache to Ref Guide - Key: HBASE-11238 URL: https://issues.apache.org/jira/browse/HBASE-11238 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.98.2 Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Attachments: HBASE-11238-2.patch, HBASE-11238-3.patch, HBASE-11238.patch Upstream issues: HBASE-11171 and HBASE-11098. Could back port some of what is in these issues, the package-info.java class for instance. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11340) Remove references to xcievers in documentation
Dima Spivak created HBASE-11340: --- Summary: Remove references to xcievers in documentation Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11340: Status: Patch Available (was: Open) Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Work stopped] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-11340 stopped by Dima Spivak. Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Work started] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-11340 started by Dima Spivak. Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11340: Attachment: HBASE-11340.patch Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11338) Expand documentation on bloom filters
[ https://issues.apache.org/jira/browse/HBASE-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11338: Component/s: (was: Filters) Expand documentation on bloom filters - Key: HBASE-11338 URL: https://issues.apache.org/jira/browse/HBASE-11338 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Ref Guide could use more info on bloom filters. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030829#comment-14030829 ] Dima Spivak commented on HBASE-11340: - Oops, sorry about that. I'll fix the formatting issue, throw in a mention of the former name, and upload a new revision shortly. Thanks for taking a look! -Dima Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11340) Remove references to xcievers in documentation
[ https://issues.apache.org/jira/browse/HBASE-11340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11340: Attachment: HBASE-11340-v2.patch Fixed the missing closing para tag and added a mention of the formerly known as name in the configuration section (along with a link to the blog post discussing it). Remove references to xcievers in documentation -- Key: HBASE-11340 URL: https://issues.apache.org/jira/browse/HBASE-11340 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak Assignee: Dima Spivak Priority: Trivial Attachments: HBASE-11340-v2.patch, HBASE-11340.patch While most references to xcievers were removed in HBase as part of [HBASE-5697], the ref guide still contains a few stray references. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11381) Isolate and rerun unit tests that fail following a code commit
Dima Spivak created HBASE-11381: --- Summary: Isolate and rerun unit tests that fail following a code commit Key: HBASE-11381 URL: https://issues.apache.org/jira/browse/HBASE-11381 Project: HBase Issue Type: Task Reporter: Dima Spivak It's not uncommon to see that, after changes are committed to a branch, a set of unit tests will begin to fail and Hudson will add a comment to the relevant JIRAs reporting on the unsuccessful build. Unfortunately, these test failures are not always indicative of regressions; sometimes, the problem is with infrastructure and simply rerunning the tests can make them go green again. I propose modifying the Jenkins job that is triggered by a code commit to address this problem. In particular, the job could use the reports generated by the Maven Surefire plugin to generate a list of tests to rerun. These tests can be set to rerun any number of times and a threshold ratio of passes-to-fails can be the deciding factor in whether a real bug has been introduced following a change in HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11381) Isolate and rerun unit tests that fail following a code commit
[ https://issues.apache.org/jira/browse/HBASE-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14038095#comment-14038095 ] Dima Spivak commented on HBASE-11381: - To provide a few more concrete details on the logistics of this, the Jenkins [Naginator Plugin|https://wiki.jenkins-ci.org/display/JENKINS/Naginator+Plugin] seems well-suited for this task. Note that the newly triggered builds could specifically be set to use the previous build's artifacts so that the exact same bits are retested. Isolate and rerun unit tests that fail following a code commit -- Key: HBASE-11381 URL: https://issues.apache.org/jira/browse/HBASE-11381 Project: HBase Issue Type: Task Reporter: Dima Spivak It's not uncommon to see that, after changes are committed to a branch, a set of unit tests will begin to fail and Hudson will add a comment to the relevant JIRAs reporting on the unsuccessful build. Unfortunately, these test failures are not always indicative of regressions; sometimes, the problem is with infrastructure and simply rerunning the tests can make them go green again. I propose modifying the Jenkins job that is triggered by a code commit to address this problem. In particular, the job could use the reports generated by the Maven Surefire plugin to generate a list of tests to rerun. These tests can be set to rerun any number of times and a threshold ratio of passes-to-fails can be the deciding factor in whether a real bug has been introduced following a change in HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11587) Improve documentation around custom filters
Dima Spivak created HBASE-11587: --- Summary: Improve documentation around custom filters Key: HBASE-11587 URL: https://issues.apache.org/jira/browse/HBASE-11587 Project: HBase Issue Type: Bug Components: documentation Reporter: Dima Spivak The instructions in the ref guide surrounding creation of custom filters by extending FilterBase are incomplete and need updating. Since 0.96 and protobufs, it looks like you need to implement the parseFrom method to make custom filter work. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-11721) jdiff script no longer works as usage instructions indicate
[ https://issues.apache.org/jira/browse/HBASE-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak reassigned HBASE-11721: --- Assignee: Dima Spivak jdiff script no longer works as usage instructions indicate --- Key: HBASE-11721 URL: https://issues.apache.org/jira/browse/HBASE-11721 Project: HBase Issue Type: Bug Components: scripts Reporter: Misty Stanley-Jones Assignee: Dima Spivak I pasted the command from the usage instructions embedded in the script, but it fails as follows: [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git 0.94 JDiff evaluation beginning: Determining if this is a local directory or a git repo. Looks like https://github.com/apache/hbase.git is a git repo Determining if this is a local directory or a git repo. Looks like https://github.com/MY_REPO/hbase.git is a git repo We are going to compare source 1 which is a git_repo and source 2, which is a git_repo 0.94 0.94 JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to /tmp/jdiff. % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448 Archive: jdiff-1.1.1-with-incompatible-option.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of jdiff-1.1.1-with-incompatible-option.zip or jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11721) jdiff script no longer works as usage instructions indicate
[ https://issues.apache.org/jira/browse/HBASE-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11721: Attachment: HBASE-11721.patch Fixes problems that prevent the script from running. Will file a separate JIRA to do a more thorough revision of the script to improve usability and readability. jdiff script no longer works as usage instructions indicate --- Key: HBASE-11721 URL: https://issues.apache.org/jira/browse/HBASE-11721 Project: HBase Issue Type: Bug Components: scripts Reporter: Misty Stanley-Jones Assignee: Dima Spivak Attachments: HBASE-11721.patch I pasted the command from the usage instructions embedded in the script, but it fails as follows: [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git 0.94 JDiff evaluation beginning: Determining if this is a local directory or a git repo. Looks like https://github.com/apache/hbase.git is a git repo Determining if this is a local directory or a git repo. Looks like https://github.com/MY_REPO/hbase.git is a git repo We are going to compare source 1 which is a git_repo and source 2, which is a git_repo 0.94 0.94 JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to /tmp/jdiff. % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448 Archive: jdiff-1.1.1-with-incompatible-option.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of jdiff-1.1.1-with-incompatible-option.zip or jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Work started] (HBASE-11721) jdiff script no longer works as usage instructions indicate
[ https://issues.apache.org/jira/browse/HBASE-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-11721 started by Dima Spivak. jdiff script no longer works as usage instructions indicate --- Key: HBASE-11721 URL: https://issues.apache.org/jira/browse/HBASE-11721 Project: HBase Issue Type: Bug Components: scripts Reporter: Misty Stanley-Jones Assignee: Dima Spivak Attachments: HBASE-11721.patch I pasted the command from the usage instructions embedded in the script, but it fails as follows: [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git 0.94 JDiff evaluation beginning: Determining if this is a local directory or a git repo. Looks like https://github.com/apache/hbase.git is a git repo Determining if this is a local directory or a git repo. Looks like https://github.com/MY_REPO/hbase.git is a git repo We are going to compare source 1 which is a git_repo and source 2, which is a git_repo 0.94 0.94 JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to /tmp/jdiff. % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448 Archive: jdiff-1.1.1-with-incompatible-option.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of jdiff-1.1.1-with-incompatible-option.zip or jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11721) jdiff script no longer works as usage instructions indicate
[ https://issues.apache.org/jira/browse/HBASE-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11721: Status: Patch Available (was: In Progress) jdiff script no longer works as usage instructions indicate --- Key: HBASE-11721 URL: https://issues.apache.org/jira/browse/HBASE-11721 Project: HBase Issue Type: Bug Components: scripts Reporter: Misty Stanley-Jones Assignee: Dima Spivak Attachments: HBASE-11721.patch I pasted the command from the usage instructions embedded in the script, but it fails as follows: [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git 0.94 JDiff evaluation beginning: Determining if this is a local directory or a git repo. Looks like https://github.com/apache/hbase.git is a git repo Determining if this is a local directory or a git repo. Looks like https://github.com/MY_REPO/hbase.git is a git repo We are going to compare source 1 which is a git_repo and source 2, which is a git_repo 0.94 0.94 JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to /tmp/jdiff. % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448 Archive: jdiff-1.1.1-with-incompatible-option.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of jdiff-1.1.1-with-incompatible-option.zip or jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11808) Update jdiff script to improve usability and readability
Dima Spivak created HBASE-11808: --- Summary: Update jdiff script to improve usability and readability Key: HBASE-11808 URL: https://issues.apache.org/jira/browse/HBASE-11808 Project: HBase Issue Type: Improvement Reporter: Dima Spivak Assignee: Dima Spivak We should update the jdiffHBasePublicAPI script in hbase/dev-support to be a bit more user-friendly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Work started] (HBASE-11808) Update jdiff script to improve usability and readability
[ https://issues.apache.org/jira/browse/HBASE-11808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-11808 started by Dima Spivak. Update jdiff script to improve usability and readability Key: HBASE-11808 URL: https://issues.apache.org/jira/browse/HBASE-11808 Project: HBase Issue Type: Improvement Reporter: Dima Spivak Assignee: Dima Spivak We should update the jdiffHBasePublicAPI script in hbase/dev-support to be a bit more user-friendly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11721) jdiff script no longer works as usage instructions indicate
[ https://issues.apache.org/jira/browse/HBASE-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11721: Attachment: HBASE-11721-v2.patch Reuploading the patch generated with git format-patch. jdiff script no longer works as usage instructions indicate --- Key: HBASE-11721 URL: https://issues.apache.org/jira/browse/HBASE-11721 Project: HBase Issue Type: Bug Components: scripts Reporter: Misty Stanley-Jones Assignee: Dima Spivak Attachments: HBASE-11721-v2.patch, HBASE-11721.patch I pasted the command from the usage instructions embedded in the script, but it fails as follows: [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git 0.94 JDiff evaluation beginning: Determining if this is a local directory or a git repo. Looks like https://github.com/apache/hbase.git is a git repo Determining if this is a local directory or a git repo. Looks like https://github.com/MY_REPO/hbase.git is a git repo We are going to compare source 1 which is a git_repo and source 2, which is a git_repo 0.94 0.94 JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to /tmp/jdiff. % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448 Archive: jdiff-1.1.1-with-incompatible-option.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of jdiff-1.1.1-with-incompatible-option.zip or jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11721) jdiff script no longer works as usage instructions indicate
[ https://issues.apache.org/jira/browse/HBASE-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11721: Attachment: HBASE-11721-v3.patch Patch modifies arguments to sed to support BSD derivatives (i.e. what you'd find on Mac OS X). You're too thorough, Misty. :) jdiff script no longer works as usage instructions indicate --- Key: HBASE-11721 URL: https://issues.apache.org/jira/browse/HBASE-11721 Project: HBase Issue Type: Bug Components: scripts Reporter: Misty Stanley-Jones Assignee: Dima Spivak Attachments: HBASE-11721-v2.patch, HBASE-11721-v3.patch, HBASE-11721.patch I pasted the command from the usage instructions embedded in the script, but it fails as follows: [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git 0.94 JDiff evaluation beginning: Determining if this is a local directory or a git repo. Looks like https://github.com/apache/hbase.git is a git repo Determining if this is a local directory or a git repo. Looks like https://github.com/MY_REPO/hbase.git is a git repo We are going to compare source 1 which is a git_repo and source 2, which is a git_repo 0.94 0.94 JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to /tmp/jdiff. % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448 Archive: jdiff-1.1.1-with-incompatible-option.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of jdiff-1.1.1-with-incompatible-option.zip or jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11721) jdiff script no longer works as usage instructions indicate
[ https://issues.apache.org/jira/browse/HBASE-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11721: Attachment: HBASE-11721-v4.patch Trim down lines that Hadoop QA complained about. Note that there are a ton of other 100 char lines in this script, but I'm not touching them since it'll be superseded by HBASE-11808. jdiff script no longer works as usage instructions indicate --- Key: HBASE-11721 URL: https://issues.apache.org/jira/browse/HBASE-11721 Project: HBase Issue Type: Bug Components: scripts Reporter: Misty Stanley-Jones Assignee: Dima Spivak Attachments: HBASE-11721-v2.patch, HBASE-11721-v3.patch, HBASE-11721-v4.patch, HBASE-11721.patch I pasted the command from the usage instructions embedded in the script, but it fails as follows: [misty@cheezel dev-support](master)$ bash ./jdiffHBasePublicAPI.sh https://github.com/apache/hbase.git 0.94 https://github.com/MY_REPO/hbase.git 0.94 JDiff evaluation beginning: Determining if this is a local directory or a git repo. Looks like https://github.com/apache/hbase.git is a git repo Determining if this is a local directory or a git repo. Looks like https://github.com/MY_REPO/hbase.git is a git repo We are going to compare source 1 which is a git_repo and source 2, which is a git_repo 0.94 0.94 JDIFF_WORKING_DIRECTORY not set. That's not an issue. We will default it to /tmp/jdiff. % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 183 100 1830 0447 0 --:--:-- --:--:-- --:--:-- 448 Archive: jdiff-1.1.1-with-incompatible-option.zip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. unzip: cannot find zipfile directory in one of jdiff-1.1.1-with-incompatible-option.zip or jdiff-1.1.1-with-incompatible-option.zip.zip, and cannot find jdiff-1.1.1-with-incompatible-option.zip.ZIP, period. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
Dima Spivak created HBASE-11885: --- Summary: Provide a Dockerfile to easily build and run HBase from source Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
[
https://issues.apache.org/jira/browse/HBASE-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak reopened HBASE-8674:
JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
Key: HBASE-8674
URL: https://issues.apache.org/jira/browse/HBASE-8674
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.98.0, 0.94.8, 0.95.1
Reporter: Andrew Purtell
people.apache.org cannot currently host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are at the moment broken if the HBase custom junit or surefire
jars are not already in cache. Even if this is a temporary condition, we
should find a new home for these artifacts, upgrade to versions that include
our submitted changes (if any), or fall back to release versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
[
https://issues.apache.org/jira/browse/HBASE-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119497#comment-14119497
]
Dima Spivak commented on HBASE-8674:
It might be worth revisiting this. My builds tonight were slowed to a crawl
after I had to rebuild my local Maven repository and I can only assume that
Gary's personal repo was sluggish because simply removing references to it in
the root POM got things speedy again.
JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
Key: HBASE-8674
URL: https://issues.apache.org/jira/browse/HBASE-8674
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.98.0, 0.94.8, 0.95.1
Reporter: Andrew Purtell
people.apache.org cannot currently host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are at the moment broken if the HBase custom junit or surefire
jars are not already in cache. Even if this is a temporary condition, we
should find a new home for these artifacts, upgrade to versions that include
our submitted changes (if any), or fall back to release versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[
https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119501#comment-14119501
]
Dima Spivak commented on HBASE-11885:
-
I have a working Dockerfile that sets up the necessary HBase dependencies (i.e.
Maven and Java), clones in the repo, runs {{mvn assembly:single}}, and then
starts HBase and the HBase shell. The main problem I've run into pertains to
the Maven step due to
[HBASE-8674|https://issues.apache.org/jira/browse/HBASE-8674]. Can someone with
knowledge of why the POMs are in the state that they are in take a look there
and chime in on if there any consequence to simply removing Gary H's repo as a
dependency?
Provide a Dockerfile to easily build and run HBase from source
--
Key: HBASE-11885
URL: https://issues.apache.org/jira/browse/HBASE-11885
Project: HBase
Issue Type: New Feature
Reporter: Dima Spivak
Assignee: Dima Spivak
[A recent email to
dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E]
highlighted the difficulty that new users can face in getting HBase compiled
from source and running locally. I'd like to provide a Dockerfile that would
allow anyone with Docker running on a machine with a reasonably current Linux
kernel to do so with ease.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120158#comment-14120158 ] Dima Spivak commented on HBASE-11885: - +1 ^^^. As I said, I had no issues at all building master from source after removing that repository altogether (0.94 is a different story, because of the dependency on version 2.12-TRUNK-HBASE-2 Surefire). Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-11885 started by Dima Spivak. --- Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11885: Status: Patch Available (was: In Progress) Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-11885.patch [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11885: Attachment: HBASE-11885.patch Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-11885.patch [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[
https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120730#comment-14120730
]
Dima Spivak commented on HBASE-11885:
-
Uploaded my first iteration of the Dockerfile. I hope the instructions on use
are clear; you basically just have to put a JDK and Maven tarball in the same
directory and type {{docker build .}} to create an image.
Provide a Dockerfile to easily build and run HBase from source
--
Key: HBASE-11885
URL: https://issues.apache.org/jira/browse/HBASE-11885
Project: HBase
Issue Type: New Feature
Reporter: Dima Spivak
Assignee: Dima Spivak
Attachments: HBASE-11885.patch
[A recent email to
dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E]
highlighted the difficulty that new users can face in getting HBase compiled
from source and running locally. I'd like to provide a Dockerfile that would
allow anyone with Docker running on a machine with a reasonably current Linux
kernel to do so with ease.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120789#comment-14120789 ] Dima Spivak commented on HBASE-11885: - I think for licensing reasons (i.e. having to do with Java) and also to avoid stale Maven versions down the line being baked into the Dockerfile itself, having a user supply their own preferred version is the way to go if we are to include a Dockerfile for users in Apache HBase. I think this also lets someone easily test different combinations of Java and Maven to validate that the code builds. Thoughts? Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-11885.patch [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120813#comment-14120813 ] Dima Spivak commented on HBASE-11885: - :) Yep, that's why I think this would belong in hbase/dev-support. Good points all around, [~tobe]. Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-11885.patch [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14121002#comment-14121002 ] Dima Spivak commented on HBASE-11885: - Thanks for the feedback, Srikanth; I'll update documentation tomorrow. The error you saw is because of what seems to be buggy implementation of the devicemapper driver. Unfortunately, this tends to get fixed by just rerunning docker build (sometimes repeatedly). The fact that the image didn't complete is why the second error about being unable to find hbase_docker showed up (Docker always tries to pull images from the central registry if they aren't present locally). Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-11885.patch [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14121605#comment-14121605 ] Dima Spivak commented on HBASE-11885: - Yeah, I could create a wrapper script to do that, I just wonder how much use it would get. Someone wanting to automatically download Oracle and Maven would still have to go to the sites to get URLs, and as you point out, in the case of JDK, would also need to pass the correct cookie accepting Oracle's license agreement. In fact, my first iteration of the Dockerfile did just that (i.e. had hardcoded URLs and the cookie passed with wget) before I worried that it would quickly become stale. What do you think? Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-11885.patch [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-11885: Attachment: HBASE-11885_v2.patch Added hbase_docker.sh into dev-support, a driver script that lets users specify locations of the Oracle JDK and Maven tarballs before running docker build (repeatedly, if necessary), and then starting the docker image. Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-11885.patch, HBASE-11885_v2.patch [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123792#comment-14123792 ] Dima Spivak commented on HBASE-11885: - Hey Andy, I've confirmed that it works as expected with master, 0.98, and branch-1. When committing into those branches, I'd just have to modify one line in the Dockerfile to reflect the desired branch. Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-11885.patch, HBASE-11885_v2.patch [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123849#comment-14123849 ] Dima Spivak commented on HBASE-11885: - Yessir! Changing that to 0.98 or branch-1 works out of box (just confirmed on my machines). Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-11885.patch, HBASE-11885_v2.patch [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11885) Provide a Dockerfile to easily build and run HBase from source
[ https://issues.apache.org/jira/browse/HBASE-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123874#comment-14123874 ] Dima Spivak commented on HBASE-11885: - Fine by me. I just copied and pasted from GitHub, but figured maybe the Apache variant was more correct. Provide a Dockerfile to easily build and run HBase from source -- Key: HBASE-11885 URL: https://issues.apache.org/jira/browse/HBASE-11885 Project: HBase Issue Type: New Feature Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-11885.patch, HBASE-11885_v2.patch [A recent email to dev@|http://mail-archives.apache.org/mod_mbox/hbase-dev/201408.mbox/%3CCAAef%2BM4q%3Da8Dqxe_EHSFTueY%2BXxz%2BtTe%2BJKsWWbXjhB_Pz7oSA%40mail.gmail.com%3E] highlighted the difficulty that new users can face in getting HBase compiled from source and running locally. I'd like to provide a Dockerfile that would allow anyone with Docker running on a machine with a reasonably current Linux kernel to do so with ease. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11945) Client writes may be reordered under contention
[ https://issues.apache.org/jira/browse/HBASE-11945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129608#comment-14129608 ] Dima Spivak commented on HBASE-11945: - [~tlipcon], Working on it now. -Dima Client writes may be reordered under contention --- Key: HBASE-11945 URL: https://issues.apache.org/jira/browse/HBASE-11945 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.98.6 Reporter: Todd Lipcon I haven't seen this bug in practice, but I was thinking about this a bit and think there may be a correctness issue with the way that we handle client batches which contain multiple operations which touch the same row. The client expects that these operations will be performed in the same order they were submitted, but under contention I believe they can get arbitrarily reordered, leading to incorrect results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-11628) Clean up tests that expect an exception
[
https://issues.apache.org/jira/browse/HBASE-11628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak reassigned HBASE-11628:
---
Assignee: Dima Spivak
Clean up tests that expect an exception
---
Key: HBASE-11628
URL: https://issues.apache.org/jira/browse/HBASE-11628
Project: HBase
Issue Type: Sub-task
Components: test
Reporter: Sean Busbey
Assignee: Dima Spivak
Priority: Trivial
Labels: beginner
We have a few tests that look for expected exceptions by manually catching
and explicitly failing in case no exception happens.
Places that need to be fixed can be found by running:
{code}
$ grep -rie [^n]expected *exception `find . -name test -path '*/src/*'
-type d | xargs`
{code}
(there are some false positives, e.g. TestIPv6NIOServerSocketChannel)
We should clean these up to use [JUnit's expected arg to the Test
annotation|http://junit.org/javadoc/4.11/org/junit/Test.html#expected()].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11628) Clean up tests that expect an exception
[
https://issues.apache.org/jira/browse/HBASE-11628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133366#comment-14133366
]
Dima Spivak commented on HBASE-11628:
-
I'd like to work on this next week, if that's alright. I'm really excited about
getting more familiar with our unit tests in general and committing myself to
this seems like good motivation. :)
Clean up tests that expect an exception
---
Key: HBASE-11628
URL: https://issues.apache.org/jira/browse/HBASE-11628
Project: HBase
Issue Type: Sub-task
Components: test
Reporter: Sean Busbey
Assignee: Dima Spivak
Priority: Trivial
Labels: beginner
We have a few tests that look for expected exceptions by manually catching
and explicitly failing in case no exception happens.
Places that need to be fixed can be found by running:
{code}
$ grep -rie [^n]expected *exception `find . -name test -path '*/src/*'
-type d | xargs`
{code}
(there are some false positives, e.g. TestIPv6NIOServerSocketChannel)
We should clean these up to use [JUnit's expected arg to the Test
annotation|http://junit.org/javadoc/4.11/org/junit/Test.html#expected()].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11960) Provide a sample to show how to use Thrift client authentication
[ https://issues.apache.org/jira/browse/HBASE-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139268#comment-14139268 ] Dima Spivak commented on HBASE-11960: - Hey Jimmy, Looks great to me. Just some minor nits: - Lines 198, 200, and 395 of DemoClient.java have stray whitespace. - You may want to update /hbase-examples/README.txt to reflect the addition of the [secure] argument to the DemoClient. Thanks so much for this. Provide a sample to show how to use Thrift client authentication Key: HBASE-11960 URL: https://issues.apache.org/jira/browse/HBASE-11960 Project: HBase Issue Type: Task Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: hbase-11960.patch Now Thrift server can authenticate clients. However, many people asked how to use it. Although we have some info in the refguide, it doesn't mention how to change the client to turn it on. A sample should help. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-5580) Publish Thrift-generated files for other languages
[ https://issues.apache.org/jira/browse/HBASE-5580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak reassigned HBASE-5580: -- Assignee: Dima Spivak Publish Thrift-generated files for other languages -- Key: HBASE-5580 URL: https://issues.apache.org/jira/browse/HBASE-5580 Project: HBase Issue Type: New Feature Components: Thrift Affects Versions: 0.90.4 Reporter: Patrick Angeles Assignee: Dima Spivak Labels: thrift, thrift2 HBase ships with Thrift-generated Java files for use with the ThriftServer. For convenience (and to save users the frustration of having to compile and install the Thrift compiler), HBase can ship with the thrift-generated files for other languages as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12098) User granted namespace table create permissions can't create a table
Dima Spivak created HBASE-12098:
---
Summary: User granted namespace table create permissions can't
create a table
Key: HBASE-12098
URL: https://issues.apache.org/jira/browse/HBASE-12098
Project: HBase
Issue Type: Bug
Components: Client, security
Affects Versions: 0.98.6
Reporter: Dima Spivak
Priority: Critical
From the HBase shell and Java API, I am seeing
{code}ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
Insufficient permissions for user 'dima' (global, action=CREATE){code}
when I try to create a table in a namespace to which I have been granted RWXCA
permissions by a global admin. Interestingly enough, this only seems to extend
to table creation; the same user is then allowed to disable and drop a table
created by a global admin in that namespace.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-12098) User granted namespace table create permissions can't create a table
[
https://issues.apache.org/jira/browse/HBASE-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak reassigned HBASE-12098:
---
Assignee: Srikanth Srungarapu
Assigning to Srikanth, because he's awesome.
User granted namespace table create permissions can't create a table
Key: HBASE-12098
URL: https://issues.apache.org/jira/browse/HBASE-12098
Project: HBase
Issue Type: Bug
Components: Client, security
Affects Versions: 0.98.6
Reporter: Dima Spivak
Assignee: Srikanth Srungarapu
Priority: Critical
Attachments: HBASE-12098.patch
From the HBase shell and Java API, I am seeing
{code}ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
Insufficient permissions for user 'dima' (global, action=CREATE){code}
when I try to create a table in a namespace to which I have been granted
RWXCA permissions by a global admin. Interestingly enough, this only seems to
extend to table creation; the same user is then allowed to disable and drop a
table created by a global admin in that namespace.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12120) HBase shell doesn't allow deleting of a cell by user with W-only permissions to it
Dima Spivak created HBASE-12120: --- Summary: HBase shell doesn't allow deleting of a cell by user with W-only permissions to it Key: HBASE-12120 URL: https://issues.apache.org/jira/browse/HBASE-12120 Project: HBase Issue Type: Bug Components: shell Reporter: Dima Spivak If cell-level ACLs are used to give a user W permissions to a cell, they should be able to delete that cell. In testing the Java API, I've confirmed that this is respected. Using the HBase shell, however, a user trying to delete a cell to which they don't have read permissions results in a Row Not Found error. [~mbertozzi] tracked this down to a change introduced by HBASE-5837 wherein an attempt is made to get the cell before deleting it to confirm that it exists. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-12120) HBase shell doesn't allow deleting of a cell by user with W-only permissions to it
[ https://issues.apache.org/jira/browse/HBASE-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak reassigned HBASE-12120: --- Assignee: Srikanth Srungarapu Awesome at it again. :) HBase shell doesn't allow deleting of a cell by user with W-only permissions to it -- Key: HBASE-12120 URL: https://issues.apache.org/jira/browse/HBASE-12120 Project: HBase Issue Type: Bug Components: shell Reporter: Dima Spivak Assignee: Srikanth Srungarapu Attachments: HBASE-12120.patch If cell-level ACLs are used to give a user W permissions to a cell, they should be able to delete that cell. In testing the Java API, I've confirmed that this is respected. Using the HBase shell, however, a user trying to delete a cell to which they don't have read permissions results in a Row Not Found error. [~mbertozzi] tracked this down to a change introduced by HBASE-5837 wherein an attempt is made to get the cell before deleting it to confirm that it exists. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12161) Add support for grant/revoke on namespaces in AccessControlClient
[ https://issues.apache.org/jira/browse/HBASE-12161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159816#comment-14159816 ] Dima Spivak commented on HBASE-12161: - Thanks for taking this on, Srikanth. Some minor suggestions: - Your method declaration for getAccessControlServiceStub should be split over two lines. Same for the AccessControlClient.grant in #testAccessControlClientGrantOnNameSpace. - The AccessControlClient#grant and #revoke methods have nearly identical logic that might as well be factored into a common private method to avoid a bit of code duplication. - Similarly (as you explained to me), your #testAccessControlClientGrantOnNameSpace is very similar to the existing #testNamespaceUserGrant. It might be worth factoring out the common bits and only leaving the different grant methods used by each test. - In #testAccessControlClientGrantOnNameSpace, you handle a Throwable by just logging it which swallow any exception that might break things. Is this intentional or might you want the test to fail at this point if such a general exception is thrown? Add support for grant/revoke on namespaces in AccessControlClient - Key: HBASE-12161 URL: https://issues.apache.org/jira/browse/HBASE-12161 Project: HBase Issue Type: Improvement Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Attachments: HBASE-12161_0.98.patch As per the description. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12181) Some tests create a table and try to use it before regions get assigned
Dima Spivak created HBASE-12181:
---
Summary: Some tests create a table and try to use it before
regions get assigned
Key: HBASE-12181
URL: https://issues.apache.org/jira/browse/HBASE-12181
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.6.1, 1.0.0, 2.0.0
Reporter: Dima Spivak
Assignee: Dima Spivak
I inadvertently did some unit test stress testing this weekend by running {{mvn
test -P runAllTests}} on a machine that also had other processes going on. The
overall sluggishness led to a number of failed tests that were caused by a
table being created using HBaseAdmin#createTable, which only blocks for meta to
get updated and not for the region to actually get assigned a RegionServer.
Quick fix is to use HBaseTestingUtility#createTable whenever possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Work started] (HBASE-12181) Some tests create a table and try to use it before regions get assigned
[
https://issues.apache.org/jira/browse/HBASE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-12181 started by Dima Spivak.
---
Some tests create a table and try to use it before regions get assigned
---
Key: HBASE-12181
URL: https://issues.apache.org/jira/browse/HBASE-12181
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.6.1
Reporter: Dima Spivak
Assignee: Dima Spivak
I inadvertently did some unit test stress testing this weekend by running
{{mvn test -P runAllTests}} on a machine that also had other processes going
on. The overall sluggishness led to a number of failed tests that were caused
by a table being created using HBaseAdmin#createTable, which only blocks for
meta to get updated and not for the region to actually get assigned a
RegionServer. Quick fix is to use HBaseTestingUtility#createTable whenever
possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12181) Some tests create a table and try to use it before regions get assigned
[
https://issues.apache.org/jira/browse/HBASE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164341#comment-14164341
]
Dima Spivak commented on HBASE-12181:
-
Good call, [~apurtell]; needed to end up doing that because a coprocessor was
being added to the HTableDescriptor instance.
Some tests create a table and try to use it before regions get assigned
---
Key: HBASE-12181
URL: https://issues.apache.org/jira/browse/HBASE-12181
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.6.1
Reporter: Dima Spivak
Assignee: Dima Spivak
I inadvertently did some unit test stress testing this weekend by running
{{mvn test -P runAllTests}} on a machine that also had other processes going
on. The overall sluggishness led to a number of failed tests that were caused
by a table being created using HBaseAdmin#createTable, which only blocks for
meta to get updated and not for the region to actually get assigned a
RegionServer. Quick fix is to use HBaseTestingUtility#createTable whenever
possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12181) Some tests create a table and try to use it before regions get assigned
[
https://issues.apache.org/jira/browse/HBASE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak updated HBASE-12181:
Attachment: HBASE-12181.patch
Some tests create a table and try to use it before regions get assigned
---
Key: HBASE-12181
URL: https://issues.apache.org/jira/browse/HBASE-12181
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.6.1
Reporter: Dima Spivak
Assignee: Dima Spivak
Attachments: HBASE-12181.patch
I inadvertently did some unit test stress testing this weekend by running
{{mvn test -P runAllTests}} on a machine that also had other processes going
on. The overall sluggishness led to a number of failed tests that were caused
by a table being created using HBaseAdmin#createTable, which only blocks for
meta to get updated and not for the region to actually get assigned a
RegionServer. Quick fix is to use HBaseTestingUtility#createTable whenever
possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12181) Some tests create a table and try to use it before regions get assigned
[
https://issues.apache.org/jira/browse/HBASE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak updated HBASE-12181:
Status: Patch Available (was: In Progress)
Some tests create a table and try to use it before regions get assigned
---
Key: HBASE-12181
URL: https://issues.apache.org/jira/browse/HBASE-12181
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.6.1, 1.0.0, 2.0.0
Reporter: Dima Spivak
Assignee: Dima Spivak
Attachments: HBASE-12181.patch
I inadvertently did some unit test stress testing this weekend by running
{{mvn test -P runAllTests}} on a machine that also had other processes going
on. The overall sluggishness led to a number of failed tests that were caused
by a table being created using HBaseAdmin#createTable, which only blocks for
meta to get updated and not for the region to actually get assigned a
RegionServer. Quick fix is to use HBaseTestingUtility#createTable whenever
possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12212) HBaseTestingUtility#waitUntilAllRegionsAssigned should wait for RegionStates
Dima Spivak created HBASE-12212: --- Summary: HBaseTestingUtility#waitUntilAllRegionsAssigned should wait for RegionStates Key: HBASE-12212 URL: https://issues.apache.org/jira/browse/HBASE-12212 Project: HBase Issue Type: Improvement Affects Versions: 0.98.6, 1.0.0, 2.0.0 Reporter: Dima Spivak At the moment, #waitUntilAllRegionsAssigned checks to see if each region has a value for info:server in the meta table, but I've encountered a few instances of tests that fail because RegionStates#getRegionsOfTable returns an empty list, despite #waitUntilAllRegionsAssigned being called after the table was created. Let's fix this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12221) Port TestFromClientSide to hbase-it
Dima Spivak created HBASE-12221: --- Summary: Port TestFromClientSide to hbase-it Key: HBASE-12221 URL: https://issues.apache.org/jira/browse/HBASE-12221 Project: HBase Issue Type: New Feature Affects Versions: 0.98.7, 1.0.0, 2.0.0 Reporter: Dima Spivak Assignee: Dima Spivak org.apache.hadoop.hbase.client.TestFromClientSide touches a lot of the HBase Java API and would be a nice integration test to have for anyone looking to verify end-to-end functionality. I'd like to turn it into an integration test and have it live in hbase-it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12221) Port TestFromClientSide to hbase-it
[ https://issues.apache.org/jira/browse/HBASE-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165991#comment-14165991 ] Dima Spivak commented on HBASE-12221: - :) Definitely not moving, just also adding it to the integration module. Port TestFromClientSide to hbase-it --- Key: HBASE-12221 URL: https://issues.apache.org/jira/browse/HBASE-12221 Project: HBase Issue Type: New Feature Affects Versions: 1.0.0, 2.0.0, 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak org.apache.hadoop.hbase.client.TestFromClientSide touches a lot of the HBase Java API and would be a nice integration test to have for anyone looking to verify end-to-end functionality. I'd like to turn it into an integration test and have it live in hbase-it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12212) HBaseTestingUtility#waitUntilAllRegionsAssigned should wait for RegionStates
[ https://issues.apache.org/jira/browse/HBASE-12212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166004#comment-14166004 ] Dima Spivak commented on HBASE-12212: - Looks good to me, just a minor nit in the comment (// returing... missing the n). +1 HBaseTestingUtility#waitUntilAllRegionsAssigned should wait for RegionStates Key: HBASE-12212 URL: https://issues.apache.org/jira/browse/HBASE-12212 Project: HBase Issue Type: Improvement Affects Versions: 1.0.0, 2.0.0, 0.98.6 Reporter: Dima Spivak Assignee: Dima Spivak Attachments: 0002-HBASE-12212-HBaseTestingUtility-waitUntilAllRegionsA.patch At the moment, #waitUntilAllRegionsAssigned checks to see if each region has a value for info:server in the meta table, but I've encountered a few instances of tests that fail because RegionStates#getRegionsOfTable returns an empty list, despite #waitUntilAllRegionsAssigned being called after the table was created. Let's fix this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12181) Some tests create a table and try to use it before regions get assigned
[
https://issues.apache.org/jira/browse/HBASE-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166449#comment-14166449
]
Dima Spivak commented on HBASE-12181:
-
Thanks for this, [~apurtell]; I missed that 0.98 wasn't declaring TableNames
from the start. Much appreciated!
Some tests create a table and try to use it before regions get assigned
---
Key: HBASE-12181
URL: https://issues.apache.org/jira/browse/HBASE-12181
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 0.98.6.1
Reporter: Dima Spivak
Assignee: Dima Spivak
Fix For: 2.0.0, 0.98.8, 0.99.1
Attachments: HBASE-12181-0.98.patch, HBASE-12181.patch
I inadvertently did some unit test stress testing this weekend by running
{{mvn test -P runAllTests}} on a machine that also had other processes going
on. The overall sluggishness led to a number of failed tests that were caused
by a table being created using HBaseAdmin#createTable, which only blocks for
meta to get updated and not for the region to actually get assigned a
RegionServer. Quick fix is to use HBaseTestingUtility#createTable whenever
possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12221) Port TestFromClientSide to hbase-it
[ https://issues.apache.org/jira/browse/HBASE-12221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167520#comment-14167520 ] Dima Spivak commented on HBASE-12221: - Thanks for the pointer, [~enis]. I'm modeling IntegrationTestFromClientSide after your example. Port TestFromClientSide to hbase-it --- Key: HBASE-12221 URL: https://issues.apache.org/jira/browse/HBASE-12221 Project: HBase Issue Type: New Feature Affects Versions: 1.0.0, 2.0.0, 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak org.apache.hadoop.hbase.client.TestFromClientSide touches a lot of the HBase Java API and would be a nice integration test to have for anyone looking to verify end-to-end functionality. I'd like to turn it into an integration test and have it live in hbase-it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12229) NullPointerException in SnapshotTestingUtils
Dima Spivak created HBASE-12229: --- Summary: NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.7, 1.0.0, 2.0.0 Reporter: Dima Spivak Assignee: Dima Spivak I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-12229: Attachment: HBASE-12229.patch Pretty trivial; waits for regions to get assigned before go to an RS and asking for its regions. NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.0.0, 2.0.0, 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-12229.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-12229: Status: Patch Available (was: Open) Uploaded a one-liner for branch-1. Can do the same for 0.98 and master if necessary. NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.7, 1.0.0, 2.0.0 Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-12229.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-12229: Attachment: HBASE-12229_v1.patch Oops, sorry - did some Git silly. Trying again with a proper patch... NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.0.0, 2.0.0, 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Attachments: HBASE-12229.patch, HBASE-12229_v1.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-12229: Priority: Minor (was: Major) Sure, [~tedyu]; I can never remember when to call something major or minor :-p. Thanks. NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.0.0, 2.0.0, 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: HBASE-12229.patch, HBASE-12229_v1.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167760#comment-14167760 ] Dima Spivak commented on HBASE-12229: - So createTable() in HBaseTestingUtility already has the waitUntilAllRegionsAssigned. In this case, I just added the call to the wait because HBTU didn't have a createTable method that takes splitKeys, but I can add one of those instead of adding the call. More fundamentally, should we consider whether HBA#createTable itself should be synchronous up through the region assignments being made? On a separate note, [~mbertozzi], is the SnapshotTestingUtils#waitForTableToBeOnline right to be doing nothing with the boolean returned by isTableAvailable? It looks like you might have created the method and I wasn't sure whether to touch it with this JIRA. NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.0.0, 2.0.0, 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: HBASE-12229.patch, HBASE-12229_v1.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168329#comment-14168329 ] Dima Spivak commented on HBASE-12229: - That's a fair point, [~apurtell], and I'll post up a patch doing that. Is it worth separately addressing why the [synchronous] HBaseAdmin#createTable method returns before a table is actually ready to accept puts? If that was fixed, there wouldn't need to be these helper methods checking for regions being available in HBTU. NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.0.0, 2.0.0, 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: HBASE-12229.patch, HBASE-12229_v1.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12074) TestLogRollingNoCluster#testContendedLogRolling() failed
[
https://issues.apache.org/jira/browse/HBASE-12074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169745#comment-14169745
]
Dima Spivak commented on HBASE-12074:
-
I've seen this a few times on branch-1, as well. Some stdout for anyone who may
want more stdout:
{code}
2014-10-13 01:50:05,121 INFO [27] wal.FSHLog(895): Rolled WAL
/root/hbase/hbase-server/target/test-data/a50fe909-a402-40d1-85ac-4b1cd45ce1a3/logs/hlog.1413165005081
with entries=270, filesize=289.07 KB; new WAL
/root/hbase/hbase-server/target/test-data/a50fe909-a402-40d1-85ac-4b1cd45ce1a3/logs/hlog.1413165005100
2014-10-13 01:50:05,130 INFO [31] wal.FSHLog(895): Rolled WAL
/root/hbase/hbase-server/target/test-data/a50fe909-a402-40d1-85ac-4b1cd45ce1a3/logs/hlog.1413165005100
with entries=20, filesize=21.50 KB; new WAL
/root/hbase/hbase-server/target/test-data/a50fe909-a402-40d1-85ac-4b1cd45ce1a3/logs/hlog.1413165005123
2014-10-13 01:50:05,141 INFO [35] wal.FSHLog(895): Rolled WAL
/root/hbase/hbase-server/target/test-data/a50fe909-a402-40d1-85ac-4b1cd45ce1a3/logs/hlog.1413165005123
with entries=70, filesize=75.01 KB; new WAL
/root/hbase/hbase-server/target/test-data/a50fe909-a402-40d1-85ac-4b1cd45ce1a3/logs/hlog.1413165005131
2014-10-13 01:50:05,151 INFO [49] wal.FSHLog(895): Rolled WAL
/root/hbase/hbase-server/target/test-data/a50fe909-a402-40d1-85ac-4b1cd45ce1a3/logs/hlog.1413165005131
with entries=140, filesize=149.93 KB; new WAL
/root/hbase/hbase-server/target/test-data/a50fe909-a402-40d1-85ac-4b1cd45ce1a3/logs/hlog.1413165005143
2014-10-13 01:50:05,161 ERROR [sync.3] wal.FSHLog$SyncRunner(1306): Error
syncing, request close of hlog
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:173)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1302)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:170)
... 2 more
2014-10-13 01:50:05,162 INFO [2] wal.FSHLog(895): Rolled WAL
/root/hbase/hbase-server/target/test-data/a50fe909-a402-40d1-85ac-4b1cd45ce1a3/logs/hlog.1413165005143
with entries=185, filesize=198.10 KB; new WAL
/root/hbase/hbase-server/target/test-data/a50fe909-a402-40d1-85ac-4b1cd45ce1a3/logs/hlog.1413165005152
2014-10-13 01:50:05,162 INFO [23] wal.TestLogRollingNoCluster$Appender(139):
Caught exception from Appender:23
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:173)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1302)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:170)
... 2 more
2014-10-13 01:50:05,162 INFO [5] wal.TestLogRollingNoCluster$Appender(139):
Caught exception from Appender:5
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:173)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1302)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:170)
... 2 more
2014-10-13 01:50:05,162 INFO [53] wal.TestLogRollingNoCluster$Appender(139):
Caught exception from Appender:53
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:173)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1302)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:170)
... 2 more
2014-10-13 01:50:05,162 INFO [15] wal.TestLogRollingNoCluster$Appender(139):
Caught exception from Appender:15
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:173)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1302)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:170)
... 2 more
2014-10-13 01:50:05,162 INFO [3] wal.TestLogRollingNoCluster$Appender(139):
Caught exception from Appender:3
java.io.IOException: java.lang.NullPointerException
at
[jira] [Work started] (HBASE-11628) Clean up tests that expect an exception
[
https://issues.apache.org/jira/browse/HBASE-11628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-11628 started by Dima Spivak.
---
Clean up tests that expect an exception
---
Key: HBASE-11628
URL: https://issues.apache.org/jira/browse/HBASE-11628
Project: HBase
Issue Type: Sub-task
Components: test
Reporter: Sean Busbey
Assignee: Dima Spivak
Priority: Trivial
Labels: beginner
We have a few tests that look for expected exceptions by manually catching
and explicitly failing in case no exception happens.
Places that need to be fixed can be found by running:
{code}
$ grep -rie [^n]expected *exception `find . -name test -path '*/src/*'
-type d | xargs`
{code}
(there are some false positives, e.g. TestIPv6NIOServerSocketChannel)
We should clean these up to use [JUnit's expected arg to the Test
annotation|http://junit.org/javadoc/4.11/org/junit/Test.html#expected()].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12252) IntegrationTestBulkLoad fails with illegal partition error
Dima Spivak created HBASE-12252:
---
Summary: IntegrationTestBulkLoad fails with illegal partition error
Key: HBASE-12252
URL: https://issues.apache.org/jira/browse/HBASE-12252
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.98.6
Reporter: Dima Spivak
Assignee: Dima Spivak
While running {{IntegrationTestBulkLoad}} for a while, I encountered the
following error:
{code}
14/10/14 05:57:12 INFO mapreduce.IntegrationTestBulkLoad: Running check
14/10/14 05:57:12 INFO client.RMProxy: Connecting to ResourceManager at
node-1.internal/10.0.0.3:8032
14/10/14 05:57:16 INFO util.RegionSizeCalculator: Calculating region sizes for
table IntegrationTestBulkLoad.
14/10/14 05:57:16 INFO mapreduce.JobSubmitter: number of splits:16
14/10/14 05:57:17 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1413261370136_0012
14/10/14 05:57:17 INFO impl.YarnClientImpl: Submitted application
application_1413261370136_0012
14/10/14 05:57:17 INFO mapreduce.Job: The url to track the job:
http://node-1.internal:8088/proxy/application_1413261370136_0012/
14/10/14 05:57:17 INFO mapreduce.Job: Running job: job_1413261370136_0012
14/10/14 05:57:32 INFO mapreduce.Job: Job job_1413261370136_0012 running in
uber mode : false
14/10/14 05:57:32 INFO mapreduce.Job: map 0% reduce 0%
14/10/14 05:57:52 INFO mapreduce.Job: map 6% reduce 0%
14/10/14 05:57:53 INFO mapreduce.Job: Task Id :
attempt_1413261370136_0012_m_13_0, Status : FAILED
Error: java.io.IOException: Illegal partition for
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkKey@133c4539 (-3)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1079)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:712)
at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkedListCheckingMapper.map(IntegrationTestBulkLoad.java:502)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkedListCheckingMapper.map(IntegrationTestBulkLoad.java:491)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
{code}
I'm pretty sure I've tracked this down to a partition generator that can return
a negative number; I'll post a fix once I've confirmed locally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Work started] (HBASE-12252) IntegrationTestBulkLoad fails with illegal partition error
[
https://issues.apache.org/jira/browse/HBASE-12252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-12252 started by Dima Spivak.
---
IntegrationTestBulkLoad fails with illegal partition error
--
Key: HBASE-12252
URL: https://issues.apache.org/jira/browse/HBASE-12252
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.98.6
Reporter: Dima Spivak
Assignee: Dima Spivak
While running {{IntegrationTestBulkLoad}} for a while, I encountered the
following error:
{code}
14/10/14 05:57:12 INFO mapreduce.IntegrationTestBulkLoad: Running check
14/10/14 05:57:12 INFO client.RMProxy: Connecting to ResourceManager at
node-1.internal/10.0.0.3:8032
14/10/14 05:57:16 INFO util.RegionSizeCalculator: Calculating region sizes
for table IntegrationTestBulkLoad.
14/10/14 05:57:16 INFO mapreduce.JobSubmitter: number of splits:16
14/10/14 05:57:17 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1413261370136_0012
14/10/14 05:57:17 INFO impl.YarnClientImpl: Submitted application
application_1413261370136_0012
14/10/14 05:57:17 INFO mapreduce.Job: The url to track the job:
http://node-1.internal:8088/proxy/application_1413261370136_0012/
14/10/14 05:57:17 INFO mapreduce.Job: Running job: job_1413261370136_0012
14/10/14 05:57:32 INFO mapreduce.Job: Job job_1413261370136_0012 running in
uber mode : false
14/10/14 05:57:32 INFO mapreduce.Job: map 0% reduce 0%
14/10/14 05:57:52 INFO mapreduce.Job: map 6% reduce 0%
14/10/14 05:57:53 INFO mapreduce.Job: Task Id :
attempt_1413261370136_0012_m_13_0, Status : FAILED
Error: java.io.IOException: Illegal partition for
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkKey@133c4539
(-3)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1079)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:712)
at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkedListCheckingMapper.map(IntegrationTestBulkLoad.java:502)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkedListCheckingMapper.map(IntegrationTestBulkLoad.java:491)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
{code}
I'm pretty sure I've tracked this down to a partition generator that can
return a negative number; I'll post a fix once I've confirmed locally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-12132) mvn install -Dtest=testclass also runs the hbase-it tests.
[
https://issues.apache.org/jira/browse/HBASE-12132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak resolved HBASE-12132.
-
Resolution: Not a Problem
Assignee: Dima Spivak
Hi Manukranth,
This is actually the designed behavior of Maven. If you run {{mvn install}}, it
will run all parts of the Maven lifecycle up to and including the phase
specified, which in this case means it will also go through the
integration-test phase. If you'd like to prevent IT tests from being run, add
{{-DskipITs}} to the end of your existing command.
mvn install -Dtest=testclass also runs the hbase-it tests.
-
Key: HBASE-12132
URL: https://issues.apache.org/jira/browse/HBASE-12132
Project: HBase
Issue Type: Bug
Reporter: Manukranth Kolloju
Assignee: Dima Spivak
Priority: Minor
Attachments: mvninstall.txt
mvn install -Dtest=testclass runs tests in it and they time out. This Jira
is intended to fix that behavior.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12252) IntegrationTestBulkLoad fails with illegal partition error
[
https://issues.apache.org/jira/browse/HBASE-12252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak updated HBASE-12252:
Attachment: HBASE-12252.patch
Simple fix; ran IntegrationTestBulkLoad successfully for 40 minutes after
applying.
IntegrationTestBulkLoad fails with illegal partition error
--
Key: HBASE-12252
URL: https://issues.apache.org/jira/browse/HBASE-12252
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.98.6
Reporter: Dima Spivak
Assignee: Dima Spivak
Attachments: HBASE-12252.patch
While running {{IntegrationTestBulkLoad}} for a while, I encountered the
following error:
{code}
14/10/14 05:57:12 INFO mapreduce.IntegrationTestBulkLoad: Running check
14/10/14 05:57:12 INFO client.RMProxy: Connecting to ResourceManager at
node-1.internal/10.0.0.3:8032
14/10/14 05:57:16 INFO util.RegionSizeCalculator: Calculating region sizes
for table IntegrationTestBulkLoad.
14/10/14 05:57:16 INFO mapreduce.JobSubmitter: number of splits:16
14/10/14 05:57:17 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1413261370136_0012
14/10/14 05:57:17 INFO impl.YarnClientImpl: Submitted application
application_1413261370136_0012
14/10/14 05:57:17 INFO mapreduce.Job: The url to track the job:
http://node-1.internal:8088/proxy/application_1413261370136_0012/
14/10/14 05:57:17 INFO mapreduce.Job: Running job: job_1413261370136_0012
14/10/14 05:57:32 INFO mapreduce.Job: Job job_1413261370136_0012 running in
uber mode : false
14/10/14 05:57:32 INFO mapreduce.Job: map 0% reduce 0%
14/10/14 05:57:52 INFO mapreduce.Job: map 6% reduce 0%
14/10/14 05:57:53 INFO mapreduce.Job: Task Id :
attempt_1413261370136_0012_m_13_0, Status : FAILED
Error: java.io.IOException: Illegal partition for
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkKey@133c4539
(-3)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1079)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:712)
at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkedListCheckingMapper.map(IntegrationTestBulkLoad.java:502)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkedListCheckingMapper.map(IntegrationTestBulkLoad.java:491)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
{code}
I'm pretty sure I've tracked this down to a partition generator that can
return a negative number; I'll post a fix once I've confirmed locally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12252) IntegrationTestBulkLoad fails with illegal partition error
[
https://issues.apache.org/jira/browse/HBASE-12252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak updated HBASE-12252:
Status: Patch Available (was: In Progress)
IntegrationTestBulkLoad fails with illegal partition error
--
Key: HBASE-12252
URL: https://issues.apache.org/jira/browse/HBASE-12252
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.98.6
Reporter: Dima Spivak
Assignee: Dima Spivak
Attachments: HBASE-12252.patch
While running {{IntegrationTestBulkLoad}} for a while, I encountered the
following error:
{code}
14/10/14 05:57:12 INFO mapreduce.IntegrationTestBulkLoad: Running check
14/10/14 05:57:12 INFO client.RMProxy: Connecting to ResourceManager at
node-1.internal/10.0.0.3:8032
14/10/14 05:57:16 INFO util.RegionSizeCalculator: Calculating region sizes
for table IntegrationTestBulkLoad.
14/10/14 05:57:16 INFO mapreduce.JobSubmitter: number of splits:16
14/10/14 05:57:17 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1413261370136_0012
14/10/14 05:57:17 INFO impl.YarnClientImpl: Submitted application
application_1413261370136_0012
14/10/14 05:57:17 INFO mapreduce.Job: The url to track the job:
http://node-1.internal:8088/proxy/application_1413261370136_0012/
14/10/14 05:57:17 INFO mapreduce.Job: Running job: job_1413261370136_0012
14/10/14 05:57:32 INFO mapreduce.Job: Job job_1413261370136_0012 running in
uber mode : false
14/10/14 05:57:32 INFO mapreduce.Job: map 0% reduce 0%
14/10/14 05:57:52 INFO mapreduce.Job: map 6% reduce 0%
14/10/14 05:57:53 INFO mapreduce.Job: Task Id :
attempt_1413261370136_0012_m_13_0, Status : FAILED
Error: java.io.IOException: Illegal partition for
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkKey@133c4539
(-3)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1079)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:712)
at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkedListCheckingMapper.map(IntegrationTestBulkLoad.java:502)
at
org.apache.hadoop.hbase.mapreduce.IntegrationTestBulkLoad$LinkedListCheckingMapper.map(IntegrationTestBulkLoad.java:491)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
{code}
I'm pretty sure I've tracked this down to a partition generator that can
return a negative number; I'll post a fix once I've confirmed locally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-12229: Attachment: HBASE-12229_v2.patch Addressed Matteo's suggestion of putting the waitUntilAllRegionsOnline call in HBaseTestingUtility#createTable. In this case, that meant adding another overloaded method and, in the process, refactoring an existing one that took a byte[] tableName to simply invoke my new TableName tableName variant. Finally, I did a bit of tidying of SnapshotTestingUtils (clean up stale imports and add a waitFor around a call to #isTableAvailable). NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.0.0, 2.0.0, 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: HBASE-12229.patch, HBASE-12229_v1.patch, HBASE-12229_v2.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-12229: Status: Open (was: Patch Available) NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.7, 1.0.0, 2.0.0 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: HBASE-12229.patch, HBASE-12229_v1.patch, HBASE-12229_v2.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-12229: Affects Version/s: (was: 2.0.0) (was: 1.0.0) Status: Patch Available (was: Open) Oops - looks like Hadoop QA tried to apply my 0.98 patch to master. Let's try again... NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: HBASE-12229.patch, HBASE-12229_v1.patch, HBASE-12229_v2.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12262) Make it possible to run hbase-it against a distributed cluster using Maven
Dima Spivak created HBASE-12262:
---
Summary: Make it possible to run hbase-it against a distributed
cluster using Maven
Key: HBASE-12262
URL: https://issues.apache.org/jira/browse/HBASE-12262
Project: HBase
Issue Type: Task
Reporter: Dima Spivak
Assignee: Dima Spivak
hbase-it is pretty awesome, especially when run against a distributed cluster.
As I've been trying to develop new tests to add to the module, I got sick of
having to run {{mvn package}} and moving JARs to my cluster and wanted to run
the tests directly using {{mvn verify}}. Unfortunately, I can't seem to do so;
this JIRA is to try to fix that.
[~enis] and [~ndimiduk], I've seen that you guys are behind a lot of these
tests. Was it previously possible to do what I'm trying to do and it just broke
at some point, or is this new functionality that I'll need to figure out?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12262) Make it possible to run hbase-it against a distributed cluster using Maven
[
https://issues.apache.org/jira/browse/HBASE-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171933#comment-14171933
]
Dima Spivak commented on HBASE-12262:
-
Awesome, thanks for the background, [~enis]. My initial thought is to have a
profile that could be invoked along with relevant system properties under
hbase-it so that the default hbase-site.xml (which normally comes from
hbase-server) is skipped and one without the minicluster settings is used
instead. So, under /hbase/hbase-it/, whereas {{mvn verify}} would start a
minicluster, something like {{mvn verify -P distributedCluster
-Dhbase.zookeeper.quorum=node-1,node-2
-Dhbase.root.dir=hdfs://node-1:8020/hbase}} would point Maven at a real
cluster. That sound kosher to you?
Make it possible to run hbase-it against a distributed cluster using Maven
--
Key: HBASE-12262
URL: https://issues.apache.org/jira/browse/HBASE-12262
Project: HBase
Issue Type: Task
Reporter: Dima Spivak
Assignee: Dima Spivak
hbase-it is pretty awesome, especially when run against a distributed
cluster. As I've been trying to develop new tests to add to the module, I got
sick of having to run {{mvn package}} and moving JARs to my cluster and
wanted to run the tests directly using {{mvn verify}}. Unfortunately, I can't
seem to do so; this JIRA is to try to fix that.
[~enis] and [~ndimiduk], I've seen that you guys are behind a lot of these
tests. Was it previously possible to do what I'm trying to do and it just
broke at some point, or is this new functionality that I'll need to figure
out?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-12229: Attachment: HBASE-12229_v2.patch Reuploading the same file to try to get Hadoop QA's attention... NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: HBASE-12229.patch, HBASE-12229_v1.patch, HBASE-12229_v2.patch, HBASE-12229_v2.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12278) Race condition in TestSecureLoadIncrementalHFilesSplitRecovery
Dima Spivak created HBASE-12278: --- Summary: Race condition in TestSecureLoadIncrementalHFilesSplitRecovery Key: HBASE-12278 URL: https://issues.apache.org/jira/browse/HBASE-12278 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.0.0 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor I sometimes get #testGroupOrSplitWhenRegionHoleExistsInMeta to error out on me because of a TableNotFoundException. Fix is pretty simple and will involve a little bit of test refactoring to get rid of some duplicate behavior that's done better in HBaseTestingUtility anyway. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-12229: Attachment: HBASE-12229_master_v1.patch Hadoop QA doesn't seem to like my 0.98 patch. Maybe one for master will be more to its liking? NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: HBASE-12229.patch, HBASE-12229_master_v1.patch, HBASE-12229_v1.patch, HBASE-12229_v2.patch, HBASE-12229_v2.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12229) NullPointerException in SnapshotTestingUtils
[ https://issues.apache.org/jira/browse/HBASE-12229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-12229: Attachment: HBASE-12229_master_v2.patch Legit test failure caused by me being dumb. :) Let's try again... NullPointerException in SnapshotTestingUtils Key: HBASE-12229 URL: https://issues.apache.org/jira/browse/HBASE-12229 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.98.7 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: HBASE-12229.patch, HBASE-12229_master_v1.patch, HBASE-12229_master_v1.patch, HBASE-12229_master_v2.patch, HBASE-12229_v1.patch, HBASE-12229_v2.patch, HBASE-12229_v2.patch I tracked down occasional flakiness in TestRestoreSnapshotFromClient to a potential NPE in SnapshotTestingUtils#waitForTableToBeOnline. In short, some tests in TestRestoreSnapshot... create a table and then invoke SnapshotTestingUtils#waitForTableToBeOnline, but this method assumes that regions have been assigned by the time it's invoked (which is not always the case). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-12284) Make TestMasterOperationsForRegionReplicas less flaky
[ https://issues.apache.org/jira/browse/HBASE-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak reassigned HBASE-12284: --- Assignee: Dima Spivak I'd be happy to take a look [~manukranthk]. Do you have example output from a failure you've observed? I can't seem to find any in the b.a.o. TRUNK job. Make TestMasterOperationsForRegionReplicas less flaky - Key: HBASE-12284 URL: https://issues.apache.org/jira/browse/HBASE-12284 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: Manukranth Kolloju Assignee: Dima Spivak Priority: Minor Fix For: 2.0.0 This test seems to be failing occassionaly, like once in 30 times. Maybe the test needs some love ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12285) Builds are failing, possibly because of SUREFIRE-1091
Dima Spivak created HBASE-12285: --- Summary: Builds are failing, possibly because of SUREFIRE-1091 Key: HBASE-12285 URL: https://issues.apache.org/jira/browse/HBASE-12285 Project: HBase Issue Type: Bug Affects Versions: 1.0.0 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Blocker Our branch-1 builds on builds.apache.org have been failing in recent days after we switched over to an official version of Surefire a few days back (HBASE-4955). The version we're using, 2.17, is hit by a bug ([SUREFIRE-1091|https://jira.codehaus.org/browse/SUREFIRE-1091]) that results in an IOException, which looks like what we're seeing on Jenkins. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-12285) Builds are failing, possibly because of SUREFIRE-1091
[ https://issues.apache.org/jira/browse/HBASE-12285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-12285 started by Dima Spivak. --- Builds are failing, possibly because of SUREFIRE-1091 - Key: HBASE-12285 URL: https://issues.apache.org/jira/browse/HBASE-12285 Project: HBase Issue Type: Bug Affects Versions: 1.0.0 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Blocker Our branch-1 builds on builds.apache.org have been failing in recent days after we switched over to an official version of Surefire a few days back (HBASE-4955). The version we're using, 2.17, is hit by a bug ([SUREFIRE-1091|https://jira.codehaus.org/browse/SUREFIRE-1091]) that results in an IOException, which looks like what we're seeing on Jenkins. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12285) Builds are failing, possibly because of SUREFIRE-1091
[ https://issues.apache.org/jira/browse/HBASE-12285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175474#comment-14175474 ] Dima Spivak commented on HBASE-12285: - Looks like there isn't yet a 2.18 version of surefire-junit47. Since this only seems to be hitting branch-1, is it worth just reverting HBASE-4955 for now, [~eclark]? Builds are failing, possibly because of SUREFIRE-1091 - Key: HBASE-12285 URL: https://issues.apache.org/jira/browse/HBASE-12285 Project: HBase Issue Type: Bug Affects Versions: 1.0.0 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Blocker Our branch-1 builds on builds.apache.org have been failing in recent days after we switched over to an official version of Surefire a few days back (HBASE-4955). The version we're using, 2.17, is hit by a bug ([SUREFIRE-1091|https://jira.codehaus.org/browse/SUREFIRE-1091]) that results in an IOException, which looks like what we're seeing on Jenkins. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12278) Race condition in TestSecureLoadIncrementalHFilesSplitRecovery
[ https://issues.apache.org/jira/browse/HBASE-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175805#comment-14175805 ] Dima Spivak commented on HBASE-12278: - Ted, I'm actually waiting for HBASE-12229 to get committed, because that adds a HBTU#createTable method that fits this case. Wanna take a look there? Race condition in TestSecureLoadIncrementalHFilesSplitRecovery -- Key: HBASE-12278 URL: https://issues.apache.org/jira/browse/HBASE-12278 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.0.0 Reporter: Dima Spivak Assignee: Dima Spivak Priority: Minor Attachments: 12278-v1.txt I sometimes get #testGroupOrSplitWhenRegionHoleExistsInMeta to error out on me because of a TableNotFoundException. Fix is pretty simple and will involve a little bit of test refactoring to get rid of some duplicate behavior that's done better in HBaseTestingUtility anyway. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
