[jira] [Commented] (HBASE-6785) Convert AggregateProtocol to protobuf defined coprocessor service
[ https://issues.apache.org/jira/browse/HBASE-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472163#comment-13472163 ] Devaraj Das commented on HBASE-6785: https://reviews.apache.org/r/7495/ is the new RB. Please ignore the previous one. Thanks, Ted for looking over the patch. I have simplified the PB definition to have only one message (all the use cases are covered by this one message). Convert AggregateProtocol to protobuf defined coprocessor service - Key: HBASE-6785 URL: https://issues.apache.org/jira/browse/HBASE-6785 Project: HBase Issue Type: Sub-task Components: Coprocessors Reporter: Gary Helmling Assignee: Devaraj Das Fix For: 0.96.0 Attachments: Aggregate.proto, Aggregate.proto With coprocessor endpoints now exposed as protobuf defined services, we should convert over all of our built-in endpoints to PB services. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6785) Convert AggregateProtocol to protobuf defined coprocessor service
[ https://issues.apache.org/jira/browse/HBASE-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472166#comment-13472166 ] Devaraj Das commented on HBASE-6785: Just to reiterate, the patch could do with some cleanup and maybe javadoc fixes here and there. I have posted it here to get a feedback on the ColumnInterpreter interface additions, and the PB definition.. I'll clean up the patch as a follow up. Thanks! Convert AggregateProtocol to protobuf defined coprocessor service - Key: HBASE-6785 URL: https://issues.apache.org/jira/browse/HBASE-6785 Project: HBase Issue Type: Sub-task Components: Coprocessors Reporter: Gary Helmling Assignee: Devaraj Das Fix For: 0.96.0 Attachments: Aggregate.proto, Aggregate.proto With coprocessor endpoints now exposed as protobuf defined services, we should convert over all of our built-in endpoints to PB services. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[
https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472172#comment-13472172
]
Elliott Clark commented on HBASE-6961:
--
This is only for in place runs by developers, when trying to run bin/hbase out
of the source repository. This patch only changes that. Nothing about the
normal start up classpath is changed. Right now on local dev builds maven is
already required. The only thing about the maven start up that's changing is
if the dev hasn't ever done a package/compile the start up will take longer
because it runs a package along with dependency:build-classpath. On trunk when
no test jars are available only the first module runs before maven errors out
so most jars aren't in the class path file.
Currently in trunk the order of precedence is:
# Things in your mavan repo
# Main Classes locally compiled
# Test Classes locally compiled
{code}
[trunk ✓] $ bin/hbase classpath
[jira] [Updated] (HBASE-5498) Secure Bulk Load
[ https://issues.apache.org/jira/browse/HBASE-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-5498: --- Attachment: HBASE-5498_trunk_2.patch Updated trunk patches incorporating review comments. Except for the following: -added coprocessor hooks for the new SecureBulkLoadCoprocessors (I'd like to address this as part of the coprocessor refactor for 0.96) -SecureTestUtil continues to load SecureBulkLoad by default, this is better for consistency and overall testing -Support for federation Secure Bulk Load Key: HBASE-5498 URL: https://issues.apache.org/jira/browse/HBASE-5498 Project: HBase Issue Type: Improvement Components: security Reporter: Francis Liu Assignee: Francis Liu Fix For: 0.94.3, 0.96.0 Attachments: HBASE-5498_94.patch, HBASE-5498_94.patch, HBASE-5498_draft_94.patch, HBASE-5498_draft.patch, HBASE-5498_trunk_2.patch, HBASE-5498_trunk.patch Design doc: https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Secure+Bulk+Load Short summary: Security as it stands does not cover the bulkLoadHFiles() feature. Users calling this method will bypass ACLs. Also loading is made more cumbersome in a secure setting because of hdfs privileges. bulkLoadHFiles() moves the data from user's directory to the hbase directory, which would require certain write access privileges set. Our solution is to create a coprocessor which makes use of AuthManager to verify if a user has write access to the table. If so, launches a MR job as the hbase user to do the importing (ie rewrite from text to hfiles). One tricky part this job will have to do is impersonate the calling user when reading the input files. We can do this by expecting the user to pass an hdfs delegation token as part of the secureBulkLoad() coprocessor call and extend an inputformat to make use of that token. The output is written to a temporary directory accessible only by hbase and then bulkloadHFiles() is called. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5498) Secure Bulk Load
[ https://issues.apache.org/jira/browse/HBASE-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-5498: --- Attachment: HBASE-5498_94_2.patch attached 0.94 addressing the same issues. Secure Bulk Load Key: HBASE-5498 URL: https://issues.apache.org/jira/browse/HBASE-5498 Project: HBase Issue Type: Improvement Components: security Reporter: Francis Liu Assignee: Francis Liu Fix For: 0.94.3, 0.96.0 Attachments: HBASE-5498_94_2.patch, HBASE-5498_94.patch, HBASE-5498_94.patch, HBASE-5498_draft_94.patch, HBASE-5498_draft.patch, HBASE-5498_trunk_2.patch, HBASE-5498_trunk.patch Design doc: https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Secure+Bulk+Load Short summary: Security as it stands does not cover the bulkLoadHFiles() feature. Users calling this method will bypass ACLs. Also loading is made more cumbersome in a secure setting because of hdfs privileges. bulkLoadHFiles() moves the data from user's directory to the hbase directory, which would require certain write access privileges set. Our solution is to create a coprocessor which makes use of AuthManager to verify if a user has write access to the table. If so, launches a MR job as the hbase user to do the importing (ie rewrite from text to hfiles). One tricky part this job will have to do is impersonate the calling user when reading the input files. We can do this by expecting the user to pass an hdfs delegation token as part of the secureBulkLoad() coprocessor call and extend an inputformat to make use of that token. The output is written to a temporary directory accessible only by hbase and then bulkloadHFiles() is called. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5498) Secure Bulk Load
[
https://issues.apache.org/jira/browse/HBASE-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472208#comment-13472208
]
Hadoop QA commented on HBASE-5498:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548376/HBASE-5498_94_2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 19 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3021//console
This message is automatically generated.
Secure Bulk Load
Key: HBASE-5498
URL: https://issues.apache.org/jira/browse/HBASE-5498
Project: HBase
Issue Type: Improvement
Components: security
Reporter: Francis Liu
Assignee: Francis Liu
Fix For: 0.94.3, 0.96.0
Attachments: HBASE-5498_94_2.patch, HBASE-5498_94.patch,
HBASE-5498_94.patch, HBASE-5498_draft_94.patch, HBASE-5498_draft.patch,
HBASE-5498_trunk_2.patch, HBASE-5498_trunk.patch
Design doc:
https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Secure+Bulk+Load
Short summary:
Security as it stands does not cover the bulkLoadHFiles() feature. Users
calling this method will bypass ACLs. Also loading is made more cumbersome in
a secure setting because of hdfs privileges. bulkLoadHFiles() moves the data
from user's directory to the hbase directory, which would require certain
write access privileges set.
Our solution is to create a coprocessor which makes use of AuthManager to
verify if a user has write access to the table. If so, launches a MR job as
the hbase user to do the importing (ie rewrite from text to hfiles). One
tricky part this job will have to do is impersonate the calling user when
reading the input files. We can do this by expecting the user to pass an hdfs
delegation token as part of the secureBulkLoad() coprocessor call and extend
an inputformat to make use of that token. The output is written to a
temporary directory accessible only by hbase and then bulkloadHFiles() is
called.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5498) Secure Bulk Load
[
https://issues.apache.org/jira/browse/HBASE-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472217#comment-13472217
]
Hadoop QA commented on HBASE-5498:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548374/HBASE-5498_trunk_2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 16 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
82 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3020//console
This message is automatically generated.
Secure Bulk Load
Key: HBASE-5498
URL: https://issues.apache.org/jira/browse/HBASE-5498
Project: HBase
Issue Type: Improvement
Components: security
Reporter: Francis Liu
Assignee: Francis Liu
Fix For: 0.94.3, 0.96.0
Attachments: HBASE-5498_94_2.patch, HBASE-5498_94.patch,
HBASE-5498_94.patch, HBASE-5498_draft_94.patch, HBASE-5498_draft.patch,
HBASE-5498_trunk_2.patch, HBASE-5498_trunk.patch
Design doc:
https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Secure+Bulk+Load
Short summary:
Security as it stands does not cover the bulkLoadHFiles() feature. Users
calling this method will bypass ACLs. Also loading is made more cumbersome in
a secure setting because of hdfs privileges. bulkLoadHFiles() moves the data
from user's directory to the hbase directory, which would require certain
write access privileges set.
Our solution is to create a coprocessor which makes use of AuthManager to
verify if a user has write access to the table. If so, launches a MR job as
the hbase user to do the importing (ie rewrite from text to hfiles). One
tricky part this job will have to do is impersonate the calling user when
reading the input files. We can do this by expecting the user to pass an hdfs
delegation token as part of the secureBulkLoad() coprocessor call and extend
an inputformat to make use of that token. The output is written to a
temporary directory accessible only by hbase and then bulkloadHFiles() is
called.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-643) Rename tables
[ https://issues.apache.org/jira/browse/HBASE-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472387#comment-13472387 ] Devin Bayer commented on HBASE-643: --- Hi. Is it safe to use the rename_table.rb script attached to this bug? Which versions does it work with? Rename tables - Key: HBASE-643 URL: https://issues.apache.org/jira/browse/HBASE-643 Project: HBase Issue Type: New Feature Reporter: Michael Bieniosek Attachments: copy_table.rb, rename_table.rb It would be nice to be able to rename tables, if this is possible. Some of our internal users are doing things like: upload table mytable - realize they screwed up - upload table mytable_2 - decide mytable_2 looks better - have to go on using mytable_2 instead of originally desired table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6785) Convert AggregateProtocol to protobuf defined coprocessor service
[ https://issues.apache.org/jira/browse/HBASE-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472406#comment-13472406 ] Ted Yu commented on HBASE-6785: --- Minor comment: 6758-simplified-pb.patch should have been named 6785-simplified-pb.patch (JIRA number was wrong) review comments to follow. Convert AggregateProtocol to protobuf defined coprocessor service - Key: HBASE-6785 URL: https://issues.apache.org/jira/browse/HBASE-6785 Project: HBase Issue Type: Sub-task Components: Coprocessors Reporter: Gary Helmling Assignee: Devaraj Das Fix For: 0.96.0 Attachments: Aggregate.proto, Aggregate.proto With coprocessor endpoints now exposed as protobuf defined services, we should convert over all of our built-in endpoints to PB services. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6962) Upgrade to hadoop 1.1
Ted Yu created HBASE-6962: - Summary: Upgrade to hadoop 1.1 Key: HBASE-6962 URL: https://issues.apache.org/jira/browse/HBASE-6962 Project: HBase Issue Type: Bug Environment: hadoop 1.1 contains multiple important fixes, including HDFS-3703 Reporter: Ted Yu -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6962) Upgrade hadoop 1 dependency to hadoop 1.1
[ https://issues.apache.org/jira/browse/HBASE-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6962: -- Summary: Upgrade hadoop 1 dependency to hadoop 1.1 (was: Upgrade to hadoop 1.1) Upgrade hadoop 1 dependency to hadoop 1.1 - Key: HBASE-6962 URL: https://issues.apache.org/jira/browse/HBASE-6962 Project: HBase Issue Type: Bug Environment: hadoop 1.1 contains multiple important fixes, including HDFS-3703 Reporter: Ted Yu -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-643) Rename tables
[ https://issues.apache.org/jira/browse/HBASE-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472458#comment-13472458 ] stack commented on HBASE-643: - @Devin No. What is attached here has rotted. It would need work to update it. Even then, the script was a band-aid. What is needed is our keeping an id rather than the table name everywhere so a rename requires our changing the tablename once in a table to id map rather than in a few places in the filesystem as well as in the metadata kept by each table region. Rename tables - Key: HBASE-643 URL: https://issues.apache.org/jira/browse/HBASE-643 Project: HBase Issue Type: New Feature Reporter: Michael Bieniosek Attachments: copy_table.rb, rename_table.rb It would be nice to be able to rename tables, if this is possible. Some of our internal users are doing things like: upload table mytable - realize they screwed up - upload table mytable_2 - decide mytable_2 looks better - have to go on using mytable_2 instead of originally desired table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6942) Endpoint implementation for bulk delete
[ https://issues.apache.org/jira/browse/HBASE-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472509#comment-13472509 ] Lars Hofhansl commented on HBASE-6942: -- Patch looks good over all (but have not looked in detail). Since this is a specific use case (we're issuing family deletes always), I'd say we should only provide the EndPoint and don't add a public API into HTableInterface. Endpoint implementation for bulk delete --- Key: HBASE-6942 URL: https://issues.apache.org/jira/browse/HBASE-6942 Project: HBase Issue Type: Improvement Components: Coprocessors, Performance Reporter: Anoop Sam John Assignee: Anoop Sam John Attachments: HBASE-6942.patch We can provide an end point implementation for doing a bulk delete (based on a scan) at the server side. This can reduce the time taken for such an operation as right now it need to do a scan to client and issue delete(s) using rowkeys. Query like delete from table1 where... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6942) Endpoint implementation for bulk delete
[ https://issues.apache.org/jira/browse/HBASE-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472516#comment-13472516 ] Ted Yu commented on HBASE-6942: --- @Anoop: Can you add a test case where deletion fails in one of the regions of the table ? Once you add that test, you would discover a small flaw in determining row count. Thanks Endpoint implementation for bulk delete --- Key: HBASE-6942 URL: https://issues.apache.org/jira/browse/HBASE-6942 Project: HBase Issue Type: Improvement Components: Coprocessors, Performance Reporter: Anoop Sam John Assignee: Anoop Sam John Attachments: HBASE-6942.patch We can provide an end point implementation for doing a bulk delete (based on a scan) at the server side. This can reduce the time taken for such an operation as right now it need to do a scan to client and issue delete(s) using rowkeys. Query like delete from table1 where... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472604#comment-13472604 ] stack commented on HBASE-6961: -- On 1., it will show mvn spew everytime I run bin/start-hbase (even if just dev context)? And slow startup everytime? That seems a little obnoxious. Can we not improve the message folks get if they fail to package first? On 2., the reorder of cp, that is good (I actually misread the ordering you were trying to achieve -- pardon me). In place scripts fail if mvn install hasn't been run Key: HBASE-6961 URL: https://issues.apache.org/jira/browse/HBASE-6961 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6961-0.patch bin/hbase tries to get dependencies of the project however it fails if mvn install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6929) Publish Hbase 0.94 artifacts build against hadoop-2.0
[ https://issues.apache.org/jira/browse/HBASE-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472607#comment-13472607 ] stack commented on HBASE-6929: -- [~lhofhansl] If you do mvn package, it will install the build into your local mvn repository (see under ~/.m2/repository). Notice how the suffix is -SNAPSHOT. The suggestion is to push one of these snapshots up to apache (Apache maintains an area where you can publish mvn snapshots; see https://repository.apache.org/content/groups/snapshots/org/apache/hbase/hbase/). Publish Hbase 0.94 artifacts build against hadoop-2.0 - Key: HBASE-6929 URL: https://issues.apache.org/jira/browse/HBASE-6929 Project: HBase Issue Type: Task Components: build Affects Versions: 0.94.2 Reporter: Enis Soztutar Downstream projects (flume, hive, pig, etc) depends on hbase, but since the hbase binaries build with hadoop-2.0 are not pushed to maven, they cannot depend on them. AFAIK, hadoop 1 and 2 are not binary compatible, so we should also push hbase jars build with hadoop2.0 profile into maven, possibly with version string like 0.94.2-hadoop2.0. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472644#comment-13472644 ] Elliott Clark commented on HBASE-6961: -- No maven spew. It's all piped to /dev/null. The computation is cached in target/cached_classpath.txt as long as that file remains this won't get re-computed. So a mvn clean or a rm target/cached_classpath.txt will force a re-run of mvn package -DskipTests dependency:build-classpath but other than that the start up shouldn't change. In place scripts fail if mvn install hasn't been run Key: HBASE-6961 URL: https://issues.apache.org/jira/browse/HBASE-6961 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6961-0.patch bin/hbase tries to get dependencies of the project however it fails if mvn install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6929) Publish Hbase 0.94 artifacts build against hadoop-2.0
[
https://issues.apache.org/jira/browse/HBASE-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472645#comment-13472645
]
stack commented on HBASE-6929:
--
You know how to fix this [~enis]?
{code}
Here is what I ran:
stack:0.94 stack$ mvn -DskipTests -Dhadoop.profile=2.0 -Dclassifier=hadoop2
install
...and I got this:
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 1:12.527s
[INFO] Finished at: Tue Oct 09 12:11:41 PDT 2012
[INFO] Final Memory: 28M/81M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-dependency-plugin:2.1:build-classpath
(create-mrapp-generated-classpath) on project hbase: not found in any
repository: aopalliance:aopalliance:java-source:hadoop2:1.0: Could not find
artifact aopalliance:aopalliance:jar:hadoop2:1.0 in apache release
(https://repository.apache.org/content/repositories/releases/)
[ERROR]
[ERROR] Try downloading the file manually from the project website.
[ERROR]
[ERROR] Then, install it using the command:
[ERROR] mvn install:install-file -DgroupId=aopalliance -DartifactId=aopalliance
-Dversion=1.0 -Dclassifier=hadoop2 -Dpackaging=java-source -Dfile=/path/to/file
[ERROR]
[ERROR] Alternatively, if you host your own repository you can deploy the file
there:
[ERROR] mvn deploy:deploy-file -DgroupId=aopalliance -DartifactId=aopalliance
-Dversion=1.0 -Dclassifier=hadoop2 -Dpackaging=java-source -Dfile=/path/to/file
-Durl=[url] -DrepositoryId=[id]
[ERROR]
[ERROR]
[ERROR] aopalliance:aopalliance:java-source:1.0
[ERROR]
[ERROR] from the specified remote repositories:
[ERROR] apache release
(https://repository.apache.org/content/repositories/releases/, releases=true,
snapshots=true),
[ERROR] apache non-releases (http://people.apache.org/~stack/m2/repository,
releases=true, snapshots=false),
[ERROR] java.net (http://download.java.net/maven/2/, releases=true,
snapshots=false),
[ERROR] codehaus (http://repository.codehaus.org/, releases=true,
snapshots=false),
[ERROR] repository.jboss.org
(http://repository.jboss.org/nexus/content/groups/public-jboss/, releases=true,
snapshots=false),
[ERROR] ghelmling.testing (http://people.apache.org/~garyh/mvn/, releases=true,
snapshots=true),
[ERROR] apache.snapshots (http://repository.apache.org/snapshots,
releases=false, snapshots=true),
[ERROR] central (http://repo1.maven.org/maven2, releases=true, snapshots=false)
[ERROR] - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
{code}
Publish Hbase 0.94 artifacts build against hadoop-2.0
-
Key: HBASE-6929
URL: https://issues.apache.org/jira/browse/HBASE-6929
Project: HBase
Issue Type: Task
Components: build
Affects Versions: 0.94.2
Reporter: Enis Soztutar
Downstream projects (flume, hive, pig, etc) depends on hbase, but since the
hbase binaries build with hadoop-2.0 are not pushed to maven, they cannot
depend on them. AFAIK, hadoop 1 and 2 are not binary compatible, so we should
also push hbase jars build with hadoop2.0 profile into maven, possibly with
version string like 0.94.2-hadoop2.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6963) unable to run hbck on a secure cluster
Jimmy Xiang created HBASE-6963:
--
Summary: unable to run hbck on a secure cluster
Key: HBASE-6963
URL: https://issues.apache.org/jira/browse/HBASE-6963
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
{noformat}
12/10/02 11:49:07 WARN util.HBaseFsck: Got AccessControlException when
preCheckPermission
org.apache.hadoop.security.AccessControlException: Permission denied:
action=WRITE path=hdfs://...:8020/hbase/-ROOT- user=hbase/...
at org.apache.hadoop.hbase.util.FSUtils.checkAccess(FSUtils.java:882)
at
org.apache.hadoop.hbase.util.HBaseFsck.preCheckPermission(HBaseFsck.java:1230)
at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3343)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3205)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6963) unable to run hbck on a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-6963:
---
Status: Patch Available (was: Open)
unable to run hbck on a secure cluster
--
Key: HBASE-6963
URL: https://issues.apache.org/jira/browse/HBASE-6963
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: trunk_6963.patch
{noformat}
12/10/02 11:49:07 WARN util.HBaseFsck: Got AccessControlException when
preCheckPermission
org.apache.hadoop.security.AccessControlException: Permission denied:
action=WRITE path=hdfs://...:8020/hbase/-ROOT- user=hbase/...
at org.apache.hadoop.hbase.util.FSUtils.checkAccess(FSUtils.java:882)
at
org.apache.hadoop.hbase.util.HBaseFsck.preCheckPermission(HBaseFsck.java:1230)
at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3343)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3205)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6963) unable to run hbck on a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-6963:
---
Attachment: trunk_6963.patch
unable to run hbck on a secure cluster
--
Key: HBASE-6963
URL: https://issues.apache.org/jira/browse/HBASE-6963
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: trunk_6963.patch
{noformat}
12/10/02 11:49:07 WARN util.HBaseFsck: Got AccessControlException when
preCheckPermission
org.apache.hadoop.security.AccessControlException: Permission denied:
action=WRITE path=hdfs://...:8020/hbase/-ROOT- user=hbase/...
at org.apache.hadoop.hbase.util.FSUtils.checkAccess(FSUtils.java:882)
at
org.apache.hadoop.hbase.util.HBaseFsck.preCheckPermission(HBaseFsck.java:1230)
at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3343)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3205)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5525) Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Odell reassigned HBASE-5525: -- Assignee: Kevin Odell Truncate and preserve region boundaries option -- Key: HBASE-5525 URL: https://issues.apache.org/jira/browse/HBASE-5525 Project: HBase Issue Type: New Feature Reporter: Jean-Daniel Cryans Assignee: Kevin Odell Labels: newbie, noob A tool that would be useful for testing (and maybe in prod too) would be a truncate option to keep the current region boundaries. Right now what you have to do is completely kill the table and recreate it with the correct regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472720#comment-13472720 ] stack commented on HBASE-6961: -- +1 on 1. and 2. then; i.e. +1 on committing the attached patch. In place scripts fail if mvn install hasn't been run Key: HBASE-6961 URL: https://issues.apache.org/jira/browse/HBASE-6961 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6961-0.patch bin/hbase tries to get dependencies of the project however it fails if mvn install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6929) Publish Hbase 0.94 artifacts build against hadoop-2.0
[ https://issues.apache.org/jira/browse/HBASE-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472723#comment-13472723 ] Lars Hofhansl commented on HBASE-6929: -- We were discussing this internally, and there'd be interest for us to have 0.94 working with Hadoop-2. Is there a top-level jira that I can check out to see the magnitude of the changes needed? Publish Hbase 0.94 artifacts build against hadoop-2.0 - Key: HBASE-6929 URL: https://issues.apache.org/jira/browse/HBASE-6929 Project: HBase Issue Type: Task Components: build Affects Versions: 0.94.2 Reporter: Enis Soztutar Downstream projects (flume, hive, pig, etc) depends on hbase, but since the hbase binaries build with hadoop-2.0 are not pushed to maven, they cannot depend on them. AFAIK, hadoop 1 and 2 are not binary compatible, so we should also push hbase jars build with hadoop2.0 profile into maven, possibly with version string like 0.94.2-hadoop2.0. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6963) unable to run hbck on a secure cluster
[
https://issues.apache.org/jira/browse/HBASE-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472729#comment-13472729
]
Hadoop QA commented on HBASE-6963:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548448/trunk_6963.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestSplitTransaction
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3022//console
This message is automatically generated.
unable to run hbck on a secure cluster
--
Key: HBASE-6963
URL: https://issues.apache.org/jira/browse/HBASE-6963
Project: HBase
Issue Type: Bug
Components: hbck
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: trunk_6963.patch
{noformat}
12/10/02 11:49:07 WARN util.HBaseFsck: Got AccessControlException when
preCheckPermission
org.apache.hadoop.security.AccessControlException: Permission denied:
action=WRITE path=hdfs://...:8020/hbase/-ROOT- user=hbase/...
at org.apache.hadoop.hbase.util.FSUtils.checkAccess(FSUtils.java:882)
at
org.apache.hadoop.hbase.util.HBaseFsck.preCheckPermission(HBaseFsck.java:1230)
at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3343)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3205)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6964) Add more instrumentation for log roll
Amitanand Aiyer created HBASE-6964: -- Summary: Add more instrumentation for log roll Key: HBASE-6964 URL: https://issues.apache.org/jira/browse/HBASE-6964 Project: HBase Issue Type: Sub-task Reporter: Amitanand Aiyer Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
Kumar Ravi created HBASE-6965: - Summary: Generic MXBean Utility class to support all JDK vendors Key: HBASE-6965 URL: https://issues.apache.org/jira/browse/HBASE-6965 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.94.1 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 This issue is related to JIRA https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to propose the use of a newly created generic org.apache.hadoop.hbase.util.OSMXBean class that can be used by other classes. JIRA HBASE-6945 contains a patch for the class org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the inclusion of this new class, HBase can be built and become functional with JDKs and JREs other than what is provided by Oracle. This class uses reflection to determine the JVM vendor (Sun, IBM) and the platform (Linux or Windows), and contains other methods that return the OS properties - 1. Number of Open File descriptors; 2. Maximum number of File Descriptors. This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well as Oracle JDK 6. Junit tests (runDevTests category) completed without any failures or errors when tested on all the three JDKs.The builds and tests were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-6961: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the review Stack. Integrated into trunk on revision 1396307 In place scripts fail if mvn install hasn't been run Key: HBASE-6961 URL: https://issues.apache.org/jira/browse/HBASE-6961 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6961-0.patch bin/hbase tries to get dependencies of the project however it fails if mvn install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumar Ravi updated HBASE-6965: -- Status: Patch Available (was: Open) Here's the source file in it's entirety. Generic MXBean Utility class to support all JDK vendors --- Key: HBASE-6965 URL: https://issues.apache.org/jira/browse/HBASE-6965 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.94.1 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 This issue is related to JIRA https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to propose the use of a newly created generic org.apache.hadoop.hbase.util.OSMXBean class that can be used by other classes. JIRA HBASE-6945 contains a patch for the class org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the inclusion of this new class, HBase can be built and become functional with JDKs and JREs other than what is provided by Oracle. This class uses reflection to determine the JVM vendor (Sun, IBM) and the platform (Linux or Windows), and contains other methods that return the OS properties - 1. Number of Open File descriptors; 2. Maximum number of File Descriptors. This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well as Oracle JDK 6. Junit tests (runDevTests category) completed without any failures or errors when tested on all the three JDKs.The builds and tests were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumar Ravi updated HBASE-6965: -- Status: Open (was: Patch Available) Generic MXBean Utility class to support all JDK vendors --- Key: HBASE-6965 URL: https://issues.apache.org/jira/browse/HBASE-6965 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.94.1 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 This issue is related to JIRA https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to propose the use of a newly created generic org.apache.hadoop.hbase.util.OSMXBean class that can be used by other classes. JIRA HBASE-6945 contains a patch for the class org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the inclusion of this new class, HBase can be built and become functional with JDKs and JREs other than what is provided by Oracle. This class uses reflection to determine the JVM vendor (Sun, IBM) and the platform (Linux or Windows), and contains other methods that return the OS properties - 1. Number of Open File descriptors; 2. Maximum number of File Descriptors. This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well as Oracle JDK 6. Junit tests (runDevTests category) completed without any failures or errors when tested on all the three JDKs.The builds and tests were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumar Ravi updated HBASE-6965: -- Attachment: OSMXBean.java Here's the source file for OSMXBean.java. This would reside in the src/main/java/org/apache/hadoop/hbase/util directory. Generic MXBean Utility class to support all JDK vendors --- Key: HBASE-6965 URL: https://issues.apache.org/jira/browse/HBASE-6965 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.94.1 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 Attachments: OSMXBean.java This issue is related to JIRA https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to propose the use of a newly created generic org.apache.hadoop.hbase.util.OSMXBean class that can be used by other classes. JIRA HBASE-6945 contains a patch for the class org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the inclusion of this new class, HBase can be built and become functional with JDKs and JREs other than what is provided by Oracle. This class uses reflection to determine the JVM vendor (Sun, IBM) and the platform (Linux or Windows), and contains other methods that return the OS properties - 1. Number of Open File descriptors; 2. Maximum number of File Descriptors. This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well as Oracle JDK 6. Junit tests (runDevTests category) completed without any failures or errors when tested on all the three JDKs.The builds and tests were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-6965 started by Kumar Ravi. Generic MXBean Utility class to support all JDK vendors --- Key: HBASE-6965 URL: https://issues.apache.org/jira/browse/HBASE-6965 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.94.1 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 Attachments: OSMXBean.java This issue is related to JIRA https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to propose the use of a newly created generic org.apache.hadoop.hbase.util.OSMXBean class that can be used by other classes. JIRA HBASE-6945 contains a patch for the class org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the inclusion of this new class, HBase can be built and become functional with JDKs and JREs other than what is provided by Oracle. This class uses reflection to determine the JVM vendor (Sun, IBM) and the platform (Linux or Windows), and contains other methods that return the OS properties - 1. Number of Open File descriptors; 2. Maximum number of File Descriptors. This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well as Oracle JDK 6. Junit tests (runDevTests category) completed without any failures or errors when tested on all the three JDKs.The builds and tests were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472741#comment-13472741 ] Kumar Ravi commented on HBASE-6945: --- I have created another JIRA https://issues.apache.org/jira/browse/HBASE-6965 to be used along with the patch I plan to submit here. This new class called OSMXBean is used by the patch here. Compilation errors when using non-Sun JDKs to build HBase-0.94 -- Key: HBASE-6945 URL: https://issues.apache.org/jira/browse/HBASE-6945 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.94.1 Environment: RHEL 6.3, IBM Java 7 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 When using IBM Java 7 to build HBase-0.94.1, the following comilation error is seen. [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] error: package com.sun.management does not exist [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] error: cannot find symbol [INFO] 4 errors [INFO] - [INFO] [INFO] BUILD FAILURE [INFO] I have a patch available which should work for all JDKs including Sun. I am in the process of testing this patch. Preliminary tests indicate the build is working fine with this patch. I will post this patch when I am done testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[
https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kumar Ravi updated HBASE-6945:
--
Status: Patch Available (was: Open)
--- curr_hbase/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java
2012-10-04 12:59:14.0 -0400
+++ new_hbase/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java
2012-10-08 16:32:00.0 -0400
@@ -20,13 +20,11 @@
package org.apache.hadoop.hbase;
-import com.sun.management.UnixOperatingSystemMXBean;
+import org.apache.hadoop.hbase.util.OSMXBean;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.hbase.client.HConnectionTestingUtility;
-import java.lang.management.ManagementFactory;
-import java.lang.management.OperatingSystemMXBean;
import java.util.*;
@@ -42,42 +40,33 @@
* On unix, we know how to get the number of open file descriptor
*/
private static class ResourceAnalyzer {
-private static final OperatingSystemMXBean osStats;
-private static final UnixOperatingSystemMXBean unixOsStats;
public long getThreadsCount() {
return Thread.getAllStackTraces().size();
}
public long getOpenFileDescriptorCount() {
- if (unixOsStats == null) {
-return 0;
- } else {
-return unixOsStats.getOpenFileDescriptorCount();
- }
+ OSMXBean osMbean = new OSMXBean();
+ if (osMbean != null osMbean.getUnix() == true) {
+return osMbean.getOpenFileDescriptorCount();
+ }
+else
+return 0;
}
public long getMaxFileDescriptorCount() {
- if (unixOsStats == null) {
-return 0;
- } else {
-return unixOsStats.getMaxFileDescriptorCount();
+ OSMXBean osMbean = new OSMXBean();
+ if (osMbean != null osMbean.getUnix() == true) {
+ return osMbean.getMaxFileDescriptorCount();
}
+ else
+ return 0;
}
public long getConnectionCount(){
return HConnectionTestingUtility.getConnectionCount();
}
-static {
- osStats =
-ManagementFactory.getOperatingSystemMXBean();
- if (osStats instanceof UnixOperatingSystemMXBean) {
-unixOsStats = (UnixOperatingSystemMXBean) osStats;
- } else {
-unixOsStats = null;
- }
-}
}
private static final ResourceAnalyzer rc = new ResourceAnalyzer();
Compilation errors when using non-Sun JDKs to build HBase-0.94
--
Key: HBASE-6945
URL: https://issues.apache.org/jira/browse/HBASE-6945
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.94.1
Environment: RHEL 6.3, IBM Java 7
Reporter: Kumar Ravi
Assignee: Kumar Ravi
Fix For: 0.94.3
When using IBM Java 7 to build HBase-0.94.1, the following comilation error
is seen.
[INFO] -
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25]
error: package com.sun.management does not exist
[ERROR]
/home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25]
error: cannot find symbol
[ERROR] symbol: class UnixOperatingSystemMXBean
location: class ResourceAnalyzer
/home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29]
error: cannot find symbol
[ERROR] symbol: class UnixOperatingSystemMXBean
location: class ResourceAnalyzer
/home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23]
error: cannot find symbol
[INFO] 4 errors
[INFO] -
[INFO]
[INFO] BUILD FAILURE
[INFO]
I have a patch available which should work for all JDKs including Sun.
I am in the process of testing this patch. Preliminary tests indicate the
build is working fine with this patch. I will post this patch when I am done
testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6965) Generic MXBean Utility class to support all JDK vendors
[ https://issues.apache.org/jira/browse/HBASE-6965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472744#comment-13472744 ] Ted Yu commented on HBASE-6965: --- Hi, You may want to put this file under hbase-server/src/main/java/org/apache/hadoop/hbase/util and produce a patch for trunk. Thanks Generic MXBean Utility class to support all JDK vendors --- Key: HBASE-6965 URL: https://issues.apache.org/jira/browse/HBASE-6965 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.94.1 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 Attachments: OSMXBean.java This issue is related to JIRA https://issues.apache.org/jira/browse/HBASE-6945. This issue is opened to propose the use of a newly created generic org.apache.hadoop.hbase.util.OSMXBean class that can be used by other classes. JIRA HBASE-6945 contains a patch for the class org.apache.hadoop.hbase.ResourceChecker that uses OSMXBean. With the inclusion of this new class, HBase can be built and become functional with JDKs and JREs other than what is provided by Oracle. This class uses reflection to determine the JVM vendor (Sun, IBM) and the platform (Linux or Windows), and contains other methods that return the OS properties - 1. Number of Open File descriptors; 2. Maximum number of File Descriptors. This class compiles without any problems with IBM JDK 7, OpenJDK 6 as well as Oracle JDK 6. Junit tests (runDevTests category) completed without any failures or errors when tested on all the three JDKs.The builds and tests were attempted on branch hbase-0.94 Revision 1396305. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6945: -- Status: Open (was: Patch Available) Compilation errors when using non-Sun JDKs to build HBase-0.94 -- Key: HBASE-6945 URL: https://issues.apache.org/jira/browse/HBASE-6945 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.94.1 Environment: RHEL 6.3, IBM Java 7 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 When using IBM Java 7 to build HBase-0.94.1, the following comilation error is seen. [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] error: package com.sun.management does not exist [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] error: cannot find symbol [INFO] 4 errors [INFO] - [INFO] [INFO] BUILD FAILURE [INFO] I have a patch available which should work for all JDKs including Sun. I am in the process of testing this patch. Preliminary tests indicate the build is working fine with this patch. I will post this patch when I am done testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472746#comment-13472746 ] Ted Yu commented on HBASE-6945: --- Hadoop QA only runs patch through trunk test suite. Can you produce a patch file for hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java and attach the patch to this JIRA ? Thanks Compilation errors when using non-Sun JDKs to build HBase-0.94 -- Key: HBASE-6945 URL: https://issues.apache.org/jira/browse/HBASE-6945 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.94.1 Environment: RHEL 6.3, IBM Java 7 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 When using IBM Java 7 to build HBase-0.94.1, the following comilation error is seen. [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] error: package com.sun.management does not exist [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] error: cannot find symbol [INFO] 4 errors [INFO] - [INFO] [INFO] BUILD FAILURE [INFO] I have a patch available which should work for all JDKs including Sun. I am in the process of testing this patch. Preliminary tests indicate the build is working fine with this patch. I will post this patch when I am done testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472749#comment-13472749 ] stack commented on HBASE-6940: -- That would work but fix the comments. They don't make sense now the default is these environment variables are on by default. Enable GC logging by default Key: HBASE-6940 URL: https://issues.apache.org/jira/browse/HBASE-6940 Project: HBase Issue Type: Improvement Components: Admin Reporter: stack Priority: Critical Fix For: 0.96.0 Attachments: 6940-trunk.txt I think we should enable gc by default. Its pretty frictionless apparently and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6958) TestAssignmentManager sometimes fails in trunk
[
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472751#comment-13472751
]
stack commented on HBASE-6958:
--
Any chance of a patch Ted?
TestAssignmentManager sometimes fails in trunk
--
Key: HBASE-6958
URL: https://issues.apache.org/jira/browse/HBASE-6958
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Fix For: 0.96.0
From
https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
:
{code}
Stacktrace
java.lang.Exception: test timed out after 5000 milliseconds
at java.lang.System.arraycopy(Native Method)
at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
at java.lang.Thread.exit(Thread.java:732)
...
2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0]
zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s)
of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set
watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640.,
state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509,
payload.length=0
2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0]
executor.EventHandler(205): Caught throwable while processing event
RS_ZK_REGION_CLOSED
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
at
org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
at
org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
at
org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
at
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread]
master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE,
server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state
from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640.
state=OFFLINE, ts=1349484372508, server=null}
{code}
Looks like NPE happened on this line:
{code}
this.gate.set(true);
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6957) TestRowCounter consistently fails against hadoop-2.0
[
https://issues.apache.org/jira/browse/HBASE-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472754#comment-13472754
]
stack commented on HBASE-6957:
--
Any chance of a patch Ted?
TestRowCounter consistently fails against hadoop-2.0
Key: HBASE-6957
URL: https://issues.apache.org/jira/browse/HBASE-6957
Project: HBase
Issue Type: Sub-task
Reporter: Ted Yu
Priority: Critical
Fix For: 0.96.0
In
https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/210/testReport/org.apache.hadoop.hbase.mapreduce/TestRowCounter/testRowCounterHiddenColumn/
, we can see:
{code}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at
org.apache.hadoop.hbase.mapreduce.TestRowCounter.runRowCount(TestRowCounter.java:135)
at
org.apache.hadoop.hbase.mapreduce.TestRowCounter.testRowCounterHiddenColumn(TestRowCounter.java:118)
...
2012-10-05 11:24:17,355 WARN [ContainersLauncher #1]
launcher.ContainerLaunch(246): Failed to launch container.
java.lang.ArithmeticException: / by zero
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:355)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
at
org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.getLogPathForWrite(LocalDirsHandlerService.java:268)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:126)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:68)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2012-10-05 11:24:17,356 WARN [DeletionService #1]
nodemanager.DefaultContainerExecutor(276): delete returned false for path:
[/home/jenkins/jenkins-slave/workspace/HBase-TRUNK-on-Hadoop-2.0.0/trunk/hbase-server/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-localDir-nm-1_0/usercache/jenkins/appcache/application_1349436189156_0003/container_1349436189156_0003_01_02]
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6940: -- Attachment: 6940-v2.txt Enable GC logging by default Key: HBASE-6940 URL: https://issues.apache.org/jira/browse/HBASE-6940 Project: HBase Issue Type: Improvement Components: Admin Reporter: stack Priority: Critical Fix For: 0.96.0 Attachments: 6940-trunk.txt, 6940-v2.txt I think we should enable gc by default. Its pretty frictionless apparently and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6940: -- Status: Patch Available (was: Open) Enable GC logging by default Key: HBASE-6940 URL: https://issues.apache.org/jira/browse/HBASE-6940 Project: HBase Issue Type: Improvement Components: Admin Reporter: stack Priority: Critical Fix For: 0.96.0 Attachments: 6940-trunk.txt, 6940-v2.txt I think we should enable gc by default. Its pretty frictionless apparently and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472762#comment-13472762 ] Kumar Ravi commented on HBASE-6945: --- A build with the above patch along with the new class in https://issues.apache.org/jira/browse/HBASE-6965 were built and tested on Revision 1396305: /hbase/branches/0.94 with IBM JDK 7, openJDK 6 as well as Oracle JDK 6. Builds were successful and the junit tests (runDevTests) ran with all three JDKs without any errors or failures. Compilation errors when using non-Sun JDKs to build HBase-0.94 -- Key: HBASE-6945 URL: https://issues.apache.org/jira/browse/HBASE-6945 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.94.1 Environment: RHEL 6.3, IBM Java 7 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 When using IBM Java 7 to build HBase-0.94.1, the following comilation error is seen. [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] error: package com.sun.management does not exist [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] error: cannot find symbol [INFO] 4 errors [INFO] - [INFO] [INFO] BUILD FAILURE [INFO] I have a patch available which should work for all JDKs including Sun. I am in the process of testing this patch. Preliminary tests indicate the build is working fine with this patch. I will post this patch when I am done testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472764#comment-13472764 ] Kumar Ravi commented on HBASE-6945: --- Ted, Thanks for your comment. Attaching a patch file.. Compilation errors when using non-Sun JDKs to build HBase-0.94 -- Key: HBASE-6945 URL: https://issues.apache.org/jira/browse/HBASE-6945 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.94.1 Environment: RHEL 6.3, IBM Java 7 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 Attachments: ResourceChecker.patch When using IBM Java 7 to build HBase-0.94.1, the following comilation error is seen. [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] error: package com.sun.management does not exist [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] error: cannot find symbol [INFO] 4 errors [INFO] - [INFO] [INFO] BUILD FAILURE [INFO] I have a patch available which should work for all JDKs including Sun. I am in the process of testing this patch. Preliminary tests indicate the build is working fine with this patch. I will post this patch when I am done testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kumar Ravi updated HBASE-6945: -- Attachment: ResourceChecker.patch Compilation errors when using non-Sun JDKs to build HBase-0.94 -- Key: HBASE-6945 URL: https://issues.apache.org/jira/browse/HBASE-6945 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.94.1 Environment: RHEL 6.3, IBM Java 7 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 Attachments: ResourceChecker.patch When using IBM Java 7 to build HBase-0.94.1, the following comilation error is seen. [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] error: package com.sun.management does not exist [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] error: cannot find symbol [INFO] 4 errors [INFO] - [INFO] [INFO] BUILD FAILURE [INFO] I have a patch available which should work for all JDKs including Sun. I am in the process of testing this patch. Preliminary tests indicate the build is working fine with this patch. I will post this patch when I am done testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[
https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472770#comment-13472770
]
stack commented on HBASE-6940:
--
+1
On commit put this line:
{code}
+export HBASE_USE_GC_LOGFILE=true
{code}
... before the uncommented HBASE_OPTS
Enable GC logging by default
Key: HBASE-6940
URL: https://issues.apache.org/jira/browse/HBASE-6940
Project: HBase
Issue Type: Improvement
Components: Admin
Reporter: stack
Priority: Critical
Fix For: 0.96.0
Attachments: 6940-trunk.txt, 6940-v2.txt
I think we should enable gc by default. Its pretty frictionless apparently
and could help in the case where folks are getting off the ground.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6966) Compressed RPCs for HBase (HBASE-5355) port to trunk
Devaraj Das created HBASE-6966: -- Summary: Compressed RPCs for HBase (HBASE-5355) port to trunk Key: HBASE-6966 URL: https://issues.apache.org/jira/browse/HBASE-6966 Project: HBase Issue Type: Improvement Components: IPC/RPC Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 This jira will address the port of the compressed RPC implementation to trunk. I am expecting the patch to be significantly different due to the PB stuff in trunk, and hence filed a separate jira. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6804) [replication] lower the amount of logging to a more human-readable level
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472774#comment-13472774 ] stack commented on HBASE-6804: -- +1 on commit. Can this go back into 0.94 too? [replication] lower the amount of logging to a more human-readable level Key: HBASE-6804 URL: https://issues.apache.org/jira/browse/HBASE-6804 Project: HBase Issue Type: Improvement Reporter: Jean-Daniel Cryans Assignee: Jean-Daniel Cryans Priority: Critical Fix For: 0.96.0 Attachments: HBASE-6804-0.94.patch, HBASE-6804-0.94-v2.patch We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5355) Compressed RPC's for HBase
[ https://issues.apache.org/jira/browse/HBASE-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472776#comment-13472776 ] Devaraj Das commented on HBASE-5355: Raised HBASE-6966 for the forward port to trunk. Compressed RPC's for HBase -- Key: HBASE-5355 URL: https://issues.apache.org/jira/browse/HBASE-5355 Project: HBase Issue Type: Improvement Components: IPC/RPC Affects Versions: 0.89.20100924 Reporter: Karthik Ranganathan Assignee: Karthik Ranganathan Attachments: HBASE-5355-0.94.patch Some application need ability to do large batched writes and reads from a remote MR cluster. These eventually get bottlenecked on the network. These results are also pretty compressible sometimes. The aim here is to add the ability to do compressed calls to the server on both the send and receive paths. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6956) Do not return back to HTablePool closed connections
[
https://issues.apache.org/jira/browse/HBASE-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472779#comment-13472779
]
stack commented on HBASE-6956:
--
You have a patch Igor?
Do not return back to HTablePool closed connections
---
Key: HBASE-6956
URL: https://issues.apache.org/jira/browse/HBASE-6956
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.90.6
Reporter: Igor Yurinok
Sometimes we see a lot of Exception about closed connections:
{code}
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068
closed
org.apache.hadoop.hbase.client.ClosedConnectionException:
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@553fd068
closed
{code}
After investigation we assumed that it occurs because closed connection
returns back into HTablePool.
For our opinion best solution is check whether the table is closed in method
HTablePool.putTable and if true don't add it into the queue and release such
HTableInterface.
But unfortunatly right now there are no access to HTable#closed field through
HTableInterface
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6951) Allow the master info server to be started in a read only mode.
[ https://issues.apache.org/jira/browse/HBASE-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472787#comment-13472787 ] stack commented on HBASE-6951: -- +1 on patch Allow the master info server to be started in a read only mode. --- Key: HBASE-6951 URL: https://issues.apache.org/jira/browse/HBASE-6951 Project: HBase Issue Type: Improvement Components: UI Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Labels: noob Attachments: HBASE-6951-trunk-0.patch There are some cases that a user could want a web ui to be accessible but might not want the split and compact functionality to be usable. Allowing the web ui to start in a readOnly mode would be good. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6958) TestAssignmentManager sometimes fails in trunk
[
https://issues.apache.org/jira/browse/HBASE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472792#comment-13472792
]
Jimmy Xiang commented on HBASE-6958:
I can take a look next week if it is not urgent.
TestAssignmentManager sometimes fails in trunk
--
Key: HBASE-6958
URL: https://issues.apache.org/jira/browse/HBASE-6958
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Fix For: 0.96.0
From
https://builds.apache.org/job/HBase-TRUNK/3432/testReport/junit/org.apache.hadoop.hbase.master/TestAssignmentManager/testBalanceOnMasterFailoverScenarioWithOpenedNode/
:
{code}
Stacktrace
java.lang.Exception: test timed out after 5000 milliseconds
at java.lang.System.arraycopy(Native Method)
at java.lang.ThreadGroup.remove(ThreadGroup.java:969)
at java.lang.ThreadGroup.threadTerminated(ThreadGroup.java:942)
at java.lang.Thread.exit(Thread.java:732)
...
2012-10-06 00:46:12,521 DEBUG [MASTER_CLOSE_REGION-mockedAMExecutor-0]
zookeeper.ZKUtil(1141): mockedServer-0x13a33892de7000e Retrieved 81 byte(s)
of data from znode /hbase/unassigned/dc01abf9cd7fd0ea256af4df02811640 and set
watcher; region=t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640.,
state=M_ZK_REGION_OFFLINE, servername=master,1,1, createTime=1349484372509,
payload.length=0
2012-10-06 00:46:12,522 ERROR [MASTER_CLOSE_REGION-mockedAMExecutor-0]
executor.EventHandler(205): Caught throwable while processing event
RS_ZK_REGION_CLOSED
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.TestAssignmentManager$MockedLoadBalancer.randomAssignment(TestAssignmentManager.java:773)
at
org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1709)
at
org.apache.hadoop.hbase.master.AssignmentManager.getRegionPlan(AssignmentManager.java:1666)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1435)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1155)
at
org.apache.hadoop.hbase.master.TestAssignmentManager$AssignmentManagerWithExtrasForTesting.assign(TestAssignmentManager.java:1035)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1130)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1125)
at
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:106)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:202)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2012-10-06 00:46:12,522 DEBUG [pool-1-thread-1-EventThread]
master.AssignmentManager(670): Handling transition=M_ZK_REGION_OFFLINE,
server=master,1,1, region=dc01abf9cd7fd0ea256af4df02811640, current state
from region state map ={t,,1349484359011.dc01abf9cd7fd0ea256af4df02811640.
state=OFFLINE, ts=1349484372508, server=null}
{code}
Looks like NPE happened on this line:
{code}
this.gate.set(true);
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-3646) When mapper writes multiple values for a key keep chronological order of values
[ https://issues.apache.org/jira/browse/HBASE-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-3646. -- Resolution: Incomplete Closing for now. Open a new issue when a patch Bob. Thanks. When mapper writes multiple values for a key keep chronological order of values --- Key: HBASE-3646 URL: https://issues.apache.org/jira/browse/HBASE-3646 Project: HBase Issue Type: New Feature Components: Client Affects Versions: 0.90.1 Environment: Cloudera 3.5 VM TableMapperImmutableBytesWritable,IntWritable TableReducerImmutableBytesWritable,IntWritable, ImmutableBytesWritable Reporter: Bob Cummins Priority: Minor When mapper writes multiple values for a key, the underlying collection class maps each of the values to the key, but not always in chronological order. If chronological order were guaranteed each of the values mapped to the key, each of the values could be understood as specific and different parameters between the mapper and the reducer. I've done little tricks like having the mapper flag one a the values by making it a negative number, which the reducer recognizes and can write it to hbase as a unique column value.This is a kluge workaround which it would be nice to not have to do. Used to formulate this suggestion: TableMapperImmutableBytesWritable,IntWritable TableReducerImmutableBytesWritable,IntWritable, ImmutableBytesWritable -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[
https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472800#comment-13472800
]
Hadoop QA commented on HBASE-6940:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548465/6940-v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3024//console
This message is automatically generated.
Enable GC logging by default
Key: HBASE-6940
URL: https://issues.apache.org/jira/browse/HBASE-6940
Project: HBase
Issue Type: Improvement
Components: Admin
Reporter: stack
Priority: Critical
Fix For: 0.96.0
Attachments: 6940-trunk.txt, 6940-v2.txt
I think we should enable gc by default. Its pretty frictionless apparently
and could help in the case where folks are getting off the ground.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3661) Handle empty qualifier better in shell for increments
[
https://issues.apache.org/jira/browse/HBASE-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472801#comment-13472801
]
stack commented on HBASE-3661:
--
Patch looks good to me Michael. Can you mark it as patch available so it runs
by hadoopqa (I can't for some reason)
Handle empty qualifier better in shell for increments
-
Key: HBASE-3661
URL: https://issues.apache.org/jira/browse/HBASE-3661
Project: HBase
Issue Type: Improvement
Components: shell
Affects Versions: 0.92.0
Reporter: Lars George
Assignee: Michael Drzal
Priority: Minor
Attachments: HBASE-3661.patch
When trying to increment a counter using the examples, which specify no
*explicit* qualifier you get an error:
{code}
hbase(main):014:0 incr 'testtable', 'cnt1', 'colfam1', 1
ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
contact region server 10.0.0.57:51640 for region
testtable,,1300267113942.cd2e7925140eb414d519621e384fb654., row 'cnt1', but
failed after 7 attempts.
Exceptions:
java.io.IOException: java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.ColumnCount.init(ColumnCount.java:47)
at
org.apache.hadoop.hbase.regionserver.ExplicitColumnTracker.init(ExplicitColumnTracker.java:69)
at
org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.init(ScanQueryMatcher.java:93)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.init(StoreScanner.java:65)
at
org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1436)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.init(HRegion.java:2412)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateInternalScanner(HRegion.java:1185)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1171)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1155)
at
org.apache.hadoop.hbase.regionserver.HRegion.getLastIncrement(HRegion.java:3087)
at
org.apache.hadoop.hbase.regionserver.HRegion.incrementColumnValue(HRegion.java:3312)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.incrementColumnValue(HRegionServer.java:2570)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:309)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1060)
Here is some help for this command:
Increments a cell 'value' at specified table/row/column coordinates.
To increment a cell value in table 't1' at row 'r1' under column
'c1' by 1 (can be omitted) or 10 do:
hbase incr 't1', 'r1', 'c1'
hbase incr 't1', 'r1', 'c1', 1
hbase incr 't1', 'r1', 'c1', 10
{code}
Handle this more gracefully (printing 5 stacktraces is ugly), improve the
help to specify what is needed more clearly. Or fix the server side to
support this, if this makes sense, and therefore never triggering this issue.
Adding a qualifier makes it work:
{code}
hbase(main):015:0 incr 'testtable', 'cnt1', 'colfam1:test', 1
COUNTER VALUE = 1
hbase(main):016:0 incr 'testtable', 'cnt1', 'colfam1:test', 1
COUNTER VALUE = 2
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6953) Incorrect javadoc description of HFileOutputFormat regarding multiple column families
[ https://issues.apache.org/jira/browse/HBASE-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6953. -- Resolution: Fixed Fix Version/s: 0.96.0 Committed to trunk. Thanks for the patch Gabriel. Incorrect javadoc description of HFileOutputFormat regarding multiple column families - Key: HBASE-6953 URL: https://issues.apache.org/jira/browse/HBASE-6953 Project: HBase Issue Type: Bug Components: mapreduce Reporter: Gabriel Reid Priority: Minor Fix For: 0.96.0 Attachments: HBASE-6953.patch The javadoc for HFileOutputFormat states that the class does not support writing multiple column families; however, this hasn't been the case since HBASE-1861 was resolved. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[ https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472810#comment-13472810 ] stack commented on HBASE-6316: -- Can I get a review for this blocker 0.96 issue? Thanks. Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting - Key: HBASE-6316 URL: https://issues.apache.org/jira/browse/HBASE-6316 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Fix For: 0.96.0 Attachments: 6316.txt, 6316v2.txt, 6316v3.txt, a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96 to start over data written by a 0.94. Need to fix it so don't have to do this -- that zk state left over gets auto-migrated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472826#comment-13472826 ] Hudson commented on HBASE-6961: --- Integrated in HBase-TRUNK #3437 (See [https://builds.apache.org/job/HBase-TRUNK/3437/]) HBASE-6961 In place scripts fail if mvn install hasn't been run (Revision 1396307) Result = SUCCESS eclark : Files : * /hbase/trunk/bin/hbase In place scripts fail if mvn install hasn't been run Key: HBASE-6961 URL: https://issues.apache.org/jira/browse/HBASE-6961 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6961-0.patch bin/hbase tries to get dependencies of the project however it fails if mvn install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[
https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472845#comment-13472845
]
Ted Yu commented on HBASE-6316:
---
{code}
+return convert(FSProtos.Reference.parseFrom(in));
+ } else {
{code}
nit: keyword else can be omitted.
You may want to submit for QA run after including
a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c in the patch.
Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
-
Key: HBASE-6316
URL: https://issues.apache.org/jira/browse/HBASE-6316
Project: HBase
Issue Type: Bug
Reporter: stack
Priority: Blocker
Fix For: 0.96.0
Attachments: 6316.txt, 6316v2.txt, 6316v3.txt,
a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c
Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96
to start over data written by a 0.94. Need to fix it so don't have to do
this -- that zk state left over gets auto-migrated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-6940: - Assignee: Ted Yu Enable GC logging by default Key: HBASE-6940 URL: https://issues.apache.org/jira/browse/HBASE-6940 Project: HBase Issue Type: Improvement Components: Admin Reporter: stack Assignee: Ted Yu Priority: Critical Fix For: 0.96.0 Attachments: 6940-trunk.txt, 6940-v2.txt I think we should enable gc by default. Its pretty frictionless apparently and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472849#comment-13472849 ] Ted Yu commented on HBASE-6940: --- Integrated to trunk. Thanks for the review, Stack. Enable GC logging by default Key: HBASE-6940 URL: https://issues.apache.org/jira/browse/HBASE-6940 Project: HBase Issue Type: Improvement Components: Admin Reporter: stack Assignee: Ted Yu Priority: Critical Fix For: 0.96.0 Attachments: 6940-trunk.txt, 6940-v2.txt I think we should enable gc by default. Its pretty frictionless apparently and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6940: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Enable GC logging by default Key: HBASE-6940 URL: https://issues.apache.org/jira/browse/HBASE-6940 Project: HBase Issue Type: Improvement Components: Admin Reporter: stack Assignee: Ted Yu Priority: Critical Fix For: 0.96.0 Attachments: 6940-trunk.txt, 6940-v2.txt I think we should enable gc by default. Its pretty frictionless apparently and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6967) Introduce a hook in StoreScanner to allow aggregation of KeyValues in a single row.
Adela Maznikar created HBASE-6967: - Summary: Introduce a hook in StoreScanner to allow aggregation of KeyValues in a single row. Key: HBASE-6967 URL: https://issues.apache.org/jira/browse/HBASE-6967 Project: HBase Issue Type: New Feature Reporter: Adela Maznikar Assignee: Adela Maznikar Priority: Minor Introduce an interface for enabling hooks in StoreScanner -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6968) Several HBase write perf improvement
Liyin Tang created HBASE-6968: - Summary: Several HBase write perf improvement Key: HBASE-6968 URL: https://issues.apache.org/jira/browse/HBASE-6968 Project: HBase Issue Type: Improvement Reporter: Liyin Tang There are two improvements in this jira: 1) Change 2 hotspot synchronized functions into double locking pattern. So it shall remove the synchronization overhead in the normal case. 2) Avoid creating HBaseConfiguraiton object for each HLog. Every time when creating a HBaseConfiguraiton object, it would parse the xml configuration files from disk, which is not cheap operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6968) Several HBase write perf improvement
[
https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HBASE-6968:
--
Description:
Here are 2 hbase write performance improvements recently found out.
1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
newWriter = createWriter(fs, newPath, conf);
{code}
2) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
public synchronized void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
m = create(name);
}
m.inc(amt);
}
{code}
new:
{code}
public void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
synchronized (this) {
if ((m = get(name)) == null) {
m = create(name);
}
}
}
m.inc(amt);
}
{code}
=
orig:
{code:title=MemStoreFlusher.java}
public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {
flushSomeRegions();
}
}
{code}
new:
{code}
public void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {
flushSomeRegions();
}
}
private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get() globalMemStoreLimit) {
return; // double check the global memstore size inside of the
synchronized block.
}
...
}
{code}
was:
There are two improvements in this jira:
1) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
2) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
Several HBase write perf improvement
Key: HBASE-6968
URL: https://issues.apache.org/jira/browse/HBASE-6968
Project: HBase
Issue Type: Improvement
Reporter: Liyin Tang
Here are 2 hbase write performance improvements recently found out.
1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
newWriter = createWriter(fs, newPath, conf);
{code}
2) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
public synchronized void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
m = create(name);
}
m.inc(amt);
}
{code}
new:
{code}
public void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
synchronized (this) {
if ((m = get(name)) == null) {
m = create(name);
}
}
}
m.inc(amt);
}
{code}
=
orig:
{code:title=MemStoreFlusher.java}
public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {
flushSomeRegions();
}
}
{code}
new:
{code}
public void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {
flushSomeRegions();
}
}
private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get() globalMemStoreLimit) {
return; // double check the global memstore size inside of the
synchronized block.
}
...
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[ https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6316: - Attachment: 6316v3.txt Here is what I committed. Thanks for the review Ted. The else was in previous code but I fixed it (Added WARNING about return in middle of a method). Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting - Key: HBASE-6316 URL: https://issues.apache.org/jira/browse/HBASE-6316 Project: HBase Issue Type: Bug Reporter: stack Priority: Blocker Fix For: 0.96.0 Attachments: 6316.txt, 6316v2.txt, 6316v3.txt, 6316v3.txt, a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96 to start over data written by a 0.94. Need to fix it so don't have to do this -- that zk state left over gets auto-migrated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6968) Several HBase write perf improvement
[
https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated HBASE-6968:
--
Description:
Here are 2 hbase write performance improvements recently:
1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
newWriter = createWriter(fs, newPath, conf);
{code}
2) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
public synchronized void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
m = create(name);
}
m.inc(amt);
}
{code}
new:
{code}
public void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
synchronized (this) {
if ((m = get(name)) == null) {
m = create(name);
}
}
}
m.inc(amt);
}
{code}
=
orig:
{code:title=MemStoreFlusher.java}
public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {
flushSomeRegions();
}
}
{code}
new:
{code}
public void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {
flushSomeRegions();
}
}
private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get() globalMemStoreLimit) {
return; // double check the global memstore size inside of the
synchronized block.
}
...
}
{code}
was:
Here are 2 hbase write performance improvements recently found out.
1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
newWriter = createWriter(fs, newPath, conf);
{code}
2) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
public synchronized void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
m = create(name);
}
m.inc(amt);
}
{code}
new:
{code}
public void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
synchronized (this) {
if ((m = get(name)) == null) {
m = create(name);
}
}
}
m.inc(amt);
}
{code}
=
orig:
{code:title=MemStoreFlusher.java}
public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {
flushSomeRegions();
}
}
{code}
new:
{code}
public void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {
flushSomeRegions();
}
}
private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get() globalMemStoreLimit) {
return; // double check the global memstore size inside of the
synchronized block.
}
...
}
{code}
Several HBase write perf improvement
Key: HBASE-6968
URL: https://issues.apache.org/jira/browse/HBASE-6968
Project: HBase
Issue Type: Improvement
Reporter: Liyin Tang
Here are 2 hbase write performance improvements recently:
1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
newWriter = createWriter(fs, newPath, conf);
{code}
2) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
public synchronized void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
m = create(name);
}
[jira] [Commented] (HBASE-6945) Compilation errors when using non-Sun JDKs to build HBase-0.94
[ https://issues.apache.org/jira/browse/HBASE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472887#comment-13472887 ] stack commented on HBASE-6945: -- [~nkeywal] You should be able to admin now (I added you as administrator) [~kumarr] Is the OSMXBean missing from your patch? You forgot to svn/git add it? Compilation errors when using non-Sun JDKs to build HBase-0.94 -- Key: HBASE-6945 URL: https://issues.apache.org/jira/browse/HBASE-6945 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.94.1 Environment: RHEL 6.3, IBM Java 7 Reporter: Kumar Ravi Assignee: Kumar Ravi Fix For: 0.94.3 Attachments: ResourceChecker.patch When using IBM Java 7 to build HBase-0.94.1, the following comilation error is seen. [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[23,25] error: package com.sun.management does not exist [ERROR] /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[46,25] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[75,29] error: cannot find symbol [ERROR] symbol: class UnixOperatingSystemMXBean location: class ResourceAnalyzer /home/hadoop/hbase-0.94/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java:[76,23] error: cannot find symbol [INFO] 4 errors [INFO] - [INFO] [INFO] BUILD FAILURE [INFO] I have a patch available which should work for all JDKs including Sun. I am in the process of testing this patch. Preliminary tests indicate the build is working fine with this patch. I will post this patch when I am done testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[ https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6316. -- Resolution: Fixed Assignee: stack Hadoop Flags: Reviewed Committed to trunk Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting - Key: HBASE-6316 URL: https://issues.apache.org/jira/browse/HBASE-6316 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.96.0 Attachments: 6316.txt, 6316v2.txt, 6316v3.txt, 6316v3.txt, a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96 to start over data written by a 0.94. Need to fix it so don't have to do this -- that zk state left over gets auto-migrated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6961) In place scripts fail if mvn install hasn't been run
[ https://issues.apache.org/jira/browse/HBASE-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472895#comment-13472895 ] Hudson commented on HBASE-6961: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #215 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/215/]) HBASE-6961 In place scripts fail if mvn install hasn't been run (Revision 1396307) Result = FAILURE eclark : Files : * /hbase/trunk/bin/hbase In place scripts fail if mvn install hasn't been run Key: HBASE-6961 URL: https://issues.apache.org/jira/browse/HBASE-6961 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6961-0.patch bin/hbase tries to get dependencies of the project however it fails if mvn install hasn't already satisfied hbase-hadoop-compat test-jar. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[ https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472896#comment-13472896 ] Hudson commented on HBASE-6316: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #215 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/215/]) HBASE-6316 Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting (Revision 1396428) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/Reference.java * /hbase/trunk/hbase-server/src/test/data/a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestReference.java Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting - Key: HBASE-6316 URL: https://issues.apache.org/jira/browse/HBASE-6316 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.96.0 Attachments: 6316.txt, 6316v2.txt, 6316v3.txt, 6316v3.txt, a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96 to start over data written by a 0.94. Need to fix it so don't have to do this -- that zk state left over gets auto-migrated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6953) Incorrect javadoc description of HFileOutputFormat regarding multiple column families
[ https://issues.apache.org/jira/browse/HBASE-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472897#comment-13472897 ] Hudson commented on HBASE-6953: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #215 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/215/]) HBASE-6953 Incorrect javadoc description of HFileOutputFormat regarding multiple column families (Revision 1396383) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java Incorrect javadoc description of HFileOutputFormat regarding multiple column families - Key: HBASE-6953 URL: https://issues.apache.org/jira/browse/HBASE-6953 Project: HBase Issue Type: Bug Components: mapreduce Reporter: Gabriel Reid Priority: Minor Fix For: 0.96.0 Attachments: HBASE-6953.patch The javadoc for HFileOutputFormat states that the class does not support writing multiple column families; however, this hasn't been the case since HBASE-1861 was resolved. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472898#comment-13472898 ] Hudson commented on HBASE-6940: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #215 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/215/]) HBASE-6940 Enable GC logging by default (Revision 1396399) Result = FAILURE tedyu : Files : * /hbase/trunk/conf/hbase-env.sh Enable GC logging by default Key: HBASE-6940 URL: https://issues.apache.org/jira/browse/HBASE-6940 Project: HBase Issue Type: Improvement Components: Admin Reporter: stack Assignee: Ted Yu Priority: Critical Fix For: 0.96.0 Attachments: 6940-trunk.txt, 6940-v2.txt I think we should enable gc by default. Its pretty frictionless apparently and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6953) Incorrect javadoc description of HFileOutputFormat regarding multiple column families
[ https://issues.apache.org/jira/browse/HBASE-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472901#comment-13472901 ] Hudson commented on HBASE-6953: --- Integrated in HBase-TRUNK #3438 (See [https://builds.apache.org/job/HBase-TRUNK/3438/]) HBASE-6953 Incorrect javadoc description of HFileOutputFormat regarding multiple column families (Revision 1396383) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java Incorrect javadoc description of HFileOutputFormat regarding multiple column families - Key: HBASE-6953 URL: https://issues.apache.org/jira/browse/HBASE-6953 Project: HBase Issue Type: Bug Components: mapreduce Reporter: Gabriel Reid Priority: Minor Fix For: 0.96.0 Attachments: HBASE-6953.patch The javadoc for HFileOutputFormat states that the class does not support writing multiple column families; however, this hasn't been the case since HBASE-1861 was resolved. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6940) Enable GC logging by default
[ https://issues.apache.org/jira/browse/HBASE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472902#comment-13472902 ] Hudson commented on HBASE-6940: --- Integrated in HBase-TRUNK #3438 (See [https://builds.apache.org/job/HBase-TRUNK/3438/]) HBASE-6940 Enable GC logging by default (Revision 1396399) Result = FAILURE tedyu : Files : * /hbase/trunk/conf/hbase-env.sh Enable GC logging by default Key: HBASE-6940 URL: https://issues.apache.org/jira/browse/HBASE-6940 Project: HBase Issue Type: Improvement Components: Admin Reporter: stack Assignee: Ted Yu Priority: Critical Fix For: 0.96.0 Attachments: 6940-trunk.txt, 6940-v2.txt I think we should enable gc by default. Its pretty frictionless apparently and could help in the case where folks are getting off the ground. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6929) Publish Hbase 0.94 artifacts build against hadoop-2.0
[
https://issues.apache.org/jira/browse/HBASE-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472932#comment-13472932
]
Enis Soztutar commented on HBASE-6929:
--
@Stack, I think -Dclassifier causes the problem.
{code}
mvn clean install -Dhadoop.profile=2.0
{code}
Works for me on 0.94. There should be a pom.xml change for deploying the local
jar with a classifier. I'll look into that.
@Lars, I did not see an umbrella issue, but I think we have to compile a list,
and see what comes out of it. If I have some time by the end of week, I'll try
to do that.
Publish Hbase 0.94 artifacts build against hadoop-2.0
-
Key: HBASE-6929
URL: https://issues.apache.org/jira/browse/HBASE-6929
Project: HBase
Issue Type: Task
Components: build
Affects Versions: 0.94.2
Reporter: Enis Soztutar
Downstream projects (flume, hive, pig, etc) depends on hbase, but since the
hbase binaries build with hadoop-2.0 are not pushed to maven, they cannot
depend on them. AFAIK, hadoop 1 and 2 are not binary compatible, so we should
also push hbase jars build with hadoop2.0 profile into maven, possibly with
version string like 0.94.2-hadoop2.0.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6316) Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting
[ https://issues.apache.org/jira/browse/HBASE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472937#comment-13472937 ] Hudson commented on HBASE-6316: --- Integrated in HBase-TRUNK #3439 (See [https://builds.apache.org/job/HBase-TRUNK/3439/]) HBASE-6316 Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting (Revision 1396428) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/Reference.java * /hbase/trunk/hbase-server/src/test/data/a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestReference.java Confirm can upgrade to 0.96 from 0.94 by just stopping and restarting - Key: HBASE-6316 URL: https://issues.apache.org/jira/browse/HBASE-6316 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Priority: Blocker Fix For: 0.96.0 Attachments: 6316.txt, 6316v2.txt, 6316v3.txt, 6316v3.txt, a6a6562b777440fd9c34885428f5cb61.21e75333ada3d5bafb34bb918f29576c Over in HBASE-6294, LarsH says you have to currently clear zk to get a 0.96 to start over data written by a 0.94. Need to fix it so don't have to do this -- that zk state left over gets auto-migrated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6969) Allow for injecting MiniZookeeperCluster instance
Micah Whitacre created HBASE-6969: - Summary: Allow for injecting MiniZookeeperCluster instance Key: HBASE-6969 URL: https://issues.apache.org/jira/browse/HBASE-6969 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.1 Reporter: Micah Whitacre While not an official part of the HBase API, it'd be nice if HBaseTestingUtility allowed for consumer to inject their own instance of the MiniZookeeperCluster when executing tests. Currently there is no way to control how the ZK instance is started which we've had to hack around when there were ZK version conflicts (3.3 vs 3.4). Or if you want to control which specific port it starts on vs random. Allowing consumers to inject an instance (unstarted) gives them the freedom to implement their own without having to fork the entire HBaseTestingUtility class -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6969) Allow for injecting MiniZookeeperCluster instance
[ https://issues.apache.org/jira/browse/HBASE-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Micah Whitacre updated HBASE-6969: -- Attachment: hbase-6969.patch Simple patch file that just allows the HBaseTestingUtility to access a base MiniZKCluster instance. Allow for injecting MiniZookeeperCluster instance - Key: HBASE-6969 URL: https://issues.apache.org/jira/browse/HBASE-6969 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.1 Reporter: Micah Whitacre Attachments: hbase-6969.patch While not an official part of the HBase API, it'd be nice if HBaseTestingUtility allowed for consumer to inject their own instance of the MiniZookeeperCluster when executing tests. Currently there is no way to control how the ZK instance is started which we've had to hack around when there were ZK version conflicts (3.3 vs 3.4). Or if you want to control which specific port it starts on vs random. Allowing consumers to inject an instance (unstarted) gives them the freedom to implement their own without having to fork the entire HBaseTestingUtility class -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5525) Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Odell updated HBASE-5525: --- Attachment: HBASE-5525.patch So I added a new command truncate_preserve, it seemed to fit with the flow of the other commands: disable_all disable_peer enable_all enable_peer alter alter_async alter_status I am not too attached to the name so I am open for suggestions. I tested this out by altering the ruby code on my box and it worked. Let me know if you see anything strange. Truncate and preserve region boundaries option -- Key: HBASE-5525 URL: https://issues.apache.org/jira/browse/HBASE-5525 Project: HBase Issue Type: New Feature Reporter: Jean-Daniel Cryans Assignee: Kevin Odell Labels: newbie, noob Attachments: HBASE-5525.patch A tool that would be useful for testing (and maybe in prod too) would be a truncate option to keep the current region boundaries. Right now what you have to do is completely kill the table and recreate it with the correct regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5525) Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Odell updated HBASE-5525: --- Fix Version/s: 0.96.0 Affects Version/s: 0.96.0 Status: Patch Available (was: Open) Truncate and preserve region boundaries option -- Key: HBASE-5525 URL: https://issues.apache.org/jira/browse/HBASE-5525 Project: HBase Issue Type: New Feature Affects Versions: 0.96.0 Reporter: Jean-Daniel Cryans Assignee: Kevin Odell Labels: newbie, noob Fix For: 0.96.0 Attachments: HBASE-5525.patch A tool that would be useful for testing (and maybe in prod too) would be a truncate option to keep the current region boundaries. Right now what you have to do is completely kill the table and recreate it with the correct regions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6969) Allow for injecting MiniZookeeperCluster instance
[ https://issues.apache.org/jira/browse/HBASE-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472946#comment-13472946 ] Micah Whitacre commented on HBASE-6969: --- It should be noted I made the patch based off of trunk but I would like it patched back to 0.92.1 (the version I'm currently on). Trunk has a number of improvements (e.g configuration for the port) but I'm on CDH4.1 which is 0.92.1. Allow for injecting MiniZookeeperCluster instance - Key: HBASE-6969 URL: https://issues.apache.org/jira/browse/HBASE-6969 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.1 Reporter: Micah Whitacre Attachments: hbase-6969.patch While not an official part of the HBase API, it'd be nice if HBaseTestingUtility allowed for consumer to inject their own instance of the MiniZookeeperCluster when executing tests. Currently there is no way to control how the ZK instance is started which we've had to hack around when there were ZK version conflicts (3.3 vs 3.4). Or if you want to control which specific port it starts on vs random. Allowing consumers to inject an instance (unstarted) gives them the freedom to implement their own without having to fork the entire HBaseTestingUtility class -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6853) IllegalArgument Exception is thrown when an empty region is spliitted.
[
https://issues.apache.org/jira/browse/HBASE-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472958#comment-13472958
]
Lars Hofhansl commented on HBASE-6853:
--
Should we commit the addendum? (Also should we limit the waiting, or just wait
until the test is killed?)
IllegalArgument Exception is thrown when an empty region is spliitted.
--
Key: HBASE-6853
URL: https://issues.apache.org/jira/browse/HBASE-6853
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1, 0.94.1
Reporter: ramkrishna.s.vasudevan
Assignee: Priyadarshini
Fix For: 0.94.2, 0.96.0
Attachments: HBASE-6853_2_splitsuccess.patch,
HBASE-6853_addendum.patch, HBASE-6853.patch, HBASE-6853_splitfailure.patch
This is w.r.t a mail sent in the dev mail list.
Empty region split should be handled gracefully. Either we should not allow
the split to happen if we know that the region is empty or we should allow
the split to happen by setting the no of threads to the thread pool executor
as 1.
{code}
int nbFiles = hstoreFilesToSplit.size();
ThreadFactoryBuilder builder = new ThreadFactoryBuilder();
builder.setNameFormat(StoreFileSplitter-%1$d);
ThreadFactory factory = builder.build();
ThreadPoolExecutor threadPool =
(ThreadPoolExecutor) Executors.newFixedThreadPool(nbFiles, factory);
ListFutureVoid futures = new ArrayListFutureVoid(nbFiles);
{code}
Here the nbFiles needs to be a non zero positive value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
Lars Hofhansl created HBASE-6970:
Summary: hbase-deamon.sh creates/updates pid file even when that
start failed.
Key: HBASE-6970
URL: https://issues.apache.org/jira/browse/HBASE-6970
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS $HBASE_HOME/bin/hbase \
--config ${HBASE_CONF_DIR} \
$command $@ $startStop $logout 21 /dev/null
echo $! $pid
{code}
So the pid file is create or update even the start of the service failed. The
next stop command will then fail, because the pid file has he wrong pid in it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
[
https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472959#comment-13472959
]
Lars Hofhansl commented on HBASE-6970:
--
Also we found that the stop command does not return a failed exit code ( 0),
if the stop fails because the running process was not found.
I am not sure how risky changing this behavior is for current installation that
might rely on the this behavior.
hbase-deamon.sh creates/updates pid file even when that start failed.
-
Key: HBASE-6970
URL: https://issues.apache.org/jira/browse/HBASE-6970
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS $HBASE_HOME/bin/hbase \
--config ${HBASE_CONF_DIR} \
$command $@ $startStop $logout 21 /dev/null
echo $! $pid
{code}
So the pid file is create or update even the start of the service failed. The
next stop command will then fail, because the pid file has he wrong pid in it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
[
https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472964#comment-13472964
]
Jean-Daniel Cryans commented on HBASE-6970:
---
lol I'm looking at the same problem and worrying about the same thing. At least
changing this behavior would help things like supervisord.
hbase-deamon.sh creates/updates pid file even when that start failed.
-
Key: HBASE-6970
URL: https://issues.apache.org/jira/browse/HBASE-6970
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS $HBASE_HOME/bin/hbase \
--config ${HBASE_CONF_DIR} \
$command $@ $startStop $logout 21 /dev/null
echo $! $pid
{code}
So the pid file is create or update even the start of the service failed. The
next stop command will then fail, because the pid file has he wrong pid in it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5525) Truncate and preserve region boundaries option
[
https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472967#comment-13472967
]
Hadoop QA commented on HBASE-5525:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548513/HBASE-5525.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestFromClientSide
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3025//console
This message is automatically generated.
Truncate and preserve region boundaries option
--
Key: HBASE-5525
URL: https://issues.apache.org/jira/browse/HBASE-5525
Project: HBase
Issue Type: New Feature
Affects Versions: 0.96.0
Reporter: Jean-Daniel Cryans
Assignee: Kevin Odell
Labels: newbie, noob
Fix For: 0.96.0
Attachments: HBASE-5525.patch
A tool that would be useful for testing (and maybe in prod too) would be a
truncate option to keep the current region boundaries. Right now what you
have to do is completely kill the table and recreate it with the correct
regions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
[
https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-6970:
-
Description:
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS $HBASE_HOME/bin/hbase \
--config ${HBASE_CONF_DIR} \
$command $@ $startStop $logout 21 /dev/null
echo $! $pid
{code}
So the pid file is created or updated even when the start of the service
failed. The next stop command will then fail, because the pid file has he wrong
pid in it.
Edit: Spelling
was:
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS $HBASE_HOME/bin/hbase \
--config ${HBASE_CONF_DIR} \
$command $@ $startStop $logout 21 /dev/null
echo $! $pid
{code}
So the pid file is create or update even the start of the service failed. The
next stop command will then fail, because the pid file has he wrong pid in it.
hbase-deamon.sh creates/updates pid file even when that start failed.
-
Key: HBASE-6970
URL: https://issues.apache.org/jira/browse/HBASE-6970
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS $HBASE_HOME/bin/hbase \
--config ${HBASE_CONF_DIR} \
$command $@ $startStop $logout 21 /dev/null
echo $! $pid
{code}
So the pid file is created or updated even when the start of the service
failed. The next stop command will then fail, because the pid file has he
wrong pid in it.
Edit: Spelling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6970) hbase-deamon.sh creates/updates pid file even when that start failed.
[
https://issues.apache.org/jira/browse/HBASE-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-6970:
-
Description:
We just ran into a strange issue where could neither start nor stop services
with hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS $HBASE_HOME/bin/hbase \
--config ${HBASE_CONF_DIR} \
$command $@ $startStop $logout 21 /dev/null
echo $! $pid
{code}
So the pid file is created or updated even when the start of the service
failed. The next stop command will then fail, because the pid file has the
wrong pid in it.
Edit: Spelling and more spelling errors.
was:
We just ran into a strange issue where could start or stop services with
hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS $HBASE_HOME/bin/hbase \
--config ${HBASE_CONF_DIR} \
$command $@ $startStop $logout 21 /dev/null
echo $! $pid
{code}
So the pid file is created or updated even when the start of the service
failed. The next stop command will then fail, because the pid file has he wrong
pid in it.
Edit: Spelling
hbase-deamon.sh creates/updates pid file even when that start failed.
-
Key: HBASE-6970
URL: https://issues.apache.org/jira/browse/HBASE-6970
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
We just ran into a strange issue where could neither start nor stop services
with hbase-deamon.sh.
The problem is this:
{code}
nohup nice -n $HBASE_NICENESS $HBASE_HOME/bin/hbase \
--config ${HBASE_CONF_DIR} \
$command $@ $startStop $logout 21 /dev/null
echo $! $pid
{code}
So the pid file is created or updated even when the start of the service
failed. The next stop command will then fail, because the pid file has the
wrong pid in it.
Edit: Spelling and more spelling errors.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6853) IllegalArgument Exception is thrown when an empty region is spliitted.
[
https://issues.apache.org/jira/browse/HBASE-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472969#comment-13472969
]
stack commented on HBASE-6853:
--
+1 on committing addendum and +1 on adding a wait time on the test.
IllegalArgument Exception is thrown when an empty region is spliitted.
--
Key: HBASE-6853
URL: https://issues.apache.org/jira/browse/HBASE-6853
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1, 0.94.1
Reporter: ramkrishna.s.vasudevan
Assignee: Priyadarshini
Fix For: 0.94.2, 0.96.0
Attachments: HBASE-6853_2_splitsuccess.patch,
HBASE-6853_addendum.patch, HBASE-6853.patch, HBASE-6853_splitfailure.patch
This is w.r.t a mail sent in the dev mail list.
Empty region split should be handled gracefully. Either we should not allow
the split to happen if we know that the region is empty or we should allow
the split to happen by setting the no of threads to the thread pool executor
as 1.
{code}
int nbFiles = hstoreFilesToSplit.size();
ThreadFactoryBuilder builder = new ThreadFactoryBuilder();
builder.setNameFormat(StoreFileSplitter-%1$d);
ThreadFactory factory = builder.build();
ThreadPoolExecutor threadPool =
(ThreadPoolExecutor) Executors.newFixedThreadPool(nbFiles, factory);
ListFutureVoid futures = new ArrayListFutureVoid(nbFiles);
{code}
Here the nbFiles needs to be a non zero positive value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6969) Allow for injecting MiniZookeeperCluster instance
[ https://issues.apache.org/jira/browse/HBASE-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472973#comment-13472973 ] stack commented on HBASE-6969: -- As written, at end of tests, won't your passed in zkcluster be shutdown? That is probably not what you want? Should there be a getMiniZookeeperCluster to answer your added setMiniZookeeperCluster? Is baseZKCluster necessary? Why not just an internal flag which has whether or not HBaseTestingUtility started the zk cluster? If we didn't start it, we shouldn't stop it on the way out? Otherwise, looks like useful functionality to add. Thanks Micah. Allow for injecting MiniZookeeperCluster instance - Key: HBASE-6969 URL: https://issues.apache.org/jira/browse/HBASE-6969 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.1 Reporter: Micah Whitacre Attachments: hbase-6969.patch While not an official part of the HBase API, it'd be nice if HBaseTestingUtility allowed for consumer to inject their own instance of the MiniZookeeperCluster when executing tests. Currently there is no way to control how the ZK instance is started which we've had to hack around when there were ZK version conflicts (3.3 vs 3.4). Or if you want to control which specific port it starts on vs random. Allowing consumers to inject an instance (unstarted) gives them the freedom to implement their own without having to fork the entire HBaseTestingUtility class -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6302) Document how to run integration tests
[ https://issues.apache.org/jira/browse/HBASE-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472975#comment-13472975 ] stack commented on HBASE-6302: -- Any luck w/ this one Enis? Document how to run integration tests - Key: HBASE-6302 URL: https://issues.apache.org/jira/browse/HBASE-6302 Project: HBase Issue Type: Sub-task Components: documentation Reporter: stack Assignee: Enis Soztutar Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-6302_v1.patch HBASE-6203 has attached the old IT doc with some mods. When we figure how ITs are to be run, update it and apply the documentation under this issue. Making a blocker against 0.96. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6733) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2]
[ https://issues.apache.org/jira/browse/HBASE-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-6733: -- Resolution: Fixed Fix Version/s: (was: 0.92.3) 0.96.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk, thanks for your patch and your patience Devaraj. [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2] --- Key: HBASE-6733 URL: https://issues.apache.org/jira/browse/HBASE-6733 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 6733-1.patch, 6733-2.patch, 6733-3.patch The failure is in TestReplication.queueFailover (fails due to unreplicated rows). I have come across two problems: 1. The sleepMultiplier is not properly reset when the currentPath is changed (in ReplicationSource.java). 2. ReplicationExecutor sometime removes files to replicate from the queue too early, resulting in corresponding edits missing. Here the problem is due to the fact the log-file length that the replication executor finds is not the most updated one, and hence it doesn't read anything from there, and ultimately, when there is a log roll, the replication-queue gets a new entry, and the executor drops the old entry out of the queue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6971) .META. directory does not contain .tableinfo serialization
Enis Soztutar created HBASE-6971: Summary: .META. directory does not contain .tableinfo serialization Key: HBASE-6971 URL: https://issues.apache.org/jira/browse/HBASE-6971 Project: HBase Issue Type: Bug Components: Filesystem Integration, master Affects Versions: 0.94.3, 0.96.0 Reporter: Enis Soztutar Assignee: Enis Soztutar This is not very critical, but I've noticed that we do not serialize the HTableDescriptor to hbase.root.dir/.META./.tableinfo, although we do it for ROOT. MasterFileSystem.createRootTableInfo() should probably have an META equivalent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6853) IllegalArgument Exception is thrown when an empty region is spliitted.
[
https://issues.apache.org/jira/browse/HBASE-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472977#comment-13472977
]
ramkrishna.s.vasudevan commented on HBASE-6853:
---
Ok, I shall commit this later today with a wait time on test.
@Lars/Stack
You mind committing it?
IllegalArgument Exception is thrown when an empty region is spliitted.
--
Key: HBASE-6853
URL: https://issues.apache.org/jira/browse/HBASE-6853
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1, 0.94.1
Reporter: ramkrishna.s.vasudevan
Assignee: Priyadarshini
Fix For: 0.94.2, 0.96.0
Attachments: HBASE-6853_2_splitsuccess.patch,
HBASE-6853_addendum.patch, HBASE-6853.patch, HBASE-6853_splitfailure.patch
This is w.r.t a mail sent in the dev mail list.
Empty region split should be handled gracefully. Either we should not allow
the split to happen if we know that the region is empty or we should allow
the split to happen by setting the no of threads to the thread pool executor
as 1.
{code}
int nbFiles = hstoreFilesToSplit.size();
ThreadFactoryBuilder builder = new ThreadFactoryBuilder();
builder.setNameFormat(StoreFileSplitter-%1$d);
ThreadFactory factory = builder.build();
ThreadPoolExecutor threadPool =
(ThreadPoolExecutor) Executors.newFixedThreadPool(nbFiles, factory);
ListFutureVoid futures = new ArrayListFutureVoid(nbFiles);
{code}
Here the nbFiles needs to be a non zero positive value.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6968) Several HBase write perf improvement
[
https://issues.apache.org/jira/browse/HBASE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472978#comment-13472978
]
ramkrishna.s.vasudevan commented on HBASE-6968:
---
Nice one.
Several HBase write perf improvement
Key: HBASE-6968
URL: https://issues.apache.org/jira/browse/HBASE-6968
Project: HBase
Issue Type: Improvement
Reporter: Liyin Tang
Here are 2 hbase write performance improvements recently:
1) Avoid creating HBaseConfiguraiton object for each HLog. Every time when
creating a HBaseConfiguraiton object, it would parse the xml configuration
files from disk, which is not cheap operation.
In HLog.java:
orig:
{code:title=HLog.java}
newWriter = createWriter(fs, newPath, HBaseConfiguration.create(conf));
{code}
new:
{code}
newWriter = createWriter(fs, newPath, conf);
{code}
2) Change 2 hotspot synchronized functions into double locking pattern. So it
shall remove the synchronization overhead in the normal case.
orig:
{code:title=HBaseRpcMetrics.java}
public synchronized void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
m = create(name);
}
m.inc(amt);
}
{code}
new:
{code}
public void inc(String name, int amt) {
MetricsTimeVaryingRate m = get(name);
if (m == null) {
synchronized (this) {
if ((m = get(name)) == null) {
m = create(name);
}
}
}
m.inc(amt);
}
{code}
=
orig:
{code:title=MemStoreFlusher.java}
public synchronized void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {
flushSomeRegions();
}
}
{code}
new:
{code}
public void reclaimMemStoreMemory() {
if (this.server.getGlobalMemstoreSize().get() = globalMemStoreLimit) {
flushSomeRegions();
}
}
private synchronized void flushSomeRegions() {
if (this.server.getGlobalMemstoreSize().get() globalMemStoreLimit) {
return; // double check the global memstore size inside of the
synchronized block.
}
...
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6758) [replication] The replication-executor should make sure the file that it is replicating is closed before declaring success on that file
[ https://issues.apache.org/jira/browse/HBASE-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472982#comment-13472982 ] Jean-Daniel Cryans commented on HBASE-6758: --- The last time I played around postLogRoll in HBASE-3515, I found that we must ensure that we have that log up in ZK before we start writing to it because it would be possible for writers to append and at the same time not be able to add the log in ZK because the session timed out. The current code blocks talking to ZK. [replication] The replication-executor should make sure the file that it is replicating is closed before declaring success on that file --- Key: HBASE-6758 URL: https://issues.apache.org/jira/browse/HBASE-6758 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 6758-1-0.92.patch, 6758-2-0.92.patch, 6758-trunk-1.patch, 6758-trunk-2.patch, 6758-trunk-3.patch, TEST-org.apache.hadoop.hbase.replication.TestReplication.xml I have seen cases where the replication-executor would lose data to replicate since the file hasn't been closed yet. Upon closing, the new data becomes visible. Before that happens the ZK node shouldn't be deleted in ReplicationSourceManager.logPositionAndCleanOldLogs. Changes need to be made in ReplicationSource.processEndOfFile as well (currentPath related). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6758) [replication] The replication-executor should make sure the file that it is replicating is closed before declaring success on that file
[ https://issues.apache.org/jira/browse/HBASE-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473005#comment-13473005 ] Devaraj Das commented on HBASE-6758: Hey [~jdcryans], this patch doesn't change that behavior at all (new log is put up in ZK before the log is being written to, and blocks talking to ZK..). This patch only changes the postLogRoll placement and that deterministically ensures the previous log file is really closed before enqueuing the new log for replication. The code changes in the replicator thread (ReplicationSource.java) makes sure that the entire iteration of the loop sees a closed log file at least once (and hence takes care of the problem reported in the jira). [replication] The replication-executor should make sure the file that it is replicating is closed before declaring success on that file --- Key: HBASE-6758 URL: https://issues.apache.org/jira/browse/HBASE-6758 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 6758-1-0.92.patch, 6758-2-0.92.patch, 6758-trunk-1.patch, 6758-trunk-2.patch, 6758-trunk-3.patch, TEST-org.apache.hadoop.hbase.replication.TestReplication.xml I have seen cases where the replication-executor would lose data to replicate since the file hasn't been closed yet. Upon closing, the new data becomes visible. Before that happens the ZK node shouldn't be deleted in ReplicationSourceManager.logPositionAndCleanOldLogs. Changes need to be made in ReplicationSource.processEndOfFile as well (currentPath related). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6733) [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2]
[ https://issues.apache.org/jira/browse/HBASE-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473012#comment-13473012 ] Hudson commented on HBASE-6733: --- Integrated in HBase-TRUNK #3440 (See [https://builds.apache.org/job/HBase-TRUNK/3440/]) HBASE-6733 TestReplication.queueFailover occasionally fails [Part-2] (Devaraj Das via JD) (Revision 1396463) Result = SUCCESS jdcryans : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2] --- Key: HBASE-6733 URL: https://issues.apache.org/jira/browse/HBASE-6733 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 6733-1.patch, 6733-2.patch, 6733-3.patch The failure is in TestReplication.queueFailover (fails due to unreplicated rows). I have come across two problems: 1. The sleepMultiplier is not properly reset when the currentPath is changed (in ReplicationSource.java). 2. ReplicationExecutor sometime removes files to replicate from the queue too early, resulting in corresponding edits missing. Here the problem is due to the fact the log-file length that the replication executor finds is not the most updated one, and hence it doesn't read anything from there, and ultimately, when there is a log roll, the replication-queue gets a new entry, and the executor drops the old entry out of the queue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
