[jira] [Commented] (HBASE-8077) Configure the job name in ImportTsv
[ https://issues.apache.org/jira/browse/HBASE-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13604889#comment-13604889 ] Hudson commented on HBASE-8077: --- Integrated in HBase-TRUNK #3967 (See [https://builds.apache.org/job/HBase-TRUNK/3967/]) HBASE-8077 Configure the job name in ImportTsv (Revision 1457632) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java Configure the job name in ImportTsv --- Key: HBASE-8077 URL: https://issues.apache.org/jira/browse/HBASE-8077 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: udy brill Priority: Minor Labels: importtsv, mapreduce Fix For: 0.95.0, 0.98.0 Attachments: ImportTsv_HBASE_8077.patch Original Estimate: 48h Remaining Estimate: 48h Currently when using the ImportTsv tool, the name of the mapreduce job is hard coded and cannot be change (importtsv_tableName) it would be nice if this was the default, but could be change. an idea is: bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=a,b,c -Dmapred.job.name=newJobName tablename hdfs-inputdir -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8081) Backport HBASE-7213 (separate hlog for meta tables) to 0.94
[ https://issues.apache.org/jira/browse/HBASE-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13604894#comment-13604894 ] Devaraj Das commented on HBASE-8081: [~lhofhansl], Thanks. I'll get the test run with the option enabled. Let me poke around a bit with this patch, and when I am satisfied, it'll be great to run it on your test cluster. Thanks [~yuzhih...@gmail.com], for running the test suite. Backport HBASE-7213 (separate hlog for meta tables) to 0.94 --- Key: HBASE-8081 URL: https://issues.apache.org/jira/browse/HBASE-8081 Project: HBase Issue Type: Bug Affects Versions: 0.94.5 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.94.7 Attachments: 7213-0.94-2.patch, 7213-0.94-3.patch, 7213-0.94.patch, 7213-0.94-with-config.patch I am interested in backporting HBASE-7213 to 0.94. Helps to address more of the MTTR story. Offline discussion with Lars indicated he is interested as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8077) Configure the job name in ImportTsv
[ https://issues.apache.org/jira/browse/HBASE-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13604899#comment-13604899 ] Hudson commented on HBASE-8077: --- Integrated in hbase-0.95 #81 (See [https://builds.apache.org/job/hbase-0.95/81/]) HBASE-8077 Configure the job name in ImportTsv (Revision 1457633) Result = SUCCESS stack : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java Configure the job name in ImportTsv --- Key: HBASE-8077 URL: https://issues.apache.org/jira/browse/HBASE-8077 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: udy brill Priority: Minor Labels: importtsv, mapreduce Fix For: 0.95.0, 0.98.0 Attachments: ImportTsv_HBASE_8077.patch Original Estimate: 48h Remaining Estimate: 48h Currently when using the ImportTsv tool, the name of the mapreduce job is hard coded and cannot be change (importtsv_tableName) it would be nice if this was the default, but could be change. an idea is: bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=a,b,c -Dmapred.job.name=newJobName tablename hdfs-inputdir -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8089) Add type support
[ https://issues.apache.org/jira/browse/HBASE-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13604944#comment-13604944 ] Anoop Sam John commented on HBASE-8089: --- ImportTSV is a very good tool used for bulk loading. We can add a type support for this tool also. When the MR reads the file lines and convert it into bytes to store into HBase, this type can be considered. We can have a sub issue for that also? Add type support Key: HBASE-8089 URL: https://issues.apache.org/jira/browse/HBASE-8089 Project: HBase Issue Type: New Feature Components: Client Reporter: Nick Dimiduk Attachments: HBASE-8089-types.txt This proposal outlines an improvement to HBase that provides for a set of types, above and beyond the existing byte-bucket strategy. This is intended to reduce user-level duplication of effort, provide better support for 3rd-party integration, and provide an overall improved experience for developers using HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8128) HTable#put improvements
[ https://issues.apache.org/jira/browse/HBASE-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13604970#comment-13604970 ] nkeywal commented on HBASE-8128: bq. Sound simple and efficient... Will it be possible to have it for 0.94 too? The patch should be directly applicable, so I can do it. As you and Lars want. bq. These classes could do with a general revamp. Agreed. I'm actually studying this currently. Committed in trunk and 0.95, thanks for the reviews! HTable#put improvements --- Key: HBASE-8128 URL: https://issues.apache.org/jira/browse/HBASE-8128 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.95.0, 0.96.0 Attachments: 8128.v1.patch 3 points: - When doing a single put, we're creating an object by calling Arrays.asList - we're doing a size check every 10 put. Not doing it seems simpler, better and allows to share some code between a single put and a list of puts. - we could call flushCommits on empty write buffer, especially for someone using a lot of HTable instead of using a pool, as it's called in close(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8128) HTable#put improvements
[ https://issues.apache.org/jira/browse/HBASE-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-8128: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) HTable#put improvements --- Key: HBASE-8128 URL: https://issues.apache.org/jira/browse/HBASE-8128 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.95.0, 0.96.0 Attachments: 8128.v1.patch 3 points: - When doing a single put, we're creating an object by calling Arrays.asList - we're doing a size check every 10 put. Not doing it seems simpler, better and allows to share some code between a single put and a list of puts. - we could call flushCommits on empty write buffer, especially for someone using a lot of HTable instead of using a pool, as it's called in close(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4955) Use the official versions of surefire junit
[ https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13604981#comment-13604981 ] nkeywal commented on HBASE-4955: bq. Tests run: 35, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 281.479 sec Useless log lines. That's SUREFIRE-969 bq. Took 2 mo. 7 d. That's SUREFIRE-970. bq. 2,158 tests (-274) It should be the flakiness of TestHTableMultiplexer.testHTableMultiplexer I'm going to retry because of the point 3. For the first 2 ones, I tend to think it should not prevent us from committing. We don't have any issue today because I built a version that included all we need. If we want to come back to an official version, we need to compromise. We can expect these points are likely to be solved in a later version, but these later version can also include regressions.. We need to jump in at a moment, and we've been waiting for more than a year now. Use the official versions of surefire junit - Key: HBASE-4955 URL: https://issues.apache.org/jira/browse/HBASE-4955 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 4955.v1.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch We currently use private versions for Surefire JUnit since HBASE-4763. This JIRA traks what we need to move to official versions. Surefire 2.11 is just out, but, after some tests, it does not contain all what we need. JUnit. Could be for JUnit 4.11. Issue to monitor: https://github.com/KentBeck/junit/issues/359: fixed in our version, no feedback for an integration on trunk Surefire: Could be for Surefire 2.12. Issues to monitor are: 329 (category support): fixed, we use the official implementation from the trunk 786 (@Category with forkMode=always): fixed, we use the official implementation from the trunk 791 (incorrect elapsed time on test failure): fixed, we use the official implementation from the trunk 793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on our version. 760 (does not take into account the test method): fixed in trunk, not fixed in our version 798 (print immediately the test class name): not fixed in trunk, not fixed in our version 799 (Allow test parallelization when forkMode=always): not fixed in trunk, not fixed in our version 800 (redirectTestOutputToFile not taken into account): not yet fix on trunk, fixed on our version 800 793 are the more important to monitor, it's the only ones that are fixed in our version but not on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4955) Use the official versions of surefire junit
[ https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-4955: --- Status: Open (was: Patch Available) Use the official versions of surefire junit - Key: HBASE-4955 URL: https://issues.apache.org/jira/browse/HBASE-4955 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 4955.v1.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch We currently use private versions for Surefire JUnit since HBASE-4763. This JIRA traks what we need to move to official versions. Surefire 2.11 is just out, but, after some tests, it does not contain all what we need. JUnit. Could be for JUnit 4.11. Issue to monitor: https://github.com/KentBeck/junit/issues/359: fixed in our version, no feedback for an integration on trunk Surefire: Could be for Surefire 2.12. Issues to monitor are: 329 (category support): fixed, we use the official implementation from the trunk 786 (@Category with forkMode=always): fixed, we use the official implementation from the trunk 791 (incorrect elapsed time on test failure): fixed, we use the official implementation from the trunk 793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on our version. 760 (does not take into account the test method): fixed in trunk, not fixed in our version 798 (print immediately the test class name): not fixed in trunk, not fixed in our version 799 (Allow test parallelization when forkMode=always): not fixed in trunk, not fixed in our version 800 (redirectTestOutputToFile not taken into account): not yet fix on trunk, fixed on our version 800 793 are the more important to monitor, it's the only ones that are fixed in our version but not on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4955) Use the official versions of surefire junit
[ https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-4955: --- Attachment: 4955.v2.patch Use the official versions of surefire junit - Key: HBASE-4955 URL: https://issues.apache.org/jira/browse/HBASE-4955 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 4955.v1.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch We currently use private versions for Surefire JUnit since HBASE-4763. This JIRA traks what we need to move to official versions. Surefire 2.11 is just out, but, after some tests, it does not contain all what we need. JUnit. Could be for JUnit 4.11. Issue to monitor: https://github.com/KentBeck/junit/issues/359: fixed in our version, no feedback for an integration on trunk Surefire: Could be for Surefire 2.12. Issues to monitor are: 329 (category support): fixed, we use the official implementation from the trunk 786 (@Category with forkMode=always): fixed, we use the official implementation from the trunk 791 (incorrect elapsed time on test failure): fixed, we use the official implementation from the trunk 793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on our version. 760 (does not take into account the test method): fixed in trunk, not fixed in our version 798 (print immediately the test class name): not fixed in trunk, not fixed in our version 799 (Allow test parallelization when forkMode=always): not fixed in trunk, not fixed in our version 800 (redirectTestOutputToFile not taken into account): not yet fix on trunk, fixed on our version 800 793 are the more important to monitor, it's the only ones that are fixed in our version but not on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4955) Use the official versions of surefire junit
[ https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-4955: --- Status: Patch Available (was: Open) Use the official versions of surefire junit - Key: HBASE-4955 URL: https://issues.apache.org/jira/browse/HBASE-4955 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 4955.v1.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch We currently use private versions for Surefire JUnit since HBASE-4763. This JIRA traks what we need to move to official versions. Surefire 2.11 is just out, but, after some tests, it does not contain all what we need. JUnit. Could be for JUnit 4.11. Issue to monitor: https://github.com/KentBeck/junit/issues/359: fixed in our version, no feedback for an integration on trunk Surefire: Could be for Surefire 2.12. Issues to monitor are: 329 (category support): fixed, we use the official implementation from the trunk 786 (@Category with forkMode=always): fixed, we use the official implementation from the trunk 791 (incorrect elapsed time on test failure): fixed, we use the official implementation from the trunk 793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on our version. 760 (does not take into account the test method): fixed in trunk, not fixed in our version 798 (print immediately the test class name): not fixed in trunk, not fixed in our version 799 (Allow test parallelization when forkMode=always): not fixed in trunk, not fixed in our version 800 (redirectTestOutputToFile not taken into account): not yet fix on trunk, fixed on our version 800 793 are the more important to monitor, it's the only ones that are fixed in our version but not on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8135) Mutation should implement HeapSize
nkeywal created HBASE-8135: -- Summary: Mutation should implement HeapSize Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.94.5, 0.95.0, 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-8135: --- Status: Patch Available (was: Open) Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.94.5, 0.95.0, 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-8135: --- Attachment: 8135.v1.patch Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0, 0.94.5 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4955) Use the official versions of surefire junit
[
https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605024#comment-13605024
]
Hadoop QA commented on HBASE-4955:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574127/4955.v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4861//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4861//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4861//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4861//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4861//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4861//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4861//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4861//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4861//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4861//console
This message is automatically generated.
Use the official versions of surefire junit
-
Key: HBASE-4955
URL: https://issues.apache.org/jira/browse/HBASE-4955
Project: HBase
Issue Type: Improvement
Components: test
Affects Versions: 0.94.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Fix For: 0.96.0
Attachments: 4955.v1.patch, 4955.v2.patch, 4955.v2.patch,
4955.v2.patch, 4955.v2.patch
We currently use private versions for Surefire JUnit since HBASE-4763.
This JIRA traks what we need to move to official versions.
Surefire 2.11 is just out, but, after some tests, it does not contain all
what we need.
JUnit. Could be for JUnit 4.11. Issue to monitor:
https://github.com/KentBeck/junit/issues/359: fixed in our version, no
feedback for an integration on trunk
Surefire: Could be for Surefire 2.12. Issues to monitor are:
329 (category support): fixed, we use the official implementation from the
trunk
786 (@Category with forkMode=always): fixed, we use the official
implementation from the trunk
791 (incorrect elapsed time on test failure): fixed, we use the official
implementation from the trunk
793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on
our version.
760 (does not take into account the test method): fixed in trunk, not fixed
in our version
798 (print immediately the test class name): not fixed in trunk, not fixed in
our version
799 (Allow test parallelization when forkMode=always): not fixed in trunk,
not fixed in our version
800 (redirectTestOutputToFile not taken into account): not yet fix on trunk,
fixed on our version
800 793 are the more important to monitor, it's the only ones that are
fixed in our version but not on trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8128) HTable#put improvements
[ https://issues.apache.org/jira/browse/HBASE-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605040#comment-13605040 ] Hudson commented on HBASE-8128: --- Integrated in HBase-TRUNK #3968 (See [https://builds.apache.org/job/HBase-TRUNK/3968/]) HBASE-8128 HTable#put improvements (Revision 1457688) Result = FAILURE nkeywal : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java HTable#put improvements --- Key: HBASE-8128 URL: https://issues.apache.org/jira/browse/HBASE-8128 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.95.0, 0.96.0 Attachments: 8128.v1.patch 3 points: - When doing a single put, we're creating an object by calling Arrays.asList - we're doing a size check every 10 put. Not doing it seems simpler, better and allows to share some code between a single put and a list of puts. - we could call flushCommits on empty write buffer, especially for someone using a lot of HTable instead of using a pool, as it's called in close(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7590) Add a costless notifications mechanism from master to regionservers clients
[ https://issues.apache.org/jira/browse/HBASE-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-7590: --- Attachment: 7590.v12.patch Add a costless notifications mechanism from master to regionservers clients - Key: HBASE-7590 URL: https://issues.apache.org/jira/browse/HBASE-7590 Project: HBase Issue Type: Bug Components: Client, master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 7590.inprogress.patch, 7590.v12.patch, 7590.v12.patch, 7590.v1.patch, 7590.v1-rebased.patch, 7590.v2.patch, 7590.v3.patch, 7590.v5.patch, 7590.v5.patch t would be very useful to add a mechanism to distribute some information to the clients and regionservers. Especially It would be useful to know globally (regionservers + clients apps) that some regionservers are dead. This would allow: - to lower the load on the system, without clients using staled information and going on dead machines - to make the recovery faster from a client point of view. It's common to use large timeouts on the client side, so the client may need a lot of time before declaring a region server dead and trying another one. If the client receives the information separatly about a region server states, it can take the right decision, and continue/stop to wait accordingly. We can also send more information, for example instructions like 'slow down' to instruct the client to increase the retries delay and so on. Technically, the master could send this information. To lower the load on the system, we should: - have a multicast communication (i.e. the master does not have to connect to all servers by tcp), with once packet every 10 seconds or so. - receivers should not depend on this: if the information is available great. If not, it should not break anything. - it should be optional. So at the end we would have a thread in the master sending a protobuf message about the dead servers on a multicast socket. If the socket is not configured, it does not do anything. On the client side, when we receive an information that a node is dead, we refresh the cache about it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7590) Add a costless notifications mechanism from master to regionservers clients
[ https://issues.apache.org/jira/browse/HBASE-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-7590: --- Status: Open (was: Patch Available) Add a costless notifications mechanism from master to regionservers clients - Key: HBASE-7590 URL: https://issues.apache.org/jira/browse/HBASE-7590 Project: HBase Issue Type: Bug Components: Client, master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 7590.inprogress.patch, 7590.v12.patch, 7590.v12.patch, 7590.v1.patch, 7590.v1-rebased.patch, 7590.v2.patch, 7590.v3.patch, 7590.v5.patch, 7590.v5.patch t would be very useful to add a mechanism to distribute some information to the clients and regionservers. Especially It would be useful to know globally (regionservers + clients apps) that some regionservers are dead. This would allow: - to lower the load on the system, without clients using staled information and going on dead machines - to make the recovery faster from a client point of view. It's common to use large timeouts on the client side, so the client may need a lot of time before declaring a region server dead and trying another one. If the client receives the information separatly about a region server states, it can take the right decision, and continue/stop to wait accordingly. We can also send more information, for example instructions like 'slow down' to instruct the client to increase the retries delay and so on. Technically, the master could send this information. To lower the load on the system, we should: - have a multicast communication (i.e. the master does not have to connect to all servers by tcp), with once packet every 10 seconds or so. - receivers should not depend on this: if the information is available great. If not, it should not break anything. - it should be optional. So at the end we would have a thread in the master sending a protobuf message about the dead servers on a multicast socket. If the socket is not configured, it does not do anything. On the client side, when we receive an information that a node is dead, we refresh the cache about it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7590) Add a costless notifications mechanism from master to regionservers clients
[ https://issues.apache.org/jira/browse/HBASE-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-7590: --- Attachment: 7590.v12.patch Add a costless notifications mechanism from master to regionservers clients - Key: HBASE-7590 URL: https://issues.apache.org/jira/browse/HBASE-7590 Project: HBase Issue Type: Bug Components: Client, master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 7590.inprogress.patch, 7590.v12.patch, 7590.v12.patch, 7590.v1.patch, 7590.v1-rebased.patch, 7590.v2.patch, 7590.v3.patch, 7590.v5.patch, 7590.v5.patch t would be very useful to add a mechanism to distribute some information to the clients and regionservers. Especially It would be useful to know globally (regionservers + clients apps) that some regionservers are dead. This would allow: - to lower the load on the system, without clients using staled information and going on dead machines - to make the recovery faster from a client point of view. It's common to use large timeouts on the client side, so the client may need a lot of time before declaring a region server dead and trying another one. If the client receives the information separatly about a region server states, it can take the right decision, and continue/stop to wait accordingly. We can also send more information, for example instructions like 'slow down' to instruct the client to increase the retries delay and so on. Technically, the master could send this information. To lower the load on the system, we should: - have a multicast communication (i.e. the master does not have to connect to all servers by tcp), with once packet every 10 seconds or so. - receivers should not depend on this: if the information is available great. If not, it should not break anything. - it should be optional. So at the end we would have a thread in the master sending a protobuf message about the dead servers on a multicast socket. If the socket is not configured, it does not do anything. On the client side, when we receive an information that a node is dead, we refresh the cache about it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7590) Add a costless notifications mechanism from master to regionservers clients
[ https://issues.apache.org/jira/browse/HBASE-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-7590: --- Status: Patch Available (was: Open) Add a costless notifications mechanism from master to regionservers clients - Key: HBASE-7590 URL: https://issues.apache.org/jira/browse/HBASE-7590 Project: HBase Issue Type: Bug Components: Client, master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 7590.inprogress.patch, 7590.v12.patch, 7590.v12.patch, 7590.v1.patch, 7590.v1-rebased.patch, 7590.v2.patch, 7590.v3.patch, 7590.v5.patch, 7590.v5.patch t would be very useful to add a mechanism to distribute some information to the clients and regionservers. Especially It would be useful to know globally (regionservers + clients apps) that some regionservers are dead. This would allow: - to lower the load on the system, without clients using staled information and going on dead machines - to make the recovery faster from a client point of view. It's common to use large timeouts on the client side, so the client may need a lot of time before declaring a region server dead and trying another one. If the client receives the information separatly about a region server states, it can take the right decision, and continue/stop to wait accordingly. We can also send more information, for example instructions like 'slow down' to instruct the client to increase the retries delay and so on. Technically, the master could send this information. To lower the load on the system, we should: - have a multicast communication (i.e. the master does not have to connect to all servers by tcp), with once packet every 10 seconds or so. - receivers should not depend on this: if the information is available great. If not, it should not break anything. - it should be optional. So at the end we would have a thread in the master sending a protobuf message about the dead servers on a multicast socket. If the socket is not configured, it does not do anything. On the client side, when we receive an information that a node is dead, we refresh the cache about it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7590) Add a costless notifications mechanism from master to regionservers clients
[ https://issues.apache.org/jira/browse/HBASE-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605046#comment-13605046 ] nkeywal commented on HBASE-7590: v12 with the comments on RB from Devaraj taken into account. Nearly there! Add a costless notifications mechanism from master to regionservers clients - Key: HBASE-7590 URL: https://issues.apache.org/jira/browse/HBASE-7590 Project: HBase Issue Type: Bug Components: Client, master, regionserver Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 7590.inprogress.patch, 7590.v12.patch, 7590.v12.patch, 7590.v1.patch, 7590.v1-rebased.patch, 7590.v2.patch, 7590.v3.patch, 7590.v5.patch, 7590.v5.patch t would be very useful to add a mechanism to distribute some information to the clients and regionservers. Especially It would be useful to know globally (regionservers + clients apps) that some regionservers are dead. This would allow: - to lower the load on the system, without clients using staled information and going on dead machines - to make the recovery faster from a client point of view. It's common to use large timeouts on the client side, so the client may need a lot of time before declaring a region server dead and trying another one. If the client receives the information separatly about a region server states, it can take the right decision, and continue/stop to wait accordingly. We can also send more information, for example instructions like 'slow down' to instruct the client to increase the retries delay and so on. Technically, the master could send this information. To lower the load on the system, we should: - have a multicast communication (i.e. the master does not have to connect to all servers by tcp), with once packet every 10 seconds or so. - receivers should not depend on this: if the information is available great. If not, it should not break anything. - it should be optional. So at the end we would have a thread in the master sending a protobuf message about the dead servers on a multicast socket. If the socket is not configured, it does not do anything. On the client side, when we receive an information that a node is dead, we refresh the cache about it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8077) Configure the job name in ImportTsv
[ https://issues.apache.org/jira/browse/HBASE-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605054#comment-13605054 ] Hudson commented on HBASE-8077: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #452 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/452/]) HBASE-8077 Configure the job name in ImportTsv (Revision 1457632) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java Configure the job name in ImportTsv --- Key: HBASE-8077 URL: https://issues.apache.org/jira/browse/HBASE-8077 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: udy brill Priority: Minor Labels: importtsv, mapreduce Fix For: 0.95.0, 0.98.0 Attachments: ImportTsv_HBASE_8077.patch Original Estimate: 48h Remaining Estimate: 48h Currently when using the ImportTsv tool, the name of the mapreduce job is hard coded and cannot be change (importtsv_tableName) it would be nice if this was the default, but could be change. an idea is: bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=a,b,c -Dmapred.job.name=newJobName tablename hdfs-inputdir -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[
https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605056#comment-13605056
]
Hadoop QA commented on HBASE-8135:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574131/8135.v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4862//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4862//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4862//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4862//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4862//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4862//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4862//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4862//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4862//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4862//console
This message is automatically generated.
Mutation should implement HeapSize
--
Key: HBASE-8135
URL: https://issues.apache.org/jira/browse/HBASE-8135
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.95.0, 0.96.0, 0.94.5
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Fix For: 0.95.0, 0.96.0
Attachments: 8135.v1.patch
Code is there already.
Doing so would allow to share some code when doing client side buffering.
patch compiles locally, should not impact tests, but not tested locally.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8128) HTable#put improvements
[ https://issues.apache.org/jira/browse/HBASE-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605055#comment-13605055 ] Hudson commented on HBASE-8128: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #452 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/452/]) HBASE-8128 HTable#put improvements (Revision 1457688) Result = FAILURE nkeywal : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java HTable#put improvements --- Key: HBASE-8128 URL: https://issues.apache.org/jira/browse/HBASE-8128 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.95.0, 0.96.0 Attachments: 8128.v1.patch 3 points: - When doing a single put, we're creating an object by calling Arrays.asList - we're doing a size check every 10 put. Not doing it seems simpler, better and allows to share some code between a single put and a list of puts. - we could call flushCommits on empty write buffer, especially for someone using a lot of HTable instead of using a pool, as it's called in close(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8077) Configure the job name in ImportTsv
[ https://issues.apache.org/jira/browse/HBASE-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605070#comment-13605070 ] Hudson commented on HBASE-8077: --- Integrated in hbase-0.95-on-hadoop2 #31 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/31/]) HBASE-8077 Configure the job name in ImportTsv (Revision 1457633) Result = FAILURE stack : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java Configure the job name in ImportTsv --- Key: HBASE-8077 URL: https://issues.apache.org/jira/browse/HBASE-8077 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: udy brill Priority: Minor Labels: importtsv, mapreduce Fix For: 0.95.0, 0.98.0 Attachments: ImportTsv_HBASE_8077.patch Original Estimate: 48h Remaining Estimate: 48h Currently when using the ImportTsv tool, the name of the mapreduce job is hard coded and cannot be change (importtsv_tableName) it would be nice if this was the default, but could be change. an idea is: bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=a,b,c -Dmapred.job.name=newJobName tablename hdfs-inputdir -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8128) HTable#put improvements
[ https://issues.apache.org/jira/browse/HBASE-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605074#comment-13605074 ] Jean-Marc Spaggiari commented on HBASE-8128: bq. The patch should be directly applicable, so I can do it. As you and Lars want. Since it's not a new feature and it's an interesting cleanup, I think we should. [~lhofhansl] do you have any objection with that? HTable#put improvements --- Key: HBASE-8128 URL: https://issues.apache.org/jira/browse/HBASE-8128 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.95.0, 0.96.0 Attachments: 8128.v1.patch 3 points: - When doing a single put, we're creating an object by calling Arrays.asList - we're doing a size check every 10 put. Not doing it seems simpler, better and allows to share some code between a single put and a list of puts. - we could call flushCommits on empty write buffer, especially for someone using a lot of HTable instead of using a pool, as it's called in close(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7590) Add a costless notifications mechanism from master to regionservers clients
[
https://issues.apache.org/jira/browse/HBASE-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605084#comment-13605084
]
Hadoop QA commented on HBASE-7590:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574136/7590.v12.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 32 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4863//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4863//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4863//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4863//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4863//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4863//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4863//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4863//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4863//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4863//console
This message is automatically generated.
Add a costless notifications mechanism from master to regionservers clients
-
Key: HBASE-7590
URL: https://issues.apache.org/jira/browse/HBASE-7590
Project: HBase
Issue Type: Bug
Components: Client, master, regionserver
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Attachments: 7590.inprogress.patch, 7590.v12.patch, 7590.v12.patch,
7590.v1.patch, 7590.v1-rebased.patch, 7590.v2.patch, 7590.v3.patch,
7590.v5.patch, 7590.v5.patch
t would be very useful to add a mechanism to distribute some information to
the clients and regionservers. Especially It would be useful to know globally
(regionservers + clients apps) that some regionservers are dead. This would
allow:
- to lower the load on the system, without clients using staled information
and going on dead machines
- to make the recovery faster from a client point of view. It's common to use
large timeouts on the client side, so the client may need a lot of time
before declaring a region server dead and trying another one. If the client
receives the information separatly about a region server states, it can take
the right decision, and continue/stop to wait accordingly.
We can also send more information, for example instructions like 'slow down'
to instruct the client to increase the retries delay and so on.
Technically, the master could send this information. To lower the load on
the system, we should:
- have a multicast communication (i.e. the master does not have to connect to
all servers by tcp), with once packet every 10 seconds or so.
- receivers should not depend on this: if the information is available great.
If not, it should not break anything.
- it should be optional.
So at the end we would have a thread in the master sending a protobuf message
about the dead servers on a multicast socket. If the socket is not
configured, it does not do anything. On the client side, when we receive an
information that a node is dead,
[jira] [Commented] (HBASE-7910) Dont use reflection for security
[ https://issues.apache.org/jira/browse/HBASE-7910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605136#comment-13605136 ] Amir Sanjar commented on HBASE-7910: Removing reflection from UserGroupInformation (IBM JVM vs SUN JVM login section) will break hadoop on any platforms that SUN/Oracle JVM is not supported - i.e. POWERFPC and S390x. The real solution will be to develop hadoop security component that is independent of platform and JVM vendor. Dont use reflection for security Key: HBASE-7910 URL: https://issues.apache.org/jira/browse/HBASE-7910 Project: HBase Issue Type: Improvement Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.95.0 security.User class uses reflection so that HBase can work with older Hadoop's not having security. Now that we require 1.x, or 0.23 or 2.x, all Hadoop versions have security code. We can get rid of most of the User class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4955) Use the official versions of surefire junit
[
https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605139#comment-13605139
]
nkeywal commented on HBASE-4955:
Likely a bad news... Among the missing tests, we have this:
@RunWith(Parameterized.class)
@Category(SmallTests.class)
public class TestFixedFileTrailer {
i.e. there could be issues with parametized tests (and that could not be enough
to explain the 200 missing tests).
Looking...
Use the official versions of surefire junit
-
Key: HBASE-4955
URL: https://issues.apache.org/jira/browse/HBASE-4955
Project: HBase
Issue Type: Improvement
Components: test
Affects Versions: 0.94.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Fix For: 0.96.0
Attachments: 4955.v1.patch, 4955.v2.patch, 4955.v2.patch,
4955.v2.patch, 4955.v2.patch
We currently use private versions for Surefire JUnit since HBASE-4763.
This JIRA traks what we need to move to official versions.
Surefire 2.11 is just out, but, after some tests, it does not contain all
what we need.
JUnit. Could be for JUnit 4.11. Issue to monitor:
https://github.com/KentBeck/junit/issues/359: fixed in our version, no
feedback for an integration on trunk
Surefire: Could be for Surefire 2.12. Issues to monitor are:
329 (category support): fixed, we use the official implementation from the
trunk
786 (@Category with forkMode=always): fixed, we use the official
implementation from the trunk
791 (incorrect elapsed time on test failure): fixed, we use the official
implementation from the trunk
793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on
our version.
760 (does not take into account the test method): fixed in trunk, not fixed
in our version
798 (print immediately the test class name): not fixed in trunk, not fixed in
our version
799 (Allow test parallelization when forkMode=always): not fixed in trunk,
not fixed in our version
800 (redirectTestOutputToFile not taken into account): not yet fix on trunk,
fixed on our version
800 793 are the more important to monitor, it's the only ones that are
fixed in our version but not on trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[
https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605176#comment-13605176
]
Ted Yu commented on HBASE-8135:
---
{code}
- long heapSize() {
+ public long heapSize() {
{code}
Add @Override.
Mutation should implement HeapSize
--
Key: HBASE-8135
URL: https://issues.apache.org/jira/browse/HBASE-8135
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.95.0, 0.96.0, 0.94.5
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Fix For: 0.95.0, 0.96.0
Attachments: 8135.v1.patch
Code is there already.
Doing so would allow to share some code when doing client side buffering.
patch compiles locally, should not impact tests, but not tested locally.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605183#comment-13605183 ] nkeywal commented on HBASE-8135: Agreed, I will do that on commit. Are you +1 otherwise? Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0, 0.94.5 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion().
[ https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605204#comment-13605204 ] ramkrishna.s.vasudevan commented on HBASE-7992: --- +1 provide pre/post region offline hooks for HMaster.offlineRegion(). --- Key: HBASE-7992 URL: https://issues.apache.org/jira/browse/HBASE-7992 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 0.95.0 Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.98.0 Attachments: HBASE-7992_trunk_2.patch, HBASE-7992_trunk.patch presently no hooks to provide access control to offline region in master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4955) Use the official versions of surefire junit
[ https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605224#comment-13605224 ] nkeywal commented on HBASE-4955: It's fun, because if I do: mvn clean test -Dsurefire.part2.skip=true -q -PrunAllTests -Dsurefire.part1.forkCount=10 the number of executed tests is a random number above 600 while with mvn clean test -Dsurefire.part2.skip=true -q -PrunAllTests -Dsurefire.part1.forkCount=1 It's always 543 more parallism == less randomness (logic) but less tests executed (not logic) I don't reproduce it on a surefire unit tests. I'm going to try a little bit more then we will have the option to wait for 2.15, hoping it will be identified and fixed. Use the official versions of surefire junit - Key: HBASE-4955 URL: https://issues.apache.org/jira/browse/HBASE-4955 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 4955.v1.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch We currently use private versions for Surefire JUnit since HBASE-4763. This JIRA traks what we need to move to official versions. Surefire 2.11 is just out, but, after some tests, it does not contain all what we need. JUnit. Could be for JUnit 4.11. Issue to monitor: https://github.com/KentBeck/junit/issues/359: fixed in our version, no feedback for an integration on trunk Surefire: Could be for Surefire 2.12. Issues to monitor are: 329 (category support): fixed, we use the official implementation from the trunk 786 (@Category with forkMode=always): fixed, we use the official implementation from the trunk 791 (incorrect elapsed time on test failure): fixed, we use the official implementation from the trunk 793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on our version. 760 (does not take into account the test method): fixed in trunk, not fixed in our version 798 (print immediately the test class name): not fixed in trunk, not fixed in our version 799 (Allow test parallelization when forkMode=always): not fixed in trunk, not fixed in our version 800 (redirectTestOutputToFile not taken into account): not yet fix on trunk, fixed on our version 800 793 are the more important to monitor, it's the only ones that are fixed in our version but not on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8131) Create table handler needs to handle failure cases.
[ https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8131: -- Status: Open (was: Patch Available) Create table handler needs to handle failure cases. --- Key: HBASE-8131 URL: https://issues.apache.org/jira/browse/HBASE-8131 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk.patch In CreateTable Handler there are number of failure cases. IOExceptions are common while creation of regioninfos, htableDescriptors etc. After this exception if i try to recreate the table using admin, we need to remove the acquired table lock and also clear the ZKTable in memory cache so that the operation can be retried. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8131) Create table handler needs to handle failure cases.
[ https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8131: -- Attachment: HBASE-8131_trunk_1.patch Updated patch with a test case that mocks the CreateTableHandler. Create table handler needs to handle failure cases. --- Key: HBASE-8131 URL: https://issues.apache.org/jira/browse/HBASE-8131 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk.patch In CreateTable Handler there are number of failure cases. IOExceptions are common while creation of regioninfos, htableDescriptors etc. After this exception if i try to recreate the table using admin, we need to remove the acquired table lock and also clear the ZKTable in memory cache so that the operation can be retried. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4955) Use the official versions of surefire junit
[ https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605239#comment-13605239 ] stack commented on HBASE-4955: -- [~nkeywal] Thanks for staying on top of this one. So, we only host surefire now? It is in Gary's repo or in yours? Use the official versions of surefire junit - Key: HBASE-4955 URL: https://issues.apache.org/jira/browse/HBASE-4955 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 4955.v1.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch We currently use private versions for Surefire JUnit since HBASE-4763. This JIRA traks what we need to move to official versions. Surefire 2.11 is just out, but, after some tests, it does not contain all what we need. JUnit. Could be for JUnit 4.11. Issue to monitor: https://github.com/KentBeck/junit/issues/359: fixed in our version, no feedback for an integration on trunk Surefire: Could be for Surefire 2.12. Issues to monitor are: 329 (category support): fixed, we use the official implementation from the trunk 786 (@Category with forkMode=always): fixed, we use the official implementation from the trunk 791 (incorrect elapsed time on test failure): fixed, we use the official implementation from the trunk 793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on our version. 760 (does not take into account the test method): fixed in trunk, not fixed in our version 798 (print immediately the test class name): not fixed in trunk, not fixed in our version 799 (Allow test parallelization when forkMode=always): not fixed in trunk, not fixed in our version 800 (redirectTestOutputToFile not taken into account): not yet fix on trunk, fixed on our version 800 793 are the more important to monitor, it's the only ones that are fixed in our version but not on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8131) Create table handler needs to handle failure cases.
[ https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8131: -- Status: Patch Available (was: Open) Create table handler needs to handle failure cases. --- Key: HBASE-8131 URL: https://issues.apache.org/jira/browse/HBASE-8131 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk.patch In CreateTable Handler there are number of failure cases. IOExceptions are common while creation of regioninfos, htableDescriptors etc. After this exception if i try to recreate the table using admin, we need to remove the acquired table lock and also clear the ZKTable in memory cache so that the operation can be retried. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8014) Backport HBASE-6915 to 0.94.
[ https://issues.apache.org/jira/browse/HBASE-8014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605242#comment-13605242 ] Ted Yu commented on HBASE-8014: --- Patch looks good. Backport HBASE-6915 to 0.94. Key: HBASE-8014 URL: https://issues.apache.org/jira/browse/HBASE-8014 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Critical Attachments: HBASE-8014-v0-0.94.patch JDK 1.7 changed some data size. Goal of this JIRA is to backport HBASE-6915 to 0.94 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605244#comment-13605244 ] stack commented on HBASE-8135: -- +1 I think I added heapsize method to Mutation when I made Increment a Mutation; didn't add the interface because wasn't sure it was needed generally. Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0, 0.94.5 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605253#comment-13605253 ] Ted Yu commented on HBASE-8135: --- Increment inherits heapSize() from Mutation. Increment has a field of TimeRange which is not accounted for in Mutation#heapSize() Should we account for this field in Increment#heapSize() ? Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0, 0.94.5 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8136) coprocessor service requires .meta. to be available all the time.
nkeywal created HBASE-8136: -- Summary: coprocessor service requires .meta. to be available all the time. Key: HBASE-8136 URL: https://issues.apache.org/jira/browse/HBASE-8136 Project: HBase Issue Type: Bug Components: Client, Coprocessors Affects Versions: 0.96.0 Reporter: nkeywal Priority: Minor HTable#getRegionLocations does not use a cache: all the calls to this function go to .META. So: - we're missing an opportunity to reuse/update the location cache in the HConnection. - this method is called by the coprocessor service. So, for people using this features, they have .meta. on their execution path, and it's not good for performances, scalability and reliability. I'm not totally clear on the fix. I think it should be possible to use the cache to see if we have all regions for the table. But it means we won't always have the last version when calling getRegionLocations. Any thought on this? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8128) HTable#put improvements
[ https://issues.apache.org/jira/browse/HBASE-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605262#comment-13605262 ] stack commented on HBASE-8128: -- bq. Agreed. I'm actually studying this currently. + Too many Interfaces/Classes implemented (Row, Mutation, CellScanner, etc.) + Deletes and Gets are the same because they hold coordinates only, no data. + Put and Result are same because they hold Cells/KeyValues. + Mutation cuts across the two groupings because Deletes and Put 'mutate'. + High-level, these should be client-side only classes I'd say. Over in the server we should never materialize them; protobufs are rich in accessors and setters and we have to materialize them server-side anyways so we might as well pass them down into the server (rather than do the protobuf object conversion to these client-side Put/Delete, etc. POJOs, first, and then pass them into the server). + CellScanner Interface was just added to these objects to faciliate rpc work. Ideally, these Put/Delete, objects become Cell vesssels only. When they get to the rpc edge, we will ask them for the Cell content and we'll put the Put/Delete metadata on the wire apart from their Cell content. Would be coolio if when you added an item, it was made into a Cell client-side and the Cell was put into a CellBlock hosted by the Put or Increment, optionally compressing as we went If you want more input to ignore (smile), just ask [~nkeywal] HTable#put improvements --- Key: HBASE-8128 URL: https://issues.apache.org/jira/browse/HBASE-8128 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.95.0, 0.96.0 Attachments: 8128.v1.patch 3 points: - When doing a single put, we're creating an object by calling Arrays.asList - we're doing a size check every 10 put. Not doing it seems simpler, better and allows to share some code between a single put and a list of puts. - we could call flushCommits on empty write buffer, especially for someone using a lot of HTable instead of using a pool, as it's called in close(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8136) coprocessor service requires .meta. to be available all the time.
[ https://issues.apache.org/jira/browse/HBASE-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605265#comment-13605265 ] Ted Yu commented on HBASE-8136: --- Looks like HBASE-6870 was talking about the same thing. coprocessor service requires .meta. to be available all the time. - Key: HBASE-8136 URL: https://issues.apache.org/jira/browse/HBASE-8136 Project: HBase Issue Type: Bug Components: Client, Coprocessors Affects Versions: 0.96.0 Reporter: nkeywal Priority: Minor HTable#getRegionLocations does not use a cache: all the calls to this function go to .META. So: - we're missing an opportunity to reuse/update the location cache in the HConnection. - this method is called by the coprocessor service. So, for people using this features, they have .meta. on their execution path, and it's not good for performances, scalability and reliability. I'm not totally clear on the fix. I think it should be possible to use the cache to see if we have all regions for the table. But it means we won't always have the last version when calling getRegionLocations. Any thought on this? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7905) Add passing of optional cell blocks over rpc
[ https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605267#comment-13605267 ] stack commented on HBASE-7905: -- [~ted_yu] Class comment says * Callable that handles multi method calls against regionservers. RegionServerCallable instead? And make it clear its multi against one server? Let me fix the not applying. Any change in trunk breaks this patch that spans the code base. Add passing of optional cell blocks over rpc Key: HBASE-7905 URL: https://issues.apache.org/jira/browse/HBASE-7905 Project: HBase Issue Type: Sub-task Components: IPC/RPC Reporter: stack Assignee: stack Fix For: 0.95.0 Attachments: 7900v12-depends-on-8101.txt, 7905.txt, 7905v13.txt, 7905v14.txt, 7905v15.txt, 7905v3.txt, 7905v4.txt, 7905v6.txt, 7905v8.txt, 7905v9.txt Make it so we can pass Cells/data w/o having to bury it all in protobuf to get it over the wire. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[
https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605268#comment-13605268
]
Matteo Bertozzi commented on HBASE-5583:
{quote}
Could we consider make creating table as a transaction, e.g. create files in
tmp directory, atomic put regions to .META.?
{quote}
We have already that, the problem is that META and the filesystem are two
separate operations... so you may fail in the middle and have META but not the
fs dir moved...
The idea of the file-tracking table (mentioned in others jira, e.g. HBASE-7806)
was exactly to solve the problem and having a single source of truth... and
have the table creation/deletion as transaction
Master restart on create table with splitkeys does not recreate table with
all the splitkey regions
---
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.95.0
- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of
regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But
table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism
on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8136) coprocessor service requires .meta. to be available all the time.
[ https://issues.apache.org/jira/browse/HBASE-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605269#comment-13605269 ] Andrew Purtell commented on HBASE-8136: --- Close this as a dup N? coprocessor service requires .meta. to be available all the time. - Key: HBASE-8136 URL: https://issues.apache.org/jira/browse/HBASE-8136 Project: HBase Issue Type: Bug Components: Client, Coprocessors Affects Versions: 0.96.0 Reporter: nkeywal Priority: Minor HTable#getRegionLocations does not use a cache: all the calls to this function go to .META. So: - we're missing an opportunity to reuse/update the location cache in the HConnection. - this method is called by the coprocessor service. So, for people using this features, they have .meta. on their execution path, and it's not good for performances, scalability and reliability. I'm not totally clear on the fix. I think it should be possible to use the cache to see if we have all regions for the table. But it means we won't always have the last version when calling getRegionLocations. Any thought on this? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7905) Add passing of optional cell blocks over rpc
[ https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605272#comment-13605272 ] Ted Yu commented on HBASE-7905: --- bq. RegionServerCallable instead? I think ServerCallable and RegionServerCallable are essentially the same. How about ServerCallableWithMulti ? Add passing of optional cell blocks over rpc Key: HBASE-7905 URL: https://issues.apache.org/jira/browse/HBASE-7905 Project: HBase Issue Type: Sub-task Components: IPC/RPC Reporter: stack Assignee: stack Fix For: 0.95.0 Attachments: 7900v12-depends-on-8101.txt, 7905.txt, 7905v13.txt, 7905v14.txt, 7905v15.txt, 7905v3.txt, 7905v4.txt, 7905v6.txt, 7905v8.txt, 7905v9.txt Make it so we can pass Cells/data w/o having to bury it all in protobuf to get it over the wire. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8136) coprocessor service requires .meta. to be available all the time.
[ https://issues.apache.org/jira/browse/HBASE-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal resolved HBASE-8136. Resolution: Duplicate Release Note: HBASE-6870 And the good news if that there is already a patch for HBASE-6870 :-). coprocessor service requires .meta. to be available all the time. - Key: HBASE-8136 URL: https://issues.apache.org/jira/browse/HBASE-8136 Project: HBase Issue Type: Bug Components: Client, Coprocessors Affects Versions: 0.96.0 Reporter: nkeywal Priority: Minor HTable#getRegionLocations does not use a cache: all the calls to this function go to .META. So: - we're missing an opportunity to reuse/update the location cache in the HConnection. - this method is called by the coprocessor service. So, for people using this features, they have .meta. on their execution path, and it's not good for performances, scalability and reliability. I'm not totally clear on the fix. I think it should be possible to use the cache to see if we have all regions for the table. But it means we won't always have the last version when calling getRegionLocations. Any thought on this? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-6870: --- Affects Version/s: 0.96.0 0.95.0 HTable#coprocessorExec always scan the whole table --- Key: HBASE-6870 URL: https://issues.apache.org/jira/browse/HBASE-6870 Project: HBase Issue Type: Improvement Components: Coprocessors Affects Versions: 0.94.1, 0.95.0, 0.96.0 Reporter: chunhui shen Assignee: chunhui shen Attachments: HBASE-6870.patch, HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch In current logic, HTable#coprocessorExec always scan the whole table, its efficiency is low and will affect the Regionserver carrying .META. under large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605323#comment-13605323 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- @Matteo My plan was to maintain the state of the operation in ZK or HDFS. Can i refresh the patch that i have and submit for review? Or you feel that will not be needed at all. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7905) Add passing of optional cell blocks over rpc
[ https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7905: - Attachment: 7905v16.txt I added more to the MultiServerCallable class comment to better explain what class is about. Add passing of optional cell blocks over rpc Key: HBASE-7905 URL: https://issues.apache.org/jira/browse/HBASE-7905 Project: HBase Issue Type: Sub-task Components: IPC/RPC Reporter: stack Assignee: stack Fix For: 0.95.0 Attachments: 7900v12-depends-on-8101.txt, 7905.txt, 7905v13.txt, 7905v14.txt, 7905v15.txt, 7905v16.txt, 7905v3.txt, 7905v4.txt, 7905v6.txt, 7905v8.txt, 7905v9.txt Make it so we can pass Cells/data w/o having to bury it all in protobuf to get it over the wire. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7910) Dont use reflection for security
[
https://issues.apache.org/jira/browse/HBASE-7910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605335#comment-13605335
]
Gary Helmling commented on HBASE-7910:
--
[~asanjar] This issue does not imply any changes to {{UserGroupInformation}},
nor could it. That is a Hadoop class and would be a Hadoop issue.
HBase has its own {{User}} class as a wrapper around {{UserGroupInformation}}.
This class uses reflection to ride over backwards incompatible API changes to
{{UserGroupInformation}} between pre-secure Hadoop and secure Hadoop. Since
HBase 0.95 requires a version of Hadoop already supporting security, we can
eliminate reflection in the {{User}} class that was previously used to check
for the API changes.
Dont use reflection for security
Key: HBASE-7910
URL: https://issues.apache.org/jira/browse/HBASE-7910
Project: HBase
Issue Type: Improvement
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.95.0
security.User class uses reflection so that HBase can work with older
Hadoop's not having security. Now that we require 1.x, or 0.23 or 2.x, all
Hadoop versions have security code. We can get rid of most of the User class.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[
https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605336#comment-13605336
]
Matteo Bertozzi commented on HBASE-5583:
{quote}
My plan was to maintain the state of the operation in ZK or HDFS. Can i refresh
the patch that i have and submit for review?
Or you feel that will not be needed at all.
{quote}
There's a long way to the file-tracking table,
so I'm +1 to have an alternative now based on a ZK or HDFS state
Master restart on create table with splitkeys does not recreate table with
all the splitkey regions
---
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.95.0
- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of
regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But
table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism
on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion().
[ https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7992: -- Attachment: 7992_trunk_3.patch provide pre/post region offline hooks for HMaster.offlineRegion(). --- Key: HBASE-7992 URL: https://issues.apache.org/jira/browse/HBASE-7992 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 0.95.0 Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.98.0 Attachments: 7992_trunk_3.patch, HBASE-7992_trunk_2.patch, HBASE-7992_trunk.patch presently no hooks to provide access control to offline region in master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion().
[ https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605338#comment-13605338 ] Ted Yu commented on HBASE-7992: --- Patch v3 fixes indentation and one syntax error. provide pre/post region offline hooks for HMaster.offlineRegion(). --- Key: HBASE-7992 URL: https://issues.apache.org/jira/browse/HBASE-7992 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 0.95.0 Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.98.0 Attachments: 7992_trunk_3.patch, HBASE-7992_trunk_2.patch, HBASE-7992_trunk.patch presently no hooks to provide access control to offline region in master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion()
[ https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7992: -- Summary: provide pre/post region offline hooks for HMaster.offlineRegion() (was: provide pre/post region offline hooks for HMaster.offlineRegion(). ) provide pre/post region offline hooks for HMaster.offlineRegion() - Key: HBASE-7992 URL: https://issues.apache.org/jira/browse/HBASE-7992 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 0.95.0 Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.98.0 Attachments: 7992_trunk_3.patch, HBASE-7992_trunk_2.patch, HBASE-7992_trunk.patch presently no hooks to provide access control to offline region in master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605341#comment-13605341 ] stack commented on HBASE-8135: -- [~ted_yu] Not in this patch. Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0, 0.94.5 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion()
[ https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605342#comment-13605342 ] Ted Yu commented on HBASE-7992: --- Integrated to trunk. Thanks for the patch, Rajesh. Thanks for the reviews, Andy, Ram and Anoop. provide pre/post region offline hooks for HMaster.offlineRegion() - Key: HBASE-7992 URL: https://issues.apache.org/jira/browse/HBASE-7992 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 0.95.0 Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.98.0 Attachments: 7992_trunk_3.patch, HBASE-7992_trunk_2.patch, HBASE-7992_trunk.patch presently no hooks to provide access control to offline region in master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion()
[ https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605347#comment-13605347 ] Anoop Sam John commented on HBASE-7992: --- Thanks Ted for integrating this. I was abt to commit this :) Do we need this in 0.95? provide pre/post region offline hooks for HMaster.offlineRegion() - Key: HBASE-7992 URL: https://issues.apache.org/jira/browse/HBASE-7992 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 0.95.0 Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.98.0 Attachments: 7992_trunk_3.patch, HBASE-7992_trunk_2.patch, HBASE-7992_trunk.patch presently no hooks to provide access control to offline region in master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion()
[
https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605349#comment-13605349
]
Hadoop QA commented on HBASE-7992:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574167/7992_trunk_3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4866//console
This message is automatically generated.
provide pre/post region offline hooks for HMaster.offlineRegion()
-
Key: HBASE-7992
URL: https://issues.apache.org/jira/browse/HBASE-7992
Project: HBase
Issue Type: Bug
Components: Coprocessors
Affects Versions: 0.95.0
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0
Attachments: 7992_trunk_3.patch, HBASE-7992_trunk_2.patch,
HBASE-7992_trunk.patch
presently no hooks to provide access control to offline region in master.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8134) delete qualified tombstones during minor compaction
[ https://issues.apache.org/jira/browse/HBASE-8134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605351#comment-13605351 ] Sergey Shelukhin commented on HBASE-8134: - To improve common case (no weird TS manipulations), we can move earliest put ts calculation from compactor to policy, to consider all files for it, including the ones left out. In that case we can determine during file selection what the drop deletes cutoff is. It is safe, and in common case will allow minor compaction of oldest files to drop deletes. delete qualified tombstones during minor compaction --- Key: HBASE-8134 URL: https://issues.apache.org/jira/browse/HBASE-8134 Project: HBase Issue Type: Brainstorming Components: Compaction Affects Versions: 0.98.0 Reporter: Liang Xie see http://mail-archives.apache.org/mod_mbox/hbase-dev/201303.mbox/%3c1363284331.32934.yahoomail...@web140606.mail.bf1.yahoo.com%3E for previous discussion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605358#comment-13605358
]
Jeffrey Zhong commented on HBASE-8127:
--
{quote}
Ideally it should not happen we need to dig more into this when it is possible.
If possible can you give some details about this?
{quote}
I think we need to agree if RIT in opening state could exist for a disabled
table. If that could not happen, we can adjust the test case to remove the
opening RITs of disabled table(or set to the table to disabling) and we don't
need to do extra work.
One time when I saw the opening RIT stuck is due to the offlineDisabledRegion
function in assignment manager. As you can see we don't handle opening RIT
inside the function.
{code}
public void offlineDisabledRegion(HRegionInfo regionInfo) {
...
try {
if (!ZKAssign.deleteClosedNode(watcher, regionInfo.getEncodedName())) {
// Could also be in OFFLINE mode
ZKAssign.deleteOfflineNode(watcher, regionInfo.getEncodedName());
}
...
}
{code}
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.94.7
Attachments: HBASE-8127_feedback.patch, HBASE-8127.patch,
hbase-8127_v1.patch, reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8104) HBase consistency and availability after replication
[ https://issues.apache.org/jira/browse/HBASE-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605360#comment-13605360 ] Chris Trezzo commented on HBASE-8104: - [~brianhbase] is your intent to make replication synchronous across HBase clusters? HBase consistency and availability after replication Key: HBASE-8104 URL: https://issues.apache.org/jira/browse/HBASE-8104 Project: HBase Issue Type: New Feature Affects Versions: 0.94.3 Reporter: Brian Fu Priority: Critical Original Estimate: 336h Remaining Estimate: 336h HBase consistency and availability after replication Scene as follows: 1. There are two HBase clusters are the Master clusters and Slave Clusters. two clusters replication function is open. 2. if master cluster have problems, so all write and read request switching to the slave cluster. 3. After a period of time ,we need to switch back to the Master cluster, there will be a part of the data is inconsistent, lead to this part of the data is not available. This feature is particularly important for providing online services HBase cluster. So, I want through a write-back program to keep the data consistency, then to improve HBase availability. we will provide a patch for this function. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8034) record on-disk data size for store file and make it available during writing
[ https://issues.apache.org/jira/browse/HBASE-8034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8034: Status: Open (was: Patch Available) record on-disk data size for store file and make it available during writing Key: HBASE-8034 URL: https://issues.apache.org/jira/browse/HBASE-8034 Project: HBase Issue Type: Task Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Attachments: HBASE-8034-v0.patch, HBASE-8034-v1.patch, HBASE-8034-v2.patch To better estimate the size of data in the file, and to be able to split files intelligently during any multi-file compactor like stripe or level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8034) record on-disk data size for store file and make it available during writing
[ https://issues.apache.org/jira/browse/HBASE-8034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin resolved HBASE-8034. - Resolution: Won't Fix record on-disk data size for store file and make it available during writing Key: HBASE-8034 URL: https://issues.apache.org/jira/browse/HBASE-8034 Project: HBase Issue Type: Task Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Priority: Minor Attachments: HBASE-8034-v0.patch, HBASE-8034-v1.patch, HBASE-8034-v2.patch To better estimate the size of data in the file, and to be able to split files intelligently during any multi-file compactor like stripe or level. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[
https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605362#comment-13605362
]
nkeywal commented on HBASE-8135:
I was trying to find a generic way to get the size of an object. A google
search on this leads to quite a lot of terrible practises :-). It should be
possible to do a static{} block for the fixed fields, but it won't bring much
actual value. With the current implementation, it's better to have unit tests
when ones adds fields. I'm going to do this in this patch (including Increment)
it will be simpler.
Mutation should implement HeapSize
--
Key: HBASE-8135
URL: https://issues.apache.org/jira/browse/HBASE-8135
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.95.0, 0.96.0, 0.94.5
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Fix For: 0.95.0, 0.96.0
Attachments: 8135.v1.patch
Code is there already.
Doing so would allow to share some code when doing client side buffering.
patch compiles locally, should not impact tests, but not tested locally.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8131) Create table handler needs to handle failure cases.
[
https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605366#comment-13605366
]
Hadoop QA commented on HBASE-8131:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574159/HBASE-8131_trunk_1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4864//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4864//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4864//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4864//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4864//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4864//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4864//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4864//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4864//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4864//console
This message is automatically generated.
Create table handler needs to handle failure cases.
---
Key: HBASE-8131
URL: https://issues.apache.org/jira/browse/HBASE-8131
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk.patch
In CreateTable Handler there are number of failure cases.
IOExceptions are common while creation of regioninfos, htableDescriptors etc.
After this exception if i try to recreate the table using admin, we need to
remove the acquired table lock and also clear the ZKTable in memory cache so
that the operation can be retried.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8014) Backport HBASE-6915 to 0.94.
[
https://issues.apache.org/jira/browse/HBASE-8014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605368#comment-13605368
]
Ted Yu commented on HBASE-8014:
---
On jdk 1.6:
{code}
Tests run: 1326, Failures: 0, Errors: 0, Skipped: 13
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 1:03:51.438s
[INFO] Finished at: Mon Mar 18 17:07:18 UTC 2013
{code}
Backport HBASE-6915 to 0.94.
Key: HBASE-8014
URL: https://issues.apache.org/jira/browse/HBASE-8014
Project: HBase
Issue Type: Bug
Reporter: Jean-Marc Spaggiari
Assignee: Jean-Marc Spaggiari
Priority: Critical
Attachments: HBASE-8014-v0-0.94.patch
JDK 1.7 changed some data size. Goal of this JIRA is to backport HBASE-6915
to 0.94
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8131) Create table handler needs to handle failure cases.
[ https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605371#comment-13605371 ] ramkrishna.s.vasudevan commented on HBASE-8131: --- This change and snapshot is related? Oh!! Let me check that too. Create table handler needs to handle failure cases. --- Key: HBASE-8131 URL: https://issues.apache.org/jira/browse/HBASE-8131 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk.patch In CreateTable Handler there are number of failure cases. IOExceptions are common while creation of regioninfos, htableDescriptors etc. After this exception if i try to recreate the table using admin, we need to remove the acquired table lock and also clear the ZKTable in memory cache so that the operation can be retried. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8131) Create table handler needs to handle failure cases.
[ https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605372#comment-13605372 ] Matteo Bertozzi commented on HBASE-8131: not looked at the error, but maybe it's just because the CloneSnapshotHandler doesn't call super? Create table handler needs to handle failure cases. --- Key: HBASE-8131 URL: https://issues.apache.org/jira/browse/HBASE-8131 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk.patch In CreateTable Handler there are number of failure cases. IOExceptions are common while creation of regioninfos, htableDescriptors etc. After this exception if i try to recreate the table using admin, we need to remove the acquired table lock and also clear the ZKTable in memory cache so that the operation can be retried. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8131) Create table handler needs to handle failure cases.
[ https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605381#comment-13605381 ] ramkrishna.s.vasudevan commented on HBASE-8131: --- Yes you are right Matteo. My previous patch actually introduced finally block where the lock was released in the latest patch i made the change in complete(). My bad. Lack of code knowledge on the snapshot side. Create table handler needs to handle failure cases. --- Key: HBASE-8131 URL: https://issues.apache.org/jira/browse/HBASE-8131 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk.patch In CreateTable Handler there are number of failure cases. IOExceptions are common while creation of regioninfos, htableDescriptors etc. After this exception if i try to recreate the table using admin, we need to remove the acquired table lock and also clear the ZKTable in memory cache so that the operation can be retried. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7905) Add passing of optional cell blocks over rpc
[
https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605386#comment-13605386
]
Hadoop QA commented on HBASE-7905:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574165/7905v16.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 55 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 javac{color}. The applied patch generated 6 javac compiler
warnings (more than the trunk's current 4 warnings).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.procedure.TestZKProcedureControllers
org.apache.hadoop.hbase.client.TestHTableMultiplexer
org.apache.hadoop.hbase.client.TestHCM
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4865//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4865//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4865//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4865//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4865//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4865//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4865//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4865//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4865//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4865//console
This message is automatically generated.
Add passing of optional cell blocks over rpc
Key: HBASE-7905
URL: https://issues.apache.org/jira/browse/HBASE-7905
Project: HBase
Issue Type: Sub-task
Components: IPC/RPC
Reporter: stack
Assignee: stack
Fix For: 0.95.0
Attachments: 7900v12-depends-on-8101.txt, 7905.txt, 7905v13.txt,
7905v14.txt, 7905v15.txt, 7905v16.txt, 7905v3.txt, 7905v4.txt, 7905v6.txt,
7905v8.txt, 7905v9.txt
Make it so we can pass Cells/data w/o having to bury it all in protobuf to
get it over the wire.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8128) HTable#put improvements
[ https://issues.apache.org/jira/browse/HBASE-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605387#comment-13605387 ] Lars Hofhansl commented on HBASE-8128: -- nice simple improvement... +1 for 0.94. HTable#put improvements --- Key: HBASE-8128 URL: https://issues.apache.org/jira/browse/HBASE-8128 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.95.0, 0.96.0 Attachments: 8128.v1.patch 3 points: - When doing a single put, we're creating an object by calling Arrays.asList - we're doing a size check every 10 put. Not doing it seems simpler, better and allows to share some code between a single put and a list of puts. - we could call flushCommits on empty write buffer, especially for someone using a lot of HTable instead of using a pool, as it's called in close(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8080) refactor default compactor to make its parts easier to reuse
[ https://issues.apache.org/jira/browse/HBASE-8080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605388#comment-13605388 ] Sergey Shelukhin commented on HBASE-8080: - Planning to commit in a few hours refactor default compactor to make its parts easier to reuse Key: HBASE-8080 URL: https://issues.apache.org/jira/browse/HBASE-8080 Project: HBase Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HBASE-8080-v0.patch, HBASE-8080-v1.patch, HBASE-8080-v2.patch, HBASE-8080-v3.patch Refactor default compactor to make its parts easier to reuse. To make eventual HBASE-7967 patch smaller. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7835) Implementation of the log splitting without creating intermediate files
[
https://issues.apache.org/jira/browse/HBASE-7835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605392#comment-13605392
]
Jeffrey Zhong commented on HBASE-7835:
--
[~anoop.hbase] Thanks for the comments and suggestions!
{quote}
Do we need a framework kind of thing within the RS for doing any RS-RS call?
{quote}
A good point. Currently I'm working on
https://issues.apache.org/jira/browse/HBASE-7836 where protobuf is used and the
code is similar as HRegionServer.replicateWALEntry for RS-RS traffic.
{quote}
This one + HBASE-6772 + multi WAL...
Can we think on all together.
{quote}
Currently I'm working on this JIRA and HBASE-6772. I agree they're related but
I think they can be developed parallelly because they're trying to improve
different phases in a RS recovery process. I like your IPC saving idea and we
can create a follow up optimization JIRA for that once current improvements are
check in because they can possibly be built on top of each other.
Implementation of the log splitting without creating intermediate files
Key: HBASE-7835
URL: https://issues.apache.org/jira/browse/HBASE-7835
Project: HBase
Issue Type: Sub-task
Components: regionserver
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.95.0
Attachments: hbase-7006_12.patch, hbase-7006_1.patch,
hbase-7006_2.patch, hbase-7006_3.patch, hbase-7006_3.patch,
hbase-7006_6.patch, hbase-7006_8.patch
The sub task is to check in a separate patch for major implementation for the
proposal.
More checkins will include a new replay command and more metrics support.
Thanks,
-Jeffrey
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8131) Create table handler needs to handle failure cases.
[ https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8131: -- Status: Open (was: Patch Available) Create table handler needs to handle failure cases. --- Key: HBASE-8131 URL: https://issues.apache.org/jira/browse/HBASE-8131 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk_2.patch, HBASE-8131_trunk.patch In CreateTable Handler there are number of failure cases. IOExceptions are common while creation of regioninfos, htableDescriptors etc. After this exception if i try to recreate the table using admin, we need to remove the acquired table lock and also clear the ZKTable in memory cache so that the operation can be retried. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8131) Create table handler needs to handle failure cases.
[ https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8131: -- Attachment: HBASE-8131_trunk_2.patch The failed test cases with this change. Added super.completed() in CloneSnapShotHandler. Create table handler needs to handle failure cases. --- Key: HBASE-8131 URL: https://issues.apache.org/jira/browse/HBASE-8131 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk_2.patch, HBASE-8131_trunk.patch In CreateTable Handler there are number of failure cases. IOExceptions are common while creation of regioninfos, htableDescriptors etc. After this exception if i try to recreate the table using admin, we need to remove the acquired table lock and also clear the ZKTable in memory cache so that the operation can be retried. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8131) Create table handler needs to handle failure cases.
[ https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8131: -- Status: Patch Available (was: Open) Create table handler needs to handle failure cases. --- Key: HBASE-8131 URL: https://issues.apache.org/jira/browse/HBASE-8131 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.98.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk_2.patch, HBASE-8131_trunk.patch In CreateTable Handler there are number of failure cases. IOExceptions are common while creation of regioninfos, htableDescriptors etc. After this exception if i try to recreate the table using admin, we need to remove the acquired table lock and also clear the ZKTable in memory cache so that the operation can be retried. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8131) Create table handler needs to handle failure cases.
[
https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605414#comment-13605414
]
Ted Yu commented on HBASE-8131:
---
{code}
+public class TestCreateTableHandler {
+ private static final HBaseTestingUtility TEST_UTIL = new
HBaseTestingUtility();
+ private static final Log LOG = LogFactory.getLog(TestMaster.class);
{code}
LOG used wrong class name.
{code}
+ public static class CustomCreateTableHandler extends CreateTableHandler {
{code}
This handler can be private, right ?
Create table handler needs to handle failure cases.
---
Key: HBASE-8131
URL: https://issues.apache.org/jira/browse/HBASE-8131
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk_2.patch,
HBASE-8131_trunk.patch
In CreateTable Handler there are number of failure cases.
IOExceptions are common while creation of regioninfos, htableDescriptors etc.
After this exception if i try to recreate the table using admin, we need to
remove the acquired table lock and also clear the ZKTable in memory cache so
that the operation can be retried.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Zhong updated HBASE-8097:
-
Attachment: hbase-8097_v3.patch
Incorporating Ted's comments.
MetaServerShutdownHandler may potentially keep bumping up
DeadServer.numProcessing
--
Key: HBASE-8097
URL: https://issues.apache.org/jira/browse/HBASE-8097
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.96.0
Attachments: 8097.txt, hbase-8097_1.patch, hbase-8097_v2.patch,
hbase-8097_v3.patch
{code}
} catch (IOException ioe) {
this.services.getExecutorService().submit(this);
this.deadServers.add(serverName);
throw new IOException(failed log splitting for +
serverName + , will retry, ioe);
}
{code}
this.deadServers.add(serverName); will keep incrementing
DeadServer.numProcessing
We can't get rid of numProcessing by just checking deadServers.size() because
deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4811) Support reverse Scan
[ https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605441#comment-13605441 ] John Carrino commented on HBASE-4811: - A common workflow for me would to be to do a forward/reverse index lookup. Basically find the first row that is after/before a given key. I think for this case both forward and reverse index lookups would be approximately the same speed. Support reverse Scan Key: HBASE-4811 URL: https://issues.apache.org/jira/browse/HBASE-4811 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.20.6 Reporter: John Carrino All the documentation I find about HBase says that if you want forward and reverse scans you should just build 2 tables and one be ascending and one descending. Is there a fundamental reason that HBase only supports forward Scan? It seems like a lot of extra space overhead and coding overhead (to keep them in sync) to support 2 tables. I am assuming this has been discussed before, but I can't find the discussions anywhere about it or why it would be infeasible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605442#comment-13605442
]
Ted Yu commented on HBASE-8097:
---
+1
MetaServerShutdownHandler may potentially keep bumping up
DeadServer.numProcessing
--
Key: HBASE-8097
URL: https://issues.apache.org/jira/browse/HBASE-8097
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.96.0
Attachments: 8097.txt, hbase-8097_1.patch, hbase-8097_v2.patch,
hbase-8097_v3.patch
{code}
} catch (IOException ioe) {
this.services.getExecutorService().submit(this);
this.deadServers.add(serverName);
throw new IOException(failed log splitting for +
serverName + , will retry, ioe);
}
{code}
this.deadServers.add(serverName); will keep incrementing
DeadServer.numProcessing
We can't get rid of numProcessing by just checking deadServers.size() because
deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8137) Add failed to open/close region state
Jimmy Xiang created HBASE-8137: -- Summary: Add failed to open/close region state Key: HBASE-8137 URL: https://issues.apache.org/jira/browse/HBASE-8137 Project: HBase Issue Type: Improvement Components: Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.95.0 Since we are going to remove assignment timeout monitor, we should add some state to show if a region failed to open/close after max retries. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8138) Using [packed=true] for repeated field of primitive numeric types (types which use the varint, 32-bit, or 64-bit wire types)
Jeffrey Zhong created HBASE-8138:
Summary: Using [packed=true] for repeated field of primitive
numeric types (types which use the varint, 32-bit, or 64-bit wire types)
Key: HBASE-8138
URL: https://issues.apache.org/jira/browse/HBASE-8138
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Priority: Trivial
Fix For: 0.98.0
It's recommended to do the following for numeric primitive types
{quote}
For historical reasons, repeated fields of basic numeric types aren't encoded
as efficiently as they could be. New code should use the special option
[packed=true] to get a more efficient encoding
{quote}
See details at https://developers.google.com/protocol-buffers/docs/proto
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8138) Using [packed=true] for repeated field of primitive numeric types (types which use the varint, 32-bit, or 64-bit wire types)
[
https://issues.apache.org/jira/browse/HBASE-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Zhong updated HBASE-8138:
-
Attachment: hbase-8138.patch
A trivial patch.
Thanks,
-Jeffrey
Using [packed=true] for repeated field of primitive numeric types (types
which use the varint, 32-bit, or 64-bit wire types)
Key: HBASE-8138
URL: https://issues.apache.org/jira/browse/HBASE-8138
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Priority: Trivial
Fix For: 0.98.0
Attachments: hbase-8138.patch
It's recommended to do the following for numeric primitive types
{quote}
For historical reasons, repeated fields of basic numeric types aren't encoded
as efficiently as they could be. New code should use the special option
[packed=true] to get a more efficient encoding
{quote}
See details at https://developers.google.com/protocol-buffers/docs/proto
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605453#comment-13605453
]
Jimmy Xiang commented on HBASE-8097:
Do you see DeadServer.numProcessing keeps bumping up?
MetaServerShutdownHandler may potentially keep bumping up
DeadServer.numProcessing
--
Key: HBASE-8097
URL: https://issues.apache.org/jira/browse/HBASE-8097
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.96.0
Attachments: 8097.txt, hbase-8097_1.patch, hbase-8097_v2.patch,
hbase-8097_v3.patch
{code}
} catch (IOException ioe) {
this.services.getExecutorService().submit(this);
this.deadServers.add(serverName);
throw new IOException(failed log splitting for +
serverName + , will retry, ioe);
}
{code}
this.deadServers.add(serverName); will keep incrementing
DeadServer.numProcessing
We can't get rid of numProcessing by just checking deadServers.size() because
deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605457#comment-13605457
]
Jimmy Xiang commented on HBASE-8097:
We have the following at the end in process:
{code}
} finally {
this.deadServers.finish(serverName);
}
{code}
Does it reduce numProcessing properly?
MetaServerShutdownHandler may potentially keep bumping up
DeadServer.numProcessing
--
Key: HBASE-8097
URL: https://issues.apache.org/jira/browse/HBASE-8097
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.96.0
Attachments: 8097.txt, hbase-8097_1.patch, hbase-8097_v2.patch,
hbase-8097_v3.patch
{code}
} catch (IOException ioe) {
this.services.getExecutorService().submit(this);
this.deadServers.add(serverName);
throw new IOException(failed log splitting for +
serverName + , will retry, ioe);
}
{code}
this.deadServers.add(serverName); will keep incrementing
DeadServer.numProcessing
We can't get rid of numProcessing by just checking deadServers.size() because
deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-8135: --- Status: Open (was: Patch Available) Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.94.5, 0.95.0, 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch, 8135.v2.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-8135: --- Attachment: 8135.v2.patch Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0, 0.94.5 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch, 8135.v2.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605470#comment-13605470 ] nkeywal commented on HBASE-8135: There is an issue: with the unit tests, I've got a huge gap I do not explain: Expected :80 Actual :168 2013-03-18 19:54:28,845 DEBUG [main] util.ClassSize(246): 0 row class [B 2013-03-18 19:54:28,845 DEBUG [main] util.ClassSize(246): 1 ts long 2013-03-18 19:54:28,846 DEBUG [main] util.ClassSize(246): 2 writeToWAL boolean 2013-03-18 19:54:28,846 DEBUG [main] util.ClassSize(246): 3 familyMap interface java.util.NavigableMap 2013-03-18 19:54:28,846 DEBUG [main] util.ClassSize(246): 4 attributes interface java.util.Map 2013-03-18 19:54:28,846 DEBUG [main] util.ClassSize(273): Primitives=9, arrays=1, references(includes 2 for object overhead)=5, refSize 8, size=80, prealign_size=73 Any hint? Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0, 0.94.5 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch, 8135.v2.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605471#comment-13605471 ] stack commented on HBASE-8135: -- Try using -d32 flag locally. Jenkins seems to do 32bit java. Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0, 0.94.5 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch, 8135.v2.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605472#comment-13605472
]
Jimmy Xiang commented on HBASE-8097:
Got it. I was looking at ServerShutdownHandler.
MetaServerShutdownHandler may potentially keep bumping up
DeadServer.numProcessing
--
Key: HBASE-8097
URL: https://issues.apache.org/jira/browse/HBASE-8097
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.96.0
Attachments: 8097.txt, hbase-8097_1.patch, hbase-8097_v2.patch,
hbase-8097_v3.patch
{code}
} catch (IOException ioe) {
this.services.getExecutorService().submit(this);
this.deadServers.add(serverName);
throw new IOException(failed log splitting for +
serverName + , will retry, ioe);
}
{code}
this.deadServers.add(serverName); will keep incrementing
DeadServer.numProcessing
We can't get rid of numProcessing by just checking deadServers.size() because
deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605475#comment-13605475 ] stack commented on HBASE-8135: -- Oh, this is just locally, no jenkins involved? You using ClassSize.align? Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0, 0.94.5 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch, 8135.v2.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8131) Create table handler needs to handle failure cases.
[
https://issues.apache.org/jira/browse/HBASE-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605476#comment-13605476
]
Hadoop QA commented on HBASE-8131:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574177/HBASE-8131_trunk_2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterFailover
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4867//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4867//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4867//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4867//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4867//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4867//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4867//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4867//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4867//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4867//console
This message is automatically generated.
Create table handler needs to handle failure cases.
---
Key: HBASE-8131
URL: https://issues.apache.org/jira/browse/HBASE-8131
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.98.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Attachments: HBASE-8131_trunk_1.patch, HBASE-8131_trunk_2.patch,
HBASE-8131_trunk.patch
In CreateTable Handler there are number of failure cases.
IOExceptions are common while creation of regioninfos, htableDescriptors etc.
After this exception if i try to recreate the table using admin, we need to
remove the acquired table lock and also clear the ZKTable in memory cache so
that the operation can be retried.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8135) Mutation should implement HeapSize
[ https://issues.apache.org/jira/browse/HBASE-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605483#comment-13605483 ] nkeywal commented on HBASE-8135: Yes, just locally (I do test before submitting :-) ) I used ClassSize.align for timerange, and it went ok. But for Put/Delete/Increment, There are 88 bytes of difference I cannot explain. The code is on the v2.patch. Mutation should implement HeapSize -- Key: HBASE-8135 URL: https://issues.apache.org/jira/browse/HBASE-8135 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.0, 0.96.0, 0.94.5 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.95.0, 0.96.0 Attachments: 8135.v1.patch, 8135.v2.patch Code is there already. Doing so would allow to share some code when doing client side buffering. patch compiles locally, should not impact tests, but not tested locally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605488#comment-13605488
]
Jimmy Xiang commented on HBASE-8097:
+1
MetaServerShutdownHandler may potentially keep bumping up
DeadServer.numProcessing
--
Key: HBASE-8097
URL: https://issues.apache.org/jira/browse/HBASE-8097
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.96.0
Attachments: 8097.txt, hbase-8097_1.patch, hbase-8097_v2.patch,
hbase-8097_v3.patch
{code}
} catch (IOException ioe) {
this.services.getExecutorService().submit(this);
this.deadServers.add(serverName);
throw new IOException(failed log splitting for +
serverName + , will retry, ioe);
}
{code}
this.deadServers.add(serverName); will keep incrementing
DeadServer.numProcessing
We can't get rid of numProcessing by just checking deadServers.size() because
deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8139) Allow job names to be overridden
Nick Dimiduk created HBASE-8139: --- Summary: Allow job names to be overridden Key: HBASE-8139 URL: https://issues.apache.org/jira/browse/HBASE-8139 Project: HBase Issue Type: Sub-task Components: mapreduce, Usability Reporter: Nick Dimiduk As a general feature, we should allow mr job names to be overridden by the user. See also HBASE-8077. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8077) Configure the job name in ImportTsv
[ https://issues.apache.org/jira/browse/HBASE-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605489#comment-13605489 ] Nick Dimiduk commented on HBASE-8077: - bq. We should probably make this a general feature of MR jobs, that you can specify job name from command line. Noted: HBASE-8139. Configure the job name in ImportTsv --- Key: HBASE-8077 URL: https://issues.apache.org/jira/browse/HBASE-8077 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: udy brill Priority: Minor Labels: importtsv, mapreduce Fix For: 0.95.0, 0.98.0 Attachments: ImportTsv_HBASE_8077.patch Original Estimate: 48h Remaining Estimate: 48h Currently when using the ImportTsv tool, the name of the mapreduce job is hard coded and cannot be change (importtsv_tableName) it would be nice if this was the default, but could be change. an idea is: bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.columns=a,b,c -Dmapred.job.name=newJobName tablename hdfs-inputdir -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605508#comment-13605508
]
Hadoop QA commented on HBASE-8097:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12574183/hbase-8097_v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4868//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4868//console
This message is automatically generated.
MetaServerShutdownHandler may potentially keep bumping up
DeadServer.numProcessing
--
Key: HBASE-8097
URL: https://issues.apache.org/jira/browse/HBASE-8097
Project: HBase
Issue Type: Bug
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.96.0
Attachments: 8097.txt, hbase-8097_1.patch, hbase-8097_v2.patch,
hbase-8097_v3.patch
{code}
} catch (IOException ioe) {
this.services.getExecutorService().submit(this);
this.deadServers.add(serverName);
throw new IOException(failed log splitting for +
serverName + , will retry, ioe);
}
{code}
this.deadServers.add(serverName); will keep incrementing
DeadServer.numProcessing
We can't get rid of numProcessing by just checking deadServers.size() because
deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4955) Use the official versions of surefire junit
[ https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605503#comment-13605503 ] nkeywal commented on HBASE-4955: Yes, JUnit 4.11 contains what we need. Surefire contains what we need as well, we just need to get a version that comes up with an acceptable set of regressions. Our Surefire is in Gary's repo. He will be the one getting the blame if Apache complains :-). BTW, I haven't done the update to JUnit in HBase 0.94, as it implies backporting a few jiras as well (in the required section). So we still need to have it in Gary's repo as well. Use the official versions of surefire junit - Key: HBASE-4955 URL: https://issues.apache.org/jira/browse/HBASE-4955 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 4955.v1.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch We currently use private versions for Surefire JUnit since HBASE-4763. This JIRA traks what we need to move to official versions. Surefire 2.11 is just out, but, after some tests, it does not contain all what we need. JUnit. Could be for JUnit 4.11. Issue to monitor: https://github.com/KentBeck/junit/issues/359: fixed in our version, no feedback for an integration on trunk Surefire: Could be for Surefire 2.12. Issues to monitor are: 329 (category support): fixed, we use the official implementation from the trunk 786 (@Category with forkMode=always): fixed, we use the official implementation from the trunk 791 (incorrect elapsed time on test failure): fixed, we use the official implementation from the trunk 793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on our version. 760 (does not take into account the test method): fixed in trunk, not fixed in our version 798 (print immediately the test class name): not fixed in trunk, not fixed in our version 799 (Allow test parallelization when forkMode=always): not fixed in trunk, not fixed in our version 800 (redirectTestOutputToFile not taken into account): not yet fix on trunk, fixed on our version 800 793 are the more important to monitor, it's the only ones that are fixed in our version but not on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8140) TableMapReduceUtils#addDependencyJar fails when nested inside another MR job
Nick Dimiduk created HBASE-8140: --- Summary: TableMapReduceUtils#addDependencyJar fails when nested inside another MR job Key: HBASE-8140 URL: https://issues.apache.org/jira/browse/HBASE-8140 Project: HBase Issue Type: Bug Components: mapreduce Reporter: Nick Dimiduk Assignee: Nick Dimiduk TableMapReduceUtils#addDependencyJar is used when configuring a mapreduce job to make sure dependencies of the job are shipped to the cluster. The code depends on finding an actual jar file containing the necessary classes. This is not always the case, for instance, when run at the end of another mapreduce job. In that case, dependency jars have already been shipped to the cluster and expanded in the parent job's run folder. Those dependencies are there, just not available as jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
