[jira] [Commented] (HBASE-20432) Cleanup related resources when remove a sync replication peer
[
https://issues.apache.org/jira/browse/HBASE-20432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445378#comment-16445378
]
Hadoop QA commented on HBASE-20432:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
51s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
4s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
2s{color} | {color:red} hbase-server: The patch generated 1 new + 0 unchanged -
0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}108m
43s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20432 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919946/HBASE-20432.HBASE-19064.v2.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 47cabb1ab4ad 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-19064 / 2fd6813846 |
| maven | version: Apache Mave
[jira] [Commented] (HBASE-20281) [DOC] upgrade section needs an explanation of changes to Bucket Cache
[
https://issues.apache.org/jira/browse/HBASE-20281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445362#comment-16445362
]
Anoop Sam John commented on HBASE-20281:
hbase.bucketcache.ioengine changes are there. That we have removed onheap
mode and also new modes like files and mmap.
hbase.bucketcache.combinedcache.enabled - I can see this is added under
section "Configuration settings no longer in HBase 2.0+" already.
> [DOC] upgrade section needs an explanation of changes to Bucket Cache
> -
>
> Key: HBASE-20281
> URL: https://issues.apache.org/jira/browse/HBASE-20281
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache, documentation
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
>
> the first pass version of the upgrade docs has a brief note about the bucket
> cache changing:
> {quote}
> * hbase.bucketcache.combinedcache.enabled
> * hbase.bucketcache.ioengine no longer supports the 'heap' value.
> {quote}
> But the changes are substantial and warrant a section of their own under the
> "Changes of Note!" banner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19782) Reject the replication request when peer is DA or A state
[
https://issues.apache.org/jira/browse/HBASE-19782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445346#comment-16445346
]
Hadoop QA commented on HBASE-19782:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} HBASE-19064 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
20s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
58s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} HBASE-19064 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} HBASE-19064 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} hbase-server: The patch generated 0 new + 403
unchanged - 1 fixed = 403 total (was 404) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
50s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}115m
37s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-19782 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919938/HBASE-19782.HBASE-19064.v8.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e8b88e0690da 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | HBASE-19064 / 2fd6813846 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12564/testReport/ |
| Max. process+thread count | 3985 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12564/console |
| Powered
[jira] [Commented] (HBASE-20457) Return immediately for a scan rpc call when we want to switch from pread to stream
[ https://issues.apache.org/jira/browse/HBASE-20457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445329#comment-16445329 ] ramkrishna.s.vasudevan commented on HBASE-20457: One more thing to consider. One case for eg the count 'table' case, where we do a scan wiht FirstKeyOnlyfiter. In this case we are switching over to STREAM after the first RPC. But when the scan moves over to the next region it is again DEFAULT type and then after first RPC it switches to STREAM. So should we do this way or after the first RPC in the first region switches to STREAM we should use the same mode in all other regions? > Return immediately for a scan rpc call when we want to switch from pread to > stream > -- > > Key: HBASE-20457 > URL: https://issues.apache.org/jira/browse/HBASE-20457 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20462) Put up 2.0.0RC1
[
https://issues.apache.org/jira/browse/HBASE-20462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445318#comment-16445318
]
stack commented on HBASE-20462:
---
* git show-branch origin/branch-2.0 origin/branch-2 and compare if anything
important left out of branch-2.0.
* Copied the refguide from master branch. See HBASE-20142 commit message at
334286dfe521b9c9206699c967a46b24e52a2a36 for what was done.
* Update RELEASENOTES.md and CHANGES.md running {{$
./release-doc-maker/releasedocmaker.py -p HBASE --fileversions -v 2.0.0 -l
--sortorder=newer --skip-credits}}. Changed generated files according to
prescription at head of existing CHANGES.md, etc. Added in the hbase-1.0.0
CHANGES.txt content to CHANGES.md. Committed it under this JIRA.
* Tagged bfada28760ec4c95cc42215a1813e0b628f1fffa locally as 2.0.0RC1. Wait
till successful build before pushing tag public (don't forget!).
* Ran ./dev-tools/make_rc.sh 2.0.0RC0
Be back later
> Put up 2.0.0RC1
> ---
>
> Key: HBASE-20462
> URL: https://issues.apache.org/jira/browse/HBASE-20462
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20276) [shell] Revert shell REPL change and document
[
https://issues.apache.org/jira/browse/HBASE-20276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445296#comment-16445296
]
stack edited comment on HBASE-20276 at 4/20/18 4:58 AM:
Opening HBASE-20463 for branch-1 fix. Resolving this so it makes it into the
RC1 of 2.0.0 release notes. Hope you don't mind lads ([~busbey] and [~apurtell])
was (Author: stack):
Opening subissue for branch-1 fix. Resolving this so it makes it into the RC1
of 2.0.0 release notes.
> [shell] Revert shell REPL change and document
> -
>
> Key: HBASE-20276
> URL: https://issues.apache.org/jira/browse/HBASE-20276
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, shell
>Affects Versions: 1.4.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-20276-branch-1.v4.patch, HBASE-20276.0.patch,
> HBASE-20276.1.patch, HBASE-20276.2.patch, HBASE-20276.3.patch
>
>
> Feedback from [~mdrob] on HBASE-19158:
> {quote}
> Shell:
> HBASE-19770. There was another issue opened where this was identified as a
> problem so maybe the shape will change further, but I can't find it now.
> {quote}
> New commentary from [~busbey]:
> This was a follow on to HBASE-15965. That change effectively makes it so none
> of our ruby wrappers can be used to build expressions in an interactive REPL.
> This is a pretty severe change (most of my tips on HBASE-15611 will break, I
> think).
> I think we should
> a) Have a DISCUSS thread, spanning dev@ and user@
> b) based on the outcome of that thread, either default to the new behavior or
> the old behavior
> c) if we keep the HBASE-15965 behavior as the default, flag it as
> incompatible, call it out in the hbase 2.0 upgrade section, and update docs
> (two examples: the output in the shell_exercises sections would be wrong, and
> the _table_variables section won't work)
> d) In either case document the new flag in the ref guide
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20276) [shell] Revert shell REPL change and document
[
https://issues.apache.org/jira/browse/HBASE-20276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20276:
--
Resolution: Fixed
Fix Version/s: (was: 1.4.4)
Status: Resolved (was: Patch Available)
Opening subissue for branch-1 fix. Resolving this so it makes it into the RC1
of 2.0.0 release notes.
> [shell] Revert shell REPL change and document
> -
>
> Key: HBASE-20276
> URL: https://issues.apache.org/jira/browse/HBASE-20276
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, shell
>Affects Versions: 1.4.0, 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: HBASE-20276-branch-1.v4.patch, HBASE-20276.0.patch,
> HBASE-20276.1.patch, HBASE-20276.2.patch, HBASE-20276.3.patch
>
>
> Feedback from [~mdrob] on HBASE-19158:
> {quote}
> Shell:
> HBASE-19770. There was another issue opened where this was identified as a
> problem so maybe the shape will change further, but I can't find it now.
> {quote}
> New commentary from [~busbey]:
> This was a follow on to HBASE-15965. That change effectively makes it so none
> of our ruby wrappers can be used to build expressions in an interactive REPL.
> This is a pretty severe change (most of my tips on HBASE-15611 will break, I
> think).
> I think we should
> a) Have a DISCUSS thread, spanning dev@ and user@
> b) based on the outcome of that thread, either default to the new behavior or
> the old behavior
> c) if we keep the HBASE-15965 behavior as the default, flag it as
> incompatible, call it out in the hbase 2.0 upgrade section, and update docs
> (two examples: the output in the shell_exercises sections would be wrong, and
> the _table_variables section won't work)
> d) In either case document the new flag in the ref guide
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20463) Fix breakage introduced on branch-1 by HBASE-20276 "[shell] Revert shell REPL change and document"
stack created HBASE-20463: - Summary: Fix breakage introduced on branch-1 by HBASE-20276 "[shell] Revert shell REPL change and document" Key: HBASE-20463 URL: https://issues.apache.org/jira/browse/HBASE-20463 Project: HBase Issue Type: Bug Reporter: stack Assignee: Sean Busbey Fix For: 1.4.4 Hope you don't mind my making an issue for fixing branch-1 breakage [~busbey] (and [~apurtell]). See parent for discussion on breakage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20281) [DOC] upgrade section needs an explanation of changes to Bucket Cache
[
https://issues.apache.org/jira/browse/HBASE-20281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445290#comment-16445290
]
stack commented on HBASE-20281:
---
[~anoop.hbase] The doc in HBASE-20059 doesn't have anything on the two configs
in the description above. On victim going away, I was suggesting it is worth
noting in the upgrade section (yes, we talk about it going away in hbase-20059
but should get mention earlier in doc too). Thanks.
> [DOC] upgrade section needs an explanation of changes to Bucket Cache
> -
>
> Key: HBASE-20281
> URL: https://issues.apache.org/jira/browse/HBASE-20281
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache, documentation
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
>
> the first pass version of the upgrade docs has a brief note about the bucket
> cache changing:
> {quote}
> * hbase.bucketcache.combinedcache.enabled
> * hbase.bucketcache.ioengine no longer supports the 'heap' value.
> {quote}
> But the changes are substantial and warrant a section of their own under the
> "Changes of Note!" banner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20454) [DOC] Add note on perf to upgrade section
[
https://issues.apache.org/jira/browse/HBASE-20454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445288#comment-16445288
]
Hudson commented on HBASE-20454:
Results for branch branch-2.0
[build #198 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/198/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/198//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/198//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/198//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [DOC] Add note on perf to upgrade section
> -
>
> Key: HBASE-20454
> URL: https://issues.apache.org/jira/browse/HBASE-20454
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20454.branch-2.0.001.patch
>
>
> Add short note on YMMV regards perf and 2.0.0 but we working on it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20059) Make sure documentation is updated for the offheap Bucket cache usage
[ https://issues.apache.org/jira/browse/HBASE-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445287#comment-16445287 ] Anoop Sam John commented on HBASE-20059: Thanks Stack. > Make sure documentation is updated for the offheap Bucket cache usage > - > > Key: HBASE-20059 > URL: https://issues.apache.org/jira/browse/HBASE-20059 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-20059-Make-sure-documentation-is-updated-for-t.patch, > HBASE-20059.patch, HBASE-20059_V2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20142) Copy master doc into branch-2 and edit to make it suit 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445286#comment-16445286 ] stack commented on HBASE-20142: --- Pushed another copy from master branch running same prescription as noted above at 334286dfe521b9c9206699c967a46b24e52a2a36 > Copy master doc into branch-2 and edit to make it suit 2.0.0 > > > Key: HBASE-20142 > URL: https://issues.apache.org/jira/browse/HBASE-20142 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Priority: Major > > Place-holder for task to be done before we RC... copy the master refguide > into branch-2 and do a pass to make sure it hbase-2 appropriate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20281) [DOC] upgrade section needs an explanation of changes to Bucket Cache
[
https://issues.apache.org/jira/browse/HBASE-20281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445285#comment-16445285
]
Anoop Sam John commented on HBASE-20281:
Patch in HBASE-20059 handles this.
> [DOC] upgrade section needs an explanation of changes to Bucket Cache
> -
>
> Key: HBASE-20281
> URL: https://issues.apache.org/jira/browse/HBASE-20281
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache, documentation
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
>
> the first pass version of the upgrade docs has a brief note about the bucket
> cache changing:
> {quote}
> * hbase.bucketcache.combinedcache.enabled
> * hbase.bucketcache.ioengine no longer supports the 'heap' value.
> {quote}
> But the changes are substantial and warrant a section of their own under the
> "Changes of Note!" banner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20462) Put up 2.0.0RC1
stack created HBASE-20462: - Summary: Put up 2.0.0RC1 Key: HBASE-20462 URL: https://issues.apache.org/jira/browse/HBASE-20462 Project: HBase Issue Type: Sub-task Reporter: stack -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20461) Implement fsync for frasyncwal
stack created HBASE-20461: - Summary: Implement fsync for frasyncwal Key: HBASE-20461 URL: https://issues.apache.org/jira/browse/HBASE-20461 Project: HBase Issue Type: Sub-task Environment: Parent issue adds a config so we can fsync rather than sync for FSHLog, the branch-1 WAL. Add same for asyncfwal, the branch-2 WAL Reporter: stack -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20432) Cleanup related resources when remove a sync replication peer
[ https://issues.apache.org/jira/browse/HBASE-20432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-20432: - Attachment: HBASE-20432.HBASE-19064.v2.patch > Cleanup related resources when remove a sync replication peer > - > > Key: HBASE-20432 > URL: https://issues.apache.org/jira/browse/HBASE-20432 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-20432.HBASE-19064.v1.patch, > HBASE-20432.HBASE-19064.v2.patch > > > When remove a sync replication peer, we should clean: > 1. the SyncReplicationState in zookeeper or table. > 2. Remote WALs & Remote WALs directory in peer clusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20459) Majority of scan CPU time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-20459: -- Attachment: 20459.txt > Majority of scan CPU time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.4.4 > > Attachments: 20459.txt, Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20459) Majority of scan CPU time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl reassigned HBASE-20459: - Assignee: Lars Hofhansl > Majority of scan CPU time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.4.4 > > Attachments: 20459.txt, Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20459) Majority of scan CPU time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445214#comment-16445214 ] Lars Hofhansl commented on HBASE-20459: --- Just this?! Avoids recalculating arrayHeaderSize over and over. > Majority of scan CPU time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.4.4 > > Attachments: 20459.txt, Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20432) Cleanup related resources when remove a sync replication peer
[ https://issues.apache.org/jira/browse/HBASE-20432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445203#comment-16445203 ] Zheng Hu commented on HBASE-20432: -- Ping [~Apache9] & [~zghaobac] for reviewing. Thanks. > Cleanup related resources when remove a sync replication peer > - > > Key: HBASE-20432 > URL: https://issues.apache.org/jira/browse/HBASE-20432 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-20432.HBASE-19064.v1.patch > > > When remove a sync replication peer, we should clean: > 1. the SyncReplicationState in zookeeper or table. > 2. Remote WALs & Remote WALs directory in peer clusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-15950) Fix memstore size estimates to be more tighter
[ https://issues.apache.org/jira/browse/HBASE-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445194#comment-16445194 ] Lars Hofhansl commented on HBASE-15950: --- Gentlemen, please see HBASE-20459. > Fix memstore size estimates to be more tighter > -- > > Key: HBASE-15950 > URL: https://issues.apache.org/jira/browse/HBASE-15950 > Project: HBase > Issue Type: Bug >Reporter: Enis Soztutar >Assignee: Enis Soztutar >Priority: Major > Fix For: 1.4.0, 2.0.0 > > Attachments: Screen Shot 2016-06-02 at 8.48.27 PM.png, > hbase-15950-v0.patch, hbase-15950-v1.patch, hbase-15950-v2.branch-1.patch, > hbase-15950-v2.branch-1.patch, hbase-15950-v2.patch > > > While testing something else, I was loading a region with a lot of data. > Writing 30M cells in 1M rows, with 1 byte values. > The memstore size turned out to be estimated as 4.5GB, while with the JFR > profiling I can see that we are using 2.8GB for all the objects in the > memstore (KV + KV byte[] + CSLM.Node + CSLM.Index). > This obviously means that there is room in the write cache that we are not > effectively using. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19782) Reject the replication request when peer is DA or A state
[ https://issues.apache.org/jira/browse/HBASE-19782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-19782: - Attachment: HBASE-19782.HBASE-19064.v8.patch > Reject the replication request when peer is DA or A state > - > > Key: HBASE-19782 > URL: https://issues.apache.org/jira/browse/HBASE-19782 > Project: HBase > Issue Type: Sub-task > Components: Replication >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-19064-HBASE-19782.v1.patch, > HBASE-19064-HBASE-19782.v2.patch, HBASE-19782.HBASE-19064.v2.patch, > HBASE-19782.HBASE-19064.v2.patch, HBASE-19782.HBASE-19064.v3.patch, > HBASE-19782.HBASE-19064.v4.patch, HBASE-19782.HBASE-19064.v4.patch, > HBASE-19782.HBASE-19064.v4.patch, HBASE-19782.HBASE-19064.v5.patch, > HBASE-19782.HBASE-19064.v5.patch, HBASE-19782.HBASE-19064.v6.patch, > HBASE-19782.HBASE-19064.v7.patch, HBASE-19782.HBASE-19064.v8.patch > > > According to the design doc, we'll initialize both of the cluster state to > DA after added the bidirectional replication path. and when a cluster in > DA state, it'll reject replication request. > so for cluster A and B in state DA, if any received replication entry whose > table or namespace match the peer,the cluster will skip to apply into > its local rs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20459) Majority of scan CPU time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-20459: -- Summary: Majority of scan CPU time in HBase-1 spent in size estimation (was: Majority of scan time in HBase-1 spent in size estimation) > Majority of scan CPU time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.4.4 > > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445145#comment-16445145 ] Lars Hofhansl edited comment on HBASE-20459 at 4/20/18 2:34 AM: If the region server is not busy the query time will actually not increase, but I see that the region server consumes between 25% and 40% more CPU time. So it's quite easy to make the region server CPU bound with just a few concurrent scans. was (Author: lhofhansl): If the region server is not busy the query time will actually not increase, but I see that the region server consumes between 25% and 40% more CPU time. > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.4.4 > > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445145#comment-16445145 ] Lars Hofhansl commented on HBASE-20459: --- If the region server is not busy the query time will actually not increase, but I see that the region server consumes between 25% and 40% more CPU time. > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.4.4 > > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19924) hbase rpc throttling does not work for multi() with request count rater.
[
https://issues.apache.org/jira/browse/HBASE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445110#comment-16445110
]
Chia-Ping Tsai commented on HBASE-19924:
+1
> hbase rpc throttling does not work for multi() with request count rater.
>
>
> Key: HBASE-19924
> URL: https://issues.apache.org/jira/browse/HBASE-19924
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 0.16.0, 1.2.6
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Major
> Attachments: HBASE-19924-master-v001.patch,
> HBASE-19924-master-v002.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch, HBASE-19924-master-v002.patch
>
>
> Basically, rpc throttling does not work for request count based rater for
> multi. for the following code, when it calls limiter's checkQuota(),
> numWrites/numReads is lost.
> {code:java}
> @Override
> public void checkQuota(int numWrites, int numReads, int numScans) throws
> ThrottlingException {
> writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);
> readConsumed = estimateConsume(OperationType.GET, numReads, 100);
> readConsumed += estimateConsume(OperationType.SCAN, numScans, 1000);
> writeAvailable = Long.MAX_VALUE;
> readAvailable = Long.MAX_VALUE;
> for (final QuotaLimiter limiter : limiters) {
> if (limiter.isBypass()) continue;
> limiter.checkQuota(writeConsumed, readConsumed);
> readAvailable = Math.min(readAvailable, limiter.getReadAvailable());
> writeAvailable = Math.min(writeAvailable, limiter.getWriteAvailable());
> }
> for (final QuotaLimiter limiter : limiters) {
> limiter.grabQuota(writeConsumed, readConsumed);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20040) Master UI should include "Cluster Key" needed to use the cluster as a replication sink
[
https://issues.apache.org/jira/browse/HBASE-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445106#comment-16445106
]
Hadoop QA commented on HBASE-20040:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}104m
43s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}116m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20040 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919927/hbase-20040.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux 2e00ace9978a 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 70377babd0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12562/testReport/ |
| Max. process+thread count | 4319 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12562/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Master UI should include "Cluster Key" needed to use the cluster as a
> replication sink
> --
>
> Key: HBASE-20040
> URL: https://issues.apache.org/jira/browse/HBASE-20040
> Project: HBase
> Issue Type: Improvement
> Components: Replication, Usability
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Minor
> Labels: beginner
> Attachments: hbase-20040.master.001.patch
>
>
> The ref guide defines a "Cluster Key" needed to add an hbase cluster as a
> replication peer
> {quote}
> CLUSTER_KEY: composed using the following template, with appropriate
> place-holders:
> {code}hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent{code}
> {quote}
> the Master UI has all of the pieces displayed currently, but it should
> include a single field that operators can copy/paste.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17391) [Shell] Add shell command to get list of servers, with filters
[
https://issues.apache.org/jira/browse/HBASE-17391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445094#comment-16445094
]
Hadoop QA commented on HBASE-17391:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 3 new + 412 unchanged - 0 fixed =
415 total (was 412) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red} 0m
4s{color} | {color:red} The patch generated 6 new + 725 unchanged - 0 fixed =
731 total (was 725) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
50s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-17391 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919933/HBASE-17391.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux 953eb639fad8 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 70377babd0 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_162 |
| rubocop | v0.54.0 |
| rubocop |
https://builds.apache.org/job/PreCommit-HBASE-Build/12563/artifact/patchprocess/diff-patch-rubocop.txt
|
| ruby-lint | v2.3.1 |
| ruby-lint |
https://builds.apache.org/job/PreCommit-HBASE-Build/12563/artifact/patchprocess/diff-patch-ruby-lint.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12563/testReport/ |
| Max. process+thread count | 2094 (vs. ulimit of 1) |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/12563/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> [Shell] Add shell command to get list of servers, with filters
> --
>
> Key: HBASE-17391
> URL: https://issues.apache.org/jira/browse/HBASE-17391
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: Sreeram Venkatasubramanian
>Priority: Major
> Labels: beginner
> Fix For: 3.0.0
>
> Attachments: HBASE-17391.master.000.patch,
> HBASE-17391.master.001.patch
>
>
> For some operations, for example calling {{update_config}}, the user needs to
> specify the full server name. For region servers that is easier to find, but
> not so much for the master (using {{zk_dump}} works but is noisy). It woould
> be good to add a utility call that lists the servers, preferably with an
[jira] [Updated] (HBASE-17391) [Shell] Add shell command to get list of servers, with filters
[
https://issues.apache.org/jira/browse/HBASE-17391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeram Venkatasubramanian updated HBASE-17391:
---
Attachment: HBASE-17391.master.001.patch
> [Shell] Add shell command to get list of servers, with filters
> --
>
> Key: HBASE-17391
> URL: https://issues.apache.org/jira/browse/HBASE-17391
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Affects Versions: 1.3.0
>Reporter: Lars George
>Assignee: Sreeram Venkatasubramanian
>Priority: Major
> Labels: beginner
> Fix For: 3.0.0
>
> Attachments: HBASE-17391.master.000.patch,
> HBASE-17391.master.001.patch
>
>
> For some operations, for example calling {{update_config}}, the user needs to
> specify the full server name. For region servers that is easier to find, but
> not so much for the master (using {{zk_dump}} works but is noisy). It woould
> be good to add a utility call that lists the servers, preferably with an
> optional filter (a regexp, server type, or globbing style format) that allows
> to whittle down the potentially long is of servers. For example:
> {noformat}
> hbase(main):001:0> list_servers "master"
> master-1.internal.larsgeorge.com,16000,1483018890074
> hbase(main):002:0> list_servers "rs"
> slave-1.internal.larsgeorge.com,16020,1482996572051
> slave-3.internal.larsgeorge.com,16020,1482996572481
> slave-2.internal.larsgeorge.com,16020,1482996570909
> hbase(main):003:0> list_servers "rs:s.*\.com.*"
> slave-1.internal.larsgeorge.com,16020,1482996572051
> slave-3.internal.larsgeorge.com,16020,1482996572481
> slave-2.internal.larsgeorge.com,16020,1482996570909
> hbase(main):004:0> list_servers ":.*160?0.*"
> master-1.internal.larsgeorge.com,16000,1483018890074
> slave-1.internal.larsgeorge.com,16020,1482996572051
> slave-3.internal.larsgeorge.com,16020,1482996572481
> slave-2.internal.larsgeorge.com,16020,1482996570909
> {noformat}
> I could imagine to have {{master}}, {{backup-master}}, {{rs}}, and maybe even
> {{zk}} too. The optional regexp shown uses a colon as a divider. This
> combines the "by-type", and using a filter. Example #4 skips the type and
> only is using the filter.
> Of course, you could also implement this differently, say with two
> parameters... just suggesting.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445073#comment-16445073 ] Lars Hofhansl commented on HBASE-20459: --- I think the added precision introduced in HBASE-15950 is not worth the added cost. > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.4.4 > > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20316) Backport HBASE-20229 "ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on" to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445060#comment-16445060 ] huaxiang sun commented on HBASE-20316: -- +1, let me run the patch locally. > Backport HBASE-20229 "ConnectionImplementation.locateRegions() returns > duplicated entries when region replication is on" to branch-1 > > > Key: HBASE-20316 > URL: https://issues.apache.org/jira/browse/HBASE-20316 > Project: HBase > Issue Type: Sub-task > Components: backport >Reporter: stack >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-20229.branch-1.002.patch, > HBASE-20229.branch-1.002.patch, HBASE-20229.branch-1.002.patch, > HBASE-20229.branch-1.002.patch > > > Issue to backport parent to branch-1. Hope you don't mind my assigning it to > you [~brfrn169] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20332) shaded mapreduce module shouldn't include hadoop
[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445052#comment-16445052
]
Sean Busbey commented on HBASE-20332:
-
those are definitely related failures. probably some additional missing test
scoped hadoop htrace instances.
> shaded mapreduce module shouldn't include hadoop
>
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch,
> HBASE-20332.2.WIP.patch
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround
> (e.g. for some library your application needs that hadoop provides) then our
> inclusion of *some but not all* hadoop classes then causes everything to fall
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're
> just wasting space
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20332) shaded mapreduce module shouldn't include hadoop
[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445051#comment-16445051
]
Sean Busbey commented on HBASE-20332:
-
okay v2 checked out via running the commands on cluster with
{{HADOOP_CLASSPATH=/etc/hbase/conf yarn jar
hbase-shaded-mapreduce-3.0.0-SNAPSHOT.jar }}. got through
everything except VerifyReplication. had a problem with auths on the zk nodes
needed to get the peer config, but doesn't look like a shading issue.
let me run through these unit test results.
> shaded mapreduce module shouldn't include hadoop
>
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch,
> HBASE-20332.2.WIP.patch
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround
> (e.g. for some library your application needs that hadoop provides) then our
> inclusion of *some but not all* hadoop classes then causes everything to fall
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're
> just wasting space
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[
https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445036#comment-16445036
]
Hadoop QA commented on HBASE-20447:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
19s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
2s{color} | {color:red} hbase-server: The patch generated 14 new + 149
unchanged - 1 fixed = 163 total (was 150) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
14s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 54s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
27s{color} | {color:red} hbase-server generated 2 new + 2 unchanged - 0 fixed =
4 total (was 2) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}104m 31s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
31s{color} | {color:green} hbase-external-blockcache in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}149m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.io.hfile.bucket.TestBucketCache |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20447 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919861/HBASE-20447.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 33dae2728c6e 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/c
[jira] [Commented] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445027#comment-16445027 ] Lars Hofhansl commented on HBASE-20459: --- Indeed when I pass -Dhbase.memorylayout.use.unsafe=false to the region server this cost goes away. > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.4.4 > > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20413) IntegrationTestRegionReplicaReplication fails
[ https://issues.apache.org/jira/browse/HBASE-20413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20413: -- Priority: Critical (was: Blocker) > IntegrationTestRegionReplicaReplication fails > - > > Key: HBASE-20413 > URL: https://issues.apache.org/jira/browse/HBASE-20413 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > > Following [~psomogyi]'s inspiration, I've been running unit tests on a GCE > instance. IntegrationTestRegionReplicaReplication fails always for > branch-2.0. It fails for branch-1 too but for a different looking reason. On > branch-2.0, its OOME'ing. Downing the size of ingested data, it seems to hang > unable to update because it is unable to flush. > Marking this a blocker for now. Fix or disable it for branch-2.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20413) IntegrationTestRegionReplicaReplication fails
[ https://issues.apache.org/jira/browse/HBASE-20413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445020#comment-16445020 ] stack commented on HBASE-20413: --- This is not getting any love. Making critical. > IntegrationTestRegionReplicaReplication fails > - > > Key: HBASE-20413 > URL: https://issues.apache.org/jira/browse/HBASE-20413 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack >Priority: Critical > Fix For: 2.0.0 > > > Following [~psomogyi]'s inspiration, I've been running unit tests on a GCE > instance. IntegrationTestRegionReplicaReplication fails always for > branch-2.0. It fails for branch-1 too but for a different looking reason. On > branch-2.0, its OOME'ing. Downing the size of ingested data, it seems to hang > unable to update because it is unable to flush. > Marking this a blocker for now. Fix or disable it for branch-2.0. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20459: --- Affects Version/s: (was: 1.3.2) > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.4.4 > > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20459: --- Fix Version/s: (was: 1.3.3) > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.4.4 > > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20459: --- Affects Version/s: 1.3.2 1.4.3 > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.3.2, 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.3.3, 1.4.4 > > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20459: --- Fix Version/s: 1.4.4 1.5.0 > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.3.2, 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.3.3, 1.4.4 > > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20459: --- Fix Version/s: 1.3.3 > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.3.2, 1.4.3 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.5.0, 1.3.3, 1.4.4 > > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20040) Master UI should include "Cluster Key" needed to use the cluster as a replication sink
[
https://issues.apache.org/jira/browse/HBASE-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445006#comment-16445006
]
Sakthi commented on HBASE-20040:
[~busbey] , Please review. I have tested the patch manually by building a tar
and checking the master UI for the presence of the "Cluster Key" field. If
there is any other way to automate the test, I would be happy to try.
Sakthi
> Master UI should include "Cluster Key" needed to use the cluster as a
> replication sink
> --
>
> Key: HBASE-20040
> URL: https://issues.apache.org/jira/browse/HBASE-20040
> Project: HBase
> Issue Type: Improvement
> Components: Replication, Usability
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Minor
> Labels: beginner
> Attachments: hbase-20040.master.001.patch
>
>
> The ref guide defines a "Cluster Key" needed to add an hbase cluster as a
> replication peer
> {quote}
> CLUSTER_KEY: composed using the following template, with appropriate
> place-holders:
> {code}hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent{code}
> {quote}
> the Master UI has all of the pieces displayed currently, but it should
> include a single field that operators can copy/paste.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20060) Add details of off heap memstore into book.
[ https://issues.apache.org/jira/browse/HBASE-20060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445003#comment-16445003 ] stack commented on HBASE-20060: --- I opened HBASE-20460 for doc on general offheap write-path. This is probably a subtask of it. > Add details of off heap memstore into book. > --- > > Key: HBASE-20060 > URL: https://issues.apache.org/jira/browse/HBASE-20060 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.1.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20040) Master UI should include "Cluster Key" needed to use the cluster as a replication sink
[
https://issues.apache.org/jira/browse/HBASE-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-20040:
---
Status: Patch Available (was: In Progress)
> Master UI should include "Cluster Key" needed to use the cluster as a
> replication sink
> --
>
> Key: HBASE-20040
> URL: https://issues.apache.org/jira/browse/HBASE-20040
> Project: HBase
> Issue Type: Improvement
> Components: Replication, Usability
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Minor
> Labels: beginner
> Attachments: hbase-20040.master.001.patch
>
>
> The ref guide defines a "Cluster Key" needed to add an hbase cluster as a
> replication peer
> {quote}
> CLUSTER_KEY: composed using the following template, with appropriate
> place-holders:
> {code}hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent{code}
> {quote}
> the Master UI has all of the pieces displayed currently, but it should
> include a single field that operators can copy/paste.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20460) Doc offheap write-path
[ https://issues.apache.org/jira/browse/HBASE-20460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20460: -- Fix Version/s: 2.1.0 > Doc offheap write-path > -- > > Key: HBASE-20460 > URL: https://issues.apache.org/jira/browse/HBASE-20460 > Project: HBase > Issue Type: Bug > Components: documentation, Offheaping >Reporter: stack >Priority: Major > Fix For: 2.1.0 > > > We have an empty section in refguide that needs filling in on how to enable > offheap write-path, how to know you've set it up right or not, how to tune > it, and how it relates to direct memory allocation and offheap read-path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20460) Doc offheap write-path
stack created HBASE-20460: - Summary: Doc offheap write-path Key: HBASE-20460 URL: https://issues.apache.org/jira/browse/HBASE-20460 Project: HBase Issue Type: Bug Components: documentation, Offheaping Reporter: stack We have an empty section in refguide that needs filling in on how to enable offheap write-path, how to know you've set it up right or not, how to tune it, and how it relates to direct memory allocation and offheap read-path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20040) Master UI should include "Cluster Key" needed to use the cluster as a replication sink
[
https://issues.apache.org/jira/browse/HBASE-20040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-20040:
---
Attachment: hbase-20040.master.001.patch
> Master UI should include "Cluster Key" needed to use the cluster as a
> replication sink
> --
>
> Key: HBASE-20040
> URL: https://issues.apache.org/jira/browse/HBASE-20040
> Project: HBase
> Issue Type: Improvement
> Components: Replication, Usability
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Minor
> Labels: beginner
> Attachments: hbase-20040.master.001.patch
>
>
> The ref guide defines a "Cluster Key" needed to add an hbase cluster as a
> replication peer
> {quote}
> CLUSTER_KEY: composed using the following template, with appropriate
> place-holders:
> {code}hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent{code}
> {quote}
> the Master UI has all of the pieces displayed currently, but it should
> include a single field that operators can copy/paste.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20281) [DOC] upgrade section needs an explanation of changes to Bucket Cache
[
https://issues.apache.org/jira/browse/HBASE-20281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444996#comment-16444996
]
stack commented on HBASE-20281:
---
Nothing here as yet on these changes.
The 'victim' config., cacheDataInL1` and `CACHE_DATA_IN_L1` for HCD no longer
work in hbase2 so could get a mention in same section.
> [DOC] upgrade section needs an explanation of changes to Bucket Cache
> -
>
> Key: HBASE-20281
> URL: https://issues.apache.org/jira/browse/HBASE-20281
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache, documentation
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
>
> the first pass version of the upgrade docs has a brief note about the bucket
> cache changing:
> {quote}
> * hbase.bucketcache.combinedcache.enabled
> * hbase.bucketcache.ioengine no longer supports the 'heap' value.
> {quote}
> But the changes are substantial and warrant a section of their own under the
> "Changes of Note!" banner.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444995#comment-16444995 ] Lars Hofhansl commented on HBASE-20459: --- This came with HBASE-15950. [~enis], FYI. > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Reporter: Lars Hofhansl >Priority: Major > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20278) [DOC] include ref guide updates for HBase 2.0 HBCK
[
https://issues.apache.org/jira/browse/HBASE-20278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444994#comment-16444994
]
stack commented on HBASE-20278:
---
No hbck2 as of now so nothing to point at.
> [DOC] include ref guide updates for HBase 2.0 HBCK
> --
>
> Key: HBASE-20278
> URL: https://issues.apache.org/jira/browse/HBASE-20278
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, hbck
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Umesh Agashe
>Priority: Critical
>
> Called out by [~stack] in HBASE-19158
> {quote}
> bq. HBCK tool from an earlier release against an HBase 2.0+ cluster will
> destructively alter said cluster in unrecoverable ways.
> Footnote or callout that says something like "Unfortunately we are unable to
> distinguish an HBCK client so cannot put in place guards against destructive
> HBCK changes." probably too much for this startup section on reread
> of what I've written here.
> bq. As of HBase 2.0, HBCK is a read-only tool that can report the status of
> some non-public system internals. You should not rely on the format nor
> content of these internals to remain consistent across HBase releases.
> Ugh. We need HBCK2 and a pointer here to it. Will do in a follow-on.
> {quote}
> then update the upgrade section to point to the docs
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-20459: -- Description: See attached screenshot. Will look into a fix later. > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Reporter: Lars Hofhansl >Priority: Major > Attachments: Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-20459: -- Attachment: Screenshot_20180419_162559.png > Majority of scan time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement >Reporter: Lars Hofhansl >Priority: Major > Attachments: Screenshot_20180419_162559.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20459) Majority of scan time in HBase-1 spent in size estimation
Lars Hofhansl created HBASE-20459: - Summary: Majority of scan time in HBase-1 spent in size estimation Key: HBASE-20459 URL: https://issues.apache.org/jira/browse/HBASE-20459 Project: HBase Issue Type: Improvement Reporter: Lars Hofhansl Attachments: Screenshot_20180419_162559.png -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20060) Add details of off heap memstore into book.
[ https://issues.apache.org/jira/browse/HBASE-20060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20060: -- Fix Version/s: (was: 2.0.0) 2.1.0 > Add details of off heap memstore into book. > --- > > Key: HBASE-20060 > URL: https://issues.apache.org/jira/browse/HBASE-20060 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.1.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20060) Add details of off heap memstore into book.
[ https://issues.apache.org/jira/browse/HBASE-20060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444981#comment-16444981 ] stack commented on HBASE-20060: --- I added an offheap read/write path. I was able to fill out the offheap read path. The write-path is a TODO. Use this issue to fill it in. > Add details of off heap memstore into book. > --- > > Key: HBASE-20060 > URL: https://issues.apache.org/jira/browse/HBASE-20060 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.1.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20059) Make sure documentation is updated for the offheap Bucket cache usage
[ https://issues.apache.org/jira/browse/HBASE-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444980#comment-16444980 ] stack commented on HBASE-20059: --- Attached 001... what I pushed. > Make sure documentation is updated for the offheap Bucket cache usage > - > > Key: HBASE-20059 > URL: https://issues.apache.org/jira/browse/HBASE-20059 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-20059-Make-sure-documentation-is-updated-for-t.patch, > HBASE-20059.patch, HBASE-20059_V2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20059) Make sure documentation is updated for the offheap Bucket cache usage
[ https://issues.apache.org/jira/browse/HBASE-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20059: -- Attachment: 0001-HBASE-20059-Make-sure-documentation-is-updated-for-t.patch > Make sure documentation is updated for the offheap Bucket cache usage > - > > Key: HBASE-20059 > URL: https://issues.apache.org/jira/browse/HBASE-20059 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-20059-Make-sure-documentation-is-updated-for-t.patch, > HBASE-20059.patch, HBASE-20059_V2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20059) Make sure documentation is updated for the offheap Bucket cache usage
[ https://issues.apache.org/jira/browse/HBASE-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-20059. --- Resolution: Fixed Pushed to master branch. > Make sure documentation is updated for the offheap Bucket cache usage > - > > Key: HBASE-20059 > URL: https://issues.apache.org/jira/browse/HBASE-20059 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-20059.patch, HBASE-20059_V2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20059) Make sure documentation is updated for the offheap Bucket cache usage
[ https://issues.apache.org/jira/browse/HBASE-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444971#comment-16444971 ] stack commented on HBASE-20059: --- So, I gave the doc a once-over, moved-stuff around, and added a section on offheap read-path and one for offheap write-path. I don't have enough to write up how to do offheap write-path with offheap memstore sizing, MSLAB chunking, and pools, so have just put a TODO there for now. Let me commit what is here. There is work to do still on making it so an operator can tell whether they have setup correctly (they have to know what log line to look for and how to interpret it currently). For offheap write, there seems to be but weak signal in logs that it is in place. There is work to do here still. My guess is that when this gets a bit of testing by others, the holes will be more plain; deferring till then. > Make sure documentation is updated for the offheap Bucket cache usage > - > > Key: HBASE-20059 > URL: https://issues.apache.org/jira/browse/HBASE-20059 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-20059.patch, HBASE-20059_V2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17097) Documentation for max off heap configuration
[
https://issues.apache.org/jira/browse/HBASE-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17097:
--
Component/s: Offheaping
> Documentation for max off heap configuration
>
>
> Key: HBASE-17097
> URL: https://issues.apache.org/jira/browse/HBASE-17097
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, Offheaping
>Affects Versions: 2.0.0
>Reporter: Anoop Sam John
>Priority: Major
>
> {quote}
> \# Uncomment below if you intend to use off heap cache. For example, to
> allocate 8G of
> \# offheap, set the value to "8G".
> \# export HBASE_OFFHEAPSIZE=1G
> {quote}
> This is what we have in hbase-env.sh now. We need more notes around this and
> some suggestion. May be more details in book also. This depends on whether we
> want L2 off heap BC as the default data cache in 2.0. (I would like to and
> will open a jira soon) Also if users use off heap MSLAB pool, then that also
> to be added up..
> As of now we will use a ByteBufferPool which pool direct BBs and this is ON
> by default. The max size, this pool can keep is 2 MB * 2 * #handlers.
> Will add more details as we turn ON other features.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-17097) [offheap] Documentation for max off heap configuration
[
https://issues.apache.org/jira/browse/HBASE-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17097:
--
Summary: [offheap] Documentation for max off heap configuration (was:
Documentation for max off heap configuration)
> [offheap] Documentation for max off heap configuration
> --
>
> Key: HBASE-17097
> URL: https://issues.apache.org/jira/browse/HBASE-17097
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, Offheaping
>Affects Versions: 2.0.0
>Reporter: Anoop Sam John
>Priority: Major
>
> {quote}
> \# Uncomment below if you intend to use off heap cache. For example, to
> allocate 8G of
> \# offheap, set the value to "8G".
> \# export HBASE_OFFHEAPSIZE=1G
> {quote}
> This is what we have in hbase-env.sh now. We need more notes around this and
> some suggestion. May be more details in book also. This depends on whether we
> want L2 off heap BC as the default data cache in 2.0. (I would like to and
> will open a jira soon) Also if users use off heap MSLAB pool, then that also
> to be added up..
> As of now we will use a ByteBufferPool which pool direct BBs and this is ON
> by default. The max size, this pool can keep is 2 MB * 2 * #handlers.
> Will add more details as we turn ON other features.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-17097) Documentation for max off heap configuration
[
https://issues.apache.org/jira/browse/HBASE-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-17097:
--
Fix Version/s: (was: 2.0.0)
> Documentation for max off heap configuration
>
>
> Key: HBASE-17097
> URL: https://issues.apache.org/jira/browse/HBASE-17097
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Anoop Sam John
>Priority: Major
>
> {quote}
> \# Uncomment below if you intend to use off heap cache. For example, to
> allocate 8G of
> \# offheap, set the value to "8G".
> \# export HBASE_OFFHEAPSIZE=1G
> {quote}
> This is what we have in hbase-env.sh now. We need more notes around this and
> some suggestion. May be more details in book also. This depends on whether we
> want L2 off heap BC as the default data cache in 2.0. (I would like to and
> will open a jira soon) Also if users use off heap MSLAB pool, then that also
> to be added up..
> As of now we will use a ByteBufferPool which pool direct BBs and this is ON
> by default. The max size, this pool can keep is 2 MB * 2 * #handlers.
> Will add more details as we turn ON other features.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17097) Documentation for max off heap configuration
[
https://issues.apache.org/jira/browse/HBASE-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444965#comment-16444965
]
stack commented on HBASE-17097:
---
HBASE-20059 adds a note to hbase-env.sh pointing to the refguide Direct Memory
section. This is something, enough to move this out of 2.0.0, but not enough
for user trying to figure sizing when offheap read/write. Leaving issue open.
> Documentation for max off heap configuration
>
>
> Key: HBASE-17097
> URL: https://issues.apache.org/jira/browse/HBASE-17097
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Anoop Sam John
>Priority: Major
>
> {quote}
> \# Uncomment below if you intend to use off heap cache. For example, to
> allocate 8G of
> \# offheap, set the value to "8G".
> \# export HBASE_OFFHEAPSIZE=1G
> {quote}
> This is what we have in hbase-env.sh now. We need more notes around this and
> some suggestion. May be more details in book also. This depends on whether we
> want L2 off heap BC as the default data cache in 2.0. (I would like to and
> will open a jira soon) Also if users use off heap MSLAB pool, then that also
> to be added up..
> As of now we will use a ByteBufferPool which pool direct BBs and this is ON
> by default. The max size, this pool can keep is 2 MB * 2 * #handlers.
> Will add more details as we turn ON other features.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20332) shaded mapreduce module shouldn't include hadoop
[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444934#comment-16444934
]
Hadoop QA commented on HBASE-20332:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
0s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-shaded hbase-shaded/hbase-shaded-mapreduce
hbase-shaded/hbase-shaded-check-invariants . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} hbase-common: The patch generated 0 new + 88
unchanged - 1 fixed = 88 total (was 89) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} The patch hbase-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hbase-replication passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} The patch hbase-server passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch hbase-backup passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch hbase-rest passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch hbase-shaded passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} The patch hbase-shaded-mapreduce passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch hbase-shaded-check-invariants passed
checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch hbase-shaded-with-hadoop-check-invariants
passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
41s{color}
[jira] [Comment Edited] (HBASE-20059) Make sure documentation is updated for the offheap Bucket cache usage
[ https://issues.apache.org/jira/browse/HBASE-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444927#comment-16444927 ] stack edited comment on HBASE-20059 at 4/19/18 10:48 PM: - Where did we decide L1 and L2 should be deprecated? nvm. I see how its DATA blocks for bucketcache and META blocks for lrublockcache. was (Author: stack): Where did we decide L1 and L2 should be deprecated? > Make sure documentation is updated for the offheap Bucket cache usage > - > > Key: HBASE-20059 > URL: https://issues.apache.org/jira/browse/HBASE-20059 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-20059.patch, HBASE-20059_V2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20059) Make sure documentation is updated for the offheap Bucket cache usage
[ https://issues.apache.org/jira/browse/HBASE-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444927#comment-16444927 ] stack commented on HBASE-20059: --- Where did we decide L1 and L2 should be deprecated? > Make sure documentation is updated for the offheap Bucket cache usage > - > > Key: HBASE-20059 > URL: https://issues.apache.org/jira/browse/HBASE-20059 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-20059.patch, HBASE-20059_V2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18792) hbase-2 needs to defend against hbck operations
[ https://issues.apache.org/jira/browse/HBASE-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-18792: - Attachment: hbase-18792.branch-1.001.patch > hbase-2 needs to defend against hbck operations > --- > > Key: HBASE-18792 > URL: https://issues.apache.org/jira/browse/HBASE-18792 > Project: HBase > Issue Type: Task > Components: hbck >Reporter: stack >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > Attachments: hbase-18792.branch-1.001.patch, > hbase-18792.master.001.patch, hbase-18792.master.002.patch > > > hbck needs updating to run against hbase2. Meantime, if an hbck from hbase1 > is run against hbck2, it may do damage. hbase2 should defend itself against > hbck1 ops. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[ https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-20447: -- Status: Patch Available (was: Open) > Only fail cacheBlock if block collisions aren't related to next block metadata > -- > > Key: HBASE-20447 > URL: https://issues.apache.org/jira/browse/HBASE-20447 > Project: HBase > Issue Type: Bug > Components: BlockCache, BucketCache >Affects Versions: 1.4.3, 2.0.0 >Reporter: Zach York >Assignee: Zach York >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.4, 2.0.1 > > Attachments: HBASE-20447.branch-1.001.patch, > HBASE-20447.master.001.patch > > > This is the issue I was originally having here: > [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] > > When we pread, we don't force the read to read all of the next block header. > However, when we get into a race condition where two opener threads try to > cache the same block and one thread read all of the next block header and the > other one didn't, it will fail the open process. This is especially important > in a splitting case where it will potentially fail the split process. > Instead, in the caches, we should only fail if the required blocks are > different. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[ https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444837#comment-16444837 ] Andrew Purtell commented on HBASE-20447: An IOE should only be thrown if there is actually IO involved, hence the *IO* part of IOE. This is an assertion failure so we could use Preconditions. Note what Preconditions throws are all subclassed from RuntimeException. https://github.com/google/guava/wiki/PreconditionsExplained Seems entirely appropriate here. > Only fail cacheBlock if block collisions aren't related to next block metadata > -- > > Key: HBASE-20447 > URL: https://issues.apache.org/jira/browse/HBASE-20447 > Project: HBase > Issue Type: Bug > Components: BlockCache, BucketCache >Affects Versions: 1.4.3, 2.0.0 >Reporter: Zach York >Assignee: Zach York >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.4, 2.0.1 > > Attachments: HBASE-20447.branch-1.001.patch, > HBASE-20447.master.001.patch > > > This is the issue I was originally having here: > [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] > > When we pread, we don't force the read to read all of the next block header. > However, when we get into a race condition where two opener threads try to > cache the same block and one thread read all of the next block header and the > other one didn't, it will fail the open process. This is especially important > in a splitting case where it will potentially fail the split process. > Instead, in the caches, we should only fail if the required blocks are > different. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20224) Web UI is broken in standalone mode
[ https://issues.apache.org/jira/browse/HBASE-20224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444802#comment-16444802 ] Umesh Agashe commented on HBASE-20224: -- +1 > Web UI is broken in standalone mode > --- > > Key: HBASE-20224 > URL: https://issues.apache.org/jira/browse/HBASE-20224 > Project: HBase > Issue Type: Bug > Components: UI, Usability >Affects Versions: 2.0.0-beta-2 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Blocker > Fix For: 2.0.0 > > Attachments: > 0001-HBASE-20224-Web-UI-is-broken-in-standalone-mode-ADDE.ADDENDUM.patch, > 20224-addendum.3.txt, 20224.addendum.4, 20224.addendum.5, 20224.addendum.6, > HBASE-20224.master.004.patch, hbase-20224.master.001.patch, > hbase-20224.master.002.patch, hbase-20224.master.003.patch, > hbase-20224.master.addendum.patch > > > Web UI doesn't show up in standalone mode on default port. This can be seen > on master and branch-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[ https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444801#comment-16444801 ] Ted Yu commented on HBASE-20447: w.r.t. RuntimeException, developers would possibly miss the fact that validateBlockAddition would throw exception if they don't look at the implementation. Throwing IOE, on the other hand, signifies that the validation may fail. I think it would facilitate maintainability of the related code. > Only fail cacheBlock if block collisions aren't related to next block metadata > -- > > Key: HBASE-20447 > URL: https://issues.apache.org/jira/browse/HBASE-20447 > Project: HBase > Issue Type: Bug > Components: BlockCache, BucketCache >Affects Versions: 1.4.3, 2.0.0 >Reporter: Zach York >Assignee: Zach York >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.4, 2.0.1 > > Attachments: HBASE-20447.branch-1.001.patch, > HBASE-20447.master.001.patch > > > This is the issue I was originally having here: > [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] > > When we pread, we don't force the read to read all of the next block header. > However, when we get into a race condition where two opener threads try to > cache the same block and one thread read all of the next block header and the > other one didn't, it will fail the open process. This is especially important > in a splitting case where it will potentially fail the split process. > Instead, in the caches, we should only fail if the required blocks are > different. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19438) Doc cleanup after removal of features across Cache/BucketCache
[ https://issues.apache.org/jira/browse/HBASE-19438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-19438. --- Resolution: Won't Fix Fix Version/s: (was: 2.0.0) Resolving. The parent issue has a patch that seems to address this JIRA. Will go with it. > Doc cleanup after removal of features across Cache/BucketCache > -- > > Key: HBASE-19438 > URL: https://issues.apache.org/jira/browse/HBASE-19438 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20293) get_splits returns duplicate split points when region replication is on
[
https://issues.apache.org/jira/browse/HBASE-20293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444775#comment-16444775
]
huaxiang sun commented on HBASE-20293:
--
Looks good to me. [~busbey], is it good to commit to branch-1? Thanks.
> get_splits returns duplicate split points when region replication is on
> ---
>
> Key: HBASE-20293
> URL: https://issues.apache.org/jira/browse/HBASE-20293
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Minor
> Attachments: HBASE-20293.branch-1.001.patch,
> HBASE-20293.branch-1.002.patch, HBASE-20293.branch-1.003.patch,
> HBASE-20293.branch-1.004.patch, HBASE-20293.branch-1.005.patch,
> HBASE-20293.master.001.patch, HBASE-20293.master.002.patch,
> HBASE-20293.master.003.patch, HBASE-20293.master.004.patch
>
>
> When region replication is on, get_splits returns duplicate split points like
> the following:
> {code}
> hbase(main):001:0> create "test", "cf", {REGION_REPLICATION => 3}, SPLITS =>
> ["10"]
> Created table test
> Took 1.0975 seconds
> hbase(main):002:0> get_splits "test"
> Total number of splits = 4
> 10
> 10
> 10
> Took 0.0941 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20450) Provide metrics for number of total active, priority and replication rpc handlers
[
https://issues.apache.org/jira/browse/HBASE-20450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444757#comment-16444757
]
Hadoop QA commented on HBASE-20450:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 21s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
25s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}111m 55s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
55s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}163m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:d8b550f |
| JIRA Issue | HBASE-20450 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919850/HBASE-20450.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 4317087f9b4d 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision |
[jira] [Commented] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[ https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444753#comment-16444753 ] Zach York commented on HBASE-20447: --- [~apurtell] Yep, Ted brought up a similar thing with the log level. I'll adjust that to debug. Thanks for the review! [~yuzhih...@gmail.com] This has been a RuntimeException for quite some time. Any hard reasoning on why this should change to an IOE? Otherwise your comments look good, I'll try to incorporate them. [~anoop.hbase] I'm running into some issues getting the test to pass in master (I think) due to the shared memory cache. What is happening is that even after adding a block to the cache, the getBlock is returning the incorrect block. I believe this might be related to it not being put in the backingMap yet, but I'm not sure. Do you have any insight on that? Thanks! > Only fail cacheBlock if block collisions aren't related to next block metadata > -- > > Key: HBASE-20447 > URL: https://issues.apache.org/jira/browse/HBASE-20447 > Project: HBase > Issue Type: Bug > Components: BlockCache, BucketCache >Affects Versions: 1.4.3, 2.0.0 >Reporter: Zach York >Assignee: Zach York >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.4, 2.0.1 > > Attachments: HBASE-20447.branch-1.001.patch, > HBASE-20447.master.001.patch > > > This is the issue I was originally having here: > [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] > > When we pread, we don't force the read to read all of the next block header. > However, when we get into a race condition where two opener threads try to > cache the same block and one thread read all of the next block header and the > other one didn't, it will fail the open process. This is especially important > in a splitting case where it will potentially fail the split process. > Instead, in the caches, we should only fail if the required blocks are > different. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20188: -- Attachment: (was: HBASE-20188.branch-2.0.001.patch) > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, hits.png, hits_with_fp_scheduler.png, > lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, perregion.png, run_ycsb.sh, > total.png, tree.txt, workloadx, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20249) Investigate behavior of hbase.client.operation.timeout
[ https://issues.apache.org/jira/browse/HBASE-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444736#comment-16444736 ] stack commented on HBASE-20249: --- What is this issue about [~appy]? It doesn't work when you set operation timeout? > Investigate behavior of hbase.client.operation.timeout > -- > > Key: HBASE-20249 > URL: https://issues.apache.org/jira/browse/HBASE-20249 > Project: HBase > Issue Type: Bug >Reporter: Appy >Priority: Critical > Fix For: 2.0.0 > > > hbase.client.operation.timeout should be hard limit on client operations, and > independent of number of retires. > Previous discussions here: > https://issues.apache.org/jira/browse/HBASE-17449?focusedCommentId=16390085&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16390085 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20454) [DOC] Add note on perf to upgrade section
[ https://issues.apache.org/jira/browse/HBASE-20454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-20454. --- Resolution: Fixed Pushed to master. > [DOC] Add note on perf to upgrade section > - > > Key: HBASE-20454 > URL: https://issues.apache.org/jira/browse/HBASE-20454 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-20454.branch-2.0.001.patch > > > Add short note on YMMV regards perf and 2.0.0 but we working on it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20454) [DOC] Add note on perf to upgrade section
[ https://issues.apache.org/jira/browse/HBASE-20454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444731#comment-16444731 ] stack commented on HBASE-20454: --- Thanks [~mdrob] Fixed on push to master branch (Pushed to branch-2.0 too but not needed here since we'll be copying back the whole refguide on release). > [DOC] Add note on perf to upgrade section > - > > Key: HBASE-20454 > URL: https://issues.apache.org/jira/browse/HBASE-20454 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-20454.branch-2.0.001.patch > > > Add short note on YMMV regards perf and 2.0.0 but we working on it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20454) [DOC] Add note on perf to upgrade section
[ https://issues.apache.org/jira/browse/HBASE-20454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444725#comment-16444725 ] Mike Drob commented on HBASE-20454: --- lgtm. you have a couple extra characters hanging about: bq. dependent on context.0.0. > [DOC] Add note on perf to upgrade section > - > > Key: HBASE-20454 > URL: https://issues.apache.org/jira/browse/HBASE-20454 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-20454.branch-2.0.001.patch > > > Add short note on YMMV regards perf and 2.0.0 but we working on it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20454) [DOC] Add note on perf to upgrade section
[ https://issues.apache.org/jira/browse/HBASE-20454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20454: -- Attachment: HBASE-20454.branch-2.0.001.patch > [DOC] Add note on perf to upgrade section > - > > Key: HBASE-20454 > URL: https://issues.apache.org/jira/browse/HBASE-20454 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-20454.branch-2.0.001.patch > > > Add short note on YMMV regards perf and 2.0.0 but we working on it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20188: -- Attachment: HBASE-20188.branch-2.0.001.patch > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.branch-2.0.001.patch, HBASE-20188.sh, HBase 2.0 performance > evaluation - 8GB(1).pdf, HBase 2.0 performance evaluation - 8GB.pdf, HBase > 2.0 performance evaluation - Basic vs None_ system settings.pdf, > ITBLL2.5B_1.2.7vs2.0.0_cpu.png, ITBLL2.5B_1.2.7vs2.0.0_gctime.png, > ITBLL2.5B_1.2.7vs2.0.0_iops.png, ITBLL2.5B_1.2.7vs2.0.0_load.png, > ITBLL2.5B_1.2.7vs2.0.0_memheap.png, ITBLL2.5B_1.2.7vs2.0.0_memstore.png, > ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, hits.png, hits_with_fp_scheduler.png, > lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, perregion.png, run_ycsb.sh, > total.png, tree.txt, workloadx, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19121) HBCK for AMv2 (A.K.A HBCK2)
[ https://issues.apache.org/jira/browse/HBASE-19121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444703#comment-16444703 ] stack commented on HBASE-19121: --- A few of us looking at a hosed cluster came up with a scenario that would help with the dev of an hbck2: run a cluster with some loading, kill the master, remove its procedure WALs, and then restart the master. HBCK2 should be able to restore cluster to a working state. This manufacture is good too for figuring scope of what hbck2 needs to provide. Chatting here a few of us, it was noted that the Master UI and shell will not be useable unless the Master completes initialization -- which it will not be able to do if the procedure WAL files held part done procedures (e.g. a Move of the namespace region where the master kill and prcedure wal files were removed after the move unassign but before the moe assign completed). > HBCK for AMv2 (A.K.A HBCK2) > --- > > Key: HBASE-19121 > URL: https://issues.apache.org/jira/browse/HBASE-19121 > Project: HBase > Issue Type: Bug > Components: hbck >Reporter: stack >Priority: Major > > We don't have an hbck for the new AM. Old hbck may actually do damage going > against AMv2. > Fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[ https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444652#comment-16444652 ] Andrew Purtell commented on HBASE-20447: The "Cached an already cached block .. This is harmless" log line should be DEBUG level I think. I have this patched as such internally. Otherwise the warnings concern operators without being actionable. So in your patch where you have introduced more WARN level logs where the comparison differs by nextBlockOnDiskSize the same thing applies I think. Otherwise lgtm > Only fail cacheBlock if block collisions aren't related to next block metadata > -- > > Key: HBASE-20447 > URL: https://issues.apache.org/jira/browse/HBASE-20447 > Project: HBase > Issue Type: Bug > Components: BlockCache, BucketCache >Affects Versions: 1.4.3, 2.0.0 >Reporter: Zach York >Assignee: Zach York >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.4, 2.0.1 > > Attachments: HBASE-20447.branch-1.001.patch, > HBASE-20447.master.001.patch > > > This is the issue I was originally having here: > [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] > > When we pread, we don't force the read to read all of the next block header. > However, when we get into a race condition where two opener threads try to > cache the same block and one thread read all of the next block header and the > other one didn't, it will fail the open process. This is especially important > in a splitting case where it will potentially fail the split process. > Instead, in the caches, we should only fail if the required blocks are > different. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[ https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20447: --- Fix Version/s: 2.0.1 1.4.4 1.5.0 2.1.0 3.0.0 > Only fail cacheBlock if block collisions aren't related to next block metadata > -- > > Key: HBASE-20447 > URL: https://issues.apache.org/jira/browse/HBASE-20447 > Project: HBase > Issue Type: Bug > Components: BlockCache, BucketCache >Affects Versions: 1.4.3, 2.0.0 >Reporter: Zach York >Assignee: Zach York >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.5.0, 1.4.4, 2.0.1 > > Attachments: HBASE-20447.branch-1.001.patch, > HBASE-20447.master.001.patch > > > This is the issue I was originally having here: > [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] > > When we pread, we don't force the read to read all of the next block header. > However, when we get into a race condition where two opener threads try to > cache the same block and one thread read all of the next block header and the > other one didn't, it will fail the open process. This is especially important > in a splitting case where it will potentially fail the split process. > Instead, in the caches, we should only fail if the required blocks are > different. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[ https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-20447: -- Attachment: HBASE-20447.master.001.patch > Only fail cacheBlock if block collisions aren't related to next block metadata > -- > > Key: HBASE-20447 > URL: https://issues.apache.org/jira/browse/HBASE-20447 > Project: HBase > Issue Type: Bug > Components: BlockCache, BucketCache >Affects Versions: 1.4.3, 2.0.0 >Reporter: Zach York >Assignee: Zach York >Priority: Major > Attachments: HBASE-20447.branch-1.001.patch, > HBASE-20447.master.001.patch > > > This is the issue I was originally having here: > [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] > > When we pread, we don't force the read to read all of the next block header. > However, when we get into a race condition where two opener threads try to > cache the same block and one thread read all of the next block header and the > other one didn't, it will fail the open process. This is especially important > in a splitting case where it will potentially fail the split process. > Instead, in the caches, we should only fail if the required blocks are > different. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20332) shaded mapreduce module shouldn't include hadoop
[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444581#comment-16444581
]
Sean Busbey edited comment on HBASE-20332 at 4/19/18 6:46 PM:
--
v2 WIP fix the issues pointed out by qabot
- checkstyle corrections
- strip trailing whitespace
- handle shellcheck warning
- hbase-backup's hadoop 3 profile was wrong.
was (Author: busbey):
- v2 WIP fix the issues pointed out by qabot
- checkstyle corrections
- strip trailing whitespace
- handle shellcheck warning
- hbase-backup's hadoop 3 profile was wrong.
> shaded mapreduce module shouldn't include hadoop
>
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch,
> HBASE-20332.2.WIP.patch
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround
> (e.g. for some library your application needs that hadoop provides) then our
> inclusion of *some but not all* hadoop classes then causes everything to fall
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're
> just wasting space
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20332) shaded mapreduce module shouldn't include hadoop
[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20332:
Status: Patch Available (was: In Progress)
- v2 WIP fix the issues pointed out by qabot
- checkstyle corrections
- strip trailing whitespace
- handle shellcheck warning
- hbase-backup's hadoop 3 profile was wrong.
> shaded mapreduce module shouldn't include hadoop
>
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch,
> HBASE-20332.2.WIP.patch
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround
> (e.g. for some library your application needs that hadoop provides) then our
> inclusion of *some but not all* hadoop classes then causes everything to fall
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're
> just wasting space
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20332) shaded mapreduce module shouldn't include hadoop
[
https://issues.apache.org/jira/browse/HBASE-20332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20332:
Attachment: HBASE-20332.2.WIP.patch
> shaded mapreduce module shouldn't include hadoop
>
>
> Key: HBASE-20332
> URL: https://issues.apache.org/jira/browse/HBASE-20332
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, shading
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-20332.0.patch, HBASE-20332.1.WIP.patch,
> HBASE-20332.2.WIP.patch
>
>
> AFAICT, we should just entirely skip including hadoop in our shaded mapreduce
> module
> 1) Folks expect to run yarn / mr apps via {{hadoop jar}} / {{yarn jar}}
> 2) those commands include all the needed Hadoop jars in your classpath by
> default (both client side and in the containers)
> 3) If you try to use "user classpath first" for your job as a workaround
> (e.g. for some library your application needs that hadoop provides) then our
> inclusion of *some but not all* hadoop classes then causes everything to fall
> over because of mixing rewritten and non-rewritten hadoop classes
> 4) if you don't use "user classpath first" then all of our
> non-relocated-but-still-shaded hadoop classes are ignored anyways so we're
> just wasting space
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20450) Provide metrics for number of total active, priority and replication rpc handlers
[ https://issues.apache.org/jira/browse/HBASE-20450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444549#comment-16444549 ] Nihal Jain commented on HBASE-20450: My bad. :P Thanks for the review [~yuzhih...@gmail.com]. Will attach a patch with the fix. > Provide metrics for number of total active, priority and replication rpc > handlers > - > > Key: HBASE-20450 > URL: https://issues.apache.org/jira/browse/HBASE-20450 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20450.master.001.patch > > > Currently hbase provides a metric for [number of total active rpc > handlers|https://github.com/apache/hbase/blob/f4f2b68238a094d7b1931dc8b7939742ccbb2b57/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java#L187] > which is a sum of the following: > * number of active general rpc handlers > * number of active priority rpc handlers > * number of active replication rpc handlers > I think we can have 3 different metrics corresponding to the above mentioned > handlers which will allow us to see detailed information about number of > active handlers running for a particular type of handler. > We can have following new metrics: > * numActiveGeneralHandler > * numActivePriorityHandler > * numActiveReplicationHandler > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20450) Provide metrics for number of total active, priority and replication rpc handlers
[ https://issues.apache.org/jira/browse/HBASE-20450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1664#comment-1664 ] Nihal Jain commented on HBASE-20450: Hi I have attached the patch where I have added three new metrics to retrieve counts of general, priority and replication rpc handlers. > Provide metrics for number of total active, priority and replication rpc > handlers > - > > Key: HBASE-20450 > URL: https://issues.apache.org/jira/browse/HBASE-20450 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20450.master.001.patch > > > Currently hbase provides a metric for [number of total active rpc > handlers|https://github.com/apache/hbase/blob/f4f2b68238a094d7b1931dc8b7939742ccbb2b57/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java#L187] > which is a sum of the following: > * number of active general rpc handlers > * number of active priority rpc handlers > * number of active replication rpc handlers > I think we can have 3 different metrics corresponding to the above mentioned > handlers which will allow us to see detailed information about number of > active handlers running for a particular type of handler. > We can have following new metrics: > * numActiveGeneralHandler > * numActivePriorityHandler > * numActiveReplicationHandler > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20450) Provide metrics for number of total active, priority and replication rpc handlers
[
https://issues.apache.org/jira/browse/HBASE-20450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1667#comment-1667
]
Ted Yu commented on HBASE-20450:
{code}
+ String NUM_ACTIVE_HANDLER_DESC = "Number of total active rpc handlers.";
{code}
Number of total -> Total number of
The rest looks okay.
> Provide metrics for number of total active, priority and replication rpc
> handlers
> -
>
> Key: HBASE-20450
> URL: https://issues.apache.org/jira/browse/HBASE-20450
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-20450.master.001.patch
>
>
> Currently hbase provides a metric for [number of total active rpc
> handlers|https://github.com/apache/hbase/blob/f4f2b68238a094d7b1931dc8b7939742ccbb2b57/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java#L187]
> which is a sum of the following:
> * number of active general rpc handlers
> * number of active priority rpc handlers
> * number of active replication rpc handlers
> I think we can have 3 different metrics corresponding to the above mentioned
> handlers which will allow us to see detailed information about number of
> active handlers running for a particular type of handler.
> We can have following new metrics:
> * numActiveGeneralHandler
> * numActivePriorityHandler
> * numActiveReplicationHandler
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20450) Provide metrics for number of total active, priority and replication rpc handlers
[ https://issues.apache.org/jira/browse/HBASE-20450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20450: --- Fix Version/s: 3.0.0 Status: Patch Available (was: Open) > Provide metrics for number of total active, priority and replication rpc > handlers > - > > Key: HBASE-20450 > URL: https://issues.apache.org/jira/browse/HBASE-20450 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20450.master.001.patch > > > Currently hbase provides a metric for [number of total active rpc > handlers|https://github.com/apache/hbase/blob/f4f2b68238a094d7b1931dc8b7939742ccbb2b57/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java#L187] > which is a sum of the following: > * number of active general rpc handlers > * number of active priority rpc handlers > * number of active replication rpc handlers > I think we can have 3 different metrics corresponding to the above mentioned > handlers which will allow us to see detailed information about number of > active handlers running for a particular type of handler. > We can have following new metrics: > * numActiveGeneralHandler > * numActivePriorityHandler > * numActiveReplicationHandler > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20450) Provide metrics for number of total active, priority and replication rpc handlers
[ https://issues.apache.org/jira/browse/HBASE-20450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-20450: --- Attachment: HBASE-20450.master.001.patch > Provide metrics for number of total active, priority and replication rpc > handlers > - > > Key: HBASE-20450 > URL: https://issues.apache.org/jira/browse/HBASE-20450 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Attachments: HBASE-20450.master.001.patch > > > Currently hbase provides a metric for [number of total active rpc > handlers|https://github.com/apache/hbase/blob/f4f2b68238a094d7b1931dc8b7939742ccbb2b57/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java#L187] > which is a sum of the following: > * number of active general rpc handlers > * number of active priority rpc handlers > * number of active replication rpc handlers > I think we can have 3 different metrics corresponding to the above mentioned > handlers which will allow us to see detailed information about number of > active handlers running for a particular type of handler. > We can have following new metrics: > * numActiveGeneralHandler > * numActivePriorityHandler > * numActiveReplicationHandler > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1631#comment-1631 ] Thiriguna Bharat Rao commented on HBASE-20369: -- Many thanks [~mdrob], for educating me about the QA bot. Point noted, will not address QA in future communication. Appreciate your support and time. Best, Triguna > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19924) hbase rpc throttling does not work for multi() with request count rater.
[
https://issues.apache.org/jira/browse/HBASE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1614#comment-1614
]
huaxiang sun commented on HBASE-19924:
--
Thanks [~stack]. I will wait for some more time to commit in case [~chia7712]
has comments.
> hbase rpc throttling does not work for multi() with request count rater.
>
>
> Key: HBASE-19924
> URL: https://issues.apache.org/jira/browse/HBASE-19924
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 0.16.0, 1.2.6
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Major
> Attachments: HBASE-19924-master-v001.patch,
> HBASE-19924-master-v002.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch, HBASE-19924-master-v002.patch
>
>
> Basically, rpc throttling does not work for request count based rater for
> multi. for the following code, when it calls limiter's checkQuota(),
> numWrites/numReads is lost.
> {code:java}
> @Override
> public void checkQuota(int numWrites, int numReads, int numScans) throws
> ThrottlingException {
> writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);
> readConsumed = estimateConsume(OperationType.GET, numReads, 100);
> readConsumed += estimateConsume(OperationType.SCAN, numScans, 1000);
> writeAvailable = Long.MAX_VALUE;
> readAvailable = Long.MAX_VALUE;
> for (final QuotaLimiter limiter : limiters) {
> if (limiter.isBypass()) continue;
> limiter.checkQuota(writeConsumed, readConsumed);
> readAvailable = Math.min(readAvailable, limiter.getReadAvailable());
> writeAvailable = Math.min(writeAvailable, limiter.getWriteAvailable());
> }
> for (final QuotaLimiter limiter : limiters) {
> limiter.grabQuota(writeConsumed, readConsumed);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1611#comment-1611 ] Mike Drob commented on HBASE-20369: --- [~trigunab] - the QA bot will automatically run whenever you upload a new patch file if the jira status is set to "patch available" - no need to address the QA bot in comments directly. > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20369) Document incompatibilities between HBase 1.1.2 and HBase 2.0
[ https://issues.apache.org/jira/browse/HBASE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444398#comment-16444398 ] Thiriguna Bharat Rao commented on HBASE-20369: -- Many thanks for verifying the version 2 patch. I reviewed the doc changes in the patch site. [~busbey] Looks like, lot of formatting has to happen and I am on it, but I will be able to submit the new version of the patch tomorrow. In my opinion, white spaces are required between various headings to make the content look neat. Will keep you and QA updated after I post the newer version of the patch with the formatting changes. Appreciate your support and time. Best, Triguna > Document incompatibilities between HBase 1.1.2 and HBase 2.0 > > > Key: HBASE-20369 > URL: https://issues.apache.org/jira/browse/HBASE-20369 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Thiriguna Bharat Rao >Assignee: Thiriguna Bharat Rao >Priority: Critical > Labels: patch > Attachments: HBASE-20369.patch, HBASE-20369_v1.patch, > HBASE-20369_v2.patch, book.adoc > > > Hi, > I compiled a draft document for the HBase incompatibilities from the raw > source content that was available in HBase Beta 1 site. Can someone please > review and provide a feedback or share your comments on this document? > Appreciate your support and time. > > Best Regards, > Triguna -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19924) hbase rpc throttling does not work for multi() with request count rater.
[
https://issues.apache.org/jira/browse/HBASE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444397#comment-16444397
]
stack commented on HBASE-19924:
---
[~huaxiang] Looks reasonable to me +1 for branch-2.
> hbase rpc throttling does not work for multi() with request count rater.
>
>
> Key: HBASE-19924
> URL: https://issues.apache.org/jira/browse/HBASE-19924
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 0.16.0, 1.2.6
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Major
> Attachments: HBASE-19924-master-v001.patch,
> HBASE-19924-master-v002.patch, HBASE-19924-master-v002.patch,
> HBASE-19924-master-v002.patch, HBASE-19924-master-v002.patch
>
>
> Basically, rpc throttling does not work for request count based rater for
> multi. for the following code, when it calls limiter's checkQuota(),
> numWrites/numReads is lost.
> {code:java}
> @Override
> public void checkQuota(int numWrites, int numReads, int numScans) throws
> ThrottlingException {
> writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);
> readConsumed = estimateConsume(OperationType.GET, numReads, 100);
> readConsumed += estimateConsume(OperationType.SCAN, numScans, 1000);
> writeAvailable = Long.MAX_VALUE;
> readAvailable = Long.MAX_VALUE;
> for (final QuotaLimiter limiter : limiters) {
> if (limiter.isBypass()) continue;
> limiter.checkQuota(writeConsumed, readConsumed);
> readAvailable = Math.min(readAvailable, limiter.getReadAvailable());
> writeAvailable = Math.min(writeAvailable, limiter.getWriteAvailable());
> }
> for (final QuotaLimiter limiter : limiters) {
> limiter.grabQuota(writeConsumed, readConsumed);
> }
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
