[jira] [Commented] (HBASE-19035) Miss metrics when coprocessor use region scanner to read data
[ https://issues.apache.org/jira/browse/HBASE-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248805#comment-16248805 ] Guanghao Zhang commented on HBASE-19035: Pushed to master and branch-2. Thanks all for reviewing. Let me check this for branch-1. > Miss metrics when coprocessor use region scanner to read data > - > > Key: HBASE-19035 > URL: https://issues.apache.org/jira/browse/HBASE-19035 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19035.master.001.patch, > HBASE-19035.master.002.patch, HBASE-19035.master.003.patch > > > Region interface is exposed to coprocessor. So coprocessor use getScanner to > get a region scanner to read data. But the scan metrics was only updated in > region server level. So we will miss some scan metrics for the read from > coprocessor. > || Region Operation || When to update requests metric || > | get | update read metric in nextRaw() | > | put | update write metric in batchMutate() | > | delete | update write metric in batchMutate() | > | increment | update read metric by get() and update write metric in > doDelta() | > | append | update read metric by get() and update write metric in doDelta() | > | mutateRow | update write metric in processRowsWithLocks() | > | mutateRowsWithLocks | update write metric in processRowsWithLocks() | > | batchMutate | update write metric in batchMutate() | > | checkAndMutate | update read metric by get() and update write metric by > mutateRow() | > | checkAndRowMutate | update read metric by get() and update write metric by > doBatchMutate() | > | processRowsWithLocks | update write metric in processRowsWithLocks() | > 1. Move read requests to region level. Because RegionScanner exposed to CP. > 2. Update write requests count in processRowsWithLocks. This was missed in > previous implemenation, too. > 3. Remove requestRowActionCount in RSRpcServices. This metric can be computed > by region's readRequestsCount and writeRequestsCount. > Upload to review board: https://reviews.apache.org/r/63579/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18601) Update Htrace to 4.2
[ https://issues.apache.org/jira/browse/HBASE-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248801#comment-16248801 ] Guanghao Zhang commented on HBASE-18601: TestHTraceHooks is failed. Is it releated? [INFO] Running org.apache.hadoop.hbase.trace.TestHTraceHooks [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 31.138 s <<< FAILURE! - in org.apache.hadoop.hbase.trace.TestHTraceHooks [ERROR] testTraceCreateTable(org.apache.hadoop.hbase.trace.TestHTraceHooks) Time elapsed: 1.752 s <<< FAILURE! java.lang.AssertionError: expected:<3> but was:<1> at org.apache.hadoop.hbase.trace.TestHTraceHooks.testTraceCreateTable(TestHTraceHooks.java:100) > Update Htrace to 4.2 > > > Key: HBASE-18601 > URL: https://issues.apache.org/jira/browse/HBASE-18601 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0, 3.0.0 >Reporter: Tamas Penzes >Assignee: Balazs Meszaros > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18601.master.001.patch, > HBASE-18601.master.002.patch, HBASE-18601.master.003 (3).patch, > HBASE-18601.master.003.patch, HBASE-18601.master.004.patch, > HBASE-18601.master.004.patch, HBASE-18601.master.005.patch, > HBASE-18601.master.006.patch, HBASE-18601.master.006.patch, > HBASE-18601.master.007.patch, HBASE-18601.master.007.patch, > HBASE-18601.master.007.patch, HBASE-18601.master.008.patch, > HBASE-18601.master.009.patch, HBASE-18601.master.009.patch, > HBASE-18601.master.010.patch, HBASE-18601.master.010.patch, > HBASE-18601.master.011.patch, HBASE-18601.master.012.patch, > HBASE-18601.master.013.patch, HBASE-18601.master.014.patch, > HBASE-18601.master.014.patch, HBASE-18601.master.015.patch > > > HTrace is not perfectly integrated into HBase, the version 3.2.0 is buggy, > the upgrade to 4.x is not trivial and would take time. It might not worth to > keep it in this state, so would be better to remove it. > Of course it doesn't mean tracing would be useless, just that in this form > the use of HTrace 3.2 might not add any value to the project and fixing it > would be far too much effort. > - > Based on the decision of the community we keep htrace now and update version -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19035) Miss metrics when coprocessor use region scanner to read data
[ https://issues.apache.org/jira/browse/HBASE-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248802#comment-16248802 ] Guanghao Zhang commented on HBASE-19035: TestHTraceHooks is not related. And it failed on master branch, too. > Miss metrics when coprocessor use region scanner to read data > - > > Key: HBASE-19035 > URL: https://issues.apache.org/jira/browse/HBASE-19035 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19035.master.001.patch, > HBASE-19035.master.002.patch, HBASE-19035.master.003.patch > > > Region interface is exposed to coprocessor. So coprocessor use getScanner to > get a region scanner to read data. But the scan metrics was only updated in > region server level. So we will miss some scan metrics for the read from > coprocessor. > || Region Operation || When to update requests metric || > | get | update read metric in nextRaw() | > | put | update write metric in batchMutate() | > | delete | update write metric in batchMutate() | > | increment | update read metric by get() and update write metric in > doDelta() | > | append | update read metric by get() and update write metric in doDelta() | > | mutateRow | update write metric in processRowsWithLocks() | > | mutateRowsWithLocks | update write metric in processRowsWithLocks() | > | batchMutate | update write metric in batchMutate() | > | checkAndMutate | update read metric by get() and update write metric by > mutateRow() | > | checkAndRowMutate | update read metric by get() and update write metric by > doBatchMutate() | > | processRowsWithLocks | update write metric in processRowsWithLocks() | > 1. Move read requests to region level. Because RegionScanner exposed to CP. > 2. Update write requests count in processRowsWithLocks. This was missed in > previous implemenation, too. > 3. Remove requestRowActionCount in RSRpcServices. This metric can be computed > by region's readRequestsCount and writeRequestsCount. > Upload to review board: https://reviews.apache.org/r/63579/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19035) Miss metrics when coprocessor use region scanner to read data
[
https://issues.apache.org/jira/browse/HBASE-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248794#comment-16248794
]
Hadoop QA commented on HBASE-19035:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
46s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
54m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}134m 55s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}208m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.trace.TestHTraceHooks |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19035 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12897223/HBASE-19035.master.003.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 0f711657db82 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / c87189d418 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/9748/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9748/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9748/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was
[jira] [Commented] (HBASE-19226) Limit the reduce tasks number of incremental load
[ https://issues.apache.org/jira/browse/HBASE-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248785#comment-16248785 ] Reid Chan commented on HBASE-19226: --- Got it, thanks for your explanation [~Yun Zhao] What about supports not only ratio but also a specified number(int) > 1, like 5 reducers wanted, i'd prefer a 5 rather than 0.005 if hbase has 1000 regions or more which always happens in production. > Limit the reduce tasks number of incremental load > - > > Key: HBASE-19226 > URL: https://issues.apache.org/jira/browse/HBASE-19226 > Project: HBase > Issue Type: Improvement >Reporter: Yun Zhao >Assignee: Yun Zhao >Priority: Minor > Attachments: HBASE-19226.master.001.patch, > HBASE-19226.master.002.patch > > > When using MapReduce job to perform an incremental load into a table,the > number of reduce tasks is the current number of regions. If there are too > many regions, will lead to network+disk I/O is too large, affecting the > real-time request. > Need to use a configuration to set a number or ratio? > Limit running reduce tasks since > [https://issues.apache.org/jira/browse/MAPREDUCE-5583], the old version can > only be set reduce number. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19226) Limit the reduce tasks number of incremental load
[ https://issues.apache.org/jira/browse/HBASE-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248779#comment-16248779 ] Yun Zhao commented on HBASE-19226: -- Thanks for your review [~reidchan] The buckets used to record the number of regions for each partition. When offset is equal to the first bucket number, use the current startKey as a splitkey and start counting the next bucket. Ignores the startkey of the first bucket, replacing sorted.remove(sorted.first()); > Limit the reduce tasks number of incremental load > - > > Key: HBASE-19226 > URL: https://issues.apache.org/jira/browse/HBASE-19226 > Project: HBase > Issue Type: Improvement >Reporter: Yun Zhao >Assignee: Yun Zhao >Priority: Minor > Attachments: HBASE-19226.master.001.patch, > HBASE-19226.master.002.patch > > > When using MapReduce job to perform an incremental load into a table,the > number of reduce tasks is the current number of regions. If there are too > many regions, will lead to network+disk I/O is too large, affecting the > real-time request. > Need to use a configuration to set a number or ratio? > Limit running reduce tasks since > [https://issues.apache.org/jira/browse/MAPREDUCE-5583], the old version can > only be set reduce number. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-15631) Backport Regionserver Groups (HBASE-6721) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248769#comment-16248769 ] Sean Busbey edited comment on HBASE-15631 at 11/12/17 2:46 AM: --- [~apurtell], any objection to me deleting the "HBASE\-15631\-branch\-1" branch? was (Author: busbey): [~apurtell], any objection to me deleting the "HBASE-15631-branch-1" branch? > Backport Regionserver Groups (HBASE-6721) to branch-1 > -- > > Key: HBASE-15631 > URL: https://issues.apache.org/jira/browse/HBASE-15631 > Project: HBase > Issue Type: New Feature >Affects Versions: 1.4.0 >Reporter: Francis Liu >Assignee: Andrew Purtell > Fix For: 1.4.0 > > Attachments: HBASE-15631-branch-1-addendum.patch, > HBASE-15631-branch-1.patch, HBASE-15631-branch-1.patch, > HBASE-15631-branch-1.patch, HBASE-15631.branch-1.patch, HBASE-15631.patch > > > Based on dev list discussion backporting region server group should not be an > issue as it does not: 1. destabilize the code. 2. cause backward > incompatibility. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15631) Backport Regionserver Groups (HBASE-6721) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248769#comment-16248769 ] Sean Busbey commented on HBASE-15631: - [~apurtell], any objection to me deleting the "HBASE-15631-branch-1" branch? > Backport Regionserver Groups (HBASE-6721) to branch-1 > -- > > Key: HBASE-15631 > URL: https://issues.apache.org/jira/browse/HBASE-15631 > Project: HBase > Issue Type: New Feature >Affects Versions: 1.4.0 >Reporter: Francis Liu >Assignee: Andrew Purtell > Fix For: 1.4.0 > > Attachments: HBASE-15631-branch-1-addendum.patch, > HBASE-15631-branch-1.patch, HBASE-15631-branch-1.patch, > HBASE-15631-branch-1.patch, HBASE-15631.branch-1.patch, HBASE-15631.patch > > > Based on dev list discussion backporting region server group should not be an > issue as it does not: 1. destabilize the code. 2. cause backward > incompatibility. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19223) Remove references to Date Tiered compaction from branch-1.2 and branch-1.1 ref guide
[ https://issues.apache.org/jira/browse/HBASE-19223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-19223: Attachment: HBASE-19223-branch-1.2.v1.patch -1 - update the upgrading section to be "0.98 -> 1.x" instead of "1.0.0" - add a note about availability of Date Tiered Compaction feature for branches newer than branch-1.2, just the changes to the upgrade section would be included. > Remove references to Date Tiered compaction from branch-1.2 and branch-1.1 > ref guide > > > Key: HBASE-19223 > URL: https://issues.apache.org/jira/browse/HBASE-19223 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Affects Versions: 1.1.5, 1.2.2 >Reporter: chilianyi >Assignee: Sean Busbey >Priority: Critical > Fix For: 1.2.7, 1.1.13 > > Attachments: HBASE-19223-branch-1.2.v0.patch, > HBASE-19223-branch-1.2.v1.patch > > > I have download hbase 1.2.6 src package. But not found class > DateTieredStoreEngine, which is present in document: > https://hbase.apache.org/1.2/book.html. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19223) Remove references to Date Tiered compaction from branch-1.2 and branch-1.1 ref guide
[ https://issues.apache.org/jira/browse/HBASE-19223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-19223: Status: Patch Available (was: In Progress) > Remove references to Date Tiered compaction from branch-1.2 and branch-1.1 > ref guide > > > Key: HBASE-19223 > URL: https://issues.apache.org/jira/browse/HBASE-19223 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Affects Versions: 1.2.2, 1.1.5 >Reporter: chilianyi >Assignee: Sean Busbey >Priority: Critical > Fix For: 1.2.7, 1.1.13 > > Attachments: HBASE-19223-branch-1.2.v0.patch, > HBASE-19223-branch-1.2.v1.patch > > > I have download hbase 1.2.6 src package. But not found class > DateTieredStoreEngine, which is present in document: > https://hbase.apache.org/1.2/book.html. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19035) Miss metrics when coprocessor use region scanner to read data
[ https://issues.apache.org/jira/browse/HBASE-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19035: --- Attachment: HBASE-19035.master.003.patch Rebase. > Miss metrics when coprocessor use region scanner to read data > - > > Key: HBASE-19035 > URL: https://issues.apache.org/jira/browse/HBASE-19035 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19035.master.001.patch, > HBASE-19035.master.002.patch, HBASE-19035.master.003.patch > > > Region interface is exposed to coprocessor. So coprocessor use getScanner to > get a region scanner to read data. But the scan metrics was only updated in > region server level. So we will miss some scan metrics for the read from > coprocessor. > || Region Operation || When to update requests metric || > | get | update read metric in nextRaw() | > | put | update write metric in batchMutate() | > | delete | update write metric in batchMutate() | > | increment | update read metric by get() and update write metric in > doDelta() | > | append | update read metric by get() and update write metric in doDelta() | > | mutateRow | update write metric in processRowsWithLocks() | > | mutateRowsWithLocks | update write metric in processRowsWithLocks() | > | batchMutate | update write metric in batchMutate() | > | checkAndMutate | update read metric by get() and update write metric by > mutateRow() | > | checkAndRowMutate | update read metric by get() and update write metric by > doBatchMutate() | > | processRowsWithLocks | update write metric in processRowsWithLocks() | > 1. Move read requests to region level. Because RegionScanner exposed to CP. > 2. Update write requests count in processRowsWithLocks. This was missed in > previous implemenation, too. > 3. Remove requestRowActionCount in RSRpcServices. This metric can be computed > by region's readRequestsCount and writeRequestsCount. > Upload to review board: https://reviews.apache.org/r/63579/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19035) Miss metrics when coprocessor use region scanner to read data
[ https://issues.apache.org/jira/browse/HBASE-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19035: --- Release Note: 1. Move read requests count to region level. Because RegionScanner is exposed to CP. 2. Update write requests count in processRowsWithLocks. 3. Remove requestRowActionCount in RSRpcServices. This metric can be computed by region's readRequestsCount and writeRequestsCount. was: 1. Move read requests to region level. Because RegionScanner exposed to CP. 2. Update write requests count in processRowsWithLocks. 3. Remove requestRowActionCount in RSRpcServices. This metric can be computed by region's readRequestsCount and writeRequestsCount. > Miss metrics when coprocessor use region scanner to read data > - > > Key: HBASE-19035 > URL: https://issues.apache.org/jira/browse/HBASE-19035 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19035.master.001.patch, > HBASE-19035.master.002.patch > > > Region interface is exposed to coprocessor. So coprocessor use getScanner to > get a region scanner to read data. But the scan metrics was only updated in > region server level. So we will miss some scan metrics for the read from > coprocessor. > || Region Operation || When to update requests metric || > | get | update read metric in nextRaw() | > | put | update write metric in batchMutate() | > | delete | update write metric in batchMutate() | > | increment | update read metric by get() and update write metric in > doDelta() | > | append | update read metric by get() and update write metric in doDelta() | > | mutateRow | update write metric in processRowsWithLocks() | > | mutateRowsWithLocks | update write metric in processRowsWithLocks() | > | batchMutate | update write metric in batchMutate() | > | checkAndMutate | update read metric by get() and update write metric by > mutateRow() | > | checkAndRowMutate | update read metric by get() and update write metric by > doBatchMutate() | > | processRowsWithLocks | update write metric in processRowsWithLocks() | > 1. Move read requests to region level. Because RegionScanner exposed to CP. > 2. Update write requests count in processRowsWithLocks. This was missed in > previous implemenation, too. > 3. Remove requestRowActionCount in RSRpcServices. This metric can be computed > by region's readRequestsCount and writeRequestsCount. > Upload to review board: https://reviews.apache.org/r/63579/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19035) Miss metrics when coprocessor use region scanner to read data
[ https://issues.apache.org/jira/browse/HBASE-19035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19035: --- Release Note: 1. Move read requests to region level. Because RegionScanner exposed to CP. 2. Update write requests count in processRowsWithLocks. 3. Remove requestRowActionCount in RSRpcServices. This metric can be computed by region's readRequestsCount and writeRequestsCount. > Miss metrics when coprocessor use region scanner to read data > - > > Key: HBASE-19035 > URL: https://issues.apache.org/jira/browse/HBASE-19035 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19035.master.001.patch, > HBASE-19035.master.002.patch > > > Region interface is exposed to coprocessor. So coprocessor use getScanner to > get a region scanner to read data. But the scan metrics was only updated in > region server level. So we will miss some scan metrics for the read from > coprocessor. > || Region Operation || When to update requests metric || > | get | update read metric in nextRaw() | > | put | update write metric in batchMutate() | > | delete | update write metric in batchMutate() | > | increment | update read metric by get() and update write metric in > doDelta() | > | append | update read metric by get() and update write metric in doDelta() | > | mutateRow | update write metric in processRowsWithLocks() | > | mutateRowsWithLocks | update write metric in processRowsWithLocks() | > | batchMutate | update write metric in batchMutate() | > | checkAndMutate | update read metric by get() and update write metric by > mutateRow() | > | checkAndRowMutate | update read metric by get() and update write metric by > doBatchMutate() | > | processRowsWithLocks | update write metric in processRowsWithLocks() | > 1. Move read requests to region level. Because RegionScanner exposed to CP. > 2. Update write requests count in processRowsWithLocks. This was missed in > previous implemenation, too. > 3. Remove requestRowActionCount in RSRpcServices. This metric can be computed > by region's readRequestsCount and writeRequestsCount. > Upload to review board: https://reviews.apache.org/r/63579/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19226) Limit the reduce tasks number of incremental load
[
https://issues.apache.org/jira/browse/HBASE-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248760#comment-16248760
]
Reid Chan commented on HBASE-19226:
---
Would you mind explaining the following codes?
{code}
for (ImmutableBytesWritable startKey : sorted) {
if (offset == bucket[bucketIndex]) {
writer.append(startKey, NullWritable.get());
bucketIndex++;
offset = 0;
}
offset++;
}
{code}
> Limit the reduce tasks number of incremental load
> -
>
> Key: HBASE-19226
> URL: https://issues.apache.org/jira/browse/HBASE-19226
> Project: HBase
> Issue Type: Improvement
>Reporter: Yun Zhao
>Assignee: Yun Zhao
>Priority: Minor
> Attachments: HBASE-19226.master.001.patch,
> HBASE-19226.master.002.patch
>
>
> When using MapReduce job to perform an incremental load into a table,the
> number of reduce tasks is the current number of regions. If there are too
> many regions, will lead to network+disk I/O is too large, affecting the
> real-time request.
> Need to use a configuration to set a number or ratio?
> Limit running reduce tasks since
> [https://issues.apache.org/jira/browse/MAPREDUCE-5583], the old version can
> only be set reduce number.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19226) Limit the reduce tasks number of incremental load
[ https://issues.apache.org/jira/browse/HBASE-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Zhao updated HBASE-19226: - Release Note: Add a configuration item `hbase.mapreduce.hfileoutputformat.reduces.ratio`, it is used to adjust the reduce tasks number for the MapReduce job when incremental load into a table，which is the ratio of the regions number between 0 and 1. > Limit the reduce tasks number of incremental load > - > > Key: HBASE-19226 > URL: https://issues.apache.org/jira/browse/HBASE-19226 > Project: HBase > Issue Type: Improvement >Reporter: Yun Zhao >Assignee: Yun Zhao >Priority: Minor > Attachments: HBASE-19226.master.001.patch, > HBASE-19226.master.002.patch > > > When using MapReduce job to perform an incremental load into a table,the > number of reduce tasks is the current number of regions. If there are too > many regions, will lead to network+disk I/O is too large, affecting the > real-time request. > Need to use a configuration to set a number or ratio? > Limit running reduce tasks since > [https://issues.apache.org/jira/browse/MAPREDUCE-5583], the old version can > only be set reduce number. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19242) Add MOB compact support for AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-19242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang updated HBASE-19242:
---
Issue Type: Sub-task (was: Task)
Parent: HBASE-17856
> Add MOB compact support for AsyncAdmin
> --
>
> Key: HBASE-19242
> URL: https://issues.apache.org/jira/browse/HBASE-19242
> Project: HBase
> Issue Type: Sub-task
> Components: Admin, mob
>Reporter: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
>
> {code}
> private CompletableFuture compact(TableName tableName, byte[]
> columnFamily, boolean major,
> CompactType compactType) {
> if (CompactType.MOB.equals(compactType)) {
> // TODO support MOB compact.
> return failedFuture(new UnsupportedOperationException("MOB compact does
> not support"));
> }
> {code}
> We need to support it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19242) Add MOB compact support for AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-19242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248753#comment-16248753
]
Guanghao Zhang commented on HBASE-19242:
Move this to a sub-task of HBASE-17856.
> Add MOB compact support for AsyncAdmin
> --
>
> Key: HBASE-19242
> URL: https://issues.apache.org/jira/browse/HBASE-19242
> Project: HBase
> Issue Type: Sub-task
> Components: Admin, mob
>Reporter: Duo Zhang
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
>
> {code}
> private CompletableFuture compact(TableName tableName, byte[]
> columnFamily, boolean major,
> CompactType compactType) {
> if (CompactType.MOB.equals(compactType)) {
> // TODO support MOB compact.
> return failedFuture(new UnsupportedOperationException("MOB compact does
> not support"));
> }
> {code}
> We need to support it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19241) Javadoc improvement for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248752#comment-16248752 ] Guanghao Zhang commented on HBASE-19241: +1 after fix the checkstyle warnings. Meanwhile, please add one line change change to trigger hbase-server test. Thanks. > Javadoc improvement for AsyncAdmin > -- > > Key: HBASE-19241 > URL: https://issues.apache.org/jira/browse/HBASE-19241 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19241.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19227) Nightly jobs should archive JVM dumpstream files
[
https://issues.apache.org/jira/browse/HBASE-19227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248749#comment-16248749
]
Hudson commented on HBASE-19227:
FAILURE: Integrated in Jenkins build HBase-1.5 #151 (See
[https://builds.apache.org/job/HBase-1.5/151/])
HBASE-19227 Nightly job should archive JVM dumpstream files. (busbey: rev
da8e6153d58519fa9dae7eabe785654ba6018f12)
* (edit) dev-support/Jenkinsfile
> Nightly jobs should archive JVM dumpstream files
>
>
> Key: HBASE-19227
> URL: https://issues.apache.org/jira/browse/HBASE-19227
> Project: HBase
> Issue Type: Improvement
> Components: build
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 2.0.0, 3.0.0, 1.0.4, 1.2.7, 1.1.13
>
> Attachments: HBASE-19227.0.patch, HBASE-19227.1.patch
>
>
> came up on dev@ discussion about some of our current nightly test failures.
> when surefire fails to launch a test JVM instance, the details go into a file
> that we currently don't archive:
> {quote}
> [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump,
> [date].dumpstream and [date]-jvmRun[N].dumpstream.
> {quote}
> Add them to the default archive pattern.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19223) Remove references to Date Tiered compaction from branch-1.2 and branch-1.1 ref guide
[ https://issues.apache.org/jira/browse/HBASE-19223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-19223: Status: In Progress (was: Patch Available) > Remove references to Date Tiered compaction from branch-1.2 and branch-1.1 > ref guide > > > Key: HBASE-19223 > URL: https://issues.apache.org/jira/browse/HBASE-19223 > Project: HBase > Issue Type: Bug > Components: Compaction, documentation >Affects Versions: 1.2.2, 1.1.5 >Reporter: chilianyi >Assignee: Sean Busbey >Priority: Critical > Fix For: 1.2.7, 1.1.13 > > Attachments: HBASE-19223-branch-1.2.v0.patch > > > I have download hbase 1.2.6 src package. But not found class > DateTieredStoreEngine, which is present in document: > https://hbase.apache.org/1.2/book.html. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19204) branch-1.2 times out and is taking 6-7 hours to complete
[ https://issues.apache.org/jira/browse/HBASE-19204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248731#comment-16248731 ] Chia-Ping Tsai commented on HBASE-19204: bq. Unrelated, I can't remember where, but in an issue you raised the notion of a 'RawCell'. HBASE-19112 bq. TestThriftServerCmdLine doesn't hang. It runs for too long though... I did the more tests on my machine having 4 cores and 24G ram. ||host||container||maven||jdk||TestThriftServerCmdLine|| |ubuntu-17.04|ubuntu-trusty|3.0.5|1.7|hang| |ubuntu-17.04|ubuntu-trusty|3.3.9|1.7|hang| |ubuntu-17.04|ubuntu-trusty|3.0.5|1.8|62s| |ubuntu-17.04|ubuntu-trusty|3.3.9|1.8|63s| |ubuntu-17.04|ubuntu-17.04|3.3.9|1.7|63s| |ubuntu-17.04|n/a|3.3.9|1.7|63s| |ubuntu-17.04|n/a|3.3.9|1.8|63s| > branch-1.2 times out and is taking 6-7 hours to complete > > > Key: HBASE-19204 > URL: https://issues.apache.org/jira/browse/HBASE-19204 > Project: HBase > Issue Type: Umbrella > Components: test >Reporter: stack > > Sean has been looking at tooling and infra. This Umbrellas is about looking > at actual tests. For example, running locally on dedicated machine I picked a > random test, TestPerColumnFamilyFlush. In my test run, it wrote 16M lines. It > seems to be having zk issues but it is catching interrupts and ignoring them > ([~carp84] fixed this in later versions over in HBASE-18441). > Let me try and do some fixup under this umbrella so we can get a 1.2.7 out > the door. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16574) Add backup / restore feature to refguide
[
https://issues.apache.org/jira/browse/HBASE-16574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248716#comment-16248716
]
stack commented on HBASE-16574:
---
Did these get addressed?
{code}
Nit: On usage, the backupid exists nowhere in the system except as output from
the backup command? Later it is explained what it is which is helpful. Also,
later again it is explained that it does live as part of the backup history.
Could be good to call out these facts earlier.
Could also say how long a backup is going to take roughly. Me as an operator
would be afraid to run a backup because I'd think the command could run for
ever on my 100 node cluster. Reading later in the manual, it seems that it
could start and return immediately and then I check in on status. Would be good
to surface some of that up here at start of doc. to allay the fears of the poor
operator.
{code}
> Add backup / restore feature to refguide
>
>
> Key: HBASE-16574
> URL: https://issues.apache.org/jira/browse/HBASE-16574
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Frank Welsch
> Labels: backup
> Fix For: 2.0.0-beta-1
>
> Attachments: B command-line tools and configuration (updated).pdf,
> Backup-and-Restore-Apache_19Sep2016.pdf, HBASE-16574.001.patch,
> HBASE-16574.002.patch, HBASE-16574.003.branch-2.patch,
> hbase-book-16574.003.pdf, hbase_reference_guide.v1.pdf
>
>
> This issue is to add backup / restore feature description to hbase refguide.
> The description should cover:
> scenarios where backup / restore is used
> backup / restore commands and sample usage
> considerations in setup
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-14141) HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backed up tables
[
https://issues.apache.org/jira/browse/HBASE-14141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248714#comment-16248714
]
stack commented on HBASE-14141:
---
This should have a release note.
Is this facility on always? The walToHfile? (If I just wanted to copy WALs
though it overcopying, it is no longer allowed?)
This looks generally useful... walToHFiles. Break it out?
What happens if convertion to hfiles fails midway? I don't see cleanup (perhaps
it is there -- in failBackup, but we don't seem to pass the tmp dir name I
see that incrementalCopyHFiles does cleanup... but don't see it in convertion
of WAL to hfile).
This commit has no test. How we know if it works or if it is broke subsequently?
bq.* Get list of WAL files eligible for incremental backup
What makes a WAL eliible for backup?
getLogFilesFromBackupSystem gets log files from backup table. Will this be a
large set. Does Will the set be large? Will it grow w/o bound?
Why do operators sometimes have spaces around them and then in same line, they
do not: "180 for (int i=0; i < logFromSystemTable.size(); i++) {"
> HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits
> from backed up tables
> ---
>
> Key: HBASE-14141
> URL: https://issues.apache.org/jira/browse/HBASE-14141
> Project: HBase
> Issue Type: New Feature
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Blocker
> Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-14141.HBASE-14123.v1.patch, HBASE-14141.v1.patch,
> HBASE-14141.v2.patch, HBASE-14141.v4.patch, HBASE-14141.v5.patch,
> HBASE-14141.v6.patch
>
>
> h2. High level design overview
> * When incremental backup request comes for tables {t} we select all the
> tables already registered in a backup system - {T} and union them with {t},
> which results in a new table set - U(t, T)
> * For every table K from U(t,T) we perform the following:
> ** Convert new WAL files into HFile applying table filter K (only edits for
> table T will pass the filter)
> ** Move these HFile(s) to backup destination
> During restore (incremental):
> * We run full restore first
> * Then collect all HFiles from intermediate incremental images and run them
> through HFileSplitterJob, which splits files into a current tables region
> boundaries
> * Load these files using LoadIncrementalHFiles tool
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19210) TestNamespacesInstanceResource fails
[
https://issues.apache.org/jira/browse/HBASE-19210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248707#comment-16248707
]
Hadoop QA commented on HBASE-19210:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 14m
58s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
15s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
57m 53s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}184m
3s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}275m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19210 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12897208/19210.v3.txt |
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux 174b65bcde0d 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / bff619ef7b |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9747/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9747/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was
[jira] [Commented] (HBASE-18601) Update Htrace to 4.2
[ https://issues.apache.org/jira/browse/HBASE-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248670#comment-16248670 ] Hudson commented on HBASE-18601: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4038 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4038/]) HBASE-18601: Update Htrace to 4.2 (stack: rev 377174d3ef95e020573a824e71c530b8391e67c3) * (edit) hbase-backup/pom.xml * (edit) hbase-endpoint/pom.xml * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java * (edit) src/main/asciidoc/_chapters/tracing.adoc * (edit) hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java * (edit) hbase-thrift/pom.xml * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * (edit) hbase-client/pom.xml * (edit) hbase-hadoop2-compat/pom.xml * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyServerCall.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/ServerCall.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/wal/WALPerformanceEvaluation.java * (edit) hbase-mapreduce/pom.xml * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * (edit) hbase-testing-util/pom.xml * (edit) hbase-external-blockcache/src/main/java/org/apache/hadoop/hbase/io/hfile/MemcachedBlockCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyServerRpcConnection.java * (edit) hbase-server/pom.xml * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcConnection.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRequestFutureImpl.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SyncFuture.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/trace/HBaseHTraceConfiguration.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/Call.java * (edit) hbase-external-blockcache/pom.xml * (edit) hbase-examples/pom.xml * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleServerCall.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServerCommandLine.java * (edit) hbase-shell/pom.xml * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java * (edit) hbase-common/pom.xml * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/trace/SpanReceiverHost.java * (edit) hbase-it/pom.xml * (edit) hbase-protocol-shaded/pom.xml * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcCall.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java * (edit) hbase-it/src/test/java/org/apache/hadoop/hbase/mttr/IntegrationTestMTTR.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleServerRpcConnection.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/ServerRpcConnection.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/trace/TraceTree.java * (edit) hbase-rest/pom.xml * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (edit) hbase-shell/src/main/ruby/shell/commands/trace.rb * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/trace/TestHTraceHooks.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java * (edit) hbase-it/src/test/java/org/apache/hadoop/hbase/trace/IntegrationTestSendTraceRequests.java * (add) hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * (edit) hbase-spark/pom.xml * (edit) hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java * (edit) hbase-procedure/pom.xml * (edit) hbase-replication/pom.xml * (edit) hbase-rsgroup/pom.xml * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/IPCUtil.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/ResultBoundedCompletionService.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSWALEntry.java * (edit)
[jira] [Commented] (HBASE-19240) Fix error-prone errors, part four?
[ https://issues.apache.org/jira/browse/HBASE-19240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248671#comment-16248671 ] Hudson commented on HBASE-19240: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4038 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4038/]) HBASE-19240 more error-prone results (stack: rev c87189d41888b504489492181dee486886414513) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeepDeletes.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationEndpoint.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFileRefresherChore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestWALProcedureStoreOnHDFS.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestServerNonceManager.java * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestVersionResource.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestTablePermissions.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStoreChunkPool.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverForAddingMutationsFromCoprocessors.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/MultiThreadedUpdaterWithACL.java * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/model/VersionModel.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java * (edit) hbase-it/src/test/java/org/apache/hadoop/hbase/ipc/IntegrationTestRpcClient.java * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/model/TestVersionModel.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestMasterProcedureSchedulerConcurrency.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestCachedMobFile.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterOperationsForRegionReplicas.java * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/HBaseRESTTestingUtility.java * (edit) hbase-endpoint/src/test/java/org/apache/hadoop/hbase/client/TestRpcControllerFactory.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRegionPlacement.java > Fix error-prone errors, part four? > -- > > Key: HBASE-19240 > URL: https://issues.apache.org/jira/browse/HBASE-19240 > Project: HBase > Issue Type: Bug >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 3.0.0 > > Attachments: HBASE-19240.patch, HBASE-19240.v2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19225) Date Tiered Compaction design doc should be on a server the project controls
[ https://issues.apache.org/jira/browse/HBASE-19225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248653#comment-16248653 ] stack commented on HBASE-19225: --- None [~Apache9] Should I create a dir under src/site/resources and make sure its show up on our site? I could check in the doc there. > Date Tiered Compaction design doc should be on a server the project controls > > > Key: HBASE-19225 > URL: https://issues.apache.org/jira/browse/HBASE-19225 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.5.0 >Reporter: Sean Busbey >Priority: Critical > Attachments: Date Tiered Compaction.pdf > > > Right now the docs on date tiered compaction refer folks to its design doc > hosted on google docs. > we should host this somewhere we control, if only as a PDF on the website, > and link there instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19240) Fix error-prone errors, part four?
[ https://issues.apache.org/jira/browse/HBASE-19240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19240: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master branch. > Fix error-prone errors, part four? > -- > > Key: HBASE-19240 > URL: https://issues.apache.org/jira/browse/HBASE-19240 > Project: HBase > Issue Type: Bug >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 3.0.0 > > Attachments: HBASE-19240.patch, HBASE-19240.v2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19240) Fix error-prone errors, part four?
[ https://issues.apache.org/jira/browse/HBASE-19240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248641#comment-16248641 ] stack commented on HBASE-19240: --- +1 > Fix error-prone errors, part four? > -- > > Key: HBASE-19240 > URL: https://issues.apache.org/jira/browse/HBASE-19240 > Project: HBase > Issue Type: Bug >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-19240.patch, HBASE-19240.v2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19240) Fix error-prone errors, part four?
[
https://issues.apache.org/jira/browse/HBASE-19240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248636#comment-16248636
]
Hadoop QA commented on HBASE-19240:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 26 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
12s{color} | {color:red} hbase-server: The patch generated 1 new + 621
unchanged - 1 fixed = 622 total (was 622) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} hbase-endpoint: The patch generated 0 new + 0
unchanged - 1 fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} hbase-it: The patch generated 0 new + 1 unchanged -
16 fixed = 1 total (was 17) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} hbase-rest: The patch generated 8 new + 18 unchanged -
1 fixed = 26 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
49m 30s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}109m
28s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
15s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
26s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
4s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black}
[jira] [Commented] (HBASE-19241) Javadoc improvement for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248633#comment-16248633 ] stack commented on HBASE-19241: --- [~appy] This good by you sir? > Javadoc improvement for AsyncAdmin > -- > > Key: HBASE-19241 > URL: https://issues.apache.org/jira/browse/HBASE-19241 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19241.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19235) CoprocessorEnvironment should be exposed to CPs
[ https://issues.apache.org/jira/browse/HBASE-19235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248630#comment-16248630 ] stack commented on HBASE-19235: --- Options: # The [~appy] interpretation of IA.Private on superclass means superclass methods are not for CPs (I think this too much attention-to-detail to expect of users...) # Hiding the problematic as per Anoop's suggestion that we restore the old hackery? # Just mark CP as LP and if a CP calls shutdown, they are shooting themselves in the foot (Annotate IA.Private?) I like option #2. > CoprocessorEnvironment should be exposed to CPs > --- > > Key: HBASE-19235 > URL: https://issues.apache.org/jira/browse/HBASE-19235 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Affects Versions: 2.0.0-alpha-4 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Minor > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19235.patch > > > Its sub interfaces are exposed with > LimitedPrivate(HBaseInterfaceAudience.COPROC). So ideally all the functions > in this are. Better we mark CoprocessorEnvironment also as CP exposed to > avoid confusion. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19146) Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
[
https://issues.apache.org/jira/browse/HBASE-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248627#comment-16248627
]
stack commented on HBASE-19146:
---
[~yqGu]
bq. Could you please upgrade protoc version to 3.5.0 that is used for
hbase-protocol-shaded when google release it?
Yes. Hopefully it is sooner rather than later.
bq. How do you like to upgrade Protoc-2.5.0 to 2.5.1?
Perhaps. If it happens soon. Would have to see what changed in 2.5.0. If it
only arm support then that should be fine.
We continue to support 2.5.0 so that Coprocessor Endpoints continue to work
when an hbase1.x client is going against an hbase2 cluster.
> Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
> --
>
> Key: HBASE-19146
> URL: https://issues.apache.org/jira/browse/HBASE-19146
> Project: HBase
> Issue Type: Bug
> Components: build, pom
>Affects Versions: 3.0.0
> Environment: OS: Ubuntu 16.04.3
> OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
> Hw platform: AARCH64
>Reporter: Yuqi Gu
>
> We are building the HBase 3.0.0-SNAPSHOT on AARCH64.
> It is noted that 'protobuf-maven-plugin' only support x86 shown as follows:
> {code:java}
>
>org.xolstice.maven.plugins
>protobuf-maven-plugin
>${protobuf.plugin.version}
>
> com.google.protobuf:protoc:${external.protobuf.version}:
> exe:${os.detected.classifier}
>
> com.google.protobuf:protoc:${external.protobuf.version}:exe:${os.detected.classifier}
>
>false
>true
>
>
> {code}
> So the build is failed.
> {code:java}
> [INFO] --- protobuf-maven-plugin:0.5.0:compile (compile-protoc) @
> hbase-protocol-shaded ---
> [INFO] Compiling 32 proto file(s) to
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> Failed to execute goal
> org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile
> (compile-protoc) on project hbase-protocol-shaded: Missing:
> {code}
> Then I installed aarch64 protobuf 2.5.0 on the host and modify the pom:
> {code:java}
> - ${basedir}/src/main/protobuf/
> + /usr/local/bin/protoc
> {code}
> The build is also failed:
> {code:java}
> [INFO] Compiling 32 proto file(s) to
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> [ERROR] PROTOC FAILED: google/protobuf/any.proto:31:10: Unrecognized syntax
> identifier "proto3". This parser only recognizes "proto2".
> {code}
> It seems that "internal.protobuf.version" in "hbase-protocol-shaded" is 3.3.0.
> How to fix it? Thanks!
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19146) Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
[
https://issues.apache.org/jira/browse/HBASE-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248627#comment-16248627
]
stack edited comment on HBASE-19146 at 11/11/17 7:12 PM:
-
[~yqGu]
bq. Could you please upgrade protoc version to 3.5.0 that is used for
hbase-protocol-shaded when google release it?
Yes. Hopefully it is sooner rather than later.
bq. How do you like to upgrade Protoc-2.5.0 to 2.5.1?
Perhaps. If it happens soon. Would have to see what changed in 2.5.0. If it
only arm support then that should be fine.
We continue to support 2.5.0 so that Coprocessor Endpoints continue to work
when an hbase1.x client is going against an hbase2 cluster (Coprocessor
Endpoints use Protobuf Service to describe the methods they implement).
was (Author: stack):
[~yqGu]
bq. Could you please upgrade protoc version to 3.5.0 that is used for
hbase-protocol-shaded when google release it?
Yes. Hopefully it is sooner rather than later.
bq. How do you like to upgrade Protoc-2.5.0 to 2.5.1?
Perhaps. If it happens soon. Would have to see what changed in 2.5.0. If it
only arm support then that should be fine.
We continue to support 2.5.0 so that Coprocessor Endpoints continue to work
when an hbase1.x client is going against an hbase2 cluster.
> Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
> --
>
> Key: HBASE-19146
> URL: https://issues.apache.org/jira/browse/HBASE-19146
> Project: HBase
> Issue Type: Bug
> Components: build, pom
>Affects Versions: 3.0.0
> Environment: OS: Ubuntu 16.04.3
> OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
> Hw platform: AARCH64
>Reporter: Yuqi Gu
>
> We are building the HBase 3.0.0-SNAPSHOT on AARCH64.
> It is noted that 'protobuf-maven-plugin' only support x86 shown as follows:
> {code:java}
>
>org.xolstice.maven.plugins
>protobuf-maven-plugin
>${protobuf.plugin.version}
>
> com.google.protobuf:protoc:${external.protobuf.version}:
> exe:${os.detected.classifier}
>
> com.google.protobuf:protoc:${external.protobuf.version}:exe:${os.detected.classifier}
>
>false
>true
>
>
> {code}
> So the build is failed.
> {code:java}
> [INFO] --- protobuf-maven-plugin:0.5.0:compile (compile-protoc) @
> hbase-protocol-shaded ---
> [INFO] Compiling 32 proto file(s) to
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> Failed to execute goal
> org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile
> (compile-protoc) on project hbase-protocol-shaded: Missing:
> {code}
> Then I installed aarch64 protobuf 2.5.0 on the host and modify the pom:
> {code:java}
> - ${basedir}/src/main/protobuf/
> + /usr/local/bin/protoc
> {code}
> The build is also failed:
> {code:java}
> [INFO] Compiling 32 proto file(s) to
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> [ERROR] PROTOC FAILED: google/protobuf/any.proto:31:10: Unrecognized syntax
> identifier "proto3". This parser only recognizes "proto2".
> {code}
> It seems that "internal.protobuf.version" in "hbase-protocol-shaded" is 3.3.0.
> How to fix it? Thanks!
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19204) branch-1.2 times out and is taking 6-7 hours to complete
[ https://issues.apache.org/jira/browse/HBASE-19204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248625#comment-16248625 ] stack commented on HBASE-19204: --- [~chia7712] Unrelated, I can't remember where, but in an issue you raised the notion of a 'RawCell'. I keep meaning to say that I like this notion. We currently have 'EnhancedCell'. I think RawCell a much better name ([~anoop.hbase] & [~ram_krish] -- what you think lads?). TestThriftServerCmdLine doesn't hang. It runs for too long though... 435 [INFO] Running org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine 3436 [INFO] Tests run: 32, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 791.874 s - in org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine On tests that hang for me, the number is too many to paste in here (smile). Running a few of the tests independent of the others, they hang... So, I need to dig in. > branch-1.2 times out and is taking 6-7 hours to complete > > > Key: HBASE-19204 > URL: https://issues.apache.org/jira/browse/HBASE-19204 > Project: HBase > Issue Type: Umbrella > Components: test >Reporter: stack > > Sean has been looking at tooling and infra. This Umbrellas is about looking > at actual tests. For example, running locally on dedicated machine I picked a > random test, TestPerColumnFamilyFlush. In my test run, it wrote 16M lines. It > seems to be having zk issues but it is catching interrupts and ignoring them > ([~carp84] fixed this in later versions over in HBASE-18441). > Let me try and do some fixup under this umbrella so we can get a 1.2.7 out > the door. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18601) Update Htrace to 4.2
[ https://issues.apache.org/jira/browse/HBASE-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18601: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-2 and master. Thanks Tamas and [~balazs.meszaros] ... I added you both as sign-offs > Update Htrace to 4.2 > > > Key: HBASE-18601 > URL: https://issues.apache.org/jira/browse/HBASE-18601 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0, 3.0.0 >Reporter: Tamas Penzes >Assignee: Balazs Meszaros > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18601.master.001.patch, > HBASE-18601.master.002.patch, HBASE-18601.master.003 (3).patch, > HBASE-18601.master.003.patch, HBASE-18601.master.004.patch, > HBASE-18601.master.004.patch, HBASE-18601.master.005.patch, > HBASE-18601.master.006.patch, HBASE-18601.master.006.patch, > HBASE-18601.master.007.patch, HBASE-18601.master.007.patch, > HBASE-18601.master.007.patch, HBASE-18601.master.008.patch, > HBASE-18601.master.009.patch, HBASE-18601.master.009.patch, > HBASE-18601.master.010.patch, HBASE-18601.master.010.patch, > HBASE-18601.master.011.patch, HBASE-18601.master.012.patch, > HBASE-18601.master.013.patch, HBASE-18601.master.014.patch, > HBASE-18601.master.014.patch, HBASE-18601.master.015.patch > > > HTrace is not perfectly integrated into HBase, the version 3.2.0 is buggy, > the upgrade to 4.x is not trivial and would take time. It might not worth to > keep it in this state, so would be better to remove it. > Of course it doesn't mean tracing would be useless, just that in this form > the use of HTrace 3.2 might not add any value to the project and fixing it > would be far too much effort. > - > Based on the decision of the community we keep htrace now and update version -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19200) make hbase-client only depend on ZKAsyncRegistry and ZNodePaths
[ https://issues.apache.org/jira/browse/HBASE-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248612#comment-16248612 ] stack commented on HBASE-19200: --- This is great. I love that we move ahead on the old Duo plan of untangling client and zk. I am a little unclear though on what exactly was delivered. hbase-client will still depend on the new hbase-zookeeper module because of ZKLeaderManager, et al? Can we have release note that states what we get here? Thanks lads. > make hbase-client only depend on ZKAsyncRegistry and ZNodePaths > --- > > Key: HBASE-19200 > URL: https://issues.apache.org/jira/browse/HBASE-19200 > Project: HBase > Issue Type: Task > Components: Client, Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19200-v1.patch, HBASE-19200-v2.patch, > HBASE-19200-v3.patch, HBASE-19200-v4.patch, HBASE-19200-v5.patch, > HBASE-19200.patch > > > So that we can move most of the zookeeper related code out of hbase-client > module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19241) Javadoc improvement for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248611#comment-16248611 ] stack commented on HBASE-19241: --- +1 Fix javadoc complaint on commit. > Javadoc improvement for AsyncAdmin > -- > > Key: HBASE-19241 > URL: https://issues.apache.org/jira/browse/HBASE-19241 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19241.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19210) TestNamespacesInstanceResource fails
[
https://issues.apache.org/jira/browse/HBASE-19210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19210:
---
Attachment: 19210.v3.txt
Patch v3 upgrades jackson for all components except for hbase-spark module.
> TestNamespacesInstanceResource fails
>
>
> Key: HBASE-19210
> URL: https://issues.apache.org/jira/browse/HBASE-19210
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 19210.v1.txt, 19210.v2.txt, 19210.v3.txt,
> HBASE-19210.WIP.patch
>
>
> This is the top flaky test.
> The following can be reproduced:
> {code}
> java.net.SocketException: Connection reset
> at
> org.apache.hadoop.hbase.rest.TestNamespacesInstanceResource.testInvalidNamespacePostsAndPuts(TestNamespacesInstanceResource.java:271)
> {code}
> With commit e320df5a0c267258c03909da8d0eee4c0e287532, the test passes.
> With commit 5facaded902a13556952b1f9d26b768cb86e6599, the test fails.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19210) TestNamespacesInstanceResource fails
[
https://issues.apache.org/jira/browse/HBASE-19210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-19210:
---
Status: Patch Available (was: Open)
> TestNamespacesInstanceResource fails
>
>
> Key: HBASE-19210
> URL: https://issues.apache.org/jira/browse/HBASE-19210
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 19210.v1.txt, 19210.v2.txt, 19210.v3.txt,
> HBASE-19210.WIP.patch
>
>
> This is the top flaky test.
> The following can be reproduced:
> {code}
> java.net.SocketException: Connection reset
> at
> org.apache.hadoop.hbase.rest.TestNamespacesInstanceResource.testInvalidNamespacePostsAndPuts(TestNamespacesInstanceResource.java:271)
> {code}
> With commit e320df5a0c267258c03909da8d0eee4c0e287532, the test passes.
> With commit 5facaded902a13556952b1f9d26b768cb86e6599, the test fails.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19240) Fix error-prone errors, part four?
[ https://issues.apache.org/jira/browse/HBASE-19240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-19240: -- Attachment: HBASE-19240.v2.patch v2: fix unit failing unit tests and checkstyle errors > Fix error-prone errors, part four? > -- > > Key: HBASE-19240 > URL: https://issues.apache.org/jira/browse/HBASE-19240 > Project: HBase > Issue Type: Bug >Reporter: Mike Drob >Assignee: Mike Drob > Attachments: HBASE-19240.patch, HBASE-19240.v2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19241) Javadoc improvement for AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248542#comment-16248542
]
Hadoop QA commented on HBASE-19241:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
27s{color} | {color:red} hbase-client: The patch generated 2 new + 135
unchanged - 2 fixed = 137 total (was 137) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
49m 25s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
19s{color} | {color:red} hbase-client generated 4 new + 2 unchanged - 0 fixed =
6 total (was 2) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 75m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19241 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12897196/HBASE-19241.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d78760343789 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / bff619ef7b |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/9745/artifact/patchprocess/diff-checkstyle-hbase-client.txt
|
| javadoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/9745/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
|
| Test Results |
[jira] [Created] (HBASE-19242) Add MOB compact support for AsyncAdmin
Duo Zhang created HBASE-19242:
-
Summary: Add MOB compact support for AsyncAdmin
Key: HBASE-19242
URL: https://issues.apache.org/jira/browse/HBASE-19242
Project: HBase
Issue Type: Task
Components: Admin, mob
Reporter: Duo Zhang
Priority: Blocker
Fix For: 2.0.0-beta-1
{code}
private CompletableFuture compact(TableName tableName, byte[]
columnFamily, boolean major,
CompactType compactType) {
if (CompactType.MOB.equals(compactType)) {
// TODO support MOB compact.
return failedFuture(new UnsupportedOperationException("MOB compact does
not support"));
}
{code}
We need to support it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19241) Javadoc improvement for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248518#comment-16248518 ] Duo Zhang commented on HBASE-19241: --- [~zghaobac] [~appy] FYI. > Javadoc improvement for AsyncAdmin > -- > > Key: HBASE-19241 > URL: https://issues.apache.org/jira/browse/HBASE-19241 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19241.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19241) Javadoc improvement for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19241: -- Attachment: HBASE-19241.patch Add more javadoc for AsyncAdmin. Also cleanup the warnings of RawAsyncHBaseAdmin. > Javadoc improvement for AsyncAdmin > -- > > Key: HBASE-19241 > URL: https://issues.apache.org/jira/browse/HBASE-19241 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19241.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19241) Javadoc improvement for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19241: -- Status: Patch Available (was: Open) > Javadoc improvement for AsyncAdmin > -- > > Key: HBASE-19241 > URL: https://issues.apache.org/jira/browse/HBASE-19241 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19241.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19187) Remove option to create on heap bucket cache
[ https://issues.apache.org/jira/browse/HBASE-19187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248515#comment-16248515 ] Hudson commented on HBASE-19187: FAILURE: Integrated in Jenkins build HBase-2.0 #835 (See [https://builds.apache.org/job/HBase-2.0/835/]) HBASE-19187 Remove option to create on heap bucket cache. (anoopsamjohn: rev 3ee76f8573387097c896d0611c30504ed7e57cb0) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHeapMemoryManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/ByteBufferIOEngine.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketWriterThread.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HeapMemoryManager.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerFromBucketCache.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAvoidCellReferencesIntoShippedBlocks.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestByteBufferIOEngine.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/util/MemorySizeUtil.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/ByteBufferAllocator.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/ByteBufferArray.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheConfig.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestByteBufferArray.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestBlockEvictionFromClient.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileMmapEngine.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java * (edit) hbase-common/src/main/resources/hbase-default.xml > Remove option to create on heap bucket cache > > > Key: HBASE-19187 > URL: https://issues.apache.org/jira/browse/HBASE-19187 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 2.0.0-alpha-3 >Reporter: Biju Nair >Assignee: Anoop Sam John >Priority: Minor > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19187.patch, HBASE-19187_V2.patch > > > Since we are moving read and write paths to use direct memory, option to > configure on heap bucket cache is irrelevant. We should remove that option. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19226) Limit the reduce tasks number of incremental load
[ https://issues.apache.org/jira/browse/HBASE-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248509#comment-16248509 ] Ted Yu commented on HBASE-19226: Can you fill out release note ? > Limit the reduce tasks number of incremental load > - > > Key: HBASE-19226 > URL: https://issues.apache.org/jira/browse/HBASE-19226 > Project: HBase > Issue Type: Improvement >Reporter: Yun Zhao >Assignee: Yun Zhao >Priority: Minor > Attachments: HBASE-19226.master.001.patch, > HBASE-19226.master.002.patch > > > When using MapReduce job to perform an incremental load into a table,the > number of reduce tasks is the current number of regions. If there are too > many regions, will lead to network+disk I/O is too large, affecting the > real-time request. > Need to use a configuration to set a number or ratio? > Limit running reduce tasks since > [https://issues.apache.org/jira/browse/MAPREDUCE-5583], the old version can > only be set reduce number. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19241) Javadoc improvement for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19241: -- Component/s: documentation > Javadoc improvement for AsyncAdmin > -- > > Key: HBASE-19241 > URL: https://issues.apache.org/jira/browse/HBASE-19241 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19241) Javadoc improvement for AsyncAdmin
Duo Zhang created HBASE-19241: - Summary: Javadoc improvement for AsyncAdmin Key: HBASE-19241 URL: https://issues.apache.org/jira/browse/HBASE-19241 Project: HBase Issue Type: Task Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 2.0.0-beta-1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19181) LogRollBackupSubprocedure will fail if we use AsyncFSWAL instead of FSHLog
[
https://issues.apache.org/jira/browse/HBASE-19181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248502#comment-16248502
]
Duo Zhang commented on HBASE-19181:
---
Use AbstractFSWAK to avoid the @SuppressWarnings("rawtypes").
> LogRollBackupSubprocedure will fail if we use AsyncFSWAL instead of FSHLog
> --
>
> Key: HBASE-19181
> URL: https://issues.apache.org/jira/browse/HBASE-19181
> Project: HBase
> Issue Type: Bug
> Components: backup
>Reporter: Duo Zhang
>Assignee: Vladimir Rodionov
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19181-v1.patch
>
>
> In the RSRollLogTask it will cast a WAL to FSHLog.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18978) Align the methods in Table and AsyncTable
[ https://issues.apache.org/jira/browse/HBASE-18978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248500#comment-16248500 ] Duo Zhang commented on HBASE-18978: --- I think the documentations of AsyncAdmin related code are stale. Let me open a issue to fix them. > Align the methods in Table and AsyncTable > - > > Key: HBASE-18978 > URL: https://issues.apache.org/jira/browse/HBASE-18978 > Project: HBase > Issue Type: Task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Peter Somogyi >Priority: Critical > Fix For: 2.0.0-beta-1 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19200) make hbase-client only depend on ZKAsyncRegistry and ZNodePaths
[ https://issues.apache.org/jira/browse/HBASE-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248499#comment-16248499 ] Duo Zhang commented on HBASE-19200: --- Thanks [~appy] for helping committing this. > make hbase-client only depend on ZKAsyncRegistry and ZNodePaths > --- > > Key: HBASE-19200 > URL: https://issues.apache.org/jira/browse/HBASE-19200 > Project: HBase > Issue Type: Task > Components: Client, Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19200-v1.patch, HBASE-19200-v2.patch, > HBASE-19200-v3.patch, HBASE-19200-v4.patch, HBASE-19200-v5.patch, > HBASE-19200.patch > > > So that we can move most of the zookeeper related code out of hbase-client > module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19226) Limit the reduce tasks number of incremental load
[
https://issues.apache.org/jira/browse/HBASE-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248461#comment-16248461
]
Hadoop QA commented on HBASE-19226:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
54s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
49m 49s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 15m
2s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 83m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19226 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12897181/HBASE-19226.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e234e4685b1e 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / bff619ef7b |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9744/testReport/ |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9744/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Limit the reduce tasks number of incremental load
> -
>
> Key: HBASE-19226
> URL:
[jira] [Commented] (HBASE-19187) Remove option to create on heap bucket cache
[ https://issues.apache.org/jira/browse/HBASE-19187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16248435#comment-16248435 ] Hudson commented on HBASE-19187: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4035 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4035/]) HBASE-19187 Remove option to create on heap bucket cache. (anoopsamjohn: rev bff619ef7b100e8b09f7f5eb0f6e289ca51de096) * (edit) hbase-common/src/main/resources/hbase-default.xml * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerFromBucketCache.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/ByteBufferArray.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/util/MemorySizeUtil.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheConfig.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestByteBufferIOEngine.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAvoidCellReferencesIntoShippedBlocks.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestBlockEvictionFromClient.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/ByteBufferIOEngine.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestByteBufferArray.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileMmapEngine.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketWriterThread.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHeapMemoryManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HeapMemoryManager.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/util/ByteBufferAllocator.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java > Remove option to create on heap bucket cache > > > Key: HBASE-19187 > URL: https://issues.apache.org/jira/browse/HBASE-19187 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 2.0.0-alpha-3 >Reporter: Biju Nair >Assignee: Anoop Sam John >Priority: Minor > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19187.patch, HBASE-19187_V2.patch > > > Since we are moving read and write paths to use direct memory, option to > configure on heap bucket cache is irrelevant. We should remove that option. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19226) Limit the reduce tasks number of incremental load
[ https://issues.apache.org/jira/browse/HBASE-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Zhao updated HBASE-19226: - Attachment: HBASE-19226.master.002.patch > Limit the reduce tasks number of incremental load > - > > Key: HBASE-19226 > URL: https://issues.apache.org/jira/browse/HBASE-19226 > Project: HBase > Issue Type: Improvement >Reporter: Yun Zhao >Assignee: Yun Zhao >Priority: Minor > Attachments: HBASE-19226.master.001.patch, > HBASE-19226.master.002.patch > > > When using MapReduce job to perform an incremental load into a table,the > number of reduce tasks is the current number of regions. If there are too > many regions, will lead to network+disk I/O is too large, affecting the > real-time request. > Need to use a configuration to set a number or ratio? > Limit running reduce tasks since > [https://issues.apache.org/jira/browse/MAPREDUCE-5583], the old version can > only be set reduce number. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
