[jira] [Updated] (HBASE-21829) Use FutureUtils.addListener instead of calling whenComplete directly
[ https://issues.apache.org/jira/browse/HBASE-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21829: -- Assignee: Duo Zhang Status: Patch Available (was: Open) > Use FutureUtils.addListener instead of calling whenComplete directly > > > Key: HBASE-21829 > URL: https://issues.apache.org/jira/browse/HBASE-21829 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0 > > Attachments: HBASE-21829.patch > > > This can eliminate the FutureReturnValueIgnored warnings, and also we can do > the CompletionFuture unwrap in the addListener method to avoid doing it > everywhere. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21829) Use FutureUtils.addListener instead of calling whenComplete directly
[ https://issues.apache.org/jira/browse/HBASE-21829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21829: -- Attachment: HBASE-21829.patch > Use FutureUtils.addListener instead of calling whenComplete directly > > > Key: HBASE-21829 > URL: https://issues.apache.org/jira/browse/HBASE-21829 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21829.patch > > > This can eliminate the FutureReturnValueIgnored warnings, and also we can do > the CompletionFuture unwrap in the addListener method to avoid doing it > everywhere. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21827) CompletableFuture may wrap the original Throwable with CompletionException while propagating it
[ https://issues.apache.org/jira/browse/HBASE-21827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21827: -- Issue Type: Umbrella (was: Bug) > CompletableFuture may wrap the original Throwable with CompletionException > while propagating it > --- > > Key: HBASE-21827 > URL: https://issues.apache.org/jira/browse/HBASE-21827 > Project: HBase > Issue Type: Umbrella > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > > See the stack overflow post > https://stackoverflow.com/questions/49230980/does-completionstage-always-wrap-exceptions-in-completionexception > It seems that, for a chain of CompletableFutures, the first child will > receive the original exception, and the latter ones will receive a > CompletionException, which is the wrapper of the original exception. > This is also what I observed when debugging HBASE-21779. We need to fix this, > at least, we should unwrap it when we want to return exceptions to users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21828) Make sure we do not return CompletionException when locating region
[ https://issues.apache.org/jira/browse/HBASE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21828: -- Status: Patch Available (was: Open) > Make sure we do not return CompletionException when locating region > --- > > Key: HBASE-21828 > URL: https://issues.apache.org/jira/browse/HBASE-21828 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0 > > Attachments: HBASE-21828.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21828) Make sure we do not return CompletionException when locating region
[ https://issues.apache.org/jira/browse/HBASE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21828: -- Attachment: HBASE-21828.patch > Make sure we do not return CompletionException when locating region > --- > > Key: HBASE-21828 > URL: https://issues.apache.org/jira/browse/HBASE-21828 > Project: HBase > Issue Type: Sub-task > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0 > > Attachments: HBASE-21828.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21829) Use FutureUtils.addListener instead of calling whenComplete directly
Duo Zhang created HBASE-21829: - Summary: Use FutureUtils.addListener instead of calling whenComplete directly Key: HBASE-21829 URL: https://issues.apache.org/jira/browse/HBASE-21829 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang This can eliminate the FutureReturnValueIgnored warnings, and also we can do the CompletionFuture unwrap in the addListener method to avoid doing it everywhere. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21828) Make sure we do not return CompletionException when locating region
Duo Zhang created HBASE-21828: - Summary: Make sure we do not return CompletionException when locating region Key: HBASE-21828 URL: https://issues.apache.org/jira/browse/HBASE-21828 Project: HBase Issue Type: Sub-task Components: asyncclient, Client Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5, 2.3.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21811) region can be opened on two servers due to race condition with procedures and server reports
[
https://issues.apache.org/jira/browse/HBASE-21811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21811:
--
Attachment: HBASE-21811-v2.patch
> region can be opened on two servers due to race condition with procedures and
> server reports
>
>
> Key: HBASE-21811
> URL: https://issues.apache.org/jira/browse/HBASE-21811
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 3.0.0
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Blocker
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21811-UT.patch, HBASE-21811-v1.patch,
> HBASE-21811-v2.patch, HBASE-21811.patch
>
>
> Looks like the region server responses are being processed incorrectly in
> places allowing te region to be opened on two servers.
> * The region server report handling in procedures should check which server
> is reporting.
> * Also although I didn't check (and it isn't implicated in this bug), RS must
> check in OPEN that it's actually the correct RS master sent open to (w.r.t.
> start timestamp)
> This was previosuly "mitigated" by master killing the RS with incorrect
> reports, but due to race conditions with reports and assignment the kill was
> replaced with a warning, so now this condition persists.
> Regardless, the kill approach is not a good fix because there's still a
> window when a region can be opened on two servers.
> A region is being opened by server_48c. The server dies, and we process the
> retry correctly (retry=3 because 2 previous similar open failures were
> processed correctly).
> We start opening it on server_1aa now.
> {noformat}
> 2019-01-28 18:12:09,862 INFO [KeepAlivePEWorker-104]
> assignment.RegionStateStore: pid=4915 updating hbase:meta
> row=8be2a423b16471b9417f0f7de04281c6, regionState=ABNORMALLY_CLOSED
> 2019-01-28 18:12:09,862 INFO [KeepAlivePEWorker-104]
> procedure.ServerCrashProcedure: pid=11944,
> state=RUNNABLE:SERVER_CRASH_ASSIGN, hasLock=true; ServerCrashProcedure
> server=server_48c,17020,1548726406632, splitWal=true, meta=false found RIT
> pid=4915, ppid=7, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED,
> hasLock=true; TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=OPENING,
> location=server_48c,17020,1548726406632, table=table,
> region=8be2a423b16471b9417f0f7de04281c6
> 2019-01-28 18:12:10,778 INFO [KeepAlivePEWorker-80]
> assignment.TransitRegionStateProcedure: Retry=3 of max=2147483647; pid=4915,
> ppid=7, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_OPENED, hasLock=true;
> TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=ABNORMALLY_CLOSED,
> location=null
> ...
> 2019-01-28 18:12:10,902 INFO [KeepAlivePEWorker-80]
> assignment.TransitRegionStateProcedure: Starting pid=4915, ppid=7,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true;
> TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=ABNORMALLY_CLOSED,
> location=null; forceNewPlan=true, retain=false
> 2019-01-28 18:12:11,114 INFO [PEWorker-7] assignment.RegionStateStore:
> pid=4915 updating hbase:meta row=8be2a423b16471b9417f0f7de04281c6,
> regionState=OPENING, regionLocation=server_1aa,17020,1548727658713
> {noformat}
> However, we get the remote procedure failure from 48c after we've already
> started that.
> It actually tried to open on the restarted RS, which makes me wonder if this
> is safe also w.r.t. other races - what if RS already initialized and didn't
> error out?
> Need to check if we verify the start code expected by master on RS when
> opening.
> {noformat}
> 2019-01-28 18:12:12,179 WARN [RSProcedureDispatcher-pool4-t362]
> assignment.RegionRemoteProcedureBase: The remote operation pid=11050,
> ppid=4915, state=SUCCESS, hasLock=false;
> org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure for region
> {ENCODED => 8be2a423b16471b9417f0f7de04281c6 ... to server
> server_48c,17020,1548726406632 failed
> org.apache.hadoop.hbase.ipc.ServerNotRunningYetException:
> org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server
> server_48c,17020,1548727752747 is not running yet
> 2019-01-28 18:12:12,179 WARN [RSProcedureDispatcher-pool4-t362]
> procedure.RSProcedureDispatcher: server server_48c,17020,1548726406632 is not
> up for a while; try a new one
> {noformat}
> Without any other reason (at least logged), the RIT immediately retries again
> and chooses a new candidate. It then retries again and goes to the new 48c,
> but that's unrelated.
> {noformat}
> 2019-01-28 18:12:12,289 INFO [KeepAlivePEWorker-100]
> assignment.TransitRegionStateProcedure: Retry=4 of
[jira] [Updated] (HBASE-21827) CompletableFuture may wrap the original Throwable with CompletionException while propagating it
[ https://issues.apache.org/jira/browse/HBASE-21827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21827: -- Fix Version/s: (was: 2.3.0) (was: 2.2.0) (was: 3.0.0) > CompletableFuture may wrap the original Throwable with CompletionException > while propagating it > --- > > Key: HBASE-21827 > URL: https://issues.apache.org/jira/browse/HBASE-21827 > Project: HBase > Issue Type: Bug > Components: asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > > See the stack overflow post > https://stackoverflow.com/questions/49230980/does-completionstage-always-wrap-exceptions-in-completionexception > It seems that, for a chain of CompletableFutures, the first child will > receive the original exception, and the latter ones will receive a > CompletionException, which is the wrapper of the original exception. > This is also what I observed when debugging HBASE-21779. We need to fix this, > at least, we should unwrap it when we want to return exceptions to users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21811) region can be opened on two servers due to race condition with procedures and server reports
[
https://issues.apache.org/jira/browse/HBASE-21811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758898#comment-16758898
]
Duo Zhang commented on HBASE-21811:
---
Forgot to add the ClassRule annotation...
> region can be opened on two servers due to race condition with procedures and
> server reports
>
>
> Key: HBASE-21811
> URL: https://issues.apache.org/jira/browse/HBASE-21811
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 3.0.0
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Blocker
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21811-UT.patch, HBASE-21811-v1.patch,
> HBASE-21811-v2.patch, HBASE-21811.patch
>
>
> Looks like the region server responses are being processed incorrectly in
> places allowing te region to be opened on two servers.
> * The region server report handling in procedures should check which server
> is reporting.
> * Also although I didn't check (and it isn't implicated in this bug), RS must
> check in OPEN that it's actually the correct RS master sent open to (w.r.t.
> start timestamp)
> This was previosuly "mitigated" by master killing the RS with incorrect
> reports, but due to race conditions with reports and assignment the kill was
> replaced with a warning, so now this condition persists.
> Regardless, the kill approach is not a good fix because there's still a
> window when a region can be opened on two servers.
> A region is being opened by server_48c. The server dies, and we process the
> retry correctly (retry=3 because 2 previous similar open failures were
> processed correctly).
> We start opening it on server_1aa now.
> {noformat}
> 2019-01-28 18:12:09,862 INFO [KeepAlivePEWorker-104]
> assignment.RegionStateStore: pid=4915 updating hbase:meta
> row=8be2a423b16471b9417f0f7de04281c6, regionState=ABNORMALLY_CLOSED
> 2019-01-28 18:12:09,862 INFO [KeepAlivePEWorker-104]
> procedure.ServerCrashProcedure: pid=11944,
> state=RUNNABLE:SERVER_CRASH_ASSIGN, hasLock=true; ServerCrashProcedure
> server=server_48c,17020,1548726406632, splitWal=true, meta=false found RIT
> pid=4915, ppid=7, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED,
> hasLock=true; TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=OPENING,
> location=server_48c,17020,1548726406632, table=table,
> region=8be2a423b16471b9417f0f7de04281c6
> 2019-01-28 18:12:10,778 INFO [KeepAlivePEWorker-80]
> assignment.TransitRegionStateProcedure: Retry=3 of max=2147483647; pid=4915,
> ppid=7, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_OPENED, hasLock=true;
> TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=ABNORMALLY_CLOSED,
> location=null
> ...
> 2019-01-28 18:12:10,902 INFO [KeepAlivePEWorker-80]
> assignment.TransitRegionStateProcedure: Starting pid=4915, ppid=7,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true;
> TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=ABNORMALLY_CLOSED,
> location=null; forceNewPlan=true, retain=false
> 2019-01-28 18:12:11,114 INFO [PEWorker-7] assignment.RegionStateStore:
> pid=4915 updating hbase:meta row=8be2a423b16471b9417f0f7de04281c6,
> regionState=OPENING, regionLocation=server_1aa,17020,1548727658713
> {noformat}
> However, we get the remote procedure failure from 48c after we've already
> started that.
> It actually tried to open on the restarted RS, which makes me wonder if this
> is safe also w.r.t. other races - what if RS already initialized and didn't
> error out?
> Need to check if we verify the start code expected by master on RS when
> opening.
> {noformat}
> 2019-01-28 18:12:12,179 WARN [RSProcedureDispatcher-pool4-t362]
> assignment.RegionRemoteProcedureBase: The remote operation pid=11050,
> ppid=4915, state=SUCCESS, hasLock=false;
> org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure for region
> {ENCODED => 8be2a423b16471b9417f0f7de04281c6 ... to server
> server_48c,17020,1548726406632 failed
> org.apache.hadoop.hbase.ipc.ServerNotRunningYetException:
> org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server
> server_48c,17020,1548727752747 is not running yet
> 2019-01-28 18:12:12,179 WARN [RSProcedureDispatcher-pool4-t362]
> procedure.RSProcedureDispatcher: server server_48c,17020,1548726406632 is not
> up for a while; try a new one
> {noformat}
> Without any other reason (at least logged), the RIT immediately retries again
> and chooses a new candidate. It then retries again and goes to the new 48c,
> but that's unrelated.
> {noformat}
> 2019-01-28 18:12:12,289 INFO [KeepAlivePEWorker-100]
>
[jira] [Commented] (HBASE-21811) region can be opened on two servers due to race condition with procedures and server reports
[
https://issues.apache.org/jira/browse/HBASE-21811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758890#comment-16758890
]
Hadoop QA commented on HBASE-21811:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 38s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
23s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}127m 54s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
53s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}173m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.assignment.TestWakeUpUnexpectedProcedure |
| | hadoop.hbase.replication.TestSyncReplicationStandbyKillRS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21811 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12957355/HBASE-21811-v1.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ca520ca841fd 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision
[jira] [Updated] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[ https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21808: Resolution: Fixed Fix Version/s: 2.2.0 3.0.0 Status: Resolved (was: Patch Available) > Ensure we can build with JDK11 targetting JDK8 > -- > > Key: HBASE-21808 > URL: https://issues.apache.org/jira/browse/HBASE-21808 > Project: HBase > Issue Type: Improvement > Components: build, community, java >Affects Versions: 3.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21808.0.patch > > > When folks "install a JDK" wether they get JDK11 now will depend on the > platform (e.g. from homebrew you will get jdk11). For new contributors this > results in confusing errors on first run. > Make it so that a non-release build using JDK11 and using our default > compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21783) Support exceed user/table/ns throttle quota if region server has available quota
[ https://issues.apache.org/jira/browse/HBASE-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21783: --- Attachment: HBASE-21783.master.003.patch > Support exceed user/table/ns throttle quota if region server has available > quota > > > Key: HBASE-21783 > URL: https://issues.apache.org/jira/browse/HBASE-21783 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21783.master.001.patch, > HBASE-21783.master.002.patch, HBASE-21783.master.003.patch > > > Currently, all types of rpc throttle quota (include region server, namespace, > table and user quota) are hard limit, which means once requests exceed the > amount, they will be throttled. > In some situation, user use out of all their own quotas but region server > still has available quota because other users don't consume at the same time, > in this case, we can allow user consume additional quota. So add a switch to > enable or disable other quotas(except region server quota) exceed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21827) CompletableFuture may wrap the original Throwable with CompletionException while propagating it
Duo Zhang created HBASE-21827: - Summary: CompletableFuture may wrap the original Throwable with CompletionException while propagating it Key: HBASE-21827 URL: https://issues.apache.org/jira/browse/HBASE-21827 Project: HBase Issue Type: Bug Components: asyncclient, Client Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 3.0.0, 2.2.0, 2.3.0 See the stack overflow post https://stackoverflow.com/questions/49230980/does-completionstage-always-wrap-exceptions-in-completionexception It seems that, for a chain of CompletableFutures, the first child will receive the original exception, and the latter ones will receive a CompletionException, which is the wrapper of the original exception. This is also what I observed when debugging HBASE-21779. We need to fix this, at least, we should unwrap it when we want to return exceptions to users. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21779) Reimplement BulkLoadHFilesTool to use AsyncClusterConnection
[ https://issues.apache.org/jira/browse/HBASE-21779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758864#comment-16758864 ] Duo Zhang commented on HBASE-21779: --- OK, things are a bit strange. The TableNotFoundException is wrapped by a CompletionException. Need to find out why. > Reimplement BulkLoadHFilesTool to use AsyncClusterConnection > > > Key: HBASE-21779 > URL: https://issues.apache.org/jira/browse/HBASE-21779 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21779-HBASE-21512-v1.patch, > HBASE-21779-HBASE-21512-v2.patch, HBASE-21779-HBASE-21512.patch > > > So we will not rely on the RpcRetryingCaller and ServiceCallable any more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21811) region can be opened on two servers due to race condition with procedures and server reports
[
https://issues.apache.org/jira/browse/HBASE-21811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21811:
--
Attachment: HBASE-21811-v1.patch
> region can be opened on two servers due to race condition with procedures and
> server reports
>
>
> Key: HBASE-21811
> URL: https://issues.apache.org/jira/browse/HBASE-21811
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 3.0.0
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Blocker
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21811-UT.patch, HBASE-21811-v1.patch,
> HBASE-21811.patch
>
>
> Looks like the region server responses are being processed incorrectly in
> places allowing te region to be opened on two servers.
> * The region server report handling in procedures should check which server
> is reporting.
> * Also although I didn't check (and it isn't implicated in this bug), RS must
> check in OPEN that it's actually the correct RS master sent open to (w.r.t.
> start timestamp)
> This was previosuly "mitigated" by master killing the RS with incorrect
> reports, but due to race conditions with reports and assignment the kill was
> replaced with a warning, so now this condition persists.
> Regardless, the kill approach is not a good fix because there's still a
> window when a region can be opened on two servers.
> A region is being opened by server_48c. The server dies, and we process the
> retry correctly (retry=3 because 2 previous similar open failures were
> processed correctly).
> We start opening it on server_1aa now.
> {noformat}
> 2019-01-28 18:12:09,862 INFO [KeepAlivePEWorker-104]
> assignment.RegionStateStore: pid=4915 updating hbase:meta
> row=8be2a423b16471b9417f0f7de04281c6, regionState=ABNORMALLY_CLOSED
> 2019-01-28 18:12:09,862 INFO [KeepAlivePEWorker-104]
> procedure.ServerCrashProcedure: pid=11944,
> state=RUNNABLE:SERVER_CRASH_ASSIGN, hasLock=true; ServerCrashProcedure
> server=server_48c,17020,1548726406632, splitWal=true, meta=false found RIT
> pid=4915, ppid=7, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED,
> hasLock=true; TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=OPENING,
> location=server_48c,17020,1548726406632, table=table,
> region=8be2a423b16471b9417f0f7de04281c6
> 2019-01-28 18:12:10,778 INFO [KeepAlivePEWorker-80]
> assignment.TransitRegionStateProcedure: Retry=3 of max=2147483647; pid=4915,
> ppid=7, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_OPENED, hasLock=true;
> TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=ABNORMALLY_CLOSED,
> location=null
> ...
> 2019-01-28 18:12:10,902 INFO [KeepAlivePEWorker-80]
> assignment.TransitRegionStateProcedure: Starting pid=4915, ppid=7,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true;
> TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=ABNORMALLY_CLOSED,
> location=null; forceNewPlan=true, retain=false
> 2019-01-28 18:12:11,114 INFO [PEWorker-7] assignment.RegionStateStore:
> pid=4915 updating hbase:meta row=8be2a423b16471b9417f0f7de04281c6,
> regionState=OPENING, regionLocation=server_1aa,17020,1548727658713
> {noformat}
> However, we get the remote procedure failure from 48c after we've already
> started that.
> It actually tried to open on the restarted RS, which makes me wonder if this
> is safe also w.r.t. other races - what if RS already initialized and didn't
> error out?
> Need to check if we verify the start code expected by master on RS when
> opening.
> {noformat}
> 2019-01-28 18:12:12,179 WARN [RSProcedureDispatcher-pool4-t362]
> assignment.RegionRemoteProcedureBase: The remote operation pid=11050,
> ppid=4915, state=SUCCESS, hasLock=false;
> org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure for region
> {ENCODED => 8be2a423b16471b9417f0f7de04281c6 ... to server
> server_48c,17020,1548726406632 failed
> org.apache.hadoop.hbase.ipc.ServerNotRunningYetException:
> org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server
> server_48c,17020,1548727752747 is not running yet
> 2019-01-28 18:12:12,179 WARN [RSProcedureDispatcher-pool4-t362]
> procedure.RSProcedureDispatcher: server server_48c,17020,1548726406632 is not
> up for a while; try a new one
> {noformat}
> Without any other reason (at least logged), the RIT immediately retries again
> and chooses a new candidate. It then retries again and goes to the new 48c,
> but that's unrelated.
> {noformat}
> 2019-01-28 18:12:12,289 INFO [KeepAlivePEWorker-100]
> assignment.TransitRegionStateProcedure: Retry=4 of max=2147483647; pid=4915,
[jira] [Commented] (HBASE-21811) region can be opened on two servers due to race condition with procedures and server reports
[
https://issues.apache.org/jira/browse/HBASE-21811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758849#comment-16758849
]
Duo Zhang commented on HBASE-21811:
---
And we need to check for start code before executing any procedures in
executeProcedures, otherwise it will also lead to multiple assigns. We need to
fail all the procedures.
> region can be opened on two servers due to race condition with procedures and
> server reports
>
>
> Key: HBASE-21811
> URL: https://issues.apache.org/jira/browse/HBASE-21811
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 3.0.0
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Blocker
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21811-UT.patch, HBASE-21811.patch
>
>
> Looks like the region server responses are being processed incorrectly in
> places allowing te region to be opened on two servers.
> * The region server report handling in procedures should check which server
> is reporting.
> * Also although I didn't check (and it isn't implicated in this bug), RS must
> check in OPEN that it's actually the correct RS master sent open to (w.r.t.
> start timestamp)
> This was previosuly "mitigated" by master killing the RS with incorrect
> reports, but due to race conditions with reports and assignment the kill was
> replaced with a warning, so now this condition persists.
> Regardless, the kill approach is not a good fix because there's still a
> window when a region can be opened on two servers.
> A region is being opened by server_48c. The server dies, and we process the
> retry correctly (retry=3 because 2 previous similar open failures were
> processed correctly).
> We start opening it on server_1aa now.
> {noformat}
> 2019-01-28 18:12:09,862 INFO [KeepAlivePEWorker-104]
> assignment.RegionStateStore: pid=4915 updating hbase:meta
> row=8be2a423b16471b9417f0f7de04281c6, regionState=ABNORMALLY_CLOSED
> 2019-01-28 18:12:09,862 INFO [KeepAlivePEWorker-104]
> procedure.ServerCrashProcedure: pid=11944,
> state=RUNNABLE:SERVER_CRASH_ASSIGN, hasLock=true; ServerCrashProcedure
> server=server_48c,17020,1548726406632, splitWal=true, meta=false found RIT
> pid=4915, ppid=7, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED,
> hasLock=true; TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=OPENING,
> location=server_48c,17020,1548726406632, table=table,
> region=8be2a423b16471b9417f0f7de04281c6
> 2019-01-28 18:12:10,778 INFO [KeepAlivePEWorker-80]
> assignment.TransitRegionStateProcedure: Retry=3 of max=2147483647; pid=4915,
> ppid=7, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_OPENED, hasLock=true;
> TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=ABNORMALLY_CLOSED,
> location=null
> ...
> 2019-01-28 18:12:10,902 INFO [KeepAlivePEWorker-80]
> assignment.TransitRegionStateProcedure: Starting pid=4915, ppid=7,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true;
> TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=ABNORMALLY_CLOSED,
> location=null; forceNewPlan=true, retain=false
> 2019-01-28 18:12:11,114 INFO [PEWorker-7] assignment.RegionStateStore:
> pid=4915 updating hbase:meta row=8be2a423b16471b9417f0f7de04281c6,
> regionState=OPENING, regionLocation=server_1aa,17020,1548727658713
> {noformat}
> However, we get the remote procedure failure from 48c after we've already
> started that.
> It actually tried to open on the restarted RS, which makes me wonder if this
> is safe also w.r.t. other races - what if RS already initialized and didn't
> error out?
> Need to check if we verify the start code expected by master on RS when
> opening.
> {noformat}
> 2019-01-28 18:12:12,179 WARN [RSProcedureDispatcher-pool4-t362]
> assignment.RegionRemoteProcedureBase: The remote operation pid=11050,
> ppid=4915, state=SUCCESS, hasLock=false;
> org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure for region
> {ENCODED => 8be2a423b16471b9417f0f7de04281c6 ... to server
> server_48c,17020,1548726406632 failed
> org.apache.hadoop.hbase.ipc.ServerNotRunningYetException:
> org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server
> server_48c,17020,1548727752747 is not running yet
> 2019-01-28 18:12:12,179 WARN [RSProcedureDispatcher-pool4-t362]
> procedure.RSProcedureDispatcher: server server_48c,17020,1548726406632 is not
> up for a while; try a new one
> {noformat}
> Without any other reason (at least logged), the RIT immediately retries again
> and chooses a new candidate. It then retries again and goes to the new 48c,
> but that's
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758847#comment-16758847
]
Hudson commented on HBASE-21512:
Results for branch HBASE-21512
[build #83 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/83/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/83//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/83//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/83//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
> --
>
> Key: HBASE-21512
> URL: https://issues.apache.org/jira/browse/HBASE-21512
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> At least for the RSProcedureDispatcher, with CompletableFuture we do not need
> to set a delay and use a thread pool any more, which could reduce the
> resource usage and also the latency.
> Once this is done, I think we can remove the ClusterConnection completely,
> and start to rewrite the old sync client based on the async client, which
> could reduce the code base a lot for our client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21794) Update the Coprocessor observer example given in section 111.1 of the ref guide.
[
https://issues.apache.org/jira/browse/HBASE-21794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758845#comment-16758845
]
Hudson commented on HBASE-21794:
Results for branch master
[build #764 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/764/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/764//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/764//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/764//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Update the Coprocessor observer example given in section 111.1 of the ref
> guide.
>
>
> Key: HBASE-21794
> URL: https://issues.apache.org/jira/browse/HBASE-21794
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, documentation
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: hbase-21794.master.001.patch,
> hbase-21794.master.002.patch, hbase-21794.master.003.patch,
> hbase-21794.master.004.patch
>
>
> The given example should be changed after the CP changes (HBASE-17732)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21645) Perform sanity check and disallow table creation/modification with region replication < 1
[
https://issues.apache.org/jira/browse/HBASE-21645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758828#comment-16758828
]
Hudson commented on HBASE-21645:
Results for branch branch-1.4
[build #653 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/653/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/653//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/653//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/653//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Perform sanity check and disallow table creation/modification with region
> replication < 1
> -
>
> Key: HBASE-21645
> URL: https://issues.apache.org/jira/browse/HBASE-21645
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.5.0, 2.1.1, 2.1.2
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5
>
> Attachments: HBASE-21645.master.001.patch,
> HBASE-21645.master.001.patch, HBASE-21645.master.002.patch
>
>
> We should perform sanity check and disallow table creation with region
> replication < 1 or modification of an existing table with new region
> replication value < 1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21811) region can be opened on two servers due to race condition with procedures and server reports
[
https://issues.apache.org/jira/browse/HBASE-21811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758819#comment-16758819
]
Duo Zhang commented on HBASE-21811:
---
I know that checking for ServerName could work but it may confuse people on
what is actually going on here.
> region can be opened on two servers due to race condition with procedures and
> server reports
>
>
> Key: HBASE-21811
> URL: https://issues.apache.org/jira/browse/HBASE-21811
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Affects Versions: 3.0.0
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Blocker
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21811-UT.patch, HBASE-21811.patch
>
>
> Looks like the region server responses are being processed incorrectly in
> places allowing te region to be opened on two servers.
> * The region server report handling in procedures should check which server
> is reporting.
> * Also although I didn't check (and it isn't implicated in this bug), RS must
> check in OPEN that it's actually the correct RS master sent open to (w.r.t.
> start timestamp)
> This was previosuly "mitigated" by master killing the RS with incorrect
> reports, but due to race conditions with reports and assignment the kill was
> replaced with a warning, so now this condition persists.
> Regardless, the kill approach is not a good fix because there's still a
> window when a region can be opened on two servers.
> A region is being opened by server_48c. The server dies, and we process the
> retry correctly (retry=3 because 2 previous similar open failures were
> processed correctly).
> We start opening it on server_1aa now.
> {noformat}
> 2019-01-28 18:12:09,862 INFO [KeepAlivePEWorker-104]
> assignment.RegionStateStore: pid=4915 updating hbase:meta
> row=8be2a423b16471b9417f0f7de04281c6, regionState=ABNORMALLY_CLOSED
> 2019-01-28 18:12:09,862 INFO [KeepAlivePEWorker-104]
> procedure.ServerCrashProcedure: pid=11944,
> state=RUNNABLE:SERVER_CRASH_ASSIGN, hasLock=true; ServerCrashProcedure
> server=server_48c,17020,1548726406632, splitWal=true, meta=false found RIT
> pid=4915, ppid=7, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED,
> hasLock=true; TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=OPENING,
> location=server_48c,17020,1548726406632, table=table,
> region=8be2a423b16471b9417f0f7de04281c6
> 2019-01-28 18:12:10,778 INFO [KeepAlivePEWorker-80]
> assignment.TransitRegionStateProcedure: Retry=3 of max=2147483647; pid=4915,
> ppid=7, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_OPENED, hasLock=true;
> TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=ABNORMALLY_CLOSED,
> location=null
> ...
> 2019-01-28 18:12:10,902 INFO [KeepAlivePEWorker-80]
> assignment.TransitRegionStateProcedure: Starting pid=4915, ppid=7,
> state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true;
> TransitRegionStateProcedure table=table,
> region=8be2a423b16471b9417f0f7de04281c6, ASSIGN; rit=ABNORMALLY_CLOSED,
> location=null; forceNewPlan=true, retain=false
> 2019-01-28 18:12:11,114 INFO [PEWorker-7] assignment.RegionStateStore:
> pid=4915 updating hbase:meta row=8be2a423b16471b9417f0f7de04281c6,
> regionState=OPENING, regionLocation=server_1aa,17020,1548727658713
> {noformat}
> However, we get the remote procedure failure from 48c after we've already
> started that.
> It actually tried to open on the restarted RS, which makes me wonder if this
> is safe also w.r.t. other races - what if RS already initialized and didn't
> error out?
> Need to check if we verify the start code expected by master on RS when
> opening.
> {noformat}
> 2019-01-28 18:12:12,179 WARN [RSProcedureDispatcher-pool4-t362]
> assignment.RegionRemoteProcedureBase: The remote operation pid=11050,
> ppid=4915, state=SUCCESS, hasLock=false;
> org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure for region
> {ENCODED => 8be2a423b16471b9417f0f7de04281c6 ... to server
> server_48c,17020,1548726406632 failed
> org.apache.hadoop.hbase.ipc.ServerNotRunningYetException:
> org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server
> server_48c,17020,1548727752747 is not running yet
> 2019-01-28 18:12:12,179 WARN [RSProcedureDispatcher-pool4-t362]
> procedure.RSProcedureDispatcher: server server_48c,17020,1548726406632 is not
> up for a while; try a new one
> {noformat}
> Without any other reason (at least logged), the RIT immediately retries again
> and chooses a new candidate. It then retries again and goes to the new 48c,
> but that's unrelated.
> {noformat}
> 2019-01-28 18:12:12,289 INFO
[jira] [Commented] (HBASE-21779) Reimplement BulkLoadHFilesTool to use AsyncClusterConnection
[ https://issues.apache.org/jira/browse/HBASE-21779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758816#comment-16758816 ] Duo Zhang commented on HBASE-21779: --- Let me check the failed UTs. > Reimplement BulkLoadHFilesTool to use AsyncClusterConnection > > > Key: HBASE-21779 > URL: https://issues.apache.org/jira/browse/HBASE-21779 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21779-HBASE-21512-v1.patch, > HBASE-21779-HBASE-21512-v2.patch, HBASE-21779-HBASE-21512.patch > > > So we will not rely on the RpcRetryingCaller and ServiceCallable any more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21816) Print source cluster replication config directory
[
https://issues.apache.org/jira/browse/HBASE-21816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758771#comment-16758771
]
Hadoop QA commented on HBASE-21816:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
22s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 30s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}131m 26s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.TestMasterRepairMode |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21816 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12957318/HBASE-21816-003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 4d3b34334e75 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 64c32720d6 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/15850/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
[jira] [Commented] (HBASE-21823) only report RS abort once
[
https://issues.apache.org/jira/browse/HBASE-21823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758754#comment-16758754
]
Hadoop QA commented on HBASE-21823:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
35s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 47s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}143m 21s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}185m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.TestSyncReplicationStandbyKillRS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21823 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12957306/HBASE-21823.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 80a94f66ca44 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 64c32720d6 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/15840/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
[jira] [Commented] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[
https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758739#comment-16758739
]
Hadoop QA commented on HBASE-20172:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} branch-1 passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} branch-1 passed with JDK v1.7.0_211 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
30s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} branch-1 passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} branch-1 passed with JDK v1.7.0_211 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.7.0_211 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
26s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 36s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed with JDK v1.8.0_202 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.7.0_211 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}153m 17s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}170m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles |
| | hadoop.hbase.mapreduce.TestLoadIncrementalHFiles |
| | hadoop.hbase.client.TestAdmin1 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-20172 |
| JIRA Patch URL |
[jira] [Commented] (HBASE-21824) change master and RS UI links to be relative
[
https://issues.apache.org/jira/browse/HBASE-21824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758733#comment-16758733
]
Hadoop QA commented on HBASE-21824:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}134m 40s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}146m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21824 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12957309/HBASE-21824.patch |
| Optional Tests | dupname asflicense javac javadoc unit |
| uname | Linux 0422fd5985b7 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 64c32720d6 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/15849/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15849/testReport/ |
| Max. process+thread count | 5235 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15849/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> change master and RS UI links to be relative
>
>
> Key: HBASE-21824
> URL: https://issues.apache.org/jira/browse/HBASE-21824
> Project: HBase
> Issue Type: Improvement
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HBASE-21824.patch
>
>
> When HBase services are accessed thru the proxy e.g.
> proxy/foo/bar/machine:port/master-status, the current links on the page lead
> to e.g. proxy/procedures.jsp, because they start with a slash. There seems to
> be no reason for them to have a slash since all the pages are on the same
> level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18620) Secure bulkload job fails when HDFS umask has limited scope
[
https://issues.apache.org/jira/browse/HBASE-18620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758709#comment-16758709
]
Hadoop QA commented on HBASE-18620:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
53s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} branch-1 passed with JDK v1.8.0_201 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
40s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} branch-1 passed with JDK v1.8.0_201 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.8.0_201 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed with JDK v1.8.0_201 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}120m
40s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}139m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-18620 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920589/HBASE-18620-branch-1-v3.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux dd80e35164dc
[jira] [Commented] (HBASE-21824) change master and RS UI links to be relative
[ https://issues.apache.org/jira/browse/HBASE-21824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758687#comment-16758687 ] Tommy Li commented on HBASE-21824: -- +1 > change master and RS UI links to be relative > > > Key: HBASE-21824 > URL: https://issues.apache.org/jira/browse/HBASE-21824 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HBASE-21824.patch > > > When HBase services are accessed thru the proxy e.g. > proxy/foo/bar/machine:port/master-status, the current links on the page lead > to e.g. proxy/procedures.jsp, because they start with a slash. There seems to > be no reason for them to have a slash since all the pages are on the same > level. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[ https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758680#comment-16758680 ] Josh Elser commented on HBASE-21808: SGTM. I didn't think it was necessary to add such an axis of compatibility. I think you're good here :) > Ensure we can build with JDK11 targetting JDK8 > -- > > Key: HBASE-21808 > URL: https://issues.apache.org/jira/browse/HBASE-21808 > Project: HBase > Issue Type: Improvement > Components: build, community, java >Affects Versions: 3.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Attachments: HBASE-21808.0.patch > > > When folks "install a JDK" wether they get JDK11 now will depend on the > platform (e.g. from homebrew you will get jdk11). For new contributors this > results in confusing errors on first run. > Make it so that a non-release build using JDK11 and using our default > compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21816) Print source cluster replication config directory
[
https://issues.apache.org/jira/browse/HBASE-21816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Palanisamy updated HBASE-21816:
---
Attachment: HBASE-21816-003.patch
> Print source cluster replication config directory
> -
>
> Key: HBASE-21816
> URL: https://issues.apache.org/jira/browse/HBASE-21816
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 3.0.0, 2.0.0
> Environment: NA
>Reporter: Karthik Palanisamy
>Assignee: Karthik Palanisamy
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21816-001.patch, HBASE-21816-002.patch,
> HBASE-21816-003.patch
>
>
> User may get confused, to understanding our HBase configurations which are
> loaded for replication. Sometimes, User may place source and destination
> cluster conf under "/etc/hbase/conf" directory. It will create uncertainty
> because our log points that all the configurations are co-located.
>
> Existing Logs,
> {code:java}
> INFO [RpcServer.FifoWFPBQ.replication.handler=2,queue=0,port=16020]
> regionserver.DefaultSourceFSConfigurationProvider: Loading source cluster
> HDP1 file system configurations from xml files under directory
> /etc/hbase/conf/
> {code}
> But it should be something like,
> {code:java}
> INFO [RpcServer.FifoWFPBQ.replication.handler=2,queue=0,port=16020]
> regionserver.DefaultSourceFSConfigurationProvider: Loading source cluster
> HDP1 file system configurations from xml files under directory
> /etc/hbase/conf/HDP1
> {code}
>
> This jira only to change the log-line, no issue with the functionality.
> {code:java}
> File confDir = new File(replicationConfDir, replicationClusterId);
> String[] listofConfFiles = FileUtil.list(confDir);
> for (String confFile : listofConfFiles) {
> if (new File(confDir, confFile).isFile() && confFile.endsWith(XML)) {
> // Add all the user provided client conf files
> sourceClusterConf.addResource(new Path(confDir.getPath(), confFile));
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21816) Print source cluster replication config directory
[
https://issues.apache.org/jira/browse/HBASE-21816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758676#comment-16758676
]
Karthik Palanisamy commented on HBASE-21816:
Please find the new patch.
> Print source cluster replication config directory
> -
>
> Key: HBASE-21816
> URL: https://issues.apache.org/jira/browse/HBASE-21816
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 3.0.0, 2.0.0
> Environment: NA
>Reporter: Karthik Palanisamy
>Assignee: Karthik Palanisamy
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21816-001.patch, HBASE-21816-002.patch,
> HBASE-21816-003.patch
>
>
> User may get confused, to understanding our HBase configurations which are
> loaded for replication. Sometimes, User may place source and destination
> cluster conf under "/etc/hbase/conf" directory. It will create uncertainty
> because our log points that all the configurations are co-located.
>
> Existing Logs,
> {code:java}
> INFO [RpcServer.FifoWFPBQ.replication.handler=2,queue=0,port=16020]
> regionserver.DefaultSourceFSConfigurationProvider: Loading source cluster
> HDP1 file system configurations from xml files under directory
> /etc/hbase/conf/
> {code}
> But it should be something like,
> {code:java}
> INFO [RpcServer.FifoWFPBQ.replication.handler=2,queue=0,port=16020]
> regionserver.DefaultSourceFSConfigurationProvider: Loading source cluster
> HDP1 file system configurations from xml files under directory
> /etc/hbase/conf/HDP1
> {code}
>
> This jira only to change the log-line, no issue with the functionality.
> {code:java}
> File confDir = new File(replicationConfDir, replicationClusterId);
> String[] listofConfFiles = FileUtil.list(confDir);
> for (String confFile : listofConfFiles) {
> if (new File(confDir, confFile).isFile() && confFile.endsWith(XML)) {
> // Add all the user provided client conf files
> sourceClusterConf.addResource(new Path(confDir.getPath(), confFile));
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21816) Print source cluster replication config directory
[
https://issues.apache.org/jira/browse/HBASE-21816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758674#comment-16758674
]
Karthik Palanisamy commented on HBASE-21816:
Thank you [~elserj] for reviewing it. Yes, This should be a simple change to
reverse it. I lined edge :)
> Print source cluster replication config directory
> -
>
> Key: HBASE-21816
> URL: https://issues.apache.org/jira/browse/HBASE-21816
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 3.0.0, 2.0.0
> Environment: NA
>Reporter: Karthik Palanisamy
>Assignee: Karthik Palanisamy
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21816-001.patch, HBASE-21816-002.patch,
> HBASE-21816-003.patch
>
>
> User may get confused, to understanding our HBase configurations which are
> loaded for replication. Sometimes, User may place source and destination
> cluster conf under "/etc/hbase/conf" directory. It will create uncertainty
> because our log points that all the configurations are co-located.
>
> Existing Logs,
> {code:java}
> INFO [RpcServer.FifoWFPBQ.replication.handler=2,queue=0,port=16020]
> regionserver.DefaultSourceFSConfigurationProvider: Loading source cluster
> HDP1 file system configurations from xml files under directory
> /etc/hbase/conf/
> {code}
> But it should be something like,
> {code:java}
> INFO [RpcServer.FifoWFPBQ.replication.handler=2,queue=0,port=16020]
> regionserver.DefaultSourceFSConfigurationProvider: Loading source cluster
> HDP1 file system configurations from xml files under directory
> /etc/hbase/conf/HDP1
> {code}
>
> This jira only to change the log-line, no issue with the functionality.
> {code:java}
> File confDir = new File(replicationConfDir, replicationClusterId);
> String[] listofConfFiles = FileUtil.list(confDir);
> for (String confFile : listofConfFiles) {
> if (new File(confDir, confFile).isFile() && confFile.endsWith(XML)) {
> // Add all the user provided client conf files
> sourceClusterConf.addResource(new Path(confDir.getPath(), confFile));
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21808) Ensure we can build with JDK11 targetting JDK8
[ https://issues.apache.org/jira/browse/HBASE-21808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758665#comment-16758665 ] Sean Busbey commented on HBASE-21808: - bq. Looking again at https://hbase.apache.org/book.html#java – is it worth our adding another table for JDK compatibility: supported JDK for building vs. support JDK for running? I don't think so. IIRC back when we were first adding jdk8 support we could run but not build and we just put that in a note on the section. I think a similar note would do here if we wanted to update on the status of jdk11. > Ensure we can build with JDK11 targetting JDK8 > -- > > Key: HBASE-21808 > URL: https://issues.apache.org/jira/browse/HBASE-21808 > Project: HBase > Issue Type: Improvement > Components: build, community, java >Affects Versions: 3.0.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Attachments: HBASE-21808.0.patch > > > When folks "install a JDK" wether they get JDK11 now will depend on the > platform (e.g. from homebrew you will get jdk11). For new contributors this > results in confusing errors on first run. > Make it so that a non-release build using JDK11 and using our default > compiler source/target versions of JDK8 can complete. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20586) SyncTable tool: Add support for cross-realm remote clusters
[
https://issues.apache.org/jira/browse/HBASE-20586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758654#comment-16758654
]
Hadoop QA commented on HBASE-20586:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
3s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 6s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 24m
37s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-20586 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12923490/HBASE-20586.master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f2c39e266ec2 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 64c32720d6 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15845/testReport/ |
| Max. process+thread count | 5469 (vs. ulimit of 1) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output |
[jira] [Updated] (HBASE-20555) Backport HBASE-18083 and related changes in branch-1
[ https://issues.apache.org/jira/browse/HBASE-20555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20555: --- Fix Version/s: (was: 1.5.0) > Backport HBASE-18083 and related changes in branch-1 > > > Key: HBASE-20555 > URL: https://issues.apache.org/jira/browse/HBASE-20555 > Project: HBase > Issue Type: Umbrella > Components: HFile, snapshots >Affects Versions: 1.4.4, 1.4.5 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Major > Fix For: 1.4.6 > > > This will be the umbrella JIRA for backporting HBASE-18083 of `Make > large/small file clean thread number configurable in HFileCleaner` from > HBase's branch-2 to HBase's branch-1 that will needs a total of 4 sub-tasks > that backport HBASE-16490, HBASE-17215, HBASE-17854 and then HBASE-18083 > The goal is to improve HFile cleaning performance that has been introduced in > branch-2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19949) TestRSGroupsWithACL fails with ExceptionInInitializerError
[
https://issues.apache.org/jira/browse/HBASE-19949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19949:
---
Fix Version/s: (was: 1.5.0)
> TestRSGroupsWithACL fails with ExceptionInInitializerError
> --
>
> Key: HBASE-19949
> URL: https://issues.apache.org/jira/browse/HBASE-19949
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 2.0.0-beta-2, 1.4.2, 2.0.0
>
> Attachments: 19949.v1.txt
>
>
> {code}
> org.apache.hadoop.hbase.rsgroup.TestRSGroupsWithACL Time elapsed: 0.168 sec
> <<< ERROR!
> java.lang.ExceptionInInitializerError
> at
> org.apache.hadoop.hbase.rsgroup.TestRSGroupsWithACL.(TestRSGroupsWithACL.java:60)
> {code}
> The cause is that HBaseClassTestRule#getTimeoutInSeconds() cannot retrieve
> annotation for test size.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20112) Include test results from nightly hadoop3 tests in jenkins test results
[ https://issues.apache.org/jira/browse/HBASE-20112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20112: --- Fix Version/s: (was: 1.5.0) > Include test results from nightly hadoop3 tests in jenkins test results > --- > > Key: HBASE-20112 > URL: https://issues.apache.org/jira/browse/HBASE-20112 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 2.1.0, 1.3.3, 1.4.4, 2.0.1, 1.2.7 > > Attachments: HBASE-20112.0.patch > > > right now our nightly tests that run atop hadoop 3 are reported on pass/fail > but aren't recorded via the jenkins reporting mechanism. > we should add them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20075) remove logic for branch-1.1 nightly testing
[ https://issues.apache.org/jira/browse/HBASE-20075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20075: --- Fix Version/s: (was: 1.5.0) > remove logic for branch-1.1 nightly testing > --- > > Key: HBASE-20075 > URL: https://issues.apache.org/jira/browse/HBASE-20075 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 1.1.13 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 1.3.2, 1.4.3, 2.0.0, 1.2.7 > > Attachments: HBASE-20075.0.patch > > > since branch-1.1 is EOM, remove the branch-1.1 specific logic from our > Jenkinsfile and delete the Jenkinsfile in branch-1.1 so we'll stop running > nightlies for it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20505) PE should support multi column family read and write cases
[ https://issues.apache.org/jira/browse/HBASE-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20505: --- Fix Version/s: (was: 1.5.0) > PE should support multi column family read and write cases > -- > > Key: HBASE-20505 > URL: https://issues.apache.org/jira/browse/HBASE-20505 > Project: HBase > Issue Type: Test > Components: Performance >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.3.3, 2.0.1, 1.4.5, 1.2.7 > > Attachments: HBASE-20505-branch-1.patch, HBASE-20505.patch > > > PerformanceEvaluation has a --columns parameter but this adjusts the number > of distinct column qualifiers to write (and, with --addColumns, to add to the > scan), not the number of column families. > We need something like a new --families parameter that will increase the > number of column families defined in the test table schema, written to, and > included in gets and scans. Default is 1, current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19514) Use random port for TestJMXListener
[
https://issues.apache.org/jira/browse/HBASE-19514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19514:
---
Fix Version/s: (was: 1.5.0)
> Use random port for TestJMXListener
> ---
>
> Key: HBASE-19514
> URL: https://issues.apache.org/jira/browse/HBASE-19514
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 1.3.2, 1.4.1, 2.0.0-beta-1, 2.0.0, 1.2.7
>
> Attachments: 19514.v1.txt, 19514.v2.txt
>
>
> In recent trunk builds, there was TestJMXListener failure with the following:
> {code}
> 2017-12-14 15:42:13,278 ERROR [RS:0;asf918:34443]
> coprocessor.CoprocessorHost(399): The coprocessor
> org.apache.hadoop.hbase.JMXListener threw java.rmi.server.
> ExportException: Port already in use: 61120; nested exception is:
> java.net.BindException: Address already in use (Bind failed)
> java.rmi.server.ExportException: Port already in use: 61120; nested exception
> is:
> java.net.BindException: Address already in use (Bind failed)
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:341)
> at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:249)
> at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
> at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
> at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:236)
> at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:213)
> at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:198)
> at java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:203)
> at
> org.apache.hadoop.hbase.JMXListener.startConnectorServer(JMXListener.java:133)
> at org.apache.hadoop.hbase.JMXListener.start(JMXListener.java:208)
> at
> org.apache.hadoop.hbase.coprocessor.BaseEnvironment.startup(BaseEnvironment.java:72)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:264)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:157)
> at
> org.apache.hadoop.hbase.regionserver.RegionServerCoprocessorHost.(RegionServerCoprocessorHost.java:68)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:942)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:163)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:110)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:147)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1726)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:305)
> at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:145)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.net.BindException: Address already in use (Bind failed)
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
> at java.net.ServerSocket.bind(ServerSocket.java:375)
> at java.net.ServerSocket.(ServerSocket.java:237)
> at java.net.ServerSocket.(ServerSocket.java:128)
> at
> sun.rmi.transport.proxy.RMIDirectSocketFactory.createServerSocket(RMIDirectSocketFactory.java:45)
> at
> sun.rmi.transport.proxy.RMIMasterSocketFactory.createServerSocket(RMIMasterSocketFactory.java:345)
> at sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
> {code}
> The test uses 61120 as connector port which would get into above exception if
> same test from another build is running on the same machine.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-15151) Rely on nightly tests for findbugs compliance on existing branch
[ https://issues.apache.org/jira/browse/HBASE-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15151: --- Fix Version/s: (was: 1.5.0) > Rely on nightly tests for findbugs compliance on existing branch > > > Key: HBASE-15151 > URL: https://issues.apache.org/jira/browse/HBASE-15151 > Project: HBase > Issue Type: Task > Components: build, test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 1.3.2, 1.4.3, 2.0.0, 1.2.7 > > Attachments: HBASE-15151.0.patch, HBASE-15151.1.patch > > > the "-1" for extant findbugs warnings has confused interpretation of our > precommit checks enough that we should switch to non-strict mode. > It will still record the number of findbugs warnings present before the > patch, but it'll vote "0" rather than calling attention to things via a -1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21826) Rebase 1.5.0 CHANGES on branch-1.4 at release 1.4.9
[ https://issues.apache.org/jira/browse/HBASE-21826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-21826. Resolution: Fixed > Rebase 1.5.0 CHANGES on branch-1.4 at release 1.4.9 > --- > > Key: HBASE-21826 > URL: https://issues.apache.org/jira/browse/HBASE-21826 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 1.5.0 > > > Release 1.5.0 CHANGES.txt is based on last 1.4 release (1.4.9). Rebase fix > versions accordingly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21826) Rebase 1.5.0 CHANGES on branch-1.4 at release 1.4.9
Andrew Purtell created HBASE-21826: -- Summary: Rebase 1.5.0 CHANGES on branch-1.4 at release 1.4.9 Key: HBASE-21826 URL: https://issues.apache.org/jira/browse/HBASE-21826 Project: HBase Issue Type: Task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.5.0 Release 1.5.0 CHANGES.txt is based on last 1.4 release (1.4.9). Rebase fix versions accordingly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20734) Colocate recovered edits directory with hbase.wal.dir
[ https://issues.apache.org/jira/browse/HBASE-20734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20734: --- Fix Version/s: (was: 1.5.0) > Colocate recovered edits directory with hbase.wal.dir > - > > Key: HBASE-20734 > URL: https://issues.apache.org/jira/browse/HBASE-20734 > Project: HBase > Issue Type: Improvement > Components: MTTR, Recovery, wal >Reporter: Ted Yu >Assignee: Zach York >Priority: Major > Fix For: 3.0.0, 1.3.3, 2.2.0, 1.4.8, 2.1.1 > > Attachments: HBASE-20734.branch-1.001.patch, > HBASE-20734.branch-1.002.patch, HBASE-20734.branch-1.003.patch, > HBASE-20734.branch-1.004.patch, HBASE-20734.branch-1.005.patch, > HBASE-20734.master.001.patch, HBASE-20734.master.002.patch, > HBASE-20734.master.003.patch, HBASE-20734.master.004.patch, > HBASE-20734.master.005.patch, HBASE-20734.master.006.patch, > HBASE-20734.master.007.patch, HBASE-20734.master.008.patch, > HBASE-20734.master.009.patch, HBASE-20734.master.010.patch, > HBASE-20734.master.011.patch, HBASE-20734.master.012.patch > > > During investigation of HBASE-20723, I realized that we wouldn't get the best > performance when hbase.wal.dir is configured to be on different (fast) media > than hbase rootdir w.r.t. recovered edits since recovered edits directory is > currently under rootdir. > Such setup may not result in fast recovery when there is region server > failover. > This issue is to find proper (hopefully backward compatible) way in > colocating recovered edits directory with hbase.wal.dir . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20459) Majority of scan CPU time in HBase-1 spent in size estimation
[ https://issues.apache.org/jira/browse/HBASE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20459: --- Fix Version/s: (was: 1.5.0) > Majority of scan CPU time in HBase-1 spent in size estimation > - > > Key: HBASE-20459 > URL: https://issues.apache.org/jira/browse/HBASE-20459 > Project: HBase > Issue Type: Improvement > Components: Performance, scan >Affects Versions: 1.4.3 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Critical > Fix For: 1.4.4, 2.0.0 > > Attachments: 20459-v2.txt, 20459.2.0.txt, 20459.txt, > HBASE-20459.branch-2.0.001.patch, Screenshot_20180419_162559.png > > > See attached screenshot. Will look into a fix later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20545) Improve performance of BaseLoadBalancer.retainAssignment
[ https://issues.apache.org/jira/browse/HBASE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20545: --- Fix Version/s: (was: 1.5.0) > Improve performance of BaseLoadBalancer.retainAssignment > > > Key: HBASE-20545 > URL: https://issues.apache.org/jira/browse/HBASE-20545 > Project: HBase > Issue Type: Improvement > Components: Balancer >Affects Versions: 1.4.4, 2.0.0 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.3.3, 1.4.5 > > Attachments: HBASE-20545.branch-1.4.001.patch, > HBASE-20545.branch-1.4.002.patch, HBASE-20545.branch-2.001.patch > > > I was measuring perf at scale with a 1m region table and noticed some > improvements can be made to BaseLoadBalancer.retainAssignment(). > retainAssignment() spends a few mins to enable a 1m regions table and also > generates a lot of objects unnecessarily. This jira is to make the most > common case go faster with very minimal changes. A slightly modified version > of this patch takes about 5 seconds for a 1m region table ignoring the time > spent in createCluster(). I think locality can be refreshed during master > startup in different ways without taking time in retainAssignment, but will > follow up on that in subsequent jiras. Leaving it untouched here, but wanted > to call out the time taken without that method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21168) BloomFilterUtil uses hardcoded randomness
[ https://issues.apache.org/jira/browse/HBASE-21168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21168: --- Fix Version/s: (was: 1.5.0) > BloomFilterUtil uses hardcoded randomness > - > > Key: HBASE-21168 > URL: https://issues.apache.org/jira/browse/HBASE-21168 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Trivial > Fix For: 3.0.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8, 2.1.1 > > Attachments: HBASE-21168.branch-1.002.patch, > HBASE-21168.master.001.patch, HBASE-21168.master.002.patch > > > This was flagged by a Fortify scan and while it doesn't appear to be a real > issue, it's pretty easy to take care of anyway. > The hard coded rand can be moved to the test class that actually needs it to > make the static analysis happy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20942) Improve RpcServer TRACE logging
[ https://issues.apache.org/jira/browse/HBASE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20942: --- Fix Version/s: (was: 1.5.0) > Improve RpcServer TRACE logging > --- > > Key: HBASE-20942 > URL: https://issues.apache.org/jira/browse/HBASE-20942 > Project: HBase > Issue Type: Task > Components: Operability >Affects Versions: 2.1.0, 1.4.6 >Reporter: Esteban Gutierrez >Assignee: Krish Dey >Priority: Major > Fix For: 3.0.0, 1.3.3, 2.2.0, 2.0.2, 1.4.7 > > Attachments: HBASE-20942.002.patch, HBASE-20942.003.patch, > HBASE-20942.004.patch, HBASE-20942.005.patch > > > Two things: > * We truncate RpcServer output to 1000 characters for trace logging. Would > be better if that value was configurable. > * There is the chance for an ArrayIndexOutOfBounds when truncating the TRACE > log message. > Esteban mentioned this to me earlier, so I'm crediting him as the reporter. > cc: [~elserj] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21265) Split up TestRSGroups
[ https://issues.apache.org/jira/browse/HBASE-21265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21265: --- Fix Version/s: (was: 1.5.0) > Split up TestRSGroups > - > > Key: HBASE-21265 > URL: https://issues.apache.org/jira/browse/HBASE-21265 > Project: HBase > Issue Type: Task > Components: rsgroup, test >Affects Versions: 1.4.8 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Labels: beginner, beginners > Fix For: 3.0.0, 2.2.0, 1.4.9 > > Attachments: HBASE-21265-branch-1.patch, HBASE-21265-branch-1.patch, > HBASE-21265-branch-2.patch, HBASE-21265-branch-2.patch, > HBASE-21265.branch-2.1.001.patch, HBASE-21265.patch, HBASE-21265.patch > > > TestRSGroups is flaky. It is stable when run in isolation but when run as > part of the suite with concurrent executors it can fail. The current running > time of this unit on my dev box is ~240 seconds (4 minutes), which is far too > much time. This unit should be broken up 5 to 8 ways, grouped by > functionality under test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21301) Heatmap for key access patterns
[
https://issues.apache.org/jira/browse/HBASE-21301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758650#comment-16758650
]
Hadoop QA commented on HBASE-21301:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
3s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 3m
8s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
17s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 17s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
22s{color} | {color:red} hbase-server: The patch generated 70 new + 72
unchanged - 0 fixed = 142 total (was 72) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 57 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 130 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
44s{color} | {color:red} patch has 10 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
0s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
6s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
16s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 17s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21301 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12946562/HBASE-21301.v0.master.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 87f5731bfe05 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 64c32720d6 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
|
[jira] [Updated] (HBASE-20905) branch-1 docker build fails
[
https://issues.apache.org/jira/browse/HBASE-20905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20905:
---
Fix Version/s: (was: 1.5.0)
> branch-1 docker build fails
> ---
>
> Key: HBASE-20905
> URL: https://issues.apache.org/jira/browse/HBASE-20905
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 1.5.0
>Reporter: Jingyun Tian
>Assignee: Mike Drob
>Priority: Major
> Fix For: 1.3.3, 1.4.6, 1.2.7
>
> Attachments: HBASE-20905.branch-1.001.patch
>
>
> Docker build for precommit fails:
> {quote}
> 19:08:29 Cleaning up...19:08:29 Command python setup.py egg_info failed with
> error code 1 in /tmp/pip_build_root/pylint*19:08:29* Storing debug log for
> failure in /root/.pip/pip.log*19:08:29* The command '/bin/sh -c pip install
> pylint' returned a non-zero code: 1*19:08:29* 19:08:29 Total Elapsed time: 0m
> 3s*19:08:29* 19:08:29 ERROR: Docker failed to build image.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20884) Replace usage of our Base64 implementation with java.util.Base64
[ https://issues.apache.org/jira/browse/HBASE-20884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20884: --- Fix Version/s: (was: 1.5.0) > Replace usage of our Base64 implementation with java.util.Base64 > > > Key: HBASE-20884 > URL: https://issues.apache.org/jira/browse/HBASE-20884 > Project: HBase > Issue Type: Task >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 3.0.0, 1.3.3, 1.4.6, 1.2.7, 2.1.1, 2.0.2 > > Attachments: HBASE-20884.branch-1.001.patch, > HBASE-20884.branch-1.002.patch, HBASE-20884.master.001.patch > > > We have a public domain implementation of Base64 that is copied into our code > base and infrequently receives updates. We should replace usage of that with > the new Java 8 java.util.Base64 where possible. > For the migration, I propose a phased approach. > * Deprecate on 1.x and 2.x to signal to users that this is going away. > * Replace usages on branch-2 and master with j.u.Base64 > * Delete our implementation of Base64 on master. > Does this seem in line with our API compatibility requirements? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20089) make_rc.sh should name SHA-512 checksum files with the extension .sha512
[
https://issues.apache.org/jira/browse/HBASE-20089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20089:
---
Fix Version/s: (was: 1.5.0)
> make_rc.sh should name SHA-512 checksum files with the extension .sha512
>
>
> Key: HBASE-20089
> URL: https://issues.apache.org/jira/browse/HBASE-20089
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 1.3.2, 1.4.3, 2.0.0
>
> Attachments: HBASE-20089.001.patch
>
>
> From [~elserj]
> {quote}
> we need to update the checksum naming convention for SHA*. Per [1], .sha
> filenames should only contain SHA1, and .sha512 file names should be used for
> SHA512 xsum. I believe this means we just need to modify make_rc.sh to put
> the xsum into .sha512 instead of .sha. We do not need to distribute SHA1
> xsums and, afaik, there is little cryptographic value to this.
> [1] http://www.apache.org/dev/release-distribution.html#sigs-and-sums
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20595) Remove the concept of 'special tables' from rsgroups
[ https://issues.apache.org/jira/browse/HBASE-20595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20595: --- Fix Version/s: (was: 1.5.0) > Remove the concept of 'special tables' from rsgroups > > > Key: HBASE-20595 > URL: https://issues.apache.org/jira/browse/HBASE-20595 > Project: HBase > Issue Type: Task > Components: Region Assignment, rsgroup >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 3.0.0, 2.1.0, 2.0.1, 1.4.5 > > Attachments: HBASE-20595-branch-1.patch, HBASE-20595.patch, > HBASE-20595.patch > > > Regionserver groups needs to specially handle what it calls "special tables", > tables upon which core or other modular functionality depends. They need to > be excluded from normal rsgroup processing during bootstrap to avoid circular > dependencies or errors due to insufficiently initialized state. I think we > also want to ensure that such tables are always given a rsgroup assignment > with nonzero servers. (IIRC another issue already raises that point, we can > link it later.) > Special tables include: > * The system tables in the 'hbase:' namespace > * The ACL table if the AccessController coprocessor is installed > * The Labels table if the VisibilityController coprocessor is installed > * The Quotas table if the FS quotas feature is active > Either we need a facility where "special tables" can be registered, which > should be in core. Or, we institute a blanket rule that core and all > extensions that need a "special table" must put them into the 'hbase:' > namespace, so the TableName#isSystemTable() test will return TRUE for all, > and then rsgroups simply needs to test for that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20088) Update copyright notices to year 2018
[ https://issues.apache.org/jira/browse/HBASE-20088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20088: --- Fix Version/s: (was: 1.5.0) > Update copyright notices to year 2018 > - > > Key: HBASE-20088 > URL: https://issues.apache.org/jira/browse/HBASE-20088 > Project: HBase > Issue Type: Task >Reporter: Andrew Purtell >Assignee: Josh Elser >Priority: Minor > Fix For: 1.3.2, 1.4.3, 2.0.0 > > Attachments: HBASE-20088.001.patch > > > NOTICE file, UIs, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20608) Remove build option of error prone profile for branch-1 after HBASE-12350
[ https://issues.apache.org/jira/browse/HBASE-20608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20608: --- Fix Version/s: (was: 1.5.0) > Remove build option of error prone profile for branch-1 after HBASE-12350 > - > > Key: HBASE-20608 > URL: https://issues.apache.org/jira/browse/HBASE-20608 > Project: HBase > Issue Type: Task > Components: build >Affects Versions: 1.4.4, 1.4.5 >Reporter: Tak Lon (Stephen) Wu >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.4.5 > > > After HBASE-12350, error prone profile was introduced/backported to branch-1 > and branch-2. However, branch-1 is still building with JDK 7 and is > incompatible with this error prone profile such that `mvn test-compile` > failed since then. > Open this issue to track the removal of `-PerrorProne` in the build command > (in Jenkins) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20931) [branch-1] Add -Dhttps.protocols=TLSv1.2 to Maven command line in make_rc.sh
[
https://issues.apache.org/jira/browse/HBASE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20931:
---
Fix Version/s: (was: 1.5.0)
> [branch-1] Add -Dhttps.protocols=TLSv1.2 to Maven command line in make_rc.sh
>
>
> Key: HBASE-20931
> URL: https://issues.apache.org/jira/browse/HBASE-20931
> Project: HBase
> Issue Type: Task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Trivial
> Fix For: 1.3.3, 1.4.6, 1.2.7
>
> Attachments: HBASE-20931-branch-1.patch
>
>
> As of June 2018 the insecure TLS 1.0 and 1.1 protocols are no longer
> supported for SSL connections to Maven Central and perhaps other public Maven
> repositories. The branch-1 builds which require Java 7, of which the latest
> public release was 7u80, need to add {{-Dhttps.protocols=TLSv1.2}} to the
> Maven command line in order to avoid artifact retrieval problems during
> builds.
> We especially need this in make_rc.sh which starts up with an empty local
> Maven cache.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21261) Add log4j.properties for hbase-rsgroup tests
[ https://issues.apache.org/jira/browse/HBASE-21261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21261: --- Fix Version/s: (was: 1.5.0) > Add log4j.properties for hbase-rsgroup tests > > > Key: HBASE-21261 > URL: https://issues.apache.org/jira/browse/HBASE-21261 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 3.0.0, 2.2.0, 1.4.8, 2.1.1, 2.0.3 > > > When I tried to debug TestRSGroups, at first I couldn't find any DEBUG log. > Turns out that under hbase-rsgroup/src/test/resources there is no > log4j.properties > This issue adds log4j.properties for hbase-rsgroup tests. > This would be useful when finding root cause for hbase-rsgroup test > failure(s). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-16459) Remove unused hbase shell --format option
[
https://issues.apache.org/jira/browse/HBASE-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-16459:
---
Fix Version/s: (was: 1.5.0)
> Remove unused hbase shell --format option
> -
>
> Key: HBASE-16459
> URL: https://issues.apache.org/jira/browse/HBASE-16459
> Project: HBase
> Issue Type: Task
> Components: shell
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>Priority: Trivial
> Labels: beginner
> Fix For: 1.4.0, 1.3.2, 2.0.0-beta-1, 1.1.13, 2.0.0, 1.2.7
>
> Attachments: HBASE-16459.master.001.patch, HBASE-16459.patch,
> HBASE-16459_v2.patch
>
>
> The usage information when running {{hbase shell}} refers to a formatter
> option that has yet to implemented in over 8 years and which will ostensibly
> never be implemented. As such, let's cleanup the [help
> message|https://github.com/apache/hbase/blob/master/bin/hirb.rb#L57-L59] and
> remove some extraneous lines of code from
> {{[hirb.rb|https://github.com/apache/hbase/blob/master/bin/hirb.rb#L74-L83]}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21076) TestTableResource fails with NPE
[
https://issues.apache.org/jira/browse/HBASE-21076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21076:
---
Fix Version/s: (was: 1.5.0)
> TestTableResource fails with NPE
>
>
> Key: HBASE-21076
> URL: https://issues.apache.org/jira/browse/HBASE-21076
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Affects Versions: 3.0.0, 1.5.0, 1.3.3, 2.1.1, 2.0.2, 1.4.7
>Reporter: Ted Yu
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 3.0.0, 1.3.3, 2.2.0, 1.2.7, 2.1.1, 2.0.2, 1.4.7
>
> Attachments: HBASE-21076.0.patch, HBASE-21076.1.patch
>
>
> The following can be observed in master branch:
> {code}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.rest.TestTableResource.setUpBeforeClass(TestTableResource.java:134)
> {code}
> The NPE comes from the following in TestEndToEndSplitTransaction :
> {code}
> compactAndBlockUntilDone(TEST_UTIL.getAdmin(),
> TEST_UTIL.getMiniHBaseCluster().getRegionServer(0),
> daughterA.getRegionName());
> {code}
> Initial check of the code shows that TestEndToEndSplitTransaction uses
> TEST_UTIL instance which is created within TestEndToEndSplitTransaction.
> However, TestTableResource creates its own instance of HBaseTestingUtility.
> Meaning TEST_UTIL.getMiniHBaseCluster() would return null, since the instance
> created by TestEndToEndSplitTransaction has hbaseCluster as null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20619) TestWeakObjectPool occasionally times out
[ https://issues.apache.org/jira/browse/HBASE-20619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20619: --- Fix Version/s: (was: 1.5.0) > TestWeakObjectPool occasionally times out > - > > Key: HBASE-20619 > URL: https://issues.apache.org/jira/browse/HBASE-20619 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 1.5.0, 1.4.4 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 1.3.3, 1.4.5 > > Attachments: HBASE-20619-branch-1.patch > > > TestWeakObjectPool occasionally times out. Failure is rare and executor is an > EC2 instance, so I think it's just a question of the timeout being too small. > [ERROR] testCongestion(org.apache.hadoop.hbase.util.TestWeakObjectPool) Time > elapsed: 1.049 s <<< ERROR! > org.junit.runners.model.TestTimedOutException: test timed out after 1000 > milliseconds > at > org.apache.hadoop.hbase.util.TestWeakObjectPool.testCongestion(TestWeakObjectPool.java:102) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20646) TestWALProcedureStoreOnHDFS failing on branch-1
[ https://issues.apache.org/jira/browse/HBASE-20646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20646: --- Fix Version/s: (was: 1.5.0) > TestWALProcedureStoreOnHDFS failing on branch-1 > --- > > Key: HBASE-20646 > URL: https://issues.apache.org/jira/browse/HBASE-20646 > Project: HBase > Issue Type: Test >Affects Versions: 1.4.4 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 3.0.0, 2.1.0, 2.0.1, 1.4.5 > > Attachments: HBASE-20646-branch-1.patch > > > TestWALProcedureStoreOnHDFS fails sometimes on branch-1 depending on junit > particulars. An @After decoration was improperly added. Remove to fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19637) Add .checkstyle to gitignore
[ https://issues.apache.org/jira/browse/HBASE-19637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19637: --- Fix Version/s: (was: 1.5.0) > Add .checkstyle to gitignore > > > Key: HBASE-19637 > URL: https://issues.apache.org/jira/browse/HBASE-19637 > Project: HBase > Issue Type: Task > Components: build >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 1.3.2, 1.4.1, 2.0.0-beta-1, 2.0.0, 1.2.7 > > Attachments: HBASE-19637.patch > > > Eclipse will add a .checkstyle file for the checkstyle plugin. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19475) Extend backporting strategy in documentation
[ https://issues.apache.org/jira/browse/HBASE-19475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19475: --- Fix Version/s: (was: 1.5.0) > Extend backporting strategy in documentation > > > Key: HBASE-19475 > URL: https://issues.apache.org/jira/browse/HBASE-19475 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Jan Hentschel >Assignee: Jan Hentschel >Priority: Trivial > Fix For: 3.0.0, 2.1.0, 1.3.3, 2.0.1, 1.4.5, 1.2.7 > > Attachments: HBASE-19475.master.001.patch, > HBASE-19475.master.002.patch, HBASE-19475.master.003.patch > > > Based on the discussion in HBASE-19362 the documentation should be extended > to include an example of when to backport a patch to other branches. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20513) Collect and emit ScanMetrics in PerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-20513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20513: --- Fix Version/s: (was: 1.5.0) > Collect and emit ScanMetrics in PerformanceEvaluation > - > > Key: HBASE-20513 > URL: https://issues.apache.org/jira/browse/HBASE-20513 > Project: HBase > Issue Type: Test > Components: test >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20513-branch-1.patch, HBASE-20513.patch > > > To better understand changes in scanning behavior between version, enable > ScanMetrics collection in PerformanceEvaluation and collect and roll up the > results into a report at termination. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21258) Add resetting of flags for RS Group pre/post hooks in TestRSGroups
[ https://issues.apache.org/jira/browse/HBASE-21258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21258: --- Fix Version/s: (was: 1.5.0) > Add resetting of flags for RS Group pre/post hooks in TestRSGroups > -- > > Key: HBASE-21258 > URL: https://issues.apache.org/jira/browse/HBASE-21258 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Major > Fix For: 3.0.0, 2.2.0, 1.4.8 > > Attachments: 21258.branch-1.04.txt, 21258.branch-1.05.txt, > 21258.branch-2.v1.patch, 21258.v1.txt > > > Over HBASE-20627, [~xucang] reminded me that the resetting of flags for RS > Group pre/post hooks in TestRSGroups was absent. > This issue is to add the resetting of these flags before each subtest starts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21138) Close HRegion instance at the end of every test in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-21138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21138:
---
Fix Version/s: (was: 1.5.0)
> Close HRegion instance at the end of every test in TestHRegion
> --
>
> Key: HBASE-21138
> URL: https://issues.apache.org/jira/browse/HBASE-21138
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mingliang Liu
>Priority: Major
> Fix For: 3.0.0, 1.3.3, 2.2.0, 1.4.8
>
> Attachments: HBASE-21138.000.patch, HBASE-21138.001.patch,
> HBASE-21138.002.patch, HBASE-21138.003.patch, HBASE-21138.004.patch,
> HBASE-21138.branch-1.004.patch, HBASE-21138.branch-1.004.patch,
> HBASE-21138.branch-2.004.patch
>
>
> TestHRegion has over 100 tests.
> The following is from one subtest:
> {code}
> public void testCompactionAffectedByScanners() throws Exception {
> byte[] family = Bytes.toBytes("family");
> this.region = initHRegion(tableName, method, CONF, family);
> {code}
> this.region is not closed at the end of the subtest.
> testToShowNPEOnRegionScannerReseek is another example.
> Every subtest should use the following construct toward the end:
> {code}
> } finally {
> HBaseTestingUtility.closeRegionAndWAL(this.region);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20052) TestRegionOpen#testNonExistentRegionReplica fails due to NPE
[
https://issues.apache.org/jira/browse/HBASE-20052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20052:
---
Fix Version/s: (was: 1.5.0)
> TestRegionOpen#testNonExistentRegionReplica fails due to NPE
>
>
> Key: HBASE-20052
> URL: https://issues.apache.org/jira/browse/HBASE-20052
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Fix For: 1.3.3, 1.4.3, 2.0.0
>
> Attachments: 20052.v1.txt, 20052.v2.txt
>
>
> After HBASE-19391 was integrated, the following test failure can be observed:
> {code}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.TestRegionOpen.testNonExistentRegionReplica(TestRegionOpen.java:122)
> {code}
> This was due null being returned from
> HRegionFileSystem#createRegionOnFileSystem().
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-15580) Tag coprocessor limitedprivate scope to StoreFile.Reader
[
https://issues.apache.org/jira/browse/HBASE-15580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-15580:
---
Fix Version/s: (was: 1.5.0)
> Tag coprocessor limitedprivate scope to StoreFile.Reader
>
>
> Key: HBASE-15580
> URL: https://issues.apache.org/jira/browse/HBASE-15580
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 1.4.1
>
> Attachments: HBASE-15580.patch, HBASE-15580_branch-1.0.patch
>
>
> For phoenix local indexing we need to have custom storefile reader
> constructor(IndexHalfStoreFileReader) to distinguish from other storefile
> readers. So wanted to mark StoreFile.Reader scope as
> InterfaceAudience.LimitedPrivate("Coprocessor")
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21645) Perform sanity check and disallow table creation/modification with region replication < 1
[ https://issues.apache.org/jira/browse/HBASE-21645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21645: --- Fix Version/s: 1.4.10 > Perform sanity check and disallow table creation/modification with region > replication < 1 > - > > Key: HBASE-21645 > URL: https://issues.apache.org/jira/browse/HBASE-21645 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 1.5.0, 2.1.1, 2.1.2 >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.1.3, 2.0.5 > > Attachments: HBASE-21645.master.001.patch, > HBASE-21645.master.001.patch, HBASE-21645.master.002.patch > > > We should perform sanity check and disallow table creation with region > replication < 1 or modification of an existing table with new region > replication value < 1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21511) Remove in progress snapshot check in SnapshotFileCache#getUnreferencedFiles
[ https://issues.apache.org/jira/browse/HBASE-21511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21511: --- Fix Version/s: (was: 1.5.0) > Remove in progress snapshot check in SnapshotFileCache#getUnreferencedFiles > --- > > Key: HBASE-21511 > URL: https://issues.apache.org/jira/browse/HBASE-21511 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Minor > Fix For: 3.0.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2 > > Attachments: 21511.branch-1.v3.txt, 21511.branch-1.v4.txt, > 21511.v1.txt, 21511.v2.txt, 21511.v3.txt > > > During review of HBASE-21387, [~Apache9] mentioned that the check for in > progress snapshots in SnapshotFileCache#getUnreferencedFiles is no longer > needed now that snapshot hfile cleaner and taking snapshot are mutually > exclusive. > This issue is to address the review comment by removing the check for in > progress snapshots. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21373) Backport to branch-1, "HBASE-21338 [balancer] If balancer is an ill-fit for cluster size, it gives little indication"
[ https://issues.apache.org/jira/browse/HBASE-21373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21373: --- Fix Version/s: (was: 1.5.0) > Backport to branch-1, "HBASE-21338 [balancer] If balancer is an ill-fit for > cluster size, it gives little indication" > - > > Key: HBASE-21373 > URL: https://issues.apache.org/jira/browse/HBASE-21373 > Project: HBase > Issue Type: Improvement > Components: Operability >Reporter: stack >Assignee: Xu Cang >Priority: Major > Fix For: 1.3.3, 1.4.9 > > Attachments: HBASE-21373.branch-1.001.patch, > HBASE-21373.branch-1.002.patch > > > Issue to backport to branch-1. Hope you don't mind my assigning it to you Xu > Cang. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21207) Add client side sorting functionality in master web UI for table and region server details.
[ https://issues.apache.org/jira/browse/HBASE-21207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21207: --- Fix Version/s: (was: 1.5.0) > Add client side sorting functionality in master web UI for table and region > server details. > --- > > Key: HBASE-21207 > URL: https://issues.apache.org/jira/browse/HBASE-21207 > Project: HBase > Issue Type: Improvement > Components: master, monitoring, UI, Usability >Reporter: Archana Katiyar >Assignee: Archana Katiyar >Priority: Minor > Fix For: 3.0.0, 2.2.0, 1.4.8 > > Attachments: 14926e82-b929-11e8-8bdd-4ce4621f1118.png, > 21207.branch-1.addendum.patch, 2724afd8-b929-11e8-8171-8b5b2ba3084e.png, > HBASE-21207-branch-1.patch, HBASE-21207-branch-1.v1.patch, > HBASE-21207-branch-2.v1.patch, HBASE-21207.patch, HBASE-21207.patch, > HBASE-21207.v1.patch, edc5c812-b928-11e8-87e2-ce6396629bbc.png > > > In Master UI, we can see region server details like requests per seconds and > number of regions etc. Similarly, for tables also we can see online regions , > offline regions. > It will help ops people in determining hot spotting if we can provide sort > functionality in the UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21263) Mention compression algorithm along with other storefile details
[ https://issues.apache.org/jira/browse/HBASE-21263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21263: --- Fix Version/s: (was: 1.5.0) > Mention compression algorithm along with other storefile details > > > Key: HBASE-21263 > URL: https://issues.apache.org/jira/browse/HBASE-21263 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Subrat Mishra >Priority: Minor > Labels: beginner, beginners > Fix For: 3.0.0, 1.3.3, 2.2.0, 2.1.1, 2.0.3, 1.4.9, 1.2.9 > > Attachments: HBASE-21263-branch-1.patch, > HBASE-21263-branch-2.0.patch, HBASE-21263.master.001.patch, > HBASE-21263.master.002.patch, HBASE-21263.master.003.patch, HBASE-21263.patch > > > Where we log storefile details we should also log the compression algorithm > used to compress blocks on disk, if any. > For example, here's a log line out of compaction: > 2018-10-02 21:59:47,594 DEBUG > [regionserver/host/1.1.1.1:8120-longCompactions-1538517461152] > compactions.Compactor: Compacting > hdfs://namenode:8020/hbase/data/default/TestTable/86037c19117a46b5b8148439ea55753b/i/3d04a7c28d6343ceb773737dbb192533, > keycount=3335873, bloomtype=ROW, size=107.5 M, encoding=ROW_INDEX_V1, > seqNum=154199, earliestPutTs=1538516084915 > Aside from bloom type, block encoding, and filename, it would be good to know > compression type in this type of DEBUG or INFO level logging. A minor > omission of information that could be helpful during debugging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20806) Split style journal for flushes and compactions
[ https://issues.apache.org/jira/browse/HBASE-20806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20806: --- Fix Version/s: (was: 1.5.0) > Split style journal for flushes and compactions > --- > > Key: HBASE-20806 > URL: https://issues.apache.org/jira/browse/HBASE-20806 > Project: HBase > Issue Type: Improvement >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Minor > Fix For: 3.0.0, 2.1.0, 1.3.3, 1.4.6, 1.2.7, 2.0.2 > > Attachments: HBASE-20806.branch-1.001.patch, > HBASE-20806.branch-1.002.patch, HBASE-20806.branch-1.003.patch, > HBASE-20806.branch-2.001.patch, HBASE-20806.master.001.patch, > HBASE-20806.master.002.patch, HBASE-20806.master.003.patch > > > In 1.x we have split transaction journal that gives a clear picture of when > various stages of splits took place. We should have a similar thing for > flushes and compactions so as to have insights into time spent in various > stages, which we can use to identify regressions that might creep up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20935) HStore.removeCompactedFiles should log in case it is unable to delete a file
[ https://issues.apache.org/jira/browse/HBASE-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20935: --- Fix Version/s: (was: 1.5.0) > HStore.removeCompactedFiles should log in case it is unable to delete a file > > > Key: HBASE-20935 > URL: https://issues.apache.org/jira/browse/HBASE-20935 > Project: HBase > Issue Type: Improvement >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Minor > Fix For: 3.0.0, 1.3.3, 2.2.0, 2.1.1, 2.0.2, 1.4.7 > > Attachments: HBASE-20935.branch-1.3.patch, > HBASE-20935.branch-1.3.v2.patch, HBASE-20935.patch, HBASE-20935.v2.patch > > > if (r != null && r.isCompactedAway() && !r.isReferencedInReads()) > If above check fails then there will be some files which are compacted but > not getting cleaned up. It is good to log which helps in debugging the issue. > This would let us know why is getting cleaned. either with reference pending > or compatedaway is not set. > This will help debug issues like : > # HBASE-20933 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20857) JMX - add Balancer status = enabled / disabled
[ https://issues.apache.org/jira/browse/HBASE-20857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20857: --- Fix Version/s: (was: 1.5.0) > JMX - add Balancer status = enabled / disabled > -- > > Key: HBASE-20857 > URL: https://issues.apache.org/jira/browse/HBASE-20857 > Project: HBase > Issue Type: Improvement > Components: API, master, metrics, REST, tooling, Usability >Reporter: Hari Sekhon >Assignee: Kiran Kumar Maturi >Priority: Major > Fix For: 3.0.0, 2.2.0, 1.4.8, 2.1.1 > > Attachments: HBASE-20857.branch-1.4.001.patch, > HBASE-20857.branch-1.4.002.patch > > > Add HBase Balancer enabled/disabled status to JMX API on HMaster. > Right now the HMaster will give a warning near the top of HMaster UI if > balancer is disabled, but scraping this is for monitoring integration is not > nice, it should be available in JMX API as there is already a > Master,sub=Balancer bean with metrics for the balancer ops etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[ https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21098: --- Fix Version/s: (was: 1.5.0) > Improve Snapshot Performance with Temporary Snapshot Directory when rootDir > on S3 > - > > Key: HBASE-21098 > URL: https://issues.apache.org/jira/browse/HBASE-21098 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 1.4.8, 2.1.1 >Reporter: Tyler Mi >Assignee: Tyler Mi >Priority: Major > Labels: s3 > Fix For: 3.0.0, 2.2.0, 1.4.9 > > Attachments: HBASE-21098.branch-1.001.patch, > HBASE-21098.branch-1.002.patch, HBASE-21098.master.001.patch, > HBASE-21098.master.002.patch, HBASE-21098.master.003.patch, > HBASE-21098.master.004.patch, HBASE-21098.master.005.patch, > HBASE-21098.master.006.patch, HBASE-21098.master.007.patch, > HBASE-21098.master.008.patch, HBASE-21098.master.009.patch, > HBASE-21098.master.010.patch, HBASE-21098.master.011.patch, > HBASE-21098.master.012.patch, HBASE-21098.master.013.patch > > > When using Apache HBase, the snapshot feature can be used to make a point in > time recovery. To do this, HBase creates a manifest of all the files in all > of the Regions so that those files can be referenced again when a user > restores a snapshot. With HBase's S3 storage mode, developers can store their > data off-cluster on Amazon S3. However, utilizing S3 as a file system is > inefficient in some operations, namely renames. Most Hadoop ecosystem > applications use an atomic rename as a method of committing data. However, > with S3, a rename is a separate copy and then a delete of every file which is > no longer atomic and, in fact, quite costly. In addition, puts and deletes on > S3 have latency issues that traditional filesystems do not encounter when > manipulating the region snapshots to consolidate into a single manifest. When > HBase on S3 users have a significant amount of regions, puts, deletes, and > renames (the final commit stage of the snapshot) become the bottleneck > causing snapshots to take many minutes or even hours to complete. > The purpose of this patch is to increase the overall performance of snapshots > while utilizing HBase on S3 through the use of a temporary directory for the > snapshots that exists on a traditional filesystem like HDFS to circumvent the > bottlenecks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21103) nightly test cache of yetus install needs to be more thorough in verification
[
https://issues.apache.org/jira/browse/HBASE-21103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21103:
---
Fix Version/s: (was: 1.5.0)
> nightly test cache of yetus install needs to be more thorough in verification
> -
>
> Key: HBASE-21103
> URL: https://issues.apache.org/jira/browse/HBASE-21103
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Labels: beginner
> Fix For: 3.0.0, 1.3.3, 1.2.8, 2.2.0, 2.1.1, 2.0.3, 1.4.9
>
> Attachments: HBASE-21103-branch-1.v0.patch, HBASE-21103.0.patch
>
>
> branch-1.2 nightly failed because it couldn't find yetus:
> https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/443/
> walking through steps, we checked and thought we had an existing install:
> https://builds.apache.org/blue/organizations/jenkins/HBase%20Nightly/detail/branch-1.2/443/pipeline/12
> {code}
> [HBase_Nightly_branch-1.2-AEAO7LJNYKPIH3O4GB2IR4TEIHRS6EXOJCJPGWYXWO6BFPUEZT3Q]
> Running shell script
> Ensure we have a copy of Apache Yetus.
> Checking for Yetus 0.6.0 in
> '/home/jenkins/jenkins-slave/workspace/HBase_Nightly_branch-1.2-AEAO7LJNYKPIH3O4GB2IR4TEIHRS6EXOJCJPGWYXWO6BFPUEZT3Q/yetus-0.6.0'
> Reusing cached download of Apache Yetus version 0.6.0.
> {code}
> So we stashed and then tried to use it.
> Examining the workspace present before the stash shows the directory we look
> for was present, but the contents were garbage:
> https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/443/execution/node/3/ws/
> {code}
> $ unzip -l yetus-0.6.0.zip
> Archive: yetus-0.6.0.zip
> Length DateTimeName
> - -- -
> 0 00-00-1980 04:08 yetus-0.6.0/lib/precommit/qbt.sh
> - ---
> 0 1 file
> {code}
> we should probably check for an executable that will successfully give us a
> version or something like that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20930) MetaScanner.metaScan should use passed variable for meta table name rather than TableName.META_TABLE_NAME
[
https://issues.apache.org/jira/browse/HBASE-20930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20930:
---
Fix Version/s: (was: 1.5.0)
> MetaScanner.metaScan should use passed variable for meta table name rather
> than TableName.META_TABLE_NAME
> -
>
> Key: HBASE-20930
> URL: https://issues.apache.org/jira/browse/HBASE-20930
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.3.3
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>Priority: Minor
> Fix For: 1.3.3, 1.2.7, 1.4.7
>
> Attachments: HBASE-20930.branch-1.3.patch,
> HBASE-20930.branch-1.3.v2.patch
>
>
> MetaScanner.metaScan
> try (Table metaTable = new HTable(TableName.META_TABLE_NAME, connection,
> null)) {
> should be changed to
> metaScan(connection, visitor, userTableName, null, Integer.MAX_VALUE,
> metaTableName)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20826) Truncate responseInfo attributes on RpcServer WARN messages
[
https://issues.apache.org/jira/browse/HBASE-20826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20826:
---
Fix Version/s: (was: 1.5.0)
> Truncate responseInfo attributes on RpcServer WARN messages
> ---
>
> Key: HBASE-20826
> URL: https://issues.apache.org/jira/browse/HBASE-20826
> Project: HBase
> Issue Type: Improvement
> Components: rpc
>Reporter: Sergey Soldatov
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.1.0, 1.3.3, 1.4.6, 1.2.7, 2.0.2
>
> Attachments: HBASE-20826.001.branch-2.0.patch,
> HBASE-20826.002.branch-2.0.patch
>
>
> With Phoenix in the picture, dumping the {{Call}} protobuf to the RS log can
> get *really* chatty, real fast. Notably, some serialized filters just spam
> the log with binary garbage.
> Let's add an upper-limit to the length of params we'll put out at WARN, and
> leave the full content for TRACE.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20837) Make IDE configuration for import order match that in our checkstyle module
[
https://issues.apache.org/jira/browse/HBASE-20837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20837:
---
Fix Version/s: (was: 2.2.0)
(was: 1.5.0)
(was: 3.0.0)
> Make IDE configuration for import order match that in our checkstyle module
> ---
>
> Key: HBASE-20837
> URL: https://issues.apache.org/jira/browse/HBASE-20837
> Project: HBase
> Issue Type: Improvement
> Components: community
>Affects Versions: 3.0.0, 2.0.1, 1.4.5
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
> Attachments: HBASE-20837.branch-1.001.patch,
> HBASE-20837.branch-1.002.patch, HBASE-20837.branch-1.003.patch,
> HBASE-20837.branch-2.001.patch, HBASE-20837.branch-2.002.patch,
> HBASE-20837.branch-2.003.patch, HBASE-20837.master.001.patch,
> HBASE-20837.master.002.patch, HBASE-20837.master.003.patch, IDEA import
> layout.png, hbase-intellij-formatter.xml
>
>
> While working on HBASE-20557 contribution, we figured out that the checkstyle
> build target (ImportOrder's `groups`
> [http://checkstyle.sourceforge.net/config_imports.html] ) was different from
> the development supported IDE (e.g. IntelliJ and Eclipse) formatter, we would
> provide a fix here to sync between
> [dev-support/hbase_eclipse_formatter.xml|https://github.com/apache/hbase/blob/master/dev-support/hbase_eclipse_formatter.xml]
> and
> [hbase/checkstyle.xml|https://github.com/apache/hbase/blob/master/hbase-checkstyle/src/main/resources/hbase/checkstyle.xml]
> This might need to backport the changes of master to branch-1 and branch-2 as
> well.
> Before this change, this is what checkstyle is expecting for import order
>
> {code:java}
> import com.google.common.annotations.VisibleForTesting;
> import java.io.IOException;
> import java.util.ArrayList;
> import java.util.List;
> import java.util.Map;
> import org.apache.commons.logging.Log;
> import org.apache.commons.logging.LogFactory;
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.hbase.classification.InterfaceAudience;
> import org.apache.hadoop.hbase.conf.ConfigurationObserver;{code}
>
> And the proposed import order with the respect to HBASE-19262 and HBASE-19552
> should be
>
> !IDEA import layout.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20651) Master, prevents hbck or shell command to reassign the split parent region
[ https://issues.apache.org/jira/browse/HBASE-20651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20651: --- Fix Version/s: (was: 1.5.0) > Master, prevents hbck or shell command to reassign the split parent region > -- > > Key: HBASE-20651 > URL: https://issues.apache.org/jira/browse/HBASE-20651 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 1.2.6 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Fix For: 1.3.3, 1.4.6, 1.2.7 > > Attachments: HBASE-20651-branch-1-v001.patch, > HBASE-20651-branch-1-v002.patch, HBASE-20651-branch-1-v003.patch > > > We are seeing that hbck brings back split parent region and this causes > region inconsistency. More details will be filled as reproduce is still > ongoing. Might need to do something at hbck or master to prevent this from > happening. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19685) Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
[
https://issues.apache.org/jira/browse/HBASE-19685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19685:
---
Fix Version/s: (was: 1.5.0)
> Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
> -
>
> Key: HBASE-19685
> URL: https://issues.apache.org/jira/browse/HBASE-19685
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 1.3.2, 1.4.1, 2.0.0-beta-2, 2.0.0, 1.2.7
>
> Attachments: HBASE-19685.v0.patch
>
>
> {code}
> java.lang.AssertionError
> at
> org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed.testFullSystemBubblesFSErrors(TestFSErrorsExposed.java:221)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20605) Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission check
[ https://issues.apache.org/jira/browse/HBASE-20605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20605: --- Fix Version/s: (was: 1.5.0) > Exclude new Azure Storage FileSystem from SecureBulkLoadEndpoint permission > check > - > > Key: HBASE-20605 > URL: https://issues.apache.org/jira/browse/HBASE-20605 > Project: HBase > Issue Type: Improvement > Components: security >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 1.3.3, 1.4.5, 1.2.7 > > Attachments: HBASE-20605.001.branch-1.patch, > HBASE-20605.002.branch-1.patch > > > Some folks in Hadoop are working on landing a new FileSystem from the Azure > team: HADOOP-15407 > At present, this FileSystem doesn't support permissions which causes the > SecureBulkLoadEndpoint to balk because it the staging directory doesn't have > the proper 711 permissions. > We have a static list of FileSystem schemes which we ignore this check on. I > have a patch on an HBase 1.1ish which: > # Adds the new FileSystem scheme > # Makes this list configurable for the future -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20701) too much logging when balancer runs from BaseLoadBalancer
[
https://issues.apache.org/jira/browse/HBASE-20701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20701:
---
Fix Version/s: (was: 1.5.0)
> too much logging when balancer runs from BaseLoadBalancer
> -
>
> Key: HBASE-20701
> URL: https://issues.apache.org/jira/browse/HBASE-20701
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Reporter: Monani Mihir
>Assignee: Monani Mihir
>Priority: Trivial
> Fix For: 1.3.3, 1.4.6
>
> Attachments: HBASE-20701-branch-1.3.patch,
> HBASE-20701-branch-1.3.patch, HBASE-20701-branch-1.3.patch,
> HBASE-20701-branch-1.4.patch, HBASE-20701-branch-1.4.patch,
> HBASE-20701.branch-1.001.patch
>
>
> When balancer runs, it tries to find least loaded server with better locality
> for current region. During this, we make debug level logging for each of
> those regions. It creates too much amount of logging at debug level , we
> should move this to trace level logging.
> {code:java}
> int getLeastLoadedTopServerForRegion (int region, int currentServer) {
> ...
> if (leastLoadedServerIndex != -1) {
> LOG.debug("Pick the least loaded server " +
> servers[leastLoadedServerIndex].getHostname()
> + " with better locality for region " + regions[region]);
> }
> ...
> }{code}
> This was fixed in branch-2.0 as part of -HBASE-14614-
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20523) PE tool should support configuring client side buffering sizes
[ https://issues.apache.org/jira/browse/HBASE-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20523: --- Fix Version/s: (was: 1.5.0) > PE tool should support configuring client side buffering sizes > -- > > Key: HBASE-20523 > URL: https://issues.apache.org/jira/browse/HBASE-20523 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.1.0, 2.0.1, 1.4.5, 1.2.7 > > Attachments: HBASE-20523.patch, HBASE-20523_1.patch, > HBASE-20523_branch-1.2.patch, HBASE-20523_branch-1.4.patch, > HBASE-20523_branch-1.patch > > > The client side buffering size impacts the write load and the write > performance. Hence for testing purpose it is better we allow client side > buffering to be configurable in PE. Already YCSB has such facility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20469) Directory used for sidelining old recovered edits files should be made configurable
[
https://issues.apache.org/jira/browse/HBASE-20469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20469:
---
Fix Version/s: (was: 1.5.0)
> Directory used for sidelining old recovered edits files should be made
> configurable
> ---
>
> Key: HBASE-20469
> URL: https://issues.apache.org/jira/browse/HBASE-20469
> Project: HBase
> Issue Type: Improvement
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0, 1.3.3, 2.2.0, 2.1.1, 1.4.7
>
> Attachments: HBASE-20469.master.001.patch
>
>
> Currently the directory used for sidelining of old recovered edit files is
> hardcoded to be "/tmp"
> {code:java}
> Path tmp = new Path("/tmp");
> {code}
> [See L484
> WALSplittter.java|https://github.com/apache/hbase/blob/273d252838e96c4b4af2401743d84e482c4ec565/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WALSplitter.java#L484]
> Instead, we can use some configurable directory in the following manner:
>
> {code:java}
> String tmpDirName = conf.get(HConstants.TEMPORARY_FS_DIRECTORY_KEY,
> HConstants.DEFAULT_TEMPORARY_HDFS_DIRECTORY);
> .
> .
> Path tmp = new Path(tmpDirName);
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20548) Master fails to startup on large clusters, refreshing block distribution
[ https://issues.apache.org/jira/browse/HBASE-20548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20548: --- Fix Version/s: (was: 1.5.0) > Master fails to startup on large clusters, refreshing block distribution > > > Key: HBASE-20548 > URL: https://issues.apache.org/jira/browse/HBASE-20548 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.4 >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.3.3, 2.0.1, 1.4.5 > > Attachments: HBASE-20548.branch-1.4.001.patch, > HBASE-20548.branch-2.0.001.patch, HBASE-20548.master.001.patch > > > On our large clusters with, master has failed to startup within specified > time and aborted itself since it was initializing HDFS block distribution. > Enable table also takes time for larger tables for the same reason. My > proposal is to refresh HDFS block distribution at the end of master > initialization and not at retainAssignment()'s createCluster(). This would > address HBASE-16570's intention, but avoid the problems we ran into. > cc [~aoxiang] [~tedyu] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20733) QABot should run checkstyle tests if the checkstyle configs change
[
https://issues.apache.org/jira/browse/HBASE-20733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20733:
---
Fix Version/s: (was: 1.5.0)
> QABot should run checkstyle tests if the checkstyle configs change
> --
>
> Key: HBASE-20733
> URL: https://issues.apache.org/jira/browse/HBASE-20733
> Project: HBase
> Issue Type: Improvement
> Components: build, community
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.3.3, 1.4.6, 1.2.7, 2.0.2
>
> Attachments: HBASE-20733-HBASE-20733.1.patch, HBASE-20733.0.patch,
> HBASE-20733.1.patch
>
>
> right now we only do checkstyle tests when java files are altered. we should
> also run if our checkstyle configs in {{hbase-checkstyle}} are altered.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21173) Remove the duplicate HRegion#close in TestHRegion
[ https://issues.apache.org/jira/browse/HBASE-21173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21173: --- Fix Version/s: (was: 1.5.0) > Remove the duplicate HRegion#close in TestHRegion > - > > Key: HBASE-21173 > URL: https://issues.apache.org/jira/browse/HBASE-21173 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.2.0 >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Minor > Fix For: 3.0.0, 1.3.3, 2.2.0, 1.4.8 > > Attachments: HBASE-21173.branch-1.001.patch, > HBASE-21173.master.001.patch, HBASE-21173.master.002.patch > > > After HBASE-21138, some test methods still have the duplicate > HRegion#close.So open this issue to remove the duplicate close -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20387) flaky infrastructure should work for all branches
[ https://issues.apache.org/jira/browse/HBASE-20387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20387: --- Fix Version/s: (was: 1.5.0) > flaky infrastructure should work for all branches > - > > Key: HBASE-20387 > URL: https://issues.apache.org/jira/browse/HBASE-20387 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.3.3, 2.2.0, 1.2.7, 2.1.1, 2.0.2, 1.4.7 > > Attachments: HBASE-20387.0.patch, HBASE-20387.1.patch > > > We need a flaky list per-branch, since what does/does not work reliably on > master isn't really relevant to our older maintenance release lines. > We should just make the invocation a step in the current per-branch nightly > jobs, prior to when we need the list in the stages that run unit tests. We > can publish it in the nightly job as well so that precommit can still get it. > (and can fetch it per-branch if needed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20186) Improve RSGroupBasedLoadBalancer#balanceCluster() to be more efficient when calculating cluster state for each rsgroup
[
https://issues.apache.org/jira/browse/HBASE-20186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20186:
---
Fix Version/s: (was: 1.5.0)
> Improve RSGroupBasedLoadBalancer#balanceCluster() to be more efficient when
> calculating cluster state for each rsgroup
> --
>
> Key: HBASE-20186
> URL: https://issues.apache.org/jira/browse/HBASE-20186
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Fix For: 3.0.0, 2.1.0, 1.4.3
>
> Attachments: HBASE-20186.master.000.patch,
> HBASE-20186.master.001.patch
>
>
> In RSGroupBasedLoadBalancer
> {code}
> public List balanceCluster(Map>
> clusterState)
> {code}
> The second half of the function is to calculate region move plan for regions
> which have been already placed according to the rsgroup assignment, and it is
> calculated one rsgroup after another.
> The following logic to check if a server belongs to the rsgroup is not quite
> efficient, as it does not make good use of the fact that servers in
> RSGroupInfo is a TreeSet.
> {code}
> for (Address sName : info.getServers()) {
> for(ServerName curr: clusterState.keySet()) {
> if(curr.getAddress().equals(sName)) {
> groupClusterState.put(curr, correctedState.get(curr));
> }
> }
> }
> {code}
> Given there are m region servers in the cluster and n region servers for each
> rsgroup in average, the code above has time complexity as O(m * n), while
> using TreeSet's contains(), the time complexity could be reduced to O (m *
> logn).
> Another improvement is we do not need to scan every server for each rsgroup.
> If the processed server could be recorded, we could skip those.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20379) shadedjars yetus plugin should add a footer link
[ https://issues.apache.org/jira/browse/HBASE-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20379: --- Fix Version/s: (was: 1.5.0) > shadedjars yetus plugin should add a footer link > > > Key: HBASE-20379 > URL: https://issues.apache.org/jira/browse/HBASE-20379 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0, 2.1.0, 1.3.3, 1.4.4, 2.0.1, 1.2.7 > > Attachments: HBASE-20379.0.patch > > > investigating the failure on HBASE-20219, it would be nice if we posted a > footer link to what failed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20401) Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable
[ https://issues.apache.org/jira/browse/HBASE-20401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20401: --- Fix Version/s: (was: 1.5.0) > Make `MAX_WAIT` and `waitIfNotFinished` in CleanerContext configurable > -- > > Key: HBASE-20401 > URL: https://issues.apache.org/jira/browse/HBASE-20401 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 3.0.0, 1.5.0, 2.0.0-beta-1, 1.4.4, 2.0.0 >Reporter: Tak Lon (Stephen) Wu >Assignee: Tak Lon (Stephen) Wu >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 1.3.3, 1.4.6, 2.2.0, 2.1.1, 2.0.2 > > Attachments: HBASE-20401.branch-1.001.patch, > HBASE-20401.branch-1.002.patch, HBASE-20401.branch-1.003.patch, > HBASE-20401.master.001.patch, HBASE-20401.master.002.patch, > HBASE-20401.master.003.patch, HBASE-20401.master.004.patch, > HBASE-20401.master.005.patch, HBASE-20401.master.006.patch, > HBASE-20401.master.007.patch > > > When backporting HBASE-18309 in HBASE-20352, the deleteFiles calls > CleanerContext.java#getResult with a waitIfNotFinished timeout to wait for > notification (notify) from the fs.delete file thread. there might be two > situation need to tune the MAX_WAIT in CleanerContext or waitIfNotFinished > when LogClearner call getResult. > # fs.delete never complete (strange but possible), then we need to wait for > a max of 60 seconds. here, 60 seconds might be too long > # getResult is waiting in the period of 500 milliseconds, but the fs.delete > has completed and setFromClear is set but yet notify(). one might want to > tune this 500 milliseconds to 200 or less . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20352) [Chore] Backport HBASE-18309 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-20352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20352: --- Fix Version/s: (was: 1.5.0) > [Chore] Backport HBASE-18309 to branch-1 > > > Key: HBASE-20352 > URL: https://issues.apache.org/jira/browse/HBASE-20352 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Major > Fix For: 1.3.3, 1.4.4 > > Attachments: HBASE-20352.branch-1.001.patch > > > Using multiple threads to scan directory and to clean old WALs -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21505) Several inconsistencies on information reported for Replication Sources by hbase shell status 'replication' command.
[
https://issues.apache.org/jira/browse/HBASE-21505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758642#comment-16758642
]
Wellington Chevreuil commented on HBASE-21505:
--
Thanks for the review [~tianjingyun]. Had generated a new patch applying your
suggestions. Since I can't comment on RB, please find my comments on your
suggestions:
{quote}use computeIfAbsent()?
{quote}
Done, should be in the new patch.
{quote}too many parameters, add a builder for this class?
{quote}
Added builder for this in the new patch.
{quote}format
{quote}
Done, should be in the new patch.
{quote}format
{quote}
Done, should be in the new patch.
{quote}If you set these parameter as required, which means we can't rolling
upgrade this cluster?
{quote}
I haven't thought on breaking compatibility before, was more concerned with
enforce this parameters to be in the message. I guess this can be relaxed, so
made those optional on the new patch.
{quote}add to hbase-protocol-shaded should be enough
{quote}
Removed extra attributes from hbase-protocol on the new patch.
{quote}Why not update metrics here?
{quote}
This was one of the issues addressed here, that point was update
AgeOfLastShippedOp even if the Op was not successfully processed on the remote
peer. On ReplicationSourceShipper.shipEdits() method, we have a central point
to define when a specific source thread has successfully shipped the edits, so
this and other edits shipment related metrics are being updated there.
{quote}Is package private enough?
{quote}
Made it "protected' in the new patch.
Let me know on any other concerns/suggestions you might have.
> Several inconsistencies on information reported for Replication Sources by
> hbase shell status 'replication' command.
>
>
> Key: HBASE-21505
> URL: https://issues.apache.org/jira/browse/HBASE-21505
> Project: HBase
> Issue Type: Bug
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Attachments:
> 0001-HBASE-21505-initial-version-for-more-detailed-report.patch,
> HBASE-21505-master.001.patch, HBASE-21505-master.002.patch,
> HBASE-21505-master.003.patch, HBASE-21505-master.004.patch,
> HBASE-21505-master.005.patch, HBASE-21505-master.006.patch
>
>
> While reviewing hbase shell status 'replication' command, noticed the
> following issues related to replication source section:
> 1) TimeStampsOfLastShippedOp keeps getting updated and increasing even when
> no new edits were added to source, so nothing was really shipped. Test steps
> performed:
> 1.1) Source cluster with only one table targeted to replication;
> 1.2) Added a new row, confirmed the row appeared in Target cluster;
> 1.3) Issued status 'replication' command in source, TimeStampsOfLastShippedOp
> shows current timestamp T1.
> 1.4) Waited 30 seconds, no new data added to source. Issued status
> 'replication' command, now shows timestamp T2.
> 2) When replication is stuck due some connectivity issues or target
> unavailability, if new edits are added in source, reported AgeOfLastShippedOp
> is wrongly showing same value as "Replication Lag". This is incorrect,
> AgeOfLastShippedOp should not change until there's indeed another edit
> shipped to target. Test steps performed:
> 2.1) Source cluster with only one table targeted to replication;
> 2.2) Stopped target cluster RS;
> 2.3) Put a new row on source. Running status 'replication' command does show
> lag increasing. TimeStampsOfLastShippedOp seems correct also, no further
> updates as described on bullet #1 above.
> 2.4) AgeOfLastShippedOp keeps increasing together with Replication Lag, even
> though there's no new edit shipped to target:
> {noformat}
> ...
> SOURCE: PeerID=1, AgeOfLastShippedOp=5581, SizeOfLogQueue=1,
> TimeStampsOfLastShippedOp=Wed Nov 21 02:50:23 GMT 2018, Replication Lag=5581
> ...
> ...
> SOURCE: PeerID=1, AgeOfLastShippedOp=8586, SizeOfLogQueue=1,
> TimeStampsOfLastShippedOp=Wed Nov 21 02:50:23 GMT 2018, Replication Lag=8586
> ...
> {noformat}
> 3) AgeOfLastShippedOp gets set to 0 even when a given edit had taken some
> time before it got finally shipped to target. Test steps performed:
> 3.1) Source cluster with only one table targeted to replication;
> 3.2) Stopped target cluster RS;
> 3.3) Put a new row on source.
> 3.4) AgeOfLastShippedOp keeps increasing together with Replication Lag, even
> though there's no new edit shipped to target:
> {noformat}
> T1:
> ...
> SOURCE: PeerID=1, AgeOfLastShippedOp=5581, SizeOfLogQueue=1,
> TimeStampsOfLastShippedOp=Wed Nov 21 02:50:23 GMT 2018, Replication Lag=5581
> ...
> T2:
> ...
> SOURCE: PeerID=1, AgeOfLastShippedOp=8586, SizeOfLogQueue=1,
> TimeStampsOfLastShippedOp=Wed Nov 21 02:50:23
[jira] [Updated] (HBASE-19570) Add hadoop3 tests to Nightly master/branch-2 runs
[ https://issues.apache.org/jira/browse/HBASE-19570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19570: --- Fix Version/s: (was: 1.5.0) > Add hadoop3 tests to Nightly master/branch-2 runs > - > > Key: HBASE-19570 > URL: https://issues.apache.org/jira/browse/HBASE-19570 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Critical > Labels: infrastructure > Fix For: 1.3.2, 1.4.1, 2.0.0-beta-1, 2.0.0, 1.2.7 > > Attachments: HBASE-19570.master.001.patch, > HBASE-19570.master.002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20307: --- Fix Version/s: (was: 1.5.0) > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Improvement > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 1.3.3, 1.2.8, 2.2.0, 1.4.8, 2.1.1, 2.0.3 > > Attachments: HBASE-20307.000.patch, HBASE-20307.001.patch > > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19491) Exclude flaky tests from nightly master run
[ https://issues.apache.org/jira/browse/HBASE-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-19491: --- Fix Version/s: (was: 1.5.0) > Exclude flaky tests from nightly master run > --- > > Key: HBASE-19491 > URL: https://issues.apache.org/jira/browse/HBASE-19491 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Major > Labels: infrastructure > Fix For: 1.3.2, 1.4.1, 2.0.0-beta-1, 2.0.0, 1.2.7 > > Attachments: HBASE-19491.master.001.patch, > HBASE-19491.master.002.patch, > test-patch-to-run-precommit-on-one-module.master.002.patch, > test-patch-to-run-precommit-on-one-module.master.patch > > > Testing infra improvements > - Exclude flaky tests from nightly master run (Old nightly master run used to > exclude flaky tests, but new nightly one which is based on Jenkins stages > wasn't using it. Adding it to new nightly job) > - Fixes findbugs check (seems like wasn't working earlier : "0 > findbugs0m 0s Findbugs executables are not available.") -- This message was sent by Atlassian JIRA (v7.6.3#76005)
