[jira] [Updated] (HBASE-21710) Add quota related methods to the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21710: -- Summary: Add quota related methods to the Admin interface (was: Add quota related method to the Admin interface) > Add quota related methods to the Admin interface > > > Key: HBASE-21710 > URL: https://issues.apache.org/jira/browse/HBASE-21710 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Priority: Major > > So we can limit the ClusterConnection usage inside the sync client > implementation. Now we will use ClusterConnection in QuotaTableUtil, which > will be a pain when we want to replace the implement of the old sync client > with the new async client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21710) Add quota related method to the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21710: -- Description: So we can limit the ClusterConnection usage inside the sync client implementation. Now we will use ClusterConnection in QuotaTableUtil, which will be a pain when we want to replace the implement of the old sync client with the new async client. > Add quota related method to the Admin interface > --- > > Key: HBASE-21710 > URL: https://issues.apache.org/jira/browse/HBASE-21710 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Priority: Major > > So we can limit the ClusterConnection usage inside the sync client > implementation. Now we will use ClusterConnection in QuotaTableUtil, which > will be a pain when we want to replace the implement of the old sync client > with the new async client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21710) Add quota related method to the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-21710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21710: -- Summary: Add quota related method to the Admin interface (was: Move QuotaTableUtil.getCurrentSnapshot to hbase-server module as they are only used in table.jsp) > Add quota related method to the Admin interface > --- > > Key: HBASE-21710 > URL: https://issues.apache.org/jira/browse/HBASE-21710 > Project: HBase > Issue Type: Task >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21710) Move QuotaTableUtil.getCurrentSnapshot to hbase-server module as they are only used in table.jsp
Duo Zhang created HBASE-21710: - Summary: Move QuotaTableUtil.getCurrentSnapshot to hbase-server module as they are only used in table.jsp Key: HBASE-21710 URL: https://issues.apache.org/jira/browse/HBASE-21710 Project: HBase Issue Type: Task Reporter: Duo Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19695) Handle disabled table for async client
[ https://issues.apache.org/jira/browse/HBASE-19695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19695: -- Attachment: HBASE-19695-v2.patch > Handle disabled table for async client > -- > > Key: HBASE-19695 > URL: https://issues.apache.org/jira/browse/HBASE-19695 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-19695-v1.patch, HBASE-19695-v2.patch, > HBASE-19695.master.001.patch, HBASE-19695.master.002.patch, HBASE-19695.patch > > > Now for async client we will not check if a table is disabled when retrying, > so we will retry until the time or count limit is reached, and will not throw > a TableNotEnabledException. > We should have the same behavior as sync client, but the implementation > should be carefully designed. For sync client, we will also check if a table > is disabled if it is a retry, no matter what the exception is. This will > double the pressure on meta table. We should try our best to eliminate > unnecessary access to meta table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21596) HBase Shell "delete" command can cause older versions to be shown even if VERSIONS is configured as 1
[ https://issues.apache.org/jira/browse/HBASE-21596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741471#comment-16741471 ] Wellington Chevreuil commented on HBASE-21596: -- Thanks for pointing out HBASE-18142, [~busbey]. This is a different problem, though. HBASE-18142 only changed behaviour of hbase shell delete command. Prior to HBASE-18142, specifying a timestamp to delete command would cause all versions older than this timestamp to be deleted. For example, if a cell had 3 versions, say T1, T2 and T3, then triggering a delete specifying timestamp T1, would all cause all 3 versions to be deleted. If T2 was specified, then T2 and T3 versions would be deleted. Now, only the specified version is actually deleted. This jira (HBASE-21596) deals with a different problem noticed on current master branch. Problem here is that if a given cell had extrapolated its CF max versions, then when one of these versions deleted while all versions are still in memstore, the extra value is shown again by get/deletes. For example, say a CF whose VERSIONS => 3. Adding versions T1, T2, T3 and T4 for a given cell, a scan/get with VERSIONS => 10, for example will show T4, T3 and T2, which is correct. If one of this versions is deleted, say T3, then same scan/get is ran, results show now are T4, T2 and T1, which is not correct, as T1 should had already been gone when T4 was inserted. This behaviour is inconsistent as well, because if memstore had been flushed prior to the delete, the scan/get result would correctly return only T4 and T2, as the extra cell was filtered out while flushing. > HBase Shell "delete" command can cause older versions to be shown even if > VERSIONS is configured as 1 > - > > Key: HBASE-21596 > URL: https://issues.apache.org/jira/browse/HBASE-21596 > Project: HBase > Issue Type: Bug >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Minor > Attachments: HBASE-21596-master.001.patch, initial-patch.txt > > > HBase Shell delete command is supposed to operate over an specific TS. If no > TS is informed, it will assume the latest TS for the cell and put delete > marker for it. > However, for a CF with VERSIONS => 1, if multiple puts were performed for > same cell, there may be multiple cell versions on the memstore, so delete > would only be "delete marking" one of those, and causing the most recent no > marked one to be shown on gets/scans, which then contradicts the CF "VERSIONS > => 1" configuration. > This issue is not seen with deleteall command or using Delete operation from > Java API. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21519) Namespace region is never assigned in a HM failover scenario when multiwal is enabled
[
https://issues.apache.org/jira/browse/HBASE-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741470#comment-16741470
]
Hadoop QA commented on HBASE-21519:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-21519 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.8.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-21519 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15554/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Namespace region is never assigned in a HM failover scenario when multiwal is
> enabled
> -
>
> Key: HBASE-21519
> URL: https://issues.apache.org/jira/browse/HBASE-21519
> Project: HBase
> Issue Type: Bug
> Components: master, wal
>Affects Versions: 2.1.1
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.2.0
>
> Attachments: HBASE-21519.branch-2.patch, providerId_value.png
>
>
> In our test env we found that namespace region is never be assigned on HM
> failover scenario when multiwal feature is enabled,
> {noformat}
> 2018-11-28 01:38:28,085 WARN [master/HM-1:16000:becomeActiveMaster]
> master.HMaster:
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15. is NOT
> online; state=\{31f6d3383af09e18e1e81ca02a93de15 state=OPEN,
> ts=1543340156928, server=RS-2,16020,1543339824397};
> ServerCrashProcedures=false. Master startup cannot progress, in
> holding-pattern until region onlined.
> {noformat}
> And finally HM abort with following error,
> {noformat}
> 2018-11-28 01:39:16,858 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Master
> failed to complete initialization after 24ms. Please consider submitting
> a bug report including a thread dump of this process.
> 2018-11-28 01:39:18,980 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Zombie
> Master exiting. Thread dump to stdout
> {noformat}
> Stack trace:
> {noformat}
> Thread 102 (master/HM-1:16000:becomeActiveMaster):
> State: TIMED_WAITING
> Blocked count: 100
> Waited count: 246
> Stack:
> java.lang.Thread.sleep(Native Method)
> org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:148)
> org.apache.hadoop.hbase.master.HMaster.isRegionOnline(HMaster.java:1166)
>
> org.apache.hadoop.hbase.master.HMaster.waitForNamespaceOnline(HMaster.java:1187)
>
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1044)
>
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2285)
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:590)
> org.apache.hadoop.hbase.master.HMaster$$Lambda$40/1078246575.run(Unknown
> Source)
> java.lang.Thread.run(Thread.java:745)
> {noformat}
>
> Step to reproduce:
> 1) Setup a HBase cluster with 1/2 HM (say HM-1) and 2 RS(say RS-1 & RS-2)
> 2) Enable multiwal feature with following configuration setting and start
> the cluster,
> {noformat}
>
> hbase.wal.provider
> multiwal
>
>
> hbase.wal.regiongrouping.strategy
> identity
>
> {noformat}
> 3) Make sure meta and namespace regions are assigned on different RS, suppose
> RS-1 & RS-2 respectively.
> 4) Create table 't1'
> 5) Flush the meta table explicitly
> 6) Kill the RS-2, so during RS-2 SCP all regions including namespace region
> will be assigned to RS-1.
> 7) Now Kill RS-1 before meta flush happen. Here both RS-2 & RS-1 are
> shutdown now.
> 8) Stop the HM and start RS-1 & RS-2.
> 9) Now start the HM.
> Meta region is assigned successfully but HM is keep waiting for the namespace
> region onlline (Master startup cannot progress, in holding-pattern until
> region onlined) and abort with timeout.
> Observation:
> 1) After step-3 namespace region was assigned to RS-2 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> 2) After step-6 namespace region was assigned to RS-1 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server,
[jira] [Commented] (HBASE-21519) Namespace region is never assigned in a HM failover scenario when multiwal is enabled
[
https://issues.apache.org/jira/browse/HBASE-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741469#comment-16741469
]
Pankaj Kumar commented on HBASE-21519:
--
Yeah, it is .meta, have attached the snapshot, kindly check that.
> Namespace region is never assigned in a HM failover scenario when multiwal is
> enabled
> -
>
> Key: HBASE-21519
> URL: https://issues.apache.org/jira/browse/HBASE-21519
> Project: HBase
> Issue Type: Bug
> Components: master, wal
>Affects Versions: 2.1.1
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.2.0
>
> Attachments: HBASE-21519.branch-2.patch, providerId_value.png
>
>
> In our test env we found that namespace region is never be assigned on HM
> failover scenario when multiwal feature is enabled,
> {noformat}
> 2018-11-28 01:38:28,085 WARN [master/HM-1:16000:becomeActiveMaster]
> master.HMaster:
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15. is NOT
> online; state=\{31f6d3383af09e18e1e81ca02a93de15 state=OPEN,
> ts=1543340156928, server=RS-2,16020,1543339824397};
> ServerCrashProcedures=false. Master startup cannot progress, in
> holding-pattern until region onlined.
> {noformat}
> And finally HM abort with following error,
> {noformat}
> 2018-11-28 01:39:16,858 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Master
> failed to complete initialization after 24ms. Please consider submitting
> a bug report including a thread dump of this process.
> 2018-11-28 01:39:18,980 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Zombie
> Master exiting. Thread dump to stdout
> {noformat}
> Stack trace:
> {noformat}
> Thread 102 (master/HM-1:16000:becomeActiveMaster):
> State: TIMED_WAITING
> Blocked count: 100
> Waited count: 246
> Stack:
> java.lang.Thread.sleep(Native Method)
> org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:148)
> org.apache.hadoop.hbase.master.HMaster.isRegionOnline(HMaster.java:1166)
>
> org.apache.hadoop.hbase.master.HMaster.waitForNamespaceOnline(HMaster.java:1187)
>
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1044)
>
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2285)
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:590)
> org.apache.hadoop.hbase.master.HMaster$$Lambda$40/1078246575.run(Unknown
> Source)
> java.lang.Thread.run(Thread.java:745)
> {noformat}
>
> Step to reproduce:
> 1) Setup a HBase cluster with 1/2 HM (say HM-1) and 2 RS(say RS-1 & RS-2)
> 2) Enable multiwal feature with following configuration setting and start
> the cluster,
> {noformat}
>
> hbase.wal.provider
> multiwal
>
>
> hbase.wal.regiongrouping.strategy
> identity
>
> {noformat}
> 3) Make sure meta and namespace regions are assigned on different RS, suppose
> RS-1 & RS-2 respectively.
> 4) Create table 't1'
> 5) Flush the meta table explicitly
> 6) Kill the RS-2, so during RS-2 SCP all regions including namespace region
> will be assigned to RS-1.
> 7) Now Kill RS-1 before meta flush happen. Here both RS-2 & RS-1 are
> shutdown now.
> 8) Stop the HM and start RS-1 & RS-2.
> 9) Now start the HM.
> Meta region is assigned successfully but HM is keep waiting for the namespace
> region onlline (Master startup cannot progress, in holding-pattern until
> region onlined) and abort with timeout.
> Observation:
> 1) After step-3 namespace region was assigned to RS-2 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> 2) After step-6 namespace region was assigned to RS-1 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339880920, value=RS-1:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339880920, value=1543339829288
> {noformat}
> 3) After Step-9, meta entry for namespace region was as follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> During SCP we do meta log split based on filter,
> {noformat}
> /**
> * Specialized method to handle
[jira] [Updated] (HBASE-21519) Namespace region is never assigned in a HM failover scenario when multiwal is enabled
[
https://issues.apache.org/jira/browse/HBASE-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-21519:
-
Attachment: providerId_value.png
> Namespace region is never assigned in a HM failover scenario when multiwal is
> enabled
> -
>
> Key: HBASE-21519
> URL: https://issues.apache.org/jira/browse/HBASE-21519
> Project: HBase
> Issue Type: Bug
> Components: master, wal
>Affects Versions: 2.1.1
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.2.0
>
> Attachments: HBASE-21519.branch-2.patch, providerId_value.png
>
>
> In our test env we found that namespace region is never be assigned on HM
> failover scenario when multiwal feature is enabled,
> {noformat}
> 2018-11-28 01:38:28,085 WARN [master/HM-1:16000:becomeActiveMaster]
> master.HMaster:
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15. is NOT
> online; state=\{31f6d3383af09e18e1e81ca02a93de15 state=OPEN,
> ts=1543340156928, server=RS-2,16020,1543339824397};
> ServerCrashProcedures=false. Master startup cannot progress, in
> holding-pattern until region onlined.
> {noformat}
> And finally HM abort with following error,
> {noformat}
> 2018-11-28 01:39:16,858 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Master
> failed to complete initialization after 24ms. Please consider submitting
> a bug report including a thread dump of this process.
> 2018-11-28 01:39:18,980 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Zombie
> Master exiting. Thread dump to stdout
> {noformat}
> Stack trace:
> {noformat}
> Thread 102 (master/HM-1:16000:becomeActiveMaster):
> State: TIMED_WAITING
> Blocked count: 100
> Waited count: 246
> Stack:
> java.lang.Thread.sleep(Native Method)
> org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:148)
> org.apache.hadoop.hbase.master.HMaster.isRegionOnline(HMaster.java:1166)
>
> org.apache.hadoop.hbase.master.HMaster.waitForNamespaceOnline(HMaster.java:1187)
>
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1044)
>
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2285)
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:590)
> org.apache.hadoop.hbase.master.HMaster$$Lambda$40/1078246575.run(Unknown
> Source)
> java.lang.Thread.run(Thread.java:745)
> {noformat}
>
> Step to reproduce:
> 1) Setup a HBase cluster with 1/2 HM (say HM-1) and 2 RS(say RS-1 & RS-2)
> 2) Enable multiwal feature with following configuration setting and start
> the cluster,
> {noformat}
>
> hbase.wal.provider
> multiwal
>
>
> hbase.wal.regiongrouping.strategy
> identity
>
> {noformat}
> 3) Make sure meta and namespace regions are assigned on different RS, suppose
> RS-1 & RS-2 respectively.
> 4) Create table 't1'
> 5) Flush the meta table explicitly
> 6) Kill the RS-2, so during RS-2 SCP all regions including namespace region
> will be assigned to RS-1.
> 7) Now Kill RS-1 before meta flush happen. Here both RS-2 & RS-1 are
> shutdown now.
> 8) Stop the HM and start RS-1 & RS-2.
> 9) Now start the HM.
> Meta region is assigned successfully but HM is keep waiting for the namespace
> region onlline (Master startup cannot progress, in holding-pattern until
> region onlined) and abort with timeout.
> Observation:
> 1) After step-3 namespace region was assigned to RS-2 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> 2) After step-6 namespace region was assigned to RS-1 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339880920, value=RS-1:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339880920, value=1543339829288
> {noformat}
> 3) After Step-9, meta entry for namespace region was as follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> During SCP we do meta log split based on filter,
> {noformat}
> /**
> * Specialized method to handle the splitting for meta WAL
> * @param serverNames logs belonging to
[jira] [Updated] (HBASE-21705) Should treat meta table specially for some methods in AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-21705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21705: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-2.0+. Thanks [~zghaobac] for reviewing. > Should treat meta table specially for some methods in AsyncAdmin > > > Key: HBASE-21705 > URL: https://issues.apache.org/jira/browse/HBASE-21705 > Project: HBase > Issue Type: Sub-task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21705-v1.patch, HBASE-21705-v2.patch, > HBASE-21705.patch > > > For example, tableExists, isTableEnabled, isTableDisabled... > For now, we will go to the meta table directly but obviously, meta table does > not contain the record for itself... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21685) Change repository urls to Gitbox
[
https://issues.apache.org/jira/browse/HBASE-21685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741463#comment-16741463
]
Duo Zhang commented on HBASE-21685:
---
Oh, seems the HBase-Flaky-Tests job is also broken... [~busbey]
{noformat}
stderr: fatal: repository 'https://git.apache.org/hbase.git/' not found
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2038)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1757)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:72)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:438)
at
jenkins.plugins.git.AbstractGitSCMSource.doRetrieve(AbstractGitSCMSource.java:351)
at
jenkins.plugins.git.AbstractGitSCMSource.retrieve(AbstractGitSCMSource.java:374)
at jenkins.scm.api.SCMSource.fetch(SCMSource.java:583)
at
org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:95)
at
org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:293)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
{noformat}
> Change repository urls to Gitbox
>
>
> Key: HBASE-21685
> URL: https://issues.apache.org/jira/browse/HBASE-21685
> Project: HBase
> Issue Type: Task
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-21685.master.001.patch,
> HBASE-21685.master.001.patch
>
>
> Moving to Gitbox is approaching and references to git-wip-us need to be
> changed to gitbox.
> Some of the Jenkins jobs are referring to git-wip-us which if going to be
> locked after the migration. We could move them to github so the build flow
> will remain intact.
> Previous discussion on dev@:
> [https://lists.apache.org/thread.html/3496568d6cc002f74f5c3bcce46ed44b7ee9e90d7d53af2c65b6f785@%3Cdev.hbase.apache.org%3E]
> After this notify INFRA to make the change
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21707) Fix warnings in hbase-rsgroup module and also make the UTs more stable
[ https://issues.apache.org/jira/browse/HBASE-21707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741462#comment-16741462 ] Duo Zhang commented on HBASE-21707: --- OK, good. All green. > Fix warnings in hbase-rsgroup module and also make the UTs more stable > -- > > Key: HBASE-21707 > URL: https://issues.apache.org/jira/browse/HBASE-21707 > Project: HBase > Issue Type: Bug > Components: Region Assignment, rsgroup >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21707-UT.patch, HBASE-21707-v1.patch, > HBASE-21707.patch > > > By adding a sleep after the stopServer call, the test will hang there > forever, need to dig more, maybe something wrong with the TRSP related stuffs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21707) Fix warnings in hbase-rsgroup module and also make the UTs more stable
[
https://issues.apache.org/jira/browse/HBASE-21707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741461#comment-16741461
]
Hadoop QA commented on HBASE-21707:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} hbase-rsgroup: The patch generated 0 new + 2
unchanged - 5 fixed = 2 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 38s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
6s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21707 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954749/HBASE-21707-v1.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 09003a92613a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3d2580cd6d |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15553/testReport/ |
| Max. process+thread count | 4017 (vs. ulimit of 1) |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15553/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This
[jira] [Commented] (HBASE-21537) Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection
[ https://issues.apache.org/jira/browse/HBASE-21537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741459#comment-16741459 ] Duo Zhang commented on HBASE-21537: --- Will commit later if no objections. Thanks. > Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection > -- > > Key: HBASE-21537 > URL: https://issues.apache.org/jira/browse/HBASE-21537 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21537-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21707) Fix warnings in hbase-rsgroup module and also make the UTs more stable
[ https://issues.apache.org/jira/browse/HBASE-21707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741458#comment-16741458 ] Duo Zhang commented on HBASE-21707: --- Review board link: https://reviews.apache.org/r/69734/ > Fix warnings in hbase-rsgroup module and also make the UTs more stable > -- > > Key: HBASE-21707 > URL: https://issues.apache.org/jira/browse/HBASE-21707 > Project: HBase > Issue Type: Bug > Components: Region Assignment, rsgroup >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21707-UT.patch, HBASE-21707-v1.patch, > HBASE-21707.patch > > > By adding a sleep after the stopServer call, the test will hang there > forever, need to dig more, maybe something wrong with the TRSP related stuffs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21708) ReviewBoard integration is busted post-gitbox move
[ https://issues.apache.org/jira/browse/HBASE-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741457#comment-16741457 ] Duo Zhang commented on HBASE-21708: --- Thanks [~busbey]. The RB is OK now. > ReviewBoard integration is busted post-gitbox move > -- > > Key: HBASE-21708 > URL: https://issues.apache.org/jira/browse/HBASE-21708 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > > per comment from [~Apache9] on HBASE-21685 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21707) Fix warnings in hbase-rsgroup module and also make the UTs more stable
[ https://issues.apache.org/jira/browse/HBASE-21707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21707: -- Attachment: HBASE-21707-v1.patch > Fix warnings in hbase-rsgroup module and also make the UTs more stable > -- > > Key: HBASE-21707 > URL: https://issues.apache.org/jira/browse/HBASE-21707 > Project: HBase > Issue Type: Bug > Components: Region Assignment, rsgroup >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21707-UT.patch, HBASE-21707-v1.patch, > HBASE-21707.patch > > > By adding a sleep after the stopServer call, the test will hang there > forever, need to dig more, maybe something wrong with the TRSP related stuffs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21537) Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection
[ https://issues.apache.org/jira/browse/HBASE-21537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741452#comment-16741452 ] Duo Zhang commented on HBASE-21537: --- The final goal is to completely remove the getAdmin method from ClusterConnection, as it exposes the blocking protobuf interface, which will be a pain when we reimplement sync client based on the async client. And the AsyncRegionServerAdmin does not expose the protobuf stub directly, so we do not let the upper layer pass the RpcController, we will create one when we call the protobuf stub in the implementation of the methods. And no need to change the hbck project, it is hbck1 here, which will be removed in the future I think. > Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection > -- > > Key: HBASE-21537 > URL: https://issues.apache.org/jira/browse/HBASE-21537 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21537-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21679) Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741450#comment-16741450 ] Andrew Purtell commented on HBASE-21679: No problem [~lhofhansl] ! Thanks anyway > Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1 > --- > > Key: HBASE-21679 > URL: https://issues.apache.org/jira/browse/HBASE-21679 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21679-branch-1.patch, HBASE-21679-branch-1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21679) Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741448#comment-16741448 ] Lars Hofhansl commented on HBASE-21679: --- Sorry [~apurtell], no time to review last week. > Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1 > --- > > Key: HBASE-21679 > URL: https://issues.apache.org/jira/browse/HBASE-21679 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.5.0 > > Attachments: HBASE-21679-branch-1.patch, HBASE-21679-branch-1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21706) Inconsistency of fs.defaultFS between active and standby masters
[
https://issues.apache.org/jira/browse/HBASE-21706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741442#comment-16741442
]
Lei Chen edited comment on HBASE-21706 at 1/13/19 2:18 AM:
---
I don't have a running apache hbase cluster right now, but I do see same code
in apache hbase 1.1.2
-
[HRegionServer.java|https://github.com/apache/hbase/blob/rel/1.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L565]
-
[MasterFileSystem.java|https://github.com/apache/hbase/blob/rel/1.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java#L121]
I believe apache master branch also has the same code.
After failover, the new active master finds the right fs ( "hdfs://DEV-CLUSTER"
), but the new standby (previous active) master will join the rest and have it
set to "hdfs://DEV-CLUSTER/hbase-root"
I would like to provide a patch if this is indeed an unexpected behavior, but
could anyone please help me identify some cases where `fs.defaultFS` from
standby masters might be used?
Thanks
was (Author: leochen4891):
I don't have a running apache hbase cluster right now, but I do see same code
in apache hbase 1.1.2
-
[HRegionServer.java|https://github.com/apache/hbase/blob/rel/1.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L565]
-
[MasterFileSystem.java|https://github.com/apache/hbase/blob/rel/1.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java#L121]I
believe apache master branch also has the same code.
After failover, the new active master finds the right fs ( "hdfs://DEV-CLUSTER"
), but the new standby (previous active) master will join the rest and have it
set to "hdfs://DEV-CLUSTER/hbase-root"
I would like to provide a patch if this is indeed an unexpected behavior, but
could anyone please help me identify some cases where `fs.defaultFS` from
standby masters might be used?
Thanks
> Inconsistency of fs.defaultFS between active and standby masters
>
>
> Key: HBASE-21706
> URL: https://issues.apache.org/jira/browse/HBASE-21706
> Project: HBase
> Issue Type: Bug
> Components: conf, master
>Affects Versions: 1.1.2
>Reporter: Lei Chen
>Priority: Minor
>
> I'm using HDP-2.6.3.22-1 with HBase HA configured. I noticed that active and
> standby masters have different `fs.defaultFS` on their /conf pages.
> Given `fs.defaultFS` is set to : and `hbase.rootdir` is
> set to :/ in core-site.xml on all the hosts, it
> looks like standby masters has `fs.defaultFS` programatically set to the same
> value as `hbase.rootdir`.
> For example, on a 3 heads cluster DEV-CLUSTER, my active master has the
> following line on the /conf page
> {code:java}
> fs.defaultFShdfs://DEV-CLUSTERprogramatically
> {code}
> but standby masters has
> {code:java}
> fs.defaultFShdfs://DEV-CLUSTER/hbase-rootprogramatically{code}
> Please correct me if this is not a bug but a feature, however I find this
> behavior surprising plus I cannot locate any related document.
> From a quick look at the code, the cause seems to be that standby masters got
> the property set in
> [HRegionServer.java|https://github.com/hortonworks/hbase-release/blob/HDP-2.6.3.22-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L649-L652],
> and active master got it set in a different way in
> [MasterFileSystem.java|https://github.com/hortonworks/hbase-release/blob/HDP-2.6.3.22-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java#L132-L137].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21706) Inconsistency of fs.defaultFS between active and standby masters
[
https://issues.apache.org/jira/browse/HBASE-21706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741442#comment-16741442
]
Lei Chen edited comment on HBASE-21706 at 1/13/19 2:17 AM:
---
I don't have a running apache hbase cluster right now, but I do see same code
in apache hbase 1.1.2
-
[HRegionServer.java|https://github.com/apache/hbase/blob/rel/1.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L565]
-
[MasterFileSystem.java|https://github.com/apache/hbase/blob/rel/1.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java#L121]I
believe apache master branch also has the same code.
After failover, the new active master finds the right fs ( "hdfs://DEV-CLUSTER"
), but the new standby (previous active) master will join the rest and have it
set to "hdfs://DEV-CLUSTER/hbase-root"
I would like to provide a patch if this is indeed an unexpected behavior, but
could anyone please help me identify some cases where `fs.defaultFS` from
standby masters might be used?
Thanks
was (Author: leochen4891):
I don't have a running apache hbase cluster right now, but I do see same code
in apache hbase 1.1.2
-
[HRegionServer.java|https://github.com/apache/hbase/blob/rel/1.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L565]
- [MasterFileSystem.java
|https://github.com/apache/hbase/blob/rel/1.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java#L121]I
believe apache master branch also has the same code.
After failover, the new active master finds the right fs ( "hdfs://DEV-CLUSTER"
), but the new standby (previous active) master will join the rest and have it
set to "hdfs://DEV-CLUSTER/hbase-root"
I would like to provide a patch if this is indeed an unexpected behavior, but
could anyone please help me identify some cases where `fs.defaultFS` from
standby masters might be used?
Thanks
> Inconsistency of fs.defaultFS between active and standby masters
>
>
> Key: HBASE-21706
> URL: https://issues.apache.org/jira/browse/HBASE-21706
> Project: HBase
> Issue Type: Bug
> Components: conf, master
>Affects Versions: 1.1.2
>Reporter: Lei Chen
>Priority: Minor
>
> I'm using HDP-2.6.3.22-1 with HBase HA configured. I noticed that active and
> standby masters have different `fs.defaultFS` on their /conf pages.
> Given `fs.defaultFS` is set to : and `hbase.rootdir` is
> set to :/ in core-site.xml on all the hosts, it
> looks like standby masters has `fs.defaultFS` programatically set to the same
> value as `hbase.rootdir`.
> For example, on a 3 heads cluster DEV-CLUSTER, my active master has the
> following line on the /conf page
> {code:java}
> fs.defaultFShdfs://DEV-CLUSTERprogramatically
> {code}
> but standby masters has
> {code:java}
> fs.defaultFShdfs://DEV-CLUSTER/hbase-rootprogramatically{code}
> Please correct me if this is not a bug but a feature, however I find this
> behavior surprising plus I cannot locate any related document.
> From a quick look at the code, the cause seems to be that standby masters got
> the property set in
> [HRegionServer.java|https://github.com/hortonworks/hbase-release/blob/HDP-2.6.3.22-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L649-L652],
> and active master got it set in a different way in
> [MasterFileSystem.java|https://github.com/hortonworks/hbase-release/blob/HDP-2.6.3.22-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java#L132-L137].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21706) Inconsistency of fs.defaultFS between active and standby masters
[
https://issues.apache.org/jira/browse/HBASE-21706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741442#comment-16741442
]
Lei Chen commented on HBASE-21706:
--
I don't have a running apache hbase cluster right now, but I do see same code
in apache hbase 1.1.2
-
[HRegionServer.java|https://github.com/apache/hbase/blob/rel/1.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L565]
- [MasterFileSystem.java
|https://github.com/apache/hbase/blob/rel/1.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java#L121]I
believe apache master branch also has the same code.
After failover, the new active master finds the right fs ( "hdfs://DEV-CLUSTER"
), but the new standby (previous active) master will join the rest and have it
set to "hdfs://DEV-CLUSTER/hbase-root"
I would like to provide a patch if this is indeed an unexpected behavior, but
could anyone please help me identify some cases where `fs.defaultFS` from
standby masters might be used?
Thanks
> Inconsistency of fs.defaultFS between active and standby masters
>
>
> Key: HBASE-21706
> URL: https://issues.apache.org/jira/browse/HBASE-21706
> Project: HBase
> Issue Type: Bug
> Components: conf, master
>Affects Versions: 1.1.2
>Reporter: Lei Chen
>Priority: Minor
>
> I'm using HDP-2.6.3.22-1 with HBase HA configured. I noticed that active and
> standby masters have different `fs.defaultFS` on their /conf pages.
> Given `fs.defaultFS` is set to : and `hbase.rootdir` is
> set to :/ in core-site.xml on all the hosts, it
> looks like standby masters has `fs.defaultFS` programatically set to the same
> value as `hbase.rootdir`.
> For example, on a 3 heads cluster DEV-CLUSTER, my active master has the
> following line on the /conf page
> {code:java}
> fs.defaultFShdfs://DEV-CLUSTERprogramatically
> {code}
> but standby masters has
> {code:java}
> fs.defaultFShdfs://DEV-CLUSTER/hbase-rootprogramatically{code}
> Please correct me if this is not a bug but a feature, however I find this
> behavior surprising plus I cannot locate any related document.
> From a quick look at the code, the cause seems to be that standby masters got
> the property set in
> [HRegionServer.java|https://github.com/hortonworks/hbase-release/blob/HDP-2.6.3.22-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L649-L652],
> and active master got it set in a different way in
> [MasterFileSystem.java|https://github.com/hortonworks/hbase-release/blob/HDP-2.6.3.22-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java#L132-L137].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21519) Namespace region is never assigned in a HM failover scenario when multiwal is enabled
[
https://issues.apache.org/jira/browse/HBASE-21519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741419#comment-16741419
]
stack commented on HBASE-21519:
---
Is .meta correct? I could commit this patch. The change is
innocuous. It just seems there is a mangling of providerid going on. What you
think [~pankaj2461]? If so, we could commit this and file follow-on?
> Namespace region is never assigned in a HM failover scenario when multiwal is
> enabled
> -
>
> Key: HBASE-21519
> URL: https://issues.apache.org/jira/browse/HBASE-21519
> Project: HBase
> Issue Type: Bug
> Components: master, wal
>Affects Versions: 2.1.1
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.2.0
>
> Attachments: HBASE-21519.branch-2.patch
>
>
> In our test env we found that namespace region is never be assigned on HM
> failover scenario when multiwal feature is enabled,
> {noformat}
> 2018-11-28 01:38:28,085 WARN [master/HM-1:16000:becomeActiveMaster]
> master.HMaster:
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15. is NOT
> online; state=\{31f6d3383af09e18e1e81ca02a93de15 state=OPEN,
> ts=1543340156928, server=RS-2,16020,1543339824397};
> ServerCrashProcedures=false. Master startup cannot progress, in
> holding-pattern until region onlined.
> {noformat}
> And finally HM abort with following error,
> {noformat}
> 2018-11-28 01:39:16,858 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Master
> failed to complete initialization after 24ms. Please consider submitting
> a bug report including a thread dump of this process.
> 2018-11-28 01:39:18,980 ERROR
> [ActiveMasterInitializationMonitor-1543338648565] master.HMaster: Zombie
> Master exiting. Thread dump to stdout
> {noformat}
> Stack trace:
> {noformat}
> Thread 102 (master/HM-1:16000:becomeActiveMaster):
> State: TIMED_WAITING
> Blocked count: 100
> Waited count: 246
> Stack:
> java.lang.Thread.sleep(Native Method)
> org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:148)
> org.apache.hadoop.hbase.master.HMaster.isRegionOnline(HMaster.java:1166)
>
> org.apache.hadoop.hbase.master.HMaster.waitForNamespaceOnline(HMaster.java:1187)
>
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1044)
>
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2285)
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:590)
> org.apache.hadoop.hbase.master.HMaster$$Lambda$40/1078246575.run(Unknown
> Source)
> java.lang.Thread.run(Thread.java:745)
> {noformat}
>
> Step to reproduce:
> 1) Setup a HBase cluster with 1/2 HM (say HM-1) and 2 RS(say RS-1 & RS-2)
> 2) Enable multiwal feature with following configuration setting and start
> the cluster,
> {noformat}
>
> hbase.wal.provider
> multiwal
>
>
> hbase.wal.regiongrouping.strategy
> identity
>
> {noformat}
> 3) Make sure meta and namespace regions are assigned on different RS, suppose
> RS-1 & RS-2 respectively.
> 4) Create table 't1'
> 5) Flush the meta table explicitly
> 6) Kill the RS-2, so during RS-2 SCP all regions including namespace region
> will be assigned to RS-1.
> 7) Now Kill RS-1 before meta flush happen. Here both RS-2 & RS-1 are
> shutdown now.
> 8) Stop the HM and start RS-1 & RS-2.
> 9) Now start the HM.
> Meta region is assigned successfully but HM is keep waiting for the namespace
> region onlline (Master startup cannot progress, in holding-pattern until
> region onlined) and abort with timeout.
> Observation:
> 1) After step-3 namespace region was assigned to RS-2 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
> {noformat}
> 2) After step-6 namespace region was assigned to RS-1 and meta entry was as
> follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339880920, value=RS-1:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339880920, value=1543339829288
> {noformat}
> 3) After Step-9, meta entry for namespace region was as follows,
> {noformat}
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:server, timestamp=1543339860920, value=RS-2:16020
> hbase:namespace,,1543339859614.31f6d3383af09e18e1e81ca02a93de15.
> column=info:serverstartcode, timestamp=1543339860920, value=1543339824397
>
[jira] [Commented] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[
https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741399#comment-16741399
]
Hadoop QA commented on HBASE-21657:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
58s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
21s{color} | {color:red} hbase-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
19s{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
32s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
22s{color} | {color:red} hbase-mapreduce in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 21s{color}
| {color:red} hbase-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 19s{color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 32s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 22s{color}
| {color:red} hbase-mapreduce in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} hbase-common: The patch generated 0 new + 211
unchanged - 3 fixed = 211 total (was 214) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
12s{color} | {color:red} hbase-server: The patch generated 2 new + 410
unchanged - 0 fixed = 412 total (was 410) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch passed checkstyle in hbase-mapreduce
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 1m
4s{color} | {color:red} patch has 16 errors when building our shaded downstream
artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m
47s{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
35s{color} | {color:red} The patch causes 16 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
15s{color} | {color:red} hbase-common
[jira] [Commented] (HBASE-21706) Inconsistency of fs.defaultFS between active and standby masters
[
https://issues.apache.org/jira/browse/HBASE-21706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741398#comment-16741398
]
stack commented on HBASE-21706:
---
Thanks for reporting. Is this an HDP issue or do you see it in apache too? If
you are confused and there is no clarifying doc, and it seems wrong, it
probably is. On failover, does the new master find the right fs? Thanks.
> Inconsistency of fs.defaultFS between active and standby masters
>
>
> Key: HBASE-21706
> URL: https://issues.apache.org/jira/browse/HBASE-21706
> Project: HBase
> Issue Type: Bug
> Components: conf, master
>Affects Versions: 1.1.2
>Reporter: Lei Chen
>Priority: Minor
>
> I'm using HDP-2.6.3.22-1 with HBase HA configured. I noticed that active and
> standby masters have different `fs.defaultFS` on their /conf pages.
> Given `fs.defaultFS` is set to : and `hbase.rootdir` is
> set to :/ in core-site.xml on all the hosts, it
> looks like standby masters has `fs.defaultFS` programatically set to the same
> value as `hbase.rootdir`.
> For example, on a 3 heads cluster DEV-CLUSTER, my active master has the
> following line on the /conf page
> {code:java}
> fs.defaultFShdfs://DEV-CLUSTERprogramatically
> {code}
> but standby masters has
> {code:java}
> fs.defaultFShdfs://DEV-CLUSTER/hbase-rootprogramatically{code}
> Please correct me if this is not a bug but a feature, however I find this
> behavior surprising plus I cannot locate any related document.
> From a quick look at the code, the cause seems to be that standby masters got
> the property set in
> [HRegionServer.java|https://github.com/hortonworks/hbase-release/blob/HDP-2.6.3.22-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L649-L652],
> and active master got it set in a different way in
> [MasterFileSystem.java|https://github.com/hortonworks/hbase-release/blob/HDP-2.6.3.22-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java#L132-L137].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21699) Create table failed when using SPLITS_FILE => 'splits.txt'
[
https://issues.apache.org/jira/browse/HBASE-21699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741394#comment-16741394
]
stack commented on HBASE-21699:
---
[~rebornHuan] Did you see [~zghaobac] comment sir?
> Create table failed when using SPLITS_FILE => 'splits.txt'
> ---
>
> Key: HBASE-21699
> URL: https://issues.apache.org/jira/browse/HBASE-21699
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0, 2.0.1, 2.1.1, 2.0.2, 2.0.3, 2.1.2, 2.0.4
>Reporter: huan
>Priority: Minor
> Attachments: hbase-21699.001.patch
>
>
> Hi all:
> When I ran
> {code:java}
> create 't1', 'f1', SPLITS_FILE => 'splits.txt'
> {code}
> on HBase2.0.0, it failed, and no detailed error info, just like below:
> {code:java}
> ERROR:
> Creates a table. Pass a table name, and a set of column family
> specifications (at least one), and, optionally, table configuration.
> Column specification can be a simple string (name), or a dictionary
> (dictionaries are described below in main help output), necessarily
> including NAME attribute.
> Examples:
> {code}
> So I opened the debug:
> {code:java}
> hbase shell -d
> {code}
> and
> {code:java}
> ERROR:
> Backtrace:
> org.apache.hadoop.hbase.util.Bytes.toBytes(org/apache/hadoop/hbase/util/Bytes.java:732)
> org.apache.hadoop.hbase.HTableDescriptor.setValue(org/apache/hadoop/hbase/HTableDescriptor.java:190){code}
> But it works on branch 1.2.0.
> so I view the source code, I find the issue is because the below code:
> {code:java}
> // admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE])
> end
> {code}
> {code:java}
> // HTableDescriptor part
> public HTableDescriptor setValue(String key, String value) {
> getDelegateeForModification().setValue(Bytes.toBytes(key),
> Bytes.toBytes(value));
> return this;
> }
> {code}
> {code:java}
> // Bytes part
> public static byte[] toBytes(String s) {
> try {
> return s.getBytes(UTF8_CSN);
> } catch (UnsupportedEncodingException e) {
> // should never happen!
> throw new IllegalArgumentException("UTF8 decoding is not supported", e);
> }
> }
> {code}
> Call flow is:
> {code:java}
> admin.rb ---> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE]) --->
> Bytes.toBytes(key) && Bytes.toBytes(value) {code}
> from Bytes.toBytes, if s is null, the function will throw
> NullPointerException, but HTableDescriptor.setValue(String key, String value)
> does not check key and value.
> in admin.rb, it use arg.delete(SPLITS_FILE) to get the value, but this means,
> after using arg.delete(SPLITS_FILE), arg[SPLITS_FILE] will return nil. so
> HTableDescriptor.setValue(String key, String value) does not check key and
> value is root cause.
> why branch below 2.0.0 works fine, because old code is :
> {code:java}
> public HTableDescriptor setValue(String key, String value) {
> if (value == null) {
> remove(key);
> } else {
> setValue(Bytes.toBytes(key), Bytes.toBytes(value));
> }
> return this;
> }
> {code}
> it check the value.
> since branch 2.0.0, HBase add new function called 'TableDescriptorBuilder'
> it included:
> {code:java}
> public ModifyableTableDescriptor setValue(String key, String value) {
> return setValue(toBytesOrNull(key, Bytes::toBytes),
> toBytesOrNull(value, Bytes::toBytes));
> }
> {code}
> it checked key and value, but HTableDescriptor.setValue(String key, String
> value) does not call it, so
> just change HTableDescriptor.setValue(String key, String value) to call it,
> it will works
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21456) Make WALFactory only used for creating WALProviders
[ https://issues.apache.org/jira/browse/HBASE-21456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741397#comment-16741397 ] stack commented on HBASE-21456: --- I'm good w/ it all. Just wondering if a write-up/plan/spec to consult to help navigate the changes. Would address comments like mine and [~reidchan]'s where our first blush is to ask why the change at all? Thanks. > Make WALFactory only used for creating WALProviders > --- > > Key: HBASE-21456 > URL: https://issues.apache.org/jira/browse/HBASE-21456 > Project: HBase > Issue Type: Sub-task > Components: wal >Reporter: Josh Elser >Assignee: Ankit Singhal >Priority: Major > Attachments: HBASE-21456.HBASE-20952.001.patch, > HBASE-21456.HBASE-20952.wip.patch > > > As a Factory, WALFactory should only have one job: creating instances of > WALProvider. > However, over the years, it has been a landing place for lots of wal-related > methods (e.g. constructing readers, WALEntryStream, and more). We want all of > this to live in the WALProvider. > We can do this in two steps: have the WALFactory methods invoke the method on > the WALProvider (handled elsewhere), then here we can replace usage of the > WALFactory "wrapper methods" with the WALProvider itself. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741392#comment-16741392 ] stack commented on HBASE-21657: --- v5 is nice and clean. Do you need the change in NoTagsByteBufferKeyValue. It inherits from ByteBufferKeyValue which has your change. Is it safe doing a return of this.length in SizeCachedKeyValue ? It caches rowLen and keyLen... Does it cache this.length? Maybe its caching what is passed in on construction? That addition to addSize in RSRpcServices is crptic. We need that sir? Say more why you are doing the accounting on the outside? Patch is great. +1. It should go back to branch-2.0? I've not done the testing on my side. I'll keep trying but lets not have this hold up this fix. > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21657.v1.patch, HBASE-21657.v2.patch, > HBASE-21657.v3.patch, HBASE-21657.v3.patch, HBASE-21657.v4.patch, > HBASE-21657.v5.patch, HBASE-21657.v5.patch, HBASE-21657.v5.patch, > HBase1.4.9-ssd-1000-rows-flamegraph.svg, > HBase1.4.9-ssd-1000-rows-qps-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-qps-latency.png, HBase2.0.4-with-patch.v2.png, > HBase2.0.4-without-patch-v2.png, debug-the-ByteBufferKeyValue.diff, > hbase2.0.4-ssd-scan-traces.2.svg, hbase2.0.4-ssd-scan-traces.svg, > hbase20-ssd-100-scan-traces.svg, image-2019-01-07-19-03-37-930.png, > image-2019-01-07-19-03-55-577.png, overview-statstics-1.png, run.log > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21657) PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.
[ https://issues.apache.org/jira/browse/HBASE-21657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21657: -- Attachment: HBASE-21657.v5.patch > PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% > scan case. > > > Key: HBASE-21657 > URL: https://issues.apache.org/jira/browse/HBASE-21657 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21657.v1.patch, HBASE-21657.v2.patch, > HBASE-21657.v3.patch, HBASE-21657.v3.patch, HBASE-21657.v4.patch, > HBASE-21657.v5.patch, HBASE-21657.v5.patch, HBASE-21657.v5.patch, > HBase1.4.9-ssd-1000-rows-flamegraph.svg, > HBase1.4.9-ssd-1000-rows-qps-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v2-ssd-1000-rows.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-patch-v3-ssd-1000-rows-qps-and-latency.png, > HBase2.0.4-patch-v4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-flamegraph.svg, > HBase2.0.4-ssd-1000-rows-qps-latency.png, HBase2.0.4-with-patch.v2.png, > HBase2.0.4-without-patch-v2.png, debug-the-ByteBufferKeyValue.diff, > hbase2.0.4-ssd-scan-traces.2.svg, hbase2.0.4-ssd-scan-traces.svg, > hbase20-ssd-100-scan-traces.svg, image-2019-01-07-19-03-37-930.png, > image-2019-01-07-19-03-55-577.png, overview-statstics-1.png, run.log > > > We are evaluating the performance of branch-2, and find that the throughput > of scan in SSD cluster is almost the same as HDD cluster. so I made a > FlameGraph on RS, and found that the > PrivateCellUtil#estimatedSerializedSizeOf cost about 29% cpu, Obviously, it > has been the bottleneck in 100% scan case. > See the [^hbase20-ssd-100-scan-traces.svg] > BTW, in our XiaoMi branch, we introduce a > HRegion#updateReadRequestsByCapacityUnitPerSecond to sum up the size of cells > (for metric monitor), so it seems the performance loss was amplified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21537) Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection
[ https://issues.apache.org/jira/browse/HBASE-21537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741383#comment-16741383 ] stack commented on HBASE-21537: --- This is unrelated but why on the async connection do I do connection.getRegionServerAdmin(server) but on sync Connection, I do getAdmin? Can they not be the same? We don't pass an rpc controller when we are doing async admin. Its because the controller is actually used doing async? Otherwise, +1. Need changes over on the hbck project to match? > Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection > -- > > Key: HBASE-21537 > URL: https://issues.apache.org/jira/browse/HBASE-21537 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21537-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21685) Change repository urls to Gitbox
[ https://issues.apache.org/jira/browse/HBASE-21685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741322#comment-16741322 ] Sean Busbey commented on HBASE-21685: - thanks for the heads up. I filed a jira to track things down. I think asf infra is about done fixing things. > Change repository urls to Gitbox > > > Key: HBASE-21685 > URL: https://issues.apache.org/jira/browse/HBASE-21685 > Project: HBase > Issue Type: Task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Critical > Fix For: 3.0.0 > > Attachments: HBASE-21685.master.001.patch, > HBASE-21685.master.001.patch > > > Moving to Gitbox is approaching and references to git-wip-us need to be > changed to gitbox. > Some of the Jenkins jobs are referring to git-wip-us which if going to be > locked after the migration. We could move them to github so the build flow > will remain intact. > Previous discussion on dev@: > [https://lists.apache.org/thread.html/3496568d6cc002f74f5c3bcce46ed44b7ee9e90d7d53af2c65b6f785@%3Cdev.hbase.apache.org%3E] > After this notify INFRA to make the change -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21709) website should have browsable release notes
Sean Busbey created HBASE-21709: --- Summary: website should have browsable release notes Key: HBASE-21709 URL: https://issues.apache.org/jira/browse/HBASE-21709 Project: HBase Issue Type: Improvement Components: website Reporter: Sean Busbey Assignee: Sean Busbey Right now the website has a couple of places to find "release notes": * In the main menu under the "Apache HBase Project" heading, but it's just a link to open issues in jira (which is like the opposite of release notes) * On the downloads page, but it's inconsistently unrendered markdown or a link to jira release notes We should consolidate and provide HTML formatted notes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21709) website should have browsable release notes
[ https://issues.apache.org/jira/browse/HBASE-21709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21709: Component/s: community > website should have browsable release notes > --- > > Key: HBASE-21709 > URL: https://issues.apache.org/jira/browse/HBASE-21709 > Project: HBase > Issue Type: Improvement > Components: community, website >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > Right now the website has a couple of places to find "release notes": > * In the main menu under the "Apache HBase Project" heading, but it's just a > link to open issues in jira (which is like the opposite of release notes) > * On the downloads page, but it's inconsistently unrendered markdown or a > link to jira release notes > We should consolidate and provide HTML formatted notes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21708) ReviewBoard integration is busted post-gitbox move
[ https://issues.apache.org/jira/browse/HBASE-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741313#comment-16741313 ] Sean Busbey commented on HBASE-21708: - filed INFRA-17632 with infra since it looks like a reviewboard configuration thing to me. > ReviewBoard integration is busted post-gitbox move > -- > > Key: HBASE-21708 > URL: https://issues.apache.org/jira/browse/HBASE-21708 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > > per comment from [~Apache9] on HBASE-21685 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-21708) ReviewBoard integration is busted post-gitbox move
[ https://issues.apache.org/jira/browse/HBASE-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-21708 started by Sean Busbey. --- > ReviewBoard integration is busted post-gitbox move > -- > > Key: HBASE-21708 > URL: https://issues.apache.org/jira/browse/HBASE-21708 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > > per comment from [~Apache9] on HBASE-21685 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[
https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741298#comment-16741298
]
Hudson commented on HBASE-21512:
Results for branch HBASE-21512
[build #51 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/51/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/51//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/51//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21512/51//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
> --
>
> Key: HBASE-21512
> URL: https://issues.apache.org/jira/browse/HBASE-21512
> Project: HBase
> Issue Type: Umbrella
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
> At least for the RSProcedureDispatcher, with CompletableFuture we do not need
> to set a delay and use a thread pool any more, which could reduce the
> resource usage and also the latency.
> Once this is done, I think we can remove the ClusterConnection completely,
> and start to rewrite the old sync client based on the async client, which
> could reduce the code base a lot for our client.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21708) ReviewBoard integration is busted post-gitbox move
Sean Busbey created HBASE-21708: --- Summary: ReviewBoard integration is busted post-gitbox move Key: HBASE-21708 URL: https://issues.apache.org/jira/browse/HBASE-21708 Project: HBase Issue Type: Task Components: community Reporter: Sean Busbey Assignee: Sean Busbey per comment from [~Apache9] on HBASE-21685 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21707) Fix warnings in hbase-rsgroup module and also make the UTs more stable
[
https://issues.apache.org/jira/browse/HBASE-21707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741289#comment-16741289
]
Hadoop QA commented on HBASE-21707:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 9 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
35s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hbase-rsgroup: The patch generated 1 new + 2 unchanged
- 5 fixed = 3 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
39s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 9s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
7s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21707 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954728/HBASE-21707.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8dd716cd0c03 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3d2580cd6d |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/15551/artifact/patchprocess/diff-checkstyle-hbase-rsgroup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15551/testReport/ |
| Max. process+thread count | 3316 (vs. ulimit of 1) |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
[jira] [Commented] (HBASE-21705) Should treat meta table specially for some methods in AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-21705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741284#comment-16741284
]
Hadoop QA commented on HBASE-21705:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} The patch passed checkstyle in hbase-client {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} hbase-server: The patch generated 0 new + 1
unchanged - 2 fixed = 1 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 31s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
16s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}133m 22s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}180m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestHbck |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21705 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954724/HBASE-21705-v2.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 04c29b67c786 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
|
[jira] [Commented] (HBASE-21685) Change repository urls to Gitbox
[ https://issues.apache.org/jira/browse/HBASE-21685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741274#comment-16741274 ] Duo Zhang commented on HBASE-21685: --- Seems it is impossible to upload patches to RB after the move? > Change repository urls to Gitbox > > > Key: HBASE-21685 > URL: https://issues.apache.org/jira/browse/HBASE-21685 > Project: HBase > Issue Type: Task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Critical > Fix For: 3.0.0 > > Attachments: HBASE-21685.master.001.patch, > HBASE-21685.master.001.patch > > > Moving to Gitbox is approaching and references to git-wip-us need to be > changed to gitbox. > Some of the Jenkins jobs are referring to git-wip-us which if going to be > locked after the migration. We could move them to github so the build flow > will remain intact. > Previous discussion on dev@: > [https://lists.apache.org/thread.html/3496568d6cc002f74f5c3bcce46ed44b7ee9e90d7d53af2c65b6f785@%3Cdev.hbase.apache.org%3E] > After this notify INFRA to make the change -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21707) Fix warnings in hbase-rsgroup module and also make the UTs more stable
[ https://issues.apache.org/jira/browse/HBASE-21707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21707: -- Assignee: Duo Zhang Status: Patch Available (was: Open) > Fix warnings in hbase-rsgroup module and also make the UTs more stable > -- > > Key: HBASE-21707 > URL: https://issues.apache.org/jira/browse/HBASE-21707 > Project: HBase > Issue Type: Bug > Components: Region Assignment, rsgroup >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21707-UT.patch, HBASE-21707.patch > > > By adding a sleep after the stopServer call, the test will hang there > forever, need to dig more, maybe something wrong with the TRSP related stuffs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21707) Fix warnings in hbase-rsgroup module and also make the UTs more stable
[ https://issues.apache.org/jira/browse/HBASE-21707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21707: -- Attachment: HBASE-21707.patch > Fix warnings in hbase-rsgroup module and also make the UTs more stable > -- > > Key: HBASE-21707 > URL: https://issues.apache.org/jira/browse/HBASE-21707 > Project: HBase > Issue Type: Bug > Components: Region Assignment, rsgroup >Reporter: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21707-UT.patch, HBASE-21707.patch > > > By adding a sleep after the stopServer call, the test will hang there > forever, need to dig more, maybe something wrong with the TRSP related stuffs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21707) Fix warnings in hbase-rsgroup module and also make the UTs more stable
[ https://issues.apache.org/jira/browse/HBASE-21707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21707: -- Fix Version/s: 2.2.0 3.0.0 > Fix warnings in hbase-rsgroup module and also make the UTs more stable > -- > > Key: HBASE-21707 > URL: https://issues.apache.org/jira/browse/HBASE-21707 > Project: HBase > Issue Type: Bug > Components: Region Assignment, rsgroup >Reporter: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21707-UT.patch > > > By adding a sleep after the stopServer call, the test will hang there > forever, need to dig more, maybe something wrong with the TRSP related stuffs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21707) Fix warnings in hbase-rsgroup module and also make the UTs more stable
[ https://issues.apache.org/jira/browse/HBASE-21707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21707: -- Summary: Fix warnings in hbase-rsgroup module and also make the UTs more stable (was: TestRSGroupsKillRS is not stable enough) > Fix warnings in hbase-rsgroup module and also make the UTs more stable > -- > > Key: HBASE-21707 > URL: https://issues.apache.org/jira/browse/HBASE-21707 > Project: HBase > Issue Type: Bug > Components: Region Assignment, rsgroup >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21707-UT.patch > > > By adding a sleep after the stopServer call, the test will hang there > forever, need to dig more, maybe something wrong with the TRSP related stuffs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21705) Should treat meta table specially for some methods in AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-21705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21705: -- Attachment: HBASE-21705-v2.patch > Should treat meta table specially for some methods in AsyncAdmin > > > Key: HBASE-21705 > URL: https://issues.apache.org/jira/browse/HBASE-21705 > Project: HBase > Issue Type: Sub-task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5 > > Attachments: HBASE-21705-v1.patch, HBASE-21705-v2.patch, > HBASE-21705.patch > > > For example, tableExists, isTableEnabled, isTableDisabled... > For now, we will go to the meta table directly but obviously, meta table does > not contain the record for itself... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-6028) Implement a cancel for in-progress compactions
[
https://issues.apache.org/jira/browse/HBASE-6028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741214#comment-16741214
]
Hudson commented on HBASE-6028:
---
Results for branch branch-1
[build #630 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/630/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/630//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/630//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/630//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Implement a cancel for in-progress compactions
> --
>
> Key: HBASE-6028
> URL: https://issues.apache.org/jira/browse/HBASE-6028
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Derek Wollenstein
>Assignee: Mohit Goel
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-6028.master.007.patch,
> HBASE-6028.master.008.patch, HBASE-6028.master.008.patch,
> HBASE-6028.master.009.patch
>
>
> Depending on current server load, it can be extremely expensive to run
> periodic minor / major compactions. It would be helpful to have a feature
> where a user could use the shell or a client tool to explicitly cancel an
> in-progress compactions. This would allow a system to recover when too many
> regions became eligible for compactions at once
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21679) Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741213#comment-16741213
]
Hudson commented on HBASE-21679:
Results for branch branch-1
[build #630 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/630/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/630//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/630//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/630//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
> ---
>
> Key: HBASE-21679
> URL: https://issues.apache.org/jira/browse/HBASE-21679
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Major
> Fix For: 1.5.0
>
> Attachments: HBASE-21679-branch-1.patch, HBASE-21679-branch-1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21620) Problem in scan query when using more than one column prefix filter in some cases.
[
https://issues.apache.org/jira/browse/HBASE-21620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741207#comment-16741207
]
Zheng Hu commented on HBASE-21620:
--
Any improvement ? another reason maybe: the filter list concated by OR will
choose the minimal forward step among all sub-filters. in this patch, we have
stricter restrictions on all sub filters, include those sub-filter whose has
non-null RC returned in calculateReturnCodeByPrevCellAndRC (previously, we will
skip to merge this sub-filter's rc, but it's wrong in some case), and merge all
of the sub-filter's RC, this is also some time cost.
> Problem in scan query when using more than one column prefix filter in some
> cases.
> --
>
> Key: HBASE-21620
> URL: https://issues.apache.org/jira/browse/HBASE-21620
> Project: HBase
> Issue Type: Bug
> Components: scan
>Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.8, 2.1.2, 2.0.4
> Environment: hbase-1.4.8, hbase-1.4.9
> hadoop-2.7.3
>Reporter: Mohamed Mohideen Meeran
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2, 2.0.4
>
> Attachments: HBASE-21620.branch-1.patch, HBASE-21620.v1.patch,
> HBASE-21620.v2.patch, HBASE-21620.v3.patch, HBASE-21620.v3.patch,
> HBaseImportData.java, columnkey.txt, file.txt, test.patch
>
>
> In some cases, unable to get the scan results when using more than one column
> prefix filter.
> Attached a java file to import the data which we used and a text file
> containing the values..
> While executing the following query (hbase shell as well as java program) it
> is waiting indefinitely and after RPC timeout we got the following error..
> Also we noticed high cpu, high load average and very frequent young gc in
> the region server containing this row...
> scan 'namespace:tablename',\{STARTROW => 'test',ENDROW => 'test', FILTER =>
> "ColumnPrefixFilter('1544770422942010001_') OR
> ColumnPrefixFilter('1544769883529010001_')"}
> ROW COLUMN+CELL
> ERROR: Call id=18,
> waitTime=60005, rpcTimetout=6
>
> Note: Table scan operation and scan with a single column prefix filter works
> fine in this case.
> When we check the same query in hbase-1.2.5 it is working fine.
> Can you please help me on this..
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21705) Should treat meta table specially for some methods in AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-21705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16741177#comment-16741177
]
Hadoop QA commented on HBASE-21705:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 50s{color}
| {color:red} hbase-server generated 1 new + 187 unchanged - 1 fixed = 188
total (was 188) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
2s{color} | {color:red} hbase-server: The patch generated 4 new + 3 unchanged -
0 fixed = 7 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
10s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 42s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
22s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}130m
43s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
52s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}177m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21705 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954688/HBASE-21705-v1.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 0e5ad2918e43 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 3d2580cd6d |
| maven | version: Apache Maven 3.5.4
