[jira] [Created] (HBASE-28148) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit
Duo Zhang created HBASE-28148: - Summary: Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit Key: HBASE-28148 URL: https://issues.apache.org/jira/browse/HBASE-28148 Project: HBase Issue Type: Task Components: dependabot, scripts, security Reporter: Duo Zhang Fix For: 4.0.0-alpha-1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28148 Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit [hbase]
Apache9 merged PR #5459: URL: https://github.com/apache/hbase/pull/5459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28148 Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit [hbase]
Apache-HBase commented on PR #5459: URL: https://github.com/apache/hbase/pull/5459#issuecomment-1759046875 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | patch | 0m 3s | https://github.com/apache/hbase/pull/5459 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/5459 | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5459/2/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Issue Type: Bug (was: Improvement) > Too many master region WAL cause HDFS disk space full > - > > Key: HBASE-28150 > URL: https://issues.apache.org/jira/browse/HBASE-28150 > Project: HBase > Issue Type: Bug >Affects Versions: 2.4.14 >Reporter: chaijunjie >Priority: Critical > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28150) Too many master region WAL cause HDFS disk space full
chaijunjie created HBASE-28150: -- Summary: Too many master region WAL cause HDFS disk space full Key: HBASE-28150 URL: https://issues.apache.org/jira/browse/HBASE-28150 Project: HBase Issue Type: Improvement Affects Versions: 2.4.14 Reporter: chaijunjie -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Description: create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then will try again and again, will write too many proc record to master:store, we find num of the master WAL in oldWALs more than 1300.. > Too many master region WAL cause HDFS disk space full > - > > Key: HBASE-28150 > URL: https://issues.apache.org/jira/browse/HBASE-28150 > Project: HBase > Issue Type: Bug >Affects Versions: 2.4.14 >Reporter: chaijunjie >Priority: Critical > > create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then > will try again and again, will write too many proc record to master:store, we > find num of the master WAL in oldWALs more than 1300.. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[
https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chaijunjie updated HBASE-28150:
---
Description:
create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then
will try again and again, will write too many proc record to master:store, we
find num of the master WAL in oldWALs more than 13000..
Q: should add a suspend time logic for create table proc retry? i see
TransitRegionStateProcedure has the logic..
---
sorry, i upload screenshot failed, just copy to here
{code:java}
// 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 |
Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900)
2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created
at hdfs://hacluster/hbase/.tmp for table themis:a |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346)
2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create
table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159)
java.io.IOException: Unable to move table from
temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase
root=hdfs://hacluster/hbase/data/themis/a
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962)
at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988)
2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917,
state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure
table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102)
2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating
{ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME =>
'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '',
ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW',
IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE',
DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER',
MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536',
REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp |
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting
for flushes and compactions to finish for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total
wait time for flushes and compaction for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing
region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,432 | INFO |
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[
https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chaijunjie updated HBASE-28150:
---
Description:
create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then
will try again and again, will write too many proc record to master:store, we
find num of the master WAL in oldWALs more than 13000..
Q: should add a suspend time logic for create table proc retry? i see
TransitRegionStateProcedure has the logic..
---
sorry, i upload screenshot failed, just copy to here
{code:java}
// 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 |
Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900)
2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created
at hdfs://hacluster/hbase/.tmp for table themis:a |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346)
2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create
table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159)
java.io.IOException: Unable to move table from
temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase
root=hdfs://hacluster/hbase/data/themis/a
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962)
at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988)
2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917,
state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure
table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102)
2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating
{ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME =>
'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '',
ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW',
IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE',
DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER',
MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536',
REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp |
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting
for flushes and compactions to finish for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total
wait time for flushes and compaction for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing
region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,432 | INFO |
Re: [PR] HBASE-28147 Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests [hbase]
Apache9 merged PR #5458: URL: https://github.com/apache/hbase/pull/5458 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
Duo Zhang created HBASE-28147: - Summary: Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests Key: HBASE-28147 URL: https://issues.apache.org/jira/browse/HBASE-28147 Project: HBase Issue Type: Task Components: dependabot, scripts, security Reporter: Duo Zhang Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28148) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit
[ https://issues.apache.org/jira/browse/HBASE-28148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28148. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to master. > Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit > --- > > Key: HBASE-28148 > URL: https://issues.apache.org/jira/browse/HBASE-28148 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28148 Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit [hbase]
Apache-HBase commented on PR #5459: URL: https://github.com/apache/hbase/pull/5459#issuecomment-1759047078 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 12s | Docker mode activated. | | -1 :x: | patch | 0m 3s | https://github.com/apache/hbase/pull/5459 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5459/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5459 | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5459/2/console | | versions | git=2.34.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28148 Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit [hbase]
Apache-HBase commented on PR #5459: URL: https://github.com/apache/hbase/pull/5459#issuecomment-1759047295 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -1 :x: | patch | 0m 4s | https://github.com/apache/hbase/pull/5459 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5459/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5459 | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5459/2/console | | versions | git=2.34.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28138 Make the connection idle timeout configurable for the embedded HTTP servers [hbase]
wchevreuil merged PR #5457: URL: https://github.com/apache/hbase/pull/5457 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[ https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28147. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.4+. > Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests > > > Key: HBASE-28147 > URL: https://issues.apache.org/jira/browse/HBASE-28147 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28149) move_servers_namespaces_rsgroup is not changing the new rs group in namespace description
[
https://issues.apache.org/jira/browse/HBASE-28149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajeshbabu Chintaguntla updated HBASE-28149:
Issue Type: Bug (was: Improvement)
> move_servers_namespaces_rsgroup is not changing the new rs group in namespace
> description
> -
>
> Key: HBASE-28149
> URL: https://issues.apache.org/jira/browse/HBASE-28149
> Project: HBase
> Issue Type: Bug
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
>
> {noformat}
> hbase:024:0> list_rsgroups
> NAME SERVER / TABLE
> rs1 server
> hostname2:16020
> table hbase:meta
> table hbase:acl
> table
> hbase:namespace
> table
> hbase:rsgroup
> default server
> hostname3:16020
> table ns_R:ta
> tenant_group1
> tenant_group server
> hostname1:16020
> table ns:tb1
> table ns:tab
> table ns:t2
> 4 row(s)
> Took 0.0129 seconds
> hbase:025:0> move_servers_namespaces_rsgroup 'rs1',['hostname1:16020'], ['ns']
> Took 0.0302 seconds
> hbase:026:0> list_rsgroups
> NAME SERVER / TABLE
> rs1 server
> hostname1:16020
> server
> hostname2:16020
> table ns:tb1
> table ns:tab
> table hbase:meta
> table hbase:acl
> table ns:t2
> table
> hbase:namespace
> table
> hbase:rsgroup
> default server
> hostname2:16020
> table ns_R:ta
> tenant_group1
> tenant_group
> 4 row(s)
> Took 0.0140 seconds
> hbase:027:0> describe_namespace 'ns'
> DESCRIPTION
> {NAME => 'ns', hbase.rsgroup.name => 'tenant_group'}
> Quota is disabled
> Took 0.0093 seconds
> hbase:028:0>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28149) move_servers_namespaces_rsgroup is not changing the new rs group in namespace description
Rajeshbabu Chintaguntla created HBASE-28149:
---
Summary: move_servers_namespaces_rsgroup is not changing the new
rs group in namespace description
Key: HBASE-28149
URL: https://issues.apache.org/jira/browse/HBASE-28149
Project: HBase
Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
{noformat}
hbase:024:0> list_rsgroups
NAME SERVER / TABLE
rs1 server
hostname2:16020
table hbase:meta
table hbase:acl
table
hbase:namespace
table
hbase:rsgroup
default server
hostname3:16020
table ns_R:ta
tenant_group1
tenant_group server
hostname1:16020
table ns:tb1
table ns:tab
table ns:t2
4 row(s)
Took 0.0129 seconds
hbase:025:0> move_servers_namespaces_rsgroup 'rs1',['hostname1:16020'], ['ns']
Took 0.0302 seconds
hbase:026:0> list_rsgroups
NAME SERVER / TABLE
rs1 server
hostname1:16020
server
hostname2:16020
table ns:tb1
table ns:tab
table hbase:meta
table hbase:acl
table ns:t2
table
hbase:namespace
table
hbase:rsgroup
default server
hostname2:16020
table ns_R:ta
tenant_group1
tenant_group
4 row(s)
Took 0.0140 seconds
hbase:027:0> describe_namespace 'ns'
DESCRIPTION
{NAME => 'ns', hbase.rsgroup.name => 'tenant_group'}
Quota is disabled
Took 0.0093 seconds
hbase:028:0>
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28135) Specify -Xms for tests
[ https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-28135. --- Fix Version/s: 2.6.0 2.4.18 2.5.6 3.0.0-beta-1 Resolution: Fixed Pushed to branch-2.4+. Thanks [~stoty]! > Specify -Xms for tests > -- > > Key: HBASE-28135 > URL: https://issues.apache.org/jira/browse/HBASE-28135 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1 > > > The default -Xms value is JVM dependent, but the host memory size is usually > included in the calculation. > -Xms in turn is used to calculate some GC parameters, for example NewSize and > OldSize, which affect the behaviour of tests. > As the memory consumption on the tests is not dependent on the host VM size, > we could set -Xms for the tests explictly, and enjoy more consistent test > results. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28135 Specify -Xms for tests [hbase]
petersomogyi merged PR #5451: URL: https://github.com/apache/hbase/pull/5451 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28140 AbstractWALProvider may miss the WAL which is under creat… [hbase]
Apache-HBase commented on PR #5455: URL: https://github.com/apache/hbase/pull/5455#issuecomment-1759078120 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 55s | master passed | | +1 :green_heart: | compile | 2m 27s | master passed | | +1 :green_heart: | checkstyle | 0m 34s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 34s | the patch passed | | +1 :green_heart: | compile | 2m 22s | the patch passed | | +1 :green_heart: | javac | 2m 22s | the patch passed | | +1 :green_heart: | checkstyle | 0m 33s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 26s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 38s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 31m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5455 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 60731907958b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 991d4aab51 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[
https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chaijunjie updated HBASE-28150:
---
Description:
create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then
will try again and again, will write too many proc record to master:store, we
find num of the master WAL in oldWALs more than 13000..
---
sorry, i upload screenshot failed, just copy to here
{code:java}
// 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 |
Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900)
2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created
at hdfs://hacluster/hbase/.tmp for table themis:a |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346)
2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create
table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159)
java.io.IOException: Unable to move table from
temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase
root=hdfs://hacluster/hbase/data/themis/a
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962)
at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988)
2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917,
state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure
table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102)
2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating
{ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME =>
'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '',
ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW',
IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE',
DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER',
MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536',
REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp |
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting
for flushes and compactions to finish for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total
wait time for flushes and compaction for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing
region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[
https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chaijunjie updated HBASE-28150:
---
Description:
create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then
will try again and again, will write too many proc record to master:store, we
find num of the master WAL in oldWALs more than 1300..
---
sorry, i upload screenshot failed, just copy to here
{code:java}
// 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 |
Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900)
2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created
at hdfs://hacluster/hbase/.tmp for table themis:a |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346)
2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create
table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159)
java.io.IOException: Unable to move table from
temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase
root=hdfs://hacluster/hbase/data/themis/a
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962)
at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988)
2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917,
state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure
table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102)
2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating
{ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME =>
'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '',
ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW',
IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE',
DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER',
MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536',
REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp |
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting
for flushes and compactions to finish for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total
wait time for flushes and compaction for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing
region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[
https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chaijunjie updated HBASE-28150:
---
Description:
create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then
will try again and again, will write too many proc record to master:store, we
find num of the master WAL in oldWALs more than 1300..
---
sorry, i upload screenshot failed, just copy to here
{code:java}
// 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 |
Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900)
2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created
at hdfs://hacluster/hbase/.tmp for table themis:a |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346)
2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create
table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159)
java.io.IOException: Unable to move table from
temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase
root=hdfs://hacluster/hbase/data/themis/a
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962)
at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988)
2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917,
state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure
table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102)
2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating
{ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME =>
'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '',
ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW',
IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE',
DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER',
MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536',
REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp |
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting
for flushes and compactions to finish for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total
wait time for flushes and compaction for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing
region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[
https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chaijunjie updated HBASE-28150:
---
Description:
create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then
will try again and again, will write too many proc record to master:store, we
find num of the master WAL in oldWALs more than 13000..
Q: should add a suspend time logic for create table proc retry? i see
TransitRegionStateProcedure has the logic..
---
sorry, i upload screenshot failed, just copy to here
{code:java}
// 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 |
Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900)
2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created
at hdfs://hacluster/hbase/.tmp for table themis:a |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346)
2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create
table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159)
java.io.IOException: Unable to move table from
temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase
root=hdfs://hacluster/hbase/data/themis/a
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962)
at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988)
2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917,
state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure
table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102)
2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating
{ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME =>
'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '',
ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW',
IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE',
DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER',
MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536',
REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp |
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting
for flushes and compactions to finish for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total
wait time for flushes and compaction for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing
region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,432 | INFO |
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[
https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chaijunjie updated HBASE-28150:
---
Description:
create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then
will try again and again, will write too many proc record to master:store, we
find num of the master WAL in oldWALs more than 13000..
Q: should add a suspend time logic for create table proc retry? i see
TransitRegionStateProcedure has the logic..
---
sorry, i upload screenshot failed, just copy to here
{code:java}
// 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 |
Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900)
2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created
at hdfs://hacluster/hbase/.tmp for table themis:a |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346)
2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create
table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159)
java.io.IOException: Unable to move table from
temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase
root=hdfs://hacluster/hbase/data/themis/a
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121)
at
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962)
at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988)
2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917,
state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure
table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT |
org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102)
2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating
{ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME =>
'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '',
ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW',
IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE',
DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER',
MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536',
REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp |
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting
for flushes and compactions to finish for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total
wait time for flushes and compaction for the region
themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms |
org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946)
2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing
region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. |
org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688)
2023-10-12 12:34:35,432 | INFO |
[jira] [Commented] (HBASE-27901) [Close-WAL-Writer-178] wal.AsyncFSWAL: close old writer failed
[ https://issues.apache.org/jira/browse/HBASE-27901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774341#comment-17774341 ] chaijunjie commented on HBASE-27901: When i use hbase 2.4, it also happpen... why we can not set hbase.wal.provider to multiwal? i also set it... > [Close-WAL-Writer-178] wal.AsyncFSWAL: close old writer failed > -- > > Key: HBASE-27901 > URL: https://issues.apache.org/jira/browse/HBASE-27901 > Project: HBase > Issue Type: Bug > Components: regionserver, wal >Affects Versions: 2.2.2 >Reporter: Jepson >Priority: Critical > > *HBase Log：* > 2023-06-02 08:07:57,423 INFO [Close-WAL-Writer-177] util.FSHDFSUtils: > Recover lease on dfs file > /hbase/WALs/bdpprd07,16020,1685646099569/bdpprd07%2C16020%2C1685646099569.1685664417370 > 2023-06-02 08:07:57,425 INFO [Close-WAL-Writer-177] util.FSHDFSUtils: Failed > to recover lease, attempt=0 on > file=/hbase/WALs/bdpprd07,16020,1685646099569/bdpprd07%2C16020%2C1685646099569.1685664417370 > after 2ms > 2023-06-02 08:08:01,427 WARN [Close-WAL-Writer-177] wal.AsyncFSWAL: close > old writer failed > java.io.FileNotFoundException: File does not exist: > /hbase/WALs/bdpprd07,16020,1685646099569/bdpprd07%2C16020%2C1685646099569.1685664417370 > at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:72) > at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:62) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLease(FSNamesystem.java:2358) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.recoverLease(NameNodeRpcServer.java:790) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.recoverLease(ClientNamenodeProtocolServerSideTranslatorPB.java:693) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:503) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:871) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:817) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2606) > at sun.reflect.GeneratedConstructorAccessor33.newInstance(Unknown > Source) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) > at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) > at org.apache.hadoop.hdfs.DFSClient.recoverLease(DFSClient.java:867) > at > org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:304) > at > org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:301) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.recoverLease(DistributedFileSystem.java:301) > at > org.apache.hadoop.hbase.util.FSHDFSUtils.recoverLease(FSHDFSUtils.java:283) > at > org.apache.hadoop.hbase.util.FSHDFSUtils.recoverDFSFileLease(FSHDFSUtils.java:216) > at > org.apache.hadoop.hbase.util.FSHDFSUtils.recoverFileLease(FSHDFSUtils.java:163) > at > org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutput.recoverAndClose(FanOutOneBlockAsyncDFSOutput.java:559) > at > org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.close(AsyncProtobufLogWriter.java:157) > at > org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.lambda$closeWriter$6(AsyncFSWAL.java:646) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: > org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): File > does not exist: > /hbase/WALs/bdpprd07,16020,1685646099569/bdpprd07%2C16020%2C1685646099569.1685664417370 > at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:72) > at >
[jira] [Commented] (HBASE-28133) TestSyncTimeRangeTracker fails with OOM with small -Xms values
[
https://issues.apache.org/jira/browse/HBASE-28133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774336#comment-17774336
]
Hudson commented on HBASE-28133:
Results for branch branch-2
[build #902 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/902/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/902/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/902/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/902/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/902/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestSyncTimeRangeTracker fails with OOM with small -Xms values
> --
>
> Key: HBASE-28133
> URL: https://issues.apache.org/jira/browse/HBASE-28133
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.17
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Labels: Arm64, test
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> Edit2: It's not the OS, it's the -Xms value determined from the host memory
> size.
> Edit: It's related to the OS and it's default java 8 , not to the processor
> architecture.
> This test seems to be cutting real close to the heap size.
> On ARM, it consistently fails on my RHEL8.8 Aarch64 VM with Java 8.
> {noformat}
> mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G
> -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server
> ...
> [ERROR]
> org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker.testConcurrentIncludeTimestampCorrectness
> Time elapsed: 1.969 s <<< ERROR!
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> It seems that Java on ARM has some higher memory overhead than x86_64.
> Simply bumping -Xmx from the default 2200m to 2300m allows it to pass.
> {noformat}
> mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G
> -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server
> -Dsurefire.Xmx=2300m
> ...
> [INFO] Running org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker
> [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.395
> s - in org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker
> {noformat}
> However, the real solution should be reducing the memory usage for this test.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28144) Canary publish read failure fails with NPE if region location is null
[
https://issues.apache.org/jira/browse/HBASE-28144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774335#comment-17774335
]
Hudson commented on HBASE-28144:
Results for branch branch-2
[build #902 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/902/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/902/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/902/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/902/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/902/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Canary publish read failure fails with NPE if region location is null
> -
>
> Key: HBASE-28144
> URL: https://issues.apache.org/jira/browse/HBASE-28144
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.5.5
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> Region with null server name causes canary failures while publishing read
> failure i.e. while updating perServerFailuresCount map:
> {code:java}
> 2023-10-09 15:24:11 [CanaryMonitor-1696864805801] ERROR tool.Canary(1480):
> Sniff region failed!
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.hadoop.hbase.tool.CanaryTool$RegionMonitor.run(CanaryTool.java:1478)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1837)
> at
> org.apache.hadoop.hbase.tool.CanaryTool$RegionStdOutSink.incFailuresCountDetails(CanaryTool.java:327)
> at
> org.apache.hadoop.hbase.tool.CanaryTool$RegionStdOutSink.publishReadFailure(CanaryTool.java:353)
> at
> org.apache.hadoop.hbase.tool.CanaryTool$RegionTask.readColumnFamily(CanaryTool.java:548)
> at
> org.apache.hadoop.hbase.tool.CanaryTool$RegionTask.read(CanaryTool.java:587)
> at
> org.apache.hadoop.hbase.tool.CanaryTool$RegionTask.call(CanaryTool.java:502)
> at
> org.apache.hadoop.hbase.tool.CanaryTool$RegionTask.call(CanaryTool.java:470)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache9 commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1759028250 This problem only affects 2.x? Master and branch-3 are safe? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28138) Make the connection idle timeout configurable for the embedded HTTP servers
[ https://issues.apache.org/jira/browse/HBASE-28138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-28138: - Fix Version/s: 4.0.0-alpha-1 (was: 3.0.0-beta-1) > Make the connection idle timeout configurable for the embedded HTTP servers > --- > > Key: HBASE-28138 > URL: https://issues.apache.org/jira/browse/HBASE-28138 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-4 >Reporter: qiuwei >Assignee: qiuwei >Priority: Minor > Fix For: 4.0.0-alpha-1 > > Attachments: > 0001-HBASE-28138-Slow-HTTP-Denial-of-Service-Attack-of-HB.patch > > > The Jetty Server may be slow HTTP denial of service attacks in hbase master > web ui due to the default value of jetty connector idle timeout is 3. > This may cause such an attack. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28138) Make the connection idle timeout configurable for the embedded HTTP servers
[ https://issues.apache.org/jira/browse/HBASE-28138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-28138: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the contribution [~qiuwei6]! I had now merged this into master branch. > Make the connection idle timeout configurable for the embedded HTTP servers > --- > > Key: HBASE-28138 > URL: https://issues.apache.org/jira/browse/HBASE-28138 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-4 >Reporter: qiuwei >Assignee: qiuwei >Priority: Minor > Fix For: 4.0.0-alpha-1 > > Attachments: > 0001-HBASE-28138-Slow-HTTP-Denial-of-Service-Attack-of-HB.patch > > > The Jetty Server may be slow HTTP denial of service attacks in hbase master > web ui due to the default value of jetty connector idle timeout is 3. > This may cause such an attack. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
bbeaudreault commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1759553079 > This problem only affects 2.x? Master and branch-3 are safe? It looks like master was fixed via your commit https://github.com/apache/hbase/commit/5d872d34225157bfbc18a6600c958c93d96636fd. Should we make a similar change here? Otherwise if we intend to just fix this bug here, then looking at ServerManager.getRsAdmin, it is not thread safe. We should probably use computeIfAbsent if the get returns null. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache9 commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1759622141 Ah, OK, we change to use async connection for master and branch-3... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache9 commented on code in PR #5461:
URL: https://github.com/apache/hbase/pull/5461#discussion_r1356835441
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java:
##
@@ -128,7 +128,8 @@ public class ServerManager {
* Map of admin interfaces per registered regionserver; these interfaces we
use to control
* regionservers out on the cluster
*/
- private final Map rsAdmins = new
HashMap<>();
+ private final Map rsAdmins =
Review Comment:
Better also change the declaration type to ConcurrentMap
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28140 AbstractWALProvider may miss the WAL which is under creat… [hbase]
Apache-HBase commented on PR #5455: URL: https://github.com/apache/hbase/pull/5455#issuecomment-1759381079 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 17s | master passed | | +1 :green_heart: | compile | 1m 2s | master passed | | +1 :green_heart: | shadedjars | 5m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 35s | the patch passed | | +1 :green_heart: | compile | 0m 42s | the patch passed | | +1 :green_heart: | javac | 0m 42s | the patch passed | | +1 :green_heart: | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 214m 21s | hbase-server in the patch failed. | | | | 237m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5455 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 89eb6b214063 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 991d4aab51 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/testReport/ | | Max. process+thread count | 4552 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28140 AbstractWALProvider may miss the WAL which is under creat… [hbase]
Apache-HBase commented on PR #5455: URL: https://github.com/apache/hbase/pull/5455#issuecomment-1759390434 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 26s | master passed | | +1 :green_heart: | compile | 0m 36s | master passed | | +1 :green_heart: | shadedjars | 5m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 14s | the patch passed | | +1 :green_heart: | compile | 0m 36s | the patch passed | | +1 :green_heart: | javac | 0m 36s | the patch passed | | +1 :green_heart: | shadedjars | 5m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 222m 56s | hbase-server in the patch failed. | | | | 244m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5455 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 497bcea4f47e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 991d4aab51 | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/testReport/ | | Max. process+thread count | 4637 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-26974 Introduce a LogRollProcedure [hbase]
Apache9 commented on code in PR #5408:
URL: https://github.com/apache/hbase/pull/5408#discussion_r1356842065
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/LogRollProcedure.java:
##
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.backup.master;
+
+import static
org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_DEFAULT;
+import static
org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_KEY;
+import static
org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED;
+import static
org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED_DEFAULT;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Optional;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.NamespaceDescriptor;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.TableNotEnabledException;
+import org.apache.hadoop.hbase.backup.impl.BackupSystemTable;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableState;
+import org.apache.hadoop.hbase.master.TableStateManager;
+import org.apache.hadoop.hbase.master.procedure.CreateNamespaceProcedure;
+import org.apache.hadoop.hbase.master.procedure.CreateTableProcedure;
+import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv;
+import org.apache.hadoop.hbase.master.procedure.TableProcedureInterface;
+import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer;
+import org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException;
+import org.apache.hadoop.hbase.procedure2.ProcedureYieldException;
+import org.apache.hadoop.hbase.procedure2.StateMachineProcedure;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.hadoop.hbase.util.ModifyRegionUtils;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollData;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollState;
+
+@InterfaceAudience.Private
+public class LogRollProcedure extends
StateMachineProcedure
+ implements TableProcedureInterface {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(RSLogRollProcedure.class);
+
+ private Configuration conf;
+
+ private String backupRoot;
+
+ public LogRollProcedure() {
+ }
+
+ public LogRollProcedure(final String backupRoot, final Configuration conf) {
+this.backupRoot = backupRoot;
+this.conf = conf;
+ }
+
+ @Override
+ protected Flow executeFromState(MasterProcedureEnv env, LogRollState state)
+throws ProcedureSuspendedException, ProcedureYieldException,
InterruptedException {
+LOG.info("{} execute state={}", this, state);
+
+try {
+ switch (state) {
+case LOG_ROLL_PRE_CHECK_NAMESPACE:
+ // create backup namespace if not exists
+ if (
+
!env.getMasterServices().getClusterSchema().getTableNamespaceManager()
+ .doesNamespaceExist(getTableName().getNamespaceAsString())
+ ) {
+NamespaceDescriptor desc =
+
NamespaceDescriptor.create(getTableName().getNamespaceAsString()).build();
+addChildProcedure(new CreateNamespaceProcedure(env, desc));
+ }
+ setNextState(LogRollState.LOG_ROLL_PRE_CHECK_TABLES);
+ return Flow.HAS_MORE_STATE;
+case LOG_ROLL_PRE_CHECK_TABLES:
+ // create backup system table and backup system bulkload table if
not exists
+ List toCreate = new ArrayList<>();
+ TableStateManager tsm =
env.getMasterServices().getTableStateManager();
+ createIfNeeded(env, tsm, BackupSystemTable.getTableName(conf),
+
BackupSystemTable.getSystemTableDescriptor(conf)).ifPresent(toCreate::add);
+ createIfNeeded(env, tsm,
Re: [PR] HBASE-26974 Introduce a LogRollProcedure [hbase]
Apache9 commented on code in PR #5408:
URL: https://github.com/apache/hbase/pull/5408#discussion_r1356848506
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/util/BackupUtils.java:
##
@@ -762,4 +773,30 @@ public static String findMostRecentBackupId(String[]
backupIds) {
return BackupRestoreConstants.BACKUPID_PREFIX + recentTimestamp;
}
+ public static void rollWALWriters(Admin admin, Map props)
throws IOException {
+byte[] ret =
Review Comment:
The execProcedure call is for zk based procedures, do we still have other
procedures besides the log roll one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache9 commented on code in PR #5461:
URL: https://github.com/apache/hbase/pull/5461#discussion_r1356884191
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java:
##
@@ -718,14 +720,19 @@ public static void
closeRegionSilentlyAndWait(ClusterConnection connection, Serv
public AdminService.BlockingInterface getRsAdmin(final ServerName sn) throws
IOException {
AdminService.BlockingInterface admin = this.rsAdmins.get(sn);
if (admin == null) {
- LOG.debug("New admin connection to " + sn.toString());
- if (sn.equals(master.getServerName()) && master instanceof
HRegionServer) {
-// A master is also a region server now, see HBASE-10569 for details
-admin = ((HRegionServer) master).getRSRpcServices();
- } else {
-admin = this.connection.getAdmin(sn);
- }
- this.rsAdmins.put(sn, admin);
+ return this.rsAdmins.computeIfAbsent(sn, server -> {
+LOG.debug("New admin connection to " + server.toString());
+if (server.equals(master.getServerName()) && master instanceof
HRegionServer) {
+ // A master is also a region server now, see HBASE-10569 for details
+ return ((HRegionServer) master).getRSRpcServices();
+} else {
+ try {
+return this.connection.getAdmin(server);
+ } catch (IOException e) {
+throw new RuntimeException(e);
Review Comment:
We should throw this IOException out? This is a problem of computeIfAbsent,
as the functional interface does not throw IOException...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
rmdmattingly commented on code in PR #5461:
URL: https://github.com/apache/hbase/pull/5461#discussion_r1356892819
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java:
##
@@ -718,14 +720,19 @@ public static void
closeRegionSilentlyAndWait(ClusterConnection connection, Serv
public AdminService.BlockingInterface getRsAdmin(final ServerName sn) throws
IOException {
AdminService.BlockingInterface admin = this.rsAdmins.get(sn);
if (admin == null) {
- LOG.debug("New admin connection to " + sn.toString());
- if (sn.equals(master.getServerName()) && master instanceof
HRegionServer) {
-// A master is also a region server now, see HBASE-10569 for details
-admin = ((HRegionServer) master).getRSRpcServices();
- } else {
-admin = this.connection.getAdmin(sn);
- }
- this.rsAdmins.put(sn, admin);
+ return this.rsAdmins.computeIfAbsent(sn, server -> {
+LOG.debug("New admin connection to " + server.toString());
+if (server.equals(master.getServerName()) && master instanceof
HRegionServer) {
+ // A master is also a region server now, see HBASE-10569 for details
+ return ((HRegionServer) master).getRSRpcServices();
+} else {
+ try {
+return this.connection.getAdmin(server);
+ } catch (IOException e) {
+throw new RuntimeException(e);
Review Comment:
Yeah, good point
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1759751872 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 49s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 55s | branch-2 passed | | +1 :green_heart: | compile | 2m 26s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 34s | branch-2 passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 27s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 39s | the patch passed | | +1 :green_heart: | compile | 2m 26s | the patch passed | | +1 :green_heart: | javac | 2m 26s | the patch passed | | +1 :green_heart: | checkstyle | 0m 34s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 58s | Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 32m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 416b440107d5 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28138) Make the connection idle timeout configurable for the embedded HTTP servers
[ https://issues.apache.org/jira/browse/HBASE-28138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774427#comment-17774427 ] Duo Zhang commented on HBASE-28138: --- This can not be applied to other branches? > Make the connection idle timeout configurable for the embedded HTTP servers > --- > > Key: HBASE-28138 > URL: https://issues.apache.org/jira/browse/HBASE-28138 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-4 >Reporter: qiuwei >Assignee: qiuwei >Priority: Minor > Fix For: 4.0.0-alpha-1 > > Attachments: > 0001-HBASE-28138-Slow-HTTP-Denial-of-Service-Attack-of-HB.patch > > > The Jetty Server may be slow HTTP denial of service attacks in hbase master > web ui due to the default value of jetty connector idle timeout is 3. > This may cause such an attack. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28140 AbstractWALProvider may miss the WAL which is under creat… [hbase]
Apache9 commented on PR #5455: URL: https://github.com/apache/hbase/pull/5455#issuecomment-1759397697 The failed UTs are not related. Not sure why TestZooKeeper can fail with time out... Saw it several times recently... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-26974 Introduce a LogRollProcedure [hbase]
Apache9 commented on code in PR #5408:
URL: https://github.com/apache/hbase/pull/5408#discussion_r1356843363
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/LogRollProcedure.java:
##
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.backup.master;
+
+import static
org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_DEFAULT;
+import static
org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_KEY;
+import static
org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED;
+import static
org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED_DEFAULT;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Optional;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.NamespaceDescriptor;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.TableNotEnabledException;
+import org.apache.hadoop.hbase.backup.impl.BackupSystemTable;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableState;
+import org.apache.hadoop.hbase.master.TableStateManager;
+import org.apache.hadoop.hbase.master.procedure.CreateNamespaceProcedure;
+import org.apache.hadoop.hbase.master.procedure.CreateTableProcedure;
+import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv;
+import org.apache.hadoop.hbase.master.procedure.TableProcedureInterface;
+import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer;
+import org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException;
+import org.apache.hadoop.hbase.procedure2.ProcedureYieldException;
+import org.apache.hadoop.hbase.procedure2.StateMachineProcedure;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.hadoop.hbase.util.ModifyRegionUtils;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollData;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollState;
+
+@InterfaceAudience.Private
+public class LogRollProcedure extends
StateMachineProcedure
+ implements TableProcedureInterface {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(RSLogRollProcedure.class);
+
+ private Configuration conf;
+
+ private String backupRoot;
+
+ public LogRollProcedure() {
+ }
+
+ public LogRollProcedure(final String backupRoot, final Configuration conf) {
+this.backupRoot = backupRoot;
+this.conf = conf;
+ }
+
+ @Override
+ protected Flow executeFromState(MasterProcedureEnv env, LogRollState state)
+throws ProcedureSuspendedException, ProcedureYieldException,
InterruptedException {
+LOG.info("{} execute state={}", this, state);
+
+try {
+ switch (state) {
+case LOG_ROLL_PRE_CHECK_NAMESPACE:
+ // create backup namespace if not exists
+ if (
+
!env.getMasterServices().getClusterSchema().getTableNamespaceManager()
+ .doesNamespaceExist(getTableName().getNamespaceAsString())
+ ) {
+NamespaceDescriptor desc =
+
NamespaceDescriptor.create(getTableName().getNamespaceAsString()).build();
+addChildProcedure(new CreateNamespaceProcedure(env, desc));
+ }
+ setNextState(LogRollState.LOG_ROLL_PRE_CHECK_TABLES);
+ return Flow.HAS_MORE_STATE;
+case LOG_ROLL_PRE_CHECK_TABLES:
+ // create backup system table and backup system bulkload table if
not exists
+ List toCreate = new ArrayList<>();
+ TableStateManager tsm =
env.getMasterServices().getTableStateManager();
+ createIfNeeded(env, tsm, BackupSystemTable.getTableName(conf),
+
BackupSystemTable.getSystemTableDescriptor(conf)).ifPresent(toCreate::add);
+ createIfNeeded(env, tsm,
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1760103112 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 21s | branch-2 passed | | +1 :green_heart: | compile | 0m 40s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 14s | the patch passed | | +1 :green_heart: | compile | 0m 40s | the patch passed | | +1 :green_heart: | javac | 0m 40s | the patch passed | | +1 :green_heart: | shadedjars | 4m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 207m 49s | hbase-server in the patch passed. | | | | 228m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 493f497d3c0f 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/testReport/ | | Max. process+thread count | 4540 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1760098522 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 47s | branch-2 passed | | +1 :green_heart: | compile | 0m 43s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 32s | the patch passed | | +1 :green_heart: | compile | 0m 43s | the patch passed | | +1 :green_heart: | javac | 0m 43s | the patch passed | | +1 :green_heart: | shadedjars | 5m 14s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 203m 0s | hbase-server in the patch passed. | | | | 225m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fa5a8c149301 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/testReport/ | | Max. process+thread count | 4656 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
bbeaudreault commented on code in PR #5461:
URL: https://github.com/apache/hbase/pull/5461#discussion_r1356965047
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java:
##
@@ -718,14 +720,19 @@ public static void
closeRegionSilentlyAndWait(ClusterConnection connection, Serv
public AdminService.BlockingInterface getRsAdmin(final ServerName sn) throws
IOException {
AdminService.BlockingInterface admin = this.rsAdmins.get(sn);
if (admin == null) {
- LOG.debug("New admin connection to " + sn.toString());
- if (sn.equals(master.getServerName()) && master instanceof
HRegionServer) {
-// A master is also a region server now, see HBASE-10569 for details
-admin = ((HRegionServer) master).getRSRpcServices();
- } else {
-admin = this.connection.getAdmin(sn);
- }
- this.rsAdmins.put(sn, admin);
+ return this.rsAdmins.computeIfAbsent(sn, server -> {
+LOG.debug("New admin connection to " + server.toString());
+if (server.equals(master.getServerName()) && master instanceof
HRegionServer) {
+ // A master is also a region server now, see HBASE-10569 for details
+ return ((HRegionServer) master).getRSRpcServices();
+} else {
+ try {
+return this.connection.getAdmin(server);
+ } catch (IOException e) {
+throw new RuntimeException(e);
Review Comment:
As written, it seems we need _some_ synchronization. A concurrent map is
pointless if you do unsyncrhonized writes on it. You could use
ConcurrentMapUtils.computeIfAbsentEx to get around the exception issue.
That said, looking at the impls of getRSRpcServices and getAdmin, I don't
think we really need to cache these at all. getRSRpcServices directly returns
an instance variable. getAdmin already does it's own caching.
So it might make sense to simply remove the rsAdmins map. @Apache9 what do
you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-26974 Introduce a LogRollProcedure [hbase]
frostruan commented on code in PR #5408:
URL: https://github.com/apache/hbase/pull/5408#discussion_r1357032646
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/LogRollProcedure.java:
##
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.backup.master;
+
+import static
org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_DEFAULT;
+import static
org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_KEY;
+import static
org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED;
+import static
org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED_DEFAULT;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Optional;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.NamespaceDescriptor;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.TableNotEnabledException;
+import org.apache.hadoop.hbase.backup.impl.BackupSystemTable;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableState;
+import org.apache.hadoop.hbase.master.TableStateManager;
+import org.apache.hadoop.hbase.master.procedure.CreateNamespaceProcedure;
+import org.apache.hadoop.hbase.master.procedure.CreateTableProcedure;
+import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv;
+import org.apache.hadoop.hbase.master.procedure.TableProcedureInterface;
+import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer;
+import org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException;
+import org.apache.hadoop.hbase.procedure2.ProcedureYieldException;
+import org.apache.hadoop.hbase.procedure2.StateMachineProcedure;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.hadoop.hbase.util.ModifyRegionUtils;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollData;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollState;
+
+@InterfaceAudience.Private
+public class LogRollProcedure extends
StateMachineProcedure
+ implements TableProcedureInterface {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(RSLogRollProcedure.class);
+
+ private Configuration conf;
+
+ private String backupRoot;
+
+ public LogRollProcedure() {
+ }
+
+ public LogRollProcedure(final String backupRoot, final Configuration conf) {
+this.backupRoot = backupRoot;
+this.conf = conf;
+ }
+
+ @Override
+ protected Flow executeFromState(MasterProcedureEnv env, LogRollState state)
+throws ProcedureSuspendedException, ProcedureYieldException,
InterruptedException {
+LOG.info("{} execute state={}", this, state);
+
+try {
+ switch (state) {
+case LOG_ROLL_PRE_CHECK_NAMESPACE:
+ // create backup namespace if not exists
+ if (
+
!env.getMasterServices().getClusterSchema().getTableNamespaceManager()
+ .doesNamespaceExist(getTableName().getNamespaceAsString())
+ ) {
+NamespaceDescriptor desc =
+
NamespaceDescriptor.create(getTableName().getNamespaceAsString()).build();
+addChildProcedure(new CreateNamespaceProcedure(env, desc));
+ }
+ setNextState(LogRollState.LOG_ROLL_PRE_CHECK_TABLES);
+ return Flow.HAS_MORE_STATE;
+case LOG_ROLL_PRE_CHECK_TABLES:
+ // create backup system table and backup system bulkload table if
not exists
+ List toCreate = new ArrayList<>();
+ TableStateManager tsm =
env.getMasterServices().getTableStateManager();
+ createIfNeeded(env, tsm, BackupSystemTable.getTableName(conf),
+
BackupSystemTable.getSystemTableDescriptor(conf)).ifPresent(toCreate::add);
+ createIfNeeded(env, tsm,
Re: [PR] HBASE-26974 Introduce a LogRollProcedure [hbase]
frostruan commented on code in PR #5408:
URL: https://github.com/apache/hbase/pull/5408#discussion_r1357033196
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/util/BackupUtils.java:
##
@@ -762,4 +773,30 @@ public static String findMostRecentBackupId(String[]
backupIds) {
return BackupRestoreConstants.BACKUPID_PREFIX + recentTimestamp;
}
+ public static void rollWALWriters(Admin admin, Map props)
throws IOException {
+byte[] ret =
Review Comment:
No, this is the last one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-28135) Specify -Xms for tests
[
https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774682#comment-17774682
]
Hudson commented on HBASE-28135:
Results for branch branch-2.5
[build #418 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Specify -Xms for tests
> --
>
> Key: HBASE-28135
> URL: https://issues.apache.org/jira/browse/HBASE-28135
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> The default -Xms value is JVM dependent, but the host memory size is usually
> included in the calculation.
> -Xms in turn is used to calculate some GC parameters, for example NewSize and
> OldSize, which affect the behaviour of tests.
> As the memory consumption on the tests is not dependent on the host VM size,
> we could set -Xms for the tests explictly, and enjoy more consistent test
> results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1760163936 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 46s | branch-2 passed | | +1 :green_heart: | compile | 2m 24s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 38s | branch-2 passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 30s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | the patch passed | | +1 :green_heart: | compile | 2m 24s | the patch passed | | +1 :green_heart: | javac | 2m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 33s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 20s | Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 43s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 31m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux c7e216628d56 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28135) Specify -Xms for tests
[
https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774653#comment-17774653
]
Hudson commented on HBASE-28135:
Results for branch master
[build #925 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Specify -Xms for tests
> --
>
> Key: HBASE-28135
> URL: https://issues.apache.org/jira/browse/HBASE-28135
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> The default -Xms value is JVM dependent, but the host memory size is usually
> included in the calculation.
> -Xms in turn is used to calculate some GC parameters, for example NewSize and
> OldSize, which affect the behaviour of tests.
> As the memory consumption on the tests is not dependent on the host VM size,
> we could set -Xms for the tests explictly, and enjoy more consistent test
> results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28133) TestSyncTimeRangeTracker fails with OOM with small -Xms values
[
https://issues.apache.org/jira/browse/HBASE-28133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774650#comment-17774650
]
Hudson commented on HBASE-28133:
Results for branch master
[build #925 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestSyncTimeRangeTracker fails with OOM with small -Xms values
> --
>
> Key: HBASE-28133
> URL: https://issues.apache.org/jira/browse/HBASE-28133
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.17
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Labels: Arm64, test
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> Edit2: It's not the OS, it's the -Xms value determined from the host memory
> size.
> Edit: It's related to the OS and it's default java 8 , not to the processor
> architecture.
> This test seems to be cutting real close to the heap size.
> On ARM, it consistently fails on my RHEL8.8 Aarch64 VM with Java 8.
> {noformat}
> mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G
> -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server
> ...
> [ERROR]
> org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker.testConcurrentIncludeTimestampCorrectness
> Time elapsed: 1.969 s <<< ERROR!
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> It seems that Java on ARM has some higher memory overhead than x86_64.
> Simply bumping -Xmx from the default 2200m to 2300m allows it to pass.
> {noformat}
> mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G
> -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server
> -Dsurefire.Xmx=2300m
> ...
> [INFO] Running org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker
> [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.395
> s - in org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker
> {noformat}
> However, the real solution should be reducing the memory usage for this test.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28148) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit
[
https://issues.apache.org/jira/browse/HBASE-28148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774652#comment-17774652
]
Hudson commented on HBASE-28148:
Results for branch master
[build #925 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit
> ---
>
> Key: HBASE-28148
> URL: https://issues.apache.org/jira/browse/HBASE-28148
> Project: HBase
> Issue Type: Task
> Components: dependabot, scripts, security
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 4.0.0-alpha-1
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[
https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774651#comment-17774651
]
Hudson commented on HBASE-28147:
Results for branch master
[build #925 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
>
>
> Key: HBASE-28147
> URL: https://issues.apache.org/jira/browse/HBASE-28147
> Project: HBase
> Issue Type: Task
> Components: dependabot, scripts, security
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[
https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774681#comment-17774681
]
Hudson commented on HBASE-28147:
Results for branch branch-2.5
[build #418 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
>
>
> Key: HBASE-28147
> URL: https://issues.apache.org/jira/browse/HBASE-28147
> Project: HBase
> Issue Type: Task
> Components: dependabot, scripts, security
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28133) TestSyncTimeRangeTracker fails with OOM with small -Xms values
[
https://issues.apache.org/jira/browse/HBASE-28133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774680#comment-17774680
]
Hudson commented on HBASE-28133:
Results for branch branch-2.5
[build #418 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestSyncTimeRangeTracker fails with OOM with small -Xms values
> --
>
> Key: HBASE-28133
> URL: https://issues.apache.org/jira/browse/HBASE-28133
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.17
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Labels: Arm64, test
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> Edit2: It's not the OS, it's the -Xms value determined from the host memory
> size.
> Edit: It's related to the OS and it's default java 8 , not to the processor
> architecture.
> This test seems to be cutting real close to the heap size.
> On ARM, it consistently fails on my RHEL8.8 Aarch64 VM with Java 8.
> {noformat}
> mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G
> -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server
> ...
> [ERROR]
> org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker.testConcurrentIncludeTimestampCorrectness
> Time elapsed: 1.969 s <<< ERROR!
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> It seems that Java on ARM has some higher memory overhead than x86_64.
> Simply bumping -Xmx from the default 2200m to 2300m allows it to pass.
> {noformat}
> mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G
> -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server
> -Dsurefire.Xmx=2300m
> ...
> [INFO] Running org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker
> [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.395
> s - in org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker
> {noformat}
> However, the real solution should be reducing the memory usage for this test.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28135) Specify -Xms for tests
[
https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774678#comment-17774678
]
Hudson commented on HBASE-28135:
Results for branch branch-3
[build #65 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Specify -Xms for tests
> --
>
> Key: HBASE-28135
> URL: https://issues.apache.org/jira/browse/HBASE-28135
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> The default -Xms value is JVM dependent, but the host memory size is usually
> included in the calculation.
> -Xms in turn is used to calculate some GC parameters, for example NewSize and
> OldSize, which affect the behaviour of tests.
> As the memory consumption on the tests is not dependent on the host VM size,
> we could set -Xms for the tests explictly, and enjoy more consistent test
> results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28133) TestSyncTimeRangeTracker fails with OOM with small -Xms values
[
https://issues.apache.org/jira/browse/HBASE-28133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774676#comment-17774676
]
Hudson commented on HBASE-28133:
Results for branch branch-3
[build #65 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestSyncTimeRangeTracker fails with OOM with small -Xms values
> --
>
> Key: HBASE-28133
> URL: https://issues.apache.org/jira/browse/HBASE-28133
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.17
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Labels: Arm64, test
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> Edit2: It's not the OS, it's the -Xms value determined from the host memory
> size.
> Edit: It's related to the OS and it's default java 8 , not to the processor
> architecture.
> This test seems to be cutting real close to the heap size.
> On ARM, it consistently fails on my RHEL8.8 Aarch64 VM with Java 8.
> {noformat}
> mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G
> -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server
> ...
> [ERROR]
> org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker.testConcurrentIncludeTimestampCorrectness
> Time elapsed: 1.969 s <<< ERROR!
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> It seems that Java on ARM has some higher memory overhead than x86_64.
> Simply bumping -Xmx from the default 2200m to 2300m allows it to pass.
> {noformat}
> mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G
> -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server
> -Dsurefire.Xmx=2300m
> ...
> [INFO] Running org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker
> [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.395
> s - in org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker
> {noformat}
> However, the real solution should be reducing the memory usage for this test.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[
https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774677#comment-17774677
]
Hudson commented on HBASE-28147:
Results for branch branch-3
[build #65 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
>
>
> Key: HBASE-28147
> URL: https://issues.apache.org/jira/browse/HBASE-28147
> Project: HBase
> Issue Type: Task
> Components: dependabot, scripts, security
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1760409599 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 11s | branch-2 passed | | +1 :green_heart: | compile | 0m 40s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 11s | the patch passed | | +1 :green_heart: | compile | 0m 40s | the patch passed | | +1 :green_heart: | javac | 0m 40s | the patch passed | | +1 :green_heart: | shadedjars | 4m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 208m 20s | hbase-server in the patch passed. | | | | 228m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b63454fe1b7a 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/testReport/ | | Max. process+thread count | 4355 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1760443461 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 40s | branch-2 passed | | +1 :green_heart: | compile | 0m 43s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 30s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | shadedjars | 5m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 227m 25s | hbase-server in the patch failed. | | | | 250m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b9f01b6807d3 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/testReport/ | | Max. process+thread count | 4800 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25222) Add a cost function to move the daughter regions of a recent split to different region servers
[ https://issues.apache.org/jira/browse/HBASE-25222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774707#comment-17774707 ] David Manning commented on HBASE-25222: --- Seems like in hbase 2.x, this is already less of an issue, because the SplitTableRegionProcedure will choose new servers to open the daughter regions. In hbase 1.x, the daughter regions would open on the same server as the parent region. > Add a cost function to move the daughter regions of a recent split to > different region servers > --- > > Key: HBASE-25222 > URL: https://issues.apache.org/jira/browse/HBASE-25222 > Project: HBase > Issue Type: Improvement >Reporter: Sandeep Pal >Assignee: Sandeep Pal >Priority: Major > > In HBase, hotspot regions are easily formed whenever there is skew and there > is high write volume. Few regions grow really fast which also becomes the > bottleneck on the few region servers. > It would be beneficial to add a cost function to move the regions after the > split to differetn region servers. In this way the writes to hot key range > will be distributed to multiple region servers. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28151) hbck -o should not allow bypassing pre transit check by default
Viraj Jasani created HBASE-28151:
Summary: hbck -o should not allow bypassing pre transit check by
default
Key: HBASE-28151
URL: https://issues.apache.org/jira/browse/HBASE-28151
Project: HBase
Issue Type: Improvement
Reporter: Viraj Jasani
When operator uses hbck assigns or unassigns with "-o", the override will also
skip pre transit checks. While this is one of the intentions with "-o", the
primary purpose should still be to only unattach existing procedure from
RegionStateNode so that newly scheduled assign proc can take exclusive region
level lock.
We should restrict bypassing preTransitCheck by only providing it as site
config.
If bypassing preTransitCheck is configured, only then any hbck "-o" should be
allowed to bypass this check, otherwise by default they should go through the
check.
It is important to keep "unset of the procedure from RegionStateNode" and
"bypassing preTransitCheck" separate so that when the cluster state is bad, we
don't explicitly deteriorate it further e.g. if a region was successfully split
and now if operator performs "hbck assigns \{region} -o" and if it bypasses the
transit check, master would bring the region online and it could compact store
files and archive the store file which is referenced by daughter region. This
would not allow daughter region to come online.
Let's introduce hbase site config to allow bypassing preTransitCheck, it should
not be doable only by operator using hbck alone.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28151) hbck -o should not allow bypassing pre transit check by default
[
https://issues.apache.org/jira/browse/HBASE-28151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated HBASE-28151:
-
Affects Version/s: 2.5.5
2.4.17
> hbck -o should not allow bypassing pre transit check by default
> ---
>
> Key: HBASE-28151
> URL: https://issues.apache.org/jira/browse/HBASE-28151
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.4.17, 2.5.5
>Reporter: Viraj Jasani
>Priority: Major
>
> When operator uses hbck assigns or unassigns with "-o", the override will
> also skip pre transit checks. While this is one of the intentions with "-o",
> the primary purpose should still be to only unattach existing procedure from
> RegionStateNode so that newly scheduled assign proc can take exclusive region
> level lock.
> We should restrict bypassing preTransitCheck by only providing it as site
> config.
> If bypassing preTransitCheck is configured, only then any hbck "-o" should be
> allowed to bypass this check, otherwise by default they should go through the
> check.
>
> It is important to keep "unset of the procedure from RegionStateNode" and
> "bypassing preTransitCheck" separate so that when the cluster state is bad,
> we don't explicitly deteriorate it further e.g. if a region was successfully
> split and now if operator performs "hbck assigns \{region} -o" and if it
> bypasses the transit check, master would bring the region online and it could
> compact store files and archive the store file which is referenced by
> daughter region. This would not allow daughter region to come online.
> Let's introduce hbase site config to allow bypassing preTransitCheck, it
> should not be doable only by operator using hbck alone.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28151) hbck -o should not allow bypassing pre transit check by default
[
https://issues.apache.org/jira/browse/HBASE-28151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated HBASE-28151:
-
Description:
When operator uses hbck assigns or unassigns with "-o", the override will also
skip pre transit checks. While this is one of the intentions with "-o", the
primary purpose should still be to only unattach existing procedure from
RegionStateNode so that newly scheduled assign proc can take exclusive region
level lock.
We should restrict bypassing preTransitCheck by only providing it as site
config.
If bypassing preTransitCheck is configured, only then any hbck "-o" should be
allowed to bypass this check, otherwise by default they should go through the
check.
It is important to keep "unset of the procedure from RegionStateNode" and
"bypassing preTransitCheck" separate so that when the cluster state is bad, we
don't explicitly deteriorate it further e.g. if a region was successfully split
and now if operator performs "hbck assigns \{region} -o" and if it bypasses the
transit check, master would bring the region online and it could compact store
files and archive the store file which is referenced by daughter region. This
would not allow daughter region to come online.
Let's introduce hbase site config to allow bypassing preTransitCheck, it should
not be doable only by operator using hbck alone.
"-o" should mean "override" the procedure that is attached to the
RegionStateNode, it should not mean forcefully skip any region transition
validation checks and perform the region assignments.
was:
When operator uses hbck assigns or unassigns with "-o", the override will also
skip pre transit checks. While this is one of the intentions with "-o", the
primary purpose should still be to only unattach existing procedure from
RegionStateNode so that newly scheduled assign proc can take exclusive region
level lock.
We should restrict bypassing preTransitCheck by only providing it as site
config.
If bypassing preTransitCheck is configured, only then any hbck "-o" should be
allowed to bypass this check, otherwise by default they should go through the
check.
It is important to keep "unset of the procedure from RegionStateNode" and
"bypassing preTransitCheck" separate so that when the cluster state is bad, we
don't explicitly deteriorate it further e.g. if a region was successfully split
and now if operator performs "hbck assigns \{region} -o" and if it bypasses the
transit check, master would bring the region online and it could compact store
files and archive the store file which is referenced by daughter region. This
would not allow daughter region to come online.
Let's introduce hbase site config to allow bypassing preTransitCheck, it should
not be doable only by operator using hbck alone.
> hbck -o should not allow bypassing pre transit check by default
> ---
>
> Key: HBASE-28151
> URL: https://issues.apache.org/jira/browse/HBASE-28151
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.4.17, 2.5.5
>Reporter: Viraj Jasani
>Priority: Major
>
> When operator uses hbck assigns or unassigns with "-o", the override will
> also skip pre transit checks. While this is one of the intentions with "-o",
> the primary purpose should still be to only unattach existing procedure from
> RegionStateNode so that newly scheduled assign proc can take exclusive region
> level lock.
> We should restrict bypassing preTransitCheck by only providing it as site
> config.
> If bypassing preTransitCheck is configured, only then any hbck "-o" should be
> allowed to bypass this check, otherwise by default they should go through the
> check.
>
> It is important to keep "unset of the procedure from RegionStateNode" and
> "bypassing preTransitCheck" separate so that when the cluster state is bad,
> we don't explicitly deteriorate it further e.g. if a region was successfully
> split and now if operator performs "hbck assigns \{region} -o" and if it
> bypasses the transit check, master would bring the region online and it could
> compact store files and archive the store file which is referenced by
> daughter region. This would not allow daughter region to come online.
> Let's introduce hbase site config to allow bypassing preTransitCheck, it
> should not be doable only by operator using hbck alone.
>
> "-o" should mean "override" the procedure that is attached to the
> RegionStateNode, it should not mean forcefully skip any region transition
> validation checks and perform the region assignments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28151) hbck -o should not allow bypassing pre transit check by default
[
https://issues.apache.org/jira/browse/HBASE-28151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated HBASE-28151:
-
Description:
When operator uses hbck assigns or unassigns with "-o", the override will also
skip pre transit checks. While this is one of the intentions with "-o", the
primary purpose should still be to only unattach existing procedure from
RegionStateNode so that newly scheduled assign proc can take exclusive region
level lock.
We should restrict bypassing preTransitCheck by only providing it as site
config.
If bypassing preTransitCheck is configured, only then any hbck "-o" should be
allowed to bypass this check, otherwise by default they should go through the
check.
It is important to keep "unset of the procedure from RegionStateNode" and
"bypassing preTransitCheck" separate so that when the cluster state is bad, we
don't explicitly deteriorate it further e.g. if a region was successfully split
and now if operator performs "hbck assigns \{region} -o" and if it bypasses the
transit check, master would bring the region online and it could compact store
files and archive the store file which is referenced by daughter region. This
would not allow daughter region to come online.
Let's introduce hbase site config to allow bypassing preTransitCheck, it should
not be doable only by operator using hbck alone.
"-o" should mean "override" the procedure that is attached to the
RegionStateNode, it should not mean forcefully skip any region transition
validation checks.
was:
When operator uses hbck assigns or unassigns with "-o", the override will also
skip pre transit checks. While this is one of the intentions with "-o", the
primary purpose should still be to only unattach existing procedure from
RegionStateNode so that newly scheduled assign proc can take exclusive region
level lock.
We should restrict bypassing preTransitCheck by only providing it as site
config.
If bypassing preTransitCheck is configured, only then any hbck "-o" should be
allowed to bypass this check, otherwise by default they should go through the
check.
It is important to keep "unset of the procedure from RegionStateNode" and
"bypassing preTransitCheck" separate so that when the cluster state is bad, we
don't explicitly deteriorate it further e.g. if a region was successfully split
and now if operator performs "hbck assigns \{region} -o" and if it bypasses the
transit check, master would bring the region online and it could compact store
files and archive the store file which is referenced by daughter region. This
would not allow daughter region to come online.
Let's introduce hbase site config to allow bypassing preTransitCheck, it should
not be doable only by operator using hbck alone.
"-o" should mean "override" the procedure that is attached to the
RegionStateNode, it should not mean forcefully skip any region transition
validation checks and perform the region assignments.
> hbck -o should not allow bypassing pre transit check by default
> ---
>
> Key: HBASE-28151
> URL: https://issues.apache.org/jira/browse/HBASE-28151
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.4.17, 2.5.5
>Reporter: Viraj Jasani
>Priority: Major
>
> When operator uses hbck assigns or unassigns with "-o", the override will
> also skip pre transit checks. While this is one of the intentions with "-o",
> the primary purpose should still be to only unattach existing procedure from
> RegionStateNode so that newly scheduled assign proc can take exclusive region
> level lock.
> We should restrict bypassing preTransitCheck by only providing it as site
> config.
> If bypassing preTransitCheck is configured, only then any hbck "-o" should be
> allowed to bypass this check, otherwise by default they should go through the
> check.
>
> It is important to keep "unset of the procedure from RegionStateNode" and
> "bypassing preTransitCheck" separate so that when the cluster state is bad,
> we don't explicitly deteriorate it further e.g. if a region was successfully
> split and now if operator performs "hbck assigns \{region} -o" and if it
> bypasses the transit check, master would bring the region online and it could
> compact store files and archive the store file which is referenced by
> daughter region. This would not allow daughter region to come online.
> Let's introduce hbase site config to allow bypassing preTransitCheck, it
> should not be doable only by operator using hbck alone.
>
> "-o" should mean "override" the procedure that is attached to the
> RegionStateNode, it should not mean forcefully skip any region transition
> validation checks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28135) Specify -Xms for tests
[
https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774781#comment-17774781
]
Hudson commented on HBASE-28135:
Results for branch branch-2.4
[build #635 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Specify -Xms for tests
> --
>
> Key: HBASE-28135
> URL: https://issues.apache.org/jira/browse/HBASE-28135
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> The default -Xms value is JVM dependent, but the host memory size is usually
> included in the calculation.
> -Xms in turn is used to calculate some GC parameters, for example NewSize and
> OldSize, which affect the behaviour of tests.
> As the memory consumption on the tests is not dependent on the host VM size,
> we could set -Xms for the tests explictly, and enjoy more consistent test
> results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28135) Specify -Xms for tests
[
https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774787#comment-17774787
]
Hudson commented on HBASE-28135:
Results for branch branch-2
[build #903 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Specify -Xms for tests
> --
>
> Key: HBASE-28135
> URL: https://issues.apache.org/jira/browse/HBASE-28135
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
>
>
> The default -Xms value is JVM dependent, but the host memory size is usually
> included in the calculation.
> -Xms in turn is used to calculate some GC parameters, for example NewSize and
> OldSize, which affect the behaviour of tests.
> As the memory consumption on the tests is not dependent on the host VM size,
> we could set -Xms for the tests explictly, and enjoy more consistent test
> results.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[
https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774786#comment-17774786
]
Hudson commented on HBASE-28147:
Results for branch branch-2
[build #903 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
>
>
> Key: HBASE-28147
> URL: https://issues.apache.org/jira/browse/HBASE-28147
> Project: HBase
> Issue Type: Task
> Components: dependabot, scripts, security
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[
https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774780#comment-17774780
]
Hudson commented on HBASE-28147:
Results for branch branch-2.4
[build #635 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
>
>
> Key: HBASE-28147
> URL: https://issues.apache.org/jira/browse/HBASE-28147
> Project: HBase
> Issue Type: Task
> Components: dependabot, scripts, security
>Reporter: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
