[jira] [Commented] (HBASE-27365) Minimise block addition failures due to no space in bucket cache writers queue by introducing wait time
[
https://issues.apache.org/jira/browse/HBASE-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613305#comment-17613305
]
Hudson commented on HBASE-27365:
Results for branch branch-2
[build #657 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/657/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/657/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/657/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/657/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/657/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Minimise block addition failures due to no space in bucket cache writers
> queue by introducing wait time
> ---
>
> Key: HBASE-27365
> URL: https://issues.apache.org/jira/browse/HBASE-27365
> Project: HBase
> Issue Type: Improvement
> Components: BucketCache
>Affects Versions: 3.0.0-alpha-3
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> Currently in bucket cache asynchronous caching mechanism introduced where
> initially the blocks to be cached will be added to queue and writer threads
> consume the blocks from the queue and write to bucket cache. In case if block
> writing to bucket cache is slow then there is a chance that queue of writer
> threads become full and following block additions will be failed. In case of
> slower storages like s3 might introduce latencies even if we enable bigger
> sizes of bucket cache using ephemeral storages. So we can allow configurable
> wait time while adding blocks to queue so that chances of queue free up is
> possible during the wait time and block addition failures can be minimised.
> To avoid the performance impact of wait time in regular read paths we can use
> the wait time mainly during background operations like compactions, flushes
> or prefetches etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #4817: HBASE-27411 Update and clean up bcprov-jdk15on dependency
Apache-HBase commented on PR #4817: URL: https://github.com/apache/hbase/pull/4817#issuecomment-1269212231 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 20s | master passed | | +1 :green_heart: | compile | 5m 59s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 5s | the patch passed | | +1 :green_heart: | compile | 5m 56s | the patch passed | | -0 :warning: | javac | 5m 56s | root generated 3 new + 706 unchanged - 3 fixed = 709 total (was 709) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 8m 9s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 37s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 32m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4817/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4817 | | Optional Tests | dupname asflicense javac hadoopcheck spotless xml compile | | uname | Linux 662828bc5516 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / eb6b2745c5 | | Default Java | Temurin-1.8.0_345-b01 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4817/1/artifact/yetus-general-check/output/diff-compile-javac-root.txt | | Max. process+thread count | 138 (vs. ulimit of 3) | | modules | C: hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4817/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27411) Update and clean up bcprov-jdk15on dependency
[
https://issues.apache.org/jira/browse/HBASE-27411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-27411:
Fix Version/s: 2.6.0
2.5.1
3.0.0-alpha-4
Status: Patch Available (was: Open)
> Update and clean up bcprov-jdk15on dependency
> --
>
> Key: HBASE-27411
> URL: https://issues.apache.org/jira/browse/HBASE-27411
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 3.0.0-alpha-4
>Reporter: Nick Dimiduk
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4
>
>
> {noformat}
> $ env
> JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home
> mvn -T0.5C clean install -DskipTests
>
> [INFO] Scanning for projects...
>
>
> [WARNING]
>
>
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.hbase:hbase-server:jar:3.0.0-alpha-4-SNAPSHOT
>
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
> be unique: org.bouncycastle:bcprov-jdk15on:jar -> duplicate declaration of
> version (?) @ line 331, column 17
> [WARNING]
>
>
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
>
> [WARNING]
>
>
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
>
> [WARNING]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] apurtell commented on a diff in pull request #4770: HBASE-27238 Backport backup restore to 2.x
apurtell commented on code in PR #4770:
URL: https://github.com/apache/hbase/pull/4770#discussion_r988469107
##
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java:
##
@@ -254,16 +255,19 @@ public void write(ImmutableBytesWritable row, V cell)
throws IOException {
byte[] tableNameBytes = null;
if (writeMultipleTables) {
tableNameBytes = MultiTableHFileOutputFormat.getTableName(row.get());
- tableNameBytes = TableName.valueOf(tableNameBytes).toBytes();
+ tableNameBytes =
TableName.valueOf(tableNameBytes).getNameWithNamespaceInclAsString()
Review Comment:
Applications (HFile readers) that read things written by HFileOutputFormat2
may need to change. There is a path building semantic change implied by
`getNameWithNamespaceInclAsString` being necessary now for something introduced
by this patch. Is this a compatibility problem? Are these changes necessary?
##
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java:
##
@@ -651,15 +651,18 @@ private void doIncrementalLoadTest(boolean
shouldChangeRegions, boolean shouldKe
Path testDir = util.getDataTestDirOnTestFS("testLocalMRIncrementalLoad");
// Generate the bulk load files
runIncrementalPELoad(conf, tableInfo, testDir, putSortReducer);
+if (writeMultipleTables) {
Review Comment:
This is what I mean. The test has to be updated, so readers/consumers of
HFileOutputFormat2 produced output will need to change too? Does this fall
under the operational compatibility guidelines?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-27411) Update and clean up bcprov-jdk15on dependency
[
https://issues.apache.org/jira/browse/HBASE-27411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-27411:
Summary: Update and clean up bcprov-jdk15on dependency (was: Dependency
on org.bouncycastle:bcprov-jdk15on:jar define multiple times)
> Update and clean up bcprov-jdk15on dependency
> --
>
> Key: HBASE-27411
> URL: https://issues.apache.org/jira/browse/HBASE-27411
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 3.0.0-alpha-4
>Reporter: Nick Dimiduk
>Assignee: Andrew Kyle Purtell
>Priority: Minor
>
> {noformat}
> $ env
> JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home
> mvn -T0.5C clean install -DskipTests
>
> [INFO] Scanning for projects...
>
>
> [WARNING]
>
>
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.hbase:hbase-server:jar:3.0.0-alpha-4-SNAPSHOT
>
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
> be unique: org.bouncycastle:bcprov-jdk15on:jar -> duplicate declaration of
> version (?) @ line 331, column 17
> [WARNING]
>
>
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
>
> [WARNING]
>
>
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
>
> [WARNING]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-27411) Dependency on org.bouncycastle:bcprov-jdk15on:jar define multiple times
[
https://issues.apache.org/jira/browse/HBASE-27411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell reassigned HBASE-27411:
---
Assignee: Andrew Kyle Purtell
> Dependency on org.bouncycastle:bcprov-jdk15on:jar define multiple times
>
>
> Key: HBASE-27411
> URL: https://issues.apache.org/jira/browse/HBASE-27411
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 3.0.0-alpha-4
>Reporter: Nick Dimiduk
>Assignee: Andrew Kyle Purtell
>Priority: Minor
>
> {noformat}
> $ env
> JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home
> mvn -T0.5C clean install -DskipTests
>
> [INFO] Scanning for projects...
>
>
> [WARNING]
>
>
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.hbase:hbase-server:jar:3.0.0-alpha-4-SNAPSHOT
>
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
> be unique: org.bouncycastle:bcprov-jdk15on:jar -> duplicate declaration of
> version (?) @ line 331, column 17
> [WARNING]
>
>
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
>
> [WARNING]
>
>
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
>
> [WARNING]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27411) Dependency on org.bouncycastle:bcprov-jdk15on:jar define multiple times
[
https://issues.apache.org/jira/browse/HBASE-27411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613268#comment-17613268
]
Andrew Kyle Purtell commented on HBASE-27411:
-
We also should bring the version up to 1.70 or later because of
[https://security.snyk.io/package/maven/org.bouncycastle:bcprov-jdk15on]
("latest non-vulnerable version 1.70")
> Dependency on org.bouncycastle:bcprov-jdk15on:jar define multiple times
>
>
> Key: HBASE-27411
> URL: https://issues.apache.org/jira/browse/HBASE-27411
> Project: HBase
> Issue Type: Task
> Components: build
>Affects Versions: 3.0.0-alpha-4
>Reporter: Nick Dimiduk
>Priority: Minor
>
> {noformat}
> $ env
> JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home
> mvn -T0.5C clean install -DskipTests
>
> [INFO] Scanning for projects...
>
>
> [WARNING]
>
>
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.hbase:hbase-server:jar:3.0.0-alpha-4-SNAPSHOT
>
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
> be unique: org.bouncycastle:bcprov-jdk15on:jar -> duplicate declaration of
> version (?) @ line 331, column 17
> [WARNING]
>
>
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
>
> [WARNING]
>
>
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
>
> [WARNING]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-25720) Sync WAL stuck when prepare flush cache will prevent flush cache and cause OOM
[ https://issues.apache.org/jira/browse/HBASE-25720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613267#comment-17613267 ] jason commented on HBASE-25720: --- confused about the status, status is 'Resolved' but resolution is "Won't Fix', i check the github commit, it doesn't accepted. interesting. > Sync WAL stuck when prepare flush cache will prevent flush cache and cause OOM > -- > > Key: HBASE-25720 > URL: https://issues.apache.org/jira/browse/HBASE-25720 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.13 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: prepare-flush-cache-stuck.png > > > We call HRegion#doSyncOfUnflushedWALChanges when preparing to flush cache. > But this WAL sync may stuck, and abort the flush of cache. > !prepare-flush-cache-stuck.png|width=519,height=246! > If we cannot aware of this problem in time, RS will OOM kill. > I think we should force abort RS when sync stuck in preparing, like in > committing snapshots. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] apurtell commented on pull request #4788: HBASE-21521 Expose master startup status via web UI
apurtell commented on PR #4788: URL: https://github.com/apache/hbase/pull/4788#issuecomment-1269148957 > From my perspective, the TaskGroup can be seen as a big MonitoredTask, which contains a list of sub-monitoredTasks. The monitored tasks in the group are still be managed by the TaskMonitor, but whether to clear/expire the monitored tasks in a task group is optional. Since the monitored task already has journals, which mark the phases in a task, we still also need a task group to monitor a big task/process because I think the journals in a task is serial but the tasks in the task group can be parallel(though haven't yet), then we have more flexible ability to monitor the process. That's the idea, yes. Grouping the tasks is not strictly necessary but in my opinion is cleaner for presentation to operators, and also is similar to how Hadoop designed the startup page for the NameNode. We might want to display the tasks in a group in a list view where each task can be collapsed (probably by default) or expanded. See HDFS-4372 and HDFS-4374, upon which our issue was originally inspired. In https://img-blog.csdnimg.cn/929a86ac0f1748f5af70d27ce14f40ee.png "Loading image" is a task group, and "Loading edits" is another task group. Although not strictly required this kind of grouping can help keep the status page layout clean. If for example in our startup status we have a task group "Deploying " for each table, the individual items in the group would be the TRSP for each region of , and there might be a lot of them, so collapsing the group detail by default would be good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25720) Sync WAL stuck when prepare flush cache will prevent flush cache and cause OOM
[ https://issues.apache.org/jira/browse/HBASE-25720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613248#comment-17613248 ] March Wang commented on HBASE-25720: Hi [~Xiaolin Ha], Could you please let me know how to fix this issue? Thanks! > Sync WAL stuck when prepare flush cache will prevent flush cache and cause OOM > -- > > Key: HBASE-25720 > URL: https://issues.apache.org/jira/browse/HBASE-25720 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.13 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: prepare-flush-cache-stuck.png > > > We call HRegion#doSyncOfUnflushedWALChanges when preparing to flush cache. > But this WAL sync may stuck, and abort the flush of cache. > !prepare-flush-cache-stuck.png|width=519,height=246! > If we cannot aware of this problem in time, RS will OOM kill. > I think we should force abort RS when sync stuck in preparing, like in > committing snapshots. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27414) Search order for locations in HFileLink
Huaxiang Sun created HBASE-27414: Summary: Search order for locations in HFileLink Key: HBASE-27414 URL: https://issues.apache.org/jira/browse/HBASE-27414 Project: HBase Issue Type: Improvement Components: Performance Reporter: Huaxiang Sun Found that search order for locations is following the order of these locations added to HFileLink object. setLocations(originPath, tempPath, mobPath, archivePath); archivePath is the last one to be searched. For most cases, hfile exists in archivePath, so we can move archivePath to the first parameter to avoid unnecessary NN query. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #4813: HBASE-27381 Still seeing 'Stuck' in static initialization creating RegionInfo instance
Apache-HBase commented on PR #4813: URL: https://github.com/apache/hbase/pull/4813#issuecomment-1265827595 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 39s | master passed | | +1 :green_heart: | compile | 4m 41s | master passed | | +1 :green_heart: | checkstyle | 1m 13s | master passed | | +1 :green_heart: | spotless | 0m 57s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 46s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 20s | the patch passed | | +1 :green_heart: | compile | 4m 20s | the patch passed | | +1 :green_heart: | javac | 4m 20s | the patch passed | | +1 :green_heart: | checkstyle | 0m 50s | the patch passed | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 10m 19s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 57s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 49s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 45m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4813/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4813 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux d5a847db3c51 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6abc1aefd1 | | Default Java | Temurin-1.8.0_345-b01 | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4813/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27381) Still seeing 'Stuck' in static initialization creating RegionInfo instance
[
https://issues.apache.org/jira/browse/HBASE-27381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613203#comment-17613203
]
Hudson commented on HBASE-27381:
Results for branch branch-2.5
[build #218 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/218/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/218/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/218/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/218/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/218/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Still seeing 'Stuck' in static initialization creating RegionInfo instance
> --
>
> Key: HBASE-27381
> URL: https://issues.apache.org/jira/browse/HBASE-27381
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> See https://issues.apache.org/jira/browse/HBASE-24896 for the original
> description. Despite having that fix, we are seeing this issue in a
> 2.4.6-based deploy. We recently started seeing it as we were moving to
> centos8. I'm not sure why the centos version would affect this, otherwise the
> hbase server version and java versions were not changing.
> We're seeing this in a non-trivial number of new centos8 servers that we spin
> up. I'm pushing a hotfix which removes RegionInfo.UNDEFINED to see if that
> resolves our issue.
> As mentioned in my last comments on that jira, it could be that this field is
> still an issue because according to
> [https://stackoverflow.com/questions/28631656/runnable-thread-state-but-in-object-wait:]
> > Such deadlocks may be caused by a [typical
> > bug|https://bugs.openjdk.org/browse/JDK-8037567] when a subclass is
> > referenced from a static initializer.
> If that's true, in this case MutableRegionInfo is a subclass/implementer of
> RegionInfo, so that could trigger it. Granted the linked bug is marked "Not
> An Issue".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27365) Minimise block addition failures due to no space in bucket cache writers queue by introducing wait time
[
https://issues.apache.org/jira/browse/HBASE-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613204#comment-17613204
]
Hudson commented on HBASE-27365:
Results for branch branch-2.5
[build #218 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/218/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/218/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/218/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/218/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/218/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Minimise block addition failures due to no space in bucket cache writers
> queue by introducing wait time
> ---
>
> Key: HBASE-27365
> URL: https://issues.apache.org/jira/browse/HBASE-27365
> Project: HBase
> Issue Type: Improvement
> Components: BucketCache
>Affects Versions: 3.0.0-alpha-3
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> Currently in bucket cache asynchronous caching mechanism introduced where
> initially the blocks to be cached will be added to queue and writer threads
> consume the blocks from the queue and write to bucket cache. In case if block
> writing to bucket cache is slow then there is a chance that queue of writer
> threads become full and following block additions will be failed. In case of
> slower storages like s3 might introduce latencies even if we enable bigger
> sizes of bucket cache using ephemeral storages. So we can allow configurable
> wait time while adding blocks to queue so that chances of queue free up is
> possible during the wait time and block addition failures can be minimised.
> To avoid the performance impact of wait time in regular read paths we can use
> the wait time mainly during background operations like compactions, flushes
> or prefetches etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #4812: HBASE-27401 Clean up current broken 'n's in our javadoc
Apache-HBase commented on PR #4812: URL: https://github.com/apache/hbase/pull/4812#issuecomment-1265819970 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 2s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 37s | master passed | | +1 :green_heart: | compile | 9m 31s | master passed | | +1 :green_heart: | checkstyle | 3m 42s | master passed | | +1 :green_heart: | spotless | 0m 56s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 10m 5s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 11s | the patch passed | | +1 :green_heart: | compile | 10m 22s | the patch passed | | -0 :warning: | javac | 0m 41s | hbase-common generated 4 new + 2 unchanged - 0 fixed = 6 total (was 2) | | -0 :warning: | javac | 0m 49s | hbase-client generated 8 new + 29 unchanged - 0 fixed = 37 total (was 29) | | -0 :warning: | javac | 2m 52s | hbase-server generated 5 new + 188 unchanged - 5 fixed = 193 total (was 193) | | -0 :warning: | javac | 0m 37s | hbase-mapreduce generated 7 new + 189 unchanged - 2 fixed = 196 total (was 191) | | -0 :warning: | javac | 0m 22s | hbase-testing-util generated 1 new + 11 unchanged - 2 fixed = 12 total (was 13) | | -0 :warning: | javac | 0m 33s | hbase-it generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -0 :warning: | javac | 0m 37s | hbase-rest generated 1 new + 152 unchanged - 1 fixed = 153 total (was 153) | | +1 :green_heart: | checkstyle | 0m 21s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-hadoop-compat | | +1 :green_heart: | checkstyle | 0m 22s | hbase-client: The patch generated 0 new + 73 unchanged - 10 fixed = 73 total (was 83) | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-zookeeper | | +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-balancer | | +1 :green_heart: | checkstyle | 0m 13s | The patch passed checkstyle in hbase-http | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-asyncfs | | +1 :green_heart: | checkstyle | 0m 13s | The patch passed checkstyle in hbase-procedure | | -0 :warning: | checkstyle | 0m 50s | hbase-server: The patch generated 3 new + 446 unchanged - 2 fixed = 449 total (was 448) | | -0 :warning: | checkstyle | 0m 16s | hbase-mapreduce: The patch generated 3 new + 75 unchanged - 1 fixed = 78 total (was 76) | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-testing-util | | +1 :green_heart: | checkstyle | 0m 35s | The patch passed checkstyle in hbase-thrift | | +1 :green_heart: | checkstyle | 0m 13s | The patch passed checkstyle in hbase-backup | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-it | | -0 :warning: | checkstyle | 0m 12s | hbase-rest: The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 58s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 52s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 11m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 1m 55s | The patch does not generate ASF License warnings. | | | | 82m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4812/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4812 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux e223eef70ef3 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality |
[GitHub] [hbase] bbeaudreault commented on a diff in pull request #4786: HBASE-26809: Report client backoff time for server overloaded
bbeaudreault commented on code in PR #4786:
URL: https://github.com/apache/hbase/pull/4786#discussion_r985899306
##
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java:
##
@@ -65,26 +68,29 @@ public RpcRetryingCaller newCaller() {
// is cheap as it does not require parsing a complex structure.
return new RpcRetryingCallerImpl<>(connectionConf.getPauseMillis(),
connectionConf.getPauseMillisForServerOverloaded(),
connectionConf.getRetriesNumber(),
- interceptor, startLogErrorsCnt, connectionConf.getRpcTimeout());
+ interceptor, startLogErrorsCnt, connectionConf.getRpcTimeout(), metrics);
}
public static RpcRetryingCallerFactory instantiate(Configuration
configuration) {
Review Comment:
Ok, sorry for the delay here but I've been meaning to look into the usages
of these methods. We basically want to ensure that our metrics object is
getting in there as often as possible.
So I opened up branch-2 in intellij and looked at the usages of these
instantiate overloads:
- `instantiate(Configuration)` -- all of the callers to this method have a
ClusterConnection in the call context, which we can pull our metrics from
(connection.getConnectionMetrics())
- `instantiate(Configuration, ServerStatisticTracker) -- all but the
SecureBulkLoadClient and LoadIncrementalHFiles have similar.
So I think we should try to add ConnectionMetrics to all of the methods
here, and we can pass null in for the 2 cases above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #4809: HBASE-27403 Remove 'Remove unhelpful javadoc stubs' spotless rule for…
Apache9 merged PR #4809: URL: https://github.com/apache/hbase/pull/4809 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] briaugenreich commented on pull request #4786: HBASE-26809: Report client backoff time for server overloaded
briaugenreich commented on PR #4786: URL: https://github.com/apache/hbase/pull/4786#issuecomment-1265551267 hmm not sure why the build was aborted, but I can kick off another. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] payert commented on a diff in pull request #4807: HBASE-27395 Adding description to Prometheus metrics
payert commented on code in PR #4807:
URL: https://github.com/apache/hbase/pull/4807#discussion_r985710610
##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/prometheus/PrometheusHadoopServlet.java:
##
@@ -57,13 +60,19 @@ static String toPrometheusName(String metricRecordName,
String metricName) {
*/
@RestrictedApi(explanation = "Should only be called in tests or self", link
= "",
allowedOnPath = ".*/src/test/.*|.*/PrometheusHadoopServlet\\.java")
- void writeMetrics(Writer writer) throws IOException {
+ void writeMetrics(Writer writer, boolean desc) throws IOException {
Review Comment:
Maybe 'describe' is better argument name than 'desc'.
##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/prometheus/PrometheusHadoopServlet.java:
##
@@ -41,7 +41,10 @@ public class PrometheusHadoopServlet extends HttpServlet {
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {
-writeMetrics(resp.getWriter());
+String tmpStr = req.getParameter("description");
+boolean description = tmpStr != null && tmpStr.equals("true");
+
+writeMetrics(resp.getWriter(), description);
Review Comment:
This could be compacted like this:
`writeMetrics(resp.getWriter(),
"true".equals(req.getParameter("description"))); `
##
hbase-http/src/main/java/org/apache/hadoop/hbase/http/prometheus/PrometheusHadoopServlet.java:
##
@@ -57,13 +60,19 @@ static String toPrometheusName(String metricRecordName,
String metricName) {
*/
@RestrictedApi(explanation = "Should only be called in tests or self", link
= "",
allowedOnPath = ".*/src/test/.*|.*/PrometheusHadoopServlet\\.java")
- void writeMetrics(Writer writer) throws IOException {
+ void writeMetrics(Writer writer, boolean desc) throws IOException {
Collection metricRecords = MetricsExportHelper.export();
for (MetricsRecord metricsRecord : metricRecords) {
for (AbstractMetric metrics : metricsRecord.metrics()) {
if (metrics.type() == MetricType.COUNTER || metrics.type() ==
MetricType.GAUGE) {
String key = toPrometheusName(metricsRecord.name(), metrics.name());
+
+ if (desc) {
+String description = metrics.description();
+if (!description.isEmpty()) writer.append("# HELP
").append(description).append("\n");
Review Comment:
When you append a single char to the writer you can use the append(char c)
instead if of append(CharSequence csq). I mean
`.append('\n'`)
instead of
`.append("\n")`
Doesn't make much difference but slightly cheaper..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ragarkar commented on a diff in pull request #4799: HBASE-27389 Add cost function in balancer to consider the cost of bui…
ragarkar commented on code in PR #4799:
URL: https://github.com/apache/hbase/pull/4799#discussion_r985399745
##
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/PrefetchCacheCostFunction.java:
##
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.balancer;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Compute the cost of a potential cluster configuration based on the number
of HFile's already
+ * cached in the bucket cache
+ */
+@InterfaceAudience.Private
+public class PrefetchCacheCostFunction extends CostFunction {
+ private String prefetchedFileListPath;
+ private float prefetchRatio;
+ private float bestPrefetchRatio;
+
+ public static final String PREFETCH_PERSISTENCE_PATH_KEY =
"hbase.prefetch.file-list.path";
Review Comment:
Done. I have made this change. It will be visible in the next patch.
##
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/PrefetchCacheCostFunction.java:
##
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.balancer;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Compute the cost of a potential cluster configuration based on the number
of HFile's already
+ * cached in the bucket cache
+ */
+@InterfaceAudience.Private
+public class PrefetchCacheCostFunction extends CostFunction {
+ private String prefetchedFileListPath;
+ private float prefetchRatio;
+ private float bestPrefetchRatio;
+
+ public static final String PREFETCH_PERSISTENCE_PATH_KEY =
"hbase.prefetch.file-list.path";
+
+ PrefetchCacheCostFunction(Configuration conf) {
+prefetchedFileListPath = conf.get(PREFETCH_PERSISTENCE_PATH_KEY);
+this.setMultiplier(prefetchedFileListPath == null ? 0 : 1);
Review Comment:
Yes. The whole logic of this cost function depends on the capability of
persisting the list of files already prefetched on the server and hence, if the
configuration "hbase.prefetch.file-list.path" isn't set, then the cost function
will be disabled.
##
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:
##
@@ -452,7 +459,10 @@ protected List balanceTable(TableName
tableName,
// Allow turning this feature off if the locality cost is not going to
// be used in any computations.
RegionHDFSBlockLocationFinder finder = null;
-if ((this.localityCost != null) || (this.rackLocalityCost != null)) {
+if (
+ (this.localityCost != null) || (this.rackLocalityCost != null)
+|| (this.prefetchCacheCost != null)
Review Comment:
I have put this code just in case, the localityCost and rackLocalityCost
functions are disabled by the user configuration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ragarkar commented on pull request #4799: HBASE-27389 Add cost function in balancer to consider the cost of bui…
ragarkar commented on PR #4799: URL: https://github.com/apache/hbase/pull/4799#issuecomment-1264968926 > Also, does this new cost function always be enabled by default on the StochasticLoadBalancer? The PrefetchCacheCostFunction will be enabled if the configuration "hbase.prefetch.file.list.path" is set. In addition to that, it will have it's own configuration "hbase.master.balancer.stochastic.prefetchCacheCost" that will define the multiplier value for this cost function. This is covered in the new patch that I will upload soon. If either "hbase.prefetch.file.list.path" is not set or "hbase.master.balancer.stochastic.prefetchCacheCost" is set to 0, the cost function will not be enabled. The default value of "hbase.master.balancer.stochastic.prefetchCacheCost" will be > 0 which will mean that the cost function will be enabled by default given that the configuration "hbase.prefetch.file.list.path" is set. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ragarkar commented on pull request #4799: HBASE-27389 Add cost function in balancer to consider the cost of bui…
ragarkar commented on PR #4799: URL: https://github.com/apache/hbase/pull/4799#issuecomment-1264965698 > I do not fully understand why higher prefetched ratio leads to lower moving cost, if a region's hfiles have all been cached, after moving we need to fetched all the hfiles again, which is costly? If we are moving a region from server A to server B, if the prefetch ratio on B is higher than that on A, then the cost of moving the region is low. During region movement, there is a likelihood that the file is already prefetched on server B and hence there will be no need to prefetch it again. The changes done in HBASE-27313 already maintain a list of files which are already prefetched on the server. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] chrajeshbabu merged pull request #4805: HBASE-27365 Minimise block addition failures due to no space in bucke…
chrajeshbabu merged PR #4805: URL: https://github.com/apache/hbase/pull/4805 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] 2005hithlj commented on pull request #4810: HBASE-27217 Revisit the DumpReplicationQueues tool
2005hithlj commented on PR #4810: URL: https://github.com/apache/hbase/pull/4810#issuecomment-1264938729 @Apache9 sir. Could you take a look? Thanks. And the relevant UT of DumpReplicationQueues will be added later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] 2005hithlj commented on pull request #4811: HBASE-27405 Fix the replication hfile/log cleaner report that the replication table does not exist
2005hithlj commented on PR #4811: URL: https://github.com/apache/hbase/pull/4811#issuecomment-1264937875 @Apache9 sir. Could you take a look? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4810: HBASE-27217 Revisit the DumpReplicationQueues tool
Apache-HBase commented on PR #4810: URL: https://github.com/apache/hbase/pull/4810#issuecomment-1264717030 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-27109/table_based_rqs Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 51s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | compile | 0m 43s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | shadedjars | 3m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | HBASE-27109/table_based_rqs passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 29s | the patch passed | | +1 :green_heart: | compile | 0m 42s | the patch passed | | +1 :green_heart: | javac | 0m 42s | the patch passed | | +1 :green_heart: | shadedjars | 3m 59s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 210m 14s | hbase-server in the patch failed. | | | | 230m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4810/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4810 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 814721bae111 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-27109/table_based_rqs / de2d20aac2 | | Default Java | Eclipse Adoptium-11.0.16.1+1 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4810/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4810/1/testReport/ | | Max. process+thread count | 2693 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4810/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4810: HBASE-27217 Revisit the DumpReplicationQueues tool
Apache-HBase commented on PR #4810: URL: https://github.com/apache/hbase/pull/4810#issuecomment-1264717762 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 22s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-27109/table_based_rqs Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | compile | 0m 39s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | shadedjars | 3m 50s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | HBASE-27109/table_based_rqs passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 15s | the patch passed | | +1 :green_heart: | compile | 0m 39s | the patch passed | | +1 :green_heart: | javac | 0m 39s | the patch passed | | +1 :green_heart: | shadedjars | 3m 49s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 217m 56s | hbase-server in the patch passed. | | | | 234m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4810/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4810 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 102a8cb800a3 5.4.0-109-generic #123-Ubuntu SMP Fri Apr 8 09:10:54 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-27109/table_based_rqs / de2d20aac2 | | Default Java | Temurin-1.8.0_345-b01 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4810/1/testReport/ | | Max. process+thread count | 2506 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4810/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4811: HBASE-27405 Fix the replication hfile/log cleaner report that the replication table does not exist
Apache-HBase commented on PR #4811: URL: https://github.com/apache/hbase/pull/4811#issuecomment-1264688288 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 14s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-27109/table_based_rqs Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 39s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | compile | 0m 43s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | shadedjars | 4m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | HBASE-27109/table_based_rqs passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 22s | the patch passed | | +1 :green_heart: | compile | 0m 40s | the patch passed | | +1 :green_heart: | javac | 0m 40s | the patch passed | | +1 :green_heart: | shadedjars | 3m 45s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 8m 44s | hbase-server in the patch failed. | | | | 30m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4811/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4811 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 336456b622af 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-27109/table_based_rqs / de2d20aac2 | | Default Java | Temurin-1.8.0_345-b01 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4811/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4811/1/testReport/ | | Max. process+thread count | 549 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4811/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4811: HBASE-27405 Fix the replication hfile/log cleaner report that the replication table does not exist
Apache-HBase commented on PR #4811: URL: https://github.com/apache/hbase/pull/4811#issuecomment-1264688003 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 14s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-27109/table_based_rqs Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 59s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | compile | 0m 47s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | shadedjars | 3m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | HBASE-27109/table_based_rqs passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 32s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed | | +1 :green_heart: | javac | 0m 47s | the patch passed | | +1 :green_heart: | shadedjars | 3m 50s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 11m 6s | hbase-server in the patch failed. | | | | 29m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4811/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4811 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c2a4daf76a57 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-27109/table_based_rqs / de2d20aac2 | | Default Java | Eclipse Adoptium-11.0.16.1+1 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4811/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4811/1/testReport/ | | Max. process+thread count | 705 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4811/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4811: HBASE-27405 Fix the replication hfile/log cleaner report that the replication table does not exist
Apache-HBase commented on PR #4811: URL: https://github.com/apache/hbase/pull/4811#issuecomment-1264687858 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-27109/table_based_rqs Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | compile | 2m 16s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | checkstyle | 0m 32s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 19s | HBASE-27109/table_based_rqs passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 2m 20s | the patch passed | | +1 :green_heart: | javac | 2m 20s | the patch passed | | +1 :green_heart: | checkstyle | 0m 30s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 7m 50s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 28m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4811/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4811 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux b8e36ab5601b 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-27109/table_based_rqs / de2d20aac2 | | Default Java | Temurin-1.8.0_345-b01 | | Max. process+thread count | 69 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4811/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] 2005hithlj opened a new pull request, #4811: HBASE-27405 Fix the replication hfile/log cleaner report that the replication table does not exist
2005hithlj opened a new pull request, #4811: URL: https://github.com/apache/hbase/pull/4811 https://issues.apache.org/jira/browse/HBASE-27405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4810: HBASE-27217 Revisit the DumpReplicationQueues tool
Apache-HBase commented on PR #4810: URL: https://github.com/apache/hbase/pull/4810#issuecomment-1264679130 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-27109/table_based_rqs Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | compile | 2m 20s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | checkstyle | 0m 33s | HBASE-27109/table_based_rqs passed | | +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 22s | HBASE-27109/table_based_rqs passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 2m 18s | the patch passed | | +1 :green_heart: | javac | 2m 18s | the patch passed | | +1 :green_heart: | checkstyle | 0m 31s | hbase-server: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 7m 56s | Patch does not cause any errors with Hadoop 3.2.4 3.3.4. | | +1 :green_heart: | spotless | 0m 39s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 29m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4810/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4810 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux d406b8ac8afd 5.4.0-122-generic #138-Ubuntu SMP Wed Jun 22 15:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-27109/table_based_rqs / de2d20aac2 | | Default Java | Temurin-1.8.0_345-b01 | | Max. process+thread count | 64 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4810/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.7.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] 2005hithlj opened a new pull request, #4810: HBASE-27217 Revisit the DumpReplicationQueues tool
2005hithlj opened a new pull request, #4810: URL: https://github.com/apache/hbase/pull/4810 https://issues.apache.org/jira/browse/HBASE-27217 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27309) Add major compact table or region operation on master web table page
[
https://issues.apache.org/jira/browse/HBASE-27309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613124#comment-17613124
]
Hudson commented on HBASE-27309:
Results for branch master
[build #694 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/694/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/694/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/694/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/694/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add major compact table or region operation on master web table page
>
>
> Key: HBASE-27309
> URL: https://issues.apache.org/jira/browse/HBASE-27309
> Project: HBase
> Issue Type: Sub-task
>Reporter: zhengsicheng
>Assignee: zhengsicheng
>Priority: Minor
> Attachments: image-2022-09-22-02-32-36-619.png
>
>
> Add major compact table or region operation on master web table page
> !image-2022-09-22-02-32-36-619.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27381) Still seeing 'Stuck' in static initialization creating RegionInfo instance
[
https://issues.apache.org/jira/browse/HBASE-27381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613123#comment-17613123
]
Hudson commented on HBASE-27381:
Results for branch master
[build #694 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/694/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/694/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/694/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/694/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Still seeing 'Stuck' in static initialization creating RegionInfo instance
> --
>
> Key: HBASE-27381
> URL: https://issues.apache.org/jira/browse/HBASE-27381
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> See https://issues.apache.org/jira/browse/HBASE-24896 for the original
> description. Despite having that fix, we are seeing this issue in a
> 2.4.6-based deploy. We recently started seeing it as we were moving to
> centos8. I'm not sure why the centos version would affect this, otherwise the
> hbase server version and java versions were not changing.
> We're seeing this in a non-trivial number of new centos8 servers that we spin
> up. I'm pushing a hotfix which removes RegionInfo.UNDEFINED to see if that
> resolves our issue.
> As mentioned in my last comments on that jira, it could be that this field is
> still an issue because according to
> [https://stackoverflow.com/questions/28631656/runnable-thread-state-but-in-object-wait:]
> > Such deadlocks may be caused by a [typical
> > bug|https://bugs.openjdk.org/browse/JDK-8037567] when a subclass is
> > referenced from a static initializer.
> If that's true, in this case MutableRegionInfo is a subclass/implementer of
> RegionInfo, so that could trigger it. Granted the linked bug is marked "Not
> An Issue".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (HBASE-26421) Use HFileLink file to replace entire file‘s reference when splitting
[ https://issues.apache.org/jira/browse/HBASE-26421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613084#comment-17613084 ] Huaxiang Sun edited comment on HBASE-26421 at 10/5/22 6:53 PM: --- Forward comments from review board here for better background of this jira. h3. *[Apache9|https://github.com/Apache9]* commented [on Nov 8, 2021|https://github.com/apache/hbase/pull/3825#issuecomment-963696081] |What is the advantage here to use HFileLink?| *[sunhelly|https://github.com/sunhelly]* commented [on Nov 8, 2021|https://github.com/apache/hbase/pull/3825#issuecomment-963870288] |Thanks, [@Apache9|https://github.com/Apache9] . The main advantage here is that the first compaction after split can be a minor compaction instead of old major compaction, because only reference files should be compacted before the next split. In HBASE-26422, I described the compaction after splitting using HFileLink. It also makes move hfiles between regions of the same table possible. Then the compaction is light weight, since it need not to read and rewrite the referred files.| *[Apache9|https://github.com/Apache9]* commented [on Nov 9, 2021|https://github.com/apache/hbase/pull/3825#issuecomment-964161491] |OK, and we do not need to use HalfStoreFileReader to read the HFile right? Sound like a good general improvement. I think this could be done on master and other active branches, not only on a feature branch. Will take a look at the PR soon.| was (Author: huaxiangsun): Forward comments from review board here for better background of this jira. h3. *[Apache9|https://github.com/Apache9]* commented [on Nov 8, 2021|https://github.com/apache/hbase/pull/3825#issuecomment-963696081] |What is the advantage here to use HFileLink?| [!https://avatars.githubusercontent.com/u/10123703?s=80=6be309b7250884321e34ad1fad94072c3ddea60f=4|width=40,height=40!|https://github.com/sunhelly] ContributorAuthor h3. *[sunhelly|https://github.com/sunhelly]* commented [on Nov 8, 2021|https://github.com/apache/hbase/pull/3825#issuecomment-963870288] |Thanks, [@Apache9|https://github.com/Apache9] . The main advantage here is that the first compaction after split can be a minor compaction instead of old major compaction, because only reference files should be compacted before the next split. In HBASE-26422, I described the compaction after splitting using HFileLink. It also makes move hfiles between regions of the same table possible. Then the compaction is light weight, since it need not to read and rewrite the referred files.| [!https://avatars.githubusercontent.com/u/4958168?s=80=fc28b222c03c02201d705b025a5293d6c471f7b3=4|width=40,height=40!|https://github.com/Apache9] Contributor h3. *[Apache9|https://github.com/Apache9]* commented [on Nov 9, 2021|https://github.com/apache/hbase/pull/3825#issuecomment-964161491] |OK, and we do not need to use HalfStoreFileReader to read the HFile right? Sound like a good general improvement. I think this could be done on master and other active branches, not only on a feature branch. Will take a look at the PR soon.| > Use HFileLink file to replace entire file‘s reference when splitting > > > Key: HBASE-26421 > URL: https://issues.apache.org/jira/browse/HBASE-26421 > Project: HBase > Issue Type: Improvement >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-2 > > > ++When splitting store files, if a file should be owned by only one child > region, then there will be an entire file's reference in the child region. We > can use HFileLink files, just like those in snapshot tables, to replace the > reference files that refer to entire files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27413) MemStoreFlusher: Cache flush failed for region
March Wang created HBASE-27413:
--
Summary: MemStoreFlusher: Cache flush failed for region
Key: HBASE-27413
URL: https://issues.apache.org/jira/browse/HBASE-27413
Project: HBase
Issue Type: Bug
Affects Versions: 2.2.6
Environment: Hadoop 3.3.1
HBase 2.2.6
jdk: openjdk version "1.8.0_342"
Reporter: March Wang
We are running an HBase cluster (Hadoop 3.3.1 + HBase 2.2.6) when clients
frequently write a lot of data into this HBase cluster, we got the message
"Cache flush failed for the region" error log. then got a lot of
"responseTooSlow" logs. below are the sample logs of the "Cache flush failed
for region" messages:
{code:java}
022-08-09 11:04:05,069 DEBUG [MemStoreFlusher.0] regionserver.HRegion: Flush
status journal:
Acquiring readlock on region at 1660067945028
Running coprocessor pre-flush hooks at 1660067945028
Obtaining lock to block concurrent updates at 1660067945028
Preparing flush snapshotting stores in dfab869db13603b7800cf7b33d58a885 at
1660067945028
Finished memstore snapshotting
ns1:htable1,77e1422d7b070b9ee3ac9466174655e884d41d87bbe9182e05661cf11d02298b,1653473214632.dfab869db13603b7800cf7b33d58a885.,
syncing WAL and waiting on mvcc, flushsize=dataSize=28417, getHeapSize=59360,
getOffHeapSize=0, getCellsCount=284 at 1660067945028
2022-08-09 11:04:05,070 ERROR [MemStoreFlusher.0] regionserver.MemStoreFlusher:
Cache flush failed for region
ns1:htable1,77e1422d7b070b9ee3ac9466174655e884d41d87bbe9182e05661cf11d02298b,1653473214632.dfab869db13603b7800cf7b33d58a885.
org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
result after 30 ms for txid=77949378, WAL system stuck?
at
org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:145)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:722)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:616)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:593)
at
org.apache.hadoop.hbase.regionserver.HRegion.doSyncOfUnflushedWALChanges(HRegion.java:2754)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2691)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2549)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2523)
at
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2413)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:611)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:580)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:360)
at java.lang.Thread.run(Thread.java:748)
2022-08-09 11:04:05,070 INFO [MemStoreFlusher.0]
hbase.RangerAuthorizationCoprocessor: Unable to get remote Address
2022-08-09 11:04:05,071 DEBUG [MemStoreFlusher.0] regionserver.HRegion: Flush
column family: ai of 326f48e574a65d7ba599a68dbf742603 because time of oldest
edit=1660064392583 is > 360 from now =1660068245071
2022-08-09 11:04:05,072 INFO [MemStoreFlusher.0] regionserver.HRegion:
Flushing 326f48e574a65d7ba599a68dbf742603 1/3 column families, dataSize=111.73
KB heapSize=184.23 KB; ai={dataSize=12.15 KB, heapSize=24.73 KB, offHeapSize=0
B}
2022-08-09 11:04:05,275 WARN
[RpcServer.default.FPBQ.Fifo.handler=42,queue=0,port=16020] ipc.RpcServer:
(responseTooSlow):
{"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1660067945231","responsesize":"1659","method":"Multi","param":"region\u003d
ns1:htable12,a8f9942ef122d7bb0d0b0036cc5d776d030f9c06b7af0ebb2908edd5a4f43b4c,1460489972562.204291859a0abda7305f73ccb03b174c.,
for 1 act
\u003cTRUNCATED\u003e","processingtimems":300043,"client":"172.16.139.217:35044","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1}
2022-08-09 11:04:05,561 WARN
[RpcServer.default.FPBQ.Fifo.handler=14,queue=2,port=16020] ipc.RpcServer:
(responseTooSlow):
{"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":0,"starttimems":"1660067945519","responsesize":"1659","method":"Multi","param":"region\u003d
ns1:htable13,a00bd53d,1502528412031.f04f3634f3822a67dccf8286cfe3e19a., for 1
action(s) and 1st row key\u003da0fd50edde8a734b63a39899f4572a8
\u003cTRUNCATED\u003e","processingtimems":300042,"client":"172.16.139.215:38654","queuetimems":0,"multi.servicecalls":0,"class":"HRegionServer","multi.mutations":1}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-26421) Use HFileLink file to replace entire file‘s reference when splitting
[ https://issues.apache.org/jira/browse/HBASE-26421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17613084#comment-17613084 ] Huaxiang Sun commented on HBASE-26421: -- Forward comments from review board here for better background of this jira. h3. *[Apache9|https://github.com/Apache9]* commented [on Nov 8, 2021|https://github.com/apache/hbase/pull/3825#issuecomment-963696081] |What is the advantage here to use HFileLink?| [!https://avatars.githubusercontent.com/u/10123703?s=80=6be309b7250884321e34ad1fad94072c3ddea60f=4|width=40,height=40!|https://github.com/sunhelly] ContributorAuthor h3. *[sunhelly|https://github.com/sunhelly]* commented [on Nov 8, 2021|https://github.com/apache/hbase/pull/3825#issuecomment-963870288] |Thanks, [@Apache9|https://github.com/Apache9] . The main advantage here is that the first compaction after split can be a minor compaction instead of old major compaction, because only reference files should be compacted before the next split. In HBASE-26422, I described the compaction after splitting using HFileLink. It also makes move hfiles between regions of the same table possible. Then the compaction is light weight, since it need not to read and rewrite the referred files.| [!https://avatars.githubusercontent.com/u/4958168?s=80=fc28b222c03c02201d705b025a5293d6c471f7b3=4|width=40,height=40!|https://github.com/Apache9] Contributor h3. *[Apache9|https://github.com/Apache9]* commented [on Nov 9, 2021|https://github.com/apache/hbase/pull/3825#issuecomment-964161491] |OK, and we do not need to use HalfStoreFileReader to read the HFile right? Sound like a good general improvement. I think this could be done on master and other active branches, not only on a feature branch. Will take a look at the PR soon.| > Use HFileLink file to replace entire file‘s reference when splitting > > > Key: HBASE-26421 > URL: https://issues.apache.org/jira/browse/HBASE-26421 > Project: HBase > Issue Type: Improvement >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-2 > > > ++When splitting store files, if a file should be owned by only one child > region, then there will be an entire file's reference in the child region. We > can use HFileLink files, just like those in snapshot tables, to replace the > reference files that refer to entire files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27412) [hbase-thirdparty] Bump dependency versions in hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-27412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27412: -- Summary: [hbase-thirdparty] Bump dependency versions in hbase-thirdparty (was: Bump dependency versions in hbase-thirdparty) > [hbase-thirdparty] Bump dependency versions in hbase-thirdparty > --- > > Key: HBASE-27412 > URL: https://issues.apache.org/jira/browse/HBASE-27412 > Project: HBase > Issue Type: Task > Components: dependencies, security, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-4.1.2 > > > There are several security issues in our dependencies, for example > https://nvd.nist.gov/vuln/detail/CVE-2022-42004 > Let's bump the dependencies. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-27412) Bump dependency versions in hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-27412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-27412 started by Duo Zhang. - > Bump dependency versions in hbase-thirdparty > > > Key: HBASE-27412 > URL: https://issues.apache.org/jira/browse/HBASE-27412 > Project: HBase > Issue Type: Task > Components: dependencies, security, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-4.1.2 > > > There are several security issues in our dependencies, for example > https://nvd.nist.gov/vuln/detail/CVE-2022-42004 > Let's bump the dependencies. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-27412) Bump dependency versions in hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-27412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-27412: - Assignee: Duo Zhang > Bump dependency versions in hbase-thirdparty > > > Key: HBASE-27412 > URL: https://issues.apache.org/jira/browse/HBASE-27412 > Project: HBase > Issue Type: Task > Components: dependencies, security, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-4.1.2 > > > There are several security issues in our dependencies, for example > https://nvd.nist.gov/vuln/detail/CVE-2022-42004 > Let's bump the dependencies. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27412) Bump dependency versions in hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-27412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27412: -- Fix Version/s: thirdparty-4.1.2 > Bump dependency versions in hbase-thirdparty > > > Key: HBASE-27412 > URL: https://issues.apache.org/jira/browse/HBASE-27412 > Project: HBase > Issue Type: Task > Components: dependencies, security, thirdparty >Reporter: Duo Zhang >Priority: Major > Fix For: thirdparty-4.1.2 > > > There are several security issues in our dependencies, for example > https://nvd.nist.gov/vuln/detail/CVE-2022-42004 > Let's bump the dependencies. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27412) Bump dependency versions in hbase-thirdparty
Duo Zhang created HBASE-27412: - Summary: Bump dependency versions in hbase-thirdparty Key: HBASE-27412 URL: https://issues.apache.org/jira/browse/HBASE-27412 Project: HBase Issue Type: Task Components: dependencies, security, thirdparty Reporter: Duo Zhang There are several security issues in our dependencies, for example https://nvd.nist.gov/vuln/detail/CVE-2022-42004 Let's bump the dependencies. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-27407) Bug in JMX metrics description
[ https://issues.apache.org/jira/browse/HBASE-27407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-27407 started by Luca Kovacs. --- > Bug in JMX metrics description > -- > > Key: HBASE-27407 > URL: https://issues.apache.org/jira/browse/HBASE-27407 > Project: HBase > Issue Type: Bug > Components: metrics >Affects Versions: 2.6.0, 3.0.0-alpha-3 >Reporter: Luca Kovacs >Assignee: Luca Kovacs >Priority: Minor > > When trying to access the JMX metrics' description via the "description=true" > URL parameter, any value is accepted. > The current version checks only if "description" is in the URL parameter, but > doesn't check the parameter value. > I would like to fix this via checking if the parameter value is 'true' and > showing the description only when this condition is met. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27381) Still seeing 'Stuck' in static initialization creating RegionInfo instance
[
https://issues.apache.org/jira/browse/HBASE-27381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17612974#comment-17612974
]
Hudson commented on HBASE-27381:
Results for branch branch-2.4
[build #434 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/434/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/434/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/434/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/434/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Still seeing 'Stuck' in static initialization creating RegionInfo instance
> --
>
> Key: HBASE-27381
> URL: https://issues.apache.org/jira/browse/HBASE-27381
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.5.1, 3.0.0-alpha-4, 2.4.15
>
>
> See https://issues.apache.org/jira/browse/HBASE-24896 for the original
> description. Despite having that fix, we are seeing this issue in a
> 2.4.6-based deploy. We recently started seeing it as we were moving to
> centos8. I'm not sure why the centos version would affect this, otherwise the
> hbase server version and java versions were not changing.
> We're seeing this in a non-trivial number of new centos8 servers that we spin
> up. I'm pushing a hotfix which removes RegionInfo.UNDEFINED to see if that
> resolves our issue.
> As mentioned in my last comments on that jira, it could be that this field is
> still an issue because according to
> [https://stackoverflow.com/questions/28631656/runnable-thread-state-but-in-object-wait:]
> > Such deadlocks may be caused by a [typical
> > bug|https://bugs.openjdk.org/browse/JDK-8037567] when a subclass is
> > referenced from a static initializer.
> If that's true, in this case MutableRegionInfo is a subclass/implementer of
> RegionInfo, so that could trigger it. Granted the linked bug is marked "Not
> An Issue".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
