[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718872112 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 50s | master passed | | +1 :green_heart: | compile | 0m 20s | master passed | | +1 :green_heart: | checkstyle | 0m 10s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | the patch passed | | +1 :green_heart: | compile | 0m 18s | the patch passed | | +1 :green_heart: | javac | 0m 18s | the patch passed | | +1 :green_heart: | checkstyle | 0m 9s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 2s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 24m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux a0dd96e0c8f8 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-asyncfs U: hbase-asyncfs | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718864359 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | master passed | | +1 :green_heart: | compile | 0m 11s | master passed | | +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 0m 11s | the patch passed | | +1 :green_heart: | javac | 0m 11s | the patch passed | | +1 :green_heart: | shadedjars | 4m 55s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 0s | hbase-asyncfs in the patch passed. | | | | 17m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 171dfd1e5090 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/testReport/ | | Max. process+thread count | 636 (vs. ulimit of 3) | | modules | C: hbase-asyncfs U: hbase-asyncfs | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache-HBase commented on PR #5401: URL: https://github.com/apache/hbase/pull/5401#issuecomment-1718864306 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 10s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 51s | master passed | | +1 :green_heart: | compile | 0m 15s | master passed | | +1 :green_heart: | shadedjars | 4m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 36s | the patch passed | | +1 :green_heart: | compile | 0m 14s | the patch passed | | +1 :green_heart: | javac | 0m 14s | the patch passed | | +1 :green_heart: | shadedjars | 4m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 58s | hbase-asyncfs in the patch passed. | | | | 18m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5401 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1b4196b913eb 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/testReport/ | | Max. process+thread count | 578 (vs. ulimit of 3) | | modules | C: hbase-asyncfs U: hbase-asyncfs | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5401/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] zhangbutao commented on a diff in pull request #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
zhangbutao commented on code in PR #5401:
URL: https://github.com/apache/hbase/pull/5401#discussion_r1325439479
##
hbase-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputHelper.java:
##
@@ -383,7 +400,7 @@ private static List>
connectToDataNodes(Configuration conf, DFSC
BlockConstructionStage stage, DataChecksum summer, EventLoopGroup
eventLoopGroup,
Class channelClass) {
StorageType[] storageTypes = locatedBlock.getStorageTypes();
-DatanodeInfo[] datanodeInfos = locatedBlock.getLocations();
+DatanodeInfo[] datanodeInfos = getLocatedBlockLocations(locatedBlock);
Review Comment:
There's another place to be fixed. :)
https://github.com/apache/hbase/blob/84ccae368f6ed94a751d576c59fc695c683f1c8b/hbase-asyncfs/src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputHelper.java#L498
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache-HBase commented on PR #5391: URL: https://github.com/apache/hbase/pull/5391#issuecomment-1718850992 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 38s | master passed | | +1 :green_heart: | compile | 3m 37s | master passed | | +1 :green_heart: | checkstyle | 1m 6s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 45s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 35s | the patch passed | | +1 :green_heart: | compile | 3m 39s | the patch passed | | +1 :green_heart: | javac | 3m 39s | the patch passed | | -0 :warning: | checkstyle | 0m 15s | hbase-client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 10s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 3m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 38m 31s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5391 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 28a19edbaab4 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/5/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765019#comment-17765019
]
Duo Zhang commented on HBASE-28061:
---
Oh, just saw your patch, and there is another place need to handle.
Let me update the PR.
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HbaseMiniClusterTest.java,
> fix_LocatedBlock__getLocations.patch, full_output.txt, pom.xml, root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765016#comment-17765016
]
Duo Zhang commented on HBASE-28061:
---
I've posted a PR to fix the problem.
https://github.com/apache/hbase/pull/5401
Please have a try. If this could solve the problem, then we can include this
and make a new 2.5.x release.
Thanks.
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HbaseMiniClusterTest.java,
> fix_LocatedBlock__getLocations.patch, full_output.txt, pom.xml, root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28058) HMaster snapshot file clean thread and the snapshot request handler thread encountered a deadlock
[
https://issues.apache.org/jira/browse/HBASE-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765014#comment-17765014
]
Hudson commented on HBASE-28058:
Results for branch branch-2
[build #879 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HMaster snapshot file clean thread and the snapshot request handler thread
> encountered a deadlock
> -
>
> Key: HBASE-28058
> URL: https://issues.apache.org/jira/browse/HBASE-28058
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-beta-1
>Reporter: Haiping lv
>Assignee: Haiping lv
>Priority: Major
> Fix For: 3.0.0
>
>
> HMaster encountered a deadlock when runing ITBLL test.
> The handler thread of HMaster holds the synchronized lock of SnapshotManager
> and competes for the readLock of takingSnapshotLock.
> {code:java}
> Thread 57 (RpcServer.default.FPBQ.Fifo.handler=22,queue=1,port=16000):
> State: WAITING
> Blocked count: 14
> Waited count: 206
> Waiting on
> java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@a7a58d4
> Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
>
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:675)
>
> org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1765)
>
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437)
> org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) {code}
> The Master Snapshot cleaning thread holds the writeLock of takingSnapshotLock
> to compete for the synchronized lock of SnapshotManager.
> {code:java}
> Thread 5405 (hfile_cleaner-dir-scan-pool-2):
> State: BLOCKED
> Blocked count: 50535
> Waited count: 92200
> Blocked on org.apache.hadoop.hbase.master.snapshot.SnapshotManager@21a16019
> Blocked by 57 (RpcServer.default.FPBQ.Fifo.handler=22,queue=1,port=16000)
> Stack:
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isTakingAnySnapshot(SnapshotManager.java:655)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getUnreferencedFiles(SnapshotFileCache.java:197)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner.getDeletableFiles(SnapshotHFileCleaner.java:78)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:375)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$1(CleanerChore.java:483)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$788/1121007598.act(Unknown
> Source)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:565)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:483)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$2(CleanerChore.java:495)
>
> org.apach
[GitHub] [hbase] Apache9 opened a new pull request, #5401: HBASE-28061 HBaseTestingUtility failed to start MiniHbaseCluster in c…
Apache9 opened a new pull request, #5401: URL: https://github.com/apache/hbase/pull/5401 …ase of Hadoop3.3.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28079) Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
[
https://issues.apache.org/jira/browse/HBASE-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765015#comment-17765015
]
Hudson commented on HBASE-28079:
Results for branch branch-2
[build #879 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/879/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unhandled TableExistsException and NamespaceExistException in
> BackupSystemTable
> ---
>
> Key: HBASE-28079
> URL: https://issues.apache.org/jira/browse/HBASE-28079
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Most backup-related operations instantiate a BackupSystemTable object. The
> first thing it does is check that the {{backup}} namespace and backup-related
> tables exist, and create them if not. One way in which this can be triggered
> is via LoadIncrementalHFiles, which can load files with a configurable number
> of threads. When bulkLoadHFile is called, a preCommitHFile hook is triggered
> which does the above. With a certain level of concurrency, one can easily
> trigger a race condition on creating the namespace or tables. This exception
> bubbles up and fails the bulkload.
> We should catch TableExistsException and NamespaceExistException in
> BackupSystemTable initialization so that these do not cause client pain.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1325395461
##
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws
Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws
IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in
progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
ok, this is now taken care of
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-10278) Provide better write predictability
[ https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764997#comment-17764997 ] Ranganath Govardhanagiri commented on HBASE-10278: -- Hello [~apurtell] - Can you please provide a pointer to the more recent work (JIRA) > Provide better write predictability > --- > > Key: HBASE-10278 > URL: https://issues.apache.org/jira/browse/HBASE-10278 > Project: HBase > Issue Type: New Feature > Components: wal >Reporter: Himanshu Vashishtha >Priority: Major > Attachments: 10278-trunk-v2.1.patch, 10278-trunk-v2.1.patch, > 10278-wip-1.1.patch, Multiwaldesigndoc.pdf, SwitchWriterFlow.pptx > > > Currently, HBase has one WAL per region server. > Whenever there is any latency in the write pipeline (due to whatever reasons > such as n/w blip, a node in the pipeline having a bad disk, etc), the overall > write latency suffers. > Jonathan Hsieh and I analyzed various approaches to tackle this issue. We > also looked at HBASE-5699, which talks about adding concurrent multi WALs. > Along with performance numbers, we also focussed on design simplicity, > minimum impact on MTTR & Replication, and compatibility with 0.96 and 0.98. > Considering all these parameters, we propose a new HLog implementation with > WAL Switching functionality. > Please find attached the design doc for the same. It introduces the WAL > Switching feature, and experiments/results of a prototype implementation, > showing the benefits of this feature. > The second goal of this work is to serve as a building block for concurrent > multiple WALs feature. > Please review the doc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1325387382
##
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws
Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws
IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in
progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
i understand but i am not sure what is the best place to keep this in
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764984#comment-17764984
]
zhangbutao commented on HBASE-28061:
Attach a patch *fix_LocatedBlock__getLocations.patch* that use reflection to
fix the issue. I recompiled Hbase2.5.5( default Hadoop 3.2.4) and tested my
case in both Hadoop3.2.4 and Hadoop3.3.1, and it worked well.
[~zhangduo] Do you think this is a correct workaround to fix this issue?
Thanks.
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HbaseMiniClusterTest.java,
> fix_LocatedBlock__getLocations.patch, full_output.txt, pom.xml, root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhangbutao updated HBASE-28061:
---
Attachment: fix_LocatedBlock__getLocations.patch
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HbaseMiniClusterTest.java,
> fix_LocatedBlock__getLocations.patch, full_output.txt, pom.xml, root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang reassigned HBASE-28061:
-
Assignee: Duo Zhang
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HbaseMiniClusterTest.java, full_output.txt, pom.xml,
> root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28058) HMaster snapshot file clean thread and the snapshot request handler thread encountered a deadlock
[
https://issues.apache.org/jira/browse/HBASE-28058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haiping lv updated HBASE-28058:
---
Fix Version/s: 3.0.0
> HMaster snapshot file clean thread and the snapshot request handler thread
> encountered a deadlock
> -
>
> Key: HBASE-28058
> URL: https://issues.apache.org/jira/browse/HBASE-28058
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0-beta-1
>Reporter: Haiping lv
>Assignee: Haiping lv
>Priority: Major
> Fix For: 3.0.0
>
>
> HMaster encountered a deadlock when runing ITBLL test.
> The handler thread of HMaster holds the synchronized lock of SnapshotManager
> and competes for the readLock of takingSnapshotLock.
> {code:java}
> Thread 57 (RpcServer.default.FPBQ.Fifo.handler=22,queue=1,port=16000):
> State: WAITING
> Blocked count: 14
> Waited count: 206
> Waiting on
> java.util.concurrent.locks.ReentrantReadWriteLock$FairSync@a7a58d4
> Stack:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
>
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:675)
>
> org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1765)
>
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437)
> org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102)
> org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) {code}
> The Master Snapshot cleaning thread holds the writeLock of takingSnapshotLock
> to compete for the synchronized lock of SnapshotManager.
> {code:java}
> Thread 5405 (hfile_cleaner-dir-scan-pool-2):
> State: BLOCKED
> Blocked count: 50535
> Waited count: 92200
> Blocked on org.apache.hadoop.hbase.master.snapshot.SnapshotManager@21a16019
> Blocked by 57 (RpcServer.default.FPBQ.Fifo.handler=22,queue=1,port=16000)
> Stack:
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isTakingAnySnapshot(SnapshotManager.java:655)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getUnreferencedFiles(SnapshotFileCache.java:197)
>
> org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner.getDeletableFiles(SnapshotHFileCleaner.java:78)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:375)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$1(CleanerChore.java:483)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$788/1121007598.act(Unknown
> Source)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:565)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:483)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$2(CleanerChore.java:495)
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$794/1250580839.run(Unknown
> Source)
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> java.lang.Thread.run(Thread.java:750){{}} {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1325295375
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##
@@ -330,6 +322,73 @@ private boolean scheduleForRetry(IOException e) {
return true;
}
+private boolean unableToConnectToServerInFirstAttempt(IOException e) {
Review Comment:
I mean we could extract a method for testing exception only, and then we
test numberOfAttemptsSoFar outside the method...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache9 commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1325293263
##
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws
Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws
IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in
progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
Please do not put it in HConstants, it is IA.Public.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (HBASE-27853) Add client side table metrics for rpc calls and request latency.
[ https://issues.apache.org/jira/browse/HBASE-27853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuyaogai reassigned HBASE-27853: - Assignee: zhuyaogai > Add client side table metrics for rpc calls and request latency. > > > Key: HBASE-27853 > URL: https://issues.apache.org/jira/browse/HBASE-27853 > Project: HBase > Issue Type: Improvement > Components: Client, metrics >Affects Versions: 3.0.0-alpha-3 >Reporter: zhuyaogai >Assignee: zhuyaogai >Priority: Major > > Now we can get client metrics to see number of calls or latency of related > RPC calls like get, mutate, scan etc... > But more often, we are more concerned about table level metrics, so we need > to subdivide them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache-HBase commented on PR #5391: URL: https://github.com/apache/hbase/pull/5391#issuecomment-1718473049 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 10s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 15s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 52s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 20s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 218m 0s | hbase-server in the patch passed. | | | | 244m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5391 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e38bec87317b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/testReport/ | | Max. process+thread count | 4638 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache-HBase commented on PR #5391: URL: https://github.com/apache/hbase/pull/5391#issuecomment-1718460178 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 6s | master passed | | +1 :green_heart: | compile | 1m 21s | master passed | | +1 :green_heart: | shadedjars | 5m 16s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 46s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | +1 :green_heart: | shadedjars | 5m 13s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 17s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 32s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 218m 16s | hbase-server in the patch passed. | | | | 248m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5391 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b87086d92b18 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/testReport/ | | Max. process+thread count | 4674 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for krb auth failures
Apache-HBase commented on PR #5391: URL: https://github.com/apache/hbase/pull/5391#issuecomment-1718274909 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 39s | master passed | | +1 :green_heart: | compile | 3m 39s | master passed | | +1 :green_heart: | checkstyle | 1m 6s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 47s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 38s | the patch passed | | +1 :green_heart: | compile | 3m 38s | the patch passed | | +1 :green_heart: | javac | 3m 38s | the patch passed | | +1 :green_heart: | checkstyle | 1m 7s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 5s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 3m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 38m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5391 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux bea567ee7318 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 84ccae368f | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5391/4/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28050) RSProcedureDispatcher to fail-fast for krb auth failures
[
https://issues.apache.org/jira/browse/HBASE-28050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated HBASE-28050:
-
Summary: RSProcedureDispatcher to fail-fast for krb auth failures (was:
RSProcedureDispatcher to fail-fast for krb auth issues)
> RSProcedureDispatcher to fail-fast for krb auth failures
>
>
> Key: HBASE-28050
> URL: https://issues.apache.org/jira/browse/HBASE-28050
> Project: HBase
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>
> As discussed on the parent Jira, let's mark the remote procedures fail when
> we encounter SaslException (GSS initiate failed) as this belongs to the
> category of known IOException where we are certain that the request has not
> yet reached to the target regionserver yet.
> This should help release dispatcher threads for other
> ExecuteProceduresRemoteCall executions.
>
> Example log:
> {code:java}
> 2023-08-25 02:21:02,821 WARN [ispatcher-pool-40777]
> procedure.RSProcedureDispatcher - request to rs1,61020,1692930044498 failed
> due to java.io.IOException: Call to address=rs1:61020 failed on local
> exception: java.io.IOException:
> org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException:
> org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): GSS
> initiate failed, try=0, retrying... {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28050) RSProcedureDispatcher to fail-fast for krb auth issues
[
https://issues.apache.org/jira/browse/HBASE-28050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani updated HBASE-28050:
-
Summary: RSProcedureDispatcher to fail-fast for krb auth issues (was:
RSProcedureDispatcher to fail-fast for SaslException)
> RSProcedureDispatcher to fail-fast for krb auth issues
> --
>
> Key: HBASE-28050
> URL: https://issues.apache.org/jira/browse/HBASE-28050
> Project: HBase
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>
> As discussed on the parent Jira, let's mark the remote procedures fail when
> we encounter SaslException (GSS initiate failed) as this belongs to the
> category of known IOException where we are certain that the request has not
> yet reached to the target regionserver yet.
> This should help release dispatcher threads for other
> ExecuteProceduresRemoteCall executions.
>
> Example log:
> {code:java}
> 2023-08-25 02:21:02,821 WARN [ispatcher-pool-40777]
> procedure.RSProcedureDispatcher - request to rs1,61020,1692930044498 failed
> due to java.io.IOException: Call to address=rs1:61020 failed on local
> exception: java.io.IOException:
> org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException:
> org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): GSS
> initiate failed, try=0, retrying... {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27991) [hbase-examples] MultiThreadedClientExample throws java.lang.ClassCastException
[
https://issues.apache.org/jira/browse/HBASE-27991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764854#comment-17764854
]
Hudson commented on HBASE-27991:
Results for branch master
[build #903 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [hbase-examples] MultiThreadedClientExample throws
> java.lang.ClassCastException
> ---
>
> Key: HBASE-27991
> URL: https://issues.apache.org/jira/browse/HBASE-27991
> Project: HBase
> Issue Type: Bug
>Reporter: Nikita Pande
>Assignee: Nikita Pande
>Priority: Minor
> Fix For: 3.0.0-beta-1, 4.0.0-alpha-1
>
>
> Tried using run() method call of
> [https://github.com/apache/hbase/blob/master/hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/MultiThreadedClientExample.java|https://github.com/apache/hbase/blob/master/hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/MultiThreadedClientExample.java.]
> Following the stack trace of error during runtime
> {code:java}
> Exception in thread "main" java.io.IOException:
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.lambda$createConnection$1(ConnectionFactory.java:235)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:160)
> at
> org.apache.hadoop.hbase.client.example.MultiThreadedClientExample.run(MultiThreadedClientExample.java:136)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at .runMultiThreadedRWOps(xx)
> at .main(xx)
> Caused by: java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1780)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:328)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.lambda$createConnection$1(ConnectionFactory.java:232)
> ... 8 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.GeneratedConstructorAccessor16.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.lambda$null$0(ConnectionFactory.java:233)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
> ... 10 more
> Caused by: java.lang.ClassCastException: java.util.concurrent.ForkJoinPool
> cannot be cast to java.util.concurrent.ThreadPoolExecutor
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:283)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:270)
> ... 17 more
> {code}
> Reason: due to HBASE-22244 Make use of MetricsConnection in async client
> following changes
> {code:java}
> // thread executor shared by all Table instances created
> // by this connection
> private volatile ExecutorService batchPool = null; got modified to private
> volatile ThreadPoolExecutor batchPool = null;
> {code}
> ExecutorService got changed to ThreadPoolExecutor. ExecutorService is
> compatible with ForkJoinPool however ThreadPoolExecutor is not compatible.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28080) correct span name in AbstractRpcBasedConnectionRegistry#getActiveMaster
[
https://issues.apache.org/jira/browse/HBASE-28080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764857#comment-17764857
]
Hudson commented on HBASE-28080:
Results for branch master
[build #903 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> correct span name in AbstractRpcBasedConnectionRegistry#getActiveMaster
> ---
>
> Key: HBASE-28080
> URL: https://issues.apache.org/jira/browse/HBASE-28080
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 3.0.0-alpha-4
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Trivial
> Fix For: 3.0.0-beta-1
>
>
> It looks like that the span name does not correspond to what is actually done.
>
> public CompletableFuture getActiveMaster() {
> return tracedFuture(
> () -> this
> . call(
> (c, s, d) -> s.getActiveMaster(c,
> GetActiveMasterRequest.getDefaultInstance(), d),
> GetActiveMasterResponse::hasServerName, "getActiveMaster()")
> .thenApply(resp -> ProtobufUtil.toServerName(resp.getServerName())),
> getClass().getSimpleName() + ".getClusterId");
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27853) Add client side table metrics for rpc calls and request latency.
[
https://issues.apache.org/jira/browse/HBASE-27853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764856#comment-17764856
]
Hudson commented on HBASE-27853:
Results for branch master
[build #903 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add client side table metrics for rpc calls and request latency.
>
>
> Key: HBASE-27853
> URL: https://issues.apache.org/jira/browse/HBASE-27853
> Project: HBase
> Issue Type: Improvement
> Components: Client, metrics
>Affects Versions: 3.0.0-alpha-3
>Reporter: zhuyaogai
>Priority: Major
>
> Now we can get client metrics to see number of calls or latency of related
> RPC calls like get, mutate, scan etc...
> But more often, we are more concerned about table level metrics, so we need
> to subdivide them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28008) Add support for tcnative
[
https://issues.apache.org/jira/browse/HBASE-28008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764855#comment-17764855
]
Hudson commented on HBASE-28008:
Results for branch master
[build #903 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/903/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add support for tcnative
>
>
> Key: HBASE-28008
> URL: https://issues.apache.org/jira/browse/HBASE-28008
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: tls
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> In investigating HBASE-27947, tcnative can make a big impact on throughput
> over built-in jdk SSL support. We need three things to make it work:
> # In X509Util, if Openssl.isAvailable() (meaning tcnative is on the
> classpath):
> ## Use SslProvider.OPENSSL_REFCNT
> ## Update default ciphers to remove CBC ciphers, which do not work with
> tcnative. We can either pull the ciphers from
> OpenSsl.availableJavaCipherSuites() or simply use the default GCM ciphers we
> already have defined.
> # Our netty is shaded, so one can't simply put the tcnative jar on the
> classpath. We might need to provide an hbase-shaded-netty-tcnative module
> which one can optionally include in their deployment. We will have to decide
> which of the many tcnative modules to provide a shaded version for.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27853) Add client side table metrics for rpc calls and request latency.
[
https://issues.apache.org/jira/browse/HBASE-27853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764852#comment-17764852
]
Hudson commented on HBASE-27853:
Results for branch branch-3
[build #44 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add client side table metrics for rpc calls and request latency.
>
>
> Key: HBASE-27853
> URL: https://issues.apache.org/jira/browse/HBASE-27853
> Project: HBase
> Issue Type: Improvement
> Components: Client, metrics
>Affects Versions: 3.0.0-alpha-3
>Reporter: zhuyaogai
>Priority: Major
>
> Now we can get client metrics to see number of calls or latency of related
> RPC calls like get, mutate, scan etc...
> But more often, we are more concerned about table level metrics, so we need
> to subdivide them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27991) [hbase-examples] MultiThreadedClientExample throws java.lang.ClassCastException
[
https://issues.apache.org/jira/browse/HBASE-27991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764850#comment-17764850
]
Hudson commented on HBASE-27991:
Results for branch branch-3
[build #44 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [hbase-examples] MultiThreadedClientExample throws
> java.lang.ClassCastException
> ---
>
> Key: HBASE-27991
> URL: https://issues.apache.org/jira/browse/HBASE-27991
> Project: HBase
> Issue Type: Bug
>Reporter: Nikita Pande
>Assignee: Nikita Pande
>Priority: Minor
> Fix For: 3.0.0-beta-1, 4.0.0-alpha-1
>
>
> Tried using run() method call of
> [https://github.com/apache/hbase/blob/master/hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/MultiThreadedClientExample.java|https://github.com/apache/hbase/blob/master/hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/MultiThreadedClientExample.java.]
> Following the stack trace of error during runtime
> {code:java}
> Exception in thread "main" java.io.IOException:
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.lambda$createConnection$1(ConnectionFactory.java:235)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:160)
> at
> org.apache.hadoop.hbase.client.example.MultiThreadedClientExample.run(MultiThreadedClientExample.java:136)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at .runMultiThreadedRWOps(xx)
> at .main(xx)
> Caused by: java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1780)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:328)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.lambda$createConnection$1(ConnectionFactory.java:232)
> ... 8 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.GeneratedConstructorAccessor16.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.lambda$null$0(ConnectionFactory.java:233)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
> ... 10 more
> Caused by: java.lang.ClassCastException: java.util.concurrent.ForkJoinPool
> cannot be cast to java.util.concurrent.ThreadPoolExecutor
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:283)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:270)
> ... 17 more
> {code}
> Reason: due to HBASE-22244 Make use of MetricsConnection in async client
> following changes
> {code:java}
> // thread executor shared by all Table instances created
> // by this connection
> private volatile ExecutorService batchPool = null; got modified to private
> volatile ThreadPoolExecutor batchPool = null;
> {code}
> ExecutorService got changed to ThreadPoolExecutor. ExecutorService is
> compatible with ForkJoinPool however ThreadPoolExecutor is not compatible.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28008) Add support for tcnative
[
https://issues.apache.org/jira/browse/HBASE-28008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764851#comment-17764851
]
Hudson commented on HBASE-28008:
Results for branch branch-3
[build #44 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add support for tcnative
>
>
> Key: HBASE-28008
> URL: https://issues.apache.org/jira/browse/HBASE-28008
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: tls
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> In investigating HBASE-27947, tcnative can make a big impact on throughput
> over built-in jdk SSL support. We need three things to make it work:
> # In X509Util, if Openssl.isAvailable() (meaning tcnative is on the
> classpath):
> ## Use SslProvider.OPENSSL_REFCNT
> ## Update default ciphers to remove CBC ciphers, which do not work with
> tcnative. We can either pull the ciphers from
> OpenSsl.availableJavaCipherSuites() or simply use the default GCM ciphers we
> already have defined.
> # Our netty is shaded, so one can't simply put the tcnative jar on the
> classpath. We might need to provide an hbase-shaded-netty-tcnative module
> which one can optionally include in their deployment. We will have to decide
> which of the many tcnative modules to provide a shaded version for.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28080) correct span name in AbstractRpcBasedConnectionRegistry#getActiveMaster
[
https://issues.apache.org/jira/browse/HBASE-28080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764853#comment-17764853
]
Hudson commented on HBASE-28080:
Results for branch branch-3
[build #44 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/44/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> correct span name in AbstractRpcBasedConnectionRegistry#getActiveMaster
> ---
>
> Key: HBASE-28080
> URL: https://issues.apache.org/jira/browse/HBASE-28080
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 3.0.0-alpha-4
>Reporter: ruanhui
>Assignee: ruanhui
>Priority: Trivial
> Fix For: 3.0.0-beta-1
>
>
> It looks like that the span name does not correspond to what is actually done.
>
> public CompletableFuture getActiveMaster() {
> return tracedFuture(
> () -> this
> . call(
> (c, s, d) -> s.getActiveMaster(c,
> GetActiveMasterRequest.getDefaultInstance(), d),
> GetActiveMasterResponse::hasServerName, "getActiveMaster()")
> .thenApply(resp -> ProtobufUtil.toServerName(resp.getServerName())),
> getClass().getSimpleName() + ".getClusterId");
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] virajjasani commented on a diff in pull request #5391: HBASE-28050 RSProcedureDispatcher to fail-fast for SaslException
virajjasani commented on code in PR #5391:
URL: https://github.com/apache/hbase/pull/5391#discussion_r1324992854
##
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java:
##
@@ -347,7 +348,7 @@ public void operationComplete(ChannelFuture future) throws
Exception {
private void sendRequest0(Call call, HBaseRpcController hrc) throws
IOException {
assert eventLoop.inEventLoop();
if (reloginInProgress) {
- throw new IOException("Can not send request because relogin is in
progress.");
+ throw new IOException(HConstants.RELOGIN_IS_IN_PROGRESS);
Review Comment:
not sure what is the best place to keep this, anywhere else in hbase-common
would also work
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java:
##
@@ -306,6 +308,10 @@ private boolean scheduleForRetry(IOException e) {
serverName, e.toString(), numberOfAttemptsSoFar);
return false;
}
+ if (isSaslError(e) && numberOfAttemptsSoFar == 0) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-28055) Performance improvement for scan over several stores.
[
https://issues.apache.org/jira/browse/HBASE-28055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764846#comment-17764846
]
Hudson commented on HBASE-28055:
Results for branch branch-2.5
[build #401 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/401/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/401/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/401/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/401/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/401/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Performance improvement for scan over several stores.
> --
>
> Key: HBASE-28055
> URL: https://issues.apache.org/jira/browse/HBASE-28055
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-4, 2.5.5
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
>
>
> During the fix of HBASE-19863, an additional check for fake cells that
> trigger reseek was added. It comes that this check produces unnecessary
> reseeks because
> matcher.compareKeyForNextColumn should be used only with indexed keys. Later
> [~larsh] suggested doing a simple check for OLD_TIMESTAMP and it looks like a
> better solution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28082) oldWALs naming can be incompatible with HBase backup
[
https://issues.apache.org/jira/browse/HBASE-28082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764782#comment-17764782
]
Bryan Beaudreault commented on HBASE-28082:
---
I think the problem is related to your usage of multiwal for
hbase.wal.provider. It seems like that feature adds a ".regiongroup-#" suffix
to the WAL path name. I do think the bug is in the backup system, we should fix
BackupUtils#parseHostFromOldLog to ignore that before trying to extract a
ServerName.
> oldWALs naming can be incompatible with HBase backup
>
>
> Key: HBASE-28082
> URL: https://issues.apache.org/jira/browse/HBASE-28082
> Project: HBase
> Issue Type: Bug
> Environment: Encountered on HBase
> a2e7d2015e9f603e46339d0582e29a86843b9324 (branch-2), running in Kubernetes.
>Reporter: Dieter De Paepe
>Priority: Major
>
> I am testing HBase backup functionality, and noticed following warning when
> running "hbase backup create incremental ...":
>
> {noformat}
> 23/09/13 15:44:10 WARN org.apache.hadoop.hbase.backup.util.BackupUtils: Skip
> log file (can't parse):
> hdfs://hdfsns/hbase/hbase/oldWALs/hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.regiongroup-0.1694609969312{noformat}
> It appears in my setup, the oldWALs are indeed given names that seem to break
> "ServerName.valueOf(s)" in "BackupUtils#parseHostFromOldLog(Path p)":
>
>
> {noformat}
> user@hadoop-client-769bc9946-xqrt2:/$ hdfs dfs -ls hdfs:///hbase/hbase/oldWALs
> Found 42 items
> -rw-r--r-- 1 hbase hbase 775421 2023-09-13 13:14
> hdfs:///hbase/hbase/oldWALs/hbase-master-0.minikube-shared%2C16000%2C1694609954719.hbase-master-0.minikube-shared%2C16000%2C1694609954719.regiongroup-0.1694609957984$masterlocalwal$
> -rw-r--r-- 1 hbase hbase 26059 2023-09-13 13:29
> hdfs:///hbase/hbase/oldWALs/hbase-master-0.minikube-shared%2C16000%2C1694609954719.hbase-master-0.minikube-shared%2C16000%2C1694609954719.regiongroup-0.1694610867894$masterlocalwal$
> ...
> -rw-r--r-- 1 hbase hbase 242479 2023-09-13 14:16
> hdfs:///hbase/hbase/oldWALs/hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.regiongroup-0.1694609969312
> -rw-r--r-- 1 hbase hbase 4364 2023-09-13 14:16
> hdfs:///hbase/hbase/oldWALs/hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.regiongroup-0.1694610188654
> ...
> -rw-r--r-- 1 hbase hbase 70802 2023-09-13 13:15
> hdfs:///hbase/hbase/oldWALs/hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.meta.1694609970025.meta
> -rw-r--r-- 1 hbase hbase 93 2023-09-13 13:04
> hdfs:///hbase/hbase/oldWALs/hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.meta.1694610188627.meta
> ...{noformat}
> I'd say this is not a bug in the backup system, but rather in whatever gives
> the oldWAL files its name. I'm however not that familiar with HBase code to
> find where these files are created. Any pointers are appreciated.
> Given that this causes some logs to be missed during backup, I guess this can
> lead to data loss in a backup restore?
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28082) oldWALs naming can be incompatible with HBase backup
Dieter De Paepe created HBASE-28082:
---
Summary: oldWALs naming can be incompatible with HBase backup
Key: HBASE-28082
URL: https://issues.apache.org/jira/browse/HBASE-28082
Project: HBase
Issue Type: Bug
Environment: Encountered on HBase
a2e7d2015e9f603e46339d0582e29a86843b9324 (branch-2), running in Kubernetes.
Reporter: Dieter De Paepe
I am testing HBase backup functionality, and noticed following warning when
running "hbase backup create incremental ...":
{noformat}
23/09/13 15:44:10 WARN org.apache.hadoop.hbase.backup.util.BackupUtils: Skip
log file (can't parse):
hdfs://hdfsns/hbase/hbase/oldWALs/hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.regiongroup-0.1694609969312{noformat}
It appears in my setup, the oldWALs are indeed given names that seem to break
"ServerName.valueOf(s)" in "BackupUtils#parseHostFromOldLog(Path p)":
{noformat}
user@hadoop-client-769bc9946-xqrt2:/$ hdfs dfs -ls hdfs:///hbase/hbase/oldWALs
Found 42 items
-rw-r--r-- 1 hbase hbase 775421 2023-09-13 13:14
hdfs:///hbase/hbase/oldWALs/hbase-master-0.minikube-shared%2C16000%2C1694609954719.hbase-master-0.minikube-shared%2C16000%2C1694609954719.regiongroup-0.1694609957984$masterlocalwal$
-rw-r--r-- 1 hbase hbase 26059 2023-09-13 13:29
hdfs:///hbase/hbase/oldWALs/hbase-master-0.minikube-shared%2C16000%2C1694609954719.hbase-master-0.minikube-shared%2C16000%2C1694609954719.regiongroup-0.1694610867894$masterlocalwal$
...
-rw-r--r-- 1 hbase hbase 242479 2023-09-13 14:16
hdfs:///hbase/hbase/oldWALs/hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.regiongroup-0.1694609969312
-rw-r--r-- 1 hbase hbase 4364 2023-09-13 14:16
hdfs:///hbase/hbase/oldWALs/hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.regiongroup-0.1694610188654
...
-rw-r--r-- 1 hbase hbase 70802 2023-09-13 13:15
hdfs:///hbase/hbase/oldWALs/hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.meta.1694609970025.meta
-rw-r--r-- 1 hbase hbase 93 2023-09-13 13:04
hdfs:///hbase/hbase/oldWALs/hbase-region-0.hbase-region.minikube-shared.svc.cluster.local%2C16020%2C1694609964681.meta.1694610188627.meta
...{noformat}
I'd say this is not a bug in the backup system, but rather in whatever gives
the oldWAL files its name. I'm however not that familiar with HBase code to
find where these files are created. Any pointers are appreciated.
Given that this causes some logs to be missed during backup, I guess this can
lead to data loss in a backup restore?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (HBASE-25382) Super Split Meta
[
https://issues.apache.org/jira/browse/HBASE-25382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764770#comment-17764770
]
ruanhui edited comment on HBASE-25382 at 9/13/23 4:16 PM:
--
Hi all
In our production, there are many large-scale clusters with very high read load
requests for meta tables. On these clusters, meta table read requests can reach
over 400,000 per second. Even if we try to improve the read performance of the
meta server by increasing the size of priority thread pool to 1500 and making
the meta table always have a minimum number of files, the latency of P99/P99 is
still very high.
There are two reasons for the high request of the meta server.
1. Our users access the cluster through a large number of C++ client SDKs
deployed in k8s.
As we can imagine, when a region server crashes or the cluster is balancing due
to expansion/shrinking, there will be a large number of retry requests from
clients that will first access the meta table to obtain the latest region
location information. Moreover, because the client is c++ SDK, region replica
is not an option.
2. We have many tables with many regions because of the large amount of data.
The largest table has nearly 100,000 regions. These tables are accessed 3 times
a day by multiple flink/spark applications with a parallelism of more than
2000. So when the application starts, there will be a large number of meta
table requests to obtain the region location.
So, the meta table is the bottlenecks limiting the horizontal expansion of our
hbase cluster. I'm sure there are many people who have the same problem as us
if their clusters are large too. We have a strong demand for the feature of
meta split. Can we consider restarting the discussion and development of such a
great feature ？
was (Author: frostruan):
Hi all
In our production environment, there are many large-scale clusters with very
high read load requests for meta tables. On these clusters, meta table read
requests can reach over 400,000 per second. Even if we try to improve the read
performance of the meta server by increasing the size of priority thread pool
to 1500 and making the meta table always have a minimum number of files, the
latency of P99/P99 is still very high.
There are two reasons for the high request of the meta server.
1. Our users access the cluster through a large number of C++ client SDKs
deployed in k8s.
As we can imagine, when a region server crashes or the cluster is balancing due
to expansion/shrinking, there will be a large number of retry requests from
clients that will first access the meta table to obtain the latest region
location information. Moreover, because the client is c++ SDK, region replica
is not an option.
2. We have many tables with many regions because of the large amount of data.
The largest table has nearly 100,000 regions. These tables are accessed 3 times
a day by multiple flink/spark applications with a parallelism of more than
2000. So when the application starts, there will be a large number of meta
table requests to obtain the region location.
So, the meta table is the bottlenecks limiting the horizontal expansion of our
hbase cluster. I'm sure there are many people who have the same problem as us
if their clusters are large too. We have a strong demand for the feature of
meta split. Can we consider restarting the discussion and development of such a
great feature ？
> Super Split Meta
>
>
> Key: HBASE-25382
> URL: https://issues.apache.org/jira/browse/HBASE-25382
> Project: HBase
> Issue Type: Umbrella
> Components: meta
>Reporter: Michael Stack
>Priority: Major
>
> Currently, the hbase:meta Table is made of one Region only. It is not
> splittable. We would like to make it so the hbase:meta table can be split
> just as we for user-space tables as they grow in size.
> Why split the hbase:meta table?
> {quote}A single Region _hbase:meta_ Table hotspots as cluster size grows. At
> extreme, hotspotting brings on special-casing with deploys dedicating nodes
> solely to the hosting of the single _hbase:meta_ Region to better carry the
> higher load (but also to isolate _hbase:meta_ if heavily trafficked neighbor
> Regions). Splitting, currently disallowed, will enable distributing the
> _hbase:meta_ Table Regions, and thereby load, across the cluster. A
> splittable _hbase:meta_ table will also alleviate concerns enlarging the
> _hbase:meta_ table whether by adding more meta data per Region entry -- e.g.
> keeping the list of Region HFiles in the _hbase:meta_ table -- or running
> with more, smaller Regions rather than a few large Regions.
> {quote}
> This is not the first issue to concern itself with meta splitting
> (HBASE-11288, HBASE-24950). By aggreement. -- see the base of HB
[jira] [Commented] (HBASE-25382) Super Split Meta
[
https://issues.apache.org/jira/browse/HBASE-25382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764770#comment-17764770
]
ruanhui commented on HBASE-25382:
-
Hi all
In our production environment, there are many large-scale clusters with very
high read load requests for meta tables. On these clusters, meta table read
requests can reach over 400,000 per second. Even if we try to improve the read
performance of the meta server by increasing the size of priority thread pool
to 1500 and making the meta table always have a minimum number of files, the
latency of P99/P99 is still very high.
There are two reasons for the high request of the meta server.
1. Our users access the cluster through a large number of C++ client SDKs
deployed in k8s.
As we can imagine, when a region server crashes or the cluster is balancing due
to expansion/shrinking, there will be a large number of retry requests from
clients that will first access the meta table to obtain the latest region
location information. Moreover, because the client is c++ SDK, region replica
is not an option.
2. We have many tables with many regions because of the large amount of data.
The largest table has nearly 100,000 regions. These tables are accessed 3 times
a day by multiple flink/spark applications with a parallelism of more than
2000. So when the application starts, there will be a large number of meta
table requests to obtain the region location.
So, the meta table is the bottlenecks limiting the horizontal expansion of our
hbase cluster. I'm sure there are many people who have the same problem as us
if their clusters are large too. We have a strong demand for the feature of
meta split. Can we consider restarting the discussion and development of such a
great feature ？
> Super Split Meta
>
>
> Key: HBASE-25382
> URL: https://issues.apache.org/jira/browse/HBASE-25382
> Project: HBase
> Issue Type: Umbrella
> Components: meta
>Reporter: Michael Stack
>Priority: Major
>
> Currently, the hbase:meta Table is made of one Region only. It is not
> splittable. We would like to make it so the hbase:meta table can be split
> just as we for user-space tables as they grow in size.
> Why split the hbase:meta table?
> {quote}A single Region _hbase:meta_ Table hotspots as cluster size grows. At
> extreme, hotspotting brings on special-casing with deploys dedicating nodes
> solely to the hosting of the single _hbase:meta_ Region to better carry the
> higher load (but also to isolate _hbase:meta_ if heavily trafficked neighbor
> Regions). Splitting, currently disallowed, will enable distributing the
> _hbase:meta_ Table Regions, and thereby load, across the cluster. A
> splittable _hbase:meta_ table will also alleviate concerns enlarging the
> _hbase:meta_ table whether by adding more meta data per Region entry -- e.g.
> keeping the list of Region HFiles in the _hbase:meta_ table -- or running
> with more, smaller Regions rather than a few large Regions.
> {quote}
> This is not the first issue to concern itself with meta splitting
> (HBASE-11288, HBASE-24950). By aggreement. -- see the base of HBASE-11288 --
> this issue supplants all previous JIRAs and design efforts. It is a reset. We
> start by listing requirements in the attached Super Split Meta Design.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28079) Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
[
https://issues.apache.org/jira/browse/HBASE-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault updated HBASE-28079:
--
Labels: (was: patch-available)
> Unhandled TableExistsException and NamespaceExistException in
> BackupSystemTable
> ---
>
> Key: HBASE-28079
> URL: https://issues.apache.org/jira/browse/HBASE-28079
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Most backup-related operations instantiate a BackupSystemTable object. The
> first thing it does is check that the {{backup}} namespace and backup-related
> tables exist, and create them if not. One way in which this can be triggered
> is via LoadIncrementalHFiles, which can load files with a configurable number
> of threads. When bulkLoadHFile is called, a preCommitHFile hook is triggered
> which does the above. With a certain level of concurrency, one can easily
> trigger a race condition on creating the namespace or tables. This exception
> bubbles up and fails the bulkload.
> We should catch TableExistsException and NamespaceExistException in
> BackupSystemTable initialization so that these do not cause client pain.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28079) Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
[
https://issues.apache.org/jira/browse/HBASE-28079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault resolved HBASE-28079.
---
Fix Version/s: 2.6.0
3.0.0-beta-1
Resolution: Fixed
Merged to master, branch-3, and branch-2. Thanks [~zhangduo] for the quick
review
> Unhandled TableExistsException and NamespaceExistException in
> BackupSystemTable
> ---
>
> Key: HBASE-28079
> URL: https://issues.apache.org/jira/browse/HBASE-28079
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> Most backup-related operations instantiate a BackupSystemTable object. The
> first thing it does is check that the {{backup}} namespace and backup-related
> tables exist, and create them if not. One way in which this can be triggered
> is via LoadIncrementalHFiles, which can load files with a configurable number
> of threads. When bulkLoadHFile is called, a preCommitHFile hook is triggered
> which does the above. With a certain level of concurrency, one can easily
> trigger a race condition on creating the namespace or tables. This exception
> bubbles up and fails the bulkload.
> We should catch TableExistsException and NamespaceExistException in
> BackupSystemTable initialization so that these do not cause client pain.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] bbeaudreault merged pull request #5399: HBASE-28079 Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
bbeaudreault merged PR #5399: URL: https://github.com/apache/hbase/pull/5399 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27921) Bump up jruby to 9.4.2.0 and related joni and jcodings to 2.1.48 and 1.0.58 respectively
[
https://issues.apache.org/jira/browse/HBASE-27921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sercan Tekin updated HBASE-27921:
-
Description:
Current version of *jruby* (9.3.9.0) has snakeyaml dependency version 1.33.
{code:java}
> find ./jruby-complete-9.3.9.0/ -name "*snakeyaml*"
./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml
./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar
{code}
*snakeyaml-1.33* has a critical CVE
[https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1471]
{*}jruby-complete-9.4.2{*}{*}.0{*} uses {*}snakeyaml-engine-2.6{*}.
was:
Current version of *jruby* (9.3.9.0) has snakeyaml dependency version 1.33.
{code:java}
> find ./jruby-complete-9.3.9.0/ -name "*snakeyaml*"
./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml
./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar
{code}
*snakeyaml-1.33* has a critical CVE
[https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1471]
*jruby-complete-9.3.11.0* uses {*}snakeyaml-engine-2.6{*}.
> Bump up jruby to 9.4.2.0 and related joni and jcodings to 2.1.48 and 1.0.58
> respectively
>
>
> Key: HBASE-27921
> URL: https://issues.apache.org/jira/browse/HBASE-27921
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-4
>Reporter: Sercan Tekin
>Priority: Major
> Labels: jruby, security, shell
> Fix For: 3.0.0-beta-1
>
>
> Current version of *jruby* (9.3.9.0) has snakeyaml dependency version 1.33.
> {code:java}
> > find ./jruby-complete-9.3.9.0/ -name "*snakeyaml*"
> ./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml
> ./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar
> {code}
> *snakeyaml-1.33* has a critical CVE
> [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1471]
> {*}jruby-complete-9.4.2{*}{*}.0{*} uses {*}snakeyaml-engine-2.6{*}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27921) Bump up jruby to 9.4.2.0 and related joni and jcodings to 2.1.48 and 1.0.58 respectively
[
https://issues.apache.org/jira/browse/HBASE-27921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sercan Tekin updated HBASE-27921:
-
Summary: Bump up jruby to 9.4.2.0 and related joni and jcodings to 2.1.48
and 1.0.58 respectively (was: Bump up jruby to 9.3.11.0 and related joni and
jcodings to 2.2.1 and 1.0.58 respectively)
> Bump up jruby to 9.4.2.0 and related joni and jcodings to 2.1.48 and 1.0.58
> respectively
>
>
> Key: HBASE-27921
> URL: https://issues.apache.org/jira/browse/HBASE-27921
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-4
>Reporter: Sercan Tekin
>Priority: Major
> Labels: jruby, security, shell
> Fix For: 3.0.0-beta-1
>
>
> Current version of *jruby* (9.3.9.0) has snakeyaml dependency version 1.33.
> {code:java}
> > find ./jruby-complete-9.3.9.0/ -name "*snakeyaml*"
> ./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml
> ./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar
> {code}
> *snakeyaml-1.33* has a critical CVE
> [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1471]
> *jruby-complete-9.3.11.0* uses {*}snakeyaml-engine-2.6{*}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27921) Bump up jruby to 9.3.11.0 and related joni and jcodings to 2.2.1 and 1.0.58 respectively
[
https://issues.apache.org/jira/browse/HBASE-27921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sercan Tekin updated HBASE-27921:
-
Description:
Current version of *jruby* (9.3.9.0) has snakeyaml dependency version 1.33.
{code:java}
> find ./jruby-complete-9.3.9.0/ -name "*snakeyaml*"
./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml
./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar
{code}
*snakeyaml-1.33* has a critical CVE
[https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1471]
*jruby-complete-9.3.11.0* uses {*}snakeyaml-engine-2.6{*}.
was:
Current version of *jruby* (9.3.9.0) has snakeyaml dependency version 1.33.
{code:java}
> find ./jruby-complete-9.3.9.0/ -name "*snakeyaml*"
./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml
./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar
{code}
*snakeyaml-1.33* has a critical CVE
[https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1471]
*jruby-complete-9.4.2.0* uses {*}snakeyaml-engine-2.6{*}.
> Bump up jruby to 9.3.11.0 and related joni and jcodings to 2.2.1 and 1.0.58
> respectively
>
>
> Key: HBASE-27921
> URL: https://issues.apache.org/jira/browse/HBASE-27921
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-4
>Reporter: Sercan Tekin
>Priority: Major
> Labels: jruby, security, shell
> Fix For: 3.0.0-beta-1
>
>
> Current version of *jruby* (9.3.9.0) has snakeyaml dependency version 1.33.
> {code:java}
> > find ./jruby-complete-9.3.9.0/ -name "*snakeyaml*"
> ./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml
> ./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar
> {code}
> *snakeyaml-1.33* has a critical CVE
> [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1471]
> *jruby-complete-9.3.11.0* uses {*}snakeyaml-engine-2.6{*}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27921) Bump up jruby to 9.3.11.0 and related joni and jcodings to 2.2.1 and 1.0.58 respectively
[
https://issues.apache.org/jira/browse/HBASE-27921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sercan Tekin updated HBASE-27921:
-
Summary: Bump up jruby to 9.3.11.0 and related joni and jcodings to 2.2.1
and 1.0.58 respectively (was: Bump up jruby to 9.4.2.0 and related joni and
jcodings to 2.1.48 and 1.0.58 respectively)
> Bump up jruby to 9.3.11.0 and related joni and jcodings to 2.2.1 and 1.0.58
> respectively
>
>
> Key: HBASE-27921
> URL: https://issues.apache.org/jira/browse/HBASE-27921
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-4
>Reporter: Sercan Tekin
>Priority: Major
> Labels: jruby, security, shell
> Fix For: 3.0.0-beta-1
>
>
> Current version of *jruby* (9.3.9.0) has snakeyaml dependency version 1.33.
> {code:java}
> > find ./jruby-complete-9.3.9.0/ -name "*snakeyaml*"
> ./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml
> ./jruby-complete-9.3.9.0/META-INF/jruby.home/lib/ruby/stdlib/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar
> {code}
> *snakeyaml-1.33* has a critical CVE
> [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-1471]
> *jruby-complete-9.4.2.0* uses {*}snakeyaml-engine-2.6{*}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5399: HBASE-28079 Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
Apache-HBase commented on PR #5399: URL: https://github.com/apache/hbase/pull/5399#issuecomment-1717601811 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | master passed | | +1 :green_heart: | compile | 0m 17s | master passed | | +1 :green_heart: | shadedjars | 5m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 18s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 33s | the patch passed | | +1 :green_heart: | compile | 0m 21s | the patch passed | | +1 :green_heart: | javac | 0m 21s | the patch passed | | +1 :green_heart: | shadedjars | 6m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 27s | hbase-backup in the patch passed. | | | | 31m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5399/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5399 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7b25bda0071b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 35667c1fa4 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5399/2/testReport/ | | Max. process+thread count | 3658 (vs. ulimit of 3) | | modules | C: hbase-backup U: hbase-backup | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5399/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5399: HBASE-28079 Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
Apache-HBase commented on PR #5399: URL: https://github.com/apache/hbase/pull/5399#issuecomment-1717588383 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 15s | master passed | | +1 :green_heart: | compile | 0m 16s | master passed | | +1 :green_heart: | shadedjars | 4m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 13s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 17s | the patch passed | | +1 :green_heart: | compile | 0m 15s | the patch passed | | +1 :green_heart: | javac | 0m 15s | the patch passed | | +1 :green_heart: | shadedjars | 4m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 25s | hbase-backup in the patch passed. | | | | 25m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5399/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5399 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 96d32e56ec10 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 35667c1fa4 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5399/2/testReport/ | | Max. process+thread count | 3587 (vs. ulimit of 3) | | modules | C: hbase-backup U: hbase-backup | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5399/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5399: HBASE-28079 Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
Apache-HBase commented on PR #5399: URL: https://github.com/apache/hbase/pull/5399#issuecomment-1717586902 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 31s | master passed | | +1 :green_heart: | compile | 0m 23s | master passed | | +1 :green_heart: | checkstyle | 0m 10s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 31s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | checkstyle | 0m 11s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 8m 58s | Patch does not cause any errors with Hadoop 3.2.4 3.3.5. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 23m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5399/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5399 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 909cd8bed080 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 35667c1fa4 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-backup U: hbase-backup | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5399/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] hiping-tech commented on pull request #5381: HBASE-28058 Adjust the order of acquiring the takeSnapshot locks to r…esolve the deadlock issue.
hiping-tech commented on PR #5381: URL: https://github.com/apache/hbase/pull/5381#issuecomment-1717569067 @Apache9 Thanks for your reminder. I will pay attention next time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-20447) Only fail cacheBlock if block collisions aren't related to next block metadata
[ https://issues.apache.org/jira/browse/HBASE-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuobin zheng updated HBASE-20447: -- Description: This is the issue I was originally having here: [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] When we pread, we don't force the read to read all of the next block header. However, when we get into a race condition where two opener threads try to cache the same block and one thread read all of the next block header and the other one didn't, it will fail the open process. This is especially important in a splitting case where it will potentially fail the split process. Instead, in the caches, we should only fail if the required blocks are different. was: This is the issue I was originally having here: [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] When we pread, we don't force the read to read all of the next block header. However, when we get into a race condition where two opener threads try to cache the same block and one thread read all of the next block header and the other one didn't, it will fail the open process. This is especially important in a splitting case where it will potentially fail the split process. Instead, in the caches, we should only fail if the required blocks are different. > Only fail cacheBlock if block collisions aren't related to next block metadata > -- > > Key: HBASE-20447 > URL: https://issues.apache.org/jira/browse/HBASE-20447 > Project: HBase > Issue Type: Bug > Components: BlockCache, BucketCache >Affects Versions: 1.4.3, 2.0.0 >Reporter: Zach York >Assignee: Zach York >Priority: Major > Fix For: 3.0.0-alpha-1, 2.1.0, 1.4.5 > > Attachments: HBASE-20447.branch-1.001.patch, > HBASE-20447.branch-1.002.patch, HBASE-20447.branch-1.003.patch, > HBASE-20447.branch-1.004.patch, HBASE-20447.branch-1.005.patch, > HBASE-20447.branch-1.006.patch, HBASE-20447.master.001.patch, > HBASE-20447.master.002.patch, HBASE-20447.master.003.patch, > HBASE-20447.master.004.patch > > > This is the issue I was originally having here: > [http://mail-archives.apache.org/mod_mbox/hbase-dev/201802.mbox/%3CCAN+qs_Pav=md_aoj4xji+kcnetubg2xou2ntxv1g6m8-5vn...@mail.gmail.com%3E] > > When we pread, we don't force the read to read all of the next block header. > However, when we get into a race condition where two opener threads try to > cache the same block and one thread read all of the next block header and the > other one didn't, it will fail the open process. This is especially important > in a splitting case where it will potentially fail the split process. > Instead, in the caches, we should only fail if the required blocks are > different. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache9 commented on pull request #5381: HBASE-28058 Adjust the order of acquiring the takeSnapshot locks to r…esolve the deadlock issue.
Apache9 commented on PR #5381: URL: https://github.com/apache/hbase/pull/5381#issuecomment-1717370972 Oh, just notice that the PR is opened against branch-3... Next time please open an PR against master branch. In HBase we will first apply changes to master branch, and then cherry-pick them back to other branches. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #5381: HBASE-28058 Adjust the order of acquiring the takeSnapshot locks to r…esolve the deadlock issue.
Apache9 merged PR #5381: URL: https://github.com/apache/hbase/pull/5381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a diff in pull request #5399: HBASE-28079 Unhandled TableExistsException and NamespaceExistException in BackupSystemTable
Apache9 commented on code in PR #5399:
URL: https://github.com/apache/hbase/pull/5399#discussion_r1324309903
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupSystemTable.java:
##
@@ -202,17 +204,25 @@ private void checkSystemTable() throws IOException {
Configuration conf = connection.getConfiguration();
if (!admin.tableExists(tableName)) {
TableDescriptor backupHTD =
BackupSystemTable.getSystemTableDescriptor(conf);
-admin.createTable(backupHTD);
+createSystemTable(admin, backupHTD);
}
if (!admin.tableExists(bulkLoadTableName)) {
TableDescriptor blHTD =
BackupSystemTable.getSystemTableForBulkLoadedDataDescriptor(conf);
-admin.createTable(blHTD);
+createSystemTable(admin, blHTD);
}
waitForSystemTable(admin, tableName);
waitForSystemTable(admin, bulkLoadTableName);
}
}
+ private void createSystemTable(Admin admin, TableDescriptor descriptor)
throws IOException {
Review Comment:
Please add more comments to say why here we could hit table exists exception?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (HBASE-28081) TestSnapshotScannerHDFSAclController is failing 100% on flaky dashboard
Duo Zhang created HBASE-28081: - Summary: TestSnapshotScannerHDFSAclController is failing 100% on flaky dashboard Key: HBASE-28081 URL: https://issues.apache.org/jira/browse/HBASE-28081 Project: HBase Issue Type: Bug Components: acl, test Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764577#comment-17764577
]
zhangbutao commented on HBASE-28061:
Sure, I have recompiled hbase code with hadoop3.3.1 and there is no problem in
my test case.
But Apache Hive needs the official released Hbase jar, would Hbase community
consider releasing a official Hbase-Hadoop3.3.1 version?
Thanks.
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Priority: Major
> Attachments: HbaseMiniClusterTest.java, full_output.txt, pom.xml,
> root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764574#comment-17764574
]
Duo Zhang commented on HBASE-28061:
---
In JVM implementation, the type of return value is also contained in the method
signature, so changing the type will affect the method signature and cause
NoSuchMethodError.
Notice that this happens when you do drop-in replacement, if you recompile your
code then there is no problem.
The only way is to use reflection here...
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Priority: Major
> Attachments: HbaseMiniClusterTest.java, full_output.txt, pom.xml,
> root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764571#comment-17764571
]
zhangbutao edited comment on HBASE-28061 at 9/13/23 9:50 AM:
-
Test and debug this issue using official released jar. Attach the error
stackTrace *root_cause.png*
{code:java}
2.5.5-hadoop3
3.3.1 {code}
>From the short message:
{code:java}
java.lang.NoSuchMethodError:
org.apache.hadoop.hdfs.protocol.LocatedBlock.getLocations()[Lorg/apache/hadoop/hdfs/protocol/DatanodeInfo;
{code}
It seems that the issue is come from the HDFS method
{*}LocatedBlock.getLocations(){*}, and the return value of this method was
changed from *DatanodeInfo* to *DatanodeInfoWithStorage* by
HDFS-15255(Hadoop3.3.1), but DatanodeInfoWithStorage extends from DatanodeInfo.
That is to say, IMO it should be compatible between Hadoop3.2.4 and Hadoop3.3.1.
I have not figured out why this *NoSuchMethodError* happend in Hbase.
[~zhangduo] Could you please give some suggestion to fix this? Many thanks!
was (Author: zhangbutao):
Test and debug this issue using official released jar. Attach the error
stackTrace *root_cause.png*
{code:java}
2.5.5-hadoop3
3.3.1 {code}
>From the short message:
{code:java}
java.lang.NoSuchMethodError:
org.apache.hadoop.hdfs.protocol.LocatedBlock.getLocations()[Lorg/apache/hadoop/hdfs/protocol/DatanodeInfo;
{code}
It seems that the issue is come from the HDFS method
{*}LocatedBlock.getLocations(){*}, and the return value of this method ** was
changed to *DatanodeInfoWithStorage* by HDFS-15255(Hadoop3.3.1), but
DatanodeInfoWithStorage extends from DatanodeInfo.
That is to say, IMO it should be compatible between Hadoop3.2.4 and Hadoop3.3.1.
I have not figured out why this *NoSuchMethodError* happend in Hbase.
[~zhangduo] Could you please give some suggestion to fix this? Many thanks!
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Priority: Major
> Attachments: HbaseMiniClusterTest.java, full_output.txt, pom.xml,
> root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:
[jira] [Commented] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764571#comment-17764571
]
zhangbutao commented on HBASE-28061:
Test and debug this issue using official released jar. Attach the error
stackTrace *root_cause.png*
{code:java}
2.5.5-hadoop3
3.3.1 {code}
>From the short message:
{code:java}
java.lang.NoSuchMethodError:
org.apache.hadoop.hdfs.protocol.LocatedBlock.getLocations()[Lorg/apache/hadoop/hdfs/protocol/DatanodeInfo;
{code}
It seems that the issue is come from the HDFS method
{*}LocatedBlock.getLocations(){*}, and the return value of this method ** was
changed to *DatanodeInfoWithStorage* by HDFS-15255(Hadoop3.3.1), but
DatanodeInfoWithStorage extends from DatanodeInfo.
That is to say, IMO it should be compatible between Hadoop3.2.4 and Hadoop3.3.1.
I have not figured out why this *NoSuchMethodError* happend in Hbase.
[~zhangduo] Could you please give some suggestion to fix this? Many thanks!
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Priority: Major
> Attachments: HbaseMiniClusterTest.java, full_output.txt, pom.xml,
> root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28061) HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
[
https://issues.apache.org/jira/browse/HBASE-28061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhangbutao updated HBASE-28061:
---
Attachment: root_cause.png
> HBaseTestingUtility failed to start MiniHbaseCluster in case of Hadoop3.3.1
> ---
>
> Key: HBASE-28061
> URL: https://issues.apache.org/jira/browse/HBASE-28061
> Project: HBase
> Issue Type: Bug
> Components: hadoop3, integration tests
>Reporter: zhangbutao
>Priority: Major
> Attachments: HbaseMiniClusterTest.java, full_output.txt, pom.xml,
> root_cause.png
>
>
> Note: My test env is *Hadoop3.3.1* & Hbase2.5.5-hadoop3. And if I switch to
> *Hadoop3.1.4* or \{*}Hadoop3.2.4{*}, the test will be ok. I attached the
> source code and pom file.
>
> I just want to use *HBaseTestingUtility::startMiniHBaseCluster* to launch a
> mini hbase cluster in case of Hadoop.3.3.1, but failed with the following
> exception:
> {code:java}
> master.HMaster (MarkerIgnoringBase.java:error(159)) - Failed to become active
> master
> org.apache.hadoop.hbase.FailedCloseWALAfterInitializedErrorException: Failed
> close after init wal failed.
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:167)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:62)
> at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:300)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:217)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.bootstrap(MasterRegion.java:240)
> at
> org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:398)
> at
> org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:132)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:942)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2405)
> at org.apache.hadoop.hbase.master.HMaster.lambda$null$0(HMaster.java:565)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at org.apache.hadoop.hbase.master.HMaster.lambda$run$1(HMaster.java:562)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.shutdown(AbstractFSWAL.java:1012)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.close(AbstractFSWAL.java:1039)
> at
> org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:165)
> ... 13 more
> Caused by: java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.closeWriter(AsyncFSWAL.java:718)
> at
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.doShutdown(AsyncFSWAL.java:775)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:984)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$2.call(AbstractFSWAL.java:979)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 more {code}
> Does that mean Hbase2.5.5-hadoop3 not be compatible with Hadoop3.3.1? Or only
> the integration testing is not work well with Hadoop3.3.1?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5381: HBASE-28058 Adjust the order of acquiring the takeSnapshot locks to r…esolve the deadlock issue.
Apache-HBase commented on PR #5381: URL: https://github.com/apache/hbase/pull/5381#issuecomment-1717127212 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-3 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 13s | branch-3 passed | | +1 :green_heart: | compile | 0m 35s | branch-3 passed | | +1 :green_heart: | shadedjars | 4m 54s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-3 passed | | -0 :warning: | patch | 5m 22s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 16s | the patch passed | | +1 :green_heart: | compile | 0m 35s | the patch passed | | +1 :green_heart: | javac | 0m 35s | the patch passed | | +1 :green_heart: | shadedjars | 4m 53s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 217m 3s | hbase-server in the patch passed. | | | | 237m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5381/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5381 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e9668eb01f13 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-3 / 05449bd763 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5381/4/testReport/ | | Max. process+thread count | 4536 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5381/4/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5381: HBASE-28058 Adjust the order of acquiring the takeSnapshot locks to r…esolve the deadlock issue.
Apache-HBase commented on PR #5381: URL: https://github.com/apache/hbase/pull/5381#issuecomment-1717123980 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-3 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 44s | branch-3 passed | | +1 :green_heart: | compile | 0m 45s | branch-3 passed | | +1 :green_heart: | shadedjars | 5m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | branch-3 passed | | -0 :warning: | patch | 5m 37s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 44s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 213m 10s | hbase-server in the patch failed. | | | | 235m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5381/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5381 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9db42286e30f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-3 / 05449bd763 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5381/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5381/4/testReport/ | | Max. process+thread count | 4681 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5381/4/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
