[jira] [Commented] (HBASE-24052) Add debug to TestMasterShutdown
[ https://issues.apache.org/jira/browse/HBASE-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067374#comment-17067374 ] Michael Stack commented on HBASE-24052: --- Pushed the attached patch to branch-2 only. Leaving open for now Will probably revert in morning. > Add debug to TestMasterShutdown > --- > > Key: HBASE-24052 > URL: https://issues.apache.org/jira/browse/HBASE-24052 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Priority: Trivial > Attachments: 0001-HBASE-24052-Add-debug-to-TestMasterShutdown.patch > > > Temporarily add debug to TestMasterShutdown overnight to learn more about a > test failure not reproducible locally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24052) Add debug to TestMasterShutdown
[ https://issues.apache.org/jira/browse/HBASE-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-24052: -- Fix Version/s: (was: 2.3.0) > Add debug to TestMasterShutdown > --- > > Key: HBASE-24052 > URL: https://issues.apache.org/jira/browse/HBASE-24052 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Priority: Trivial > Attachments: 0001-HBASE-24052-Add-debug-to-TestMasterShutdown.patch > > > Temporarily add debug to TestMasterShutdown overnight to learn more about a > test failure not reproducible locally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24052) Add debug to TestMasterShutdown
[ https://issues.apache.org/jira/browse/HBASE-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-24052: -- Attachment: 0001-HBASE-24052-Add-debug-to-TestMasterShutdown.patch > Add debug to TestMasterShutdown > --- > > Key: HBASE-24052 > URL: https://issues.apache.org/jira/browse/HBASE-24052 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Priority: Trivial > Fix For: 2.3.0 > > Attachments: 0001-HBASE-24052-Add-debug-to-TestMasterShutdown.patch > > > Temporarily add debug to TestMasterShutdown overnight to learn more about a > test failure not reproducible locally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23808) [Flakey Test] TestMasterShutdown#testMasterShutdownBeforeStartingAnyRegionServer
[
https://issues.apache.org/jira/browse/HBASE-23808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067372#comment-17067372
]
Michael Stack commented on HBASE-23808:
---
I just saw this bit in TestMasterShutdown:
{code}
// Switching to master registry exacerbated a race in the master bootstrap that
can result
// in a lost shutdown command (HBASE-8422, HBASE-23836). The race is
essentially because
// the server manager in HMaster is not initialized by the time
shutdown() RPC (below) is
// made to the master. The suspected reason as to why it was uncommon
before HBASE-18095
// is because the connection creation with ZK registry is so slow that
by then the server
// manager is usually init'ed in time for the RPC to be made. For now,
adding an explicit
// wait() in the test, waiting for the server manager to become
available.
final long timeout = TimeUnit.MINUTES.toMillis(10);
assertNotEquals("timeout waiting for server manager to become
available.",
-1, Waiter.waitFor(htu.getConfiguration(), timeout,
() -> masterThread.getMaster().getServerManager() != null...
{code}
... which probably explains the 'hang' I see.
In RSProcedureDispatcher#start, we were getting NPEs... which correlated to the
test fails. Above, I added catch and returning failed start which seemed to be
because Master had already been stopped. Made a subtask adding more debug for
now while tests run over night.
> [Flakey Test]
> TestMasterShutdown#testMasterShutdownBeforeStartingAnyRegionServer
>
>
> Key: HBASE-23808
> URL: https://issues.apache.org/jira/browse/HBASE-23808
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.4
>
> Attachments:
> TEST-org.apache.hadoop.hbase.master.TestMasterShutdown.xml
>
>
> Reproduces locally from time to time. Not much to go on here. Looks like the
> test is trying to do some fancy HBase cluster initialization order on top of
> a mini-cluster. Failure seems related to trying to start the HBase master
> before HDFS is fully initialized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24052) Add debug to TestMasterShutdown
[ https://issues.apache.org/jira/browse/HBASE-24052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HBASE-24052: -- Fix Version/s: 2.3.0 > Add debug to TestMasterShutdown > --- > > Key: HBASE-24052 > URL: https://issues.apache.org/jira/browse/HBASE-24052 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Priority: Trivial > Fix For: 2.3.0 > > > Temporarily add debug to TestMasterShutdown overnight to learn more about a > test failure not reproducible locally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24052) Add debug to TestMasterShutdown
Michael Stack created HBASE-24052: - Summary: Add debug to TestMasterShutdown Key: HBASE-24052 URL: https://issues.apache.org/jira/browse/HBASE-24052 Project: HBase Issue Type: Sub-task Reporter: Michael Stack Temporarily add debug to TestMasterShutdown overnight to learn more about a test failure not reproducible locally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0 URL: https://github.com/apache/hbase/pull/1335#issuecomment-604231759 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 7m 17s | master passed | | +1 :green_heart: | compile | 2m 2s | master passed | | +1 :green_heart: | shadedjars | 6m 27s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-common in master failed. | | -0 :warning: | javadoc | 0m 16s | hbase-procedure in master failed. | | -0 :warning: | javadoc | 0m 41s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 7m 1s | the patch passed | | +1 :green_heart: | compile | 1m 57s | the patch passed | | +1 :green_heart: | javac | 1m 57s | the patch passed | | +1 :green_heart: | shadedjars | 6m 32s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 16s | hbase-common in the patch failed. | | -0 :warning: | javadoc | 0m 16s | hbase-procedure in the patch failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 25s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 36s | hbase-procedure in the patch passed. | | +1 :green_heart: | unit | 100m 43s | hbase-server in the patch passed. | | | | 140m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1335 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 26e338b4e269 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76e3db6720 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/testReport/ | | Max. process+thread count | 4150 (vs. ulimit of 1) | | modules | C: hbase-common hbase-procedure hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067355#comment-17067355 ] Anoop Sam John commented on HBASE-24051: Within the HBase version that you use, is there unbuffer() calls. Am not sure which jira added this in Hbase to make we call unbuffer() after the HFile open. > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but after the fix, hbase did not return to normal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24032) [RSGroup] Assign created tables to respective rsgroup automatically instead of manual operations
[ https://issues.apache.org/jira/browse/HBASE-24032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067354#comment-17067354 ] Anoop Sam John commented on HBASE-24032: Can u pls put an eg: of the script also in RN > [RSGroup] Assign created tables to respective rsgroup automatically instead > of manual operations > - > > Key: HBASE-24032 > URL: https://issues.apache.org/jira/browse/HBASE-24032 > Project: HBase > Issue Type: Improvement > Components: master, rsgroup >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Major > Fix For: 2.3.0, master, 1.7.0, 1.4.14, 2.2.5 > > > In prod practise, it's time consuming and tiring to assign tables to their > rsgroups for HBase admin by shell, especially in 1.x rsgroup client is > IA.private, we could not even write some programs to do that. > And I'm here proposing a solution to do the automated rsgroup assignment by > allowing running a shell in postCreateTable to determine which table should > go to which rsgroup (way in branch-1 not master). The benefit is obvious, > admin can determine or change at any time the mapping rule in script without > restarting or re-config anything. It's flexible, convenience and time saving. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Description: When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer See HDFS-14308，but after the fix, hbase did not return to normal. was: When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer See HDFS-14308，but > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but after the fix, hbase did not return to normal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Description: When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer See HDFS-14308，but was: When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: External issue ID: (was: HDFS-14308) > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: External issue ID: HDFS-14308 > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Description: When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer was: When the data directory is set to EC, during the continuous load process, regionserver is found down to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Description: When the data directory is set to EC, during the continuous load process, regionserver is found down to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Attachment: 1.png > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1354: HBASE-24050 Deprecated PBType on all 2.x branches
Apache-HBase commented on issue #1354: HBASE-24050 Deprecated PBType on all 2.x branches URL: https://github.com/apache/hbase/pull/1354#issuecomment-604210058 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 45s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 25s | branch-2 passed | | +1 :green_heart: | spotbugs | 0m 47s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 10s | the patch passed | | +1 :green_heart: | checkstyle | 0m 22s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 10m 49s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. | | +1 :green_heart: | spotbugs | 0m 50s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 31m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1354/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1354 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 075f7e493337 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / dcd9a81528 | | Max. process+thread count | 93 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1354/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0 URL: https://github.com/apache/hbase/pull/1335#issuecomment-604210063 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 2s | master passed | | +1 :green_heart: | compile | 1m 43s | master passed | | +1 :green_heart: | shadedjars | 5m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 11s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 55s | the patch passed | | +1 :green_heart: | compile | 1m 39s | the patch passed | | +1 :green_heart: | javac | 1m 39s | the patch passed | | +1 :green_heart: | shadedjars | 5m 27s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 13s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 41s | hbase-procedure in the patch passed. | | -1 :x: | unit | 16m 59s | hbase-server in the patch failed. | | | | 51m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1335 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4b00781154a9 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76e3db6720 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/testReport/ | | Max. process+thread count | 916 (vs. ulimit of 1) | | modules | C: hbase-common hbase-procedure hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Summary: When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in frequent gc and eventual failure (was: When the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in frequent gc and eventual failure) > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24051) When the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in frequent gc and eventual failur
shenshengli created HBASE-24051: --- Summary: When the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in frequent gc and eventual failure Key: HBASE-24051 URL: https://issues.apache.org/jira/browse/HBASE-24051 Project: HBase Issue Type: Bug Affects Versions: 2.0.1 Reporter: shenshengli -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1354: HBASE-24050 Deprecated PBType on all 2.x branches
Apache-HBase commented on issue #1354: HBASE-24050 Deprecated PBType on all 2.x branches URL: https://github.com/apache/hbase/pull/1354#issuecomment-604208938 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 31s | branch-2 passed | | +1 :green_heart: | compile | 0m 26s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 16s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | hbase-common in branch-2 failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 2s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | shadedjars | 5m 20s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | hbase-common in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 17s | hbase-common in the patch passed. | | | | 27m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1354/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1354 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 483d96712f40 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / dcd9a81528 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1354/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1354/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1354/1/testReport/ | | Max. process+thread count | 537 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1354/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1354: HBASE-24050 Deprecated PBType on all 2.x branches
Apache-HBase commented on issue #1354: HBASE-24050 Deprecated PBType on all 2.x branches URL: https://github.com/apache/hbase/pull/1354#issuecomment-604208183 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 44s | branch-2 passed | | +1 :green_heart: | compile | 0m 25s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 16s | the patch passed | | +1 :green_heart: | compile | 0m 22s | the patch passed | | +1 :green_heart: | javac | 0m 22s | the patch passed | | +1 :green_heart: | shadedjars | 4m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 3s | hbase-common in the patch passed. | | | | 24m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1354/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1354 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ae2ab2fc7b72 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / dcd9a81528 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1354/1/testReport/ | | Max. process+thread count | 609 (vs. ulimit of 1) | | modules | C: hbase-common U: hbase-common | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1354/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0 URL: https://github.com/apache/hbase/pull/1335#issuecomment-604207566 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 36s | master passed | | +1 :green_heart: | checkstyle | 1m 47s | master passed | | +1 :green_heart: | spotbugs | 3m 13s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 22s | the patch passed | | -0 :warning: | checkstyle | 0m 22s | hbase-common: The patch generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 0s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. | | +1 :green_heart: | spotbugs | 3m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. | | | | 41m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1335 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 0fb88ac35eea 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 76e3db6720 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | Max. process+thread count | 93 (vs. ulimit of 1) | | modules | C: hbase-common hbase-procedure hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #1292: HBASE-23994:Add WebUI to Canary
Apache9 commented on a change in pull request #1292: HBASE-23994:Add WebUI to
Canary
URL: https://github.com/apache/hbase/pull/1292#discussion_r398291858
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/tool/CanaryTool.java
##
@@ -274,16 +303,50 @@ public void publishReadTiming(String znode, String
server, long msTime) {
private Map perTableReadLatency = new HashMap<>();
private LongAdder writeLatency = new LongAdder();
private final Map> regionMap = new
ConcurrentHashMap<>();
+private Map perServerFailuresCount = new
ConcurrentHashMap<>();
Review comment:
Declare as ConcurrentMap?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #1292: HBASE-23994:Add WebUI to Canary
Apache9 commented on a change in pull request #1292: HBASE-23994:Add WebUI to
Canary
URL: https://github.com/apache/hbase/pull/1292#discussion_r398291625
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/tool/CanaryTool.java
##
@@ -123,6 +127,31 @@
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.TOOLS)
public class CanaryTool implements Tool, Canary {
+
+ private void putUpWebUI() throws IOException {
+if (zookeeperMode) {
+ LOG.info("WebUI is not supported in Zookeeper mode");
+} else if (regionServerMode) {
+ LOG.info("WebUI is not supported in RegionServer mode");
+} else {
+ Configuration conf = new Configuration();
+ int port = conf.getInt(HConstants.HBASE_CANARY_INFO_PORT,
DEFAULT_CANARY_INFOPORT);
+ // -1 is for disabling info server
+ if (port < 0) return;
+ String addr = conf.get(HBASE_CANARY_INFO_BINDADDRESS, "0.0.0.0");
+ try {
+InfoServer infoServer = new InfoServer("canary", addr, port, false,
conf);
+infoServer.addUnprivilegedServlet("canary", "/canary-status",
CanaryStatusServlet.class);
+infoServer.setAttribute("sink", this.sink);
+infoServer.start();
+LOG.info("Bind Canary http info server to port: " + port);
+ } catch (BindException e) {
+e.printStackTrace();
Review comment:
Avoid using e.printStackTrace directly?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #1292: HBASE-23994:Add WebUI to Canary
Apache9 commented on a change in pull request #1292: HBASE-23994:Add WebUI to Canary URL: https://github.com/apache/hbase/pull/1292#discussion_r398292304 ## File path: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java ## @@ -1464,6 +1464,12 @@ public static final String HBASE_CANARY_READ_RAW_SCAN_KEY = "hbase.canary.read.raw.enabled"; + public static final String HBASE_CANARY_INFO_PORT = "hbase.canary.info.port"; Review comment: Is it possible to not add these configurations in HConstants? Just put them in the CanaryTool class? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 opened a new pull request #1354: HBASE-24050 Deprecated PBType on all 2.x branches
Apache9 opened a new pull request #1354: HBASE-24050 Deprecated PBType on all 2.x branches URL: https://github.com/apache/hbase/pull/1354 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-24050) Deprecated PBType on all 2.x branches
[ https://issues.apache.org/jira/browse/HBASE-24050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-24050: -- Release Note: org.apache.hadoop.hbase.types.PBType is marked as deprecated without any replacement. It will be moved to hbase-example module and marked as IA.Private in 3.0.0. This is a mistake as it should not be part of our public API. Users who depend on this class should just copy the code your own code base. > Deprecated PBType on all 2.x branches > - > > Key: HBASE-24050 > URL: https://issues.apache.org/jira/browse/HBASE-24050 > Project: HBase > Issue Type: Sub-task > Components: Protobufs >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.3.0, 2.1.10, 2.2.5 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23981) PBType which is marked as IA.Public references protobuf Message
[ https://issues.apache.org/jira/browse/HBASE-23981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23981: -- Release Note: Move org.apache.hadoop.hbase.types.PBType to hbase-examples and mark it as IA.Private, without replacement. It should not be our public API. Users who still wants to use it should just copy the code into your own code base. (was: Move PBType to hbase-examples and mark it as IA.Private, without replacement. It should not be our public API. Users who still wants to use it should just copy the code into your own code base.) > PBType which is marked as IA.Public references protobuf Message > --- > > Key: HBASE-23981 > URL: https://issues.apache.org/jira/browse/HBASE-23981 > Project: HBase > Issue Type: Task > Components: Protobufs >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 3.0.0 > > > This is a very critical problem as we leak protobuf in our user facing public > API? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24050) Deprecated PBType on all 2.x branches
Duo Zhang created HBASE-24050: - Summary: Deprecated PBType on all 2.x branches Key: HBASE-24050 URL: https://issues.apache.org/jira/browse/HBASE-24050 Project: HBase Issue Type: Sub-task Components: Protobufs Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 2.3.0, 2.1.10, 2.2.5 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23981) PBType which is marked as IA.Public references protobuf Message
[ https://issues.apache.org/jira/browse/HBASE-23981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-23981. --- Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change) Resolution: Fixed Merged to master. Thanks [~ndimiduk] for reviewing. > PBType which is marked as IA.Public references protobuf Message > --- > > Key: HBASE-23981 > URL: https://issues.apache.org/jira/browse/HBASE-23981 > Project: HBase > Issue Type: Task > Components: Protobufs >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 3.0.0 > > > This is a very critical problem as we leak protobuf in our user facing public > API? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 merged pull request #1323: HBASE-23981 PBType which is marked as IA.Public references protobuf M…
Apache9 merged pull request #1323: HBASE-23981 PBType which is marked as IA.Public references protobuf M… URL: https://github.com/apache/hbase/pull/1323 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
Apache9 commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0 URL: https://github.com/apache/hbase/pull/1335#issuecomment-604193880 The test is strange, we wrap a already created procedure store with mockito and then call the recoverLease method again. Not sure why but after calling the real method of the procStore, the current stream is still fine but after we returned from the method of the mocked instance, the current stream has already been closed and cause the test to fail. Since the class is only used for data migration and will be removed in 4.0.0, do not want to spend much time on it. Just catch the FSError exception when closing... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-24032) [RSGroup] Assign created tables to respective rsgroup automatically instead of manual operations
[ https://issues.apache.org/jira/browse/HBASE-24032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-24032: -- Release Note: Admin can determine which tables go to which rsgroup by script on Master side which aims to lighten the burden of admin operations. Note, since HBase 3+, rsgroup can be specified in TableDescriptor as well, if clients specify this, master will skip the determination from script. > [RSGroup] Assign created tables to respective rsgroup automatically instead > of manual operations > - > > Key: HBASE-24032 > URL: https://issues.apache.org/jira/browse/HBASE-24032 > Project: HBase > Issue Type: Improvement > Components: master, rsgroup >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Major > Fix For: 2.3.0, master, 1.7.0, 1.4.14, 2.2.5 > > > In prod practise, it's time consuming and tiring to assign tables to their > rsgroups for HBase admin by shell, especially in 1.x rsgroup client is > IA.private, we could not even write some programs to do that. > And I'm here proposing a solution to do the automated rsgroup assignment by > allowing running a shell in postCreateTable to determine which table should > go to which rsgroup (way in branch-1 not master). The benefit is obvious, > admin can determine or change at any time the mapping rule in script without > restarting or re-config anything. It's flexible, convenience and time saving. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604192120 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 20s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 52s | master passed | | +1 :green_heart: | compile | 3m 33s | master passed | | +1 :green_heart: | shadedjars | 5m 30s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 54s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 16s | the patch passed | | +1 :green_heart: | compile | 3m 52s | the patch passed | | +1 :green_heart: | javac | 3m 52s | the patch passed | | +1 :green_heart: | shadedjars | 5m 43s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 3s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 150m 43s | root in the patch passed. | | | | 193m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1352 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0096a7abb8b7 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Default Java | 1.8.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/testReport/ | | Max. process+thread count | 5921 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-8868) add metric to report client shortcircuit reads
[ https://issues.apache.org/jira/browse/HBASE-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067315#comment-17067315 ] Reid Chan commented on HBASE-8868: -- [~weichiu] is it possible to be back-ported to branch-1 as well? > add metric to report client shortcircuit reads > -- > > Key: HBASE-8868 > URL: https://issues.apache.org/jira/browse/HBASE-8868 > Project: HBase > Issue Type: Improvement > Components: metrics, regionserver >Affects Versions: 0.94.8, 0.95.1 >Reporter: Viral Bajaria >Assignee: Wei-Chiu Chuang >Priority: Minor > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-8868.master.001.patch > > > With the availability of shortcircuit reads, when the feature is enabled > there is no metric which exposes how many times the regionserver was able to > shortcircuit the read and not make a IPC to the datanode. > It will be great to add the metric and expose it via Ganglia. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] WenFeiYi commented on issue #1338: HBASE-24040 WALFactory.Providers.multiwal causes StackOverflowError
WenFeiYi commented on issue #1338: HBASE-24040 WALFactory.Providers.multiwal causes StackOverflowError URL: https://github.com/apache/hbase/pull/1338#issuecomment-604185407 I refer to the processing method of other configurations. If the configuration is invalid, the default value is returned. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24047) [Flakey Tests] Disable TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
[
https://issues.apache.org/jira/browse/HBASE-24047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067293#comment-17067293
]
Josh Elser commented on HBASE-24047:
Thanks Stack. I'll investigate after I get that other one wrapped up. A couple
of follow on things for it, as well.
> [Flakey Tests] Disable
> TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
>
>
> Key: HBASE-24047
> URL: https://issues.apache.org/jira/browse/HBASE-24047
> Project: HBase
> Issue Type: Task
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.0
>
> Attachments:
> 0001-HBASE-24047-Flakey-Tests-Disable-TestCustomSaslAuthe.patch
>
>
> Fails sporadically with relogin in progress complaints. See
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html
> FYI [~elserj]
> i.e.
> org.apache.hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider.testNegativeAuthentication[2:
> rpcClientImpl=org.apache.hadoop.hbase.ipc.NettyRpcClient,
> rpcServerImpl=org.apache.hadoop.hbase.ipc.SimpleRpcServer]
> {code}
> Failed after attempts=4, exceptions:
> 2020-03-25T21:25:05.239Z, RpcRetryingCaller{globalStartTime=1585171502143,
> pause=100, maxAttempts=4},
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> 2020-03-25T21:25:04.624Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.726Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.929Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:05.235Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> {code}
> Disabling rather than spending time in here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on issue #1338: HBASE-24040 WALFactory.Providers.multiwal causes StackOverflowError
Apache9 commented on issue #1338: HBASE-24040 WALFactory.Providers.multiwal causes StackOverflowError URL: https://github.com/apache/hbase/pull/1338#issuecomment-604179753 In general, I think this is a misusage of the configuration. So my concern is that, should we fall back to default or just abort with a clear message that you should not set the delegation to multiwal? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604179388 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 26s | master passed | | +1 :green_heart: | compile | 3m 52s | master passed | | +1 :green_heart: | shadedjars | 5m 50s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 20s | root in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 11s | the patch passed | | +1 :green_heart: | compile | 3m 55s | the patch passed | | +1 :green_heart: | javac | 3m 55s | the patch passed | | +1 :green_heart: | shadedjars | 5m 48s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | root in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 111m 23s | root in the patch passed. | | | | 147m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1352 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7d590604a1a8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/testReport/ | | Max. process+thread count | 9148 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23808) [Flakey Test] TestMasterShutdown#testMasterShutdownBeforeStartingAnyRegionServer
[
https://issues.apache.org/jira/browse/HBASE-23808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067262#comment-17067262
]
Michael Stack commented on HBASE-23808:
---
The nature of this was changed by me here:
commit 50161f2de4a4efb48e250b07dd609175f82a7331
Author: stack
Date: Mon Mar 23 11:45:46 2020 -0700
HBASE-24034 [Flakey Tests] A couple of fixes and cleanups
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
index 2f4bea8799..08043ef4e2 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
@@ -1541,7 +1541,9 @@ public class HMaster extends HRegionServer implements
MasterServices {
// startProcedureExecutor. See the javadoc for
finishActiveMasterInitialization for more
// details.
procedureExecutor.init(numThreads, abortOnCorruption);
-procEnv.getRemoteDispatcher().start();
+if (!procEnv.getRemoteDispatcher().start()) {
+ throw new HBaseIOException("Failed start of remote dispatcher");
+}
}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
index b469cb86e6..10b823ca67 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
@@ -29,6 +29,7 @@ import org.apache.hadoop.hbase.client.RegionInfo;
import org.apache.hadoop.hbase.ipc.ServerNotRunningYetException;
import org.apache.hadoop.hbase.master.MasterServices;
import org.apache.hadoop.hbase.master.ServerListener;
+import org.apache.hadoop.hbase.master.ServerManager;
import org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher;
import org.apache.hadoop.hbase.regionserver.RegionServerAbortedException;
import org.apache.hadoop.hbase.regionserver.RegionServerStoppedException;
@@ -93,11 +94,16 @@ public class RSProcedureDispatcher
if (!super.start()) {
return false;
}
-
-master.getServerManager().registerListener(this);
-procedureEnv = master.getMasterProcedureExecutor().getEnvironment();
-for (ServerName serverName:
master.getServerManager().getOnlineServersList()) {
- addNode(serverName);
+// Around startup, if failed, some of the below may be set back to null so
NPE is possible.
+try {
+ master.getServerManager().registerListener(this);
+ procedureEnv = master.getMasterProcedureExecutor().getEnvironment();
+ for (ServerName serverName :
master.getServerManager().getOnlineServersList()) {
+addNode(serverName);
+ }
+} catch (Exception e) {
+ LOG.info("Failed start", e);
+ return false;
}
return true;
}
So, now we get an HBASEIOE out of the run loop which causes us to exit... but
in this case we've set 'stop' on master so we don't 'abort' the Master... we'll
try to run a nice shutdown. We are hungup though where , I'm not sure as yet.
Meantime the Admin shutdown command is hung... hasn't completed as though we
are not sending out an answer.
I can't make this fail locally. Been at it a while now.
Will keep at it. Probably good to push more debug.
Thanks for taking a look [~bharathv]. Always good to get second opinion.
> [Flakey Test]
> TestMasterShutdown#testMasterShutdownBeforeStartingAnyRegionServer
>
>
> Key: HBASE-23808
> URL: https://issues.apache.org/jira/browse/HBASE-23808
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.4
>
> Attachments:
> TEST-org.apache.hadoop.hbase.master.TestMasterShutdown.xml
>
>
> Reproduces locally from time to time. Not much to go on here. Looks like the
> test is trying to do some fancy HBase cluster initialization order on top of
> a mini-cluster. Failure seems related to trying to start the HBase master
> before HDFS is fully initialized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1353: HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh ADDENDUM
Apache-HBase commented on issue #1353: HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh ADDENDUM URL: https://github.com/apache/hbase/pull/1353#issuecomment-604159238 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1353/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1353 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux b54f8e593ead 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Max. process+thread count | 49 (vs. ulimit of 1) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1353/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1353: HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh ADDENDUM
Apache-HBase commented on issue #1353: HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh ADDENDUM URL: https://github.com/apache/hbase/pull/1353#issuecomment-604159188 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1353/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1353 | | Optional Tests | | | uname | Linux 8d405de07db2 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Max. process+thread count | 52 (vs. ulimit of 1) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1353/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1353: HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh ADDENDUM
Apache-HBase commented on issue #1353: HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh ADDENDUM URL: https://github.com/apache/hbase/pull/1353#issuecomment-604159156 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1353/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1353 | | Optional Tests | | | uname | Linux 5f6aba89e385 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Max. process+thread count | 56 (vs. ulimit of 1) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1353/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk opened a new pull request #1353: HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh ADDENDUM
ndimiduk opened a new pull request #1353: HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh ADDENDUM URL: https://github.com/apache/hbase/pull/1353 * Correctly handle the hadoop2 profile. * Create the output directory if it doesn't exist This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23808) [Flakey Test] TestMasterShutdown#testMasterShutdownBeforeStartingAnyRegionServer
[ https://issues.apache.org/jira/browse/HBASE-23808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067249#comment-17067249 ] Nick Dimiduk commented on HBASE-23808: -- Looks like this went out in 2.2.4 so a guess a new JIRA is needed. Yes, I can repro it locally, though not every time. the adhoc test runner should get it though. > [Flakey Test] > TestMasterShutdown#testMasterShutdownBeforeStartingAnyRegionServer > > > Key: HBASE-23808 > URL: https://issues.apache.org/jira/browse/HBASE-23808 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 2.3.0 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.4 > > Attachments: > TEST-org.apache.hadoop.hbase.master.TestMasterShutdown.xml > > > Reproduces locally from time to time. Not much to go on here. Looks like the > test is trying to do some fancy HBase cluster initialization order on top of > a mini-cluster. Failure seems related to trying to start the HBase master > before HDFS is fully initialized. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24043) [Flakey Tests] TestAsyncRegionAdminApi, TestRegionMergeTransactionOnCluster fixes and debug
[
https://issues.apache.org/jira/browse/HBASE-24043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067243#comment-17067243
]
Hudson commented on HBASE-24043:
Results for branch branch-2
[build #2566 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> [Flakey Tests] TestAsyncRegionAdminApi, TestRegionMergeTransactionOnCluster
> fixes and debug
> ---
>
> Key: HBASE-24043
> URL: https://issues.apache.org/jira/browse/HBASE-24043
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.5
>
>
> Add some fixes, some hopeful changes that will hopeful result in fixes, and
> then a bit of debug:
> {code}
>
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java
> Edit of log about archiving that shows in middle of a table create;
> try to make it less disorientating.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
> Loosen assert. Compaction may have produced a single file only. Allow
> for this.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableGetMultiThreaded.java
> Make this test less furious given it is inline w/ a bunch of unit
> tests.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
> Add debug
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestQuotaObserverChoreRegionReports.java
> Add wait on quota table to show up before moving forward; otherwise,
> attempt at quota setting fails.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
> Debug
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
> Remove asserts that expected regions to still have a presence in fs
> after merge when a catalogjanitor may have cleaned up parent dirs.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
> Catch exception on way out and log it rather than let it fail test.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController.java
> Wait on acl table before proceeding.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23949) refactor loadBalancer implements for rsgroup balance by table to achieve overallbalanced
[
https://issues.apache.org/jira/browse/HBASE-23949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067246#comment-17067246
]
Hudson commented on HBASE-23949:
Results for branch branch-2
[build #2566 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> refactor loadBalancer implements for rsgroup balance by table to achieve
> overallbalanced
> --
>
> Key: HBASE-23949
> URL: https://issues.apache.org/jira/browse/HBASE-23949
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.2.0
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.4.0, 2.2.5
>
>
> now can not achieve overallbalanced when use rsgroup balancer and by table
> is on,
> because balance every table actually use the clusterload only contain one
> table's load.
> we should use clusterload contain all this rsgroup table's load to balance
> overall
>
> hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> {code:java}
> public boolean balance(boolean force) throws IOException {
> ..
> boolean isByTable =
> getConfiguration().getBoolean("hbase.master.loadbalance.bytable", false);
> Map>> assignments =
> this.assignmentManager.getRegionStates()
> .getAssignmentsForBalancer(tableStateManager,
> this.serverManager.getOnlineServersList(),
> isByTable);
> for (Map> serverMap : assignments.values()) {
>
> serverMap.keySet().removeAll(this.serverManager.getDrainingServersList());
> }
> //Give the balancer the current cluster state.
> this.balancer.setClusterMetrics(getClusterMetricsWithoutCoprocessor());
> this.balancer.setClusterLoad(assignments);
> List plans = new ArrayList<>();
> for (Entry>> e :
> assignments.entrySet()) {
> List partialPlans =
> this.balancer.balanceCluster(e.getKey(), e.getValue());
> if (partialPlans != null) {
> plans.addAll(partialPlans);
> }
> }
> {code}
> now do refactor:
> # add method 'balanceTable' in interface LoadBalancer
> # SimpleLoadBalancer and StochasticLoadBalancer do the real 'balanceTable' ,
> and 'balanceTable' is not support in BaseLoadBalancer and
> RSGroupBasedLoadBalancer
> # RSGroupBasedLoadBalancer invoke balanceCluster , and pass GroupClusterLoad
> to internal balacer by group
> # internal balancer balance cluster invoke 'balanceTable'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-8868) add metric to report client shortcircuit reads
[
https://issues.apache.org/jira/browse/HBASE-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067244#comment-17067244
]
Hudson commented on HBASE-8868:
---
Results for branch branch-2
[build #2566 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> add metric to report client shortcircuit reads
> --
>
> Key: HBASE-8868
> URL: https://issues.apache.org/jira/browse/HBASE-8868
> Project: HBase
> Issue Type: Improvement
> Components: metrics, regionserver
>Affects Versions: 0.94.8, 0.95.1
>Reporter: Viral Bajaria
>Assignee: Wei-Chiu Chuang
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-8868.master.001.patch
>
>
> With the availability of shortcircuit reads, when the feature is enabled
> there is no metric which exposes how many times the regionserver was able to
> shortcircuit the read and not make a IPC to the datanode.
> It will be great to add the metric and expose it via Ganglia.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24037) Add ut for root dir and wal root dir are different
[
https://issues.apache.org/jira/browse/HBASE-24037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067245#comment-17067245
]
Hudson commented on HBASE-24037:
Results for branch branch-2
[build #2566 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//JDK11_Nightly_Build_Report/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2566//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Add ut for root dir and wal root dir are different
> --
>
> Key: HBASE-24037
> URL: https://issues.apache.org/jira/browse/HBASE-24037
> Project: HBase
> Issue Type: Sub-task
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.4.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23949) refactor loadBalancer implements for rsgroup balance by table to achieve overallbalanced
[
https://issues.apache.org/jira/browse/HBASE-23949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067239#comment-17067239
]
Hudson commented on HBASE-23949:
Results for branch branch-2.2
[build #825 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> refactor loadBalancer implements for rsgroup balance by table to achieve
> overallbalanced
> --
>
> Key: HBASE-23949
> URL: https://issues.apache.org/jira/browse/HBASE-23949
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.2.0
>Reporter: niuyulin
>Assignee: niuyulin
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.4.0, 2.2.5
>
>
> now can not achieve overallbalanced when use rsgroup balancer and by table
> is on,
> because balance every table actually use the clusterload only contain one
> table's load.
> we should use clusterload contain all this rsgroup table's load to balance
> overall
>
> hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> {code:java}
> public boolean balance(boolean force) throws IOException {
> ..
> boolean isByTable =
> getConfiguration().getBoolean("hbase.master.loadbalance.bytable", false);
> Map>> assignments =
> this.assignmentManager.getRegionStates()
> .getAssignmentsForBalancer(tableStateManager,
> this.serverManager.getOnlineServersList(),
> isByTable);
> for (Map> serverMap : assignments.values()) {
>
> serverMap.keySet().removeAll(this.serverManager.getDrainingServersList());
> }
> //Give the balancer the current cluster state.
> this.balancer.setClusterMetrics(getClusterMetricsWithoutCoprocessor());
> this.balancer.setClusterLoad(assignments);
> List plans = new ArrayList<>();
> for (Entry>> e :
> assignments.entrySet()) {
> List partialPlans =
> this.balancer.balanceCluster(e.getKey(), e.getValue());
> if (partialPlans != null) {
> plans.addAll(partialPlans);
> }
> }
> {code}
> now do refactor:
> # add method 'balanceTable' in interface LoadBalancer
> # SimpleLoadBalancer and StochasticLoadBalancer do the real 'balanceTable' ,
> and 'balanceTable' is not support in BaseLoadBalancer and
> RSGroupBasedLoadBalancer
> # RSGroupBasedLoadBalancer invoke balanceCluster , and pass GroupClusterLoad
> to internal balacer by group
> # internal balancer balance cluster invoke 'balanceTable'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24043) [Flakey Tests] TestAsyncRegionAdminApi, TestRegionMergeTransactionOnCluster fixes and debug
[
https://issues.apache.org/jira/browse/HBASE-24043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067238#comment-17067238
]
Hudson commented on HBASE-24043:
Results for branch branch-2.2
[build #825 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Flakey Tests] TestAsyncRegionAdminApi, TestRegionMergeTransactionOnCluster
> fixes and debug
> ---
>
> Key: HBASE-24043
> URL: https://issues.apache.org/jira/browse/HBASE-24043
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.5
>
>
> Add some fixes, some hopeful changes that will hopeful result in fixes, and
> then a bit of debug:
> {code}
>
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java
> Edit of log about archiving that shows in middle of a table create;
> try to make it less disorientating.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
> Loosen assert. Compaction may have produced a single file only. Allow
> for this.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableGetMultiThreaded.java
> Make this test less furious given it is inline w/ a bunch of unit
> tests.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
> Add debug
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestQuotaObserverChoreRegionReports.java
> Add wait on quota table to show up before moving forward; otherwise,
> attempt at quota setting fails.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
> Debug
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
> Remove asserts that expected regions to still have a presence in fs
> after merge when a catalogjanitor may have cleaned up parent dirs.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
> Catch exception on way out and log it rather than let it fail test.
>
> hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController.java
> Wait on acl table before proceeding.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23957) [flakey test] client.TestMultiParallel fails to read hbase-site.xml
[
https://issues.apache.org/jira/browse/HBASE-23957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067240#comment-17067240
]
Hudson commented on HBASE-23957:
Results for branch branch-2.2
[build #825 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/825//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [flakey test] client.TestMultiParallel fails to read hbase-site.xml
> ---
>
> Key: HBASE-23957
> URL: https://issues.apache.org/jira/browse/HBASE-23957
> Project: HBase
> Issue Type: Test
> Components: test
>Affects Versions: 2.3.0, master, 1.4.13, 2.2.4
>Reporter: Nick Dimiduk
>Priority: Minor
>
> Saw this on a [PreCommit
> run|https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1258/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt].
> Log file says
> {noformat}
> 2020-03-09 22:48:26,295 FATAL [Time-limited test] conf.Configuration(2853):
> error parsing conf hbase-site.xml
> java.io.FileNotFoundException:
> /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1258@2/yetus-jdk8-hadoop2-check/src/hbase-server/target/test-classes/hbase-site.xml
> (No such file or directory)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at
> sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
> at
> sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
> at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2672)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2746)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2706)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2579)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1091)
> at
> org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1145)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2363)
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2793)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:181)
> at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:85)
> at org.apache.hadoop.hbase.fs.HFileSystem.get(HFileSystem.java:465)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getTestFileSystem(HBaseTestingUtility.java:3180)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getNewDataTestDirOnTestFS(HBaseTestingUtility.java:507)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.setupDataTestDirOnTestFS(HBaseTestingUtility.java:496)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getDataTestDirOnTestFS(HBaseTestingUtility.java:469)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getDataTestDirOnTestFS(HBaseTestingUtility.java:483)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.createDirsAndSetProperties(HBaseTestingUtility.java:651)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:603)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:586)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1039)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1004)
> at
>
[jira] [Commented] (HBASE-23853) [Flakey Test] TestBlockEvictionFromClient#testBlockRefCountAfterSplits
[ https://issues.apache.org/jira/browse/HBASE-23853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067234#comment-17067234 ] Huaxiang Sun commented on HBASE-23853: -- Have a rough idea about the root cause, will try to put up a patch and root cause analysis later, assign it to me. > [Flakey Test] TestBlockEvictionFromClient#testBlockRefCountAfterSplits > -- > > Key: HBASE-23853 > URL: https://issues.apache.org/jira/browse/HBASE-23853 > Project: HBase > Issue Type: Bug > Components: flakies >Reporter: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: > 0001-HBASE-23853-Flakey-Test-TestBlockEvictionFromClient-.patch, Screen Shot > 2020-02-15 at 2.36.47 PM.png > > > Fails 12-25% of the time in the GCE build. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23853) [Flakey Test] TestBlockEvictionFromClient#testBlockRefCountAfterSplits
[ https://issues.apache.org/jira/browse/HBASE-23853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun reassigned HBASE-23853: Assignee: Hua Xiang > [Flakey Test] TestBlockEvictionFromClient#testBlockRefCountAfterSplits > -- > > Key: HBASE-23853 > URL: https://issues.apache.org/jira/browse/HBASE-23853 > Project: HBase > Issue Type: Bug > Components: flakies >Reporter: Michael Stack >Assignee: Hua Xiang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: > 0001-HBASE-23853-Flakey-Test-TestBlockEvictionFromClient-.patch, Screen Shot > 2020-02-15 at 2.36.47 PM.png > > > Fails 12-25% of the time in the GCE build. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24045) Support setting `-Dhadoop.profile` in adhoc_run_tests.sh
[ https://issues.apache.org/jira/browse/HBASE-24045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reopened HBASE-24045: -- Broke it for JDK8/Hadoop2... > Support setting `-Dhadoop.profile` in adhoc_run_tests.sh > > > Key: HBASE-24045 > URL: https://issues.apache.org/jira/browse/HBASE-24045 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0, 2.3.0 > > > Allow the caller to override the hadoop version used. This is required in > order to use this tool with JDK11. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604144487 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 12m 0s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 19m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1352 | | Optional Tests | dupname asflicense shellcheck shelldocs hadoopcheck xml | | uname | Linux 128b9d25c03e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Max. process+thread count | 63 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-24049) "Packaging and Integration" check fails
Nick Dimiduk created HBASE-24049:
Summary: "Packaging and Integration" check fails
Key: HBASE-24049
URL: https://issues.apache.org/jira/browse/HBASE-24049
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 3.0.0, 2.3.0
Reporter: Nick Dimiduk
Looks like this is running with Hadoop 2.8.5, despite HBASE-23986
{noformat}
[2020-03-24T09:14:00.930Z] ++ ls -1
/home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/hadoop-2.8.5-bin.tar.gz
[2020-03-24T09:14:00.930Z] ++ head -n 1
[2020-03-24T09:14:00.930Z] +
artifact=/home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/hadoop-2.8.5-bin.tar.gz
[2020-03-24T09:14:00.930Z] + tar --strip-components=1 -xzf
/home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/hadoop-2.8.5-bin.tar.gz
-C hadoop-2
[2020-03-24T09:14:06.186Z] +
/home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/component/dev-support/hbase_nightly_pseudo-distributed-test.sh
--single-process --working-dir output-integration/hadoop-2
--hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop
hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.8.5-tests.jar
hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.5-tests.jar
hadoop-2/bin/mapred
[2020-03-24T17:50:28.859Z] Sending interrupt signal to process
[2020-03-24T17:50:30.005Z] Sending interrupt signal to process
[2020-03-24T17:50:42.558Z] + echo '(x) {color:red}-1 client integration
test{color}
[2020-03-24T17:50:42.558Z] --Failed when running client tests on top of Hadoop
2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/master/1677//artifact/output-integration/hadoop-2.log].
(note that this means we didn'\''t run on Hadoop 3)'
[2020-03-24T17:50:42.558Z] + exit 2
[2020-03-24T17:50:42.866Z] script returned exit code 2
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604137823 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604137796 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604137830 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/2/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24044) Rubocop configuration needs updated
[
https://issues.apache.org/jira/browse/HBASE-24044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067213#comment-17067213
]
Nick Dimiduk commented on HBASE-24044:
--
rubocop.yaml and Dockerfiles are identical on master, branch-2. Why do we see
it on branch-2 but not master?
branch-1 has it's own dockerfile with its own rubocop install path, and looks
like it's working there.
> Rubocop configuration needs updated
> ---
>
> Key: HBASE-24044
> URL: https://issues.apache.org/jira/browse/HBASE-24044
> Project: HBase
> Issue Type: Test
> Components: build
>Affects Versions: 2.4.0
>Reporter: Nick Dimiduk
>Priority: Major
>
> From
> https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2565/consoleFull
> {noformat}
> 15:30:40 Running rubocop against identified ruby scripts.
> 15:30:40 The following cops were added to RuboCop, but are not configured.
> Please set Enabled to either `true` or `false` in your `.rubocop.yml` file:
> 15:30:40 - Style/HashEachMethods
> 15:30:40 - Style/HashTransformKeys
> 15:30:40 - Style/HashTransformValues
> {noformat}
> Is out rubocop versions somehow not pinned? HBASE-23943 was just fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on issue #1352: spotbugs automatically skip inapplicable modules
ndimiduk commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604135254 Supposed to. If i can just get the SpotBugs plugin to run. Over in our Nightly General reports we've been seeing the below. I went to see what that was about and found this juicy comment. ``` 0 | spotbugs | 38m 26s | Both FindBugs and SpotBugs are enabled, using SpotBugs. -- | -- | -- | -- 0 | spotbugs | 0m 10s | branch/hbase-annotations no spotbugs output file (spotbugsXml.xml) 0 | spotbugs | 0m 28s | branch/hbase-assembly no spotbugs output file (spotbugsXml.xml) 0 | spotbugs | 0m 12s | branch/hbase-archetypes/hbase-archetype-builder no spotbugs output file (spotbugsXml.xml) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11
ndimiduk commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11 URL: https://github.com/apache/hbase/pull/1351#issuecomment-604133615 `TestVerifyBucketCacheFile` strikes again :'( This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24047) [Flakey Tests] Disable TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
[
https://issues.apache.org/jira/browse/HBASE-24047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067210#comment-17067210
]
Michael Stack commented on HBASE-24047:
---
[~bharathv] would be cool if related. Can retry once it goes in...
> [Flakey Tests] Disable
> TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
>
>
> Key: HBASE-24047
> URL: https://issues.apache.org/jira/browse/HBASE-24047
> Project: HBase
> Issue Type: Task
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.0
>
> Attachments:
> 0001-HBASE-24047-Flakey-Tests-Disable-TestCustomSaslAuthe.patch
>
>
> Fails sporadically with relogin in progress complaints. See
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html
> FYI [~elserj]
> i.e.
> org.apache.hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider.testNegativeAuthentication[2:
> rpcClientImpl=org.apache.hadoop.hbase.ipc.NettyRpcClient,
> rpcServerImpl=org.apache.hadoop.hbase.ipc.SimpleRpcServer]
> {code}
> Failed after attempts=4, exceptions:
> 2020-03-25T21:25:05.239Z, RpcRetryingCaller{globalStartTime=1585171502143,
> pause=100, maxAttempts=4},
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> 2020-03-25T21:25:04.624Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.726Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.929Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:05.235Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> {code}
> Disabling rather than spending time in here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604131195 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 46s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 4m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1352 | | Optional Tests | | | uname | Linux c72039abf887 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Max. process+thread count | 47 (vs. ulimit of 1) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24047) [Flakey Tests] Disable TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
[
https://issues.apache.org/jira/browse/HBASE-24047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067207#comment-17067207
]
Bharath Vissapragada commented on HBASE-24047:
--
Nvm, ignore my last comment, taking deeper look, this is a different issue.
> [Flakey Tests] Disable
> TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
>
>
> Key: HBASE-24047
> URL: https://issues.apache.org/jira/browse/HBASE-24047
> Project: HBase
> Issue Type: Task
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.0
>
> Attachments:
> 0001-HBASE-24047-Flakey-Tests-Disable-TestCustomSaslAuthe.patch
>
>
> Fails sporadically with relogin in progress complaints. See
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html
> FYI [~elserj]
> i.e.
> org.apache.hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider.testNegativeAuthentication[2:
> rpcClientImpl=org.apache.hadoop.hbase.ipc.NettyRpcClient,
> rpcServerImpl=org.apache.hadoop.hbase.ipc.SimpleRpcServer]
> {code}
> Failed after attempts=4, exceptions:
> 2020-03-25T21:25:05.239Z, RpcRetryingCaller{globalStartTime=1585171502143,
> pause=100, maxAttempts=4},
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> 2020-03-25T21:25:04.624Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.726Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.929Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:05.235Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> {code}
> Disabling rather than spending time in here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on issue #1352: spotbugs automatically skip inapplicable modules
saintstack commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604131006 Oh, does it come up w/ same answer? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604130351 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1352 | | Optional Tests | | | uname | Linux e5c5d8355c1b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Max. process+thread count | 59 (vs. ulimit of 1) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604130348 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 2s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1352 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux 1e5e3d982527 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Max. process+thread count | 47 (vs. ulimit of 1) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) shellcheck=0.4.6 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604129922 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604129843 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules
Apache-HBase commented on issue #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352#issuecomment-604129818 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1352/1/console in case of problems. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24044) Rubocop configuration needs updated
[
https://issues.apache.org/jira/browse/HBASE-24044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067204#comment-17067204
]
Jan Hentschel commented on HBASE-24044:
---
It's the same Rubocop version. Haven't seen this messages on master yet.
Probably would be good to go against master first and then push down the
changes. Does this problem also exist in branch-1?
> Rubocop configuration needs updated
> ---
>
> Key: HBASE-24044
> URL: https://issues.apache.org/jira/browse/HBASE-24044
> Project: HBase
> Issue Type: Test
> Components: build
>Affects Versions: 2.4.0
>Reporter: Nick Dimiduk
>Priority: Major
>
> From
> https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2565/consoleFull
> {noformat}
> 15:30:40 Running rubocop against identified ruby scripts.
> 15:30:40 The following cops were added to RuboCop, but are not configured.
> Please set Enabled to either `true` or `false` in your `.rubocop.yml` file:
> 15:30:40 - Style/HashEachMethods
> 15:30:40 - Style/HashTransformKeys
> 15:30:40 - Style/HashTransformValues
> {noformat}
> Is out rubocop versions somehow not pinned? HBASE-23943 was just fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24047) [Flakey Tests] Disable TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
[
https://issues.apache.org/jira/browse/HBASE-24047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067201#comment-17067201
]
Bharath Vissapragada commented on HBASE-24047:
--
The patch for this has been +1ed, ready to be merged.
https://github.com/apache/hbase/pull/1260
> [Flakey Tests] Disable
> TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
>
>
> Key: HBASE-24047
> URL: https://issues.apache.org/jira/browse/HBASE-24047
> Project: HBase
> Issue Type: Task
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.0
>
> Attachments:
> 0001-HBASE-24047-Flakey-Tests-Disable-TestCustomSaslAuthe.patch
>
>
> Fails sporadically with relogin in progress complaints. See
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html
> FYI [~elserj]
> i.e.
> org.apache.hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider.testNegativeAuthentication[2:
> rpcClientImpl=org.apache.hadoop.hbase.ipc.NettyRpcClient,
> rpcServerImpl=org.apache.hadoop.hbase.ipc.SimpleRpcServer]
> {code}
> Failed after attempts=4, exceptions:
> 2020-03-25T21:25:05.239Z, RpcRetryingCaller{globalStartTime=1585171502143,
> pause=100, maxAttempts=4},
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> 2020-03-25T21:25:04.624Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.726Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.929Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:05.235Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> {code}
> Disabling rather than spending time in here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24047) [Flakey Tests] Disable TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
[
https://issues.apache.org/jira/browse/HBASE-24047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067199#comment-17067199
]
Michael Stack commented on HBASE-24047:
---
Attach patch of one-liner disable. Did it on branch-2 and branch-2.3 only for
now. Master seems more stable and is using different technique according to
comment in test method so leaving for now.
> [Flakey Tests] Disable
> TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
>
>
> Key: HBASE-24047
> URL: https://issues.apache.org/jira/browse/HBASE-24047
> Project: HBase
> Issue Type: Task
> Components: flakies
>Reporter: Michael Stack
>Priority: Major
> Attachments:
> 0001-HBASE-24047-Flakey-Tests-Disable-TestCustomSaslAuthe.patch
>
>
> Fails sporadically with relogin in progress complaints. See
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html
> FYI [~elserj]
> i.e.
> org.apache.hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider.testNegativeAuthentication[2:
> rpcClientImpl=org.apache.hadoop.hbase.ipc.NettyRpcClient,
> rpcServerImpl=org.apache.hadoop.hbase.ipc.SimpleRpcServer]
> {code}
> Failed after attempts=4, exceptions:
> 2020-03-25T21:25:05.239Z, RpcRetryingCaller{globalStartTime=1585171502143,
> pause=100, maxAttempts=4},
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> 2020-03-25T21:25:04.624Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.726Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.929Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:05.235Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> {code}
> Disabling rather than spending time in here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24047) [Flakey Tests] Disable TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
[
https://issues.apache.org/jira/browse/HBASE-24047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-24047.
---
Fix Version/s: 2.3.0
Assignee: Michael Stack
Resolution: Fixed
Pushed on branch-2.3 and branch-2 but not on master.
> [Flakey Tests] Disable
> TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
>
>
> Key: HBASE-24047
> URL: https://issues.apache.org/jira/browse/HBASE-24047
> Project: HBase
> Issue Type: Task
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.0
>
> Attachments:
> 0001-HBASE-24047-Flakey-Tests-Disable-TestCustomSaslAuthe.patch
>
>
> Fails sporadically with relogin in progress complaints. See
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html
> FYI [~elserj]
> i.e.
> org.apache.hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider.testNegativeAuthentication[2:
> rpcClientImpl=org.apache.hadoop.hbase.ipc.NettyRpcClient,
> rpcServerImpl=org.apache.hadoop.hbase.ipc.SimpleRpcServer]
> {code}
> Failed after attempts=4, exceptions:
> 2020-03-25T21:25:05.239Z, RpcRetryingCaller{globalStartTime=1585171502143,
> pause=100, maxAttempts=4},
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> 2020-03-25T21:25:04.624Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.726Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.929Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:05.235Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> {code}
> Disabling rather than spending time in here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #1352: spotbugs automatically skip inapplicable modules
ndimiduk opened a new pull request #1352: spotbugs automatically skip inapplicable modules URL: https://github.com/apache/hbase/pull/1352 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-24047) [Flakey Tests] Disable TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
[
https://issues.apache.org/jira/browse/HBASE-24047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-24047:
--
Attachment: 0001-HBASE-24047-Flakey-Tests-Disable-TestCustomSaslAuthe.patch
> [Flakey Tests] Disable
> TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
>
>
> Key: HBASE-24047
> URL: https://issues.apache.org/jira/browse/HBASE-24047
> Project: HBase
> Issue Type: Task
> Components: flakies
>Reporter: Michael Stack
>Priority: Major
> Attachments:
> 0001-HBASE-24047-Flakey-Tests-Disable-TestCustomSaslAuthe.patch
>
>
> Fails sporadically with relogin in progress complaints. See
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html
> FYI [~elserj]
> i.e.
> org.apache.hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider.testNegativeAuthentication[2:
> rpcClientImpl=org.apache.hadoop.hbase.ipc.NettyRpcClient,
> rpcServerImpl=org.apache.hadoop.hbase.ipc.SimpleRpcServer]
> {code}
> Failed after attempts=4, exceptions:
> 2020-03-25T21:25:05.239Z, RpcRetryingCaller{globalStartTime=1585171502143,
> pause=100, maxAttempts=4},
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> 2020-03-25T21:25:04.624Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.726Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.929Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:05.235Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> {code}
> Disabling rather than spending time in here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24047) [Flakey Tests] Disable TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
[
https://issues.apache.org/jira/browse/HBASE-24047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-24047:
--
Summary: [Flakey Tests] Disable
TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
(was: [Flakey Tests] Disable
TestCustomSaslAuthenticationProvider#testNegativeAuthentication)
> [Flakey Tests] Disable
> TestCustomSaslAuthenticationProvider#testNegativeAuthentication on branch-2.3
>
>
> Key: HBASE-24047
> URL: https://issues.apache.org/jira/browse/HBASE-24047
> Project: HBase
> Issue Type: Task
> Components: flakies
>Reporter: Michael Stack
>Priority: Major
> Attachments:
> 0001-HBASE-24047-Flakey-Tests-Disable-TestCustomSaslAuthe.patch
>
>
> Fails sporadically with relogin in progress complaints. See
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html
> FYI [~elserj]
> i.e.
> org.apache.hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider.testNegativeAuthentication[2:
> rpcClientImpl=org.apache.hadoop.hbase.ipc.NettyRpcClient,
> rpcServerImpl=org.apache.hadoop.hbase.ipc.SimpleRpcServer]
> {code}
> Failed after attempts=4, exceptions:
> 2020-03-25T21:25:05.239Z, RpcRetryingCaller{globalStartTime=1585171502143,
> pause=100, maxAttempts=4},
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=4, exceptions:
> 2020-03-25T21:25:04.624Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.726Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:04.929Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> 2020-03-25T21:25:05.235Z, RpcRetryingCaller{globalStartTime=1585171504623,
> pause=100, maxAttempts=4}, java.io.IOException: Call to
> asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
> java.io.IOException: Can not send request because relogin is in progress.
> {code}
> Disabling rather than spending time in here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24048) [Flakey Tests] Re-enable TestCustomSaslAuthenticationProvider#testNegativeAuthentication
Michael Stack created HBASE-24048: - Summary: [Flakey Tests] Re-enable TestCustomSaslAuthenticationProvider#testNegativeAuthentication Key: HBASE-24048 URL: https://issues.apache.org/jira/browse/HBASE-24048 Project: HBase Issue Type: Sub-task Components: flakies Reporter: Michael Stack See parent issue which disables flakey test. This task is to reenable once flakey-ness has been cleared. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24047) [Flakey Tests] Disable TestCustomSaslAuthenticationProvider#testNegativeAuthentication
Michael Stack created HBASE-24047:
-
Summary: [Flakey Tests] Disable
TestCustomSaslAuthenticationProvider#testNegativeAuthentication
Key: HBASE-24047
URL: https://issues.apache.org/jira/browse/HBASE-24047
Project: HBase
Issue Type: Task
Components: flakies
Reporter: Michael Stack
Fails sporadically with relogin in progress complaints. See
https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-2/lastSuccessfulBuild/artifact/dashboard.html
FYI [~elserj]
i.e.
org.apache.hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider.testNegativeAuthentication[2:
rpcClientImpl=org.apache.hadoop.hbase.ipc.NettyRpcClient,
rpcServerImpl=org.apache.hadoop.hbase.ipc.SimpleRpcServer]
{code}
Failed after attempts=4, exceptions:
2020-03-25T21:25:05.239Z, RpcRetryingCaller{globalStartTime=1585171502143,
pause=100, maxAttempts=4},
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=4, exceptions:
2020-03-25T21:25:04.624Z, RpcRetryingCaller{globalStartTime=1585171504623,
pause=100, maxAttempts=4}, java.io.IOException: Call to
asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
java.io.IOException: Can not send request because relogin is in progress.
2020-03-25T21:25:04.726Z, RpcRetryingCaller{globalStartTime=1585171504623,
pause=100, maxAttempts=4}, java.io.IOException: Call to
asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
java.io.IOException: Can not send request because relogin is in progress.
2020-03-25T21:25:04.929Z, RpcRetryingCaller{globalStartTime=1585171504623,
pause=100, maxAttempts=4}, java.io.IOException: Call to
asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
java.io.IOException: Can not send request because relogin is in progress.
2020-03-25T21:25:05.235Z, RpcRetryingCaller{globalStartTime=1585171504623,
pause=100, maxAttempts=4}, java.io.IOException: Call to
asf905.gq1.ygridcore.net/67.195.81.149:45715 failed on local exception:
java.io.IOException: Can not send request because relogin is in progress.
{code}
Disabling rather than spending time in here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23995) Snapshoting a splitting region results in corrupted snapshot
[
https://issues.apache.org/jira/browse/HBASE-23995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067182#comment-17067182
]
Josh Elser commented on HBASE-23995:
{quote}it looks like we grab a lock in the Master before taking a snapshot (via
LockManager/MasterLock), rather than a lock in PV2 which is what the
split/merge code would be grabbing
{quote}
Szabolcs was talking with me again this morning which had me looking again at
the code – I definitely did not look far enough :). One more layer deep, I
would've seen the LockProcedure which does this. I think Szabolcs missed
copying an update here, but he was able to confirm that hbase 2.2 wasn't
suffering this problem – the locks were working correctly.
Seems like something not happening quite right with PV2 table locks in the 2.0
version. Likely, something that's already been fixed :)
> Snapshoting a splitting region results in corrupted snapshot
>
>
> Key: HBASE-23995
> URL: https://issues.apache.org/jira/browse/HBASE-23995
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 2.0.2
>Reporter: Szabolcs Bukros
>Priority: Major
>
> The problem seems to originate from the fact that while the region split
> itself runs in a lock, the compactions following it run in separate threads.
> Alternatively the use of space quota policies can prevent compaction after a
> split and leads to the same issue.
> In both cases the resulting snapshot will keep the split status of the parent
> region, but do not keep the references to the daughter regions, because they
> (splitA, splitB qualifiers) are stored separately in the meta table and do
> not propagate with the snapshot.
> This is important because the in the freshly cloned table CatalogJanitor will
> find the parent region, realizes it is in split state, but because it can not
> find the daughter region references (haven't propagated) assumes parent could
> be cleaned up and deletes it. The archived region used in the snaphost only
> has back reference to the now also archived parent region and if the snapshot
> is deleted they both gets cleaned up. Unfortunately the daughter regions only
> contains hfile links, so at this point the data is lost.
> How to reproduce:
> {code:java}
> hbase shell < create 'test', 'cf'
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val"}
> flush 'test'
> split 'test'
> snapshot 'test', 'testshot'
> EOF
> {code}
> This should make sure the snapshot is made before the compaction could be
> finished even with small amount of data.
> {code:java}
> sudo -u habse hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> testshot -copy-to hdfs://target:8020/apps/hbase/data/
> {code}
> I export the snapshot to make the usecase cleaner but deleting both the
> snapshot and the original table after the cloning should have the same effect.
> {code:java}
> clone_snapshot 'testshot', 'test2'
> delete_snapshot "testshot"
> {code}
> I'm not sure what would be the best way to fix this. Preventing snapshots
> when a region is in split state, would make snapshot creation problematic.
> Forcing to run compaction as part of the split thread would make it rather
> slow. Propagating the daughter region references could prevent the deletion
> of the cloned parent region and the data would not be broken anymore but I'm
> not sure we have a logic in place that could pick up the pieces and finish
> the split process.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11
Apache-HBase commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11 URL: https://github.com/apache/hbase/pull/1351#issuecomment-604112045 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 29s | master passed | | +1 :green_heart: | compile | 3m 33s | master passed | | +1 :green_heart: | shadedjars | 5m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 47s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 53s | the patch passed | | +1 :green_heart: | compile | 3m 29s | the patch passed | | +1 :green_heart: | javac | 3m 29s | the patch passed | | +1 :green_heart: | shadedjars | 5m 32s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 47s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 11m 55s | root in the patch failed. | | | | 50m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1351 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 225d91121d52 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Default Java | 1.8.0_232 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/testReport/ | | Max. process+thread count | 870 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11
Apache-HBase commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11 URL: https://github.com/apache/hbase/pull/1351#issuecomment-604111333 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 58s | master passed | | +1 :green_heart: | compile | 5m 8s | master passed | | +1 :green_heart: | shadedjars | 7m 57s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 24s | root in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 27s | the patch passed | | +1 :green_heart: | compile | 5m 17s | the patch passed | | +1 :green_heart: | javac | 5m 17s | the patch passed | | +1 :green_heart: | shadedjars | 7m 49s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 21s | root in the patch failed. | ||| _ Other Tests _ | | -0 :warning: | unit | 2m 8s | root in the patch failed. | | | | 48m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1351 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 458afc272f14 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Default Java | 2020-01-14 | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/testReport/ | | Max. process+thread count | 487 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-24007) Get `-PrunLargeTests` passing on JDK11
[ https://issues.apache.org/jira/browse/HBASE-24007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067107#comment-17067107 ] Nick Dimiduk commented on HBASE-24007: -- Over on the PR, mighty [~weichiu] [chimes in|https://github.com/apache/hbase/pull/1351#issuecomment-604095131]: bq. We've seen performance issue with JDK11 too. They took extra time to run that breaks our internal functional tests. Later we realized it's because JDK11 defaults to G1GC and thus the memory consumption is different/more > Get `-PrunLargeTests` passing on JDK11 > -- > > Key: HBASE-24007 > URL: https://issues.apache.org/jira/browse/HBASE-24007 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > Build on HBASE-23829 and HBASE-24006, now looking at large tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11
Apache-HBase commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11 URL: https://github.com/apache/hbase/pull/1351#issuecomment-604100869 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 12m 39s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 21m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1351 | | Optional Tests | dupname asflicense hadoopcheck xml | | uname | Linux be55078c58c4 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5e63509781 | | Max. process+thread count | 63 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1351/1/console | | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23808) [Flakey Test] TestMasterShutdown#testMasterShutdownBeforeStartingAnyRegionServer
[ https://issues.apache.org/jira/browse/HBASE-23808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067097#comment-17067097 ] Bharath Vissapragada commented on HBASE-23808: -- Ya, let me make an addendum patch with debug logging. Where did you see this? Are you able to repro it locally? > [Flakey Test] > TestMasterShutdown#testMasterShutdownBeforeStartingAnyRegionServer > > > Key: HBASE-23808 > URL: https://issues.apache.org/jira/browse/HBASE-23808 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 2.3.0 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.4 > > Attachments: > TEST-org.apache.hadoop.hbase.master.TestMasterShutdown.xml > > > Reproduces locally from time to time. Not much to go on here. Looks like the > test is trying to do some fancy HBase cluster initialization order on top of > a mini-cluster. Failure seems related to trying to start the HBase master > before HDFS is fully initialized. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] jojochuang commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11
jojochuang commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11 URL: https://github.com/apache/hbase/pull/1351#issuecomment-604095131 We've seen performance issue with JDK11 too. They took extra time to run that breaks our internal functional tests. Later we realized it's because JDK11 defaults to G1GC and thus the memory consumption is different/more This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1350: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-1
Apache-HBase commented on issue #1350: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-1 URL: https://github.com/apache/hbase/pull/1350#issuecomment-604091306 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 2s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 1m 27s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 9s | Maven dependency ordering for patch | | +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 9m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1350/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1350 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux ea8157489b93 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1350/out/precommit/personality/provided.sh | | git revision | branch-1 / 61f9048 | | Max. process+thread count | 42 (vs. ulimit of 1) | | modules | C: U: | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1350/1/console | | versions | git=1.9.1 maven=3.0.5 shellcheck=0.7.0 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11
ndimiduk commented on issue #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11 URL: https://github.com/apache/hbase/pull/1351#issuecomment-604089905 Anyone know of a better way of managing the minimum hadoop version dependency? Some enforcer-plugin foo? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk opened a new pull request #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11
ndimiduk opened a new pull request #1351: HBASE-24007 Get `-PrunLargeTests` passing on JDK11 URL: https://github.com/apache/hbase/pull/1351 Minor tweaks required to get passing runs of `-PrunLargeTests`. * Minimum Hadoop version is 3.2.0 due to [HADOOP-12760](https://issues.apache.org/jira/browse/HADOOP-12760). * JDK11 looks like it consumes more memory than JDK8, so failures due to OOME see more common here. Bumping heap allocated to surefire forks allows better pass rate. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk opened a new pull request #1350: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-1
ndimiduk opened a new pull request #1350: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-1 URL: https://github.com/apache/hbase/pull/1350 Signed-off-by: stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk commented on issue #1350: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-1
ndimiduk commented on issue #1350: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-1 URL: https://github.com/apache/hbase/pull/1350#issuecomment-604085146 @apurtell @bharathv @Reidd this one relevant to branch-1 also? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-24045) Support setting `-Dhadoop.profile` in adhoc_run_tests.sh
[ https://issues.apache.org/jira/browse/HBASE-24045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-24045: - Fix Version/s: 2.3.0 > Support setting `-Dhadoop.profile` in adhoc_run_tests.sh > > > Key: HBASE-24045 > URL: https://issues.apache.org/jira/browse/HBASE-24045 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0, 2.3.0 > > > Allow the caller to override the hadoop version used. This is required in > order to use this tool with JDK11. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #1349: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-2.3
ndimiduk opened a new pull request #1349: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-2.3 URL: https://github.com/apache/hbase/pull/1349 Signed-off-by: stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk merged pull request #1349: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-2.3
ndimiduk merged pull request #1349: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-2.3 URL: https://github.com/apache/hbase/pull/1349 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-24045) Support setting `-Dhadoop.profile` in adhoc_run_tests.sh
[ https://issues.apache.org/jira/browse/HBASE-24045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-24045. -- Resolution: Fixed > Support setting `-Dhadoop.profile` in adhoc_run_tests.sh > > > Key: HBASE-24045 > URL: https://issues.apache.org/jira/browse/HBASE-24045 > Project: HBase > Issue Type: Sub-task > Components: tooling >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 3.0.0, 2.3.0 > > > Allow the caller to override the hadoop version used. This is required in > order to use this tool with JDK11. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #1348: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-2
ndimiduk merged pull request #1348: Backport "HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh" to branch-2 URL: https://github.com/apache/hbase/pull/1348 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk opened a new pull request #1348: HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh
ndimiduk opened a new pull request #1348: HBASE-24045 Support setting `-Dhadoop.profile` in adhoc_run_tests.sh URL: https://github.com/apache/hbase/pull/1348 Signed-off-by: stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
