[GitHub] [hbase] Apache-HBase commented on pull request #3369: [BACKPORT] HBASE-25859 Reference class incorrectly parses the protobuf magic marker
Apache-HBase commented on pull request #3369: URL: https://github.com/apache/hbase/pull/3369#issuecomment-871938518 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 8m 48s | Docker failed to build yetus/hbase:7ff737df0f. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/3369 | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3369/2/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #3369: [BACKPORT] HBASE-25859 Reference class incorrectly parses the protobuf magic marker
saintstack commented on pull request #3369: URL: https://github.com/apache/hbase/pull/3369#issuecomment-871934720 I retriggered the build. Do you want to point this fix at 1.7 since 1.4 is EOL'd? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25596) Fix NPE in ReplicationSourceManager as well as avoid permanently unreplicated data due to EOFException from WAL
[
https://issues.apache.org/jira/browse/HBASE-25596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372396#comment-17372396
]
Michael Stack commented on HBASE-25596:
---
This issue seems unfinished reading the comments but it is resolved. Should it
be reopened? Or is there another issue that takes up where this left off?
Thanks.
> Fix NPE in ReplicationSourceManager as well as avoid permanently unreplicated
> data due to EOFException from WAL
> ---
>
> Key: HBASE-25596
> URL: https://issues.apache.org/jira/browse/HBASE-25596
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.4.2
>
>
> There seems to be a major issue with how we handle the EOF exception from
> WALEntryStream.
> Problem:
> When we see EOFException, we try to handle it and remove it from the log
> queue, but we never try to ship the existing batch of entries. *This is a
> permanent data loss in replication.*
>
> Secondly, we do not stop the reader on encountering the EOFException and thus
> if EOFException was on the last WAL, we still try to process the WALEntry
> stream and ship the empty batch with lastWALPath set to null. This is the
> reason of NPE as below which *crash* the region server.
> {code:java}
> 2021-02-16 15:33:21,293 ERROR [,60020,1613262147968]
> regionserver.ReplicationSource - Unexpected exception in
> ReplicationSourceWorkerThread,
> currentPath=nulljava.lang.NullPointerExceptionat
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:193)at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource$ReplicationSourceShipperThread.updateLogPosition(ReplicationSource.java:831)at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource$ReplicationSourceShipperThread.shipEdits(ReplicationSource.java:746)at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource$ReplicationSourceShipperThread.run(ReplicationSource.java:650)2021-02-16
> 15:33:21,294 INFO [,60020,1613262147968] regionserver.HRegionServer -
> STOPPED: Unexpected exception in ReplicationSourceWorkerThread
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #3372: HBASE-25986 set default value of normalization enabled from hbase site
virajjasani commented on a change in pull request #3372:
URL: https://github.com/apache/hbase/pull/3372#discussion_r661975280
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptor.java
##
@@ -275,11 +275,11 @@
/**
* Check if normalization enable flag of the table is true. If flag is false
- * then no region normalizer won't attempt to normalize this table.
+ * then region normalizer won't attempt to normalize this table.
*
- * @return true if region normalization is enabled for this table
+ * @return value of NORMALIZATION_ENABLED key for this table if present else
return null
*/
- boolean isNormalizationEnabled();
+ Boolean isNormalizationEnabled();
Review comment:
I think null handling could cause compatibility issue and hence might
not make this eligible to land on branch-2 and branch-1 minor releases.
@Apache9 @ndimiduk Wondering if it is good idea to deprecate this one and add
new API that returns an enum with values: {TRUE, FALSE, NOT_PRESENT_IN_TD} to
clearly migrate from here to new one?
This is to maintain source compatibility and with new API, we can make
decision of whether to use site config or TD property value and then it can go
to minor releases on branch-2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (HBASE-25821) Sorting by "Start time" in the Master UI does so alphabetically and not by date
[ https://issues.apache.org/jira/browse/HBASE-25821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25821. --- Resolution: Duplicate Resolving as duplicate. Looks like it was done a while ago and will show in hbase-2.4.x. See HBASE-25165 Shout if it ain't what you want [~ekrettek] > Sorting by "Start time" in the Master UI does so alphabetically and not by > date > --- > > Key: HBASE-25821 > URL: https://issues.apache.org/jira/browse/HBASE-25821 > Project: HBase > Issue Type: Bug > Components: master, UI >Affects Versions: 2.4.0, 2.3.4, 2.4.1, 2.3.5, 2.4.2 >Reporter: Evan Krettek >Priority: Minor > > When sorting by the start time column in the Maser UI it sorts alphabetically > and not by date -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3448: HBASE-26057 Remove reflections used to access Hadoop 2 API in FanOutOneBlockAsyncDFSOutputHelper
Apache-HBase commented on pull request #3448: URL: https://github.com/apache/hbase/pull/3448#issuecomment-871903890 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 1s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 53s | master passed | | +1 :green_heart: | compile | 3m 43s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | spotbugs | 2m 37s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 38s | the patch passed | | +1 :green_heart: | compile | 3m 37s | the patch passed | | +1 :green_heart: | javac | 3m 37s | the patch passed | | -0 :warning: | checkstyle | 0m 12s | hbase-asyncfs: The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | -0 :warning: | checkstyle | 1m 2s | hbase-server: The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 12s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 50m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3448/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3448 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux c5a0ab86b9c1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3448/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-asyncfs.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3448/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3448/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed' org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372361#comment-17372361
]
Michael Stack commented on HBASE-26042:
---
{quote}What is the ring buffer consume thread doing? If it could write out
entries, finally we could release the slots of the ring buffer and move forward?
{quote}
I didn't take the thread dumps. They are w/o thread names. There doesn't seem
to be any 'consume' going on. We skipped out scheduling a consume on 'sync
failed'? Trying to figure it...
> WAL lockup on 'sync failed'
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer
>
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5
>Reporter: Michael Stack
>Priority: Major
> Attachments: js1, js2
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state =
[GitHub] [hbase] jojochuang opened a new pull request #3448: HBASE-26057 Remove reflections used to access Hadoop 2 API in FanOutOneBlockAsyncDFSOutputHelper
jojochuang opened a new pull request #3448: URL: https://github.com/apache/hbase/pull/3448 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26057) Remove reflections used to access Hadoop 2 API in FanOutOneBlockAsyncDFSOutputHelper
[ https://issues.apache.org/jira/browse/HBASE-26057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HBASE-26057: Fix Version/s: 3.0.0-alpha-2 > Remove reflections used to access Hadoop 2 API in > FanOutOneBlockAsyncDFSOutputHelper > > > Key: HBASE-26057 > URL: https://issues.apache.org/jira/browse/HBASE-26057 > Project: HBase > Issue Type: Sub-task >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Fix For: 3.0.0-alpha-2 > > > Remove the reflections used to access Hadoop 2 APIs in HBase 3.x. > There are still a number of reflections we can't remove now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-26057) Remove reflections used to access Hadoop 2 API in FanOutOneBlockAsyncDFSOutputHelper
[ https://issues.apache.org/jira/browse/HBASE-26057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HBASE-26057: --- Assignee: Wei-Chiu Chuang > Remove reflections used to access Hadoop 2 API in > FanOutOneBlockAsyncDFSOutputHelper > > > Key: HBASE-26057 > URL: https://issues.apache.org/jira/browse/HBASE-26057 > Project: HBase > Issue Type: Sub-task >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > > Remove the reflections used to access Hadoop 2 APIs in HBase 3.x. > There are still a number of reflections we can't remove now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26057) Remove reflections used to access Hadoop 2 API in FanOutOneBlockAsyncDFSOutputHelper
Wei-Chiu Chuang created HBASE-26057: --- Summary: Remove reflections used to access Hadoop 2 API in FanOutOneBlockAsyncDFSOutputHelper Key: HBASE-26057 URL: https://issues.apache.org/jira/browse/HBASE-26057 Project: HBase Issue Type: Sub-task Reporter: Wei-Chiu Chuang Remove the reflections used to access Hadoop 2 APIs in HBase 3.x. There are still a number of reflections we can't remove now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-26028) The view as json page shows exception when using TinyLfuBlockCache
[ https://issues.apache.org/jira/browse/HBASE-26028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-26028: --- Fix Version/s: 2.4.5 2.3.6 2.5.0 > The view as json page shows exception when using TinyLfuBlockCache > -- > > Key: HBASE-26028 > URL: https://issues.apache.org/jira/browse/HBASE-26028 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 2.5.0, 2.3.6, 3.0.0-alpha-2, 2.4.5 > > Attachments: HBASE-26028-afterpatch.jpg, HBASE-26028-beforepatch.jpg > > > Some variable in TinyLfuBlockCache should be marked as transient. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26056) Cell TTL set to -1 should mean never expire, the same as CF TTL
Baiqiang Zhao created HBASE-26056: - Summary: Cell TTL set to -1 should mean never expire, the same as CF TTL Key: HBASE-26056 URL: https://issues.apache.org/jira/browse/HBASE-26056 Project: HBase Issue Type: Bug Reporter: Baiqiang Zhao Assignee: Baiqiang Zhao We met a user who took it for granted that setting the cell TTL to -1 is the same as the CF TTL, which means that it will never expire. However, if the cell TTL is set to -1, the cell will expire immediately. In fact, if the cell TTL is not set, it means that it will never expire, but he just set it.. We should unify the meaning of cell TTL and CF TTL on the value of -1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23956) Use less resources running tests
[
https://issues.apache.org/jira/browse/HBASE-23956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372315#comment-17372315
]
Hudson commented on HBASE-23956:
Results for branch branch-2.4
[build #154 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/154/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/154/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/154/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/154/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/154/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use less resources running tests
>
>
> Key: HBASE-23956
> URL: https://issues.apache.org/jira/browse/HBASE-23956
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments: Screen Shot 2020-03-09 at 10.11.26 PM.png, Screen Shot
> 2020-03-09 at 10.11.35 PM.png, Screen Shot 2020-03-09 at 10.20.08 PM.png,
> Screen Shot 2020-03-10 at 7.10.15 AM.png, Screen Shot 2020-03-10 at 7.10.28
> AM.png
>
>
> Our tests can create thousands of threads all up in the one JVM. Using less
> means less memory, less contention, likelier passes, and later, more possible
> parallelism.
> I've been studying the likes of TestNamespaceReplicationWithBulkLoadedData to
> see what it does as it runs (this test puts up 4 clusters with replication
> between). It peaks at 2k threads. After some configuration and using less
> HDFS, its possible to get it down to ~800 threads and about 1/2 the
> memory-used. HDFS is a main offender. DataXceivers (Server and Client), jetty
> threads, Volume threads (async disk 'worker' then another for cleanup...),
> image savers, ipc clients -- new thread per incoming connection w/o bound (or
> reuse), block responder threads, anonymous threads, and so on. Many are not
> configurable or boundable or are hard-coded; e.g. each volume gets 4 workers
> regardless. Biggest impact was just downing the count of data nodes. TODO: a
> follow-on that turns down DN counts in all tests.
> I've been using Java Flight Recorder during this study. Here is how you get a
> flight recorder for the a single test run: \{code:java} MAVEN_OPTS="
> -XX:StartFlightRecording=disk=true,dumponexit=true,filename=recording.jfr,settings=profile,path-to-gc-roots=true,maxsize=1024m"
> mvn test -Dtest=TestNamespaceReplicationWithBulkLoadedData
> -Dsurefire.firstPartForkCount=0 -Dsurefire.secondPartForkCount=0 \{code} i.e.
> start recording on mvn launch, bound the size of the recording, and have the
> test run in the mvn context (DON'T fork). Useful is connecting to the running
> test at the same time from JDK Mission Control. We do the latter because the
> thread reporting screen is overwhelmed by the count of running threads and if
> you connect live, you can at least get a 'live threads' graph w/ count as the
> test progresses. Useful. When the test finishes, it dumps a .jfr file which
> can be opened in JDK MC.
> I've been compiling w/ JDK8 and then running w/ JDK11 so I can use JDK MC
> Version 7, the non-commercial latest. Works pretty well. Let me put up a
> patch for tests that cuts down thread counts where we can.
> Let me put up a patch that does first pass on curtailing resource usage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26039) TestReplicationKillRS is useless after HBASE-23956
[
https://issues.apache.org/jira/browse/HBASE-26039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372314#comment-17372314
]
Hudson commented on HBASE-26039:
Results for branch branch-2.4
[build #154 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/154/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/154/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/154/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/154/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/154/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestReplicationKillRS is useless after HBASE-23956
> --
>
> Key: HBASE-26039
> URL: https://issues.apache.org/jira/browse/HBASE-26039
> Project: HBase
> Issue Type: Bug
> Components: Replication, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> In TestReplicationKillRS, we assume there are at least 2 regionservers but in
> HBASE-23956, we set the region server number to 1.
> So in fact, we do not kill any region servers which makes the tests useless...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-26028) The view as json page shows exception when using TinyLfuBlockCache
[ https://issues.apache.org/jira/browse/HBASE-26028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-26028: --- Fix Version/s: 3.0.0-alpha-2 > The view as json page shows exception when using TinyLfuBlockCache > -- > > Key: HBASE-26028 > URL: https://issues.apache.org/jira/browse/HBASE-26028 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Fix For: 3.0.0-alpha-2 > > Attachments: HBASE-26028-afterpatch.jpg, HBASE-26028-beforepatch.jpg > > > Some variable in TinyLfuBlockCache should be marked as transient. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-26028) The view as json page shows exception when using TinyLfuBlockCache
[ https://issues.apache.org/jira/browse/HBASE-26028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang resolved HBASE-26028. Resolution: Fixed > The view as json page shows exception when using TinyLfuBlockCache > -- > > Key: HBASE-26028 > URL: https://issues.apache.org/jira/browse/HBASE-26028 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Attachments: HBASE-26028-afterpatch.jpg, HBASE-26028-beforepatch.jpg > > > Some variable in TinyLfuBlockCache should be marked as transient. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26028) The view as json page shows exception when using TinyLfuBlockCache
[ https://issues.apache.org/jira/browse/HBASE-26028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372313#comment-17372313 ] Zheng Wang commented on HBASE-26028: Thanks for the reviewing. [~vjasani] > The view as json page shows exception when using TinyLfuBlockCache > -- > > Key: HBASE-26028 > URL: https://issues.apache.org/jira/browse/HBASE-26028 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Attachments: HBASE-26028-afterpatch.jpg, HBASE-26028-beforepatch.jpg > > > Some variable in TinyLfuBlockCache should be marked as transient. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3447: HBASE-26055 Master local region "table" should be considered a system…
Apache-HBase commented on pull request #3447: URL: https://github.com/apache/hbase/pull/3447#issuecomment-871849184 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 0s | master passed | | +1 :green_heart: | compile | 1m 45s | master passed | | +1 :green_heart: | shadedjars | 9m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 56s | the patch passed | | +1 :green_heart: | compile | 1m 43s | the patch passed | | +1 :green_heart: | javac | 1m 43s | the patch passed | | +1 :green_heart: | shadedjars | 9m 10s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 11s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 183m 30s | hbase-server in the patch passed. | | | | 223m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3447 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7830fd98ca57 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/testReport/ | | Max. process+thread count | 2429 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3447: HBASE-26055 Master local region "table" should be considered a system…
Apache-HBase commented on pull request #3447: URL: https://github.com/apache/hbase/pull/3447#issuecomment-871848171 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 15s | master passed | | +1 :green_heart: | compile | 1m 24s | master passed | | +1 :green_heart: | shadedjars | 9m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 57s | the patch passed | | +1 :green_heart: | compile | 1m 25s | the patch passed | | +1 :green_heart: | javac | 1m 25s | the patch passed | | +1 :green_heart: | shadedjars | 9m 0s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 2s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 184m 37s | hbase-server in the patch passed. | | | | 220m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3447 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 384ee11222a9 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/testReport/ | | Max. process+thread count | 2223 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3447: HBASE-26055 Master local region "table" should be considered a system…
Apache9 commented on pull request #3447: URL: https://github.com/apache/hbase/pull/3447#issuecomment-871843519 Do you guys face any problems? The master local region is not a typical table, so in general it should not be conflict with any real tables or regions. You could have a user level table called master:local, no problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed' org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372298#comment-17372298
]
Duo Zhang commented on HBASE-26042:
---
What is the ring buffer consume thread doing? If it could write out entries,
finally we could release the slots of the ring buffer and move forward?
> WAL lockup on 'sync failed'
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer
>
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5
>Reporter: Michael Stack
>Priority: Major
> Attachments: js1, js2
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> -
[GitHub] [hbase] jojochuang commented on pull request #3446: HBASE-26051 Remove reflections used to access HDFS EC APIs
jojochuang commented on pull request #3446: URL: https://github.com/apache/hbase/pull/3446#issuecomment-871841648 Yeah this one is HBase 3.x only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed' org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372295#comment-17372295
]
Michael Stack commented on HBASE-26042:
---
Deadlock in coarse form is 116 of 200 handlers are in mvcc#begin of which 70
are down on the ringbuffer waiting on minimum sequence to catch up. 86
handlers are trying to do mvcc#complete but can't get in because more are in
mvcc#begins holding the synchronize lock.
The log roll wouldn't fix this issue; even if it managed to write the WAL
header (where it is currently stuck) it would then go into a wait on safe point
before progressing w/ replacement..
> WAL lockup on 'sync failed'
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer
>
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5
>Reporter: Michael Stack
>Priority: Major
> Attachments: js1, js2
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to
[GitHub] [hbase] taklwu commented on pull request #3447: HBASE-26055 Master local region "table" should be considered a system…
taklwu commented on pull request #3447: URL: https://github.com/apache/hbase/pull/3447#issuecomment-871833475 The master namespace logic looks good to me, but I have a minor question, do you see anywhere we have any procedure or cleanup logic delete the master namespace ? > Should we prevent a user from creating a "master" namespace? Currently createNamespace doesn't fail for system namespaces (because it's assumed that the namespace already exists) I found below that the `prepareCreate` as part fo the `CreateNamespaceProcedure` should fail if the namespace already exists, and you're right that it didn't protect the system namespace(s). so, can you explain in what case the system namespaces in the meta table before it comes online would have the pre-existed namespaces? is it from a previous version ? https://github.com/apache/hbase/blob/202b17f4fc3229e91d584837776d53ed3e2e8adb/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateNamespaceProcedure.java#L180-L193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed' org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372272#comment-17372272
]
Michael Stack commented on HBASE-26042:
---
Looking at thread dumps we seem to be stuck on the ring buffer. There are 415
threads total, 200 of which are handlers. ~70 are stuck here:
{code:java}
6037 Thread 8667: (state = BLOCKED)
6038 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
may be imprecise)
6039 - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11,
line=338 (Compiled frame)
6040 - com.lmax.disruptor.MultiProducerSequencer.next(int) @bci=82, line=136
(Compiled frame)
6041 - com.lmax.disruptor.MultiProducerSequencer.next() @bci=2, line=105
(Interpreted frame)
6042 - com.lmax.disruptor.RingBuffer.next() @bci=4, line=263 (Compiled frame)
6043 -
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.lambda$stampSequenceIdAndPublishToRingBuffer$1(org.apache.commons.lang3.mutable.MutableLong,
com.lmax.disruptor.RingBuffer) @bci=2, line=1031 (Compiled frame)
6044 -
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL$$Lambda$270.run() @bci=8
(Compiled frame)
6045 -
org.apache.hadoop.hbase.regionserver.MultiVersionConcurrencyControl.begin(java.lang.Runnable)
@bci=36, line=140 (Interpreted frame)
6046 -
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.stampSequenceIdAndPublishToRingBuffer(org.apache.hadoop.hbase.client.RegionInfo,
org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit,
boolean, com.lma x.disruptor.RingBuffer) @bci=62, line=1030 (Compiled frame)
6047 -
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.append(org.apache.hadoop.hbase.client.RegionInfo,
org.apache.hadoop.hbase.wal.WALKeyImpl, org.apache.hadoop.hbase.wal.WALEdit,
boolean) @bci=10, line=589 (Compiled frame) {code}
waiting for a minimum sequence number to catch up
[https://github.com/LMAX-Exchange/disruptor/blob/master/src/main/java/com/lmax/disruptor/MultiProducerSequencer.java#L111]
> WAL lockup on 'sync failed'
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer
>
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5
>Reporter: Michael Stack
>Priority: Major
> Attachments: js1, js2
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
>
[GitHub] [hbase] Apache-HBase commented on pull request #3447: HBASE-26055 Master local region "table" should be considered a system…
Apache-HBase commented on pull request #3447: URL: https://github.com/apache/hbase/pull/3447#issuecomment-871782251 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 51s | master passed | | +1 :green_heart: | compile | 4m 4s | master passed | | +1 :green_heart: | checkstyle | 1m 28s | master passed | | +1 :green_heart: | spotbugs | 2m 47s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 37s | the patch passed | | +1 :green_heart: | compile | 4m 0s | the patch passed | | +1 :green_heart: | javac | 4m 0s | the patch passed | | +1 :green_heart: | checkstyle | 1m 27s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 24s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 3m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 53m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3447 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux ea775843c42e 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3447/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3438: HBASE-22923 min version of RegionServer to move system table regions
Apache-HBase commented on pull request #3438: URL: https://github.com/apache/hbase/pull/3438#issuecomment-871762414 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 15m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 23s | branch-1 passed | | +1 :green_heart: | compile | 0m 52s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 52s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 2m 3s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 24s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 43s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 3m 16s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 12s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 2s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 47s | the patch passed | | -1 :x: | checkstyle | 1m 50s | hbase-server: The patch generated 4 new + 231 unchanged - 0 fixed = 235 total (was 231) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 6s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 46s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 43s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 3m 7s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 145m 55s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 30s | The patch does not generate ASF License warnings. | | | | 205m 16s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.mapreduce.TestLoadIncrementalHFiles | | | hadoop.hbase.security.visibility.TestVisibilityLabelsWithACL | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3438/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3438 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 774aa1b1dcb2 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-3438/out/precommit/personality/provided.sh | | git revision | branch-1 / 28f36f4 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3438/3/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3438/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3438/3/testReport/ | | Max. process+thread count | 4488 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output |
[GitHub] [hbase] z-york commented on pull request #3447: HBASE-26055 Master local region "table" should be considered a system…
z-york commented on pull request #3447: URL: https://github.com/apache/hbase/pull/3447#issuecomment-871761858 One open question with this one: Should we prevent a user from creating a "master" namespace? Currently createNamespace doesn't fail for system namespaces (because it's assumed that the namespace already exists). In this case, the namespace doesn't exist, but the master namespace prefix is being used for a system table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] z-york opened a new pull request #3447: HBASE-26055 Master local region "table" should be considered a system…
z-york opened a new pull request #3447: URL: https://github.com/apache/hbase/pull/3447 … table The master local region is not a typical Table/Region, however, we should consider it a system table when creating the TableName so that if we call isSystemTable() it returns true. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26055) Master local region should be considered as a system table
Zach York created HBASE-26055: - Summary: Master local region should be considered as a system table Key: HBASE-26055 URL: https://issues.apache.org/jira/browse/HBASE-26055 Project: HBase Issue Type: Bug Components: master Affects Versions: 2.4.4, 3.0.0-alpha-1 Reporter: Zach York Assignee: Zach York The title says most of it. The master local region is not a typical Table/Region, however, we should consider it a system table when creating the TableName so that if we call isSystemTable() it returns true. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
Apache-HBase commented on pull request #3439: URL: https://github.com/apache/hbase/pull/3439#issuecomment-871756565 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 2s | master passed | | +1 :green_heart: | compile | 1m 1s | master passed | | +1 :green_heart: | shadedjars | 8m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 43s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 8m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 129m 9s | hbase-server in the patch passed. | | | | 159m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3439 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4689563b174d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/3/testReport/ | | Max. process+thread count | 3322 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
Apache-HBase commented on pull request #3439: URL: https://github.com/apache/hbase/pull/3439#issuecomment-871754569 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 34s | master passed | | +1 :green_heart: | compile | 1m 13s | master passed | | +1 :green_heart: | shadedjars | 8m 17s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 14s | the patch passed | | +1 :green_heart: | compile | 1m 14s | the patch passed | | +1 :green_heart: | javac | 1m 14s | the patch passed | | +1 :green_heart: | shadedjars | 8m 15s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 123m 35s | hbase-server in the patch failed. | | | | 155m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3439 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6deea6df1763 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/3/testReport/ | | Max. process+thread count | 3863 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] z-york commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
z-york commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661827001
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
Review comment:
Stephen beat me to it :) I'm not sure if it's setup correctly for
hbase-operator-tools (the test report lists spotbugs as 'spotbugs executables
are not available').
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] taklwu commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
taklwu commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661825885
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
Review comment:
I may be out of the picture, but in this tool, did we enable any
findbugs plugin?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] z-york commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
z-york commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661827001
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
Review comment:
Btw, are we actually running findbugs? I'm not sure if it's setup
correctly for hbase-operator-tools (the test report lists spotbugs as 'spotbugs
executables are not available').
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] taklwu commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
taklwu commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661825885
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
Review comment:
I may be out of the picture, but in this tool, did we enable any
findbugs plugin?
if we're issuing findbugs warnings, we're doing it somewhat right with
`@CheckForNull` but this annotation is for JSR-305 and it's a bit old?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] taklwu commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
taklwu commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661825885
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
Review comment:
I may be out of the picture, but in this tool, did we enable any
findbugs plugin ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] saintstack commented on pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
saintstack commented on pull request #90: URL: https://github.com/apache/hbase-operator-tools/pull/90#issuecomment-871741704 Thanks @z-york for explaining why the changes needed. Suggest that updating the hbase dependency is another issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] saintstack commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
saintstack commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661824460
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
Review comment:
@busbey just the usual... lets minimize dependencies if we can. Was
wondering if the dependency was just because of some silly, minor usage that we
could as well do w/o.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] Apache-HBase commented on pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
Apache-HBase commented on pull request #90: URL: https://github.com/apache/hbase-operator-tools/pull/90#issuecomment-871736324 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | spotbugs | 0m 0s | spotbugs executables are not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 0m 29s | master passed | | +1 :green_heart: | compile | 0m 20s | master passed | | +1 :green_heart: | checkstyle | 0m 11s | master passed | | +1 :green_heart: | javadoc | 0m 14s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 4s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 0m 28s | the patch passed | | +1 :green_heart: | compile | 0m 20s | the patch passed | | +1 :green_heart: | javac | 0m 20s | the patch passed | | +1 :green_heart: | checkstyle | 0m 8s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | javadoc | 0m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 5m 13s | hbase-hbck2 in the patch passed. | | -1 :x: | unit | 7m 9s | root in the patch failed. | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 16m 17s | | | Reason | Tests | |---:|:--| | Failed junit tests | hbase.TestRegionsMerger | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-90/2/artifact/yetus-precommit-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase-operator-tools/pull/90 | | Optional Tests | dupname asflicense javac javadoc unit xml compile spotbugs findbugs checkstyle | | uname | Linux 00697b26d0a5 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux | | Build tool | maven | | git revision | master / 2317f13 | | Default Java | Oracle Corporation-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-90/2/artifact/yetus-precommit-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-90/2/testReport/ | | Max. process+thread count | 1218 (vs. ulimit of 5000) | | modules | C: hbase-hbck2 . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-90/2/console | | versions | git=2.20.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] taklwu commented on pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
taklwu commented on pull request #90: URL: https://github.com/apache/hbase-operator-tools/pull/90#issuecomment-871729919 I tried to build with `-Dhbase.version=2.3.5` and it worked. IMO if we would like to move with `hbase.version`, should we do it with the stable pointer as of today it's [2.3.5](https://hbase.apache.org/downloads.html)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] busbey commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
busbey commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661812267
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
Review comment:
Looks like it's all `CheckForNull` annotations to make spotbugs /
findbugs do a better job of warning about usage.
https://spotbugs.readthedocs.io/en/stable/annotations.html#edu-umd-cs-findbugs-annotations-checkfornull
Any particular reason you want to get this out of here @saintstack ? we have
the stephenc clean room findbugs dependency in lots of places across the
project already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] busbey commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
busbey commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661812267
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
Review comment:
Looks like it's all `CheckForNull` annotations to make spotbugs /
findbugs do a better job of warning about usage.
https://spotbugs.readthedocs.io/en/stable/annotations.html#edu-umd-cs-findbugs-annotations-checkfornull
Any particular reason you want to get this out of here @saintstack ? we have
this one in lots of places across the project already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] z-york commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
z-york commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661810398
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
+ compile
+
Review comment:
Good call out, fixed in latest update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] taklwu commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
taklwu commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661808404
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
+ compile
+
Review comment:
[nit] indentation is off 2 spaces
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] z-york commented on pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
z-york commented on pull request #90: URL: https://github.com/apache/hbase-operator-tools/pull/90#issuecomment-871724033 Also, I'm not sure what our procedure is for upgrading the HBase version in operator tools. I left it at the lower version since the code change works for both versions, but I can upgrade too if it makes sense. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] z-york commented on pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
z-york commented on pull request #90: URL: https://github.com/apache/hbase-operator-tools/pull/90#issuecomment-871722145 Added build output from building with 2.4.4 to the JIRA -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26054) Fix hbase-operator-tools build with HBase 2.4.4
[
https://issues.apache.org/jira/browse/HBASE-26054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zach York updated HBASE-26054:
--
Description:
I tried building hbase-operator-tools with HBase 2.4.4 and encountered a number
of issues. Opening this for the fix.
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile)
on project hbase-hbck2: Compilation failure: Compilation failure:
[ERROR]
/Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKAbstractFileStatusFilter.java:[20,39]
package edu.umd.cs.findbugs.annotations does not exist
[ERROR]
/Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKAbstractFileStatusFilter.java:[52,46]
cannot find symbol
[ERROR] symbol: class CheckForNull
[ERROR] location: class org.apache.hbase.HBCKAbstractFileStatusFilter
[ERROR]
/Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKAbstractFileStatusFilter.java:[64,44]
cannot find symbol
[ERROR] symbol: class CheckForNull
[ERROR] location: class org.apache.hbase.HBCKAbstractFileStatusFilter
[ERROR]
/Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKAbstractFileStatusFilter.java:[68,49]
cannot find symbol
[ERROR] symbol: class CheckForNull
[ERROR] location: class org.apache.hbase.HBCKAbstractFileStatusFilter
[ERROR]
/Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKFsUtils.java:[367,39]
cannot find symbol
[ERROR] symbol: class CheckForNull
[ERROR] location: class org.apache.hbase.HBCKFsUtils.FamilyDirFilter
[ERROR]
/Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKFsUtils.java:[402,39]
cannot find symbol
[ERROR] symbol: class CheckForNull
[ERROR] location: class org.apache.hbase.HBCKFsUtils.RegionDirFilter
[ERROR] -> [Help 1]
{noformat}
The Threads change is masked by these changes, but the method is clearly
removed in HBASE-24750
`
was:I tried building hbase-operator-tools with HBase 2.4.4 and encountered a
number of issues. Opening this for the fix.
> Fix hbase-operator-tools build with HBase 2.4.4
> ---
>
> Key: HBASE-26054
> URL: https://issues.apache.org/jira/browse/HBASE-26054
> Project: HBase
> Issue Type: Bug
> Components: hbase-operator-tools
>Affects Versions: hbase-operator-tools-1.2.0
>Reporter: Zach York
>Assignee: Zach York
>Priority: Minor
>
> I tried building hbase-operator-tools with HBase 2.4.4 and encountered a
> number of issues. Opening this for the fix.
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile
> (default-compile) on project hbase-hbck2: Compilation failure: Compilation
> failure:
> [ERROR]
> /Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKAbstractFileStatusFilter.java:[20,39]
> package edu.umd.cs.findbugs.annotations does not exist
> [ERROR]
> /Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKAbstractFileStatusFilter.java:[52,46]
> cannot find symbol
> [ERROR] symbol: class CheckForNull
> [ERROR] location: class org.apache.hbase.HBCKAbstractFileStatusFilter
> [ERROR]
> /Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKAbstractFileStatusFilter.java:[64,44]
> cannot find symbol
> [ERROR] symbol: class CheckForNull
> [ERROR] location: class org.apache.hbase.HBCKAbstractFileStatusFilter
> [ERROR]
> /Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKAbstractFileStatusFilter.java:[68,49]
> cannot find symbol
> [ERROR] symbol: class CheckForNull
> [ERROR] location: class org.apache.hbase.HBCKAbstractFileStatusFilter
> [ERROR]
> /Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKFsUtils.java:[367,39]
> cannot find symbol
> [ERROR] symbol: class CheckForNull
> [ERROR] location: class org.apache.hbase.HBCKFsUtils.FamilyDirFilter
> [ERROR]
> /Users/zyork/workspace/apache/hbase-operator-tools/hbase-hbck2/src/main/java/org/apache/hbase/HBCKFsUtils.java:[402,39]
> cannot find symbol
> [ERROR] symbol: class CheckForNull
> [ERROR] location: class org.apache.hbase.HBCKFsUtils.RegionDirFilter
> [ERROR] -> [Help 1]
> {noformat}
> The Threads change is masked by these changes, but the method is clearly
> removed in HBASE-24750
> `
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase-operator-tools] z-york commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
z-york commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661803931
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
Review comment:
I'm not very familiar with the findbugs annotations. I guess we could
change to using Preconditions or something, but that will be a runtime check.
If findbugs is doing something at compile time with those annotations, it might
not be a comparable switch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-operator-tools] z-york commented on pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
z-york commented on pull request #90: URL: https://github.com/apache/hbase-operator-tools/pull/90#issuecomment-871719190 @saintstack The executor changes are caused by https://github.com/apache/hbase/commit/8ccf643fdc96a5f70106a50fb08f0e37dcf80ec5 (basically the method was removed in favor of using the guava method). https://github.com/apache/hbase-operator-tools/commit/b461d58833cee465b18e46780ebb85c7b159df44 Adds the CheckForNull. I also see that it's referenced in core HBase (for example in FSUtils.java). The reason this fails for 2.4.4 is due to this change: https://issues.apache.org/jira/browse/HBASE-23077 that adds the dependency as compile scope where needed within HBase core. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3417: HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade
Apache-HBase commented on pull request #3417: URL: https://github.com/apache/hbase/pull/3417#issuecomment-871712281 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.4 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 45s | branch-2.4 passed | | +1 :green_heart: | compile | 1m 46s | branch-2.4 passed | | +1 :green_heart: | shadedjars | 7m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 10s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 29s | the patch passed | | +1 :green_heart: | compile | 1m 49s | the patch passed | | +1 :green_heart: | javac | 1m 49s | the patch passed | | +1 :green_heart: | shadedjars | 7m 49s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 54s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 213m 9s | hbase-server in the patch passed. | | | | 250m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3417/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3417 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a7a25f59752e 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / c5a6e19be0 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3417/3/testReport/ | | Max. process+thread count | 2554 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3417/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] saintstack commented on a change in pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
saintstack commented on a change in pull request #90:
URL:
https://github.com/apache/hbase-operator-tools/pull/90#discussion_r661788980
##
File path: hbase-hbck2/pom.xml
##
@@ -235,6 +235,12 @@
2.1.0
test
+
+ com.github.stephenc.findbugs
+ findbugs-annotations
+ ${findbugs-annotations.version}
Review comment:
Do we need this thing? Can we purge the annotation referenced?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3441: HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade
Apache-HBase commented on pull request #3441: URL: https://github.com/apache/hbase/pull/3441#issuecomment-871705835 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 1s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 2s | master passed | | +1 :green_heart: | compile | 1m 50s | master passed | | +1 :green_heart: | shadedjars | 9m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 7s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 42s | the patch passed | | +1 :green_heart: | compile | 1m 48s | the patch passed | | +1 :green_heart: | javac | 1m 48s | the patch passed | | +1 :green_heart: | shadedjars | 9m 1s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 40s | hbase-client in the patch passed. | | -1 :x: | unit | 193m 4s | hbase-server in the patch failed. | | | | 232m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3441/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3441 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6d7691569e39 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3441/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3441/2/testReport/ | | Max. process+thread count | 2624 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3441/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25729) Upgrade to latest hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-25729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25729. --- Hadoop Flags: Reviewed Assignee: Andrew Kyle Purtell Resolution: Duplicate Making this a duplicate of [~pankajkumar] issue; Pankaj did the work of this issues subject matter over in HBASE-25918 . This is a re-resolve because we reopened the issue to discuss whether or not latest hbase-thirdparty appropriate in branch-2.4 and conclude it fine. Re-resolving. > Upgrade to latest hbase-thirdparty > -- > > Key: HBASE-25729 > URL: https://issues.apache.org/jira/browse/HBASE-25729 > Project: HBase > Issue Type: Sub-task > Components: build, thirdparty >Affects Versions: 2.4.2 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.5 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
Apache-HBase commented on pull request #3439: URL: https://github.com/apache/hbase/pull/3439#issuecomment-871697223 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 44s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 3m 24s | master passed | | +1 :green_heart: | checkstyle | 1m 1s | master passed | | +1 :green_heart: | spotbugs | 2m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 54s | the patch passed | | +1 :green_heart: | compile | 3m 28s | the patch passed | | +1 :green_heart: | javac | 3m 28s | the patch passed | | +1 :green_heart: | checkstyle | 1m 5s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 22s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. | | | | 49m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3439 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux ac41f7582bed 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] Apache-HBase commented on pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
Apache-HBase commented on pull request #90: URL: https://github.com/apache/hbase-operator-tools/pull/90#issuecomment-871676451 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | spotbugs | 0m 0s | spotbugs executables are not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 31s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 0m 29s | master passed | | +1 :green_heart: | compile | 0m 20s | master passed | | +1 :green_heart: | checkstyle | 0m 10s | master passed | | +1 :green_heart: | javadoc | 0m 12s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 4s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 0m 27s | the patch passed | | +1 :green_heart: | compile | 0m 19s | the patch passed | | +1 :green_heart: | javac | 0m 19s | the patch passed | | +1 :green_heart: | checkstyle | 0m 8s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | javadoc | 0m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 5m 17s | hbase-hbck2 in the patch passed. | | +1 :green_heart: | unit | 7m 15s | root in the patch passed. | | +1 :green_heart: | asflicense | 0m 9s | The patch does not generate ASF License warnings. | | | | 16m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-90/1/artifact/yetus-precommit-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase-operator-tools/pull/90 | | Optional Tests | dupname asflicense javac javadoc unit xml compile spotbugs findbugs checkstyle | | uname | Linux 8f4f1c022aec 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux | | Build tool | maven | | git revision | master / 2317f13 | | Default Java | Oracle Corporation-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-90/1/testReport/ | | Max. process+thread count | 1186 (vs. ulimit of 5000) | | modules | C: hbase-hbck2 . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-90/1/console | | versions | git=2.20.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3417: HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade
Apache-HBase commented on pull request #3417: URL: https://github.com/apache/hbase/pull/3417#issuecomment-871670774 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.4 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 40s | branch-2.4 passed | | +1 :green_heart: | compile | 1m 24s | branch-2.4 passed | | +1 :green_heart: | shadedjars | 6m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 31s | the patch passed | | +1 :green_heart: | compile | 1m 25s | the patch passed | | +1 :green_heart: | javac | 1m 25s | the patch passed | | +1 :green_heart: | shadedjars | 6m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 16s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 150m 56s | hbase-server in the patch passed. | | | | 181m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3417/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3417 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9e54743a6b0c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / c5a6e19be0 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3417/3/testReport/ | | Max. process+thread count | 4423 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3417/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] z-york commented on pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
z-york commented on pull request #90: URL: https://github.com/apache/hbase-operator-tools/pull/90#issuecomment-871669261 I guess the build wouldn't catch this, but it will ensure no regressions with older releases. This change is compatible with the older versions and 2.4.4. Build with 2.4.4: hbase-operator-tools % mvn clean install -DskipTests -Dhbase.version=2.4.4 `[INFO] [INFO] Reactor Summary for Apache HBase Operator Tools 1.2.0-SNAPSHOT: [INFO] [INFO] Apache HBase Operator Tools SUCCESS [ 10.389 s] [INFO] Apache HBase - Table Reporter .. SUCCESS [ 41.494 s] [INFO] Apache HBase - HBCK2 ... SUCCESS [ 52.119 s] [INFO] Apache HBase - HBase Tools . SUCCESS [ 12.464 s] [INFO] Apache HBase Operator Tools - Assembly . SUCCESS [ 0.423 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:58 min [INFO] Finished at: 2021-06-30T12:19:53-07:00 [INFO] ` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-operator-tools] z-york opened a new pull request #90: HBASE-26054 Fix hbase-operator-tools build with HBase 2.4.4
z-york opened a new pull request #90: URL: https://github.com/apache/hbase-operator-tools/pull/90 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
Apache-HBase commented on pull request #3439: URL: https://github.com/apache/hbase/pull/3439#issuecomment-871665953 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 17s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 0s | master passed | | +1 :green_heart: | compile | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 9m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 41s | the patch passed | | +1 :green_heart: | compile | 1m 19s | the patch passed | | +1 :green_heart: | javac | 1m 19s | the patch passed | | +1 :green_heart: | shadedjars | 8m 59s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 188m 11s | hbase-server in the patch passed. | | | | 222m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3439 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bc695f051ed1 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/2/testReport/ | | Max. process+thread count | 2295 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3441: HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade
Apache-HBase commented on pull request #3441: URL: https://github.com/apache/hbase/pull/3441#issuecomment-871664976 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 54s | master passed | | +1 :green_heart: | compile | 1m 25s | master passed | | +1 :green_heart: | shadedjars | 8m 17s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 42s | the patch passed | | +1 :green_heart: | compile | 1m 27s | the patch passed | | +1 :green_heart: | javac | 1m 27s | the patch passed | | +1 :green_heart: | shadedjars | 8m 13s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 17s | hbase-client in the patch passed. | | -1 :x: | unit | 130m 20s | hbase-server in the patch failed. | | | | 164m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3441/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3441 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2b3d802c5ffd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3441/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3441/2/testReport/ | | Max. process+thread count | 3821 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3441/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26054) Fix hbase-operator-tools build with HBase 2.4.4
Zach York created HBASE-26054: - Summary: Fix hbase-operator-tools build with HBase 2.4.4 Key: HBASE-26054 URL: https://issues.apache.org/jira/browse/HBASE-26054 Project: HBase Issue Type: Bug Components: hbase-operator-tools Affects Versions: hbase-operator-tools-1.2.0 Reporter: Zach York Assignee: Zach York I tried building hbase-operator-tools with HBase 2.4.4 and encountered a number of issues. Opening this for the fix. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3438: HBASE-22923 min version of RegionServer to move system table regions
Apache-HBase commented on pull request #3438: URL: https://github.com/apache/hbase/pull/3438#issuecomment-871646080 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 11m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 54s | branch-1 passed | | +1 :green_heart: | compile | 0m 42s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 45s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 1m 55s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 49s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 42s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 3m 16s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 12s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 2s | the patch passed | | +1 :green_heart: | compile | 0m 42s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 42s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 47s | the patch passed | | -1 :x: | checkstyle | 1m 48s | hbase-server: The patch generated 4 new + 231 unchanged - 0 fixed = 235 total (was 231) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 0s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 43s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 3m 9s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 155m 4s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 210m 58s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3438/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3438 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux eddc05d43a0b 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-3438/out/precommit/personality/provided.sh | | git revision | branch-1 / 28f36f4 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3438/2/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3438/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3438/2/testReport/ | | Max. process+thread count | 5023 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3438/2/console | | versions |
[GitHub] [hbase] virajjasani commented on pull request #3446: HBASE-26051 Remove reflections used to access HDFS EC APIs
virajjasani commented on pull request #3446: URL: https://github.com/apache/hbase/pull/3446#issuecomment-871641686 > +1. For master branch only. Ah, looks like I missed this earlier. Sounds good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
Apache-HBase commented on pull request #3439: URL: https://github.com/apache/hbase/pull/3439#issuecomment-871628605 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 1m 6s | master passed | | +1 :green_heart: | shadedjars | 8m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | +1 :green_heart: | compile | 1m 2s | the patch passed | | +1 :green_heart: | javac | 1m 2s | the patch passed | | +1 :green_heart: | shadedjars | 8m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 131m 12s | hbase-server in the patch passed. | | | | 162m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3439 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 39111479fa56 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/2/testReport/ | | Max. process+thread count | 3567 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26029) It is not reliable to use nodeDeleted event to track region server's death
[
https://issues.apache.org/jira/browse/HBASE-26029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372142#comment-17372142
]
Hudson commented on HBASE-26029:
Results for branch branch-2
[build #289 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289//console].
> It is not reliable to use nodeDeleted event to track region server's death
> --
>
> Key: HBASE-26029
> URL: https://issues.apache.org/jira/browse/HBASE-26029
> Project: HBase
> Issue Type: Bug
> Components: Replication, Zookeeper
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.5.0
>
>
> When implementing HBASE-26011, [~sunxin] pointed out an interesting scenario,
> where a region server up and down between two sync requests, then we can not
> know the death of the region server.
> https://github.com/apache/hbase/pull/3405#discussion_r656720923
> This is a valid point, and when thinking of a solution, I noticed that, the
> current zk iplementation has the same problem. Notice that, a watcher on zk
> can only be triggered once, so after zk triggers the watcher, and before you
> set a new watcher, it is possible that a region server is up and down, and
> you will miss the nodeDeleted event for this region server.
> I think, the general approach here, which could works for both master based
> and zk based replication tracker is that, we should not rely on the tracker
> to tell you which region server is dead. Instead, we just provide the list of
> live regionservers, and the upper layer should compare this list with the
> expected list(for replication, the list should be gotten by listing
> replicators), to detect the dead region servers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23956) Use less resources running tests
[
https://issues.apache.org/jira/browse/HBASE-23956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372141#comment-17372141
]
Hudson commented on HBASE-23956:
Results for branch branch-2
[build #289 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289//console].
> Use less resources running tests
>
>
> Key: HBASE-23956
> URL: https://issues.apache.org/jira/browse/HBASE-23956
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments: Screen Shot 2020-03-09 at 10.11.26 PM.png, Screen Shot
> 2020-03-09 at 10.11.35 PM.png, Screen Shot 2020-03-09 at 10.20.08 PM.png,
> Screen Shot 2020-03-10 at 7.10.15 AM.png, Screen Shot 2020-03-10 at 7.10.28
> AM.png
>
>
> Our tests can create thousands of threads all up in the one JVM. Using less
> means less memory, less contention, likelier passes, and later, more possible
> parallelism.
> I've been studying the likes of TestNamespaceReplicationWithBulkLoadedData to
> see what it does as it runs (this test puts up 4 clusters with replication
> between). It peaks at 2k threads. After some configuration and using less
> HDFS, its possible to get it down to ~800 threads and about 1/2 the
> memory-used. HDFS is a main offender. DataXceivers (Server and Client), jetty
> threads, Volume threads (async disk 'worker' then another for cleanup...),
> image savers, ipc clients -- new thread per incoming connection w/o bound (or
> reuse), block responder threads, anonymous threads, and so on. Many are not
> configurable or boundable or are hard-coded; e.g. each volume gets 4 workers
> regardless. Biggest impact was just downing the count of data nodes. TODO: a
> follow-on that turns down DN counts in all tests.
> I've been using Java Flight Recorder during this study. Here is how you get a
> flight recorder for the a single test run: \{code:java} MAVEN_OPTS="
> -XX:StartFlightRecording=disk=true,dumponexit=true,filename=recording.jfr,settings=profile,path-to-gc-roots=true,maxsize=1024m"
> mvn test -Dtest=TestNamespaceReplicationWithBulkLoadedData
> -Dsurefire.firstPartForkCount=0 -Dsurefire.secondPartForkCount=0 \{code} i.e.
> start recording on mvn launch, bound the size of the recording, and have the
> test run in the mvn context (DON'T fork). Useful is connecting to the running
> test at the same time from JDK Mission Control. We do the latter because the
> thread reporting screen is overwhelmed by the count of running threads and if
> you connect live, you can at least get a 'live threads' graph w/ count as the
> test progresses. Useful. When the test finishes, it dumps a .jfr file which
> can be opened in JDK MC.
> I've been compiling w/ JDK8 and then running w/ JDK11 so I can use JDK MC
> Version 7, the non-commercial latest. Works pretty well. Let me put up a
> patch for tests that cuts down thread counts where we can.
> Let me put up a patch that does first pass on curtailing resource usage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26039) TestReplicationKillRS is useless after HBASE-23956
[
https://issues.apache.org/jira/browse/HBASE-26039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372140#comment-17372140
]
Hudson commented on HBASE-26039:
Results for branch branch-2
[build #289 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/289//console].
> TestReplicationKillRS is useless after HBASE-23956
> --
>
> Key: HBASE-26039
> URL: https://issues.apache.org/jira/browse/HBASE-26039
> Project: HBase
> Issue Type: Bug
> Components: Replication, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> In TestReplicationKillRS, we assume there are at least 2 regionservers but in
> HBASE-23956, we set the region server number to 1.
> So in fact, we do not kill any region servers which makes the tests useless...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] bharathv commented on a change in pull request #3438: HBASE-22923 min version of RegionServer to move system table regions
bharathv commented on a change in pull request #3438:
URL: https://github.com/apache/hbase/pull/3438#discussion_r661697488
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##
@@ -2551,19 +2566,44 @@ private int setOfflineInZooKeeper(final RegionState
state, final ServerName dest
* know the version. So in fact we will never assign a system region to a RS
without registering on zk.
*/
public List getExcludedServersForSystemTable() {
+return getExcludedServersForSystemTable(false);
+ }
+
+ /**
+ * Get a list of servers that this region can not assign to.
+ * For system table, we must assign them to a server with highest version.
+ * We can disable this exclusion using config:
+ * "hbase.min.version.move.system.tables" if checkForMinVersion is true.
+ *
+ * @param checkForMinVersion if true, check for minVersionToMoveSysTables
+ * and decide moving system table regions accordingly.
+ * @return List of Excluded servers for System table regions.
+ */
+ private List getExcludedServersForSystemTable(
+ boolean checkForMinVersion) {
List> serverList = new ArrayList<>();
for (ServerName s : serverManager.getOnlineServersList()) {
serverList.add(new Pair<>(s, server.getRegionServerVersion(s)));
}
if (serverList.isEmpty()) {
- return new ArrayList<>();
+ return Collections.emptyList();
}
-String highestVersion = Collections.max(serverList, new
Comparator>() {
+String highestVersion = Collections.max(serverList,
+new Comparator>() {
@Override
public int compare(Pair o1, Pair
o2) {
return VersionInfo.compareVersion(o1.getSecond(), o2.getSecond());
}
}).getSecond();
+if (checkForMinVersion) {
+ if
(!DEFAULT_MIN_VERSION_MOVE_SYS_TABLES_CONFIG.equals(minVersionToMoveSysTables))
{
Review comment:
I don't have a strong opinion. If you ask me, I like the following
version but its subjective. I'm fine with whatever you think is good.
```
checkForMinVersion = checkForMinVersion &&
!DEFAULT_MIN_VERSION_MOVE_SYS_TABLES_CONFIG.equals(minVersionToMoveSysTables);
if (checkForMinVersion) {
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] bharathv commented on a change in pull request #3438: HBASE-22923 min version of RegionServer to move system table regions
bharathv commented on a change in pull request #3438:
URL: https://github.com/apache/hbase/pull/3438#discussion_r661697488
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##
@@ -2551,19 +2566,44 @@ private int setOfflineInZooKeeper(final RegionState
state, final ServerName dest
* know the version. So in fact we will never assign a system region to a RS
without registering on zk.
*/
public List getExcludedServersForSystemTable() {
+return getExcludedServersForSystemTable(false);
+ }
+
+ /**
+ * Get a list of servers that this region can not assign to.
+ * For system table, we must assign them to a server with highest version.
+ * We can disable this exclusion using config:
+ * "hbase.min.version.move.system.tables" if checkForMinVersion is true.
+ *
+ * @param checkForMinVersion if true, check for minVersionToMoveSysTables
+ * and decide moving system table regions accordingly.
+ * @return List of Excluded servers for System table regions.
+ */
+ private List getExcludedServersForSystemTable(
+ boolean checkForMinVersion) {
List> serverList = new ArrayList<>();
for (ServerName s : serverManager.getOnlineServersList()) {
serverList.add(new Pair<>(s, server.getRegionServerVersion(s)));
}
if (serverList.isEmpty()) {
- return new ArrayList<>();
+ return Collections.emptyList();
}
-String highestVersion = Collections.max(serverList, new
Comparator>() {
+String highestVersion = Collections.max(serverList,
+new Comparator>() {
@Override
public int compare(Pair o1, Pair
o2) {
return VersionInfo.compareVersion(o1.getSecond(), o2.getSecond());
}
}).getSecond();
+if (checkForMinVersion) {
+ if
(!DEFAULT_MIN_VERSION_MOVE_SYS_TABLES_CONFIG.equals(minVersionToMoveSysTables))
{
Review comment:
I don't have a strong opinion. If you ask me, I like the following
version but its subjective. I'm fine with whatever you think is good.
```
checkForMinVersion = checkForMinVersion &
!DEFAULT_MIN_VERSION_MOVE_SYS_TABLES_CONFIG.equals(minVersionToMoveSysTables);
if (checkForMinVersion) {
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
virajjasani commented on a change in pull request #3439:
URL: https://github.com/apache/hbase/pull/3439#discussion_r661696967
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -2280,24 +2295,48 @@ private void addToPendingAssignment(final
HashMap r
* For system tables, we must assign them to a server with highest version.
*/
public List getExcludedServersForSystemTable() {
Review comment:
Yes, this is used by HMaster and AM both.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
virajjasani commented on pull request #3439: URL: https://github.com/apache/hbase/pull/3439#issuecomment-871612756 All nice comments, I agree, doc can be better. On it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
saintstack commented on pull request #3439: URL: https://github.com/apache/hbase/pull/3439#issuecomment-871610467 My fault for not working on the PR where the action was. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
virajjasani commented on pull request #3439: URL: https://github.com/apache/hbase/pull/3439#issuecomment-871609617 Sorry @saintstack for opening master branch PR later. Here is where other reviews took place: #3438 But sure let me address these in both places. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
saintstack commented on a change in pull request #3439:
URL: https://github.com/apache/hbase/pull/3439#discussion_r661685304
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -163,6 +163,19 @@
private final RegionStates regionStates = new RegionStates();
private final RegionStateStore regionStateStore;
+ /**
+ * When the operator uses this configuration option, any version between
+ * the current version and the new value of
Review comment:
Is it 'new value' or just 'value'.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -163,6 +163,19 @@
private final RegionStates regionStates = new RegionStates();
private final RegionStateStore regionStateStore;
+ /**
+ * When the operator uses this configuration option, any version between
+ * the current version and the new value of
+ * "hbase.min.version.move.system.tables" does not trigger any region
movement.
+ * It is assumed that the configured range of versions do not require special
+ * handling of moving system table regions to higher versioned RegionServer.
Review comment:
s/do/does/ and s/of//. Perhaps point at the location in the code where
this is done.
Can we add examples of how this would work -- the ones from the JIRA?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -2280,24 +2295,48 @@ private void addToPendingAssignment(final
HashMap r
* For system tables, we must assign them to a server with highest version.
*/
public List getExcludedServersForSystemTable() {
+return getExcludedServersForSystemTable(false);
+ }
+
+ /**
+ * Get a list of servers that this region can not assign to.
+ * For system table, we must assign them to a server with highest version.
+ * We can disable this exclusion using config:
+ * "hbase.min.version.move.system.tables" if checkForMinVersion is true.
+ *
+ * @param checkForMinVersion if true, check for minVersionToMoveSysTables
+ * and decide moving system table regions accordingly.
Review comment:
This method doesn't do move? It just looks for servers to exclude
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -163,6 +163,19 @@
private final RegionStates regionStates = new RegionStates();
private final RegionStateStore regionStateStore;
+ /**
+ * When the operator uses this configuration option, any version between
+ * the current version and the new value of
+ * "hbase.min.version.move.system.tables" does not trigger any region
movement.
Review comment:
Do you want to explain what the 'region movement' is here? The
auto-migration of system tables to newer server versions. A dev reading this
w/o context might not tie the 'reigon movement' to the auto-movement to newer
versions.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -2280,24 +2295,48 @@ private void addToPendingAssignment(final
HashMap r
* For system tables, we must assign them to a server with highest version.
*/
public List getExcludedServersForSystemTable() {
Review comment:
Is this used?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -2280,24 +2295,48 @@ private void addToPendingAssignment(final
HashMap r
* For system tables, we must assign them to a server with highest version.
*/
public List getExcludedServersForSystemTable() {
+return getExcludedServersForSystemTable(false);
+ }
+
+ /**
+ * Get a list of servers that this region can not assign to.
Review comment:
There doesn't seem to be a 'region' passed to this method So is it
any system table region?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3438: HBASE-22923 min version of RegionServer to move system table regions
virajjasani commented on a change in pull request #3438:
URL: https://github.com/apache/hbase/pull/3438#discussion_r661689913
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##
@@ -2551,19 +2566,44 @@ private int setOfflineInZooKeeper(final RegionState
state, final ServerName dest
* know the version. So in fact we will never assign a system region to a RS
without registering on zk.
*/
public List getExcludedServersForSystemTable() {
+return getExcludedServersForSystemTable(false);
+ }
+
+ /**
+ * Get a list of servers that this region can not assign to.
+ * For system table, we must assign them to a server with highest version.
+ * We can disable this exclusion using config:
+ * "hbase.min.version.move.system.tables" if checkForMinVersion is true.
+ *
+ * @param checkForMinVersion if true, check for minVersionToMoveSysTables
+ * and decide moving system table regions accordingly.
+ * @return List of Excluded servers for System table regions.
+ */
+ private List getExcludedServersForSystemTable(
+ boolean checkForMinVersion) {
List> serverList = new ArrayList<>();
for (ServerName s : serverManager.getOnlineServersList()) {
serverList.add(new Pair<>(s, server.getRegionServerVersion(s)));
}
if (serverList.isEmpty()) {
- return new ArrayList<>();
+ return Collections.emptyList();
}
-String highestVersion = Collections.max(serverList, new
Comparator>() {
+String highestVersion = Collections.max(serverList,
+new Comparator>() {
@Override
public int compare(Pair o1, Pair
o2) {
return VersionInfo.compareVersion(o1.getSecond(), o2.getSecond());
}
}).getSecond();
+if (checkForMinVersion) {
+ if
(!DEFAULT_MIN_VERSION_MOVE_SYS_TABLES_CONFIG.equals(minVersionToMoveSysTables))
{
Review comment:
Since the argument `checkForMinVersion` controls this specific flow, I
thought keeping it this way would make it more readable. Thought?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3441: HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade
Apache-HBase commented on pull request #3441: URL: https://github.com/apache/hbase/pull/3441#issuecomment-871606348 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 24s | master passed | | +1 :green_heart: | compile | 5m 9s | master passed | | +1 :green_heart: | checkstyle | 1m 58s | master passed | | +1 :green_heart: | spotbugs | 4m 9s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 52s | the patch passed | | +1 :green_heart: | compile | 5m 14s | the patch passed | | +1 :green_heart: | javac | 5m 14s | the patch passed | | +1 :green_heart: | checkstyle | 2m 15s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 24m 9s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 27s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 70m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3441/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3441 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux a482d9ea7315 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3441/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bharathv commented on a change in pull request #3438: HBASE-22923 min version of RegionServer to move system table regions
bharathv commented on a change in pull request #3438:
URL: https://github.com/apache/hbase/pull/3438#discussion_r661684889
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
##
@@ -2551,19 +2566,44 @@ private int setOfflineInZooKeeper(final RegionState
state, final ServerName dest
* know the version. So in fact we will never assign a system region to a RS
without registering on zk.
*/
public List getExcludedServersForSystemTable() {
+return getExcludedServersForSystemTable(false);
+ }
+
+ /**
+ * Get a list of servers that this region can not assign to.
+ * For system table, we must assign them to a server with highest version.
+ * We can disable this exclusion using config:
+ * "hbase.min.version.move.system.tables" if checkForMinVersion is true.
+ *
+ * @param checkForMinVersion if true, check for minVersionToMoveSysTables
+ * and decide moving system table regions accordingly.
+ * @return List of Excluded servers for System table regions.
+ */
+ private List getExcludedServersForSystemTable(
+ boolean checkForMinVersion) {
List> serverList = new ArrayList<>();
for (ServerName s : serverManager.getOnlineServersList()) {
serverList.add(new Pair<>(s, server.getRegionServerVersion(s)));
}
if (serverList.isEmpty()) {
- return new ArrayList<>();
+ return Collections.emptyList();
}
-String highestVersion = Collections.max(serverList, new
Comparator>() {
+String highestVersion = Collections.max(serverList,
+new Comparator>() {
@Override
public int compare(Pair o1, Pair
o2) {
return VersionInfo.compareVersion(o1.getSecond(), o2.getSecond());
}
}).getSecond();
+if (checkForMinVersion) {
+ if
(!DEFAULT_MIN_VERSION_MOVE_SYS_TABLES_CONFIG.equals(minVersionToMoveSysTables))
{
Review comment:
super nit: club the conditions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-23956) Use less resources running tests
[
https://issues.apache.org/jira/browse/HBASE-23956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372130#comment-17372130
]
Hudson commented on HBASE-23956:
Results for branch branch-2.3
[build #247 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/247/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/247/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/247/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/247/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/247/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use less resources running tests
>
>
> Key: HBASE-23956
> URL: https://issues.apache.org/jira/browse/HBASE-23956
> Project: HBase
> Issue Type: Test
> Components: test
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments: Screen Shot 2020-03-09 at 10.11.26 PM.png, Screen Shot
> 2020-03-09 at 10.11.35 PM.png, Screen Shot 2020-03-09 at 10.20.08 PM.png,
> Screen Shot 2020-03-10 at 7.10.15 AM.png, Screen Shot 2020-03-10 at 7.10.28
> AM.png
>
>
> Our tests can create thousands of threads all up in the one JVM. Using less
> means less memory, less contention, likelier passes, and later, more possible
> parallelism.
> I've been studying the likes of TestNamespaceReplicationWithBulkLoadedData to
> see what it does as it runs (this test puts up 4 clusters with replication
> between). It peaks at 2k threads. After some configuration and using less
> HDFS, its possible to get it down to ~800 threads and about 1/2 the
> memory-used. HDFS is a main offender. DataXceivers (Server and Client), jetty
> threads, Volume threads (async disk 'worker' then another for cleanup...),
> image savers, ipc clients -- new thread per incoming connection w/o bound (or
> reuse), block responder threads, anonymous threads, and so on. Many are not
> configurable or boundable or are hard-coded; e.g. each volume gets 4 workers
> regardless. Biggest impact was just downing the count of data nodes. TODO: a
> follow-on that turns down DN counts in all tests.
> I've been using Java Flight Recorder during this study. Here is how you get a
> flight recorder for the a single test run: \{code:java} MAVEN_OPTS="
> -XX:StartFlightRecording=disk=true,dumponexit=true,filename=recording.jfr,settings=profile,path-to-gc-roots=true,maxsize=1024m"
> mvn test -Dtest=TestNamespaceReplicationWithBulkLoadedData
> -Dsurefire.firstPartForkCount=0 -Dsurefire.secondPartForkCount=0 \{code} i.e.
> start recording on mvn launch, bound the size of the recording, and have the
> test run in the mvn context (DON'T fork). Useful is connecting to the running
> test at the same time from JDK Mission Control. We do the latter because the
> thread reporting screen is overwhelmed by the count of running threads and if
> you connect live, you can at least get a 'live threads' graph w/ count as the
> test progresses. Useful. When the test finishes, it dumps a .jfr file which
> can be opened in JDK MC.
> I've been compiling w/ JDK8 and then running w/ JDK11 so I can use JDK MC
> Version 7, the non-commercial latest. Works pretty well. Let me put up a
> patch for tests that cuts down thread counts where we can.
> Let me put up a patch that does first pass on curtailing resource usage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26039) TestReplicationKillRS is useless after HBASE-23956
[
https://issues.apache.org/jira/browse/HBASE-26039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372129#comment-17372129
]
Hudson commented on HBASE-26039:
Results for branch branch-2.3
[build #247 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/247/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/247/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/247/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/247/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/247/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestReplicationKillRS is useless after HBASE-23956
> --
>
> Key: HBASE-26039
> URL: https://issues.apache.org/jira/browse/HBASE-26039
> Project: HBase
> Issue Type: Bug
> Components: Replication, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.5
>
>
> In TestReplicationKillRS, we assume there are at least 2 regionservers but in
> HBASE-23956, we set the region server number to 1.
> So in fact, we do not kill any region servers which makes the tests useless...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3417: HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade
Apache-HBase commented on pull request #3417: URL: https://github.com/apache/hbase/pull/3417#issuecomment-871589733 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.4 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 31s | branch-2.4 passed | | +1 :green_heart: | compile | 4m 10s | branch-2.4 passed | | +1 :green_heart: | checkstyle | 1m 40s | branch-2.4 passed | | +1 :green_heart: | spotbugs | 3m 12s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 16s | the patch passed | | +1 :green_heart: | compile | 4m 16s | the patch passed | | +1 :green_heart: | javac | 4m 16s | the patch passed | | +1 :green_heart: | checkstyle | 1m 42s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 16m 58s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 34s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 51m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3417/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3417 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 4ba9f41e7214 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / c5a6e19be0 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 95 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3417/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #3417: HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade
saintstack commented on a change in pull request #3417:
URL: https://github.com/apache/hbase/pull/3417#discussion_r661668806
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -953,9 +956,26 @@ private void
finishActiveMasterInitialization(MonitoredTask status)
if (!waitForMetaOnline()) {
return;
}
+TableDescriptor metaDescriptor = tableDescriptors.get(
+TableName.META_TABLE_NAME);
+final ColumnFamilyDescriptor tableFamilyDesc = metaDescriptor
+.getColumnFamily(HConstants.TABLE_FAMILY);
+final ColumnFamilyDescriptor replBarrierFamilyDesc =
+metaDescriptor.getColumnFamily(HConstants.REPLICATION_BARRIER_FAMILY);
+
this.assignmentManager.joinCluster();
// The below depends on hbase:meta being online.
-this.tableStateManager.start();
+try {
+ this.tableStateManager.start();
+} catch (NoSuchColumnFamilyException e) {
+ if (tableFamilyDesc == null && replBarrierFamilyDesc == null) {
Review comment:
Ok. Good. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25520) Exclude CHANGES and RELEASENOTES files from source control
[
https://issues.apache.org/jira/browse/HBASE-25520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372118#comment-17372118
]
Michael Stack commented on HBASE-25520:
---
{quote}For the place to hold CHANGES and RELEASENOTES, maybe we could just use
JIRA's release note page?
{quote}
We used to do it this way in the old days.
> Exclude CHANGES and RELEASENOTES files from source control
> --
>
> Key: HBASE-25520
> URL: https://issues.apache.org/jira/browse/HBASE-25520
> Project: HBase
> Issue Type: Task
> Components: build, community
>Reporter: Nick Dimiduk
>Priority: Major
>
> Over on the thread [Project management (JIRA fix version tracking) is in
> crisis
> |https://lists.apache.org/thread.html/r8db075fd974de32a90174be2db106ef710cf38cfde9ea9071fda4152%40%3Cdev.hbase.apache.org%3E],
> one of the suggested improvements to our process is to remove the CHANGES
> and RELEASENOTES files from version control. To accomplish this, we'll need to
> * delete existing files from existing branches
> * modify the release automation to not commit this files
> * find someplace to deliver these files on the web
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3446: HBASE-26051 Remove reflections used to access HDFS EC APIs
Apache-HBase commented on pull request #3446: URL: https://github.com/apache/hbase/pull/3446#issuecomment-871564249 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 0s | master passed | | +1 :green_heart: | compile | 1m 21s | master passed | | +1 :green_heart: | shadedjars | 9m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 1m 21s | the patch passed | | +1 :green_heart: | javac | 1m 21s | the patch passed | | +1 :green_heart: | shadedjars | 9m 1s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 41s | hbase-asyncfs in the patch passed. | | +1 :green_heart: | unit | 192m 41s | hbase-server in the patch passed. | | | | 228m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3446/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3446 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 18d7848ce743 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3446/2/testReport/ | | Max. process+thread count | 2414 (vs. ulimit of 3) | | modules | C: hbase-asyncfs hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3446/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3439: HBASE-22923 min version of RegionServer to move system table regions
Apache-HBase commented on pull request #3439: URL: https://github.com/apache/hbase/pull/3439#issuecomment-871562622 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 40s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 15s | master passed | | +1 :green_heart: | compile | 4m 5s | master passed | | +1 :green_heart: | checkstyle | 1m 32s | master passed | | +1 :green_heart: | spotbugs | 2m 50s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 53s | the patch passed | | +1 :green_heart: | compile | 3m 51s | the patch passed | | +1 :green_heart: | javac | 3m 51s | the patch passed | | +1 :green_heart: | checkstyle | 1m 15s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 23m 38s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 49s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 61m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3439 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux eec38fb4a50c 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3439/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-26042) WAL lockup on 'sync failed' org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372109#comment-17372109
]
Michael Stack edited comment on HBASE-26042 at 6/30/21, 4:38 PM:
-
[~apurtell] My attempt at repro did not pan out unfortunately.
[~bharathv] attached
was (Author: stack):
[~apurtell] My attempt at repro did not pan out unfortunately.
[~bharathv] coming...
> WAL lockup on 'sync failed'
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer
>
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5
>Reporter: Michael Stack
>Priority: Major
> Attachments: js1, js2
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
[jira] [Updated] (HBASE-26042) WAL lockup on 'sync failed' org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-26042:
--
Attachment: js2
js1
> WAL lockup on 'sync failed'
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer
>
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5
>Reporter: Michael Stack
>Priority: Major
> Attachments: js1, js2
>
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next(int) @bci=82, line=136
> (Compiled frame)
> -
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed' org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372109#comment-17372109
]
Michael Stack commented on HBASE-26042:
---
[~apurtell] My attempt at repro did not pan out unfortunately.
[~bharathv] coming...
> WAL lockup on 'sync failed'
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer
>
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5
>Reporter: Michael Stack
>Priority: Major
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append the WAL w/ flush markers etc.
> unable to add the ringbuffer:
>
> {code:java}
> Thread 9465: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338
> (Compiled frame)
> - com.lmax.disruptor.MultiProducerSequencer.next(int) @bci=82, line=136
> (Compiled frame)
[jira] [Commented] (HBASE-26042) WAL lockup on 'sync failed' org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
[
https://issues.apache.org/jira/browse/HBASE-26042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372108#comment-17372108
]
Andrew Kyle Purtell commented on HBASE-26042:
-
{quote}2021-06-29 20:33:35,183 WARN
[AsyncFSWAL-0-hdfs://localhost.localdomain:37680/user/mstack/test-data/329a3111-d35d-18ae-9107-065eee2a4e62]
wal.AsyncFSWAL(299): sync
failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
Injected(..) failed: Connection reset by peer{quote}
So this can be simulated by injecting an IOE? Is it possible to provide a unit
test that repros the issue? Then we can investigate and fix in any dev
environment...
> WAL lockup on 'sync failed'
> org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer
>
>
> Key: HBASE-26042
> URL: https://issues.apache.org/jira/browse/HBASE-26042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.3.5
>Reporter: Michael Stack
>Priority: Major
>
> Making note of issue seen in production cluster.
> Node had been struggling under load for a few days with slow syncs up to 10
> seconds, a few STUCK MVCCs from which it recovered and some java pauses up to
> three seconds in length.
> Then the below happened:
> {code:java}
> 2021-06-27 13:41:27,604 WARN [AsyncFSWAL-0-hdfs://:8020/hbase]
> wal.AsyncFSWAL: sync
> failedorg.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException:
> readAddress(..) failed: Connection reset by peer {code}
> ... and WAL turned dead in the water. Scanners start expiring. RPC prints
> text versions of requests complaining requestsTooSlow. Then we start to see
> these:
> {code:java}
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Failed to get sync
> result after 30 ms for txid=552128301, WAL system stuck? {code}
> Whats supposed to happen when other side goes away like this is that we will
> roll the WAL – go set up a new one. You can see it happening if you run
> {code:java}
> mvn test
> -Dtest=org.apache.hadoop.hbase.regionserver.wal.TestAsyncFSWAL#testBrokenWriter
> {code}
> I tried hacking the test to repro the above hang by throwing same exception
> in above test (on linux because need epoll to repro) but all just worked.
> Thread dumps of the hungup WAL subsystem are a little odd. The log roller is
> stuck w/o timeout trying to write a long on the WAL header:
>
> {code:java}
> Thread 9464: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information
> may be imprecise)
> - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14,
> line=175 (Compiled frame)
> - java.util.concurrent.CompletableFuture$Signaller.block() @bci=19,
> line=1707 (Compiled frame)
> -
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> @bci=119, line=3323 (Compiled frame)
> - java.util.concurrent.CompletableFuture.waitingGet(boolean) @bci=115,
> line=1742 (Compiled frame)
> - java.util.concurrent.CompletableFuture.get() @bci=11, line=1908 (Compiled
> frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.write(java.util.function.Consumer)
> @bci=16, line=189 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncProtobufLogWriter.writeMagicAndWALHeader(byte[],
> org.apache.hadoop.hbase.shaded.protobuf.generated.WALProtos$WALHeader)
> @bci=9, line=202 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AbstractProtobufLogWriter.init(org.apache.hadoop.fs.FileSystem,
> org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration, boolean,
> long) @bci=107, line=170 (Compiled frame)
> -
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(org.apache.hadoop.conf.Configuration,
> org.apache.hadoop.fs.FileSystem, org.apache.hadoop.fs.Path, boolean, long,
> org.apache.hbase.thirdparty.io.netty.channel.EventLoopGroup, java.lang.Class)
> @bci=61, line=113 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=22, line=651 (Compiled frame)
> -
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(org.apache.hadoop.fs.Path)
> @bci=2, line=128 (Compiled frame)
> - org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(boolean)
> @bci=101, line=797 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(long)
> @bci=18, line=263 (Compiled frame)
> - org.apache.hadoop.hbase.wal.AbstractWALRoller.run() @bci=198, line=179
> (Compiled frame) {code}
>
> Other threads are BLOCKED trying to append
[GitHub] [hbase] Apache-HBase commented on pull request #3444: HBASE-26049 Remove DfsBuilderUtility
Apache-HBase commented on pull request #3444: URL: https://github.com/apache/hbase/pull/3444#issuecomment-871555797 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 47s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 6s | master passed | | +1 :green_heart: | compile | 4m 39s | master passed | | +1 :green_heart: | checkstyle | 1m 48s | master passed | | +1 :green_heart: | spotbugs | 3m 25s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 1s | the patch passed | | +1 :green_heart: | compile | 4m 33s | the patch passed | | +1 :green_heart: | javac | 4m 33s | the patch passed | | -0 :warning: | checkstyle | 0m 23s | hbase-common: The patch generated 3 new + 1 unchanged - 2 fixed = 4 total (was 3) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | hadoopcheck | 20m 4s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 61m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3444 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile xml | | uname | Linux 3f7fd030e0f4 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-common hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3417: HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade
virajjasani commented on a change in pull request #3417:
URL: https://github.com/apache/hbase/pull/3417#discussion_r661632460
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
##
@@ -953,9 +956,26 @@ private void
finishActiveMasterInitialization(MonitoredTask status)
if (!waitForMetaOnline()) {
return;
}
+TableDescriptor metaDescriptor = tableDescriptors.get(
+TableName.META_TABLE_NAME);
+final ColumnFamilyDescriptor tableFamilyDesc = metaDescriptor
+.getColumnFamily(HConstants.TABLE_FAMILY);
+final ColumnFamilyDescriptor replBarrierFamilyDesc =
+metaDescriptor.getColumnFamily(HConstants.REPLICATION_BARRIER_FAMILY);
+
this.assignmentManager.joinCluster();
// The below depends on hbase:meta being online.
-this.tableStateManager.start();
+try {
+ this.tableStateManager.start();
+} catch (NoSuchColumnFamilyException e) {
+ if (tableFamilyDesc == null && replBarrierFamilyDesc == null) {
Review comment:
> Is there a case that one might be in place but not the other -- I
don't know-- and does the code do right thing(I've not checked)
Exactly, the case where either 'table' or 'repl_barrier' is missing might be
weird case and should not happen given that HBase 2 upgrades are not going to
face missing CF issues. That's why I kept `&&` here instead of `||` to make
sure we are specifically handling HBase 1 to 2.3+ upgrade case.
FYI @anoopsjohn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on pull request #3446: HBASE-26051 Remove reflections used to access HDFS EC APIs
virajjasani commented on pull request #3446: URL: https://github.com/apache/hbase/pull/3446#issuecomment-871544093 Raised question on the parent Jira. For all these nice changes, we will restrict to trunk only right? Because branch-2 still supports Hadoop 2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26040) Replace reflections that are redundant
[ https://issues.apache.org/jira/browse/HBASE-26040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372101#comment-17372101 ] Viraj Jasani commented on HBASE-26040: -- If the intention is to remove redundant reflections based on Hadoop 3+ support, then I believe we should just merge these changes in trunk and not on branch-2 because branch-2 still supports Hadoop 2? > Replace reflections that are redundant > -- > > Key: HBASE-26040 > URL: https://issues.apache.org/jira/browse/HBASE-26040 > Project: HBase > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > > A number of reflections were used to access (back in the time) new Hadoop > APIs that were only available in newer Hadoop versions. > Some of them are no longer needed with the default Hadoop dependency 3.1.2, > so they can be removed to avoid the brittle code. Also, makes it possible to > determine compile time dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3444: HBASE-26049 Remove DfsBuilderUtility
Apache-HBase commented on pull request #3444: URL: https://github.com/apache/hbase/pull/3444#issuecomment-871540807 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 24s | master passed | | +1 :green_heart: | compile | 2m 2s | master passed | | +1 :green_heart: | shadedjars | 8m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 25s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 13s | the patch passed | | +1 :green_heart: | compile | 1m 59s | the patch passed | | +1 :green_heart: | javac | 1m 59s | the patch passed | | +1 :green_heart: | shadedjars | 8m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 8s | hbase-common in the patch passed. | | -1 :x: | unit | 0m 41s | hbase-procedure in the patch failed. | | -1 :x: | unit | 8m 36s | hbase-server in the patch failed. | | | | 46m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3444 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a794e7345f3e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-procedure.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/testReport/ | | Max. process+thread count | 925 (vs. ulimit of 3) | | modules | C: hbase-common hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3444: HBASE-26049 Remove DfsBuilderUtility
Apache-HBase commented on pull request #3444: URL: https://github.com/apache/hbase/pull/3444#issuecomment-871537925 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 39s | master passed | | +1 :green_heart: | compile | 1m 47s | master passed | | +1 :green_heart: | shadedjars | 8m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 16s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 40s | the patch passed | | +1 :green_heart: | compile | 1m 45s | the patch passed | | +1 :green_heart: | javac | 1m 45s | the patch passed | | +1 :green_heart: | shadedjars | 8m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 46s | hbase-common in the patch passed. | | -1 :x: | unit | 0m 37s | hbase-procedure in the patch failed. | | -1 :x: | unit | 7m 31s | hbase-server in the patch failed. | | | | 42m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3444 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c9e1783e6a51 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 147b030c1f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-procedure.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/testReport/ | | Max. process+thread count | 889 (vs. ulimit of 3) | | modules | C: hbase-common hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3444/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25520) Exclude CHANGES and RELEASENOTES files from source control
[ https://issues.apache.org/jira/browse/HBASE-25520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372091#comment-17372091 ] Duo Zhang commented on HBASE-25520: --- For the place to hold CHANGES and RELEASENOTES, maybe we could just use JIRA's release note page? Like this: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753=12332342 > Exclude CHANGES and RELEASENOTES files from source control > -- > > Key: HBASE-25520 > URL: https://issues.apache.org/jira/browse/HBASE-25520 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Nick Dimiduk >Priority: Major > > Over on the thread [Project management (JIRA fix version tracking) is in > crisis > |https://lists.apache.org/thread.html/r8db075fd974de32a90174be2db106ef710cf38cfde9ea9071fda4152%40%3Cdev.hbase.apache.org%3E], > one of the suggested improvements to our process is to remove the CHANGES > and RELEASENOTES files from version control. To accomplish this, we'll need to > * delete existing files from existing branches > * modify the release automation to not commit this files > * find someplace to deliver these files on the web -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26053) Remove CHANGES and RELEASENOTES
Duo Zhang created HBASE-26053: - Summary: Remove CHANGES and RELEASENOTES Key: HBASE-26053 URL: https://issues.apache.org/jira/browse/HBASE-26053 Project: HBase Issue Type: Sub-task Components: build Reporter: Duo Zhang Fix For: 3.0.0-alpha-1 Let's do this first. The release script has a flag to skip the generating stage so it could be done later. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-7767) Get rid of ZKTable, and table enable/disable state in ZK
[
https://issues.apache.org/jira/browse/HBASE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372086#comment-17372086
]
Hudson commented on HBASE-7767:
---
Results for branch branch-1
[build #141 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/141/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/141//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/141//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/141//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Get rid of ZKTable, and table enable/disable state in ZK
> -
>
> Key: HBASE-7767
> URL: https://issues.apache.org/jira/browse/HBASE-7767
> Project: HBase
> Issue Type: Sub-task
> Components: Zookeeper
>Affects Versions: 2.0.0
>Reporter: Enis Soztutar
>Assignee: Andrey Stepachev
>Priority: Major
> Fix For: 2.0.0, 1.7.0
>
> Attachments:
> 0001-HBASE-7767-Get-rid-of-ZKTable-and-table-enable-disab.patch,
> 0001-rebase.patch, 7767v2.txt, HBASE-7767.patch, HBASE-7767.patch,
> HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch,
> HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch,
> HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch
>
>
> As discussed table state in zookeeper for enable/disable state breaks our
> zookeeper contract. It is also very intrusive, used from the client side,
> master and region servers. We should get rid of it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26021) HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
[
https://issues.apache.org/jira/browse/HBASE-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372085#comment-17372085
]
Hudson commented on HBASE-26021:
Results for branch branch-1
[build #141 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/141/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/141//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/141//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/141//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> HBase 1.7 to 2.4 upgrade issue due to incompatible deserialization
> --
>
> Key: HBASE-26021
> URL: https://issues.apache.org/jira/browse/HBASE-26021
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.7.0, 2.4.4
>Reporter: Viraj Jasani
>Assignee: Bharath Vissapragada
>Priority: Major
> Labels: compatibility, incompatibility, serialization, upgrade
> Fix For: 1.7.1
>
> Attachments: Screenshot 2021-06-22 at 12.54.21 PM.png, Screenshot
> 2021-06-22 at 12.54.30 PM.png
>
>
> As of today, if we bring up HBase cluster using branch-1 and upgrade to
> branch-2.4, we are facing issue while parsing namespace from HDFS fileinfo.
> Instead of "*hbase:meta*" and "*hbase:namespace*", parsing using ProtobufUtil
> seems to be producing "*\n hbase meta*" and "*\n hbase namespace*"
> {code:java}
> 2021-06-22 00:05:56,611 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> regionserver.RSRpcServices: Open hbase:meta,,1.1588230740
> 2021-06-22 00:05:56,648 INFO
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> regionserver.RSRpcServices: Open
> hbase:namespace,,1624297762817.396cb6cc00cd4334cb1ea3a792d7529a.
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=5,queue=1,port=16025]
> ipc.RpcServer: Unexpected throwable object
> java.lang.IllegalArgumentException: Illegal character <
> > at 0. Namespaces may only contain 'alphanumeric characters' from any
> > language and digits:
> ^Ehbase^R namespace
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:246)
> at
> org.apache.hadoop.hbase.TableName.isLegalNamespaceName(TableName.java:220)
> at org.apache.hadoop.hbase.TableName.(TableName.java:348)
> at
> org.apache.hadoop.hbase.TableName.createTableNameIfNecessary(TableName.java:385)
> at org.apache.hadoop.hbase.TableName.valueOf(TableName.java:508)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableName(ProtobufUtil.java:2292)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toTableDescriptor(ProtobufUtil.java:2937)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.parseFrom(TableDescriptorBuilder.java:1625)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder$ModifyableTableDescriptor.access$200(TableDescriptorBuilder.java:597)
> at
> org.apache.hadoop.hbase.client.TableDescriptorBuilder.parseFrom(TableDescriptorBuilder.java:320)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.readTableDescriptor(FSTableDescriptors.java:511)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:496)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:482)
> at
> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:210)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.openRegion(RSRpcServices.java:2112)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:35218)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:395)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> 2021-06-22 00:05:56,759 ERROR
> [RpcServer.priority.RWQ.Fifo.read.handler=3,queue=1,port=16025]
> ipc.RpcServer:
