[GitHub] [hbase] Apache-HBase commented on pull request #3870: HBASE-26479: Print too slow/big scan's operation_id in region server log
Apache-HBase commented on pull request #3870: URL: https://github.com/apache/hbase/pull/3870#issuecomment-976235081 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 3s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 9m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 17s | the patch passed | | +1 :green_heart: | compile | 1m 3s | the patch passed | | +1 :green_heart: | javac | 1m 3s | the patch passed | | +1 :green_heart: | shadedjars | 9m 0s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 239m 35s | hbase-server in the patch passed. | | | | 278m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3870 | | JIRA Issue | HBASE-26479 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b5d7c739b6ba 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b870b6d9ac | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/testReport/ | | Max. process+thread count | 2398 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26482) HMaster may clean wals that is replicating in rare cases
[
https://issues.apache.org/jira/browse/HBASE-26482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447819#comment-17447819
]
Duo Zhang commented on HBASE-26482:
---
I think this is definately a problem...
> HMaster may clean wals that is replicating in rare cases
>
>
> Key: HBASE-26482
> URL: https://issues.apache.org/jira/browse/HBASE-26482
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: zhuobin zheng
>Priority: Critical
>
> In our cluster, i can found some FileNotFoundException when
> ReplicationSourceWALReader running for replication recovery queue.
> I guss the wal most likely removed by hmaste. And i found something to
> support it.
> The method getAllWALs:
> [https://github.com/apache/hbase/blob/master/hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ZKReplicationQueueStorage.java#L509
>
> |https://github.com/apache/hbase/blob/master/hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ZKReplicationQueueStorage.java#L509]Use
> zk cversion of /hbase/replication/rs as an optimistic lock to control
> concurrent ops.
> But, zk cversion *only can only reflect the changes of child nodes, but not
> the changes of grandchildren.*
> So, HMaster may loss some wal from this method in follow situation.
> # HMaster do log clean , and invoke getAllWALs to filter log which should
> not be deleted.
> # HMaster cache current cversion of /hbase/replication/rs as *v0*
> # HMaster cache all RS server name, and traverse them, get the WAL in each
> Queue
> # *RS2* dead after HMaster traverse {*}RS1{*}, and before traverse *RS2*
> # *RS1* claim one queue of *RS2,* which named *peerid-RS2* now
> # By the way , the cversion of /hbase/replication/rs not changed before all
> of *RS2* queue is removed, because the children of /hbase/replication/rs not
> change.
> # So, Hmaster will lost the wals in *peerid-RS2,* because we have already
> traversed *RS1 ,* and ** this queue not exists in *RS2*
> The above expression is currently only speculation, not confirmed
> Flie Not Found Log.
>
> {code:java}
> // code placeholder
> 2021-11-22 15:18:39,593 ERROR
> [ReplicationExecutor-0.replicationSource,peer_id-hostname,60020,1636802867348.replicationSource.wal-reader.hostname%2C60020%2C1636802867348,peer_id-hostname,60020,1636802867348]
> regionserver.WALEntryStream: Couldn't locate log:
> hdfs://namenode/hbase/oldWALs/hostname%2C60020%2C1636802867348.1636944748704
> 2021-11-22 15:18:39,593 ERROR
> [ReplicationExecutor-0.replicationSource,peer_id-hostname,60020,1636802867348.replicationSource.wal-reader.hostname%2C60020%2C1636802867348,peer_id-hostname,60020,1636802867348]
> regionserver.ReplicationSourceWALReader: Failed to read stream of
> replication entries
> java.io.FileNotFoundException: File does not exist:
> hdfs://namenode/hbase/oldWALs/hostname%2C60020%2C1636802867348.1636944748704
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1612)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1605)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1620)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:64)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.init(ProtobufLogReader.java:168)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:321)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:303)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:291)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:427)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:355)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openNextLog(WALEntryStream.java:303)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.checkReader(WALEntryStream.java:294)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:175)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:101)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.readWALEntries(ReplicationSourceWALReader.java:192)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:138)
> {code}
>
>
--
This message was sent by
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Apache-HBase commented on pull request #3852: URL: https://github.com/apache/hbase/pull/3852#issuecomment-976232207 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 17s | master passed | | +1 :green_heart: | compile | 3m 33s | master passed | | +1 :green_heart: | shadedjars | 10m 27s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 33s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 1s | the patch passed | | +1 :green_heart: | compile | 3m 31s | the patch passed | | +1 :green_heart: | javac | 3m 31s | the patch passed | | +1 :green_heart: | shadedjars | 10m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 505m 5s | root in the patch passed. | | | | 556m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3852 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 39cbe1f7cbd2 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/testReport/ | | Max. process+thread count | 4002 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly merged pull request #3874: Backport HBASE-26249 Ameliorate compaction made by bulk-loading files…
sunhelly merged pull request #3874: URL: https://github.com/apache/hbase/pull/3874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3874: Backport HBASE-26249 Ameliorate compaction made by bulk-loading files…
Apache-HBase commented on pull request #3874: URL: https://github.com/apache/hbase/pull/3874#issuecomment-976224029 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 29s | branch-2 passed | | +1 :green_heart: | compile | 1m 14s | branch-2 passed | | +1 :green_heart: | shadedjars | 8m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 46s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 8m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 205m 28s | hbase-server in the patch passed. | | | | 242m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3874/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3874 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f2c92d7d4fc8 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / ad0ed956c2 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3874/1/testReport/ | | Max. process+thread count | 3082 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3874/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3869: HBASE-26476 Make DefaultMemStore extensible for HStore.memstore
Apache-HBase commented on pull request #3869: URL: https://github.com/apache/hbase/pull/3869#issuecomment-976224035 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 5s | master passed | | +1 :green_heart: | compile | 1m 4s | master passed | | +1 :green_heart: | shadedjars | 8m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 42s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | +1 :green_heart: | compile | 1m 3s | the patch passed | | +1 :green_heart: | javac | 1m 3s | the patch passed | | +1 :green_heart: | shadedjars | 8m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 147m 44s | hbase-server in the patch passed. | | | | 180m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3869 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bad0dbe5735a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b870b6d9ac | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/3/testReport/ | | Max. process+thread count | 4785 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26482) HMaster may clean wals that is replicating in rare cases
[
https://issues.apache.org/jira/browse/HBASE-26482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447810#comment-17447810
]
zhuobin zheng commented on HBASE-26482:
---
One way to solve the problem: We compare cversion of znode
/hbase/replication/rs and all znodes of /hbase/replication/rs/${servername}.
Because we only focus on regionserver add and replication queue add. We don't
care wal add/remove under replicaiton queue.
> HMaster may clean wals that is replicating in rare cases
>
>
> Key: HBASE-26482
> URL: https://issues.apache.org/jira/browse/HBASE-26482
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: zhuobin zheng
>Priority: Critical
>
> In our cluster, i can found some FileNotFoundException when
> ReplicationSourceWALReader running for replication recovery queue.
> I guss the wal most likely removed by hmaste. And i found something to
> support it.
> The method getAllWALs:
> [https://github.com/apache/hbase/blob/master/hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ZKReplicationQueueStorage.java#L509
>
> |https://github.com/apache/hbase/blob/master/hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ZKReplicationQueueStorage.java#L509]Use
> zk cversion of /hbase/replication/rs as an optimistic lock to control
> concurrent ops.
> But, zk cversion *only can only reflect the changes of child nodes, but not
> the changes of grandchildren.*
> So, HMaster may loss some wal from this method in follow situation.
> # HMaster do log clean , and invoke getAllWALs to filter log which should
> not be deleted.
> # HMaster cache current cversion of /hbase/replication/rs as *v0*
> # HMaster cache all RS server name, and traverse them, get the WAL in each
> Queue
> # *RS2* dead after HMaster traverse {*}RS1{*}, and before traverse *RS2*
> # *RS1* claim one queue of *RS2,* which named *peerid-RS2* now
> # By the way , the cversion of /hbase/replication/rs not changed before all
> of *RS2* queue is removed, because the children of /hbase/replication/rs not
> change.
> # So, Hmaster will lost the wals in *peerid-RS2,* because we have already
> traversed *RS1 ,* and ** this queue not exists in *RS2*
> The above expression is currently only speculation, not confirmed
> Flie Not Found Log.
>
> {code:java}
> // code placeholder
> 2021-11-22 15:18:39,593 ERROR
> [ReplicationExecutor-0.replicationSource,peer_id-hostname,60020,1636802867348.replicationSource.wal-reader.hostname%2C60020%2C1636802867348,peer_id-hostname,60020,1636802867348]
> regionserver.WALEntryStream: Couldn't locate log:
> hdfs://namenode/hbase/oldWALs/hostname%2C60020%2C1636802867348.1636944748704
> 2021-11-22 15:18:39,593 ERROR
> [ReplicationExecutor-0.replicationSource,peer_id-hostname,60020,1636802867348.replicationSource.wal-reader.hostname%2C60020%2C1636802867348,peer_id-hostname,60020,1636802867348]
> regionserver.ReplicationSourceWALReader: Failed to read stream of
> replication entries
> java.io.FileNotFoundException: File does not exist:
> hdfs://namenode/hbase/oldWALs/hostname%2C60020%2C1636802867348.1636944748704
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1612)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1605)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1620)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:64)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.init(ProtobufLogReader.java:168)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:321)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:303)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:291)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:427)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:355)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openNextLog(WALEntryStream.java:303)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.checkReader(WALEntryStream.java:294)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:175)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:101)
> at
>
[jira] [Updated] (HBASE-26482) HMaster may clean wals that is replicating in rare cases
[
https://issues.apache.org/jira/browse/HBASE-26482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuobin zheng updated HBASE-26482:
--
Summary: HMaster may clean wals that is replicating in rare cases (was:
HMaster may clean replication wals in rare cases)
> HMaster may clean wals that is replicating in rare cases
>
>
> Key: HBASE-26482
> URL: https://issues.apache.org/jira/browse/HBASE-26482
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: zhuobin zheng
>Priority: Critical
>
> In our cluster, i can found some FileNotFoundException when
> ReplicationSourceWALReader running for replication recovery queue.
> I guss the wal most likely removed by hmaste. And i found something to
> support it.
> The method getAllWALs:
> [https://github.com/apache/hbase/blob/master/hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ZKReplicationQueueStorage.java#L509
>
> |https://github.com/apache/hbase/blob/master/hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ZKReplicationQueueStorage.java#L509]Use
> zk cversion of /hbase/replication/rs as an optimistic lock to control
> concurrent ops.
> But, zk cversion *only can only reflect the changes of child nodes, but not
> the changes of grandchildren.*
> So, HMaster may loss some wal from this method in follow situation.
> # HMaster do log clean , and invoke getAllWALs to filter log which should
> not be deleted.
> # HMaster cache current cversion of /hbase/replication/rs as *v0*
> # HMaster cache all RS server name, and traverse them, get the WAL in each
> Queue
> # *RS2* dead after HMaster traverse {*}RS1{*}, and before traverse *RS2*
> # *RS1* claim one queue of *RS2,* which named *peerid-RS2* now
> # By the way , the cversion of /hbase/replication/rs not changed before all
> of *RS2* queue is removed, because the children of /hbase/replication/rs not
> change.
> # So, Hmaster will lost the wals in *peerid-RS2,* because we have already
> traversed *RS1 ,* and ** this queue not exists in *RS2*
> The above expression is currently only speculation, not confirmed
> Flie Not Found Log.
>
> {code:java}
> // code placeholder
> 2021-11-22 15:18:39,593 ERROR
> [ReplicationExecutor-0.replicationSource,peer_id-hostname,60020,1636802867348.replicationSource.wal-reader.hostname%2C60020%2C1636802867348,peer_id-hostname,60020,1636802867348]
> regionserver.WALEntryStream: Couldn't locate log:
> hdfs://namenode/hbase/oldWALs/hostname%2C60020%2C1636802867348.1636944748704
> 2021-11-22 15:18:39,593 ERROR
> [ReplicationExecutor-0.replicationSource,peer_id-hostname,60020,1636802867348.replicationSource.wal-reader.hostname%2C60020%2C1636802867348,peer_id-hostname,60020,1636802867348]
> regionserver.ReplicationSourceWALReader: Failed to read stream of
> replication entries
> java.io.FileNotFoundException: File does not exist:
> hdfs://namenode/hbase/oldWALs/hostname%2C60020%2C1636802867348.1636944748704
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1612)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1605)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1620)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:64)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.init(ProtobufLogReader.java:168)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:321)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:303)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:291)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:427)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:355)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openNextLog(WALEntryStream.java:303)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.checkReader(WALEntryStream.java:294)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:175)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:101)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.readWALEntries(ReplicationSourceWALReader.java:192)
> at
>
[jira] [Created] (HBASE-26482) HMaster may clean replication wals in rare cases
zhuobin zheng created HBASE-26482:
-
Summary: HMaster may clean replication wals in rare cases
Key: HBASE-26482
URL: https://issues.apache.org/jira/browse/HBASE-26482
Project: HBase
Issue Type: Bug
Components: Replication
Reporter: zhuobin zheng
In our cluster, i can found some FileNotFoundException when
ReplicationSourceWALReader running for replication recovery queue.
I guss the wal most likely removed by hmaste. And i found something to support
it.
The method getAllWALs:
[https://github.com/apache/hbase/blob/master/hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ZKReplicationQueueStorage.java#L509
|https://github.com/apache/hbase/blob/master/hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ZKReplicationQueueStorage.java#L509]Use
zk cversion of /hbase/replication/rs as an optimistic lock to control
concurrent ops.
But, zk cversion *only can only reflect the changes of child nodes, but not the
changes of grandchildren.*
So, HMaster may loss some wal from this method in follow situation.
# HMaster do log clean , and invoke getAllWALs to filter log which should not
be deleted.
# HMaster cache current cversion of /hbase/replication/rs as *v0*
# HMaster cache all RS server name, and traverse them, get the WAL in each
Queue
# *RS2* dead after HMaster traverse {*}RS1{*}, and before traverse *RS2*
# *RS1* claim one queue of *RS2,* which named *peerid-RS2* now
# By the way , the cversion of /hbase/replication/rs not changed before all of
*RS2* queue is removed, because the children of /hbase/replication/rs not
change.
# So, Hmaster will lost the wals in *peerid-RS2,* because we have already
traversed *RS1 ,* and ** this queue not exists in *RS2*
The above expression is currently only speculation, not confirmed
Flie Not Found Log.
{code:java}
// code placeholder
2021-11-22 15:18:39,593 ERROR
[ReplicationExecutor-0.replicationSource,peer_id-hostname,60020,1636802867348.replicationSource.wal-reader.hostname%2C60020%2C1636802867348,peer_id-hostname,60020,1636802867348]
regionserver.WALEntryStream: Couldn't locate log:
hdfs://namenode/hbase/oldWALs/hostname%2C60020%2C1636802867348.1636944748704
2021-11-22 15:18:39,593 ERROR
[ReplicationExecutor-0.replicationSource,peer_id-hostname,60020,1636802867348.replicationSource.wal-reader.hostname%2C60020%2C1636802867348,peer_id-hostname,60020,1636802867348]
regionserver.ReplicationSourceWALReader: Failed to read stream of replication
entries
java.io.FileNotFoundException: File does not exist:
hdfs://namenode/hbase/oldWALs/hostname%2C60020%2C1636802867348.1636944748704
at
org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1612)
at
org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1605)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1620)
at
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:64)
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.init(ProtobufLogReader.java:168)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:321)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:303)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:291)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:427)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:355)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openNextLog(WALEntryStream.java:303)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.checkReader(WALEntryStream.java:294)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:175)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:101)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.readWALEntries(ReplicationSourceWALReader.java:192)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:138)
{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3874: Backport HBASE-26249 Ameliorate compaction made by bulk-loading files…
Apache-HBase commented on pull request #3874: URL: https://github.com/apache/hbase/pull/3874#issuecomment-976199373 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 40s | branch-2 passed | | +1 :green_heart: | compile | 1m 0s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 34s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 6m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 148m 50s | hbase-server in the patch passed. | | | | 182m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3874/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3874 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4689abb3004f 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / ad0ed956c2 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3874/1/testReport/ | | Max. process+thread count | 3998 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3874/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3869: HBASE-26476 Make DefaultMemStore extensible for HStore.memstore
Apache-HBase commented on pull request #3869: URL: https://github.com/apache/hbase/pull/3869#issuecomment-976180977 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 34s | master passed | | +1 :green_heart: | compile | 3m 53s | master passed | | +1 :green_heart: | checkstyle | 1m 19s | master passed | | +1 :green_heart: | spotbugs | 2m 38s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 45s | the patch passed | | +1 :green_heart: | compile | 3m 48s | the patch passed | | +1 :green_heart: | javac | 3m 48s | the patch passed | | +1 :green_heart: | checkstyle | 1m 17s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 23m 40s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 63m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3869 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux a5ebb059d129 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b870b6d9ac | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3857: HBASE-26458 Add UNSET_SNAPSHOT_PROP and fix TTL defaulting
Apache-HBase commented on pull request #3857: URL: https://github.com/apache/hbase/pull/3857#issuecomment-976171497 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 2m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 9m 0s | branch-1 passed | | +1 :green_heart: | compile | 2m 26s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 2m 38s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 8m 29s | branch-1 passed | | +0 :ok: | refguide | 5m 29s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 5m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 40s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 4m 43s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 4m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 23m 34s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 40s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 9s | the patch passed | | +1 :green_heart: | compile | 2m 48s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 2m 48s | the patch passed | | +1 :green_heart: | compile | 2m 50s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 2m 50s | the patch passed | | +1 :green_heart: | checkstyle | 10m 55s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 5m 9s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 5m 2s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 7m 10s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 5m 34s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 6m 24s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 25m 24s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 218m 57s | root in the patch failed. | | +1 :green_heart: | asflicense | 2m 9s | The patch does not generate ASF License warnings. | | | | 372m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3857/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3857 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide | | uname | Linux a3f95186131c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-3857/out/precommit/personality/provided.sh | | git revision | branch-1 / 1701f0de0a | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3857/3/artifact/out/branch-site/book.html | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3857/3/artifact/out/patch-site/book.html | | unit |
[GitHub] [hbase] anoopsjohn commented on a change in pull request #3859: HBASE-26465 MemStoreLAB may be released early when its SegmentScanner…
anoopsjohn commented on a change in pull request #3859:
URL: https://github.com/apache/hbase/pull/3859#discussion_r754799573
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
##
@@ -1222,6 +1225,14 @@ private boolean updateStorefiles(List sfs,
long snapshotId) throws I
this.lock.writeLock().lock();
try {
this.storeEngine.getStoreFileManager().insertNewFiles(sfs);
+ /**
+ * NOTE:we should keep clearSnapshot method inside the write lock
because clearSnapshot may
+ * close {@link DefaultMemStore#snapshot}, which may be used by
+ * {@link DefaultMemStore#getScanners}.
+ */
+ if (snapshotId > 0) {
Review comment:
So this is the actual fix part. Moving the clearSnapshot back under the
writelock. Correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-26465) MemStoreLAB may be released early when its SegmentScanner is scanning
[
https://issues.apache.org/jira/browse/HBASE-26465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-26465:
---
Affects Version/s: 3.0.0-alpha-2
(was: 3.0.0-alpha-1)
> MemStoreLAB may be released early when its SegmentScanner is scanning
> -
>
> Key: HBASE-26465
> URL: https://issues.apache.org/jira/browse/HBASE-26465
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.5.0, 3.0.0-alpha-2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-2
>
>
> HBASE-26144 moved {{MemStore.clearSnapshot}} out of write lock of
> {{HStore#lock}}, but because {{MemStore.clearSnapshot}} closed
> {{DefaultMemStore#snapshot}} which may be used by
> {{DefaultMemStore#getScanners}}, when {{DefaultMemStore#getScanners}} and
> {{MemStore}} flushing execute conurrently, {{MemStoreLAB}} used by
> {{DefaultMemStore#snapshot}} may be released early when its
> {{SegmentScanner}} is scanning.
> Considering follow thread sequences:
> 1.The flush thread starts {{DefaultMemStore}} flushing after some cells have
> be added to {{DefaultMemStore}}.
> 2.The flush thread stopping before {{DefaultMemStore#clearSnapshot}} in
> following line 1238 in
>{{HStore#updateStorefiles}} after completed flushing memStore to hfile.
> {code:java}
> 1221private boolean updateStorefiles(List sfs, long
> snapshotId) throws IOException {
> 1222this.lock.writeLock().lock();
> 1223 try {
> 1224 this.storeEngine.getStoreFileManager().insertNewFiles(sfs);
> 1225} finally {
> 1226 // We need the lock, as long as we are updating the storeFiles
> 1227 // or changing the memstore. Let us release it before calling
> 1228 // notifyChangeReadersObservers. See HBASE-4485 for a possible
> 1229 // deadlock scenario that could have happened if continue to hold
> 1230 // the lock.
> 1231 this.lock.writeLock().unlock();
> 1232 }
> 1233// We do not need to call clearSnapshot method inside the write lock.
> 1234// The clearSnapshot itself is thread safe, which can be called at
> the same time with other
> 1235// memstore operations expect snapshot and clearSnapshot. And for
> these two methods, in HRegion
> 1236// we can guarantee that there is only one onging flush, so they will
> be no race.
> 1237if (snapshotId > 0) {
> 1238 this.memstore.clearSnapshot(snapshotId);
> 1239}
> ..
> {code}
> 3.The scan thread starts and stopping after
> {{DefaultMemStore.snapshot.getAllSegments}} in following line 141 in
>{{DefaultMemStore#getScanners}},here the scan thread gets the
> {{DefaultMemStore#snapshot}} which is created by the flush thread.
> {code:java}
> 138 public List getScanners(long readPt) throws IOException {
> 139List list = new ArrayList<>();
> 140addToScanners(getActive(), readPt, list);
> 141addToScanners(snapshot.getAllSegments(), readPt, list);
> 142return list;
> 143 }
> {code}
> 4.The flush thread continues {{DefaultMemStore#clearSnapshot}} and close
> {{DefaultMemStore#snapshot}} in following line 253,because the reference
> count of the corresponding {{MemStoreLABImpl}} is decreased to 0, the Chunks
> in corresponding {{MemStoreLABImpl}} are recycled.
> {code:java}
> 240 public void clearSnapshot(long id) throws UnexpectedStateException {
> ..
> 248Segment oldSnapshot = this.snapshot;
> 249if (!this.snapshot.isEmpty()) {
> 250 this.snapshot =
> SegmentFactory.instance().createImmutableSegment(this.comparator);
> 251}
> 252this.snapshotId = NO_SNAPSHOT_ID;
> 253oldSnapshot.close();
> 254 }
> {code}
> 5.The scan thread continues {{DefaultMemStore#getScanners}}, and create a
> {{SegmentScanner}} for this {{DefaultMemStore#snapshot}}, and invokes
> {{MemStoreLABImpl.incScannerCount}} in line 281 to increase the reference
> count , but here {{MemStoreLABImpl.incScannerCount}} does not check that the
> reference count is already 0 and {{Chunks}} are already recycled by step 4,
> so these {{Chunks}} may be overwritten by other write threads when the
> {{SegmentScanner}} is scanning, which may cause serious problem.
> {code:java}
> 280 public void incScannerCount() {
> 281 this.openScannerCount.incrementAndGet();
> 282 }
> {code}
> I simulate above case in my PR, and my Fix is as following:
> 1.Moving {{MemStore.clearSnapshot}} back under write lock of
> {{HStore#lock}}, because {{DefaultMemStore#getScanners}} is protected under
> the read lock of {{HStore#lock}}, so {{DefaultMemStore#getScanners}} and
>{{DefaultMemStore#clearSnapshot}} could not execute concurrently.
> 2. Introducing {{RefCnt}} into {{MemStoreLABImpl}} to replace its flawed
> reference count
[jira] [Updated] (HBASE-26465) MemStoreLAB may be released early when its SegmentScanner is scanning
[
https://issues.apache.org/jira/browse/HBASE-26465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-26465:
---
Affects Version/s: 2.5.0
(was: 2.4.8)
> MemStoreLAB may be released early when its SegmentScanner is scanning
> -
>
> Key: HBASE-26465
> URL: https://issues.apache.org/jira/browse/HBASE-26465
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Critical
>
> HBASE-26144 moved {{MemStore.clearSnapshot}} out of write lock of
> {{HStore#lock}}, but because {{MemStore.clearSnapshot}} closed
> {{DefaultMemStore#snapshot}} which may be used by
> {{DefaultMemStore#getScanners}}, when {{DefaultMemStore#getScanners}} and
> {{MemStore}} flushing execute conurrently, {{MemStoreLAB}} used by
> {{DefaultMemStore#snapshot}} may be released early when its
> {{SegmentScanner}} is scanning.
> Considering follow thread sequences:
> 1.The flush thread starts {{DefaultMemStore}} flushing after some cells have
> be added to {{DefaultMemStore}}.
> 2.The flush thread stopping before {{DefaultMemStore#clearSnapshot}} in
> following line 1238 in
>{{HStore#updateStorefiles}} after completed flushing memStore to hfile.
> {code:java}
> 1221private boolean updateStorefiles(List sfs, long
> snapshotId) throws IOException {
> 1222this.lock.writeLock().lock();
> 1223 try {
> 1224 this.storeEngine.getStoreFileManager().insertNewFiles(sfs);
> 1225} finally {
> 1226 // We need the lock, as long as we are updating the storeFiles
> 1227 // or changing the memstore. Let us release it before calling
> 1228 // notifyChangeReadersObservers. See HBASE-4485 for a possible
> 1229 // deadlock scenario that could have happened if continue to hold
> 1230 // the lock.
> 1231 this.lock.writeLock().unlock();
> 1232 }
> 1233// We do not need to call clearSnapshot method inside the write lock.
> 1234// The clearSnapshot itself is thread safe, which can be called at
> the same time with other
> 1235// memstore operations expect snapshot and clearSnapshot. And for
> these two methods, in HRegion
> 1236// we can guarantee that there is only one onging flush, so they will
> be no race.
> 1237if (snapshotId > 0) {
> 1238 this.memstore.clearSnapshot(snapshotId);
> 1239}
> ..
> {code}
> 3.The scan thread starts and stopping after
> {{DefaultMemStore.snapshot.getAllSegments}} in following line 141 in
>{{DefaultMemStore#getScanners}},here the scan thread gets the
> {{DefaultMemStore#snapshot}} which is created by the flush thread.
> {code:java}
> 138 public List getScanners(long readPt) throws IOException {
> 139List list = new ArrayList<>();
> 140addToScanners(getActive(), readPt, list);
> 141addToScanners(snapshot.getAllSegments(), readPt, list);
> 142return list;
> 143 }
> {code}
> 4.The flush thread continues {{DefaultMemStore#clearSnapshot}} and close
> {{DefaultMemStore#snapshot}} in following line 253,because the reference
> count of the corresponding {{MemStoreLABImpl}} is decreased to 0, the Chunks
> in corresponding {{MemStoreLABImpl}} are recycled.
> {code:java}
> 240 public void clearSnapshot(long id) throws UnexpectedStateException {
> ..
> 248Segment oldSnapshot = this.snapshot;
> 249if (!this.snapshot.isEmpty()) {
> 250 this.snapshot =
> SegmentFactory.instance().createImmutableSegment(this.comparator);
> 251}
> 252this.snapshotId = NO_SNAPSHOT_ID;
> 253oldSnapshot.close();
> 254 }
> {code}
> 5.The scan thread continues {{DefaultMemStore#getScanners}}, and create a
> {{SegmentScanner}} for this {{DefaultMemStore#snapshot}}, and invokes
> {{MemStoreLABImpl.incScannerCount}} in line 281 to increase the reference
> count , but here {{MemStoreLABImpl.incScannerCount}} does not check that the
> reference count is already 0 and {{Chunks}} are already recycled by step 4,
> so these {{Chunks}} may be overwritten by other write threads when the
> {{SegmentScanner}} is scanning, which may cause serious problem.
> {code:java}
> 280 public void incScannerCount() {
> 281 this.openScannerCount.incrementAndGet();
> 282 }
> {code}
> I simulate above case in my PR, and my Fix is as following:
> 1.Moving {{MemStore.clearSnapshot}} back under write lock of
> {{HStore#lock}}, because {{DefaultMemStore#getScanners}} is protected under
> the read lock of {{HStore#lock}}, so {{DefaultMemStore#getScanners}} and
>{{DefaultMemStore#clearSnapshot}} could not execute concurrently.
> 2. Introducing {{RefCnt}} into {{MemStoreLABImpl}} to replace its flawed
> reference count implementation, so checking and increasing or decreasing is
>
[jira] [Updated] (HBASE-26465) MemStoreLAB may be released early when its SegmentScanner is scanning
[
https://issues.apache.org/jira/browse/HBASE-26465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-26465:
---
Fix Version/s: 2.5.0
3.0.0-alpha-2
> MemStoreLAB may be released early when its SegmentScanner is scanning
> -
>
> Key: HBASE-26465
> URL: https://issues.apache.org/jira/browse/HBASE-26465
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-2
>
>
> HBASE-26144 moved {{MemStore.clearSnapshot}} out of write lock of
> {{HStore#lock}}, but because {{MemStore.clearSnapshot}} closed
> {{DefaultMemStore#snapshot}} which may be used by
> {{DefaultMemStore#getScanners}}, when {{DefaultMemStore#getScanners}} and
> {{MemStore}} flushing execute conurrently, {{MemStoreLAB}} used by
> {{DefaultMemStore#snapshot}} may be released early when its
> {{SegmentScanner}} is scanning.
> Considering follow thread sequences:
> 1.The flush thread starts {{DefaultMemStore}} flushing after some cells have
> be added to {{DefaultMemStore}}.
> 2.The flush thread stopping before {{DefaultMemStore#clearSnapshot}} in
> following line 1238 in
>{{HStore#updateStorefiles}} after completed flushing memStore to hfile.
> {code:java}
> 1221private boolean updateStorefiles(List sfs, long
> snapshotId) throws IOException {
> 1222this.lock.writeLock().lock();
> 1223 try {
> 1224 this.storeEngine.getStoreFileManager().insertNewFiles(sfs);
> 1225} finally {
> 1226 // We need the lock, as long as we are updating the storeFiles
> 1227 // or changing the memstore. Let us release it before calling
> 1228 // notifyChangeReadersObservers. See HBASE-4485 for a possible
> 1229 // deadlock scenario that could have happened if continue to hold
> 1230 // the lock.
> 1231 this.lock.writeLock().unlock();
> 1232 }
> 1233// We do not need to call clearSnapshot method inside the write lock.
> 1234// The clearSnapshot itself is thread safe, which can be called at
> the same time with other
> 1235// memstore operations expect snapshot and clearSnapshot. And for
> these two methods, in HRegion
> 1236// we can guarantee that there is only one onging flush, so they will
> be no race.
> 1237if (snapshotId > 0) {
> 1238 this.memstore.clearSnapshot(snapshotId);
> 1239}
> ..
> {code}
> 3.The scan thread starts and stopping after
> {{DefaultMemStore.snapshot.getAllSegments}} in following line 141 in
>{{DefaultMemStore#getScanners}},here the scan thread gets the
> {{DefaultMemStore#snapshot}} which is created by the flush thread.
> {code:java}
> 138 public List getScanners(long readPt) throws IOException {
> 139List list = new ArrayList<>();
> 140addToScanners(getActive(), readPt, list);
> 141addToScanners(snapshot.getAllSegments(), readPt, list);
> 142return list;
> 143 }
> {code}
> 4.The flush thread continues {{DefaultMemStore#clearSnapshot}} and close
> {{DefaultMemStore#snapshot}} in following line 253,because the reference
> count of the corresponding {{MemStoreLABImpl}} is decreased to 0, the Chunks
> in corresponding {{MemStoreLABImpl}} are recycled.
> {code:java}
> 240 public void clearSnapshot(long id) throws UnexpectedStateException {
> ..
> 248Segment oldSnapshot = this.snapshot;
> 249if (!this.snapshot.isEmpty()) {
> 250 this.snapshot =
> SegmentFactory.instance().createImmutableSegment(this.comparator);
> 251}
> 252this.snapshotId = NO_SNAPSHOT_ID;
> 253oldSnapshot.close();
> 254 }
> {code}
> 5.The scan thread continues {{DefaultMemStore#getScanners}}, and create a
> {{SegmentScanner}} for this {{DefaultMemStore#snapshot}}, and invokes
> {{MemStoreLABImpl.incScannerCount}} in line 281 to increase the reference
> count , but here {{MemStoreLABImpl.incScannerCount}} does not check that the
> reference count is already 0 and {{Chunks}} are already recycled by step 4,
> so these {{Chunks}} may be overwritten by other write threads when the
> {{SegmentScanner}} is scanning, which may cause serious problem.
> {code:java}
> 280 public void incScannerCount() {
> 281 this.openScannerCount.incrementAndGet();
> 282 }
> {code}
> I simulate above case in my PR, and my Fix is as following:
> 1.Moving {{MemStore.clearSnapshot}} back under write lock of
> {{HStore#lock}}, because {{DefaultMemStore#getScanners}} is protected under
> the read lock of {{HStore#lock}}, so {{DefaultMemStore#getScanners}} and
>{{DefaultMemStore#clearSnapshot}} could not execute concurrently.
> 2. Introducing {{RefCnt}} into {{MemStoreLABImpl}} to replace its flawed
> reference count implementation, so checking
[jira] [Updated] (HBASE-26476) Make DefaultMemStore extensible for HStore.memstore
[
https://issues.apache.org/jira/browse/HBASE-26476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26476:
-
Description:
When {{HStore.memstore}} is {{DefaultMemStore}} , I find that it is
impossible to extend {{DefaultMemStore}} to add some custom logic when I was
work on HBASE-26465, because following code in {{HStore.getMemstore}} and
{{HStore.getMemstore}} is a private method and is called in {{HStore}} ctor ,
we should make the {{DefaultMemStore}} extensible just as in branch-1.
{code:java}
switch (inMemoryCompaction) {
case NONE:
ms = ReflectionUtils.newInstance(DefaultMemStore.class,
new Object[] { conf, getComparator(),
this.getHRegion().getRegionServicesForStores()});
break;
default:
Class clz =
conf.getClass(MEMSTORE_CLASS_NAME,
CompactingMemStore.class, CompactingMemStore.class);
ms = ReflectionUtils.newInstance(clz, new Object[]{conf,
getComparator(), this,
this.getHRegion().getRegionServicesForStores(),
inMemoryCompaction});
}
return ms;
}
{code}
was:
When {{HStore.memstore}} is {{DefaultMemStore}} , I find that it is
impossible to extend {{DefaultMemStore}} to add some custom logic when I was
work on HBASE-26465, because following code in {{HStore.getMemstore}} and
{{HStore.getMemstore}} is a private method and is called in {{HStore}} ctor ,
we should make the :
{code:java}
switch (inMemoryCompaction) {
case NONE:
ms = ReflectionUtils.newInstance(DefaultMemStore.class,
new Object[] { conf, getComparator(),
this.getHRegion().getRegionServicesForStores()});
break;
default:
Class clz =
conf.getClass(MEMSTORE_CLASS_NAME,
CompactingMemStore.class, CompactingMemStore.class);
ms = ReflectionUtils.newInstance(clz, new Object[]{conf,
getComparator(), this,
this.getHRegion().getRegionServicesForStores(),
inMemoryCompaction});
}
return ms;
}
{code}
> Make DefaultMemStore extensible for HStore.memstore
>
>
> Key: HBASE-26476
> URL: https://issues.apache.org/jira/browse/HBASE-26476
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.4.8
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> When {{HStore.memstore}} is {{DefaultMemStore}} , I find that it is
> impossible to extend {{DefaultMemStore}} to add some custom logic when I was
> work on HBASE-26465, because following code in {{HStore.getMemstore}} and
> {{HStore.getMemstore}} is a private method and is called in {{HStore}} ctor ,
> we should make the {{DefaultMemStore}} extensible just as in branch-1.
> {code:java}
> switch (inMemoryCompaction) {
> case NONE:
> ms = ReflectionUtils.newInstance(DefaultMemStore.class,
> new Object[] { conf, getComparator(),
> this.getHRegion().getRegionServicesForStores()});
> break;
> default:
> Class clz =
> conf.getClass(MEMSTORE_CLASS_NAME,
> CompactingMemStore.class, CompactingMemStore.class);
> ms = ReflectionUtils.newInstance(clz, new Object[]{conf,
> getComparator(), this,
> this.getHRegion().getRegionServicesForStores(),
> inMemoryCompaction});
> }
> return ms;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26476) Make DefaultMemStore extensible for HStore.memstore
[
https://issues.apache.org/jira/browse/HBASE-26476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26476:
-
Description:
When {{HStore.memstore}} is {{DefaultMemStore}} , I find that it is
impossible to extend {{DefaultMemStore}} to add some custom logic when I was
work on HBASE-26465, because following code in {{HStore.getMemstore}} and
{{HStore.getMemstore}} is a private method and is called in {{HStore}} ctor .We
should make the {{DefaultMemStore}} extensible just as in branch-1.
{code:java}
switch (inMemoryCompaction) {
case NONE:
ms = ReflectionUtils.newInstance(DefaultMemStore.class,
new Object[] { conf, getComparator(),
this.getHRegion().getRegionServicesForStores()});
break;
default:
Class clz =
conf.getClass(MEMSTORE_CLASS_NAME,
CompactingMemStore.class, CompactingMemStore.class);
ms = ReflectionUtils.newInstance(clz, new Object[]{conf,
getComparator(), this,
this.getHRegion().getRegionServicesForStores(),
inMemoryCompaction});
}
return ms;
}
{code}
was:
When {{HStore.memstore}} is {{DefaultMemStore}} , I find that it is
impossible to extend {{DefaultMemStore}} to add some custom logic when I was
work on HBASE-26465, because following code in {{HStore.getMemstore}} and
{{HStore.getMemstore}} is a private method and is called in {{HStore}} ctor ,
we should make the {{DefaultMemStore}} extensible just as in branch-1.
{code:java}
switch (inMemoryCompaction) {
case NONE:
ms = ReflectionUtils.newInstance(DefaultMemStore.class,
new Object[] { conf, getComparator(),
this.getHRegion().getRegionServicesForStores()});
break;
default:
Class clz =
conf.getClass(MEMSTORE_CLASS_NAME,
CompactingMemStore.class, CompactingMemStore.class);
ms = ReflectionUtils.newInstance(clz, new Object[]{conf,
getComparator(), this,
this.getHRegion().getRegionServicesForStores(),
inMemoryCompaction});
}
return ms;
}
{code}
> Make DefaultMemStore extensible for HStore.memstore
>
>
> Key: HBASE-26476
> URL: https://issues.apache.org/jira/browse/HBASE-26476
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.4.8
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> When {{HStore.memstore}} is {{DefaultMemStore}} , I find that it is
> impossible to extend {{DefaultMemStore}} to add some custom logic when I was
> work on HBASE-26465, because following code in {{HStore.getMemstore}} and
> {{HStore.getMemstore}} is a private method and is called in {{HStore}} ctor
> .We should make the {{DefaultMemStore}} extensible just as in branch-1.
> {code:java}
> switch (inMemoryCompaction) {
> case NONE:
> ms = ReflectionUtils.newInstance(DefaultMemStore.class,
> new Object[] { conf, getComparator(),
> this.getHRegion().getRegionServicesForStores()});
> break;
> default:
> Class clz =
> conf.getClass(MEMSTORE_CLASS_NAME,
> CompactingMemStore.class, CompactingMemStore.class);
> ms = ReflectionUtils.newInstance(clz, new Object[]{conf,
> getComparator(), this,
> this.getHRegion().getRegionServicesForStores(),
> inMemoryCompaction});
> }
> return ms;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26476) Make DefaultMemStore extensible for HStore.memstore
[
https://issues.apache.org/jira/browse/HBASE-26476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26476:
-
Description:
When {{HStore.memstore}} is {{DefaultMemStore}} , I find that it is
impossible to extend {{DefaultMemStore}} to add some custom logic when I was
work on HBASE-26465, because following code in {{HStore.getMemstore}} and
{{HStore.getMemstore}} is a private method and is called in {{HStore}} ctor ,
we should make the :
{code:java}
switch (inMemoryCompaction) {
case NONE:
ms = ReflectionUtils.newInstance(DefaultMemStore.class,
new Object[] { conf, getComparator(),
this.getHRegion().getRegionServicesForStores()});
break;
default:
Class clz =
conf.getClass(MEMSTORE_CLASS_NAME,
CompactingMemStore.class, CompactingMemStore.class);
ms = ReflectionUtils.newInstance(clz, new Object[]{conf,
getComparator(), this,
this.getHRegion().getRegionServicesForStores(),
inMemoryCompaction});
}
return ms;
}
{code}
was:
When {{HStore.memstore}} is {{DefaultMemStore}} , I find that it is
impossible to extend {{DefaultMemStore}} to add some custom logic when I was
work on HBASE-26465, because following code in {{HStore.getMemstore}} and
{{HStore.getMemstore}} is a private method and is called in {{HStore}} ctor :
{code:java}
switch (inMemoryCompaction) {
case NONE:
ms = ReflectionUtils.newInstance(DefaultMemStore.class,
new Object[] { conf, getComparator(),
this.getHRegion().getRegionServicesForStores()});
break;
default:
Class clz =
conf.getClass(MEMSTORE_CLASS_NAME,
CompactingMemStore.class, CompactingMemStore.class);
ms = ReflectionUtils.newInstance(clz, new Object[]{conf,
getComparator(), this,
this.getHRegion().getRegionServicesForStores(),
inMemoryCompaction});
}
return ms;
}
{code}
> Make DefaultMemStore extensible for HStore.memstore
>
>
> Key: HBASE-26476
> URL: https://issues.apache.org/jira/browse/HBASE-26476
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.4.8
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> When {{HStore.memstore}} is {{DefaultMemStore}} , I find that it is
> impossible to extend {{DefaultMemStore}} to add some custom logic when I was
> work on HBASE-26465, because following code in {{HStore.getMemstore}} and
> {{HStore.getMemstore}} is a private method and is called in {{HStore}} ctor ,
> we should make the :
> {code:java}
> switch (inMemoryCompaction) {
> case NONE:
> ms = ReflectionUtils.newInstance(DefaultMemStore.class,
> new Object[] { conf, getComparator(),
> this.getHRegion().getRegionServicesForStores()});
> break;
> default:
> Class clz =
> conf.getClass(MEMSTORE_CLASS_NAME,
> CompactingMemStore.class, CompactingMemStore.class);
> ms = ReflectionUtils.newInstance(clz, new Object[]{conf,
> getComparator(), this,
> this.getHRegion().getRegionServicesForStores(),
> inMemoryCompaction});
> }
> return ms;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3870: HBASE-26479: Print too slow/big scan's operation_id in region server log
Apache-HBase commented on pull request #3870: URL: https://github.com/apache/hbase/pull/3870#issuecomment-976157024 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 8m 57s | master passed | | +1 :green_heart: | compile | 3m 30s | master passed | | +1 :green_heart: | checkstyle | 1m 48s | master passed | | +1 :green_heart: | spotbugs | 2m 19s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 14s | the patch passed | | +1 :green_heart: | compile | 3m 25s | the patch passed | | +1 :green_heart: | javac | 3m 25s | the patch passed | | +1 :green_heart: | checkstyle | 1m 11s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 22m 23s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 60m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3870 | | JIRA Issue | HBASE-26479 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 457fb5cafbd6 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b870b6d9ac | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26470) Use openlabtesting protoc on linux arm64 in HBASE 2.x
[
https://issues.apache.org/jira/browse/HBASE-26470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447750#comment-17447750
]
Hudson commented on HBASE-26470:
Results for branch branch-2.4
[build #244 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/244/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/244/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/244/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/244/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/244/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Use openlabtesting protoc on linux arm64 in HBASE 2.x
> -
>
> Key: HBASE-26470
> URL: https://issues.apache.org/jira/browse/HBASE-26470
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: Mark Jens
>Assignee: Mark Jens
>Priority: Major
> Fix For: 2.5.0, 2.4.9
>
>
> This is the same issue as HBASE-23612 but for HBase 2.x.
> Please backport the fix to HBase 2.x because at the moment it is not possible
> to use Apache Phoenix on Linux ARM64.
> I will send a new Pull Request!
> -
> Currently, there is no protoc 2.5.0 for release [1]. So we can make a new one
> for ARM specific. For make sure that could work on ARM.
> We will introduce a new ARM artifact for protoc, group_id is
> org.openlabtesting.protobuf .. This is just used for protobuf-maven-plugin to
> compile .proto files. As the 3.X version of protoc support ARM already. So
> this won't affect the internal protoc usage, which is 3.5.1-1 now.
>
> [1][https://github.com/protocolbuffers/protobuf/issues/3844#issuecomment-343355946]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] sunhelly opened a new pull request #3874: Backport HBASE-26249 Ameliorate compaction made by bulk-loading files…
sunhelly opened a new pull request #3874: URL: https://github.com/apache/hbase/pull/3874 … (#3831) Signed-off-by: Duo Zhang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly merged pull request #3831: HBASE-26249 Ameliorate compaction made by bul…
sunhelly merged pull request #3831: URL: https://github.com/apache/hbase/pull/3831 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Apache-HBase commented on pull request #3852: URL: https://github.com/apache/hbase/pull/3852#issuecomment-976118025 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 6s | master passed | | +1 :green_heart: | compile | 3m 31s | master passed | | +1 :green_heart: | shadedjars | 9m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | +1 :green_heart: | compile | 3m 36s | the patch passed | | +1 :green_heart: | javac | 3m 36s | the patch passed | | +1 :green_heart: | shadedjars | 9m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 213m 21s | root in the patch passed. | | | | 264m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3852 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 08b0618e844b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/testReport/ | | Max. process+thread count | 6845 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-26414) Tracing INSTRUMENTATION_NAME is incorrect
[
https://issues.apache.org/jira/browse/HBASE-26414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuobin zheng reassigned HBASE-26414:
-
Assignee: zhuobin zheng (was: Nick Dimiduk)
> Tracing INSTRUMENTATION_NAME is incorrect
> -
>
> Key: HBASE-26414
> URL: https://issues.apache.org/jira/browse/HBASE-26414
> Project: HBase
> Issue Type: Bug
> Components: tracing
>Affects Versions: 2.5.0, 3.0.0-alpha-2
>Reporter: Nick Dimiduk
>Assignee: zhuobin zheng
>Priority: Blocker
> Fix For: 2.5.0, 3.0.0-alpha-2
>
>
> I believe the value we use for {{TraceUtil#INSTRUMENTATION_NAME}},
> {{"io.opentelemetry.contrib.hbase"}}, is incorrect. According to the java
> docs,
> {noformat}
>* @param instrumentationName The name of the instrumentation library, not
> the name of the
>* instrument*ed* library (e.g., "io.opentelemetry.contrib.mongodb").
> Must not be null.
> {noformat}
> This namespace appears to be reserved for implementations shipped by the otel
> project, found under
> https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation
> I don't have a suggestion for a suitable name at this time. Will report back.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
Apache-HBase commented on pull request #3852: URL: https://github.com/apache/hbase/pull/3852#issuecomment-976028431 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 2s | master passed | | +1 :green_heart: | compile | 9m 23s | master passed | | +1 :green_heart: | checkstyle | 2m 20s | master passed | | +0 :ok: | refguide | 3m 31s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 15m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 8m 54s | the patch passed | | +1 :green_heart: | javac | 8m 54s | the patch passed | | +1 :green_heart: | checkstyle | 1m 56s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 3m 25s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 19m 14s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 14m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 56s | The patch does not generate ASF License warnings. | | | | 97m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3852 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile refguide | | uname | Linux ab9684e32957 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | refguide | https://nightlies.apache.org/hbase/HBase/HBase-PreCommit-GitHub-PR/PR-3852/3/yetus-general-check/output/branch-site/book.html | | refguide | https://nightlies.apache.org/hbase/HBase/HBase-PreCommit-GitHub-PR/PR-3852/3/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 141 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3852/3/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3803: HBASE-26304: Reflect out of band locality improvements in metrics and balancer
Apache-HBase commented on pull request #3803: URL: https://github.com/apache/hbase/pull/3803#issuecomment-975998279 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 22s | master passed | | +1 :green_heart: | compile | 2m 10s | master passed | | +1 :green_heart: | shadedjars | 9m 24s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 26s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 13s | the patch passed | | +1 :green_heart: | compile | 2m 8s | the patch passed | | +1 :green_heart: | javac | 2m 8s | the patch passed | | +1 :green_heart: | shadedjars | 9m 10s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 13s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 11m 38s | hbase-balancer in the patch passed. | | -1 :x: | unit | 241m 16s | hbase-server in the patch failed. | | | | 297m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3803 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux db19719d0901 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-11.0.10+9 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/8/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/8/testReport/ | | Max. process+thread count | 3319 (vs. ulimit of 3) | | modules | C: hbase-common hbase-balancer hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/8/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3831: HBASE-26249 Ameliorate compaction made by bul…
Apache-HBase commented on pull request #3831: URL: https://github.com/apache/hbase/pull/3831#issuecomment-975989931 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 49s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 20s | master passed | | +1 :green_heart: | compile | 1m 38s | master passed | | +1 :green_heart: | shadedjars | 11m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 56s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 21s | the patch passed | | +1 :green_heart: | compile | 1m 50s | the patch passed | | +1 :green_heart: | javac | 1m 50s | the patch passed | | +1 :green_heart: | shadedjars | 10m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 234m 36s | hbase-server in the patch passed. | | | | 278m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3831 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a965056145a1 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/5/testReport/ | | Max. process+thread count | 2864 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joswiatek commented on pull request #3852: HBASE-26458 Update Snapshot TTL doc
joswiatek commented on pull request #3852: URL: https://github.com/apache/hbase/pull/3852#issuecomment-975977333 I've included the additional doc suggestions, thanks for the review y'all -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joswiatek commented on pull request #3856: HBASE-26458 Update snapshot doc
joswiatek commented on pull request #3856: URL: https://github.com/apache/hbase/pull/3856#issuecomment-975971306 > For a ref guide, usually we will only commit it to master branch as it contains the information for all versions, and we will only build ref guide for master branch and upload it to our website... Okay, I will close this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joswiatek closed pull request #3856: HBASE-26458 Update snapshot doc
joswiatek closed pull request #3856: URL: https://github.com/apache/hbase/pull/3856 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3803: HBASE-26304: Reflect out of band locality improvements in metrics and balancer
Apache-HBase commented on pull request #3803: URL: https://github.com/apache/hbase/pull/3803#issuecomment-975864670 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 36s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 44s | master passed | | +1 :green_heart: | compile | 6m 6s | master passed | | +1 :green_heart: | checkstyle | 2m 22s | master passed | | +0 :ok: | refguide | 4m 24s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 4m 20s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 16s | the patch passed | | +1 :green_heart: | compile | 6m 2s | the patch passed | | +1 :green_heart: | javac | 6m 2s | the patch passed | | -0 :warning: | checkstyle | 1m 19s | hbase-server: The patch generated 3 new + 72 unchanged - 0 fixed = 75 total (was 72) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 4m 18s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 30m 7s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 6m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 51s | The patch does not generate ASF License warnings. | | | | 92m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3803 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile refguide xml | | uname | Linux 9afb529865ea 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | refguide | https://nightlies.apache.org/hbase/HBase/HBase-PreCommit-GitHub-PR/PR-3803/8/yetus-general-check/output/branch-site/book.html | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/8/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | refguide | https://nightlies.apache.org/hbase/HBase/HBase-PreCommit-GitHub-PR/PR-3803/8/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-common hbase-balancer hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/8/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3873: HBASE-26456 Limit the size for one replicating
Apache-HBase commented on pull request #3873: URL: https://github.com/apache/hbase/pull/3873#issuecomment-975845283 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-26233 Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 48s | HBASE-26233 passed | | +1 :green_heart: | compile | 2m 4s | HBASE-26233 passed | | +1 :green_heart: | shadedjars | 12m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | HBASE-26233 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 7m 37s | the patch passed | | +1 :green_heart: | compile | 1m 52s | the patch passed | | +1 :green_heart: | javac | 1m 52s | the patch passed | | +1 :green_heart: | shadedjars | 12m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 173m 5s | hbase-server in the patch passed. | | | | 222m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3873/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3873 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b902348eccf5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26233 / 584884486b | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3873/1/testReport/ | | Max. process+thread count | 3817 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3873/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3869: HBASE-26476 Make DefaultMemStore extensible for HStore.memstore
Apache-HBase commented on pull request #3869: URL: https://github.com/apache/hbase/pull/3869#issuecomment-975843693 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 5s | master passed | | +1 :green_heart: | compile | 1m 21s | master passed | | +1 :green_heart: | shadedjars | 10m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 46s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 7s | the patch passed | | +1 :green_heart: | compile | 1m 16s | the patch passed | | +1 :green_heart: | javac | 1m 16s | the patch passed | | +1 :green_heart: | shadedjars | 10m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 251m 24s | hbase-server in the patch passed. | | | | 289m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3869 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4ad5b297d89a 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/2/testReport/ | | Max. process+thread count | 2652 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3831: HBASE-26249 Ameliorate compaction made by bul…
Apache-HBase commented on pull request #3831: URL: https://github.com/apache/hbase/pull/3831#issuecomment-975841075 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | compile | 3m 26s | master passed | | +1 :green_heart: | checkstyle | 1m 6s | master passed | | +1 :green_heart: | spotbugs | 2m 9s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 28s | the patch passed | | +1 :green_heart: | compile | 3m 35s | the patch passed | | +1 :green_heart: | javac | 3m 35s | the patch passed | | +1 :green_heart: | checkstyle | 1m 11s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 22m 25s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 54m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3831 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux ea796d371c68 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/5/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3872: HBASE-26340 - fix RegionSizeCalculator getLEngth to bytes instead of …
Apache-HBase commented on pull request #3872: URL: https://github.com/apache/hbase/pull/3872#issuecomment-975832718 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | master passed | | +1 :green_heart: | compile | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 9m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 22s | the patch passed | | +1 :green_heart: | compile | 1m 6s | the patch passed | | +1 :green_heart: | javac | 1m 6s | the patch passed | | +1 :green_heart: | shadedjars | 9m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 248m 12s | hbase-server in the patch passed. | | | | 282m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3872/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3872 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fa271012255b 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3872/1/testReport/ | | Max. process+thread count | 3521 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3872/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3869: HBASE-26476 Make DefaultMemStore extensible for HStore.memstore
Apache-HBase commented on pull request #3869: URL: https://github.com/apache/hbase/pull/3869#issuecomment-975830834 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 34s | master passed | | +1 :green_heart: | compile | 1m 22s | master passed | | +1 :green_heart: | shadedjars | 9m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 6s | the patch passed | | +1 :green_heart: | compile | 1m 22s | the patch passed | | +1 :green_heart: | javac | 1m 22s | the patch passed | | +1 :green_heart: | shadedjars | 10m 21s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 236m 58s | hbase-server in the patch passed. | | | | 274m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3869 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4bc083d78339 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/2/testReport/ | | Max. process+thread count | 2940 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3873: HBASE-26456 Limit the size for one replicating
Apache-HBase commented on pull request #3873: URL: https://github.com/apache/hbase/pull/3873#issuecomment-975819485 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-26233 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 38s | HBASE-26233 passed | | +1 :green_heart: | compile | 1m 9s | HBASE-26233 passed | | +1 :green_heart: | shadedjars | 9m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | HBASE-26233 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 25s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 9m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 155m 51s | hbase-server in the patch passed. | | | | 190m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3873/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3873 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3f6509727d09 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26233 / 584884486b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3873/1/testReport/ | | Max. process+thread count | 4764 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3873/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26463) Unreadable table names after HBASE-24605
[
https://issues.apache.org/jira/browse/HBASE-26463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447563#comment-17447563
]
Hudson commented on HBASE-26463:
Results for branch branch-2
[build #398 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unreadable table names after HBASE-24605
>
>
> Key: HBASE-26463
> URL: https://issues.apache.org/jira/browse/HBASE-26463
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 3.0.0-alpha-1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Trivial
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.9
>
> Attachments: After.png, Before.png
>
>
> During the fix HBASE-24605 (Break long region names in the web UI) the 'word
> break' rule was applied to the 'table-striped' style, so literally, all
> tables in Master Web UI were affected. The most noticeable is the 'User
> Tables' when table names, as well as descriptions, and even State are wrapped
> in a very ugly way.
> We have two options here:
> 1. Fix the user tables only as it was done previously for the procedures table
> 2. Apply 'word break' to those tables that require that.
> Since most of the users are comfortable with the current UI we may go through
> the first way, so the changes would affect only the User table.
> Sample screenshots before and after are attached.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26467) Wrong Cell Generated by MemStoreLABImpl.forceCopyOfBigCellInto when Cell size bigger than data chunk size
[
https://issues.apache.org/jira/browse/HBASE-26467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447565#comment-17447565
]
Hudson commented on HBASE-26467:
Results for branch branch-2
[build #398 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Wrong Cell Generated by MemStoreLABImpl.forceCopyOfBigCellInto when Cell size
> bigger than data chunk size
> --
>
> Key: HBASE-26467
> URL: https://issues.apache.org/jira/browse/HBASE-26467
> Project: HBase
> Issue Type: Bug
> Components: in-memory-compaction
>Reporter: zhuobin zheng
>Assignee: zhuobin zheng
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.9
>
>
> In our company 2.X cluster. I found some region compaction keeps failling
> because some cell can't construct succefully. In fact , we even can't read
> these cell.
> From follow stack , we can found the bug cause KeyValue can't constructed.
> Simple Log and Stack:
> {code:java}
> // code placeholder
> 2021-11-18 16:50:47,708 ERROR [regionserver/:60020-longCompactions-4]
> regionserver.CompactSplit: Compaction failed
> region=xx_table,3610ff49595a0fc4a824f2a575f37a31,1570874723992.dac703ceb35e8d8703233bebf34ae49f.,
> storeName=c, priority=-319, startTime=1637225447127
> java.lang.IllegalArgumentException: Invalid tag length at position=4659867,
> tagLength=0,
> at
> org.apache.hadoop.hbase.KeyValueUtil.checkKeyValueTagBytes(KeyValueUtil.java:685)
> at
> org.apache.hadoop.hbase.KeyValueUtil.checkKeyValueBytes(KeyValueUtil.java:643)
> at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:345)
> at
> org.apache.hadoop.hbase.SizeCachedKeyValue.(SizeCachedKeyValue.java:43)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.getCell(HFileReaderImpl.java:981)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:233)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:418)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:322)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:288)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:487)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:248)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:318)
> at
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:65)
> at
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:126)
> at
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1468)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2266)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:624)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:666)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748) {code}
> From further observation, I found the following characteristics:
> # Cell size more than 2M
> # We can reproduce the bug only after in memory compact
> # Cell bytes end with \x00\x02\x00\x00
>
[jira] [Commented] (HBASE-24605) Break long region names in the web UI
[
https://issues.apache.org/jira/browse/HBASE-24605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447564#comment-17447564
]
Hudson commented on HBASE-24605:
Results for branch branch-2
[build #398 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/398/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Break long region names in the web UI
> -
>
> Key: HBASE-24605
> URL: https://issues.apache.org/jira/browse/HBASE-24605
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Affects Versions: 3.0.0-alpha-1
>Reporter: song XinCun
>Assignee: song XinCun
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.2.6
>
> Attachments: image-2020-06-21-20-18-37-041.png,
> image-2020-06-21-20-19-25-183.png, image-2020-06-21-20-20-02-782.png,
> image-2020-06-21-20-27-23-474.png, image-2020-06-21-20-28-36-464.png,
> image-2020-06-21-20-29-07-819.png
>
>
> Before this patch, when it comes to the long region name, the UI content will
> be out of the screen, making it unreadable. Like this:
> !image-2020-06-21-20-18-37-041.png|width=542,height=50!
> !image-2020-06-21-20-19-25-183.png|width=531,height=23!
> !image-2020-06-21-20-20-02-782.png|width=542,height=146!
>
> After this patch, the long region name wil be break to the new line, like
> this:
> !image-2020-06-21-20-27-23-474.png|width=529,height=35!
> !image-2020-06-21-20-28-36-464.png|width=533,height=33!
> !image-2020-06-21-20-29-07-819.png|width=531,height=117!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3831: HBASE-26249 Ameliorate compaction made by bul…
Apache-HBase commented on pull request #3831: URL: https://github.com/apache/hbase/pull/3831#issuecomment-975796303 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 20s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 51s | master passed | | +1 :green_heart: | compile | 1m 34s | master passed | | +1 :green_heart: | shadedjars | 10m 49s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 13s | the patch passed | | +1 :green_heart: | compile | 1m 36s | the patch passed | | +1 :green_heart: | javac | 1m 36s | the patch passed | | +1 :green_heart: | shadedjars | 10m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 231m 12s | hbase-server in the patch passed. | | | | 272m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3831 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7104e92f9f45 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/4/testReport/ | | Max. process+thread count | 2825 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3803: HBASE-26304: Reflect out of band locality improvements in metrics and balancer
Apache-HBase commented on pull request #3803: URL: https://github.com/apache/hbase/pull/3803#issuecomment-975793621 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 3s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 13s | master passed | | +1 :green_heart: | compile | 2m 10s | master passed | | +1 :green_heart: | shadedjars | 9m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 23s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 4s | the patch passed | | +1 :green_heart: | compile | 2m 7s | the patch passed | | +1 :green_heart: | javac | 2m 7s | the patch passed | | +1 :green_heart: | shadedjars | 9m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 13s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 11m 37s | hbase-balancer in the patch passed. | | +1 :green_heart: | unit | 207m 44s | hbase-server in the patch passed. | | | | 261m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3803 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2d5d29badc2e 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/7/testReport/ | | Max. process+thread count | 3270 (vs. ulimit of 3) | | modules | C: hbase-common hbase-balancer hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/7/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3871: HBASE-26480 Close NamedQueueRecorder to allow HMaster/RS to shutdown gracefully
Apache-HBase commented on pull request #3871: URL: https://github.com/apache/hbase/pull/3871#issuecomment-975776719 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 8m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 39s | branch-1 passed | | +1 :green_heart: | compile | 0m 44s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 47s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 1m 44s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 17s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 41s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 3m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 8s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 5s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | compile | 0m 48s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 48s | the patch passed | | +1 :green_heart: | checkstyle | 1m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 2s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 2s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 42s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 2m 59s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 164m 45s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 41s | The patch does not generate ASF License warnings. | | | | 217m 3s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.TestMasterBalanceThrottling | | | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint | | | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles | | | hadoop.hbase.mapreduce.TestLoadIncrementalHFiles | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3871/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3871 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux a71cb59037f4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-3871/out/precommit/personality/provided.sh | | git revision | branch-1 / 1701f0de0a | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3871/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3871/1/testReport/ | | Max. process+thread count | 4286 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3871/1/console | | versions | git=2.17.1 maven=3.6.0
[GitHub] [hbase] Apache-HBase commented on pull request #3872: HBASE-26340 - fix RegionSizeCalculator getLEngth to bytes instead of …
Apache-HBase commented on pull request #3872: URL: https://github.com/apache/hbase/pull/3872#issuecomment-975775047 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 7s | master passed | | +1 :green_heart: | compile | 2m 5s | master passed | | +1 :green_heart: | shadedjars | 9m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 21s | the patch passed | | +1 :green_heart: | compile | 1m 21s | the patch passed | | +1 :green_heart: | javac | 1m 21s | the patch passed | | +1 :green_heart: | shadedjars | 8m 48s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 169m 42s | hbase-server in the patch passed. | | | | 207m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3872/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3872 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3d3e8b4f9d57 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3872/1/testReport/ | | Max. process+thread count | 3901 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3872/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3803: HBASE-26304: Reflect out of band locality improvements in metrics and balancer
Apache-HBase commented on pull request #3803: URL: https://github.com/apache/hbase/pull/3803#issuecomment-975749247 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 5s | master passed | | +1 :green_heart: | compile | 2m 21s | master passed | | +1 :green_heart: | shadedjars | 11m 21s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 31s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 14s | the patch passed | | +1 :green_heart: | compile | 2m 7s | the patch passed | | +1 :green_heart: | javac | 2m 7s | the patch passed | | +1 :green_heart: | shadedjars | 11m 14s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 4s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 7m 7s | hbase-balancer in the patch passed. | | +1 :green_heart: | unit | 152m 33s | hbase-server in the patch passed. | | | | 206m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3803 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 170fbef568ac 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/7/testReport/ | | Max. process+thread count | 3666 (vs. ulimit of 3) | | modules | C: hbase-common hbase-balancer hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/7/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3873: HBASE-26456 Limit the size for one replicating
Apache-HBase commented on pull request #3873: URL: https://github.com/apache/hbase/pull/3873#issuecomment-975704012 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-26233 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 27s | HBASE-26233 passed | | +1 :green_heart: | compile | 3m 22s | HBASE-26233 passed | | +1 :green_heart: | checkstyle | 1m 12s | HBASE-26233 passed | | +1 :green_heart: | spotbugs | 2m 14s | HBASE-26233 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 18s | the patch passed | | +1 :green_heart: | compile | 3m 21s | the patch passed | | +1 :green_heart: | javac | 3m 21s | the patch passed | | +1 :green_heart: | checkstyle | 1m 7s | the patch passed | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 21m 35s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 54m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3873/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3873 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 28d1b2e60f1e 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-26233 / 584884486b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 85 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3873/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26263) [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables
[ https://issues.apache.org/jira/browse/HBASE-26263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447503#comment-17447503 ] Wellington Chevreuil commented on HBASE-26263: -- I was trying to backport some of the commits in this branch to our internal, branch-2 based one, and noticed this PR relies on a few commits that are only in master (HBASE-26348, HBASE-22662). Are there no plans to support those on any hbase-2 branches? > [Rolling Upgrading] Persist the StoreFileTracker configurations to > TableDescriptor for existing tables > -- > > Key: HBASE-26263 > URL: https://issues.apache.org/jira/browse/HBASE-26263 > Project: HBase > Issue Type: Sub-task > Components: HFile, master >Reporter: Duo Zhang >Assignee: Zhuoyue Huang >Priority: Major > Fix For: HBASE-26067 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26481) Consider rolling upgrading from old region replication framework
Duo Zhang created HBASE-26481: - Summary: Consider rolling upgrading from old region replication framework Key: HBASE-26481 URL: https://issues.apache.org/jira/browse/HBASE-26481 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang In the old implementation, we will cerate a replication peer for region replication. And in the new framework, we do not need the peer any more, so we have deleted the RegionReplicaReplicationEndpoint class. But when rolling upgrading, the peer is still there so we will try to load it and it will lead to a class not found exception. We need to handle it. And during the upgrading, some of the region servers may not have the correct replicateToReplica method yet. Maybe we could try to catch the no such method error and try to fallback to use replay method. Although it is not perfect, but finally a flush can fix everything. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26481) Consider rolling upgrading from old region replication framework
[ https://issues.apache.org/jira/browse/HBASE-26481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26481: -- Component/s: read replicas > Consider rolling upgrading from old region replication framework > > > Key: HBASE-26481 > URL: https://issues.apache.org/jira/browse/HBASE-26481 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Reporter: Duo Zhang >Priority: Major > > In the old implementation, we will cerate a replication peer for region > replication. And in the new framework, we do not need the peer any more, so > we have deleted the RegionReplicaReplicationEndpoint class. > But when rolling upgrading, the peer is still there so we will try to load it > and it will lead to a class not found exception. We need to handle it. > And during the upgrading, some of the region servers may not have the correct > replicateToReplica method yet. Maybe we could try to catch the no such method > error and try to fallback to use replay method. Although it is not perfect, > but finally a flush can fix everything. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26305) Move NavigableSet add operation to writer thread in BucketCache
[
https://issues.apache.org/jira/browse/HBASE-26305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447495#comment-17447495
]
Hudson commented on HBASE-26305:
Results for branch branch-1
[build #183 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/183/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/183//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/183//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/183//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Move NavigableSet add operation to writer thread in BucketCache
> ---
>
> Key: HBASE-26305
> URL: https://issues.apache.org/jira/browse/HBASE-26305
> Project: HBase
> Issue Type: Improvement
> Components: BucketCache, Performance
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Minor
> Fix For: 2.5.0, 3.0.0-alpha-2
>
> Attachments: logn in WriterThreads.png, logn in cacheBlock.png
>
>
> We currently use a ConcurrentSkipList to store blocks by HFile in bucket
> cache. The average time complexity of the add function is O(logn). We can
> move this time costly to the writer threads to reduce the response latency of
> read requests. I have tested the time cost of function cacheBlock in
> BucketCache and attached the metrics screenshots.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26325) Backport HBASE-26305 to branch-1
[
https://issues.apache.org/jira/browse/HBASE-26325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447494#comment-17447494
]
Hudson commented on HBASE-26325:
Results for branch branch-1
[build #183 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/183/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/183//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/183//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/183//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Backport HBASE-26305 to branch-1
>
>
> Key: HBASE-26325
> URL: https://issues.apache.org/jira/browse/HBASE-26325
> Project: HBase
> Issue Type: Improvement
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Minor
> Fix For: 1.7.2
>
>
> Backport the HBASE-26305 to branch-1. Move NavigableSet add operation to
> writer thread in BucketCache
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3870: HBASE-26479: Print too slow/big scan's operation_id in region server log
Apache-HBase commented on pull request #3870: URL: https://github.com/apache/hbase/pull/3870#issuecomment-975657892 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 14s | master passed | | +1 :green_heart: | compile | 1m 2s | master passed | | +1 :green_heart: | shadedjars | 8m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | shadedjars | 8m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 153m 6s | hbase-server in the patch passed. | | | | 183m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3870 | | JIRA Issue | HBASE-26479 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 067b5a2dbe59 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/testReport/ | | Max. process+thread count | 4821 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on pull request #3832: HBASE-26435 The log rolling request maybe canceled immediately in LogRoller due to a race
shahrs87 commented on pull request #3832: URL: https://github.com/apache/hbase/pull/3832#issuecomment-975639501 Some tests are failing due to the patch. Currently I am working on production incidents and won't get time to look into failures until next week. Sorry for the delay. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3832: HBASE-26435 The log rolling request maybe canceled immediately in LogRoller due to a race
Apache9 commented on pull request #3832: URL: https://github.com/apache/hbase/pull/3832#issuecomment-975636127 @Reidd PTAL? This is for branch-1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26470) Use openlabtesting protoc on linux arm64 in HBASE 2.x
[ https://issues.apache.org/jira/browse/HBASE-26470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26470. --- Resolution: Fixed Pushed to branch-2 and branch-2.4. Thanks [~mark.jens] for contributing. > Use openlabtesting protoc on linux arm64 in HBASE 2.x > - > > Key: HBASE-26470 > URL: https://issues.apache.org/jira/browse/HBASE-26470 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Mark Jens >Assignee: Mark Jens >Priority: Major > Fix For: 2.5.0, 2.4.9 > > > This is the same issue as HBASE-23612 but for HBase 2.x. > Please backport the fix to HBase 2.x because at the moment it is not possible > to use Apache Phoenix on Linux ARM64. > I will send a new Pull Request! > - > Currently, there is no protoc 2.5.0 for release [1]. So we can make a new one > for ARM specific. For make sure that could work on ARM. > We will introduce a new ARM artifact for protoc, group_id is > org.openlabtesting.protobuf .. This is just used for protobuf-maven-plugin to > compile .proto files. As the 3.X version of protoc support ARM already. So > this won't affect the internal protoc usage, which is 3.5.1-1 now. > > [1][https://github.com/protocolbuffers/protobuf/issues/3844#issuecomment-343355946] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26470) Use openlabtesting protoc on linux arm64 in HBASE 2.x
[ https://issues.apache.org/jira/browse/HBASE-26470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-26470: - Assignee: Mark Jens (was: zhao bo) > Use openlabtesting protoc on linux arm64 in HBASE 2.x > - > > Key: HBASE-26470 > URL: https://issues.apache.org/jira/browse/HBASE-26470 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Mark Jens >Assignee: Mark Jens >Priority: Major > > This is the same issue as HBASE-23612 but for HBase 2.x. > Please backport the fix to HBase 2.x because at the moment it is not possible > to use Apache Phoenix on Linux ARM64. > I will send a new Pull Request! > - > Currently, there is no protoc 2.5.0 for release [1]. So we can make a new one > for ARM specific. For make sure that could work on ARM. > We will introduce a new ARM artifact for protoc, group_id is > org.openlabtesting.protobuf .. This is just used for protobuf-maven-plugin to > compile .proto files. As the 3.X version of protoc support ARM already. So > this won't affect the internal protoc usage, which is 3.5.1-1 now. > > [1][https://github.com/protocolbuffers/protobuf/issues/3844#issuecomment-343355946] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26470) Use openlabtesting protoc on linux arm64 in HBASE 2.x
[ https://issues.apache.org/jira/browse/HBASE-26470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26470: -- Fix Version/s: 2.5.0 2.4.9 > Use openlabtesting protoc on linux arm64 in HBASE 2.x > - > > Key: HBASE-26470 > URL: https://issues.apache.org/jira/browse/HBASE-26470 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Mark Jens >Assignee: Mark Jens >Priority: Major > Fix For: 2.5.0, 2.4.9 > > > This is the same issue as HBASE-23612 but for HBase 2.x. > Please backport the fix to HBase 2.x because at the moment it is not possible > to use Apache Phoenix on Linux ARM64. > I will send a new Pull Request! > - > Currently, there is no protoc 2.5.0 for release [1]. So we can make a new one > for ARM specific. For make sure that could work on ARM. > We will introduce a new ARM artifact for protoc, group_id is > org.openlabtesting.protobuf .. This is just used for protobuf-maven-plugin to > compile .proto files. As the 3.X version of protoc support ARM already. So > this won't affect the internal protoc usage, which is 3.5.1-1 now. > > [1][https://github.com/protocolbuffers/protobuf/issues/3844#issuecomment-343355946] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work started] (HBASE-26456) Limit the size for one replicating
[ https://issues.apache.org/jira/browse/HBASE-26456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-26456 started by Duo Zhang. - > Limit the size for one replicating > -- > > Key: HBASE-26456 > URL: https://issues.apache.org/jira/browse/HBASE-26456 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Now we will always try to replicate all the pending entries, which could be > too large. We should have a size/count limit, to avoid putting too much > pressure to the target region server for some extreme cases. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HBASE-26456) Limit the size for one replicating
[ https://issues.apache.org/jira/browse/HBASE-26456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-26456: - Assignee: Duo Zhang > Limit the size for one replicating > -- > > Key: HBASE-26456 > URL: https://issues.apache.org/jira/browse/HBASE-26456 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > Now we will always try to replicate all the pending entries, which could be > too large. We should have a size/count limit, to avoid putting too much > pressure to the target region server for some extreme cases. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26470) Use openlabtesting protoc on linux arm64 in HBASE 2.x
[ https://issues.apache.org/jira/browse/HBASE-26470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26470: -- Summary: Use openlabtesting protoc on linux arm64 in HBASE 2.x (was: CLONE - Update pom.xml to use another 2.5.0 protoc as external protobuf) > Use openlabtesting protoc on linux arm64 in HBASE 2.x > - > > Key: HBASE-26470 > URL: https://issues.apache.org/jira/browse/HBASE-26470 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Mark Jens >Assignee: zhao bo >Priority: Major > > This is the same issue as HBASE-23612 but for HBase 2.x. > Please backport the fix to HBase 2.x because at the moment it is not possible > to use Apache Phoenix on Linux ARM64. > I will send a new Pull Request! > - > Currently, there is no protoc 2.5.0 for release [1]. So we can make a new one > for ARM specific. For make sure that could work on ARM. > We will introduce a new ARM artifact for protoc, group_id is > org.openlabtesting.protobuf .. This is just used for protobuf-maven-plugin to > compile .proto files. As the 3.X version of protoc support ARM already. So > this won't affect the internal protoc usage, which is 3.5.1-1 now. > > [1][https://github.com/protocolbuffers/protobuf/issues/3844#issuecomment-343355946] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache9 merged pull request #3860: HBASE-26470 Use openlabtesting protoc on linux arm64 in HBASE 2.x
Apache9 merged pull request #3860: URL: https://github.com/apache/hbase/pull/3860 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26416) Implement a new method for region replication instead of using replay
[ https://issues.apache.org/jira/browse/HBASE-26416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26416. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~Xiaolin Ha] for reviewing. Will rewrite the design doc section soon. > Implement a new method for region replication instead of using replay > - > > Key: HBASE-26416 > URL: https://issues.apache.org/jira/browse/HBASE-26416 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: HBASE-26233 > > > The replay method is design to be used for DLR, which has been removed for a > long time. > We should have our own version for replay the edits for region replication, > which could be more flexible. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26416) Implement a new method for region replication instead of using replay
[ https://issues.apache.org/jira/browse/HBASE-26416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-26416: -- Fix Version/s: HBASE-26233 > Implement a new method for region replication instead of using replay > - > > Key: HBASE-26416 > URL: https://issues.apache.org/jira/browse/HBASE-26416 > Project: HBase > Issue Type: Sub-task > Components: read replicas >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: HBASE-26233 > > > The replay method is design to be used for DLR, which has been removed for a > long time. > We should have our own version for replay the edits for region replication, > which could be more flexible. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3872: HBASE-26340 - fix RegionSizeCalculator getLEngth to bytes instead of …
Apache-HBase commented on pull request #3872: URL: https://github.com/apache/hbase/pull/3872#issuecomment-975627082 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | compile | 3m 11s | master passed | | +1 :green_heart: | checkstyle | 1m 5s | master passed | | +1 :green_heart: | spotbugs | 2m 4s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | +1 :green_heart: | compile | 3m 10s | the patch passed | | +1 :green_heart: | javac | 3m 10s | the patch passed | | +1 :green_heart: | checkstyle | 1m 4s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 10s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 48m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3872/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3872 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux dd5a118fcf08 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3872/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26468) Region Server doesn't exit cleanly incase it crashes.
[
https://issues.apache.org/jira/browse/HBASE-26468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447467#comment-17447467
]
Duo Zhang commented on HBASE-26468:
---
Maybe we could add a delay? For example, if the process does not exit for 30
seconds, we call System.exit to force quit, and the return value should be
something other than 0 to indicate that this is a force terminate.
> Region Server doesn't exit cleanly incase it crashes.
> -
>
> Key: HBASE-26468
> URL: https://issues.apache.org/jira/browse/HBASE-26468
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.6.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 1.7.2, 2.3.8, 2.4.9
>
>
> Observed this in our production cluster running 1.6 version.
> RS crashed due to some reason but the process was still running. On debugging
> more, found out there was 1 non-daemon thread running and that was not
> allowing RS to exit cleanly. Our clusters are managed by Ambari and have auto
> restart capability within them. But since the process was running and pid
> file was present, Ambari also couldn't do much. There will be some bug where
> we will miss to stop some non daemon thread. Shutdown hook will not be called
> unless one of the following 2 conditions are met:
> # The Java virtual machine shuts down in response to two kinds of events:
> The program exits normally, when the last non-daemon thread exits or when the
> exit (equivalently, System.exit) method is invoked, or
> # The virtual machine is terminated in response to a user interrupt, such as
> typing ^C, or a system-wide event, such as user logoff or system shutdown.
> Considering the first condition, when the last non-daemon thread exits or
> when the exit method is invoked.
> Below is the code snippet from
> [HRegionServerCommandLine.java|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServerCommandLine.java#L51]
> {code:java}
> private int start() throws Exception {
> try {
> if (LocalHBaseCluster.isLocal(conf)) {
> // Ignore this.
> } else {
> HRegionServer hrs =
> HRegionServer.constructRegionServer(regionServerClass, conf);
> hrs.start();
> hrs.join();
> if (hrs.isAborted()) {
> throw new RuntimeException("HRegionServer Aborted");
> }
> }
> } catch (Throwable t) {
> LOG.error("Region server exiting", t);
> return 1;
> }
> return 0;
> }
> {code}
> Within HRegionServer, there is a subtle difference between when a server is
> aborted v/s when it is stopped. If it is stopped, then isAborted will return
> false and it will exit with return code 0.
> Below is the code from
> [ServerCommandLine.java|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerCommandLine.java#L147]
> {code:java}
> public void doMain(String args[]) {
> try {
> int ret = ToolRunner.run(HBaseConfiguration.create(), this, args);
> if (ret != 0) {
> System.exit(ret);
> }
> } catch (Exception e) {
> LOG.error("Failed to run", e);
> System.exit(-1);
> }
> }
> {code}
> If return code is 0, then it won't call System.exit. This means JVM will wait
> to call ShutdownHook until all non daemon threads are stopped which means
> infinite wait if we don't close all non-daemon threads cleanly.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3803: HBASE-26304: Reflect out of band locality improvements in metrics and balancer
Apache-HBase commented on pull request #3803: URL: https://github.com/apache/hbase/pull/3803#issuecomment-975613184 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | compile | 4m 33s | master passed | | +1 :green_heart: | checkstyle | 1m 45s | master passed | | +0 :ok: | refguide | 3m 57s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 3m 22s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 47s | the patch passed | | +1 :green_heart: | compile | 4m 31s | the patch passed | | +1 :green_heart: | javac | 4m 31s | the patch passed | | -0 :warning: | checkstyle | 1m 4s | hbase-server: The patch generated 1 new + 72 unchanged - 0 fixed = 73 total (was 72) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 3m 20s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 19m 36s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 3m 56s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 40s | The patch does not generate ASF License warnings. | | | | 64m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3803 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile refguide xml | | uname | Linux 6c3ae67a4b65 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | refguide | https://nightlies.apache.org/hbase/HBase/HBase-PreCommit-GitHub-PR/PR-3803/7/yetus-general-check/output/branch-site/book.html | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | refguide | https://nightlies.apache.org/hbase/HBase/HBase-PreCommit-GitHub-PR/PR-3803/7/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-common hbase-balancer hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3803/7/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26468) Region Server doesn't exit cleanly incase it crashes.
[
https://issues.apache.org/jira/browse/HBASE-26468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447451#comment-17447451
]
Viraj Jasani commented on HBASE-26468:
--
[~zhangduo] Although HBASE-26480 will have it's own fix, but I think we might
still want to make this change. Graceful exit of JVM with status code 0 is a
behaviour change, but I feel it's for the good.
With this behaviour, we might not be able to know of any other non-daemon
threads not shutting down properly but on the other hand, we will not see
zombie processes either (disallowing CD/Monitoring systems to automatically
start services when significant regionservers are affected).
> Region Server doesn't exit cleanly incase it crashes.
> -
>
> Key: HBASE-26468
> URL: https://issues.apache.org/jira/browse/HBASE-26468
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.6.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 1.7.2, 2.3.8, 2.4.9
>
>
> Observed this in our production cluster running 1.6 version.
> RS crashed due to some reason but the process was still running. On debugging
> more, found out there was 1 non-daemon thread running and that was not
> allowing RS to exit cleanly. Our clusters are managed by Ambari and have auto
> restart capability within them. But since the process was running and pid
> file was present, Ambari also couldn't do much. There will be some bug where
> we will miss to stop some non daemon thread. Shutdown hook will not be called
> unless one of the following 2 conditions are met:
> # The Java virtual machine shuts down in response to two kinds of events:
> The program exits normally, when the last non-daemon thread exits or when the
> exit (equivalently, System.exit) method is invoked, or
> # The virtual machine is terminated in response to a user interrupt, such as
> typing ^C, or a system-wide event, such as user logoff or system shutdown.
> Considering the first condition, when the last non-daemon thread exits or
> when the exit method is invoked.
> Below is the code snippet from
> [HRegionServerCommandLine.java|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServerCommandLine.java#L51]
> {code:java}
> private int start() throws Exception {
> try {
> if (LocalHBaseCluster.isLocal(conf)) {
> // Ignore this.
> } else {
> HRegionServer hrs =
> HRegionServer.constructRegionServer(regionServerClass, conf);
> hrs.start();
> hrs.join();
> if (hrs.isAborted()) {
> throw new RuntimeException("HRegionServer Aborted");
> }
> }
> } catch (Throwable t) {
> LOG.error("Region server exiting", t);
> return 1;
> }
> return 0;
> }
> {code}
> Within HRegionServer, there is a subtle difference between when a server is
> aborted v/s when it is stopped. If it is stopped, then isAborted will return
> false and it will exit with return code 0.
> Below is the code from
> [ServerCommandLine.java|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerCommandLine.java#L147]
> {code:java}
> public void doMain(String args[]) {
> try {
> int ret = ToolRunner.run(HBaseConfiguration.create(), this, args);
> if (ret != 0) {
> System.exit(ret);
> }
> } catch (Exception e) {
> LOG.error("Failed to run", e);
> System.exit(-1);
> }
> }
> {code}
> If return code is 0, then it won't call System.exit. This means JVM will wait
> to call ShutdownHook until all non daemon threads are stopped which means
> infinite wait if we don't close all non-daemon threads cleanly.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] bbeaudreault commented on a change in pull request #3803: HBASE-26304: Reflect out of band locality improvements in metrics and balancer
bbeaudreault commented on a change in pull request #3803:
URL: https://github.com/apache/hbase/pull/3803#discussion_r754340866
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/InputStreamBlockDistribution.java
##
@@ -0,0 +1,143 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.regionserver;
+
+import com.google.errorprone.annotations.RestrictedApi;
+import java.io.IOException;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.hbase.HDFSBlocksDistribution;
+import org.apache.hadoop.hbase.io.FileLink;
+import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
+import org.apache.hadoop.hbase.util.FSUtils;
+import org.apache.hadoop.hdfs.client.HdfsDataInputStream;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Computes the HDFSBlockDistribution for a file based on the underlying
located blocks
+ * for an HdfsDataInputStream reading that file. This computation may involve
a call to
+ * the namenode, so the value is cached based on
+ * {@link #HBASE_LOCALITY_INPUTSTREAM_DERIVE_CACHE_PERIOD}.
+ */
+@InterfaceAudience.Private
+public class InputStreamBlockDistribution {
+ private static final Logger LOG =
LoggerFactory.getLogger(InputStreamBlockDistribution.class);
+
+ private static final String HBASE_LOCALITY_INPUTSTREAM_DERIVE_ENABLED =
+"hbase.locality.inputstream.derive.enabled";
+ private static final boolean
DEFAULT_HBASE_LOCALITY_INPUTSTREAM_DERIVE_ENABLED = false;
+
+ private static final String HBASE_LOCALITY_INPUTSTREAM_DERIVE_CACHE_PERIOD =
+"hbase.locality.inputstream.derive.cache.period";
+ private static final int
DEFAULT_HBASE_LOCALITY_INPUTSTREAM_DERIVE_CACHE_PERIOD = 60_000;
+
+ private final FSDataInputStream stream;
+ private final StoreFileInfo fileInfo;
+ private final int cachePeriodMs;
+
+ private HDFSBlocksDistribution hdfsBlocksDistribution;
+ private long lastCachedAt;
+ private boolean streamUnsupported;
+
+ public InputStreamBlockDistribution(FSDataInputStream stream, StoreFileInfo
fileInfo) {
+this.stream = stream;
Review comment:
That's correct, I added comments and cleaned up checkstyle/compile
warnings.
Thanks again for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3831: HBASE-26249 Ameliorate compaction made by bul…
Apache-HBase commented on pull request #3831: URL: https://github.com/apache/hbase/pull/3831#issuecomment-975592483 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 1s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 45s | master passed | | +1 :green_heart: | compile | 3m 14s | master passed | | +1 :green_heart: | checkstyle | 1m 7s | master passed | | +1 :green_heart: | spotbugs | 2m 1s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | +1 :green_heart: | compile | 3m 12s | the patch passed | | +1 :green_heart: | javac | 3m 12s | the patch passed | | -0 :warning: | checkstyle | 1m 7s | hbase-server: The patch generated 1 new + 147 unchanged - 0 fixed = 148 total (was 147) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 24s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 50m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3831 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux e8daaef893f0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3869: HBASE-26476 Make DefaultMemStore extensible for HStore.memstore
Apache-HBase commented on pull request #3869: URL: https://github.com/apache/hbase/pull/3869#issuecomment-975582994 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 8s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 15s | master passed | | +1 :green_heart: | compile | 1m 16s | master passed | | +1 :green_heart: | shadedjars | 10m 27s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 4s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | +1 :green_heart: | shadedjars | 11m 25s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 252m 32s | hbase-server in the patch passed. | | | | 292m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3869 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 18f0275fac23 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/1/testReport/ | | Max. process+thread count | 2818 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nkalmar opened a new pull request #3872: HBASE-26340 - fix RegionSizeCalculator getLEngth to bytes instead of …
nkalmar opened a new pull request #3872: URL: https://github.com/apache/hbase/pull/3872 If we have store files under 1MB, list_regions will report region size of 0. This fix will use 1MB of value if there are store files available for a region. It makes more sense than using 0MB if store file size are less than 1MB. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] markjens commented on pull request #3860: HBASE-26470 Use openlabtesting protoc on linux arm64 in HBASE 2.x
markjens commented on pull request #3860: URL: https://github.com/apache/hbase/pull/3860#issuecomment-975575091 I did it before creating the PR. The build passed here! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26468) Region Server doesn't exit cleanly incase it crashes.
[
https://issues.apache.org/jira/browse/HBASE-26468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447429#comment-17447429
]
Rushabh Shah commented on HBASE-26468:
--
[~zhangduo] Created this jira with more details on which non daemon thread:
HBASE-26480
> Region Server doesn't exit cleanly incase it crashes.
> -
>
> Key: HBASE-26468
> URL: https://issues.apache.org/jira/browse/HBASE-26468
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.6.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2, 1.7.2, 2.3.8, 2.4.9
>
>
> Observed this in our production cluster running 1.6 version.
> RS crashed due to some reason but the process was still running. On debugging
> more, found out there was 1 non-daemon thread running and that was not
> allowing RS to exit cleanly. Our clusters are managed by Ambari and have auto
> restart capability within them. But since the process was running and pid
> file was present, Ambari also couldn't do much. There will be some bug where
> we will miss to stop some non daemon thread. Shutdown hook will not be called
> unless one of the following 2 conditions are met:
> # The Java virtual machine shuts down in response to two kinds of events:
> The program exits normally, when the last non-daemon thread exits or when the
> exit (equivalently, System.exit) method is invoked, or
> # The virtual machine is terminated in response to a user interrupt, such as
> typing ^C, or a system-wide event, such as user logoff or system shutdown.
> Considering the first condition, when the last non-daemon thread exits or
> when the exit method is invoked.
> Below is the code snippet from
> [HRegionServerCommandLine.java|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServerCommandLine.java#L51]
> {code:java}
> private int start() throws Exception {
> try {
> if (LocalHBaseCluster.isLocal(conf)) {
> // Ignore this.
> } else {
> HRegionServer hrs =
> HRegionServer.constructRegionServer(regionServerClass, conf);
> hrs.start();
> hrs.join();
> if (hrs.isAborted()) {
> throw new RuntimeException("HRegionServer Aborted");
> }
> }
> } catch (Throwable t) {
> LOG.error("Region server exiting", t);
> return 1;
> }
> return 0;
> }
> {code}
> Within HRegionServer, there is a subtle difference between when a server is
> aborted v/s when it is stopped. If it is stopped, then isAborted will return
> false and it will exit with return code 0.
> Below is the code from
> [ServerCommandLine.java|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerCommandLine.java#L147]
> {code:java}
> public void doMain(String args[]) {
> try {
> int ret = ToolRunner.run(HBaseConfiguration.create(), this, args);
> if (ret != 0) {
> System.exit(ret);
> }
> } catch (Exception e) {
> LOG.error("Failed to run", e);
> System.exit(-1);
> }
> }
> {code}
> If return code is 0, then it won't call System.exit. This means JVM will wait
> to call ShutdownHook until all non daemon threads are stopped which means
> infinite wait if we don't close all non-daemon threads cleanly.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (HBASE-26480) Close NamedQueueRecorder to allow HMaster/RS to shutdown gracefully
Rushabh Shah created HBASE-26480:
Summary: Close NamedQueueRecorder to allow HMaster/RS to shutdown
gracefully
Key: HBASE-26480
URL: https://issues.apache.org/jira/browse/HBASE-26480
Project: HBase
Issue Type: Bug
Affects Versions: 1.7.0
Reporter: Rushabh Shah
Assignee: Rushabh Shah
Saw one case in our production cluster where RS was not exiting. Saw this
non-daemon thread in hung RS stack trace:
{noformat}
"main.slowlog.append-pool-pool1-t1" #26 prio=5 os_prio=31
tid=0x7faf23bf7800 nid=0x6d07 waiting on condition [0x73f4d000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0004039e3840> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:47)
at
com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
at
com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:159)
at
com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
This is coming from
[NamedQueueRecorder|https://github.com/apache/hbase/blob/branch-1/hbase-server/src/main/java/org/apache/hadoop/hbase/namequeues/NamedQueueRecorder.java#L65]
implementation.
This bug doesn't exists in branch-2 and master since the Disruptor
initialization has changed and we set daemon=true also. See [this
code|https://github.com/apache/hbase/blob/branch-2/hbase-server/src/main/java/org/apache/hadoop/hbase/namequeues/NamedQueueRecorder.java#L68]
FYI [~vjasani] [~zhangduo]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache9 commented on a change in pull request #3870: HBASE-26479: Print too slow/big scan's operation_id in region server log
Apache9 commented on a change in pull request #3870:
URL: https://github.com/apache/hbase/pull/3870#discussion_r754298130
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
##
@@ -1290,6 +1292,12 @@ public String getScanDetailsWithRequest(ScanRequest
request) {
StringBuilder builder = new StringBuilder();
builder.append("table:
").append(region.getRegionInfo().getTable().getNameAsString());
builder.append(" region:
").append(region.getRegionInfo().getRegionNameAsString());
+ for (NameBytesPair pair : request.getScan().getAttributeList()) {
+if (OperationWithAttributes.ID_ATRIBUTE.equals(pair.getName())) {
Review comment:
ID_ATRIBUTE? Typo? Anyway, not your fault...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] nkalmar commented on pull request #3737: HBASE-26340 - fix RegionSizeCalculator getLEngth to bytes instead of …
nkalmar commented on pull request #3737: URL: https://github.com/apache/hbase/pull/3737#issuecomment-975544617 Closed this as I started fresh (force pushed auto-closed it to be precise) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #3864: HBASE-26416 Implement a new method for region replication instead of …
Apache9 merged pull request #3864: URL: https://github.com/apache/hbase/pull/3864 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26340) TableSplit returns false size under 1MB
[
https://issues.apache.org/jira/browse/HBASE-26340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447414#comment-17447414
]
Norbert Kalmár commented on HBASE-26340:
Made another patch, just tested it:
{code:java}
hbase:001:0> create 'emp', 'personal data', 'professional data'
2021-11-22T14:47:57,545 INFO [RPCClient-NioEventLoopGroup-1-2]
client.AsyncHBaseAdmin: Operation: CREATE, Table Name: default:emp completed
Created table emp
Took 1.5477 seconds
=> Hbase::Table - emp
hbase:002:0> list_regions 'emp'
SERVER_NAME |
REGION_NAME | START_KEY |END_KEY | SIZE | REQ | LOCALITY |
|
| -- | --
| - | - | -- |
192.168.1.75,16020,1637588865433 |
emp,,1637588876835.d6b9eaddeedbf2342c8e834a1151cc2a. ||
| 0.0MB | 0 |0.0 |
1 rows
Took 0.3317 seconds
hbase:003:0> put 'emp','1','personal data:name','raju'
Took 0.0650 seconds
hbase:004:0> flush 'emp'
Took 0.3037 seconds
hbase:005:0> list_regions 'emp'
SERVER_NAME |
REGION_NAME | START_KEY |END_KEY | SIZE | REQ | LOCALITY |
|
| -- | --
| - | - | -- |
192.168.1.75,16020,1637588865433 |
emp,,1637588876835.d6b9eaddeedbf2342c8e834a1151cc2a. ||
| 1.0MB | 1 |1.0 |
1 rows
Took 0.0175 seconds
{code}
> TableSplit returns false size under 1MB
> ---
>
> Key: HBASE-26340
> URL: https://issues.apache.org/jira/browse/HBASE-26340
> Project: HBase
> Issue Type: Bug
>Reporter: Norbert Kalmár
>Assignee: Norbert Kalmár
>Priority: Major
>
> We calculate region size in the mapreduce package by getting the size in MB
> first and multiplying:
> https://github.com/apache/hbase/blob/39a20c528e2bf27cedf12734dbdb1b7b1e538076/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/RegionSizeCalculator.java#L87
> This will give a size of 0 until at least 1MB is reached. (And it will have
> an unwanted rounding affect as well).
> Spark for example can be tuned to do some performance tuning by eliminating
> the 0 sized regions. This will eliminate any small regions which are not
> actually empty. The hadoop interface states the size is returned in bytes,
> and while this is true do to the multiplication, we multiply by 0 until 1MB
> is reached. I'm not sure why we get the size in MB units and not in bytes
> straight up.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3869: HBASE-26476 Make DefaultMemStore extensible for HStore.memstore
Apache-HBase commented on pull request #3869: URL: https://github.com/apache/hbase/pull/3869#issuecomment-975541684 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | -1 :x: | mvninstall | 5m 20s | root in master failed. | | +1 :green_heart: | compile | 1m 25s | master passed | | +1 :green_heart: | shadedjars | 9m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 46s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 10s | the patch passed | | +1 :green_heart: | compile | 1m 23s | the patch passed | | +1 :green_heart: | javac | 1m 23s | the patch passed | | +1 :green_heart: | shadedjars | 9m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 213m 25s | hbase-server in the patch passed. | | | | 253m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3869 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1ac3694e23e2 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-11.0.10+9 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/1/artifact/yetus-jdk11-hadoop3-check/output/branch-mvninstall-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/1/testReport/ | | Max. process+thread count | 3360 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 edited a comment on pull request #3862: HBASE-26468 Region Server doesn't exit cleanly incase it crashes.
shahrs87 edited a comment on pull request #3862: URL: https://github.com/apache/hbase/pull/3862#issuecomment-975538500 > The problem here is that, why a crashed region server could return zero here? Why it does not return a non zero value? I have added all the details in HBASE-26468 but adding here for the context. Within RS (atleast in branch-1.6) there is a subtle difference when RS is aborted vs when it is stopped. In this particular case, the RS was stopped. In HRegionServerCommandLine, if a server is aborted, then it throws RTE which in turns converts into System.exit(1). But if RS is stopped, it returns return code of 0. Sure we can make changes to this code to return -1 when RS is stopped so it behaves exactly alike when it is aborted. But there will be bugs in future when RS will exit with return code 0 when it should have return with non zero exit code. > A zero return value means a normal shutdown, so here we want all the threads clean themselves up and then exit gracefully. We can always wait for few seconds for threads to clean themselves up if RS exits with 0 return code. But we shouldn't wait infinitely for threads to clean themselves up. > Sorry for sticking on the behavior here, the modification is pretty simple but it is very foundamental... Don't be sorry. It is a good discussion and we should always try to understand the implications before we make any fundamental changes. @Apache9 Let me know what you think ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on pull request #3862: HBASE-26468 Region Server doesn't exit cleanly incase it crashes.
shahrs87 commented on pull request #3862: URL: https://github.com/apache/hbase/pull/3862#issuecomment-975538500 > The problem here is that, why a crashed region server could return zero here? Why it does not return a non zero value? I have added all the details in HBASE-26468 but adding here for the context. Within RS (atleast in branch-1.6) there is a subtle difference when RS is aborted vs when it is stopped. In this particular case, the RS was stopped. In HRegionServerCommandLine, if a server is aborted, then it throws RTE which in turns converts into System.exit(1). But if RS is stopped, it returns return code of 0. Sure we can make changes to this code to return -1 when RS is stopped so it behaves exactly alike when it is aborted. But there will be bugs in future when RS will exit with return code 0 when it should have return with non zero exit code. > A zero return value means a normal shutdown, so here we want all the threads clean themselves up and then exit gracefully. We can always wait for few seconds for threads to clean themselves up if RS exits with 0 return code. But we shouldn't wait infinitely for threads to clean themselves up. > Sorry for sticking on the behavior here, the modification is pretty simple but it is very foundamental... Don't be sorry. It is a good discussion and we should always try to understand the implications before we make any fundamental changes. @Apache9 Let me know what you think ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3831: HBASE-26249 Ameliorate compaction made by bul…
Apache-HBase commented on pull request #3831: URL: https://github.com/apache/hbase/pull/3831#issuecomment-975536834 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 5s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 45s | master passed | | +1 :green_heart: | compile | 1m 45s | master passed | | +1 :green_heart: | shadedjars | 10m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 55s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 1s | the patch passed | | +1 :green_heart: | compile | 1m 37s | the patch passed | | +1 :green_heart: | javac | 1m 37s | the patch passed | | +1 :green_heart: | shadedjars | 10m 27s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 237m 12s | hbase-server in the patch passed. | | | | 280m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3831 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 816d9ea36324 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/3/testReport/ | | Max. process+thread count | 2731 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3831/3/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3870: HBASE-26479: Print too slow/big scan's operation_id in region server log
Apache-HBase commented on pull request #3870: URL: https://github.com/apache/hbase/pull/3870#issuecomment-975524420 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | compile | 3m 17s | master passed | | +1 :green_heart: | checkstyle | 1m 4s | master passed | | +1 :green_heart: | spotbugs | 2m 5s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 52s | the patch passed | | +1 :green_heart: | compile | 3m 13s | the patch passed | | +1 :green_heart: | javac | 3m 13s | the patch passed | | -0 :warning: | checkstyle | 1m 3s | hbase-server: The patch generated 1 new + 32 unchanged - 0 fixed = 33 total (was 32) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 36s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 49m 29s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3870 | | JIRA Issue | HBASE-26479 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 8a8dcfd9ca18 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26467) Wrong Cell Generated by MemStoreLABImpl.forceCopyOfBigCellInto when Cell size bigger than data chunk size
[
https://issues.apache.org/jira/browse/HBASE-26467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447403#comment-17447403
]
Hudson commented on HBASE-26467:
Results for branch master
[build #448 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Wrong Cell Generated by MemStoreLABImpl.forceCopyOfBigCellInto when Cell size
> bigger than data chunk size
> --
>
> Key: HBASE-26467
> URL: https://issues.apache.org/jira/browse/HBASE-26467
> Project: HBase
> Issue Type: Bug
> Components: in-memory-compaction
>Reporter: zhuobin zheng
>Assignee: zhuobin zheng
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.9
>
>
> In our company 2.X cluster. I found some region compaction keeps failling
> because some cell can't construct succefully. In fact , we even can't read
> these cell.
> From follow stack , we can found the bug cause KeyValue can't constructed.
> Simple Log and Stack:
> {code:java}
> // code placeholder
> 2021-11-18 16:50:47,708 ERROR [regionserver/:60020-longCompactions-4]
> regionserver.CompactSplit: Compaction failed
> region=xx_table,3610ff49595a0fc4a824f2a575f37a31,1570874723992.dac703ceb35e8d8703233bebf34ae49f.,
> storeName=c, priority=-319, startTime=1637225447127
> java.lang.IllegalArgumentException: Invalid tag length at position=4659867,
> tagLength=0,
> at
> org.apache.hadoop.hbase.KeyValueUtil.checkKeyValueTagBytes(KeyValueUtil.java:685)
> at
> org.apache.hadoop.hbase.KeyValueUtil.checkKeyValueBytes(KeyValueUtil.java:643)
> at org.apache.hadoop.hbase.KeyValue.(KeyValue.java:345)
> at
> org.apache.hadoop.hbase.SizeCachedKeyValue.(SizeCachedKeyValue.java:43)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.getCell(HFileReaderImpl.java:981)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:233)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:418)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:322)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:288)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:487)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:248)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:318)
> at
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:65)
> at
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:126)
> at
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1468)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2266)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:624)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:666)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748) {code}
> From further observation, I found the following characteristics:
> # Cell size more than 2M
> # We can reproduce the bug only after in memory compact
> # Cell bytes end with \x00\x02\x00\x00
>
> In fact, the root reason is method (MemStoreLABImpl.forceCopyOfBigCellInto)
> which only invoked when cell bigger than data chunk size construct cell with
> wrong length. So there are 4 bytes (chunk head size) append end of
[jira] [Commented] (HBASE-24605) Break long region names in the web UI
[
https://issues.apache.org/jira/browse/HBASE-24605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447402#comment-17447402
]
Hudson commented on HBASE-24605:
Results for branch master
[build #448 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Break long region names in the web UI
> -
>
> Key: HBASE-24605
> URL: https://issues.apache.org/jira/browse/HBASE-24605
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Affects Versions: 3.0.0-alpha-1
>Reporter: song XinCun
>Assignee: song XinCun
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.2.6
>
> Attachments: image-2020-06-21-20-18-37-041.png,
> image-2020-06-21-20-19-25-183.png, image-2020-06-21-20-20-02-782.png,
> image-2020-06-21-20-27-23-474.png, image-2020-06-21-20-28-36-464.png,
> image-2020-06-21-20-29-07-819.png
>
>
> Before this patch, when it comes to the long region name, the UI content will
> be out of the screen, making it unreadable. Like this:
> !image-2020-06-21-20-18-37-041.png|width=542,height=50!
> !image-2020-06-21-20-19-25-183.png|width=531,height=23!
> !image-2020-06-21-20-20-02-782.png|width=542,height=146!
>
> After this patch, the long region name wil be break to the new line, like
> this:
> !image-2020-06-21-20-27-23-474.png|width=529,height=35!
> !image-2020-06-21-20-28-36-464.png|width=533,height=33!
> !image-2020-06-21-20-29-07-819.png|width=531,height=117!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26463) Unreadable table names after HBASE-24605
[
https://issues.apache.org/jira/browse/HBASE-26463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447401#comment-17447401
]
Hudson commented on HBASE-26463:
Results for branch master
[build #448 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/448/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unreadable table names after HBASE-24605
>
>
> Key: HBASE-26463
> URL: https://issues.apache.org/jira/browse/HBASE-26463
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 3.0.0-alpha-1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Trivial
> Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.9
>
> Attachments: After.png, Before.png
>
>
> During the fix HBASE-24605 (Break long region names in the web UI) the 'word
> break' rule was applied to the 'table-striped' style, so literally, all
> tables in Master Web UI were affected. The most noticeable is the 'User
> Tables' when table names, as well as descriptions, and even State are wrapped
> in a very ugly way.
> We have two options here:
> 1. Fix the user tables only as it was done previously for the procedures table
> 2. Apply 'word break' to those tables that require that.
> Since most of the users are comfortable with the current UI we may go through
> the first way, so the changes would affect only the User table.
> Sample screenshots before and after are attached.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache9 commented on pull request #3860: HBASE-26470 Use openlabtesting protoc on linux arm64 in HBASE 2.x
Apache9 commented on pull request #3860: URL: https://github.com/apache/hbase/pull/3860#issuecomment-975518542 Just apply the patch on branch-2, and try the build command. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] markjens commented on pull request #3860: HBASE-26470 Use openlabtesting protoc on linux arm64 in HBASE 2.x
markjens commented on pull request #3860: URL: https://github.com/apache/hbase/pull/3860#issuecomment-975486755 Screenshot of what exactly ? The end of Maven build output ? You can easily get an access to a Linux ARM64 at Oracle Cloud. You can spin off a VM with up to 4 vCPU @ 2.8GHz and 24GB RAM in their Free plan. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] liangxs opened a new pull request #3870: HBASE-26479: Print too slow/big scan's operation_id in region server log
liangxs opened a new pull request #3870: URL: https://github.com/apache/hbase/pull/3870 Tracing is very important in large-scale distributed systems. The attribute `_operation.attributes.id` of a scan request can be used as trace id between hbase client and region server. We should print operation id in region server's log if the scan request is too slow or too big. It will be very helpful for finding problematic requests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26479) Print too slow/big scan's operation_id in region server log
[ https://issues.apache.org/jira/browse/HBASE-26479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuesen Liang updated HBASE-26479: - Attachment: HBASE-26479.patch Status: Patch Available (was: Open) > Print too slow/big scan's operation_id in region server log > --- > > Key: HBASE-26479 > URL: https://issues.apache.org/jira/browse/HBASE-26479 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Xuesen Liang >Assignee: Xuesen Liang >Priority: Minor > Attachments: HBASE-26479.patch > > > Tracing is very important in large-scale distributed systems. > The attribute *__operation.attributes.id_* of a scan request can be used as > trace id between hbase client and region server. > We should print operation id in region server's log if the scan request is > too slow or too big. > It will be very helpful for finding problematic requests. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3868: HBASE-26475 The flush and compact methods in HTU should skip processi…
Apache-HBase commented on pull request #3868: URL: https://github.com/apache/hbase/pull/3868#issuecomment-975437920 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 1s | master passed | | +1 :green_heart: | compile | 1m 25s | master passed | | +1 :green_heart: | shadedjars | 8m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 53s | the patch passed | | +1 :green_heart: | compile | 1m 23s | the patch passed | | +1 :green_heart: | javac | 1m 23s | the patch passed | | +1 :green_heart: | shadedjars | 8m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 146m 7s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 1m 59s | hbase-testing-util in the patch passed. | | | | 180m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3868/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3868 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 24e78ff962ca 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3868/1/testReport/ | | Max. process+thread count | 4641 (vs. ulimit of 3) | | modules | C: hbase-server hbase-testing-util U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3868/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26479) Print too slow/big scan's operation_id in region server log
Xuesen Liang created HBASE-26479: Summary: Print too slow/big scan's operation_id in region server log Key: HBASE-26479 URL: https://issues.apache.org/jira/browse/HBASE-26479 Project: HBase Issue Type: Improvement Components: regionserver Reporter: Xuesen Liang Assignee: Xuesen Liang Tracing is very important in large-scale distributed systems. The attribute *__operation.attributes.id_* of a scan request can be used as trace id between hbase client and region server. We should print operation id in region server's log if the scan request is too slow or too big. It will be very helpful for finding problematic requests. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #3868: HBASE-26475 The flush and compact methods in HTU should skip processi…
Apache-HBase commented on pull request #3868: URL: https://github.com/apache/hbase/pull/3868#issuecomment-975432445 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 41s | master passed | | +1 :green_heart: | compile | 1m 38s | master passed | | +1 :green_heart: | shadedjars | 8m 20s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 27s | the patch passed | | +1 :green_heart: | compile | 1m 39s | the patch passed | | +1 :green_heart: | javac | 1m 39s | the patch passed | | +1 :green_heart: | shadedjars | 8m 19s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 136m 22s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 1m 49s | hbase-testing-util in the patch passed. | | | | 172m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3868/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3868 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6c2666f955c8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3868/1/testReport/ | | Max. process+thread count | 4054 (vs. ulimit of 3) | | modules | C: hbase-server hbase-testing-util U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3868/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26325) Backport HBASE-26305 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-26325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-26325. --- Fix Version/s: 1.7.2 Hadoop Flags: Reviewed Resolution: Fixed > Backport HBASE-26305 to branch-1 > > > Key: HBASE-26325 > URL: https://issues.apache.org/jira/browse/HBASE-26325 > Project: HBase > Issue Type: Improvement >Reporter: Yutong Xiao >Assignee: Yutong Xiao >Priority: Minor > Fix For: 1.7.2 > > > Backport the HBASE-26305 to branch-1. Move NavigableSet add operation to > writer thread in BucketCache -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Reidddddd merged pull request #3776: HBASE-26325 Backport the HBASE-26305 to branch-1. Move NavigableSet add operation to writer thread in BucketCache
Reidd merged pull request #3776: URL: https://github.com/apache/hbase/pull/3776 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly commented on a change in pull request #3831: HBASE-26249 Ameliorate compaction made by bul…
sunhelly commented on a change in pull request #3831: URL: https://github.com/apache/hbase/pull/3831#discussion_r754141891 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequester.java ## @@ -44,6 +44,12 @@ void requestCompaction(HRegion region, String why, int priority, void requestCompaction(HRegion region, HStore store, String why, int priority, CompactionLifeCycleTracker tracker, @Nullable User user) throws IOException; + /** + * Request system compaction on the given store. + */ + void requestSystemCompaction(HRegion region, HStore store, String why, boolean checkUnderCompact) Review comment: Got it, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] YutSean commented on pull request #3776: HBASE-26325 Backport the HBASE-26305 to branch-1. Move NavigableSet add operation to writer thread in BucketCache
YutSean commented on pull request #3776: URL: https://github.com/apache/hbase/pull/3776#issuecomment-975377318 The failed UT is timeout but not related to the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3869: HBASE-26476 Make DefaultMemStore extensible for HStore.memstore
Apache-HBase commented on pull request #3869: URL: https://github.com/apache/hbase/pull/3869#issuecomment-975376269 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | master passed | | +1 :green_heart: | compile | 3m 22s | master passed | | +1 :green_heart: | checkstyle | 1m 6s | master passed | | +1 :green_heart: | spotbugs | 2m 37s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | +1 :green_heart: | compile | 3m 19s | the patch passed | | +1 :green_heart: | javac | 3m 19s | the patch passed | | -0 :warning: | checkstyle | 1m 7s | hbase-server: The patch generated 3 new + 29 unchanged - 0 fixed = 32 total (was 29) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 20m 39s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 22s | The patch does not generate ASF License warnings. | | | | 52m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3869 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 088644ac1d82 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 04635f405a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3869/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] YutSean commented on pull request #3867: HBASE-26477 Fix MiniMapReduceCluster failure under Java17
YutSean commented on pull request #3867: URL: https://github.com/apache/hbase/pull/3867#issuecomment-975375074 It seems that the failed UT is not related to the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
