[jira] [Updated] (HBASE-22862) Region Server crash with: Added a key not lexically larger than previous
[
https://issues.apache.org/jira/browse/HBASE-22862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Hu updated HBASE-22862:
-
Attachment: HBASE-22862.UT.v02.patch
> Region Server crash with: Added a key not lexically larger than previous
>
>
> Key: HBASE-22862
> URL: https://issues.apache.org/jira/browse/HBASE-22862
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.10
> Environment: {code}
> openjdk version "1.8.0_181"
> OpenJDK Runtime Environment (Zulu 8.31.0.1-linux64) (build 1.8.0_181-b02)
> OpenJDK 64-Bit Server VM (Zulu 8.31.0.1-linux64) (build 25.181-b02, mixed
> mode)
> {code}
>Reporter: Alex Batyrshin
>Assignee: Zheng Hu
>Priority: Critical
> Attachments: HBASE-22862.UT.v01.patch, HBASE-22862.UT.v02.patch
>
>
> We observe error "Added a key not lexically larger than previous” that cause
> most of our region-servers to crash in our cluster.
> {code}
> 2019-08-15 18:02:10,554 INFO [MemStoreFlusher.0] regionserver.HRegion:
> Flushing 1/1 column families, memstore=56.08 MB
> 2019-08-15 18:02:10,727 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=0
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2200)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2125)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:512)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:482)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
>at java.lang.Thread.run(Thread.java:748)
> 2019-08-15 18:02:21,776 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=9
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2200)
>at
[jira] [Commented] (HBASE-22862) Region Server crash with: Added a key not lexically larger than previous
[
https://issues.apache.org/jira/browse/HBASE-22862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916458#comment-16916458
]
Zheng Hu commented on HBASE-22862:
--
Created a UT by testing it in the whole RPC path (patch.v02), it seems also
OK. Still don't find the way to reproduce the bug.
> Region Server crash with: Added a key not lexically larger than previous
>
>
> Key: HBASE-22862
> URL: https://issues.apache.org/jira/browse/HBASE-22862
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.10
> Environment: {code}
> openjdk version "1.8.0_181"
> OpenJDK Runtime Environment (Zulu 8.31.0.1-linux64) (build 1.8.0_181-b02)
> OpenJDK 64-Bit Server VM (Zulu 8.31.0.1-linux64) (build 25.181-b02, mixed
> mode)
> {code}
>Reporter: Alex Batyrshin
>Assignee: Zheng Hu
>Priority: Critical
> Attachments: HBASE-22862.UT.v01.patch, HBASE-22862.UT.v02.patch
>
>
> We observe error "Added a key not lexically larger than previous” that cause
> most of our region-servers to crash in our cluster.
> {code}
> 2019-08-15 18:02:10,554 INFO [MemStoreFlusher.0] regionserver.HRegion:
> Flushing 1/1 column families, memstore=56.08 MB
> 2019-08-15 18:02:10,727 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=0
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2200)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2125)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:512)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:482)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
>at java.lang.Thread.run(Thread.java:748)
> 2019-08-15 18:02:21,776 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=9
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regionserver.
[jira] [Commented] (HBASE-22862) Region Server crash with: Added a key not lexically larger than previous
[
https://issues.apache.org/jira/browse/HBASE-22862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916459#comment-16916459
]
Zheng Hu commented on HBASE-22862:
--
[~0x62ash], How did you write the row & delete the column into HBase ? would
you mind to share your hbase client desgin ? I think it would be helpful to
fix this bug.
> Region Server crash with: Added a key not lexically larger than previous
>
>
> Key: HBASE-22862
> URL: https://issues.apache.org/jira/browse/HBASE-22862
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.10
> Environment: {code}
> openjdk version "1.8.0_181"
> OpenJDK Runtime Environment (Zulu 8.31.0.1-linux64) (build 1.8.0_181-b02)
> OpenJDK 64-Bit Server VM (Zulu 8.31.0.1-linux64) (build 25.181-b02, mixed
> mode)
> {code}
>Reporter: Alex Batyrshin
>Assignee: Zheng Hu
>Priority: Critical
> Attachments: HBASE-22862.UT.v01.patch, HBASE-22862.UT.v02.patch
>
>
> We observe error "Added a key not lexically larger than previous” that cause
> most of our region-servers to crash in our cluster.
> {code}
> 2019-08-15 18:02:10,554 INFO [MemStoreFlusher.0] regionserver.HRegion:
> Flushing 1/1 column families, memstore=56.08 MB
> 2019-08-15 18:02:10,727 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=0
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2200)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2125)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:512)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:482)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
>at java.lang.Thread.run(Thread.java:748)
> 2019-08-15 18:02:21,776 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=9
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>a
[GitHub] [hbase] Apache-HBase commented on issue #543: HBASE-22922 Lock all regions to be merged in case of multi region merge (2.2)
Apache-HBase commented on issue #543: HBASE-22922 Lock all regions to be merged in case of multi region merge (2.2) URL: https://github.com/apache/hbase/pull/543#issuecomment-525172266 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 75 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +1 | mvninstall | 314 | branch-2.2 passed | | +1 | compile | 54 | branch-2.2 passed | | +1 | checkstyle | 79 | branch-2.2 passed | | +1 | shadedjars | 241 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 37 | branch-2.2 passed | | 0 | spotbugs | 181 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 179 | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 284 | the patch passed | | +1 | compile | 52 | the patch passed | | +1 | javac | 52 | the patch passed | | +1 | checkstyle | 74 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 237 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 880 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 34 | the patch passed | | +1 | findbugs | 167 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 14206 | hbase-server in the patch failed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 17324 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-543/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/543 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d69c87f23498 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-543/out/precommit/personality/provided.sh | | git revision | branch-2.2 / da72e2186b | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-543/4/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-543/4/testReport/ | | Max. process+thread count | 4301 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-543/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22862) Region Server crash with: Added a key not lexically larger than previous
[
https://issues.apache.org/jira/browse/HBASE-22862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916467#comment-16916467
]
Alex Batyrshin commented on HBASE-22862:
We use Phoenix-4.14.2 and don't have explicit delete, but have UPSERT with NULL
values that is translated to HBase delete.
> Region Server crash with: Added a key not lexically larger than previous
>
>
> Key: HBASE-22862
> URL: https://issues.apache.org/jira/browse/HBASE-22862
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.10
> Environment: {code}
> openjdk version "1.8.0_181"
> OpenJDK Runtime Environment (Zulu 8.31.0.1-linux64) (build 1.8.0_181-b02)
> OpenJDK 64-Bit Server VM (Zulu 8.31.0.1-linux64) (build 25.181-b02, mixed
> mode)
> {code}
>Reporter: Alex Batyrshin
>Assignee: Zheng Hu
>Priority: Critical
> Attachments: HBASE-22862.UT.v01.patch, HBASE-22862.UT.v02.patch
>
>
> We observe error "Added a key not lexically larger than previous” that cause
> most of our region-servers to crash in our cluster.
> {code}
> 2019-08-15 18:02:10,554 INFO [MemStoreFlusher.0] regionserver.HRegion:
> Flushing 1/1 column families, memstore=56.08 MB
> 2019-08-15 18:02:10,727 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=0
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2200)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2125)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:512)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:482)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
>at java.lang.Thread.run(Thread.java:748)
> 2019-08-15 18:02:21,776 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=9
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regions
[jira] [Comment Edited] (HBASE-22862) Region Server crash with: Added a key not lexically larger than previous
[
https://issues.apache.org/jira/browse/HBASE-22862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916467#comment-16916467
]
Alex Batyrshin edited comment on HBASE-22862 at 8/27/19 7:28 AM:
-
We use Phoenix-4.14.2 on table with secondary global index and don't have
explicit delete, but have UPSERT with NULL values that is translated to HBase
delete.
was (Author: 0x62ash):
We use Phoenix-4.14.2 on indexed table and don't have explicit delete, but have
UPSERT with NULL values that is translated to HBase delete.
> Region Server crash with: Added a key not lexically larger than previous
>
>
> Key: HBASE-22862
> URL: https://issues.apache.org/jira/browse/HBASE-22862
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.10
> Environment: {code}
> openjdk version "1.8.0_181"
> OpenJDK Runtime Environment (Zulu 8.31.0.1-linux64) (build 1.8.0_181-b02)
> OpenJDK 64-Bit Server VM (Zulu 8.31.0.1-linux64) (build 25.181-b02, mixed
> mode)
> {code}
>Reporter: Alex Batyrshin
>Assignee: Zheng Hu
>Priority: Critical
> Attachments: HBASE-22862.UT.v01.patch, HBASE-22862.UT.v02.patch
>
>
> We observe error "Added a key not lexically larger than previous” that cause
> most of our region-servers to crash in our cluster.
> {code}
> 2019-08-15 18:02:10,554 INFO [MemStoreFlusher.0] regionserver.HRegion:
> Flushing 1/1 column families, memstore=56.08 MB
> 2019-08-15 18:02:10,727 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=0
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2200)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2125)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:512)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:482)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
>at java.lang.Thread.run(Thread.java:748)
> 2019-08-15 18:02:21,776 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=9
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523
[jira] [Comment Edited] (HBASE-22862) Region Server crash with: Added a key not lexically larger than previous
[
https://issues.apache.org/jira/browse/HBASE-22862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916467#comment-16916467
]
Alex Batyrshin edited comment on HBASE-22862 at 8/27/19 7:28 AM:
-
We use Phoenix-4.14.2 on indexed table and don't have explicit delete, but have
UPSERT with NULL values that is translated to HBase delete.
was (Author: 0x62ash):
We use Phoenix-4.14.2 and don't have explicit delete, but have UPSERT with NULL
values that is translated to HBase delete.
> Region Server crash with: Added a key not lexically larger than previous
>
>
> Key: HBASE-22862
> URL: https://issues.apache.org/jira/browse/HBASE-22862
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.10
> Environment: {code}
> openjdk version "1.8.0_181"
> OpenJDK Runtime Environment (Zulu 8.31.0.1-linux64) (build 1.8.0_181-b02)
> OpenJDK 64-Bit Server VM (Zulu 8.31.0.1-linux64) (build 25.181-b02, mixed
> mode)
> {code}
>Reporter: Alex Batyrshin
>Assignee: Zheng Hu
>Priority: Critical
> Attachments: HBASE-22862.UT.v01.patch, HBASE-22862.UT.v02.patch
>
>
> We observe error "Added a key not lexically larger than previous” that cause
> most of our region-servers to crash in our cluster.
> {code}
> 2019-08-15 18:02:10,554 INFO [MemStoreFlusher.0] regionserver.HRegion:
> Flushing 1/1 column families, memstore=56.08 MB
> 2019-08-15 18:02:10,727 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=0
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2200)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2125)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:512)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:482)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
>at java.lang.Thread.run(Thread.java:748)
> 2019-08-15 18:02:21,776 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=9
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hba
[jira] [Commented] (HBASE-22923) hbase:meta is assigned to localhost when we downgrade the hbase version
[
https://issues.apache.org/jira/browse/HBASE-22923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916473#comment-16916473
]
wenbang commented on HBASE-22923:
-
When the nodes of system table group are all in the list of exclude nodes, can
we assign the system table across the group?
> hbase:meta is assigned to localhost when we downgrade the hbase version
> ---
>
> Key: HBASE-22923
> URL: https://issues.apache.org/jira/browse/HBASE-22923
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.8
>Reporter: wenbang
>Priority: Major
>
> When we downgrade the hbase version（rsgroup enable）, we found that the
> hbase:meta table could not be assigned.
> {code:java}
> master.AssignmentManager: Failed assignment of hbase:meta,,1.1588230740 to
> localhost,1,1, trying to assign elsewhere instead; try=1 of 10
> java.io.IOException: Call to localhost/127.0.0.1:1 failed on local exception:
> org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the
> failed servers list: localhost/127.0.0.1:1
> {code}
> hbase group list:
> HBASE_META group（hbase:meta and other system tables）
> default group
> 1.Down grade all servers in HBASE_META first
> 2.higher version servers is in default
> 3.hbase:meta assigned to localhost
> For system table, we assign them to a server with highest version.
> AssignmentManager#getExcludedServersForSystemTable
> But did not consider the rsgroup.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] ravowlga123 opened a new pull request #545: HBASE-22893 Change the comment in HBaseClassTestRule of test timeouts
ravowlga123 opened a new pull request #545: HBASE-22893 Change the comment in HBaseClassTestRule of test timeouts URL: https://github.com/apache/hbase/pull/545 HBASE-22893 Change the comment in HBaseClassTestRule to reflect change in default test timeouts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22912) [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads
[ https://issues.apache.org/jira/browse/HBASE-22912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-22912: - Summary: [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads (was: [Backport] HBASE-22867 to branch-1) > [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands > of threads > -- > > Key: HBASE-22912 > URL: https://issues.apache.org/jira/browse/HBASE-22912 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Zheng Hu >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22893) Change the comment in HBaseClassTestRule to reflect change in default test timeouts
[
https://issues.apache.org/jira/browse/HBASE-22893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916487#comment-16916487
]
Rabi Kumar K C commented on HBASE-22893:
Thank you for the response. I did check in the whole project but didn't find
any. I have edited the comment and made a git PR.
> Change the comment in HBaseClassTestRule to reflect change in default test
> timeouts
> ---
>
> Key: HBASE-22893
> URL: https://issues.apache.org/jira/browse/HBASE-22893
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Rabi Kumar K C
>Priority: Trivial
> Labels: beginner, noob
>
> The default timeout is 13 minutes. Change the comment accordingly.
> {code}
> // All tests have a 10minute timeout.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916497#comment-16916497
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #77 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/77/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/77//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/77//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/77//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (HBASE-22928) ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
Pankaj Kumar created HBASE-22928:
Summary: ScanMetrics counter update may not happen in case of
exception in TableRecordReaderImpl
Key: HBASE-22928
URL: https://issues.apache.org/jira/browse/HBASE-22928
Project: HBase
Issue Type: Bug
Components: mapreduce
Reporter: Pankaj Kumar
Assignee: Pankaj Kumar
ScanMetrics counter update may not happen in case of exception in
TableRecordReaderImpl.nextKeyValue(),
There are multiple cases as below,
1. In case of DoNotRetryIOException
{code}
// do not retry if the exception tells us not to do so
if (e instanceof DoNotRetryIOException) {
throw e;
}
{code}
2. While creating a new scanner,
{code}
if (lastSuccessfulRow == null) {
restart(scan.getStartRow());
} else {
restart(lastSuccessfulRow);
scanner.next(); // skip presumed already mapped row
}
{code}
3. In case of IOException
{code}
catch (IOException ioe) {
if (logScannerActivity) {
long now = System.currentTimeMillis();
LOG.info("Mapper took " + (now-timestamp)
+ "ms to process " + rowcount + " rows");
LOG.info(ioe.toString(), ioe);
String lastRow = lastSuccessfulRow == null ?
"null" : Bytes.toStringBinary(lastSuccessfulRow);
LOG.info("lastSuccessfulRow=" + lastRow);
}
throw ioe;
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #251: HBASE-22114 Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1
Apache-HBase commented on issue #251: HBASE-22114 Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1 URL: https://github.com/apache/hbase/pull/251#issuecomment-525192550 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 40 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | -1 | hbaseanti | 0 | The patch appears use Hadoop classification instead of HBase. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ branch-1 Compile Tests _ | | 0 | mvndep | 364 | Maven dependency ordering for branch | | +1 | mvninstall | 204 | branch-1 passed | | +1 | compile | 113 | branch-1 passed | | +1 | checkstyle | 539 | branch-1 passed | | 0 | refguide | 208 | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | -1 | shadedjars | 14 | branch has 10 errors when building our shaded downstream artifacts. | | +1 | javadoc | 214 | branch-1 passed | | 0 | spotbugs | 525 | Used deprecated FindBugs config; considering switching to SpotBugs. | | 0 | findbugs | 18 | branch/hbase-resource-bundle no findbugs output file (findbugsXml.xml) | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 120 | the patch passed | | +1 | compile | 115 | the patch passed | | +1 | javac | 115 | the patch passed | | +1 | checkstyle | 8 | The patch passed checkstyle in hbase-resource-bundle | | +1 | checkstyle | 27 | The patch passed checkstyle in hbase-common | | +1 | checkstyle | 106 | hbase-server: The patch generated 0 new + 83 unchanged - 11 fixed = 83 total (was 94) | | -1 | checkstyle | 10 | hbase-tinylfu-blockcache: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -1 | checkstyle | 366 | root: The patch generated 1 new + 92 unchanged - 11 fixed = 93 total (was 103) | | +1 | checkstyle | 27 | The patch passed checkstyle in hbase-it | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 7 | The patch has no ill-formed XML file. | | 0 | refguide | 213 | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | -1 | shadedjars | 15 | patch has 10 errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 348 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 | javadoc | 264 | the patch passed | | 0 | findbugs | 10 | hbase-resource-bundle has no data from findbugs | ||| _ Other Tests _ | | +1 | unit | 10174 | root in the patch passed. | | +1 | asflicense | 118 | The patch does not generate ASF License warnings. | | | | 15586 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/15/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/251 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml | | uname | Linux 1eb949cf291a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-251/out/precommit/personality/provided.sh | | git revision | branch-1 / f240ca0 | | Default Java | 1.8.0_222 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/15/artifact/out/branch-site/book.html | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/15/artifact/out/branch-shadedjars.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/15/artifact/out/diff-checkstyle-hbase-tinylfu-blockcache.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/15/artifact/out/diff-checkstyle-root.txt | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/15/artifact/out/patch-site/book.html | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/15/artifact/out/patch-shadedjars.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/15/testReport/ | | Max. process+thread count | 3888 (vs. ulimit of 1) | | modules | C: hbase-resource-bundle hbase-common hbase-server hbase-tinylfu-blockcache . hbase-it U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-251/15/console | | versions | git=1.9.1 maven=3.0.5 findbugs=
[jira] [Commented] (HBASE-22380) break circle replication when doing bulkload
[
https://issues.apache.org/jira/browse/HBASE-22380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916507#comment-16916507
]
Wellington Chevreuil commented on HBASE-22380:
--
[~anoop.hbase]
{quote}I mean why the user/client need to pass clusterId? When the bulk load
req comes to the 1st cluster, that RS can add its clusterId in the WALEdit cell.
{quote}
We are already doing that, in a way. User/client who first triggers the bulk
load (calling LoadIncrementalHFiles class) does not set the cluster Id. This is
set only at server side, when the bulk load reaches 1st cluster, as you
mentioned above. But we can't set it on the WAL edit, because bulk load
replication does not send WAL edits to peer clusters. Once bulk load happens,
it does generate a single special WAL edit at the original cluster only.
Replication source reads that, then triggers a bulk load on the peer cluster
directly (it never ships any wal for the original bulk load event in the
source). That's why we had to put cluster id in the bullk load request.
{quote}Only passing the 1st clusterId through all the clusters might not be
enough. We need to add current clusterId into the existing list while passing
to next. And same way check to see whether this cluster already saw/handled
this replication or not.
{quote}
I can't yet see the need for such, but let me try change current test to
include a 3rd cluster in the circle. Or do you mean, to avoid replication if
same hfiles are bulkloaded again on original cluster?
{quote}There's a scenario we should consider about, when MOB feature enabled,
after compaction PartitionedMobCompactor#bulkloadRefFile will be called
{quote}
Let me review that scenario as well.
> break circle replication when doing bulkload
>
>
> Key: HBASE-22380
> URL: https://issues.apache.org/jira/browse/HBASE-22380
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.0.5, 2.3.0, 2.1.5, 1.3.5
>Reporter: chenxu
>Assignee: Wellington Chevreuil
>Priority: Critical
> Labels: bulkload
> Fix For: 3.0.0, 1.5.0, 2.3.0, 1.4.11, 2.1.7, 2.2.2
>
>
> when enabled master-master bulkload replication, HFiles will be replicated
> circularly between two clusters
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] pankaj72981 opened a new pull request #546: HBASE-22928 ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl
pankaj72981 opened a new pull request #546: HBASE-22928 ScanMetrics counter update may not happen in case of exception in TableRecordReaderImpl URL: https://github.com/apache/hbase/pull/546 Updating the counter in multiple exception scenario. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ravowlga123 opened a new pull request #547: HBASE-22899 logging improvements for snapshot operations w/large manifests
ravowlga123 opened a new pull request #547: HBASE-22899 logging improvements for snapshot operations w/large manifests URL: https://github.com/apache/hbase/pull/547 HBASE-22899 logging improvements for snapshot operations w/large manifests This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317922121 ## File path: hbase-hbtop/README.md ## @@ -0,0 +1,246 @@ + + +# hbtop + +## Overview + +`hbtop` is a real-time monitoring tool for HBase like Unix's top command. +It can display summary information as well as metrics per Region/Namespace/Table/RegionServer. +In this tool, you can see the metrics sorted by a selected field and filter the metrics to see only metrics you really want to see. +Also, with the drill-down feature, you can find hot regions easily in a top-down manner. + +## Usage + +You can run hbtop with the following command: + +``` +$ hbase top +``` + +Also, you can specify your own zookeeper quorum and znode parent as follows: Review comment: The values of `hbase.client.zookeeper.quorum` and `zookeeper.znode.parent` in hbase-site.xml in the classpath or the default values would be used. That's the same as the other hbase commands I think. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317946197
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/terminal/impl/KeyPressGenerator.java
##
@@ -0,0 +1,477 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop.terminal.impl;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.Reader;
+import java.nio.charset.StandardCharsets;
+import java.util.Queue;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import org.apache.hadoop.hbase.hbtop.terminal.KeyPress;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * This generates {@link KeyPress} objects from the given input stream and
offers them to the
+ * given queue.
+ */
+@InterfaceAudience.Private
+public class KeyPressGenerator {
+
+ private static final Logger LOGGER =
LoggerFactory.getLogger(KeyPressGenerator.class);
+
+ private enum ParseState {
+START, ESCAPE, ESCAPE_SEQUENCE_PARAM1, ESCAPE_SEQUENCE_PARAM2
+ }
+
+ private final Queue keyPressQueue;
+ private final BlockingQueue inputCharacterQueue = new
LinkedBlockingQueue<>();
+ private final Reader input;
+ private final InputStream inputStream;
+ private final AtomicBoolean stopThreads = new AtomicBoolean();
+ private final Thread readerThread = new Thread(this::readerThread);
+ private final Thread generatorThread = new Thread(this::generatorThread);
+
+ private ParseState parseState;
+ private int param1;
+ private int param2;
+
+ public KeyPressGenerator(InputStream inputStream, Queue
keyPressQueue) {
+this.inputStream = inputStream;
+input = new InputStreamReader(inputStream, StandardCharsets.UTF_8);
+this.keyPressQueue = keyPressQueue;
+
+initState();
+ }
+
+ public void start() {
+readerThread.start();
Review comment:
Okay, will change to use an ExecutorService. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317944079
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/terminal/TerminalSize.java
##
@@ -0,0 +1,61 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop.terminal;
+
+import java.util.Objects;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Terminal dimensions in 2-d space, measured in number of rows and columns.
+ */
+@InterfaceAudience.Private
+public class TerminalSize {
Review comment:
Yes, `TerminalSize` and `CursorPosition` are very similar. However, I think
the role is different as `TerminalSize` represents a size and `CursorPosition`
represents a position. Also, if unifying the 2 classes, I can't think of a good
name. What do you think?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317932301
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/field/FieldValueType.java
##
@@ -0,0 +1,29 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop.field;
+
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Represents the type of a {@link FieldValue}.
+ */
+@InterfaceAudience.Private
+public enum FieldValueType {
+ STRING, INTEGER, LONG, FLOAT, SIZE, PERCENT
Review comment:
Do I need to change here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317931219
##
File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/Record.java
##
@@ -0,0 +1,180 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop;
+
+import java.util.Collection;
+import java.util.Collections;
+import java.util.EnumMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.validation.constraints.NotNull;
+import org.apache.hadoop.hbase.hbtop.field.Field;
+import org.apache.hadoop.hbase.hbtop.field.FieldValue;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Represents a record of the metrics in the top screen.
+ */
+@InterfaceAudience.Private
+public class Record implements Map {
+
+ private Map values;
+
+ public static final class Entry implements Map.Entry {
+private final Field key;
+private FieldValue value;
+
+private Entry(Field key, FieldValue value) {
+ this.key = Objects.requireNonNull(key);
+ this.value = Objects.requireNonNull(value);
+}
+
+@Override
+public Field getKey() {
+ return key;
+}
+
+@Override
+public FieldValue getValue() {
+ return value;
+}
+
+@Override
+public FieldValue setValue(FieldValue value) {
+ FieldValue oldValue = this.value;
+ this.value = value;
+ return oldValue;
+}
+ }
+
+ public static Entry entry(Field field, Object value) {
+return new Entry(field, field.newValue(value));
+ }
+
+ public static Entry entry(Field field, FieldValue value) {
+return new Entry(field, value);
+ }
+
+ public static Record ofEntries(List entries) {
+return ofEntries(entries.stream());
+ }
+
+ public static Record ofEntries(Entry... entries) {
+return ofEntries(Stream.of(entries));
+ }
+
+ public static Record ofEntries(Stream entries) {
+return entries
+ .collect(Record::new, (r, e) -> r.put(e.getKey(), e.getValue()), (r1,
r2) -> {});
+ }
+
+ public Record() {
+this(new EnumMap<>(Field.class));
+ }
+
+ private Record(Map values) {
+this.values = values;
+ }
+
+ @Override
+ public int size() {
+return values.size();
+ }
+
+ @Override
+ public boolean isEmpty() {
+return values.isEmpty();
+ }
+
+ @Override
+ public boolean containsKey(Object key) {
+return values.containsKey(key);
+ }
+
+ @Override
+ public boolean containsValue(Object value) {
+return values.containsValue(value);
+ }
+
+ @Override
+ public FieldValue get(Object key) {
+return values.get(key);
+ }
+
+ @Override
+ public FieldValue put(Field key, FieldValue value) {
+return values.put(key, value);
+ }
+
+ public FieldValue put(Field key, Object value) {
+return values.put(key, key.newValue(value));
+ }
+
+ @Override
+ public FieldValue remove(Object key) {
+return values.remove(key);
+ }
+
+ @Override
+ public void putAll(@NotNull Map m) {
+values.putAll(m);
+ }
+
+ @Override
+ public void clear() {
+values.clear();
Review comment:
Will make this object immutable, thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317932067
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/RecordFilter.java
##
@@ -0,0 +1,336 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import org.apache.hadoop.hbase.hbtop.field.Field;
+import org.apache.hadoop.hbase.hbtop.field.FieldValue;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Represents a filter that's filtering the metric {@link Record}s.
+ */
+@InterfaceAudience.Private
+public final class RecordFilter {
+
+ private enum Operator {
+EQUAL("="),
+DOUBLE_EQUALS("=="),
+GREATER(">"),
+GREATER_OR_EQUAL(">="),
+LESS("<"),
+LESS_OR_EQUAL("<=");
+
+private final String operator;
+
+Operator(String operator) {
+ this.operator = operator;
+}
+
+@Override
+public String toString() {
+ return operator;
+}
+ }
Review comment:
Do I need to change here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317931848
##
File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/Record.java
##
@@ -0,0 +1,180 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop;
+
+import java.util.Collection;
+import java.util.Collections;
+import java.util.EnumMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.validation.constraints.NotNull;
+import org.apache.hadoop.hbase.hbtop.field.Field;
+import org.apache.hadoop.hbase.hbtop.field.FieldValue;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Represents a record of the metrics in the top screen.
+ */
+@InterfaceAudience.Private
+public class Record implements Map {
+
+ private Map values;
+
+ public static final class Entry implements Map.Entry {
+private final Field key;
+private FieldValue value;
+
+private Entry(Field key, FieldValue value) {
+ this.key = Objects.requireNonNull(key);
+ this.value = Objects.requireNonNull(value);
+}
+
+@Override
+public Field getKey() {
+ return key;
+}
+
+@Override
+public FieldValue getValue() {
+ return value;
+}
+
+@Override
+public FieldValue setValue(FieldValue value) {
+ FieldValue oldValue = this.value;
+ this.value = value;
+ return oldValue;
+}
+ }
+
+ public static Entry entry(Field field, Object value) {
+return new Entry(field, field.newValue(value));
+ }
+
+ public static Entry entry(Field field, FieldValue value) {
+return new Entry(field, value);
+ }
+
+ public static Record ofEntries(List entries) {
+return ofEntries(entries.stream());
+ }
+
+ public static Record ofEntries(Entry... entries) {
+return ofEntries(Stream.of(entries));
+ }
+
+ public static Record ofEntries(Stream entries) {
+return entries
+ .collect(Record::new, (r, e) -> r.put(e.getKey(), e.getValue()), (r1,
r2) -> {});
+ }
+
+ public Record() {
+this(new EnumMap<>(Field.class));
+ }
+
+ private Record(Map values) {
+this.values = values;
+ }
+
+ @Override
+ public int size() {
+return values.size();
+ }
+
+ @Override
+ public boolean isEmpty() {
+return values.isEmpty();
+ }
+
+ @Override
+ public boolean containsKey(Object key) {
+return values.containsKey(key);
+ }
+
+ @Override
+ public boolean containsValue(Object value) {
+return values.containsValue(value);
+ }
+
+ @Override
+ public FieldValue get(Object key) {
+return values.get(key);
+ }
+
+ @Override
+ public FieldValue put(Field key, FieldValue value) {
+return values.put(key, value);
+ }
+
+ public FieldValue put(Field key, Object value) {
+return values.put(key, key.newValue(value));
+ }
+
+ @Override
+ public FieldValue remove(Object key) {
+return values.remove(key);
+ }
+
+ @Override
+ public void putAll(@NotNull Map m) {
+values.putAll(m);
+ }
+
+ @Override
+ public void clear() {
+values.clear();
+ }
+
+ @Override
+ @NotNull
+ public Set keySet() {
+return values.keySet();
+ }
+
+ @Override
+ @NotNull
+ public Collection values() {
+return values.values();
+ }
+
+ @Override
+ @NotNull
+ public Set> entrySet() {
+return values.entrySet();
+ }
+
+ public Record combine(Record o) {
+return ofEntries(values.keySet().stream()
+ .map(k -> {
+switch (k.getFieldValueType()) {
+ case STRING:
+return entry(k, values.get(k));
+ default:
+return entry(k, values.get(k).plus(o.values.get(k)));
+}
+ }));
+ }
+
+ public Record toImmutable() {
+return new Record(Collections.unmodifiableMap(values));
+ }
+}
Review comment:
I think we can use ImmutableMap in Guava here. Will modify to use it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For quer
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317926127 ## File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/HBTop.java ## @@ -0,0 +1,141 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.hbtop; + +import java.io.InputStream; +import java.util.Objects; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.conf.Configured; +import org.apache.hadoop.hbase.HBaseConfiguration; +import org.apache.hadoop.hbase.hbtop.mode.Mode; +import org.apache.hadoop.hbase.hbtop.screen.Screen; +import org.apache.hadoop.util.Tool; +import org.apache.hadoop.util.ToolRunner; +import org.apache.log4j.PropertyConfigurator; Review comment: Yes, I know references to log4j directly is not good and we should use Logger via slf4j. But what I want to do here is changing log level forcibly because I want to suppress logging spam which messes up the console rendering. It looks like slf4j itself doesn't have a way to change log level so I had to use log4j directly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317937751
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/screen/top/TopScreenModel.java
##
@@ -0,0 +1,202 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop.screen.top;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.time.DateFormatUtils;
+import org.apache.hadoop.hbase.ClusterMetrics;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.hbtop.Record;
+import org.apache.hadoop.hbase.hbtop.RecordFilter;
+import org.apache.hadoop.hbase.hbtop.field.Field;
+import org.apache.hadoop.hbase.hbtop.field.FieldInfo;
+import org.apache.hadoop.hbase.hbtop.field.FieldValue;
+import org.apache.hadoop.hbase.hbtop.mode.DrillDownInfo;
+import org.apache.hadoop.hbase.hbtop.mode.Mode;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * The data and business logic for the top screen.
+ */
+@InterfaceAudience.Private
+public class TopScreenModel {
+
+ private static final Logger LOGGER =
LoggerFactory.getLogger(TopScreenModel.class);
+
+ private final Admin admin;
+
+ private Mode currentMode;
+ private Field currentSortField;
+ private List fieldInfos;
+ private List fields;
+
+ private Summary summary;
+ private List records;
+
+ private final List filters = new ArrayList<>();
+ private final List filterHistories = new ArrayList<>();
+
+ private boolean ascendingSort;
+
+ public TopScreenModel(Admin admin, Mode initialMode) {
+this.admin = Objects.requireNonNull(admin);
+switchMode(Objects.requireNonNull(initialMode), null, false);
+ }
+
+ public void switchMode(Mode nextMode, List initialFilters,
+boolean keepSortFieldAndSortOrderIfPossible) {
+
+currentMode = nextMode;
+fieldInfos = Collections.unmodifiableList(new
ArrayList<>(currentMode.getFieldInfos()));
+fields = Collections.unmodifiableList(new
ArrayList<>(currentMode.getFieldInfos().stream()
+ .map(FieldInfo::getField).collect(Collectors.toList(;
+
+if (keepSortFieldAndSortOrderIfPossible) {
+ boolean match = fields.stream().anyMatch(f -> f == currentSortField);
+ if (!match) {
+currentSortField = nextMode.getDefaultSortField();
+ascendingSort = false;
+ }
+} else {
+ currentSortField = nextMode.getDefaultSortField();
+ ascendingSort = false;
+}
+
+clearFilters();
+if (initialFilters != null) {
+ filters.addAll(initialFilters);
+}
+ }
+
+ public void setSortFieldAndFields(Field sortField, List fields) {
+this.currentSortField = sortField;
+this.fields = Collections.unmodifiableList(new ArrayList<>(fields));
+ }
+
+ public void refreshMetricsData() {
Review comment:
> it seems like we only call it from the "main thread" of the application.
Yes. That's correct. In the current implementation, only the main thread
calls the method. So we don't need synchronization things here now.
> Assuming I got it right, I think a comment here explaining that "HBTop
only calls this from a single thread, and if that ever changes, this needs
synchronization" would be really helpful!
Thanks. That would be really helpful. Will add this comment.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317946868 ## File path: hbase-hbtop/src/main/resources/log4j-hbtop.properties ## @@ -0,0 +1,24 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +log4j.rootLogger=WARN,console Review comment: Okay, will add the log level setting. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317935817
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/screen/field/FieldScreenPresenter.java
##
@@ -0,0 +1,186 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop.screen.field;
+
+import java.util.ArrayList;
+import java.util.EnumMap;
+import java.util.List;
+import java.util.Objects;
+
+import org.apache.hadoop.hbase.hbtop.field.Field;
+import org.apache.hadoop.hbase.hbtop.screen.ScreenView;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * The presentation logic for the field screen.
+ */
+@InterfaceAudience.Private
+public class FieldScreenPresenter {
+
+ @FunctionalInterface
+ public interface ResultListener {
+void accept(Field sortField, List fields, EnumMap
fieldDisplayMap);
+ }
+
+ private final FieldScreenView fieldScreenView;
+ private Field sortField;
+ private final List fields;
+ private final EnumMap fieldDisplayMap;
+ private final ResultListener resultListener;
+ private final ScreenView nextScreenView;
+
+ private final int headerMaxLength;
+ private final int descriptionMaxLength;
+
+ private int currentPosition;
+ private boolean moveMode;
+
+ public FieldScreenPresenter(FieldScreenView fieldScreenView, Field
sortField, List fields,
+EnumMap fieldDisplayMap, ResultListener resultListener,
+ScreenView nextScreenView) {
+this.fieldScreenView = Objects.requireNonNull(fieldScreenView);
+this.sortField = Objects.requireNonNull(sortField);
+this.fields = new ArrayList<>(Objects.requireNonNull(fields));
+this.fieldDisplayMap = new
EnumMap<>(Objects.requireNonNull(fieldDisplayMap));
+this.resultListener = Objects.requireNonNull(resultListener);
+this.nextScreenView = Objects.requireNonNull(nextScreenView);
+
+int headerLength = 0;
+int descriptionLength = 0;
+for (int i = 0; i < fields.size(); i ++) {
+ Field field = fields.get(i);
+
+ if (field == sortField) {
Review comment:
Yes. `Field` is an enum type (singleton), so we can just use `==` operator
to compare.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317952124 ## File path: hbase-hbtop/src/main/resources/log4j-hbtop.properties ## @@ -0,0 +1,24 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +log4j.rootLogger=WARN,console Review comment: I wanted to avoid conflicting between a default log4j configuration file (`log4j.properties`) and this file (`log4j-hbtop.properties`). If this file is named `log4j.properties` and another `log4j.properties` exists in the classpath, we are not sure which file is read (maybe it depends on the order of loading the files). So I thought we needed a unique name for this file. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317920128 ## File path: hbase-hbtop/README.md ## @@ -0,0 +1,246 @@ + + +# hbtop + +## Overview + +`hbtop` is a real-time monitoring tool for HBase like Unix's top command. +It can display summary information as well as metrics per Region/Namespace/Table/RegionServer. +In this tool, you can see the metrics sorted by a selected field and filter the metrics to see only metrics you really want to see. +Also, with the drill-down feature, you can find hot regions easily in a top-down manner. + +## Usage + +You can run hbtop with the following command: + +``` +$ hbase top +``` + +Also, you can specify your own zookeeper quorum and znode parent as follows: + +``` +$ hbase top -Dhbase.client.zookeeper.quorum= -Dzookeeper.znode.parent= +``` + + + +The top screen consists of a summary part and of a metrics part. +In the summary part, you can see `HBase Version`, `Cluster ID`, `The number of region servers`, `Region count`, `Average Cluster Load` and `Aggregated Request/s`. +In the metrics part, you can see metrics per Region/Namespace/Table/RegionServer depending on the selected mode. +The top screen is refreshed in a certain period – 3 seconds by default. + +### Scrolling metric records + +You can scroll the metric records in the metrics part. + + + +### Command line arguments + +| Argument | Description | +|---|---| +| -delay| The refresh delay (in seconds); default is 3 seconds | +| -help | Print usage; for help while the tool is running press `h` key | +| -mode | The mode; `n` (Namespace)|`t` (Table)|r (Region)|`s` (RegionServer), default is `r` (Region) | + +### Modes + +There are the following 4 modes in hbtop: + +| Mode | Description | Review comment: That would be really nice. Thank you for the feedback. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317931117
##
File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/Record.java
##
@@ -0,0 +1,180 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop;
+
+import java.util.Collection;
+import java.util.Collections;
+import java.util.EnumMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Stream;
+import javax.validation.constraints.NotNull;
+import org.apache.hadoop.hbase.hbtop.field.Field;
+import org.apache.hadoop.hbase.hbtop.field.FieldValue;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Represents a record of the metrics in the top screen.
+ */
+@InterfaceAudience.Private
+public class Record implements Map {
+
+ private Map values;
+
+ public static final class Entry implements Map.Entry {
+private final Field key;
+private FieldValue value;
Review comment:
Will make this object immutable, thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317945353
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/terminal/impl/EscapeSequences.java
##
@@ -0,0 +1,140 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop.terminal.impl;
+
+import org.apache.hadoop.hbase.hbtop.terminal.Color;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Utility class for escape sequences.
+ */
+@InterfaceAudience.Private
+public final class EscapeSequences {
+ private EscapeSequences() {
+ }
+
+ public static String clearAll() {
+return "\033[0;37;40m\033[2J";
+ }
+
+ public static String setTitle(String title) {
+return "\033]2;" + title + "\007";
+ }
+
+ public static String cursor(boolean on) {
+if (on) {
+ return "\033[?25h";
+}
+return "\033[?25l";
+ }
+
+ public static String moveCursor(int column, int row) {
+return String.format("\033[%d;%dH", row + 1, column + 1);
+ }
+
+ public static String clearRemainingLine() {
+return "\033[0;37;40m\033[K";
+ }
+
+ public static String color(Color foregroundColor, Color backgroundColor,
boolean bold,
+boolean reverse, boolean blink, boolean underline) {
+
+int foregroundColorValue = getColorValue(foregroundColor, true);
+int backgroundColorValue = getColorValue(backgroundColor, false);
+
+StringBuilder sb = new StringBuilder();
+if (bold && reverse && blink && !underline) {
+ sb.append("\033[0;1;7;5;");
+} else if (bold && reverse && !blink && !underline) {
+ sb.append("\033[0;1;7;");
+} else if (!bold && reverse && blink && !underline) {
+ sb.append("\033[0;7;5;");
+} else if (bold && !reverse && blink && !underline) {
+ sb.append("\033[0;1;5;");
+} else if (bold && !reverse && !blink && !underline) {
+ sb.append("\033[0;1;");
+} else if (!bold && reverse && !blink && !underline) {
+ sb.append("\033[0;7;");
+} else if (!bold && !reverse && blink && !underline) {
+ sb.append("\033[0;5;");
+} else if (bold && reverse && blink /* && underline */) {
Review comment:
I intended that they were just for readability. Will remove the comments.
Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317934725
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/mode/RegionModeStrategy.java
##
@@ -0,0 +1,183 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop.mode;
+
+import edu.umd.cs.findbugs.annotations.Nullable;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.commons.lang3.time.FastDateFormat;
+import org.apache.hadoop.hbase.ClusterMetrics;
+import org.apache.hadoop.hbase.RegionMetrics;
+import org.apache.hadoop.hbase.ServerMetrics;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.hbtop.Record;
+import org.apache.hadoop.hbase.hbtop.field.Field;
+import org.apache.hadoop.hbase.hbtop.field.FieldInfo;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Implementation for {@link ModeStrategy} for Region Mode.
+ */
+@InterfaceAudience.Private
+public final class RegionModeStrategy implements ModeStrategy {
+
+ private final List fieldInfos = Arrays.asList(
+new FieldInfo(Field.REGION_NAME, 0, false),
+new FieldInfo(Field.NAMESPACE, 0, true),
+new FieldInfo(Field.TABLE, 0, true),
+new FieldInfo(Field.START_CODE, 13, false),
+new FieldInfo(Field.REPLICA_ID, 5, false),
+new FieldInfo(Field.REGION, 32, true),
+new FieldInfo(Field.REGION_SERVER, 0, true),
+new FieldInfo(Field.LONG_REGION_SERVER, 0, false),
+new FieldInfo(Field.REQUEST_COUNT_PER_SECOND, 8, true),
+new FieldInfo(Field.READ_REQUEST_COUNT_PER_SECOND, 8, true),
+new FieldInfo(Field.FILTERED_READ_REQUEST_COUNT_PER_SECOND, 8, true),
+new FieldInfo(Field.WRITE_REQUEST_COUNT_PER_SECOND, 8, true),
+new FieldInfo(Field.STORE_FILE_SIZE, 10, true),
+new FieldInfo(Field.UNCOMPRESSED_STORE_FILE_SIZE, 12, false),
+new FieldInfo(Field.NUM_STORE_FILES,4, true),
+new FieldInfo(Field.MEM_STORE_SIZE, 8, true),
+new FieldInfo(Field.LOCALITY, 8, true),
+new FieldInfo(Field.START_KEY, 0, false),
+new FieldInfo(Field.COMPACTING_CELL_COUNT, 12, false),
+new FieldInfo(Field.COMPACTED_CELL_COUNT, 12, false),
+new FieldInfo(Field.COMPACTION_PROGRESS, 7, false),
+new FieldInfo(Field.LAST_MAJOR_COMPACTION_TIME, 19, false)
+ );
+
+ private final Map requestCountPerSecondMap =
new HashMap<>();
+
+ RegionModeStrategy() {
+ }
+
+ @Override
+ public List getFieldInfos() {
+return fieldInfos;
+ }
+
+ @Override
+ public Field getDefaultSortField() {
+return Field.REQUEST_COUNT_PER_SECOND;
+ }
+
+ @Override
+ public List getRecords(ClusterMetrics clusterMetrics) {
+List ret = new ArrayList<>();
+for (ServerMetrics sm : clusterMetrics.getLiveServerMetrics().values()) {
+ long lastReportTimestamp = sm.getLastReportTimestamp();
+ for (RegionMetrics rm : sm.getRegionMetrics().values()) {
+ret.add(createRecord(sm, rm, lastReportTimestamp));
+ }
+}
+return ret;
+ }
+
+ private Record createRecord(ServerMetrics serverMetrics, RegionMetrics
regionMetrics,
+long lastReportTimestamp) {
+
+Record ret = new Record();
+
+String regionName = regionMetrics.getNameAsString();
+ret.put(Field.REGION_NAME, regionName);
+
+String namespaceName = "";
+String tableName = "";
+String region = "";
+String startKey = "";
+String startCode = "";
+String replicaId = "";
+try {
+ byte[][] elements =
RegionInfo.parseRegionName(regionMetrics.getRegionName());
+ TableName tn = TableName.valueOf(elements[0]);
+ namespaceName = tn.getNamespaceAsString();
+ tableName = tn.getQualifierAsString();
+ startKey = Bytes.toStringBinary(elements[1]);
+ startCode = Bytes.toString(elements[2]);
+ replicaId = elements.length == 4 ?
+Integer.valueOf(Bytes.toString(elements[3])).toString() : "";
+ region = RegionInfo.encodeRegionName(regionMetrics.getRegionNa
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317927870
##
File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/HBTop.java
##
@@ -0,0 +1,141 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop;
+
+import java.io.InputStream;
+import java.util.Objects;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.hbtop.mode.Mode;
+import org.apache.hadoop.hbase.hbtop.screen.Screen;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+import org.apache.log4j.PropertyConfigurator;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
+import org.apache.hbase.thirdparty.org.apache.commons.cli.DefaultParser;
+import org.apache.hbase.thirdparty.org.apache.commons.cli.HelpFormatter;
+import org.apache.hbase.thirdparty.org.apache.commons.cli.Options;
+
+
+/**
+ * A real-time monitoring tool for HBase like Unix top command.
+ */
+@InterfaceAudience.Private
+public class HBTop extends Configured implements Tool {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(HBTop.class);
+
+ public HBTop() {
+this(HBaseConfiguration.create());
+ }
+
+ public HBTop(Configuration conf) {
+super(Objects.requireNonNull(conf));
+ }
+
+ @Override
+ public int run(String[] args) throws Exception {
+// In order to change the log level forcibly, read a custom log4j
configuration file
+InputStream log4jConfigInputStream = this.getClass().getClassLoader()
+ .getResourceAsStream("log4j-hbtop.properties");
+PropertyConfigurator.configure(log4jConfigInputStream);
Review comment:
> If we can't find this log4j-hbtop on the classpath, what's the
side-effect: you might get logging spam which messes up the console rendering?
Yes, we will get logging spam.
> Is it worth some big warning if log4jConfigInputStream is null?
`log4j-hbtop.properties` is in this module actually, so I think we can
always find this file.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317941008
##
File path:
hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/terminal/KeyPress.java
##
@@ -0,0 +1,128 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop.terminal;
+
+import edu.umd.cs.findbugs.annotations.Nullable;
+import java.util.Objects;
+import org.apache.yetus.audience.InterfaceAudience;
+
+
+/**
+ * Represents the user pressing a key on the keyboard.
+ */
+@InterfaceAudience.Private
+public class KeyPress {
+ public enum Type {
+Character,
+Escape,
+Backspace,
+ArrowLeft,
+ArrowRight,
+ArrowUp,
+ArrowDown,
+Insert,
+Delete,
+Home,
+End,
+PageUp,
+PageDown,
+ReverseTab,
+Tab,
+Enter,
+F1,
+F2,
+F3,
+F4,
+F5,
+F6,
+F7,
+F8,
+F9,
+F10,
+F11,
+F12,
+Unknown
+ }
Review comment:
Yes, ideally we should use a terminal lib to avoid reinventing the wheel. If
we find a good lib, we can replace the current implementation with it by making
another implementation of `Terminal` interface using the lib.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
URL: https://github.com/apache/hbase/pull/476#discussion_r317930535
##
File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/HBTop.java
##
@@ -0,0 +1,141 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.hbtop;
+
+import java.io.InputStream;
+import java.util.Objects;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.conf.Configured;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.hbtop.mode.Mode;
+import org.apache.hadoop.hbase.hbtop.screen.Screen;
+import org.apache.hadoop.util.Tool;
+import org.apache.hadoop.util.ToolRunner;
+import org.apache.log4j.PropertyConfigurator;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine;
+import org.apache.hbase.thirdparty.org.apache.commons.cli.DefaultParser;
+import org.apache.hbase.thirdparty.org.apache.commons.cli.HelpFormatter;
+import org.apache.hbase.thirdparty.org.apache.commons.cli.Options;
+
+
+/**
+ * A real-time monitoring tool for HBase like Unix top command.
+ */
+@InterfaceAudience.Private
+public class HBTop extends Configured implements Tool {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(HBTop.class);
+
+ public HBTop() {
+this(HBaseConfiguration.create());
+ }
+
+ public HBTop(Configuration conf) {
+super(Objects.requireNonNull(conf));
+ }
+
+ @Override
+ public int run(String[] args) throws Exception {
+// In order to change the log level forcibly, read a custom log4j
configuration file
+InputStream log4jConfigInputStream = this.getClass().getClassLoader()
+ .getResourceAsStream("log4j-hbtop.properties");
+PropertyConfigurator.configure(log4jConfigInputStream);
+
+long initialRefreshDelay = 3 * 1000;
+Mode initialMode = Mode.REGION;
+try {
+ // Command line options
+ Options opts = new Options();
+ opts.addOption("help", false,
+"Print usage; for help while the tool is running press 'h'");
+ opts.addOption("delay", true,
+"The refresh delay (in seconds); default is 3 seconds");
Review comment:
> On this delay, the master is only going to get ClusterMetrics at a
particular rate from RegionServers. A refresh of this client that happens more
frequently than Master update will show no changes?
Yes. Of course, we can decrease `hbase.regionserver.msginterval` (3 seconds
by default) to increase the actual refresh rate, but that may not be good.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #537: HBASE-22883 Duplacate codes of method Threads.newDaemonThreadFactory(…
Apache-HBase commented on issue #537: HBASE-22883 Duplacate codes of method Threads.newDaemonThreadFactory(… URL: https://github.com/apache/hbase/pull/537#issuecomment-525199430 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 37 | Maven dependency ordering for branch | | +1 | mvninstall | 310 | master passed | | +1 | compile | 134 | master passed | | +1 | checkstyle | 136 | master passed | | +1 | shadedjars | 268 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 88 | master passed | | 0 | spotbugs | 238 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 333 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 301 | the patch passed | | +1 | compile | 136 | the patch passed | | +1 | javac | 136 | the patch passed | | +1 | checkstyle | 26 | The patch passed checkstyle in hbase-common | | +1 | checkstyle | 80 | hbase-server: The patch generated 0 new + 60 unchanged - 3 fixed = 60 total (was 63) | | +1 | checkstyle | 15 | The patch passed checkstyle in hbase-backup | | +1 | checkstyle | 16 | The patch passed checkstyle in hbase-it | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 272 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 927 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 88 | the patch passed | | +1 | findbugs | 344 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 183 | hbase-common in the patch passed. | | +1 | unit | 8982 | hbase-server in the patch passed. | | +1 | unit | 876 | hbase-backup in the patch passed. | | +1 | unit | 67 | hbase-it in the patch passed. | | +1 | asflicense | 122 | The patch does not generate ASF License warnings. | | | | 14217 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-537/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/537 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 4e732f5caca1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-537/out/precommit/personality/provided.sh | | git revision | master / c88267202a | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-537/3/testReport/ | | Max. process+thread count | 5189 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server hbase-backup hbase-it U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-537/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] jatsakthi merged pull request #545: HBASE-22893 Change the comment in HBaseClassTestRule of test timeouts
jatsakthi merged pull request #545: HBASE-22893 Change the comment in HBaseClassTestRule of test timeouts URL: https://github.com/apache/hbase/pull/545 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
Yechao Chen created HBASE-22929: --- Summary: MemStoreLAB ChunkCreator may memory leak Key: HBASE-22929 URL: https://issues.apache.org/jira/browse/HBASE-22929 Project: HBase Issue Type: Bug Affects Versions: 2.1.2 Reporter: Yechao Chen We use hbase 2.1.2 with memstorelab enable RegionServer crashed case of oom I dump the heap ,found the ChunkCreator may be memory leak The heap is 32GB, hbase.regionserver.global.memstore.size=0.4, hbase.hregion.memstore.mslab.enabled=true hbase.hregion.memstore.chunkpool.initialsize=0.5, hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-22929: Attachment: hbase_heap_monitor.png > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase_heap_monitor.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-22929: Attachment: rs_mem_used1.png > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase_heap_monitor.png, rs_mem_used1.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] brfrn169 commented on issue #476: HBASE-11062 hbtop
brfrn169 commented on issue #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#issuecomment-525201984 @joshelser > One question I haven't been able to find the answer to on my own yet: when do byte-sizes get rolled up to the next unit? (e.g. MB to GB) > > I see my storefile size is still reporting `13818.0MB`. I feel like at this point `13.8GB` is sufficient for what I care about, but I'm not sure if this is unique to your code, or just something else in HBase you're re-using. Actually, I'm feeling that should be `Size` class's responsible, but I will check that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-22929: Attachment: (was: rs_mem_used1.png) > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase_heap_monitor.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-22929: Attachment: hbase_rs_mem_used.png > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase_heap_monitor.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22893) Change the comment in HBaseClassTestRule to reflect change in default test timeouts
[
https://issues.apache.org/jira/browse/HBASE-22893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-22893:
---
Fix Version/s: 2.1.6
2.2.1
2.3.0
3.0.0
> Change the comment in HBaseClassTestRule to reflect change in default test
> timeouts
> ---
>
> Key: HBASE-22893
> URL: https://issues.apache.org/jira/browse/HBASE-22893
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Rabi Kumar K C
>Priority: Trivial
> Labels: beginner, noob
> Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.6
>
>
> The default timeout is 13 minutes. Change the comment accordingly.
> {code}
> // All tests have a 10minute timeout.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22893) Change the comment in HBaseClassTestRule to reflect change in default test timeouts
[
https://issues.apache.org/jira/browse/HBASE-22893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916522#comment-16916522
]
Sakthi commented on HBASE-22893:
Pushed to 2.1+ Thanks for the patch [~rabikumar.kc]
> Change the comment in HBaseClassTestRule to reflect change in default test
> timeouts
> ---
>
> Key: HBASE-22893
> URL: https://issues.apache.org/jira/browse/HBASE-22893
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Rabi Kumar K C
>Priority: Trivial
> Labels: beginner, noob
> Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.6
>
>
> The default timeout is 13 minutes. Change the comment accordingly.
> {code}
> // All tests have a 10minute timeout.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-22929: Attachment: hbase_rs_heap_dump_mat_1.png > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase_heap_monitor.png, hbase_rs_heap_dump_mat_1.png, > hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (HBASE-22893) Change the comment in HBaseClassTestRule to reflect change in default test timeouts
[
https://issues.apache.org/jira/browse/HBASE-22893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi resolved HBASE-22893.
Resolution: Fixed
> Change the comment in HBaseClassTestRule to reflect change in default test
> timeouts
> ---
>
> Key: HBASE-22893
> URL: https://issues.apache.org/jira/browse/HBASE-22893
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Rabi Kumar K C
>Priority: Trivial
> Labels: beginner, noob
> Fix For: 3.0.0, 2.3.0, 2.2.1, 2.1.6
>
>
> The default timeout is 13 minutes. Change the comment accordingly.
> {code}
> // All tests have a 10minute timeout.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-22929: Attachment: hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase_heap_monitor.png, hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916524#comment-16916524 ] Yechao Chen commented on HBASE-22929: - !hbase_heap_monitor.png! !hbase_rs_mem_used.png! !hbase_rs_heap_dump_mat_1.png! !hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png! > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase_heap_monitor.png, hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HBASE-22930) Set unique name to longCompactions/shortCompactions threads
Pankaj Kumar created HBASE-22930: Summary: Set unique name to longCompactions/shortCompactions threads Key: HBASE-22930 URL: https://issues.apache.org/jira/browse/HBASE-22930 Project: HBase Issue Type: Improvement Reporter: Pankaj Kumar Assignee: Pankaj Kumar Sometime I observed multiple compaction threads have the same name which is misleading while analysing the log. Currently we just append the current timestamp which can be same for multiple threads, https://github.com/apache/hbase/blob/ec68bf30ba91ea7484d3b56b0809ff5434f9406e/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java#L144 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916525#comment-16916525 ] Yechao Chen commented on HBASE-22929: - It seems : the chunkIdMap of the ChunkCreator the maxCount of the chunks should be (32GB*0.4)/2MB = 6553 the chunkIdMap size should be smaller than 6553 but the heap dump shows the size of chunkIdMap has 14579 entries > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase_heap_monitor.png, hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916525#comment-16916525 ] Yechao Chen edited comment on HBASE-22929 at 8/27/19 8:53 AM: -- It seems : the chunkIdMap of the ChunkCreator increament incorrect the maxCount of the chunks should be (32GB*0.4)/2MB = 6553 the chunkIdMap size should be smaller than 6553 but the heap dump shows the size of chunkIdMap has 14579 entries was (Author: chenyechao): It seems : the chunkIdMap of the ChunkCreator the maxCount of the chunks should be (32GB*0.4)/2MB = 6553 the chunkIdMap size should be smaller than 6553 but the heap dump shows the size of chunkIdMap has 14579 entries > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase_heap_monitor.png, hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] pankaj72981 opened a new pull request #548: HBASE-22930 Set unique name to longCompactions/shortCompactions threads
pankaj72981 opened a new pull request #548: HBASE-22930 Set unique name to longCompactions/shortCompactions threads URL: https://github.com/apache/hbase/pull/548 Appended the counter in the thread name. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] openinx opened a new pull request #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads
openinx opened a new pull request #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads URL: https://github.com/apache/hbase/pull/549 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-22929: Attachment: hbase-site.xml > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916535#comment-16916535 ] Yechao Chen commented on HBASE-22929: - chunkIdMap add entries may happened in here [createChunk method |https://github.com/apache/hbase/blob/rel/2.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java#L268] but the code has checked the maxCount before createChunk [createChunkForPool| https://github.com/apache/hbase/blob/rel/2.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java#L379] [MemStoreChunkPool init|https://github.com/apache/hbase/blob/rel/2.1.2/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java#L344] > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916536#comment-16916536 ] Yechao Chen commented on HBASE-22929: - !OOM_log.png! > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: OOM_log.png, hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-22929: Attachment: OOM_log.png > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: OOM_log.png, hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22930) Set unique name to longCompactions/shortCompactions threads
[ https://issues.apache.org/jira/browse/HBASE-22930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-22930: - Description: Sometime observed multiple compaction threads have the same name which is misleading while analysing the log. Currently we just append the current timestamp which can be same for multiple threads, [https://github.com/apache/hbase/blob/ec68bf30ba91ea7484d3b56b0809ff5434f9406e/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java#L144] was: Sometime I observed multiple compaction threads have the same name which is misleading while analysing the log. Currently we just append the current timestamp which can be same for multiple threads, https://github.com/apache/hbase/blob/ec68bf30ba91ea7484d3b56b0809ff5434f9406e/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java#L144 > Set unique name to longCompactions/shortCompactions threads > --- > > Key: HBASE-22930 > URL: https://issues.apache.org/jira/browse/HBASE-22930 > Project: HBase > Issue Type: Improvement >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Minor > > Sometime observed multiple compaction threads have the same name which is > misleading while analysing the log. > Currently we just append the current timestamp which can be same for multiple > threads, > [https://github.com/apache/hbase/blob/ec68bf30ba91ea7484d3b56b0809ff5434f9406e/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java#L144] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #494: HBASE-22380 break circle replication when doing bulkload
Apache-HBase commented on issue #494: HBASE-22380 break circle replication when doing bulkload URL: https://github.com/apache/hbase/pull/494#issuecomment-525212070 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 49 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ branch-2 Compile Tests _ | | 0 | mvndep | 15 | Maven dependency ordering for branch | | +1 | mvninstall | 393 | branch-2 passed | | +1 | compile | 137 | branch-2 passed | | +1 | checkstyle | 163 | branch-2 passed | | +1 | shadedjars | 308 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 78 | branch-2 passed | | 0 | spotbugs | 221 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 474 | branch-2 passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 370 | the patch passed | | +1 | compile | 131 | the patch passed | | +1 | cc | 131 | the patch passed | | +1 | javac | 131 | the patch passed | | -1 | checkstyle | 86 | hbase-server: The patch generated 1 new + 302 unchanged - 0 fixed = 303 total (was 302) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 259 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 943 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | hbaseprotoc | 120 | the patch passed | | +1 | javadoc | 76 | the patch passed | | -1 | findbugs | 188 | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 40 | hbase-protocol-shaded in the patch passed. | | +1 | unit | 217 | hbase-client in the patch passed. | | -1 | unit | 9625 | hbase-server in the patch failed. | | +1 | asflicense | 76 | The patch does not generate ASF License warnings. | | | | 14485 | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(List, CellScanner, String, String, String) makes inefficient use of keySet iterator instead of entrySet iterator At ReplicationSink.java:makes inefficient use of keySet iterator instead of entrySet iterator At ReplicationSink.java:[line 257] | | Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/34/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/494 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc | | uname | Linux 4be39ec73ed7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-494/out/precommit/personality/provided.sh | | git revision | branch-2 / caba5cf990 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/34/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/34/artifact/out/new-findbugs-hbase-server.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/34/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/34/testReport/ | | Max. process+thread count | 4769 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-494/34/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #536: HBASE-22881 Fix non-daemon threads in hbase server implementation
Apache-HBase commented on issue #536: HBASE-22881 Fix non-daemon threads in hbase server implementation URL: https://github.com/apache/hbase/pull/536#issuecomment-525212224 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 44 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ branch-2 Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for branch | | +1 | mvninstall | 312 | branch-2 passed | | +1 | compile | 139 | branch-2 passed | | +1 | checkstyle | 184 | branch-2 passed | | +1 | shadedjars | 251 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 114 | branch-2 passed | | 0 | spotbugs | 90 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 373 | branch-2 passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 18 | Maven dependency ordering for patch | | +1 | mvninstall | 291 | the patch passed | | +1 | compile | 139 | the patch passed | | +1 | javac | 139 | the patch passed | | +1 | checkstyle | 28 | The patch passed checkstyle in hbase-common | | +1 | checkstyle | 37 | The patch passed checkstyle in hbase-client | | +1 | checkstyle | 80 | hbase-server: The patch generated 0 new + 153 unchanged - 4 fixed = 153 total (was 157) | | +1 | checkstyle | 39 | hbase-thrift: The patch generated 0 new + 10 unchanged - 2 fixed = 10 total (was 12) | | +1 | whitespace | 1 | The patch has no whitespace issues. | | +1 | shadedjars | 250 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 904 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 112 | the patch passed | | +1 | findbugs | 413 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 176 | hbase-common in the patch passed. | | +1 | unit | 206 | hbase-client in the patch passed. | | -1 | unit | 9320 | hbase-server in the patch failed. | | +1 | unit | 239 | hbase-thrift in the patch passed. | | +1 | asflicense | 101 | The patch does not generate ASF License warnings. | | | | 14211 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestBlockEvictionFromClient | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-536/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/536 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux b9b542473dbe 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-536/out/precommit/personality/provided.sh | | git revision | branch-2 / caba5cf990 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-536/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-536/3/testReport/ | | Max. process+thread count | 4459 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-server hbase-thrift U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-536/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22872) Don't create normalization plan unnecesarily when split and merge both are disabled
[
https://issues.apache.org/jira/browse/HBASE-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916539#comment-16916539
]
Hudson commented on HBASE-22872:
Results for branch branch-1
[build #1032 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1032/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1032//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1032//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1032//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Don't create normalization plan unnecesarily when split and merge both are
> disabled
> ---
>
> Key: HBASE-22872
> URL: https://issues.apache.org/jira/browse/HBASE-22872
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.10
>Reporter: Aman Poonia
>Assignee: Aman Poonia
>Priority: Minor
> Fix For: 1.5.0, 2.2.1, 1.3.6, 1.4.11, 2.1.7
>
> Attachments: HBASE-22872.branch-1.4.001.patch,
> HBASE-22872.branch-1.4.002.patch, HBASE-22872.branch-1.4.003.patch,
> HBASE-22872.branch-1.4.004.patch, HBASE-22872.branch-1.4.005.patch,
> HBASE-22872.branch-2.patch, HBASE-22872.master.v01.patch
>
>
> We should not proceed futher in normalization plan creation if split and
> merge both are disabled on a table.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916525#comment-16916525 ] Yechao Chen edited comment on HBASE-22929 at 8/27/19 9:07 AM: -- It seems : the chunkIdMap of the ChunkCreator increment incorrect the chunk size default is 2MB,global memstore limit is 32GB*0.4 the maxCount of the chunks should be (32GB*0.4)/2MB = 6553 the chunkIdMap size should be smaller than 6553 but the heap dump shows the size of chunkIdMap has 14579 entries was (Author: chenyechao): It seems : the chunkIdMap of the ChunkCreator increament incorrect the maxCount of the chunks should be (32GB*0.4)/2MB = 6553 the chunkIdMap size should be smaller than 6553 but the heap dump shows the size of chunkIdMap has 14579 entries > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: OOM_log.png, hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yechao Chen updated HBASE-22929: Description: We use hbase 2.1.2 with memstorelab enable RegionServer crashed case of oom I dump the heap ,found the ChunkCreator may be memory leak The heap is 32GB, hbase.regionserver.global.memstore.size=0.4, hbase.hregion.memstore.mslab.enabled=true hbase.hregion.memstore.chunkpool.initialsize=0.5, hbase.hregion.memstore.chunkpool.maxsize=1.0 BucketCache with offheap was: We use hbase 2.1.2 with memstorelab enable RegionServer crashed case of oom I dump the heap ,found the ChunkCreator may be memory leak The heap is 32GB, hbase.regionserver.global.memstore.size=0.4, hbase.hregion.memstore.mslab.enabled=true hbase.hregion.memstore.chunkpool.initialsize=0.5, hbase.hregion.memstore.chunkpool.maxsize=1.0 > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: OOM_log.png, hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 > BucketCache with offheap -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916549#comment-16916549 ] ramkrishna.s.vasudevan commented on HBASE-22929: [~chenyechao] Are you seeing that the chunkIdMap is getting added with new chunks that are not from the pool? Any chance you have enabled CompactingMemstore? > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: OOM_log.png, hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 > BucketCache with offheap -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22918) RegionServer violates failfast fault assumption
[ https://issues.apache.org/jira/browse/HBASE-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916551#comment-16916551 ] ranpanfeng commented on HBASE-22918: [~busbey] Thanks for explanation, I got it! > RegionServer violates failfast fault assumption > --- > > Key: HBASE-22918 > URL: https://issues.apache.org/jira/browse/HBASE-22918 > Project: HBase > Issue Type: Bug >Reporter: ranpanfeng >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > hbase 2.1.5 is tested and veriflied seriously before it will be deployed in > our production environment. we give NP(network partition) fault a very > important care. so NP fault injection tests are conducted in our test > environment. Some findings are exposed. > I use ycsb to write data into table SYSTEM:test, which resides on > regionserver0; during the writting, I use iptables to drop any packet from > regionserver0 to zookeeper quorums. after > a default zookeeper.session.out(90'), regionserver0 throws > YouAreDeadException after retries to connect to zookeeper on > TimeoutException error. then, regionserver0 suicides itself, before > regionserver0 invokes completeFile on WAL, the active master already > considered regionserver0 has dead pre-maturely, so invokes recoverLease to > close the WAL on regionserver0 forcely. > In trusted idc, distributed storage assumes that the error are always > failstop/failfast faults, there are no Byzantine failures. so in above > scenario, active master should take over the WAL on regionserver0 after > regionserver0 is suicided successfully. According to lease protocol, RS > should suicide in a lease period, and active master should take over the WAL > after a grace period has elapsed, and invariant "lease period < grace > period" should always hold. in hbase-site.xml, only on config property > "zookeeper.session.timeout" is given, I think we should provide two > properties: > 1. regionserver.zookeeper.session.timeout > 2. master.zookeeper.session.timeout > HBase admin then can tune regionserver.zookeeper.session.timeout less than > master.zookeeper.session.timeout. In this way, failstop assumption is > guaranteed. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916553#comment-16916553 ] binlijin commented on HBASE-22929: -- @chenxu write in HBASE-20373 If memStoreScanners can't be closed properly in StoreScanner#updateReaders, there will be a memory leak in MemStoreLABImpl#close, because the Chunks can't be reclaimed correctly. > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: OOM_log.png, hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 > BucketCache with offheap -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22380) break circle replication when doing bulkload
[ https://issues.apache.org/jira/browse/HBASE-22380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916554#comment-16916554 ] Anoop Sam John commented on HBASE-22380: Seems then I misunderstood ur initial comments thinking that the user need to pass the clusterId. bq.Or do you mean, to avoid replication if same hfiles are bulkloaded again on original cluster? No. I dont mean that. A cluster which has replicated one file already seeing the replication request for same again. And this cluster is not the original cluster where the end user bulk loaded the file.For the normal mutations we pass the ids of all clusters via which this mutation passed through in the replication flow. So my ask was same is needed for hfile replication also (?) Ya I got the point that it is not the WAL cells being replicated. But my meaning is the RPC. I agree to the point that RPC need to pass the clusterId. But my thinking is it should be Ids. > break circle replication when doing bulkload > > > Key: HBASE-22380 > URL: https://issues.apache.org/jira/browse/HBASE-22380 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.0.5, 2.3.0, 2.1.5, 1.3.5 >Reporter: chenxu >Assignee: Wellington Chevreuil >Priority: Critical > Labels: bulkload > Fix For: 3.0.0, 1.5.0, 2.3.0, 1.4.11, 2.1.7, 2.2.2 > > > when enabled master-master bulkload replication, HFiles will be replicated > circularly between two clusters -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916555#comment-16916555 ] Yechao Chen commented on HBASE-22929: - [~ram_krish] thanks for reply, no,I use the DefaultMemStore > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: OOM_log.png, hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 > BucketCache with offheap -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317991637 ## File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/HBTop.java ## @@ -0,0 +1,141 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.hbtop; + +import java.io.InputStream; +import java.util.Objects; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.conf.Configured; +import org.apache.hadoop.hbase.HBaseConfiguration; +import org.apache.hadoop.hbase.hbtop.mode.Mode; +import org.apache.hadoop.hbase.hbtop.screen.Screen; +import org.apache.hadoop.util.Tool; Review comment: Added an appropriate dependency. Resolving this. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317992062 ## File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/HBTop.java ## @@ -0,0 +1,141 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.hbtop; + +import java.io.InputStream; +import java.util.Objects; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.conf.Configured; +import org.apache.hadoop.hbase.HBaseConfiguration; +import org.apache.hadoop.hbase.hbtop.mode.Mode; +import org.apache.hadoop.hbase.hbtop.screen.Screen; +import org.apache.hadoop.util.Tool; +import org.apache.hadoop.util.ToolRunner; +import org.apache.log4j.PropertyConfigurator; +import org.apache.yetus.audience.InterfaceAudience; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine; Review comment: Added an appropriate dependency. Resolving this. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317992000 ## File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/HBTop.java ## @@ -0,0 +1,141 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.hbtop; + +import java.io.InputStream; +import java.util.Objects; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.conf.Configured; +import org.apache.hadoop.hbase.HBaseConfiguration; +import org.apache.hadoop.hbase.hbtop.mode.Mode; +import org.apache.hadoop.hbase.hbtop.screen.Screen; +import org.apache.hadoop.util.Tool; +import org.apache.hadoop.util.ToolRunner; +import org.apache.log4j.PropertyConfigurator; +import org.apache.yetus.audience.InterfaceAudience; +import org.slf4j.Logger; Review comment: Added an appropriate dependency. Resolving this. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317993415 ## File path: hbase-hbtop/src/test/java/org/apache/hadoop/hbase/hbtop/TestUtils.java ## @@ -0,0 +1,402 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.hbtop; + +import static org.hamcrest.CoreMatchers.is; Review comment: Added an appropriate dependency. Resolving this. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317993482 ## File path: hbase-hbtop/src/test/java/org/apache/hadoop/hbase/hbtop/TestUtils.java ## @@ -0,0 +1,402 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.hbtop; + +import static org.hamcrest.CoreMatchers.is; +import static org.junit.Assert.assertThat; +import static org.junit.Assert.fail; + +import java.text.ParseException; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import org.apache.commons.lang3.time.FastDateFormat; Review comment: Added an appropriate dependency. Resolving this. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317994409 ## File path: hbase-hbtop/pom.xml ## @@ -0,0 +1,59 @@ + +http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> + + 4.0.0 + +hbase Review comment: I changed the parent to `hbase-build-configuration`. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r317994666 ## File path: hbase-hbtop/src/main/java/org/apache/hadoop/hbase/hbtop/HBTop.java ## @@ -0,0 +1,141 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.hbtop; + +import java.io.InputStream; +import java.util.Objects; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.conf.Configured; +import org.apache.hadoop.hbase.HBaseConfiguration; +import org.apache.hadoop.hbase.hbtop.mode.Mode; +import org.apache.hadoop.hbase.hbtop.screen.Screen; +import org.apache.hadoop.util.Tool; +import org.apache.hadoop.util.ToolRunner; +import org.apache.log4j.PropertyConfigurator; +import org.apache.yetus.audience.InterfaceAudience; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.apache.hbase.thirdparty.org.apache.commons.cli.CommandLine; +import org.apache.hbase.thirdparty.org.apache.commons.cli.DefaultParser; +import org.apache.hbase.thirdparty.org.apache.commons.cli.HelpFormatter; +import org.apache.hbase.thirdparty.org.apache.commons.cli.Options; + + +/** + * A real-time monitoring tool for HBase like Unix top command. + */ +@InterfaceAudience.Private Review comment: I modified that. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22872) Don't create normalization plan unnecesarily when split and merge both are disabled
[
https://issues.apache.org/jira/browse/HBASE-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916565#comment-16916565
]
Hudson commented on HBASE-22872:
Results for branch branch-1.4
[build #982 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/982/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/970//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/978//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Don't create normalization plan unnecesarily when split and merge both are
> disabled
> ---
>
> Key: HBASE-22872
> URL: https://issues.apache.org/jira/browse/HBASE-22872
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.10
>Reporter: Aman Poonia
>Assignee: Aman Poonia
>Priority: Minor
> Fix For: 1.5.0, 2.2.1, 1.3.6, 1.4.11, 2.1.7
>
> Attachments: HBASE-22872.branch-1.4.001.patch,
> HBASE-22872.branch-1.4.002.patch, HBASE-22872.branch-1.4.003.patch,
> HBASE-22872.branch-1.4.004.patch, HBASE-22872.branch-1.4.005.patch,
> HBASE-22872.branch-2.patch, HBASE-22872.master.v01.patch
>
>
> We should not proceed futher in normalization plan creation if split and
> merge both are disabled on a table.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916571#comment-16916571 ] Yechao Chen commented on HBASE-22929: - [~aoxiang] thanks, seems different problem, this case found the chunkIdMap size over the maxCount of chunkSize > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: OOM_log.png, hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 > BucketCache with offheap -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22872) Don't create normalization plan unnecesarily when split and merge both are disabled
[
https://issues.apache.org/jira/browse/HBASE-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916576#comment-16916576
]
Hudson commented on HBASE-22872:
Results for branch branch-1.3
[build #942 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/942/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Don't create normalization plan unnecesarily when split and merge both are
> disabled
> ---
>
> Key: HBASE-22872
> URL: https://issues.apache.org/jira/browse/HBASE-22872
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.4.10
>Reporter: Aman Poonia
>Assignee: Aman Poonia
>Priority: Minor
> Fix For: 1.5.0, 2.2.1, 1.3.6, 1.4.11, 2.1.7
>
> Attachments: HBASE-22872.branch-1.4.001.patch,
> HBASE-22872.branch-1.4.002.patch, HBASE-22872.branch-1.4.003.patch,
> HBASE-22872.branch-1.4.004.patch, HBASE-22872.branch-1.4.005.patch,
> HBASE-22872.branch-2.patch, HBASE-22872.master.v01.patch
>
>
> We should not proceed futher in normalization plan creation if split and
> merge both are disabled on a table.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22013) SpaceQuotas - getNumRegions() returning wrong number of regions due to region replicas
[
https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916604#comment-16916604
]
HBase QA commented on HBASE-22013:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
2s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
58s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 17s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}267m 1s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}333m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/838/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22013 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978570/HBASE-22013.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux cf20bb47a5c5 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2
18:22:20 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / c88267202a |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/838/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/838/testReport/ |
| Max. process+thread count | 4836 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server
[GitHub] [hbase] Reidddddd commented on issue #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads
Reidd commented on issue #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads URL: https://github.com/apache/hbase/pull/549#issuecomment-525249133 Skimmed, LGTM, I'll take a detailed review late this day. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] openinx commented on issue #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads
openinx commented on issue #549: HBASE-22912 [Backport] HBASE-22867 to branch-1 to avoid ForkJoinPool to spawn thousands of threads URL: https://github.com/apache/hbase/pull/549#issuecomment-525254673 The patch is not good enough now, because the traverseAndDelete is not async and easy to block the threads... will provide a new one. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (HBASE-22846) Internal Error 500 when Using HBASE REST API to Create Namespace.
[ https://issues.apache.org/jira/browse/HBASE-22846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913690#comment-16913690 ] Wellington Chevreuil edited comment on HBASE-22846 at 8/27/19 11:16 AM: Submitted an initial PR for branch-2.1. Noticed additional two issues that may be good to address in separate jiras: -1) Lack of UTs for resources;- 2) REST API documentation in the ref guides don't describe the payload format. was (Author: wchevreuil): Submitted an initial PR for branch-2.1. Noticed additional two issues that may be good to address in separate jiras: 1) Lack of UTs for resources; 2) REST API documentation in the ref guides don't describe the payload format. > Internal Error 500 when Using HBASE REST API to Create Namespace. > - > > Key: HBASE-22846 > URL: https://issues.apache.org/jira/browse/HBASE-22846 > Project: HBase > Issue Type: Improvement > Components: hbase-connectors >Affects Versions: 2.1.1 >Reporter: Sailesh Patel >Assignee: Wellington Chevreuil >Priority: Major > > When trying to the following URL to create namespace: > Secured cluster: curl --negotiate -u : -i -k -vi -X POST > "http://HBASE_REST_API_HOST:20550/namespaces/datasparktest"; > UnSecured cluster: curl -vi -X POST > "http://HBASE_REST_API_HOST:20550/namespaces/datasparktest"; > The following is returned on the console: > HTTP/1.1 500 Request failed. > The Error in Hbase REST Server log is: > 2019-08-13 15:44:55,080 WARN org.eclipse.jetty.servlet.ServletHandler: > javax.servlet.ServletException: java.lang.NullPointerException > at > org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489) > at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) > ... > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hbase.rest.NamespacesInstanceResource.createOrUpdate(NamespacesInstanceResource.java:250) > at > org.apache.hadoop.hbase.rest.NamespacesInstanceResource.processUpdate(NamespacesInstanceResource.java:243) > at > org.apache.hadoop.hbase.rest.NamespacesInstanceResource.post(NamespacesInstanceResource.java:183) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22846) Internal Error 500 when Using HBASE REST API to Create Namespace.
[ https://issues.apache.org/jira/browse/HBASE-22846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916633#comment-16916633 ] Wellington Chevreuil commented on HBASE-22846: -- So we do have UTs for this resource under *TestNamespacesInstanceResource.testNamespaceCreateAndDeletePBAndNoBody.* Noticed the same already tests condition of _no body_ POST requests, and these are always passing. Upon further investigation, I had found out this is due the http request content-type. The passing tests are properly setting it to "application/octet-stream". Apparently, http clients such as curl default it to "text/html", if not explicitly set. In this case, jersey is then directing the request to the wrong resource method, triggering the NPE. > Internal Error 500 when Using HBASE REST API to Create Namespace. > - > > Key: HBASE-22846 > URL: https://issues.apache.org/jira/browse/HBASE-22846 > Project: HBase > Issue Type: Improvement > Components: hbase-connectors >Affects Versions: 2.1.1 >Reporter: Sailesh Patel >Assignee: Wellington Chevreuil >Priority: Major > > When trying to the following URL to create namespace: > Secured cluster: curl --negotiate -u : -i -k -vi -X POST > "http://HBASE_REST_API_HOST:20550/namespaces/datasparktest"; > UnSecured cluster: curl -vi -X POST > "http://HBASE_REST_API_HOST:20550/namespaces/datasparktest"; > The following is returned on the console: > HTTP/1.1 500 Request failed. > The Error in Hbase REST Server log is: > 2019-08-13 15:44:55,080 WARN org.eclipse.jetty.servlet.ServletHandler: > javax.servlet.ServletException: java.lang.NullPointerException > at > org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489) > at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) > ... > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hbase.rest.NamespacesInstanceResource.createOrUpdate(NamespacesInstanceResource.java:250) > at > org.apache.hadoop.hbase.rest.NamespacesInstanceResource.processUpdate(NamespacesInstanceResource.java:243) > at > org.apache.hadoop.hbase.rest.NamespacesInstanceResource.post(NamespacesInstanceResource.java:183) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HBASE-22931) SpaceQuotas- Hbase:meta is too busy when space quotas are turned on.
Yiran Wu created HBASE-22931:
Summary: SpaceQuotas- Hbase:meta is too busy when space quotas are
turned on.
Key: HBASE-22931
URL: https://issues.apache.org/jira/browse/HBASE-22931
Project: HBase
Issue Type: Improvement
Affects Versions: 3.0.0
Reporter: Yiran Wu
In a large cluster, there are many tables and regions. META tables will be
scanned frequently after opening space quota. We need optimize it to support
larger cluster.
some code...
{code:java}
for (TableName table : Iterables.concat(tablesWithTableQuotas,
tablesWithNamespaceQuotas)) {
// Don't recompute a table we've already computed
if (tablesToRemove.contains(table)) {
continue;
}
final int numRegionsInTable = getNumRegions(table);
// If the table doesn't exist (no regions), bail out.
if (numRegionsInTable == 0) {
if (LOG.isTraceEnabled()) {
LOG.trace("Filtering " + table + " because no regions were
reported");
}
tablesToRemove.add(table);
continue;
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916647#comment-16916647 ] Jan Hentschel commented on HBASE-12081: --- Agree with [~psomogyi]. Should we close this one in favor of Java 11? > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Major > Fix For: 3.0.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22929) MemStoreLAB ChunkCreator may memory leak
[ https://issues.apache.org/jira/browse/HBASE-22929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916654#comment-16916654 ] Allan Yang commented on HBASE-22929: is HBASE-21031 related? > MemStoreLAB ChunkCreator may memory leak > - > > Key: HBASE-22929 > URL: https://issues.apache.org/jira/browse/HBASE-22929 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.2 >Reporter: Yechao Chen >Priority: Major > Attachments: OOM_log.png, hbase-site.xml, hbase_heap_monitor.png, > hbase_rs_heap_dump_mat_1.png, > hbase_rs_heap_dump_mat_ChunkCreator_chunkIdMap.png, hbase_rs_mem_used.png > > > We use hbase 2.1.2 with memstorelab enable > RegionServer crashed case of oom > I dump the heap ,found the ChunkCreator may be memory leak > The heap is 32GB, > hbase.regionserver.global.memstore.size=0.4, > hbase.hregion.memstore.mslab.enabled=true > hbase.hregion.memstore.chunkpool.initialsize=0.5, > hbase.hregion.memstore.chunkpool.maxsize=1.0 > BucketCache with offheap -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HBASE-22932) Add rs group managment methods in Admin and AsyncAdmin
Duo Zhang created HBASE-22932: - Summary: Add rs group managment methods in Admin and AsyncAdmin Key: HBASE-22932 URL: https://issues.apache.org/jira/browse/HBASE-22932 Project: HBase Issue Type: Sub-task Components: Admin, asyncclient, Client Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22932) Add rs group management methods in Admin and AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-22932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22932: -- Summary: Add rs group management methods in Admin and AsyncAdmin (was: Add rs group managment methods in Admin and AsyncAdmin) > Add rs group management methods in Admin and AsyncAdmin > --- > > Key: HBASE-22932 > URL: https://issues.apache.org/jira/browse/HBASE-22932 > Project: HBase > Issue Type: Sub-task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HBASE-22729) Start RSGroupInfoManager as default
[ https://issues.apache.org/jira/browse/HBASE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-22729: - Assignee: Duo Zhang > Start RSGroupInfoManager as default > --- > > Key: HBASE-22729 > URL: https://issues.apache.org/jira/browse/HBASE-22729 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Duo Zhang >Priority: Major > > Start RSGroupInfoManager as default. If no rsgroup information, all > RegionServers belong to default group. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work started] (HBASE-22729) Start RSGroupInfoManager as default
[ https://issues.apache.org/jira/browse/HBASE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-22729 started by Duo Zhang. - > Start RSGroupInfoManager as default > --- > > Key: HBASE-22729 > URL: https://issues.apache.org/jira/browse/HBASE-22729 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Duo Zhang >Priority: Major > > Start RSGroupInfoManager as default. If no rsgroup information, all > RegionServers belong to default group. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22862) Region Server crash with: Added a key not lexically larger than previous
[
https://issues.apache.org/jira/browse/HBASE-22862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916688#comment-16916688
]
Duo Zhang commented on HBASE-22862:
---
[~openinx] Is this branch-1 only? I haven't found RawBytesComparator on
master...
> Region Server crash with: Added a key not lexically larger than previous
>
>
> Key: HBASE-22862
> URL: https://issues.apache.org/jira/browse/HBASE-22862
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 1.4.10
> Environment: {code}
> openjdk version "1.8.0_181"
> OpenJDK Runtime Environment (Zulu 8.31.0.1-linux64) (build 1.8.0_181-b02)
> OpenJDK 64-Bit Server VM (Zulu 8.31.0.1-linux64) (build 25.181-b02, mixed
> mode)
> {code}
>Reporter: Alex Batyrshin
>Assignee: Zheng Hu
>Priority: Critical
> Attachments: HBASE-22862.UT.v01.patch, HBASE-22862.UT.v02.patch
>
>
> We observe error "Added a key not lexically larger than previous” that cause
> most of our region-servers to crash in our cluster.
> {code}
> 2019-08-15 18:02:10,554 INFO [MemStoreFlusher.0] regionserver.HRegion:
> Flushing 1/1 column families, memstore=56.08 MB
> 2019-08-15 18:02:10,727 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=0
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2200)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2125)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:512)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:482)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
>at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
>at java.lang.Thread.run(Thread.java:748)
> 2019-08-15 18:02:21,776 WARN [MemStoreFlusher.0] regionserver.HStore: Failed
> flushing store file, retrying num=9
> java.io.IOException: Added a key not lexically larger than previous. Current
> cell =
> \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567,
> lastCell =
> \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
>at
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
>at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
>at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
>at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
>at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
>at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
>at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
>at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
[jira] [Comment Edited] (HBASE-22846) Internal Error 500 when Using HBASE REST API to Create Namespace.
[ https://issues.apache.org/jira/browse/HBASE-22846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916633#comment-16916633 ] Wellington Chevreuil edited comment on HBASE-22846 at 8/27/19 1:09 PM: --- So we do have UTs for this resource under *TestNamespacesInstanceResource.testNamespaceCreateAndDeletePBAndNoBody.* Noticed the same already tests condition of _no body_ POST requests, and these are always passing. Upon further investigation, I had found out this is due the http request content-type. The passing tests are properly setting it to "application/octet-stream". -Apparently, http clients such as curl default it to "text/html", if not explicitly set-. In this case, jersey is then directing the request to the wrong resource method, triggering the NPE. was (Author: wchevreuil): So we do have UTs for this resource under *TestNamespacesInstanceResource.testNamespaceCreateAndDeletePBAndNoBody.* Noticed the same already tests condition of _no body_ POST requests, and these are always passing. Upon further investigation, I had found out this is due the http request content-type. The passing tests are properly setting it to "application/octet-stream". Apparently, http clients such as curl default it to "text/html", if not explicitly set. In this case, jersey is then directing the request to the wrong resource method, triggering the NPE. > Internal Error 500 when Using HBASE REST API to Create Namespace. > - > > Key: HBASE-22846 > URL: https://issues.apache.org/jira/browse/HBASE-22846 > Project: HBase > Issue Type: Improvement > Components: hbase-connectors >Affects Versions: 2.1.1 >Reporter: Sailesh Patel >Assignee: Wellington Chevreuil >Priority: Major > > When trying to the following URL to create namespace: > Secured cluster: curl --negotiate -u : -i -k -vi -X POST > "http://HBASE_REST_API_HOST:20550/namespaces/datasparktest"; > UnSecured cluster: curl -vi -X POST > "http://HBASE_REST_API_HOST:20550/namespaces/datasparktest"; > The following is returned on the console: > HTTP/1.1 500 Request failed. > The Error in Hbase REST Server log is: > 2019-08-13 15:44:55,080 WARN org.eclipse.jetty.servlet.ServletHandler: > javax.servlet.ServletException: java.lang.NullPointerException > at > org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489) > at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) > ... > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hbase.rest.NamespacesInstanceResource.createOrUpdate(NamespacesInstanceResource.java:250) > at > org.apache.hadoop.hbase.rest.NamespacesInstanceResource.processUpdate(NamespacesInstanceResource.java:243) > at > org.apache.hadoop.hbase.rest.NamespacesInstanceResource.post(NamespacesInstanceResource.java:183) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work stopped] (HBASE-22729) Start RSGroupInfoManager as default
[ https://issues.apache.org/jira/browse/HBASE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-22729 stopped by Duo Zhang. - > Start RSGroupInfoManager as default > --- > > Key: HBASE-22729 > URL: https://issues.apache.org/jira/browse/HBASE-22729 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Duo Zhang >Priority: Major > > Start RSGroupInfoManager as default. If no rsgroup information, all > RegionServers belong to default group. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22729) Start RSGroupInfoManager as default
[ https://issues.apache.org/jira/browse/HBASE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916707#comment-16916707 ] Duo Zhang commented on HBASE-22729: --- Checked the way we deal with AccessController, seems we still make use of Coprocessor? What is the plan there? > Start RSGroupInfoManager as default > --- > > Key: HBASE-22729 > URL: https://issues.apache.org/jira/browse/HBASE-22729 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Duo Zhang >Priority: Major > > Start RSGroupInfoManager as default. If no rsgroup information, all > RegionServers belong to default group. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22846) Internal Error 500 when Using HBASE REST API to Create Namespace.
[
https://issues.apache.org/jira/browse/HBASE-22846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916708#comment-16916708
]
Wellington Chevreuil commented on HBASE-22846:
--
Actually, if not set, http requests are coming with no "content-type" header
param at all. Here's an example from curl command _curl -vi -X POST
"http://localhost:20550/namespaces/datasparktest22"_:
{noformat}
2019-08-27 13:04:10,424 DEBUG [qtp293508253-41] server.HttpChannel: REQUEST for
//localhost:20550/namespaces/datasparktest22 on
HttpChannelOverHttp@7837ac6a{r=1,c=false,a=IDLE,uri=//localhost:20550/namespaces/datasparktest22}
POST //localhost:20550/namespaces/datasparktest22 HTTP/1.1
User-Agent: curl/7.29.0
Host: localhost:20550
Accept: */* {noformat}
Another example using _postman_ http client:
{noformat}
2019-08-27 13:07:22,191 DEBUG [qtp293508253-44] server.HttpChannel: REQUEST for
//192.168.56.105:20550/namespaces/datasparktest182 on
HttpChannelOverHttp@1e9640f3{r=1,c=false,a=IDLE,uri=//192.168.56.105:20550/namespaces/datasparktest182}
POST //192.168.56.105:20550/namespaces/datasparktest182 HTTP/1.1
User-Agent: PostmanRuntime/7.15.2
Accept: */*
Cache-Control: no-cache
Postman-Token: c825af07-60fb-4135-9429-60bb52d410f9
Host: 192.168.56.105:20550
Accept-Encoding: gzip, deflate
Content-Length: 0
Connection: keep-alive {noformat}
Both result in the NPE observed here. If we set any "content-type", then jersey
directs to the proper "noBody" method, in the case of no payload specified. If
we agree that even when no "content-type" is specified, HBase REST should still
fulfil the request, then current PR changes should be ok. I can add extra
testing logic for such cases. Otherwise, we would need to define extra
validations, in order to return a 400 code, since this would be a client
invalid request.
> Internal Error 500 when Using HBASE REST API to Create Namespace.
> -
>
> Key: HBASE-22846
> URL: https://issues.apache.org/jira/browse/HBASE-22846
> Project: HBase
> Issue Type: Improvement
> Components: hbase-connectors
>Affects Versions: 2.1.1
>Reporter: Sailesh Patel
>Assignee: Wellington Chevreuil
>Priority: Major
>
> When trying to the following URL to create namespace:
> Secured cluster: curl --negotiate -u : -i -k -vi -X POST
> "http://HBASE_REST_API_HOST:20550/namespaces/datasparktest";
> UnSecured cluster: curl -vi -X POST
> "http://HBASE_REST_API_HOST:20550/namespaces/datasparktest";
> The following is returned on the console:
> HTTP/1.1 500 Request failed.
> The Error in Hbase REST Server log is:
> 2019-08-13 15:44:55,080 WARN org.eclipse.jetty.servlet.ServletHandler:
> javax.servlet.ServletException: java.lang.NullPointerException
> at
> org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489)
> at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
> ...
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.rest.NamespacesInstanceResource.createOrUpdate(NamespacesInstanceResource.java:250)
> at
> org.apache.hadoop.hbase.rest.NamespacesInstanceResource.processUpdate(NamespacesInstanceResource.java:243)
> at
> org.apache.hadoop.hbase.rest.NamespacesInstanceResource.post(NamespacesInstanceResource.java:183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (HBASE-20818) Use TableDescriptor to replace HTableDescriptor in admin.rb
[ https://issues.apache.org/jira/browse/HBASE-20818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-20818. --- Resolution: Duplicate > Use TableDescriptor to replace HTableDescriptor in admin.rb > --- > > Key: HBASE-20818 > URL: https://issues.apache.org/jira/browse/HBASE-20818 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 2.0.0 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Minor > > HTableDescriptor is deprecated as of release 2.0.0. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-19975) Remove dead servers from rsgroup
[ https://issues.apache.org/jira/browse/HBASE-19975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916716#comment-16916716 ] Duo Zhang commented on HBASE-19975: --- Is this still alive? I think we should not remove dead servers automatically, at least should not remove it immediately, otherwise it will confuse users. Maybe we could introduce a config that, if the servers dies for a long time, then we remove it from the rs group? > Remove dead servers from rsgroup > > > Key: HBASE-19975 > URL: https://issues.apache.org/jira/browse/HBASE-19975 > Project: HBase > Issue Type: Improvement > Components: rsgroup >Affects Versions: 2.0.0-beta-2 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > Currently only decommissioned servers are allowed to be removed from a > rsgroup. > There are no regions will be assigned to dead servers too, and may dead > servers need to be removed more often by users. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HBASE-22933) Do not need to kick reassign for rs group change any more
Duo Zhang created HBASE-22933:
-
Summary: Do not need to kick reassign for rs group change any more
Key: HBASE-22933
URL: https://issues.apache.org/jira/browse/HBASE-22933
Project: HBase
Issue Type: Improvement
Reporter: Duo Zhang
The old implementation is a bit strange, the isStuck method is like this:
{code}
public boolean isStuck() {
return isInState(State.FAILED_OPEN) && getProcedure() != null;
}
{code}
It can only return true when there are ongoing procedures. But if we have a
procedure, then the procedure will try to reassign region. Scheduling a new
procedure does not make sense here, at least for branch-2.2+.
I suggest we just remove the related code, since the default retry number for
assigning a region is Integer.MAX_VALUE. And even if user set this to small
value and finally the region is left in FAILED_OPEN state without a procedure,
HBCK2 is used to deal with this, it is not necessary to deal it automatically.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
