[jira] [Updated] (HBASE-28656) Optimize the checksum logic in ExportSnapshot

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-28656:

Description: In 
[HBASE-28625|https://issues.apache.org/jira/browse/HBASE-28625], we added 
checksum logic comparison for ExportSnapshot files, but the checksum 
verification scenarios were too simple. This issue aims to address the existing 
problem.  (was: In 
[HBASE-28652|https://issues.apache.org/jira/browse/HBASE-28652], we added 
checksum logic comparison for ExportSnapshot files, but the checksum 
verification scenarios were too simple. This issue aims to address the existing 
problem.)

> Optimize the checksum logic in ExportSnapshot
> -
>
> Key: HBASE-28656
> URL: https://issues.apache.org/jira/browse/HBASE-28656
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Critical
>
> In [HBASE-28625|https://issues.apache.org/jira/browse/HBASE-28625], we added 
> checksum logic comparison for ExportSnapshot files, but the checksum 
> verification scenarios were too simple. This issue aims to address the 
> existing problem.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-28656) Optimize the checksum logic in ExportSnapshot

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-28656:

Description: In 
[HBASE-28652|https://issues.apache.org/jira/browse/HBASE-28652], we added 
checksum logic comparison for ExportSnapshot files, but the checksum 
verification scenarios were too simple. This issue aims to address the existing 
problem.  (was: ExportSnapshot should verify checksums for the source file and 
the target file)

> Optimize the checksum logic in ExportSnapshot
> -
>
> Key: HBASE-28656
> URL: https://issues.apache.org/jira/browse/HBASE-28656
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Critical
>
> In [HBASE-28652|https://issues.apache.org/jira/browse/HBASE-28652], we added 
> checksum logic comparison for ExportSnapshot files, but the checksum 
> verification scenarios were too simple. This issue aims to address the 
> existing problem.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-28656) Optimize the checksum logic in ExportSnapshot

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-28656:

Description: ExportSnapshot should verify checksums for the source file and 
the target file

> Optimize the checksum logic in ExportSnapshot
> -
>
> Key: HBASE-28656
> URL: https://issues.apache.org/jira/browse/HBASE-28656
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Critical
>
> ExportSnapshot should verify checksums for the source file and the target file



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-28656) Optimize the checksum logic in ExportSnapshot

[Liangjun He (Jira)]

Liangjun He created HBASE-28656:
---

 Summary: Optimize the checksum logic in ExportSnapshot
 Key: HBASE-28656
 URL: https://issues.apache.org/jira/browse/HBASE-28656
 Project: HBase
  Issue Type: Improvement
Reporter: Liangjun He
Assignee: Liangjun He






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-28625) ExportSnapshot should verify checksums for the source file and the target file

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-28625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852728#comment-17852728
 ] 

Liangjun He commented on HBASE-28625:
-

I have cherry-picked this pr to branch-3, branch-2, branch-2.6, branch-2.5. 

> ExportSnapshot should verify checksums for the source file and the target file
> --
>
> Key: HBASE-28625
> URL: https://issues.apache.org/jira/browse/HBASE-28625
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
> In our cluster, we encountered cases where the target hfile was corrupted 
> after executing ExportSnapshot. [HBASE-13588 
> |https://issues.apache.org/jira/browse/HBASE-13588] can only  checksum data 
> transferred, but cannot solve our problem. Therefore, we believe it is 
> necessary to verify checksums on the files exported by ExportSnapshot.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-28625) ExportSnapshot should verify checksums for the source file and the target file

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-28625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852727#comment-17852727
 ] 

Liangjun He commented on HBASE-28625:
-

I have cherry-picked this pr to branch-3, branch-2, branch-2.6, branch-2.5. 
[~zhangduo]

> ExportSnapshot should verify checksums for the source file and the target file
> --
>
> Key: HBASE-28625
> URL: https://issues.apache.org/jira/browse/HBASE-28625
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
> In our cluster, we encountered cases where the target hfile was corrupted 
> after executing ExportSnapshot. [HBASE-13588 
> |https://issues.apache.org/jira/browse/HBASE-13588] can only  checksum data 
> transferred, but cannot solve our problem. Therefore, we believe it is 
> necessary to verify checksums on the files exported by ExportSnapshot.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] (HBASE-28625) ExportSnapshot should verify checksums for the source file and the target file

[Liangjun He (Jira)]

[ https://issues.apache.org/jira/browse/HBASE-28625 ]


Liangjun He deleted comment on HBASE-28625:
-

was (Author: heliangjun):
I have cherry-picked this pr to branch-3, branch-2, branch-2.6, branch-2.5. 
[~zhangduo]

> ExportSnapshot should verify checksums for the source file and the target file
> --
>
> Key: HBASE-28625
> URL: https://issues.apache.org/jira/browse/HBASE-28625
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
> In our cluster, we encountered cases where the target hfile was corrupted 
> after executing ExportSnapshot. [HBASE-13588 
> |https://issues.apache.org/jira/browse/HBASE-13588] can only  checksum data 
> transferred, but cannot solve our problem. Therefore, we believe it is 
> necessary to verify checksums on the files exported by ExportSnapshot.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (HBASE-28625) ExportSnapshot should verify checksums for the source file and the target file

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He resolved HBASE-28625.
-
Fix Version/s: 3.0.0-beta-2
   Resolution: Fixed

> ExportSnapshot should verify checksums for the source file and the target file
> --
>
> Key: HBASE-28625
> URL: https://issues.apache.org/jira/browse/HBASE-28625
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
>
> In our cluster, we encountered cases where the target hfile was corrupted 
> after executing ExportSnapshot. [HBASE-13588 
> |https://issues.apache.org/jira/browse/HBASE-13588] can only  checksum data 
> transferred, but cannot solve our problem. Therefore, we believe it is 
> necessary to verify checksums on the files exported by ExportSnapshot.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-28625) ExportSnapshot should verify checksums for the source file and the target file

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-28625:

Description: In our cluster, we encountered cases where the target hfile 
was corrupted after executing ExportSnapshot. [HBASE-13588 
|https://issues.apache.org/jira/browse/HBASE-13588] can only  checksum data 
transferred, but cannot solve our problem. Therefore, we believe it is 
necessary to verify checksums on the files exported by ExportSnapshot.  (was: 
In our cluster, we encountered cases where the target hfile was corrupted after 
executing ExportSnapshot. [HBASE-13588 
|https://issues.apache.org/jira/browse/HBASE-13588] can only  checksum data 
transferred, but cannot solve our problem. Therefore, we believe it is 
necessary to perform checksum verification on the files exported by 
ExportSnapshot.)

> ExportSnapshot should verify checksums for the source file and the target file
> --
>
> Key: HBASE-28625
> URL: https://issues.apache.org/jira/browse/HBASE-28625
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Major
>
> In our cluster, we encountered cases where the target hfile was corrupted 
> after executing ExportSnapshot. [HBASE-13588 
> |https://issues.apache.org/jira/browse/HBASE-13588] can only  checksum data 
> transferred, but cannot solve our problem. Therefore, we believe it is 
> necessary to verify checksums on the files exported by ExportSnapshot.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-28625) ExportSnapshot should verify checksums for the source file and the target file

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-28625:

Description: In our cluster, we encountered cases where the target hfile 
was corrupted after executing ExportSnapshot. [HBASE-13588 
|https://issues.apache.org/jira/browse/HBASE-13588] can only  checksum data 
transferred, but cannot solve our problem. Therefore, we believe it is 
necessary to perform checksum verification on the files exported by 
ExportSnapshot.

> ExportSnapshot should verify checksums for the source file and the target file
> --
>
> Key: HBASE-28625
> URL: https://issues.apache.org/jira/browse/HBASE-28625
> Project: HBase
>  Issue Type: Improvement
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Major
>
> In our cluster, we encountered cases where the target hfile was corrupted 
> after executing ExportSnapshot. [HBASE-13588 
> |https://issues.apache.org/jira/browse/HBASE-13588] can only  checksum data 
> transferred, but cannot solve our problem. Therefore, we believe it is 
> necessary to perform checksum verification on the files exported by 
> ExportSnapshot.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-28625) ExportSnapshot should verify checksums for the source file and the target file

[Liangjun He (Jira)]

Liangjun He created HBASE-28625:
---

 Summary: ExportSnapshot should verify checksums for the source 
file and the target file
 Key: HBASE-28625
 URL: https://issues.apache.org/jira/browse/HBASE-28625
 Project: HBase
  Issue Type: Improvement
Reporter: Liangjun He
Assignee: Liangjun He






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-28587) Remove deprecated methods in Cell

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He reassigned HBASE-28587:
---

Assignee: (was: Liangjun He)

> Remove deprecated methods in Cell
> -
>
> Key: HBASE-28587
> URL: https://issues.apache.org/jira/browse/HBASE-28587
> Project: HBase
>  Issue Type: Sub-task
>  Components: API, Client
>Reporter: Duo Zhang
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Work started] (HBASE-28588) Remove deprecated methods in WAL

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-28588 started by Liangjun He.
---
> Remove deprecated methods in WAL
> 
>
> Key: HBASE-28588
> URL: https://issues.apache.org/jira/browse/HBASE-28588
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Work stopped] (HBASE-28587) Remove deprecated methods in Cell

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-28587 stopped by Liangjun He.
---
> Remove deprecated methods in Cell
> -
>
> Key: HBASE-28587
> URL: https://issues.apache.org/jira/browse/HBASE-28587
> Project: HBase
>  Issue Type: Sub-task
>  Components: API, Client
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Work stopped] (HBASE-28588) Remove deprecated methods in WAL

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-28588 stopped by Liangjun He.
---
> Remove deprecated methods in WAL
> 
>
> Key: HBASE-28588
> URL: https://issues.apache.org/jira/browse/HBASE-28588
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Work started] (HBASE-28588) Remove deprecated methods in WAL

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-28588 started by Liangjun He.
---
> Remove deprecated methods in WAL
> 
>
> Key: HBASE-28588
> URL: https://issues.apache.org/jira/browse/HBASE-28588
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Work started] (HBASE-28587) Remove deprecated methods in Cell

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-28587 started by Liangjun He.
---
> Remove deprecated methods in Cell
> -
>
> Key: HBASE-28587
> URL: https://issues.apache.org/jira/browse/HBASE-28587
> Project: HBase
>  Issue Type: Sub-task
>  Components: API, Client
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-28581) Remove deprecated methods in TableDescriotorBuilder

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-28581:

Summary: Remove deprecated methods in TableDescriotorBuilder  (was: Remove 
deprecated methods in TableDescriptor)

> Remove deprecated methods in TableDescriotorBuilder
> ---
>
> Key: HBASE-28581
> URL: https://issues.apache.org/jira/browse/HBASE-28581
> Project: HBase
>  Issue Type: Sub-task
>  Components: API, Client
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-28588) Remove deprecated methods in WAL

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He reassigned HBASE-28588:
---

Assignee: Liangjun He

> Remove deprecated methods in WAL
> 
>
> Key: HBASE-28588
> URL: https://issues.apache.org/jira/browse/HBASE-28588
> Project: HBase
>  Issue Type: Sub-task
>  Components: wal
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-28587) Remove deprecated methods in Cell

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He reassigned HBASE-28587:
---

Assignee: Liangjun He

> Remove deprecated methods in Cell
> -
>
> Key: HBASE-28587
> URL: https://issues.apache.org/jira/browse/HBASE-28587
> Project: HBase
>  Issue Type: Sub-task
>  Components: API, Client
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Work started] (HBASE-28581) Remove deprecated methods in TableDescriptor

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-28581 started by Liangjun He.
---
> Remove deprecated methods in TableDescriptor
> 
>
> Key: HBASE-28581
> URL: https://issues.apache.org/jira/browse/HBASE-28581
> Project: HBase
>  Issue Type: Sub-task
>  Components: API, Client
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Work started] (HBASE-28576) Remove FirstKeyValueMatchingQualifiersFilter

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-28576 started by Liangjun He.
---
> Remove FirstKeyValueMatchingQualifiersFilter
> 
>
> Key: HBASE-28576
> URL: https://issues.apache.org/jira/browse/HBASE-28576
> Project: HBase
>  Issue Type: Sub-task
>  Components: Filters
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-28581) Remove deprecated methods in TableDescriptor

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He reassigned HBASE-28581:
---

Assignee: Liangjun He

> Remove deprecated methods in TableDescriptor
> 
>
> Key: HBASE-28581
> URL: https://issues.apache.org/jira/browse/HBASE-28581
> Project: HBase
>  Issue Type: Sub-task
>  Components: API, Client
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-28576) Remove FirstKeyValueMatchingQualifiersFilter

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He reassigned HBASE-28576:
---

Assignee: Liangjun He

> Remove FirstKeyValueMatchingQualifiersFilter
> 
>
> Key: HBASE-28576
> URL: https://issues.apache.org/jira/browse/HBASE-28576
> Project: HBase
>  Issue Type: Sub-task
>  Components: Filters
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27936) NPE in StoreFileReader.passesGeneralRowPrefixBloomFilter()

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735816#comment-17735816
 ] 

Liangjun He commented on HBASE-27936:
-

[~zhangduo] sir, Perhaps “CONFIGURATION => 
{'RowPrefixBloomFilter.prefix_length' => '10'}” cause the problem?

> NPE in StoreFileReader.passesGeneralRowPrefixBloomFilter()
> --
>
> Key: HBASE-27936
> URL: https://issues.apache.org/jira/browse/HBASE-27936
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Reporter: Liangjun He
>Assignee: Duo Zhang
>Priority: Major
>
> When executing itbll, we encountered the following NPE exception：
> {code:java}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
> at 
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1176)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2407)
> at 
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:667)
> at 
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:716)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27936) NPE in StoreFileReader.passesGeneralRowPrefixBloomFilter()

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733098#comment-17733098
 ] 

Liangjun He commented on HBASE-27936:
-

This issue is discussed in HBASE-27919.

> NPE in StoreFileReader.passesGeneralRowPrefixBloomFilter()
> --
>
> Key: HBASE-27936
> URL: https://issues.apache.org/jira/browse/HBASE-27936
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Reporter: Liangjun He
>Priority: Major
>
> When executing itbll, we encountered the following NPE exception：
> {code:java}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
> at 
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1176)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2407)
> at 
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:667)
> at 
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:716)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27919) Run ITBLL for branch-3

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733096#comment-17733096
 ] 

Liangjun He commented on HBASE-27919:
-

I have created HBASE-27936 to track NPE problem.

> Run ITBLL for branch-3
> --
>
> Key: HBASE-27919
> URL: https://issues.apache.org/jira/browse/HBASE-27919
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (HBASE-27919) Run ITBLL for branch-3

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732962#comment-17732962
 ] 

Liangjun He edited comment on HBASE-27919 at 6/15/23 2:21 PM:
--

Setup a 5 slave nodes cluster, and run this command: 


{code:java}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll -m slowDeterministic   Loop 10 10 
1 /tmp/biglinkedlist 100
{code}

Some RegionServer are experiencing the following error:
{code:java}
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
at 
org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
at 
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
at 
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1176)
at 
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2407)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:667)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:716)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
{code}

And the definition of the itbll table is as follows: 
{code:java}
'itbll', {TABLE_ATTRIBUTES => {METADATA => {'SPLIT_POLICY' => 
'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy', 
'hbase.store.file-tracker.impl' => 'DEFAULT'}}}, {NAME => 'EPvNy', VERSIONS => 
'3', MIN_VERSIONS => '3', METADATA => {'COMPRESSION_COMPACT' => 'BZIP2'}}, 
{NAME => 'IrePz', VERSIONS => '3', MIN_VERSIONS => '3', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}, {NAME => 'big', VERSIONS => '3', 
MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}, {NAME => 'meta', VERSIONS => '3', 
MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}, CONFIGURATION => 
{'RowPrefixBloomFilter.prefix_length' => '10'}}, {NAME => 'tiny', VERSIONS => 
'3', MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}
{code}

[~zhangduo] sir, can you take a look at this issue, thanks?
I have created HBASE-27936 to track it.





was (Author: heliangjun):
Setup a 5 slave nodes cluster, and run this command: 


{code:java}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll -m slowDeterministic   Loop 10 10 
1 /tmp/biglinkedlist 100
{code}

Some RegionServer are experiencing the following error:
{code:java}
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
at 
org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
at 
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
at 

[jira] [Updated] (HBASE-27936) NPE in StoreFileReader.passesGeneralRowPrefixBloomFilter()

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-27936:

External issue URL:   (was: 
https://issues.apache.org/jira/browse/HBASE-27919)

> NPE in StoreFileReader.passesGeneralRowPrefixBloomFilter()
> --
>
> Key: HBASE-27936
> URL: https://issues.apache.org/jira/browse/HBASE-27936
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Reporter: Liangjun He
>Priority: Major
>
> When executing itbll, we encountered the following NPE exception：
> {code:java}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
> at 
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1176)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2407)
> at 
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:667)
> at 
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:716)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27936) NPE in StoreFileReader.passesGeneralRowPrefixBloomFilter()

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-27936:

 External issue ID:   (was: HBASE-27919)
External issue URL: https://issues.apache.org/jira/browse/HBASE-27919

> NPE in StoreFileReader.passesGeneralRowPrefixBloomFilter()
> --
>
> Key: HBASE-27936
> URL: https://issues.apache.org/jira/browse/HBASE-27936
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Reporter: Liangjun He
>Priority: Major
>
> When executing itbll, we encountered the following NPE exception：
> {code:java}
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
> at 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
> at 
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
> at 
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
> at 
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
> at 
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1176)
> at 
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2407)
> at 
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:667)
> at 
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:716)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27936) NPE in StoreFileReader.passesGeneralRowPrefixBloomFilter()

[Liangjun He (Jira)]

Liangjun He created HBASE-27936:
---

 Summary: NPE in StoreFileReader.passesGeneralRowPrefixBloomFilter()
 Key: HBASE-27936
 URL: https://issues.apache.org/jira/browse/HBASE-27936
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: Liangjun He


When executing itbll, we encountered the following NPE exception：
{code:java}
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
at 
org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
at 
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
at 
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1176)
at 
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2407)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:667)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:716)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
{code}






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27919) Run ITBLL for branch-3

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732985#comment-17732985
 ] 

Liangjun He commented on HBASE-27919:
-

Another error info：
{code:java}
2023-06-09 14:44:15,386 ERROR [regionserver/core-1-2:16020-shortCompactions-0] 
util.ResourceLeakDetector: LEAK: RefCnt.release() was not called before it's 
garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for 
more information.
Recent access records:
Created at:
org.apache.hadoop.hbase.nio.RefCnt.(RefCnt.java:59)
org.apache.hadoop.hbase.nio.RefCnt.create(RefCnt.java:54)

org.apache.hadoop.hbase.regionserver.MemStoreLABImpl.(MemStoreLABImpl.java:108)
sun.reflect.GeneratedConstructorAccessor13.newInstance(Unknown Source)

sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)

org.apache.hadoop.hbase.util.ReflectionUtils.instantiate(ReflectionUtils.java:55)

org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:43)

org.apache.hadoop.hbase.regionserver.MemStoreLAB.newInstance(MemStoreLAB.java:116)

org.apache.hadoop.hbase.regionserver.SegmentFactory.createMutableSegment(SegmentFactory.java:81)

org.apache.hadoop.hbase.regionserver.AbstractMemStore.resetActive(AbstractMemStore.java:93)

org.apache.hadoop.hbase.regionserver.AbstractMemStore.(AbstractMemStore.java:83)

org.apache.hadoop.hbase.regionserver.DefaultMemStore.(DefaultMemStore.java:79)
sun.reflect.GeneratedConstructorAccessor12.newInstance(Unknown Source)

sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)

org.apache.hadoop.hbase.util.ReflectionUtils.instantiate(ReflectionUtils.java:55)

org.apache.hadoop.hbase.util.ReflectionUtils.newInstance(ReflectionUtils.java:92)
org.apache.hadoop.hbase.regionserver.HStore.getMemstore(HStore.java:377)
org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:283)

org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6904)
org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1173)
org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1170)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
java.util.concurrent.FutureTask.run(FutureTask.java:266)

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:750)
{code}


> Run ITBLL for branch-3
> --
>
> Key: HBASE-27919
> URL: https://issues.apache.org/jira/browse/HBASE-27919
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (HBASE-27919) Run ITBLL for branch-3

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732962#comment-17732962
 ] 

Liangjun He edited comment on HBASE-27919 at 6/15/23 9:44 AM:
--

Setup a 5 slave nodes cluster, and run this command: 


{code:java}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll -m slowDeterministic   Loop 10 10 
1 /tmp/biglinkedlist 100
{code}

Some RegionServer are experiencing the following error:
{code:java}
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
at 
org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
at 
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
at 
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1176)
at 
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2407)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:667)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:716)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
{code}

And the definition of the itbll table is as follows: 
{code:java}
'itbll', {TABLE_ATTRIBUTES => {METADATA => {'SPLIT_POLICY' => 
'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy', 
'hbase.store.file-tracker.impl' => 'DEFAULT'}}}, {NAME => 'EPvNy', VERSIONS => 
'3', MIN_VERSIONS => '3', METADATA => {'COMPRESSION_COMPACT' => 'BZIP2'}}, 
{NAME => 'IrePz', VERSIONS => '3', MIN_VERSIONS => '3', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}, {NAME => 'big', VERSIONS => '3', 
MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}, {NAME => 'meta', VERSIONS => '3', 
MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}, CONFIGURATION => 
{'RowPrefixBloomFilter.prefix_length' => '10'}}, {NAME => 'tiny', VERSIONS => 
'3', MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}
{code}

[~zhangduo] sir, can you take a look at this issue, thanks?





was (Author: heliangjun):
Setup a 5 slave nodes cluster, and run this command: 


{code:java}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll -m slowDeterministic   Loop 10 10 
1 /tmp/biglinkedlist 100
{code}

Some RegionServer are experiencing the following error:
{code:java}
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
at 
org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
at 
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
at 
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
at 

[jira] [Comment Edited] (HBASE-27919) Run ITBLL for branch-3

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732962#comment-17732962
 ] 

Liangjun He edited comment on HBASE-27919 at 6/15/23 9:42 AM:
--

Setup a 5 slave nodes cluster, and run this command: 


{code:java}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll -m slowDeterministic   Loop 10 10 
1 /tmp/biglinkedlist 100
{code}

Some RegionServer are experiencing the following error:
{code:java}
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
at 
org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
at 
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
at 
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1176)
at 
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2407)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:667)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:716)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
{code}

And the definition of the itbll table is as follows: 
{code:java}
'itbll', {TABLE_ATTRIBUTES => {METADATA => {'SPLIT_POLICY' => 
'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy', 
'hbase.store.file-tracker.impl' => 'DEFAULT'}}}, {NAME => 'EPvNy', VERSIONS => 
'3', MIN_VERSIONS => '3', METADATA => {'COMPRESSION_COMPACT' => 'BZIP2'}}, 
{NAME => 'IrePz', VERSIONS => '3', MIN_VERSIONS => '3', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}, {NAME => 'big', VERSIONS => '3', 
MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}, {NAME => 'meta', VERSIONS => '3', 
MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}, CONFIGURATION => 
{'RowPrefixBloomFilter.prefix_length' => '10'}}, {NAME => 'tiny', VERSIONS => 
'3', MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}
{code}






was (Author: heliangjun):
Setup a 5 slave nodes cluster, and run this command: 


{code:java}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll_table -m slowDeterministic   Loop 10 
10 1 /tmp/biglinkedlist 100
{code}

Some RegionServer are experiencing the following error:
{code:java}
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
at 
org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
at 
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
at 
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
at 

[jira] [Comment Edited] (HBASE-27919) Run ITBLL for branch-3

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732962#comment-17732962
 ] 

Liangjun He edited comment on HBASE-27919 at 6/15/23 9:41 AM:
--

Setup a 5 slave nodes cluster, and run this command: 


{code:java}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll_table -m slowDeterministic   Loop 10 
10 1 /tmp/biglinkedlist 100
{code}

Some RegionServer are experiencing the following error:
{code:java}
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesGeneralRowPrefixBloomFilter(StoreFileReader.java:352)
at 
org.apache.hadoop.hbase.regionserver.StoreFileReader.passesBloomFilter(StoreFileReader.java:265)
at 
org.apache.hadoop.hbase.regionserver.StoreFileScanner.shouldUseScanner(StoreFileScanner.java:483)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.selectScannersFrom(StoreScanner.java:467)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:320)
at 
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:289)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:544)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor$1.createScanner(Compactor.java:269)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:358)
at 
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
at 
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1176)
at 
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2407)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:667)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:716)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
{code}

And the definition of the itbll table is as follows: 
{code:java}
'itbll', {TABLE_ATTRIBUTES => {METADATA => {'SPLIT_POLICY' => 
'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy', 
'hbase.store.file-tracker.impl' => 'DEFAULT'}}}, {NAME => 'EPvNy', VERSIONS => 
'3', MIN_VERSIONS => '3', METADATA => {'COMPRESSION_COMPACT' => 'BZIP2'}}, 
{NAME => 'IrePz', VERSIONS => '3', MIN_VERSIONS => '3', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}, {NAME => 'big', VERSIONS => '3', 
MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}, {NAME => 'meta', VERSIONS => '3', 
MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}, CONFIGURATION => 
{'RowPrefixBloomFilter.prefix_length' => '10'}}, {NAME => 'tiny', VERSIONS => 
'3', MIN_VERSIONS => '3', BLOOMFILTER => 'ROWCOL', METADATA => 
{'COMPRESSION_COMPACT' => 'BZIP2'}}
{code}






was (Author: heliangjun):
Setup a 5 slave nodes cluster, and run this command: 


{code:java}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll_table -m slowDeterministic   Loop 10 
10 1 /tmp/biglinkedlist 100
{code}




> Run ITBLL for branch-3
> --
>
> Key: HBASE-27919
> URL: https://issues.apache.org/jira/browse/HBASE-27919
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (HBASE-27919) Run ITBLL for branch-3

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732962#comment-17732962
 ] 

Liangjun He edited comment on HBASE-27919 at 6/15/23 9:36 AM:
--

Setup a 5 slave nodes cluster, and run this command: 


{code:shell}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll_table -m slowDeterministic   Loop 10 
10 1 /tmp/biglinkedlist 100
{code}





was (Author: heliangjun):
Setup a 5 slave nodes cluster, and run this command: 

{quote}hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll_table -m slowDeterministic   Loop 10 
10 1 /tmp/biglinkedlist 100{quote}

> Run ITBLL for branch-3
> --
>
> Key: HBASE-27919
> URL: https://issues.apache.org/jira/browse/HBASE-27919
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (HBASE-27919) Run ITBLL for branch-3

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732962#comment-17732962
 ] 

Liangjun He edited comment on HBASE-27919 at 6/15/23 9:36 AM:
--

Setup a 5 slave nodes cluster, and run this command: 


{code:java}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll_table -m slowDeterministic   Loop 10 
10 1 /tmp/biglinkedlist 100
{code}





was (Author: heliangjun):
Setup a 5 slave nodes cluster, and run this command: 


{code:shell}
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll_table -m slowDeterministic   Loop 10 
10 1 /tmp/biglinkedlist 100
{code}




> Run ITBLL for branch-3
> --
>
> Key: HBASE-27919
> URL: https://issues.apache.org/jira/browse/HBASE-27919
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27919) Run ITBLL for branch-3

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732962#comment-17732962
 ] 

Liangjun He commented on HBASE-27919:
-

Setup a 5 slave nodes cluster, and run this command: 

{quote}hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList 
-DIntegrationTestBigLinkedList.table=itbll_table -m slowDeterministic   Loop 10 
10 1 /tmp/biglinkedlist 100{quote}

> Run ITBLL for branch-3
> --
>
> Key: HBASE-27919
> URL: https://issues.apache.org/jira/browse/HBASE-27919
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-27919) Run ITBLL for branch-3

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He reassigned HBASE-27919:
---

Assignee: Liangjun He

> Run ITBLL for branch-3
> --
>
> Key: HBASE-27919
> URL: https://issues.apache.org/jira/browse/HBASE-27919
> Project: HBase
>  Issue Type: Sub-task
>  Components: integration tests
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27809) Attach move replication queue storage from zookeeper to a separated HBase table design doc to git repo

[Liangjun He (Jira)]

Liangjun He created HBASE-27809:
---

 Summary: Attach move replication queue storage from zookeeper to a 
separated HBase table design doc to git repo
 Key: HBASE-27809
 URL: https://issues.apache.org/jira/browse/HBASE-27809
 Project: HBase
  Issue Type: Sub-task
  Components: documentation, Replication
Reporter: Liangjun He
Assignee: Liangjun He






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-22978) Online slow response log

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-22978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-22978:

Attachment: (was: Alluxio 开源AI和大数据存储编排平台.pdf)

> Online slow response log
> 
>
> Key: HBASE-22978
> URL: https://issues.apache.org/jira/browse/HBASE-22978
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Operability, regionserver, shell
>Affects Versions: 3.0.0-alpha-1, 2.3.0
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments: 
> NamedQueue_Framework_Design_HBASE-24528_HBASE-22978_HBASE-24718.pdf, Screen 
> Shot 2019-10-19 at 2.31.59 AM.png, Screen Shot 2019-10-19 at 2.32.54 AM.png, 
> Screen Shot 2019-10-19 at 2.34.11 AM.png, Screen Shot 2019-10-19 at 2.36.14 
> AM.png
>
>
> Today when an individual RPC exceeds a configurable time bound we log a 
> complaint by way of the logging subsystem. These log lines look like:
> {noformat}
> 2019-08-30 22:10:36,195 WARN [,queue=15,port=60020] ipc.RpcServer - 
> (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1567203007549,
> "responsesize":6819737,
> "method":"Scan",
> "param":"region { type: REGION_NAME value: 
> \"tsdb,\\000\\000\\215\\f)o\\024\\302\\220\\000\\000\\000\\000\\000\\001\\000\\000\\000\\000\\000\\006\\000\\000\\000\\000\\000\\005\\000\\000",
> "processingtimems":28646,
> "client":"10.253.196.215:41116",
> "queuetimems":22453,
> "class":"HRegionServer"}
> {noformat}
> Unfortunately we often truncate the request parameters, like in the above 
> example. We do this because the human readable representation is verbose, the 
> rate of too slow warnings may be high, and the combination of these things 
> can overwhelm the log capture system. The truncation is unfortunate because 
> it eliminates much of the utility of the warnings. For example, the region 
> name, the start and end keys, and the filter hierarchy are all important 
> clues for debugging performance problems caused by moderate to low 
> selectivity queries or queries made at a high rate.
> We can maintain an in-memory ring buffer of requests that were judged to be 
> too slow in addition to the responseTooSlow logging. The in-memory 
> representation can be complete and compressed. A new admin API and shell 
> command can provide access to the ring buffer for online performance 
> debugging. A modest sizing of the ring buffer will prevent excessive memory 
> utilization for a minor performance debugging feature by limiting the total 
> number of retained records. There is some chance a high rate of requests will 
> cause information on other interesting requests to be overwritten before it 
> can be read. This is the nature of a ring buffer and an acceptable trade off.
> The write request types do not require us to retain all information submitted 
> in the request. We don't need to retain all key-values in the mutation, which 
> may be too large to comfortably retain. We only need a unique set of row 
> keys, or even a min/max range, and total counts.
> The consumers of this information will be debugging tools. We can afford to 
> apply fast compression to ring buffer entries (if codec support is 
> available), something like snappy or zstandard, and decompress on the fly 
> when servicing the retrieval API request. This will minimize the impact of 
> retaining more information about slow requests than we do today.
> This proposal is for retention of request information only, the same 
> information provided by responseTooSlow warnings. Total size of response 
> serialization, possibly also total cell or row counts, should be sufficient 
> to characterize the response.
> Optionally persist new entries added to the ring buffer into one or more 
> files in HDFS in a write-behind manner. If the HDFS writer blocks or falls 
> behind and we are unable to persist an entry before it is overwritten, that 
> is fine. Response too slow logging is best effort. If we can detect this make 
> a note of it in the log file. Provide a tool for parsing, dumping, filtering, 
> and pretty printing the slow logs written to HDFS. The tool and the shell can 
> share and reuse some utility classes and methods for accomplishing that. 
> —
> New shell commands:
> {{get_slow_responses [  ... ,  ] [ , \{  
> } ]}}
> Retrieve, decode, and pretty print the contents of the too slow response ring 
> buffer maintained by the given list of servers; or all servers in the cluster 
> if no list is provided. Optionally provide a map of parameters for filtering 
> as additional argument. The TABLE filter, which expects a string containing a 
> table name, will include only entries pertaining to that table. The REGION 
> 

[jira] [Updated] (HBASE-22978) Online slow response log

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-22978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-22978:

Attachment: (was: Flink Table Store 流计算存储.pptx)

> Online slow response log
> 
>
> Key: HBASE-22978
> URL: https://issues.apache.org/jira/browse/HBASE-22978
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Operability, regionserver, shell
>Affects Versions: 3.0.0-alpha-1, 2.3.0
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments: 
> NamedQueue_Framework_Design_HBASE-24528_HBASE-22978_HBASE-24718.pdf, Screen 
> Shot 2019-10-19 at 2.31.59 AM.png, Screen Shot 2019-10-19 at 2.32.54 AM.png, 
> Screen Shot 2019-10-19 at 2.34.11 AM.png, Screen Shot 2019-10-19 at 2.36.14 
> AM.png
>
>
> Today when an individual RPC exceeds a configurable time bound we log a 
> complaint by way of the logging subsystem. These log lines look like:
> {noformat}
> 2019-08-30 22:10:36,195 WARN [,queue=15,port=60020] ipc.RpcServer - 
> (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1567203007549,
> "responsesize":6819737,
> "method":"Scan",
> "param":"region { type: REGION_NAME value: 
> \"tsdb,\\000\\000\\215\\f)o\\024\\302\\220\\000\\000\\000\\000\\000\\001\\000\\000\\000\\000\\000\\006\\000\\000\\000\\000\\000\\005\\000\\000",
> "processingtimems":28646,
> "client":"10.253.196.215:41116",
> "queuetimems":22453,
> "class":"HRegionServer"}
> {noformat}
> Unfortunately we often truncate the request parameters, like in the above 
> example. We do this because the human readable representation is verbose, the 
> rate of too slow warnings may be high, and the combination of these things 
> can overwhelm the log capture system. The truncation is unfortunate because 
> it eliminates much of the utility of the warnings. For example, the region 
> name, the start and end keys, and the filter hierarchy are all important 
> clues for debugging performance problems caused by moderate to low 
> selectivity queries or queries made at a high rate.
> We can maintain an in-memory ring buffer of requests that were judged to be 
> too slow in addition to the responseTooSlow logging. The in-memory 
> representation can be complete and compressed. A new admin API and shell 
> command can provide access to the ring buffer for online performance 
> debugging. A modest sizing of the ring buffer will prevent excessive memory 
> utilization for a minor performance debugging feature by limiting the total 
> number of retained records. There is some chance a high rate of requests will 
> cause information on other interesting requests to be overwritten before it 
> can be read. This is the nature of a ring buffer and an acceptable trade off.
> The write request types do not require us to retain all information submitted 
> in the request. We don't need to retain all key-values in the mutation, which 
> may be too large to comfortably retain. We only need a unique set of row 
> keys, or even a min/max range, and total counts.
> The consumers of this information will be debugging tools. We can afford to 
> apply fast compression to ring buffer entries (if codec support is 
> available), something like snappy or zstandard, and decompress on the fly 
> when servicing the retrieval API request. This will minimize the impact of 
> retaining more information about slow requests than we do today.
> This proposal is for retention of request information only, the same 
> information provided by responseTooSlow warnings. Total size of response 
> serialization, possibly also total cell or row counts, should be sufficient 
> to characterize the response.
> Optionally persist new entries added to the ring buffer into one or more 
> files in HDFS in a write-behind manner. If the HDFS writer blocks or falls 
> behind and we are unable to persist an entry before it is overwritten, that 
> is fine. Response too slow logging is best effort. If we can detect this make 
> a note of it in the log file. Provide a tool for parsing, dumping, filtering, 
> and pretty printing the slow logs written to HDFS. The tool and the shell can 
> share and reuse some utility classes and methods for accomplishing that. 
> —
> New shell commands:
> {{get_slow_responses [  ... ,  ] [ , \{  
> } ]}}
> Retrieve, decode, and pretty print the contents of the too slow response ring 
> buffer maintained by the given list of servers; or all servers in the cluster 
> if no list is provided. Optionally provide a map of parameters for filtering 
> as additional argument. The TABLE filter, which expects a string containing a 
> table name, will include only entries pertaining to that table. The REGION 
> 

[jira] [Assigned] (HBASE-27516) Document the table based replication queue storage in ref guide

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He reassigned HBASE-27516:
---

Assignee: Liangjun He

> Document the table based replication queue storage in ref guide
> ---
>
> Key: HBASE-27516
> URL: https://issues.apache.org/jira/browse/HBASE-27516
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation, Replication
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27274) Re-enable the disabled tests when implementing HBASE-27212

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711930#comment-17711930
 ] 

Liangjun He commented on HBASE-27274:
-

Only TestHBaseFsckReplication needs to be enabled now. 
I  will create a PR for it.

> Re-enable the disabled tests when implementing HBASE-27212
> --
>
> Key: HBASE-27274
> URL: https://issues.apache.org/jira/browse/HBASE-27274
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>
> For landing the first basic framework of the new replication queue storage 
> implementation, we have to disable several UTs. This issue is used to track 
> these tests to make sure we finally re-enabled them all.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-22978) Online slow response log

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-22978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-22978:

Attachment: Alluxio 开源AI和大数据存储编排平台.pdf
Flink Table Store 流计算存储.pptx

> Online slow response log
> 
>
> Key: HBASE-22978
> URL: https://issues.apache.org/jira/browse/HBASE-22978
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Operability, regionserver, shell
>Affects Versions: 3.0.0-alpha-1, 2.3.0
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments: Alluxio 开源AI和大数据存储编排平台.pdf, Flink Table Store 
> 流计算存储.pptx, 
> NamedQueue_Framework_Design_HBASE-24528_HBASE-22978_HBASE-24718.pdf, Screen 
> Shot 2019-10-19 at 2.31.59 AM.png, Screen Shot 2019-10-19 at 2.32.54 AM.png, 
> Screen Shot 2019-10-19 at 2.34.11 AM.png, Screen Shot 2019-10-19 at 2.36.14 
> AM.png
>
>
> Today when an individual RPC exceeds a configurable time bound we log a 
> complaint by way of the logging subsystem. These log lines look like:
> {noformat}
> 2019-08-30 22:10:36,195 WARN [,queue=15,port=60020] ipc.RpcServer - 
> (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1567203007549,
> "responsesize":6819737,
> "method":"Scan",
> "param":"region { type: REGION_NAME value: 
> \"tsdb,\\000\\000\\215\\f)o\\024\\302\\220\\000\\000\\000\\000\\000\\001\\000\\000\\000\\000\\000\\006\\000\\000\\000\\000\\000\\005\\000\\000",
> "processingtimems":28646,
> "client":"10.253.196.215:41116",
> "queuetimems":22453,
> "class":"HRegionServer"}
> {noformat}
> Unfortunately we often truncate the request parameters, like in the above 
> example. We do this because the human readable representation is verbose, the 
> rate of too slow warnings may be high, and the combination of these things 
> can overwhelm the log capture system. The truncation is unfortunate because 
> it eliminates much of the utility of the warnings. For example, the region 
> name, the start and end keys, and the filter hierarchy are all important 
> clues for debugging performance problems caused by moderate to low 
> selectivity queries or queries made at a high rate.
> We can maintain an in-memory ring buffer of requests that were judged to be 
> too slow in addition to the responseTooSlow logging. The in-memory 
> representation can be complete and compressed. A new admin API and shell 
> command can provide access to the ring buffer for online performance 
> debugging. A modest sizing of the ring buffer will prevent excessive memory 
> utilization for a minor performance debugging feature by limiting the total 
> number of retained records. There is some chance a high rate of requests will 
> cause information on other interesting requests to be overwritten before it 
> can be read. This is the nature of a ring buffer and an acceptable trade off.
> The write request types do not require us to retain all information submitted 
> in the request. We don't need to retain all key-values in the mutation, which 
> may be too large to comfortably retain. We only need a unique set of row 
> keys, or even a min/max range, and total counts.
> The consumers of this information will be debugging tools. We can afford to 
> apply fast compression to ring buffer entries (if codec support is 
> available), something like snappy or zstandard, and decompress on the fly 
> when servicing the retrieval API request. This will minimize the impact of 
> retaining more information about slow requests than we do today.
> This proposal is for retention of request information only, the same 
> information provided by responseTooSlow warnings. Total size of response 
> serialization, possibly also total cell or row counts, should be sufficient 
> to characterize the response.
> Optionally persist new entries added to the ring buffer into one or more 
> files in HDFS in a write-behind manner. If the HDFS writer blocks or falls 
> behind and we are unable to persist an entry before it is overwritten, that 
> is fine. Response too slow logging is best effort. If we can detect this make 
> a note of it in the log file. Provide a tool for parsing, dumping, filtering, 
> and pretty printing the slow logs written to HDFS. The tool and the shell can 
> share and reuse some utility classes and methods for accomplishing that. 
> —
> New shell commands:
> {{get_slow_responses [  ... ,  ] [ , \{  
> } ]}}
> Retrieve, decode, and pretty print the contents of the too slow response ring 
> buffer maintained by the given list of servers; or all servers in the cluster 
> if no list is provided. Optionally provide a map of parameters for filtering 
> as additional argument. The TABLE filter, which expects a string 

[jira] [Updated] (HBASE-27623) Start a new ReplicationSyncUp after the previous failed

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-27623:

Summary: Start a new ReplicationSyncUp after the previous failed  (was: 
Start a new ReplicationSyncUp after the previous ReplicationSyncUp failed)

> Start a new ReplicationSyncUp after the previous failed
> ---
>
> Key: HBASE-27623
> URL: https://issues.apache.org/jira/browse/HBASE-27623
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication, tooling
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27623) Start a new ReplicationSyncUp after the previous ReplicationSyncUp failed

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-27623:

Summary: Start a new ReplicationSyncUp after the previous ReplicationSyncUp 
failed  (was: Avoid starting two ReplicationSyncUp at the same time)

> Start a new ReplicationSyncUp after the previous ReplicationSyncUp failed
> -
>
> Key: HBASE-27623
> URL: https://issues.apache.org/jira/browse/HBASE-27623
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication, tooling
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27623) Avoid starting two ReplicationSyncUp at the same time

[Liangjun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17705400#comment-17705400
 ] 

Liangjun He commented on HBASE-27623:
-

[~zhangduo] Yeah, Sir.
If we want to start a new ReplicationSyncUp after the previous 
ReplicationSyncUp failed, this is indeed a good idea. I will create a pr for 
this.

> Avoid starting two ReplicationSyncUp at the same time
> -
>
> Key: HBASE-27623
> URL: https://issues.apache.org/jira/browse/HBASE-27623
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication, tooling
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-27274) Re-enable the disabled tests when implementing HBASE-27212

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He reassigned HBASE-27274:
---

Assignee: Liangjun He

> Re-enable the disabled tests when implementing HBASE-27212
> --
>
> Key: HBASE-27274
> URL: https://issues.apache.org/jira/browse/HBASE-27274
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>
> For landing the first basic framework of the new replication queue storage 
> implementation, we have to disable several UTs. This issue is used to track 
> these tests to make sure we finally re-enabled them all.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27623) Avoid starting two ReplicationSyncUp at the same time

[Liangjun He (Jira)]

Liangjun He created HBASE-27623:
---

 Summary: Avoid starting two ReplicationSyncUp at the same time
 Key: HBASE-27623
 URL: https://issues.apache.org/jira/browse/HBASE-27623
 Project: HBase
  Issue Type: Sub-task
  Components: Replication, tooling
Reporter: Liangjun He
Assignee: Liangjun He






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Work started] (HBASE-27216) Revisit the ReplicationSyncUp tool

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-27216 started by Liangjun He.
---
> Revisit the ReplicationSyncUp tool
> --
>
> Key: HBASE-27216
> URL: https://issues.apache.org/jira/browse/HBASE-27216
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication, tooling
>Reporter: Duo Zhang
>Assignee: Liangjun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27489) Fix several problems after HBASE-27304

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-27489:

Description: In the HBASE-27304, we support using IP to expose master/rs 
service to client.  But there are several problems that need to be fixed, see 
#[4732|https://github.com/apache/hbase/pull/4732] for details. This issue is 
used to track and fix these problems.  (was: In the HBASE-27304, we support 
using IP to expose master/rs service to client.  But there are several problems 
that need to be fixed. This issue is used to track and fix these problems.)

> Fix several problems after HBASE-27304
> --
>
> Key: HBASE-27489
> URL: https://issues.apache.org/jira/browse/HBASE-27489
> Project: HBase
>  Issue Type: Bug
>  Components: master, regionserver
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Minor
>
> In the HBASE-27304, we support using IP to expose master/rs service to 
> client.  But there are several problems that need to be fixed, see 
> #[4732|https://github.com/apache/hbase/pull/4732] for details. This issue is 
> used to track and fix these problems.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27483) Expose table and region storefiles accessed days and size to the metrics

[Liangjun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liangjun He updated HBASE-27483:

Description: 
At present, we have no metrics to show the hot and cold status of region/table 
storefiles, but from the perspective of storage cost, users are very concerned 
about the hot and cold status of their table data.
So, it's necessary to expose region/table storefiles accessed days and size to 
the metrics.

  was:
At present, we have no metrics to show the hot and cold status of region/table 
data, but from the perspective of storage cost, users are very concerned about 
the hot and cold status of their table data.
So, it's necessary to expose region storefiles accessed days and size to the 
metrics.


> Expose table and region storefiles accessed days and size to the metrics
> 
>
> Key: HBASE-27483
> URL: https://issues.apache.org/jira/browse/HBASE-27483
> Project: HBase
>  Issue Type: New Feature
>  Components: regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: Liangjun He
>Assignee: Liangjun He
>Priority: Minor
> Fix For: 3.0.0-alpha-4
>
>
> At present, we have no metrics to show the hot and cold status of 
> region/table storefiles, but from the perspective of storage cost, users are 
> very concerned about the hot and cold status of their table data.
> So, it's necessary to expose region/table storefiles accessed days and size 
> to the metrics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27483) Expose table and region storefiles accessed days and size to the metrics

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27483:

Priority: Minor  (was: Major)

> Expose table and region storefiles accessed days and size to the metrics
> 
>
> Key: HBASE-27483
> URL: https://issues.apache.org/jira/browse/HBASE-27483
> Project: HBase
>  Issue Type: New Feature
>  Components: regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
> Fix For: 3.0.0-alpha-4
>
>
> At present, we have no metrics to show the hot and cold status of 
> region/table data, but from the perspective of storage cost, users are very 
> concerned about the hot and cold status of their table data.
> So, it's necessary to expose region storefiles accessed days and size to the 
> metrics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27444) Add tool commands list_enabled_tables and list_disabled_tables

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27444:

Release Note: Introduce two shell commands 'list_enabled_tables' and 
'list_disabled_tables' to list enabled or disabled tables.

> Add tool commands list_enabled_tables and list_disabled_tables
> --
>
> Key: HBASE-27444
> URL: https://issues.apache.org/jira/browse/HBASE-27444
> Project: HBase
>  Issue Type: New Feature
>  Components: master
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> Currently, there is no command line tool to get the all enabled or disabled 
> tables, which can only be seen through the HMaster UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-27505) [hbase-thirdparty] Set updatePomFile for maven-flatten-plugin

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He reassigned HBASE-27505:
---

Assignee: LiangJun He

> [hbase-thirdparty] Set updatePomFile for maven-flatten-plugin
> -
>
> Key: HBASE-27505
> URL: https://issues.apache.org/jira/browse/HBASE-27505
> Project: HBase
>  Issue Type: Bug
>  Components: build, pom, thirdparty
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27483) Expose table/region storefiles accessed days and size to the metrics

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27483:

Summary: Expose table/region storefiles accessed days and size to the 
metrics  (was: Expose region storefiles accessed days and size to the metrics)

> Expose table/region storefiles accessed days and size to the metrics
> 
>
> Key: HBASE-27483
> URL: https://issues.apache.org/jira/browse/HBASE-27483
> Project: HBase
>  Issue Type: New Feature
>  Components: regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> At present, we have no metrics to show the hot and cold status of 
> region/table data, but from the perspective of storage cost, users are very 
> concerned about the hot and cold status of their table data.
> So, it's necessary to expose region storefiles accessed days and size to the 
> metrics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27483) Expose table and region storefiles accessed days and size to the metrics

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27483:

Summary: Expose table and region storefiles accessed days and size to the 
metrics  (was: Expose table/region storefiles accessed days and size to the 
metrics)

> Expose table and region storefiles accessed days and size to the metrics
> 
>
> Key: HBASE-27483
> URL: https://issues.apache.org/jira/browse/HBASE-27483
> Project: HBase
>  Issue Type: New Feature
>  Components: regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> At present, we have no metrics to show the hot and cold status of 
> region/table data, but from the perspective of storage cost, users are very 
> concerned about the hot and cold status of their table data.
> So, it's necessary to expose region storefiles accessed days and size to the 
> metrics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27489) Fix several problems after HBASE-27304

[LiangJun He (Jira)]

LiangJun He created HBASE-27489:
---

 Summary: Fix several problems after HBASE-27304
 Key: HBASE-27489
 URL: https://issues.apache.org/jira/browse/HBASE-27489
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Reporter: LiangJun He
Assignee: LiangJun He


In the HBASE-27304, we support using IP to expose master/rs service to client.  
But there are several problems that need to be fixed. This issue is used to 
track and fix these problems.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27483) Expose region storefiles accessed days and size to the metrics

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27483:

Description: 
At present, we have no metrics to show the hot and cold status of region/table 
data, but from the perspective of storage cost, users are very concerned about 
the hot and cold status of their table data.
So, it's necessary to expose region storefiles accessed days and size to the 
metrics.

  was:
At present, we have no metrics to show the hot and cold status of region/table 
data, but from the perspective of storage cost, users are very concerned about 
the hot and cold status of their table data.
So, it's necessary to 


> Expose region storefiles accessed days and size to the metrics
> --
>
> Key: HBASE-27483
> URL: https://issues.apache.org/jira/browse/HBASE-27483
> Project: HBase
>  Issue Type: New Feature
>  Components: regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> At present, we have no metrics to show the hot and cold status of 
> region/table data, but from the perspective of storage cost, users are very 
> concerned about the hot and cold status of their table data.
> So, it's necessary to expose region storefiles accessed days and size to the 
> metrics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27483) Expose region storefiles accessed days and size to the metrics

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27483:

Description: 
At present, we have no metrics to show the hot and cold status of region/table 
data, but from the perspective of storage cost, users are very concerned about 
the hot and cold status of their table data.
So, it's necessary to 

> Expose region storefiles accessed days and size to the metrics
> --
>
> Key: HBASE-27483
> URL: https://issues.apache.org/jira/browse/HBASE-27483
> Project: HBase
>  Issue Type: New Feature
>  Components: regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> At present, we have no metrics to show the hot and cold status of 
> region/table data, but from the perspective of storage cost, users are very 
> concerned about the hot and cold status of their table data.
> So, it's necessary to 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27482) Expose region/table datalocality data to the metrics

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27482:

Description: It is necessary to expose the datalocality of region/table to 
the metrics for collection and monitoring.

> Expose region/table datalocality data to the metrics
> 
>
> Key: HBASE-27482
> URL: https://issues.apache.org/jira/browse/HBASE-27482
> Project: HBase
>  Issue Type: New Feature
>  Components: regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> It is necessary to expose the datalocality of region/table to the metrics for 
> collection and monitoring.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27483) Expose region storefiles accessed days and size to the metrics

[LiangJun He (Jira)]

LiangJun He created HBASE-27483:
---

 Summary: Expose region storefiles accessed days and size to the 
metrics
 Key: HBASE-27483
 URL: https://issues.apache.org/jira/browse/HBASE-27483
 Project: HBase
  Issue Type: New Feature
  Components: regionserver
Affects Versions: 3.0.0-alpha-4
Reporter: LiangJun He
Assignee: LiangJun He
 Fix For: 3.0.0-alpha-4






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27482) Expose region/table datalocality data to the metrics

[LiangJun He (Jira)]

LiangJun He created HBASE-27482:
---

 Summary: Expose region/table datalocality data to the metrics
 Key: HBASE-27482
 URL: https://issues.apache.org/jira/browse/HBASE-27482
 Project: HBase
  Issue Type: New Feature
  Components: regionserver
Affects Versions: 3.0.0-alpha-4
Reporter: LiangJun He
Assignee: LiangJun He
 Fix For: 3.0.0-alpha-4






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27444) Add tool commands list_enabled_tables and list_disabled_tables

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629366#comment-17629366
 ] 

LiangJun He commented on HBASE-27444:
-

[~zhangduo] OK, sir. I will start a new PR for branch-2.

> Add tool commands list_enabled_tables and list_disabled_tables
> --
>
> Key: HBASE-27444
> URL: https://issues.apache.org/jira/browse/HBASE-27444
> Project: HBase
>  Issue Type: New Feature
>  Components: master
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
>
> Currently, there is no command line tool to get the all enabled or disabled 
> tables, which can only be seen through the HMaster UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27444) Add tool commands list_enabled_tables and list_disabled_tables

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27444:

Summary: Add tool commands list_enabled_tables and list_disabled_tables  
(was: Add tool commands list_disabled_tables and list_enabled_tables)

> Add tool commands list_enabled_tables and list_disabled_tables
> --
>
> Key: HBASE-27444
> URL: https://issues.apache.org/jira/browse/HBASE-27444
> Project: HBase
>  Issue Type: New Feature
>  Components: master
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
>
> Currently, there is no command line tool to get the all enabled or disabled 
> tables, which can only be seen through the HMaster UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27444) Add tool commands list_disabled_tables and list_enabled_tables

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27444:

Description: Currently, there is no command line tool to get the all 
enabled or disabled tables, which can only be seen through the HMaster UI.  
(was: Currently, there is no command line tool to get the all disabled tables, 
which can only be seen through the HMaster UI.)

> Add tool commands list_disabled_tables and list_enabled_tables
> --
>
> Key: HBASE-27444
> URL: https://issues.apache.org/jira/browse/HBASE-27444
> Project: HBase
>  Issue Type: New Feature
>  Components: master
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
>
> Currently, there is no command line tool to get the all enabled or disabled 
> tables, which can only be seen through the HMaster UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27444) Add tool commands list_disabled_tables and list_enabled_tables

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27444:

Summary: Add tool commands list_disabled_tables and list_enabled_tables  
(was: Add a tool command list_disabled_tables)

> Add tool commands list_disabled_tables and list_enabled_tables
> --
>
> Key: HBASE-27444
> URL: https://issues.apache.org/jira/browse/HBASE-27444
> Project: HBase
>  Issue Type: New Feature
>  Components: master
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
>
> Currently, there is no command line tool to get the all disabled tables, 
> which can only be seen through the HMaster UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27448) Add an admin method to get replication enabled state

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17626361#comment-17626361
 ] 

LiangJun He commented on HBASE-27448:
-

[~zhangduo] OK, sir. I will start a new PR for branch-2.

> Add an admin method to get replication enabled state
> 
>
> Key: HBASE-27448
> URL: https://issues.apache.org/jira/browse/HBASE-27448
> Project: HBase
>  Issue Type: Improvement
>  Components: Admin, Client
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
>
> When implementing HBASE-27109, it is a bit surprise to me that there is no 
> way for me to get the enabled state of a replication peer directly through 
> Admin interface, I need to call listReplicationPeers to find out the 
> ReplicationPeerDescription and then get the enabled state.
> I think we should add a method in Admin interface for this.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27444) Add a tool command list_disabled_tables

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27444:

Description: Currently, there is no command line tool to get the all 
disabled tables, which can only be seen through the HMaster UI.  (was: 
Currently, there is no command tool to get the all disabled tables, which can 
only be seen through the HMaster UI.)

> Add a tool command list_disabled_tables
> ---
>
> Key: HBASE-27444
> URL: https://issues.apache.org/jira/browse/HBASE-27444
> Project: HBase
>  Issue Type: New Feature
>  Components: master
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
>
> Currently, there is no command line tool to get the all disabled tables, 
> which can only be seen through the HMaster UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27444) Add a tool command list_disabled_tables

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27444:

Description: Currently, there is no command tool to get the all disabled 
tables, which can only be seen through the HMaster UI.

> Add a tool command list_disabled_tables
> ---
>
> Key: HBASE-27444
> URL: https://issues.apache.org/jira/browse/HBASE-27444
> Project: HBase
>  Issue Type: New Feature
>  Components: master
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
>
> Currently, there is no command tool to get the all disabled tables, which can 
> only be seen through the HMaster UI.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-27448) Add an admin method to get replication enabled state

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He reassigned HBASE-27448:
---

Assignee: LiangJun He

> Add an admin method to get replication enabled state
> 
>
> Key: HBASE-27448
> URL: https://issues.apache.org/jira/browse/HBASE-27448
> Project: HBase
>  Issue Type: Improvement
>  Components: Admin, Client
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
>
> When implementing HBASE-27109, it is a bit surprise to me that there is no 
> way for me to get the enabled state of a replication peer directly through 
> Admin interface, I need to call listReplicationPeers to find out the 
> ReplicationPeerDescription and then get the enabled state.
> I think we should add a method in Admin interface for this.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27444) Add a tool command list_disabled_tables

[LiangJun He (Jira)]

LiangJun He created HBASE-27444:
---

 Summary: Add a tool command list_disabled_tables
 Key: HBASE-27444
 URL: https://issues.apache.org/jira/browse/HBASE-27444
 Project: HBase
  Issue Type: New Feature
  Components: master
Affects Versions: 3.0.0-alpha-4
Reporter: LiangJun He
Assignee: LiangJun He






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27405) Fix the replication hfile/log cleaner report that the replication table does not exist

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27405:

Summary: Fix the replication hfile/log cleaner report that the replication 
table does not exist  (was: Fix the replication hfile/log cleaner report that 
the replication table does not exist issue)

> Fix the replication hfile/log cleaner report that the replication table does 
> not exist
> --
>
> Key: HBASE-27405
> URL: https://issues.apache.org/jira/browse/HBASE-27405
> Project: HBase
>  Issue Type: Sub-task
>  Components: master, Replication
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27405) Fix the replication hfile/log cleaner report that the replication table does not exist issue

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27405:

Summary: Fix the replication hfile/log cleaner report that the replication 
table does not exist issue  (was: Fix the exception thrown by the replication 
hfile/log cleaner that the replication table does not exist)

> Fix the replication hfile/log cleaner report that the replication table does 
> not exist issue
> 
>
> Key: HBASE-27405
> URL: https://issues.apache.org/jira/browse/HBASE-27405
> Project: HBase
>  Issue Type: Sub-task
>  Components: master, Replication
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Work started] (HBASE-27405) Fix the exception thrown by the replication hfile/log cleaner that the replication table does not exist

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-27405 started by LiangJun He.
---
> Fix the exception thrown by the replication hfile/log cleaner that the 
> replication table does not exist
> ---
>
> Key: HBASE-27405
> URL: https://issues.apache.org/jira/browse/HBASE-27405
> Project: HBase
>  Issue Type: Sub-task
>  Components: master, Replication
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-27405) Fix the exception thrown by the replication hfile/log cleaner that the replication table does not exist

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He reassigned HBASE-27405:
---

Component/s: master
 Replication
   Assignee: LiangJun He

> Fix the exception thrown by the replication hfile/log cleaner that the 
> replication table does not exist
> ---
>
> Key: HBASE-27405
> URL: https://issues.apache.org/jira/browse/HBASE-27405
> Project: HBase
>  Issue Type: Sub-task
>  Components: master, Replication
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27405) Fix the exception thrown by the replication hfile/log cleaner that the replication table does not exist

[LiangJun He (Jira)]

LiangJun He created HBASE-27405:
---

 Summary: Fix the exception thrown by the replication hfile/log 
cleaner that the replication table does not exist
 Key: HBASE-27405
 URL: https://issues.apache.org/jira/browse/HBASE-27405
 Project: HBase
  Issue Type: Sub-task
Reporter: LiangJun He






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Work started] (HBASE-27217) Revisit the DumpReplicationQueues tool

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-27217 started by LiangJun He.
---
> Revisit the DumpReplicationQueues  tool
> ---
>
> Key: HBASE-27217
> URL: https://issues.apache.org/jira/browse/HBASE-27217
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication, tooling
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27304) Support using IP to expose master/rs servers for some special scenarios

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27304:

Release Note: 
In some scenarios, such as the elastic scaling scenario on the cloud, the HBase 
client may not be able to resolve the hostname of the newly added node. If the 
network is interconnected, the client can actually access the HBase cluster 
nodes through ip. However, since the HBase client obtains the Master/RS address 
info from or the ZK or the meta table, so the Master/RS of the HBase cluster 
needs to expose the service with ip instead of the hostname. Therefore, We can 
use hostname by default, but at the same time, we can also provide a config 
‘hbase.server.useip.enabled’
to support whether to use ip for Master/RS service.

> Support using IP to expose master/rs servers for some special scenarios
> ---
>
> Key: HBASE-27304
> URL: https://issues.apache.org/jira/browse/HBASE-27304
> Project: HBase
>  Issue Type: New Feature
>  Components: master, regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> In our environment, when accessing the HBase cluster from another cluster 
> (such as the Flink cluster), you cannot use the hostname to access the HBase 
> cluster, but you can use IP. Because it is possible that the hostname of the 
> HBase cluster automatic expansion node is not maintained in the Flink 
> cluster. So we should support using IP to expose master/rs servers for this 
> special scenarios.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27304) Support using IP to expose master/rs servers for some special scenarios

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608277#comment-17608277
 ] 

LiangJun He commented on HBASE-27304:
-

OK, sir. I will start a new PR for branch-2.

> Support using IP to expose master/rs servers for some special scenarios
> ---
>
> Key: HBASE-27304
> URL: https://issues.apache.org/jira/browse/HBASE-27304
> Project: HBase
>  Issue Type: New Feature
>  Components: master, regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
> Fix For: 3.0.0-alpha-4
>
>
> In our environment, when accessing the HBase cluster from another cluster 
> (such as the Flink cluster), you cannot use the hostname to access the HBase 
> cluster, but you can use IP. Because it is possible that the hostname of the 
> HBase cluster automatic expansion node is not maintained in the Flink 
> cluster. So we should support using IP to expose master/rs servers for this 
> special scenarios.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27369) BufferedMutator#mutate supports mutation type verification

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27369:

Description: At present, BufferedMutator#mutate method only tell the user 
in the comment that it only supports Put / Delete.  But it is not verified in 
the code implementation, so we need to deal with it.  (was: At present, 
BufferedMutator#mutate only tell the user in the comment that it only supports 
Put / Delete.  But it is not verified in the code implementation, so we need to 
deal with it.)

> BufferedMutator#mutate supports mutation type verification
> --
>
> Key: HBASE-27369
> URL: https://issues.apache.org/jira/browse/HBASE-27369
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
>
> At present, BufferedMutator#mutate method only tell the user in the comment 
> that it only supports Put / Delete.  But it is not verified in the code 
> implementation, so we need to deal with it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27369) BufferedMutator#mutate supports mutation type verification

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27369:

Summary: BufferedMutator#mutate supports mutation type verification  (was: 
BufferedMutator#mutate method supports mutation type verification)

> BufferedMutator#mutate supports mutation type verification
> --
>
> Key: HBASE-27369
> URL: https://issues.apache.org/jira/browse/HBASE-27369
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
>
> At present, BufferedMutator#mutate method only tell the user in the comment 
> that it only supports Put / Delete.  But it is not verified in the code 
> implementation, so we need to deal with it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27369) BufferedMutator#mutate supports mutation type verification

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27369:

Description: At present, BufferedMutator#mutate only tell the user in the 
comment that it only supports Put / Delete.  But it is not verified in the code 
implementation, so we need to deal with it.  (was: At present, 
BufferedMutator#mutate method only tell the user in the comment that it only 
supports Put / Delete.  But it is not verified in the code implementation, so 
we need to deal with it.)

> BufferedMutator#mutate supports mutation type verification
> --
>
> Key: HBASE-27369
> URL: https://issues.apache.org/jira/browse/HBASE-27369
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
>
> At present, BufferedMutator#mutate only tell the user in the comment that it 
> only supports Put / Delete.  But it is not verified in the code 
> implementation, so we need to deal with it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27369) BufferedMutator#mutate method supports mutation type verification

[LiangJun He (Jira)]

LiangJun He created HBASE-27369:
---

 Summary: BufferedMutator#mutate method supports mutation type 
verification
 Key: HBASE-27369
 URL: https://issues.apache.org/jira/browse/HBASE-27369
 Project: HBase
  Issue Type: Bug
  Components: Client
Affects Versions: 3.0.0-alpha-4
Reporter: LiangJun He
Assignee: LiangJun He


At present, BufferedMutator#mutate method only tell the user in the comment 
that it only supports Put / Delete.  But it is not verified in the code 
implementation, so we need to deal with it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27304) Support using IP to expose master/rs servers for some special scenarios

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602929#comment-17602929
 ] 

LiangJun He commented on HBASE-27304:
-

[~anoop.hbase] . Can you explain your way in detail, especially in the 
hadoop-common part. Thanks.

 

> Support using IP to expose master/rs servers for some special scenarios
> ---
>
> Key: HBASE-27304
> URL: https://issues.apache.org/jira/browse/HBASE-27304
> Project: HBase
>  Issue Type: New Feature
>  Components: master, regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
> Fix For: 3.0.0-alpha-4
>
>
> In our environment, when accessing the HBase cluster from another cluster 
> (such as the Flink cluster), you cannot use the hostname to access the HBase 
> cluster, but you can use IP. Because it is possible that the hostname of the 
> HBase cluster automatic expansion node is not maintained in the Flink 
> cluster. So we should support using IP to expose master/rs servers for this 
> special scenarios.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27304) Support using IP to expose master/rs servers for some special scenarios

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601681#comment-17601681
 ] 

LiangJun He commented on HBASE-27304:
-

Thanks for your comments.  [~anoop.hbase] 
As [~frostruan]  said above, if hbase.unsafe.regionserver.hostname.value is 
used to solve this scenario, cluster management will be troublesome, the config 
value needs to be dynamically modified according to the node ip.

> Support using IP to expose master/rs servers for some special scenarios
> ---
>
> Key: HBASE-27304
> URL: https://issues.apache.org/jira/browse/HBASE-27304
> Project: HBase
>  Issue Type: New Feature
>  Components: master, regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
> Fix For: 3.0.0-alpha-4
>
>
> In our environment, when accessing the HBase cluster from another cluster 
> (such as the Flink cluster), you cannot use the hostname to access the HBase 
> cluster, but you can use IP. Because it is possible that the hostname of the 
> HBase cluster automatic expansion node is not maintained in the Flink 
> cluster. So we should support using IP to expose master/rs servers for this 
> special scenarios.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27304) Support using IP to expose master/rs servers for some special scenarios

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601680#comment-17601680
 ] 

LiangJun He commented on HBASE-27304:
-

Thanks for your comments.  [~frostruan]

I'm very glad that this PR can solve your problem. :)

> Support using IP to expose master/rs servers for some special scenarios
> ---
>
> Key: HBASE-27304
> URL: https://issues.apache.org/jira/browse/HBASE-27304
> Project: HBase
>  Issue Type: New Feature
>  Components: master, regionserver
>Affects Versions: 3.0.0-alpha-4
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Minor
> Fix For: 3.0.0-alpha-4
>
>
> In our environment, when accessing the HBase cluster from another cluster 
> (such as the Flink cluster), you cannot use the hostname to access the HBase 
> cluster, but you can use IP. Because it is possible that the hostname of the 
> HBase cluster automatic expansion node is not maintained in the Flink 
> cluster. So we should support using IP to expose master/rs servers for this 
> special scenarios.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-27348) releasing resources when quiting IRB

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He reassigned HBASE-27348:
---

Assignee: LiangJun He

> releasing resources when quiting IRB
> 
>
> Key: HBASE-27348
> URL: https://issues.apache.org/jira/browse/HBASE-27348
> Project: HBase
>  Issue Type: Bug
>Reporter: LiangJun He
>Assignee: LiangJun He
>Priority: Major
>
> Please refer to #[4737|https://github.com/apache/hbase/pull/4737] for details.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27348) releasing resources when quiting IRB

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27348:

Description: Please refer to 
#[4737|https://github.com/apache/hbase/pull/4737] for details.

> releasing resources when quiting IRB
> 
>
> Key: HBASE-27348
> URL: https://issues.apache.org/jira/browse/HBASE-27348
> Project: HBase
>  Issue Type: Bug
>Reporter: LiangJun He
>Priority: Major
>
> Please refer to #[4737|https://github.com/apache/hbase/pull/4737] for details.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-27348) releasing resources when quiting IRB

[LiangJun He (Jira)]

LiangJun He created HBASE-27348:
---

 Summary: releasing resources when quiting IRB
 Key: HBASE-27348
 URL: https://issues.apache.org/jira/browse/HBASE-27348
 Project: HBase
  Issue Type: Bug
Reporter: LiangJun He






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27336) The region visualizer shows 'undefined' region server

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17597492#comment-17597492
 ] 

LiangJun He commented on HBASE-27336:
-

[~ndimiduk] sir.

I am sorry, I looked up the commit history, but I didn't find the commit about 
the change of this problem.

> The region visualizer shows 'undefined' region server
> -
>
> Key: HBASE-27336
> URL: https://issues.apache.org/jira/browse/HBASE-27336
> Project: HBase
>  Issue Type: Bug
>  Components: master, UI
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-4
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
> Attachments: Screen Shot 2022-08-26 at 3.31.27 PM.png, good.png
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27335) HBase shell hang for a minute when quiting

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17586100#comment-17586100
 ] 

LiangJun He commented on HBASE-27335:
-

After I change it to daemon thread, hbase shell can exit normally without 
waiting for 1 minute.

> HBase shell hang for a minute when quiting
> --
>
> Key: HBASE-27335
> URL: https://issues.apache.org/jira/browse/HBASE-27335
> Project: HBase
>  Issue Type: Bug
>  Components: shell
>Affects Versions: 2.5.0, 2.6.0, 2.5.1
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
> Attachments: jstack.log
>
>
> Found this when trying the 2.5.0 RC. The way to reproduce
> Use './bin/hbase shell' to connect to a cluster, and then run these two 
> commands
> {code}
> count "hbase:meta"
> quit
> {code}
> We will hang about 1 minute before the final exit of the shell process.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27335) HBase shell hang for a minute when quiting

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27335:

Affects Version/s: 2.5.0
   2.6.0
   2.5.1

> HBase shell hang for a minute when quiting
> --
>
> Key: HBASE-27335
> URL: https://issues.apache.org/jira/browse/HBASE-27335
> Project: HBase
>  Issue Type: Bug
>  Components: shell
>Affects Versions: 2.5.0, 2.6.0, 2.5.1
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
> Attachments: jstack.log
>
>
> Found this when trying the 2.5.0 RC. The way to reproduce
> Use './bin/hbase shell' to connect to a cluster, and then run these two 
> commands
> {code}
> count "hbase:meta"
> quit
> {code}
> We will hang about 1 minute before the final exit of the shell process.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27335) HBase shell hang for a minute when quiting

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17586096#comment-17586096
 ] 

LiangJun He commented on HBASE-27335:
-

I have seen that this part of the code of branch-1 and master branch sets the 
thread as the daemon thread, only branch-2 is not set. The master branch code 
is as follows:

 
{code:java}
private ThreadPoolExecutor createThreadPool() {
  Configuration conf = conn.getConfiguration();
  int threads = conf.getInt("hbase.hconnection.threads.max", 256);
  long keepAliveTime = conf.getLong("hbase.hconnection.threads.keepalivetime", 
60);
  BlockingQueue workQueue =
new LinkedBlockingQueue<>(threads * 
conf.getInt(HConstants.HBASE_CLIENT_MAX_TOTAL_TASKS,
  HConstants.DEFAULT_HBASE_CLIENT_MAX_TOTAL_TASKS));
  ThreadPoolExecutor tpe = new ThreadPoolExecutor(threads, threads, 
keepAliveTime,
TimeUnit.SECONDS, workQueue,
new ThreadFactoryBuilder().setDaemon(true).setNameFormat(toString() + 
"-shared-%d").build());
  tpe.allowCoreThreadTimeOut(true);
  return tpe;
} {code}
The branch-2 branch code is as follows:
{code:java}
private ThreadPoolExecutor getThreadPool(int maxThreads, int coreThreads, 
String nameHint,
  BlockingQueue passedWorkQueue) {
  // shared HTable thread executor not yet initialized
  if (maxThreads == 0) {
maxThreads = Runtime.getRuntime().availableProcessors() * 8;
  }
  if (coreThreads == 0) {
coreThreads = Runtime.getRuntime().availableProcessors() * 8;
  }
  long keepAliveTime = conf.getLong("hbase.hconnection.threads.keepalivetime", 
60);
  BlockingQueue workQueue = passedWorkQueue;
  if (workQueue == null) {
workQueue =
  new LinkedBlockingQueue<>(maxThreads * 
conf.getInt(HConstants.HBASE_CLIENT_MAX_TOTAL_TASKS,
HConstants.DEFAULT_HBASE_CLIENT_MAX_TOTAL_TASKS));
coreThreads = maxThreads;
  }
  ThreadPoolExecutor tpe =
new ThreadPoolExecutor(coreThreads, maxThreads, keepAliveTime, 
TimeUnit.SECONDS, workQueue,
  new ThreadFactoryBuilder().setNameFormat(toString() + nameHint + 
"-pool-%d")

.setUncaughtExceptionHandler(Threads.LOGGING_EXCEPTION_HANDLER).build());
  tpe.allowCoreThreadTimeOut(true);
  return tpe;
} {code}
I will submit a PR to solve this problem.

 

> HBase shell hang for a minute when quiting
> --
>
> Key: HBASE-27335
> URL: https://issues.apache.org/jira/browse/HBASE-27335
> Project: HBase
>  Issue Type: Bug
>  Components: shell
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
> Attachments: jstack.log
>
>
> Found this when trying the 2.5.0 RC. The way to reproduce
> Use './bin/hbase shell' to connect to a cluster, and then run these two 
> commands
> {code}
> count "hbase:meta"
> quit
> {code}
> We will hang about 1 minute before the final exit of the shell process.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (HBASE-27335) HBase shell hang for a minute when quiting

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He reassigned HBASE-27335:
---

Assignee: LiangJun He

> HBase shell hang for a minute when quiting
> --
>
> Key: HBASE-27335
> URL: https://issues.apache.org/jira/browse/HBASE-27335
> Project: HBase
>  Issue Type: Bug
>  Components: shell
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
> Attachments: jstack.log
>
>
> Found this when trying the 2.5.0 RC. The way to reproduce
> Use './bin/hbase shell' to connect to a cluster, and then run these two 
> commands
> {code}
> count "hbase:meta"
> quit
> {code}
> We will hang about 1 minute before the final exit of the shell process.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27336) The region visualizer shows 'undefined' region server

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17586086#comment-17586086
 ] 

LiangJun He commented on HBASE-27336:
-

[~zhangduo] please see the image 'good.png'.

> The region visualizer shows 'undefined' region server
> -
>
> Key: HBASE-27336
> URL: https://issues.apache.org/jira/browse/HBASE-27336
> Project: HBase
>  Issue Type: Bug
>  Components: master, UI
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-4
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
> Attachments: Screen Shot 2022-08-26 at 3.31.27 PM.png, good.png
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (HBASE-27336) The region visualizer shows 'undefined' region server

[LiangJun He (Jira)]

[ 
https://issues.apache.org/jira/browse/HBASE-27336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17586075#comment-17586075
 ] 

LiangJun He commented on HBASE-27336:
-

I have submitted a PR to solve the problem and uploaded good.png to show the 
effect after the problem is solved.

> The region visualizer shows 'undefined' region server
> -
>
> Key: HBASE-27336
> URL: https://issues.apache.org/jira/browse/HBASE-27336
> Project: HBase
>  Issue Type: Bug
>  Components: master, UI
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-4
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
> Attachments: Screen Shot 2022-08-26 at 3.31.27 PM.png, good.png
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27336) The region visualizer shows 'undefined' region server

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27336:

Attachment: good.png

> The region visualizer shows 'undefined' region server
> -
>
> Key: HBASE-27336
> URL: https://issues.apache.org/jira/browse/HBASE-27336
> Project: HBase
>  Issue Type: Bug
>  Components: master, UI
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-4
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
> Attachments: Screen Shot 2022-08-26 at 3.31.27 PM.png, good.png
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HBASE-27336) The region visualizer shows 'undefined' region server

[LiangJun He (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

LiangJun He updated HBASE-27336:

Affects Version/s: 2.5.0
   2.6.0
   3.0.0-alpha-4

> The region visualizer shows 'undefined' region server
> -
>
> Key: HBASE-27336
> URL: https://issues.apache.org/jira/browse/HBASE-27336
> Project: HBase
>  Issue Type: Bug
>  Components: master, UI
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-4
>Reporter: Duo Zhang
>Assignee: LiangJun He
>Priority: Major
> Attachments: Screen Shot 2022-08-26 at 3.31.27 PM.png
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)