[jira] [Comment Edited] (HBASE-20714) Document REST curl commands for supported (but missing in docs) operations
[ https://issues.apache.org/jira/browse/HBASE-20714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508253#comment-16508253 ] Nihal Jain edited comment on HBASE-20714 at 9/20/24 6:44 PM: - Yes. Had a look at the latest patch in HBASE-7129. It takes care of it to some extent, but not all. Recently HBASE-18671 added support for append and increment for REST. Either we can increase the scope of HBASE-7129 to take care of this or this JIRA can be used to document the above two ops. What do you say? was (Author: nihaljain.cs): Yes. Had a look at the latest patch in HBASE-7129. It takes care of it to some extent, but not all. Recently HBASE-18761 added support for append and increment for REST. Either we can increase the scope of HBASE-7129 to take care of this or this JIRA can be used to document the above two ops. What do you say? > Document REST curl commands for supported (but missing in docs) operations > -- > > Key: HBASE-20714 > URL: https://issues.apache.org/jira/browse/HBASE-20714 > Project: HBase > Issue Type: Sub-task > Components: REST >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > > While going through some recent REST JIRAs, I found REST supports row > operations like delete, append, increment etc. > I think we should document all such missing ops. > Others can give some input regarding other missing ops apart from the > above-mentioned ones. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28437) Region Server crash in our production environment.
[
https://issues.apache.org/jira/browse/HBASE-28437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883368#comment-17883368
]
Rushabh Shah commented on HBASE-28437:
--
This is fixed by HBASE-28850. Closing this jira as duplicate.
> Region Server crash in our production environment.
> --
>
> Key: HBASE-28437
> URL: https://issues.apache.org/jira/browse/HBASE-28437
> Project: HBase
> Issue Type: Bug
>Reporter: Rushabh Shah
>Priority: Major
>
> Recently we are seeing lot of RS crash in our production environment creating
> core dump file and hs_err_pid.log file.
> HBase: hbase-2.5
> Java: openjdk 1.8
> Copying contents from hs_err_pid.log below:
> {noformat}
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x7f9fb1415ba2, pid=50172, tid=0x7f92a97ec700
> #
> # JRE version: OpenJDK Runtime Environment (Zulu 8.76.0.18-SA-linux64)
> (8.0_402-b06) (build 1.8.0_402-b06)
> # Java VM: OpenJDK 64-Bit Server VM (25.402-b06 mixed mode linux-amd64 )
> # Problematic frame:
> # J 19801 C2
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyBufferToStream(Ljava/io/OutputStream;Ljava/nio/ByteBuffer;II)V
> (75 bytes) @ 0x7f9fb1415ba2 [0x7f9fb14159a0+0x202]
> #
> # Core dump written. Default location: /home/sfdc/core or core.50172
> #
> # If you would like to submit a bug report, please visit:
> # http://www.azul.com/support/
> #
> --- T H R E A D ---
> Current thread (0x7f9fa2d13000): JavaThread "RS-EventLoopGroup-1-92"
> daemon [_thread_in_Java, id=54547,
> stack(0x7f92a96ec000,0x7f92a97ed000)]
> siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr:
> 0x559869daf000
> Registers:
> RAX=0x7f9dbd8b6460, RBX=0x0008, RCX=0x0005c86b,
> RDX=0x7f9dbd8b6460
> RSP=0x7f92a97eaf20, RBP=0x0002, RSI=0x7f92d225e970,
> RDI=0x0069
> R8 =0x55986975f028, R9 =0x0064ffd8, R10=0x005f,
> R11=0x7f94a778b290
> R12=0x7f9e62855ae8, R13=0x, R14=0x7f9e5a14b1e0,
> R15=0x7f9fa2d13000
> RIP=0x7f9fb1415ba2, EFLAGS=0x00010216, CSGSFS=0x0033,
> ERR=0x0004
> TRAPNO=0x000e
> Top of Stack: (sp=0x7f92a97eaf20)
> 0x7f92a97eaf20: 00690064ff79 7f9dbd8b6460
> 0x7f92a97eaf30: 7f9dbd8b6460 00570003
> 0x7f92a97eaf40: 7f94a778b290 000400010004
> 0x7f92a97eaf50: 0004d090c130 7f9db550
> 0x7f92a97eaf60: 000800040001 7f92a97eaf90
> 0x7f92a97eaf70: 7f92d0908648 0001
> 0x7f92a97eaf80: 0001 005c
> 0x7f92a97eaf90: 7f94ee8078d0 0206
> 0x7f92a97eafa0: 7f9db5545a00 7f9fafb63670
> 0x7f92a97eafb0: 7f9e5a13ed70 00690001
> 0x7f92a97eafc0: 7f93ab8965b8 7f93b9959210
> 0x7f92a97eafd0: 7f9db5545a00 7f9fb04b3e30
> 0x7f92a97eafe0: 7f9e5a13ed70 7f930001
> 0x7f92a97eaff0: 7f93ab8965b8 7f93a8ae3920
> 0x7f92a97eb000: 7f93b9959210 7f94a778b290
> 0x7f92a97eb010: 7f9b60707c20 7f93a8938c28
> 0x7f92a97eb020: 7f94ee8078d0 7f9b60708608
> 0x7f92a97eb030: 7f9b60707bc0 7f9b60707c20
> 0x7f92a97eb040: 0069 7f93ab8965b8
> 0x7f92a97eb050: 7f94a778b290 7f94a778b290
> 0x7f92a97eb060: 0005c80d0005c80c a828a590
> 0x7f92a97eb070: 7f9e5a13ed70 0001270e
> 0x7f92a97eb080: 7f9db5545790 01440022
> 0x7f92a97eb090: 7f95ddc800c0 7f93ab89a6c8
> 0x7f92a97eb0a0: 7f93ae65c270 7f9fb24af990
> 0x7f92a97eb0b0: 7f93ae65c290 7f93ae65c270
> 0x7f92a97eb0c0: 7f9e5a13ed70 7f92ca328528
> 0x7f92a97eb0d0: 7f9e5a13ed98 7f9e5e1e88b0
> 0x7f92a97eb0e0: 7f92ca32d870 7f9e5a13ed98
> 0x7f92a97eb0f0: 7f9e5e1e88b0 7f93b9956288
> 0x7f92a97eb100: 7f9e5a13ed70 7f9fb23c3aac
> 0x7f92a97eb110: 7f9317c9c8d0 7f9b60708608
> Instructions: (pc=0x7f9fb1415ba2)
> 0x7f9fb1415b82: 44 3b d7 0f 8d 6d fe ff ff 4c 8b 40 10 45 8b ca
> 0x7f9fb1415b92: 44 03 0c 24 c4 c1 f9 7e c3 4d 8b 5b 18 4d 63 c9
> 0x7f9fb1415ba2: 47 0f be 04 08 4d 85 db 0f 84 49 03 00 00 4d 8b
> 0x7f9fb1415bb2: 4b 08 48 b9 10 1c be 10 93 7f 00 00 4c 3b c9 0f
> Register
[jira] [Resolved] (HBASE-28437) Region Server crash in our production environment.
[
https://issues.apache.org/jira/browse/HBASE-28437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rushabh Shah resolved HBASE-28437.
--
Resolution: Duplicate
> Region Server crash in our production environment.
> --
>
> Key: HBASE-28437
> URL: https://issues.apache.org/jira/browse/HBASE-28437
> Project: HBase
> Issue Type: Bug
>Reporter: Rushabh Shah
>Priority: Major
>
> Recently we are seeing lot of RS crash in our production environment creating
> core dump file and hs_err_pid.log file.
> HBase: hbase-2.5
> Java: openjdk 1.8
> Copying contents from hs_err_pid.log below:
> {noformat}
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x7f9fb1415ba2, pid=50172, tid=0x7f92a97ec700
> #
> # JRE version: OpenJDK Runtime Environment (Zulu 8.76.0.18-SA-linux64)
> (8.0_402-b06) (build 1.8.0_402-b06)
> # Java VM: OpenJDK 64-Bit Server VM (25.402-b06 mixed mode linux-amd64 )
> # Problematic frame:
> # J 19801 C2
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyBufferToStream(Ljava/io/OutputStream;Ljava/nio/ByteBuffer;II)V
> (75 bytes) @ 0x7f9fb1415ba2 [0x7f9fb14159a0+0x202]
> #
> # Core dump written. Default location: /home/sfdc/core or core.50172
> #
> # If you would like to submit a bug report, please visit:
> # http://www.azul.com/support/
> #
> --- T H R E A D ---
> Current thread (0x7f9fa2d13000): JavaThread "RS-EventLoopGroup-1-92"
> daemon [_thread_in_Java, id=54547,
> stack(0x7f92a96ec000,0x7f92a97ed000)]
> siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr:
> 0x559869daf000
> Registers:
> RAX=0x7f9dbd8b6460, RBX=0x0008, RCX=0x0005c86b,
> RDX=0x7f9dbd8b6460
> RSP=0x7f92a97eaf20, RBP=0x0002, RSI=0x7f92d225e970,
> RDI=0x0069
> R8 =0x55986975f028, R9 =0x0064ffd8, R10=0x005f,
> R11=0x7f94a778b290
> R12=0x7f9e62855ae8, R13=0x, R14=0x7f9e5a14b1e0,
> R15=0x7f9fa2d13000
> RIP=0x7f9fb1415ba2, EFLAGS=0x00010216, CSGSFS=0x0033,
> ERR=0x0004
> TRAPNO=0x000e
> Top of Stack: (sp=0x7f92a97eaf20)
> 0x7f92a97eaf20: 00690064ff79 7f9dbd8b6460
> 0x7f92a97eaf30: 7f9dbd8b6460 00570003
> 0x7f92a97eaf40: 7f94a778b290 000400010004
> 0x7f92a97eaf50: 0004d090c130 7f9db550
> 0x7f92a97eaf60: 000800040001 7f92a97eaf90
> 0x7f92a97eaf70: 7f92d0908648 0001
> 0x7f92a97eaf80: 0001 005c
> 0x7f92a97eaf90: 7f94ee8078d0 0206
> 0x7f92a97eafa0: 7f9db5545a00 7f9fafb63670
> 0x7f92a97eafb0: 7f9e5a13ed70 00690001
> 0x7f92a97eafc0: 7f93ab8965b8 7f93b9959210
> 0x7f92a97eafd0: 7f9db5545a00 7f9fb04b3e30
> 0x7f92a97eafe0: 7f9e5a13ed70 7f930001
> 0x7f92a97eaff0: 7f93ab8965b8 7f93a8ae3920
> 0x7f92a97eb000: 7f93b9959210 7f94a778b290
> 0x7f92a97eb010: 7f9b60707c20 7f93a8938c28
> 0x7f92a97eb020: 7f94ee8078d0 7f9b60708608
> 0x7f92a97eb030: 7f9b60707bc0 7f9b60707c20
> 0x7f92a97eb040: 0069 7f93ab8965b8
> 0x7f92a97eb050: 7f94a778b290 7f94a778b290
> 0x7f92a97eb060: 0005c80d0005c80c a828a590
> 0x7f92a97eb070: 7f9e5a13ed70 0001270e
> 0x7f92a97eb080: 7f9db5545790 01440022
> 0x7f92a97eb090: 7f95ddc800c0 7f93ab89a6c8
> 0x7f92a97eb0a0: 7f93ae65c270 7f9fb24af990
> 0x7f92a97eb0b0: 7f93ae65c290 7f93ae65c270
> 0x7f92a97eb0c0: 7f9e5a13ed70 7f92ca328528
> 0x7f92a97eb0d0: 7f9e5a13ed98 7f9e5e1e88b0
> 0x7f92a97eb0e0: 7f92ca32d870 7f9e5a13ed98
> 0x7f92a97eb0f0: 7f9e5e1e88b0 7f93b9956288
> 0x7f92a97eb100: 7f9e5a13ed70 7f9fb23c3aac
> 0x7f92a97eb110: 7f9317c9c8d0 7f9b60708608
> Instructions: (pc=0x7f9fb1415ba2)
> 0x7f9fb1415b82: 44 3b d7 0f 8d 6d fe ff ff 4c 8b 40 10 45 8b ca
> 0x7f9fb1415b92: 44 03 0c 24 c4 c1 f9 7e c3 4d 8b 5b 18 4d 63 c9
> 0x7f9fb1415ba2: 47 0f be 04 08 4d 85 db 0f 84 49 03 00 00 4d 8b
> 0x7f9fb1415bb2: 4b 08 48 b9 10 1c be 10 93 7f 00 00 4c 3b c9 0f
> Register to memory mapping:
> RAX=0x7f9dbd8b6460 is an oop
> java.nio.DirectByteBuffer
> -
[jira] [Commented] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883338#comment-17883338 ] Andrew Kyle Purtell commented on HBASE-28864: - Thanks for looking. Let me try to find and account for a local difference. > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28868) Add missing permission check for updateRSGroupConfig in branch-2
[
https://issues.apache.org/jira/browse/HBASE-28868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883319#comment-17883319
]
Nihal Jain edited comment on HBASE-28868 at 9/20/24 4:13 PM:
-
Even though the check is missing, the method invocation will still fail for a
non privileged user as we do a ADMIN check in
{{{}rpcPreCheck("execMasterService"){}}}, so technically this is not a security
issue. See
[https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java#L905]
was (Author: nihaljain.cs):
Even though the check is missing, the method invocation will still fail as we
do a ADMIN check in {{rpcPreCheck("execMasterService")}}, so technically this
is not a security issue. See
[https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java#L905]
> Add missing permission check for updateRSGroupConfig in branch-2
>
>
> Key: HBASE-28868
> URL: https://issues.apache.org/jira/browse/HBASE-28868
> Project: HBase
> Issue Type: Task
> Components: rsgroup
>Affects Versions: 2.7.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Labels: pull-request-available
>
> Found this during HBASE-28867, we do not have security check for
> updateRSGroupConfig in branch-2. See
> [https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L450]
> Same check exists in master
> [https://github.com/apache/hbase/blob/52082bc5b80a60406bfaaa630ed5cb23027436c1/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L2279]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28868) Add missing permission check for updateRSGroupConfig in branch-2
[ https://issues.apache.org/jira/browse/HBASE-28868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-28868: --- Status: Patch Available (was: Open) > Add missing permission check for updateRSGroupConfig in branch-2 > > > Key: HBASE-28868 > URL: https://issues.apache.org/jira/browse/HBASE-28868 > Project: HBase > Issue Type: Task > Components: rsgroup >Affects Versions: 2.7.0 >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > Labels: pull-request-available > > Found this during HBASE-28867, we do not have security check for > updateRSGroupConfig in branch-2. See > [https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L450] > Same check exists in master > [https://github.com/apache/hbase/blob/52082bc5b80a60406bfaaa630ed5cb23027436c1/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L2279] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28868) Add missing permission check for updateRSGroupConfig in branch-2
[ https://issues.apache.org/jira/browse/HBASE-28868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HBASE-28868: --- Labels: pull-request-available (was: ) > Add missing permission check for updateRSGroupConfig in branch-2 > > > Key: HBASE-28868 > URL: https://issues.apache.org/jira/browse/HBASE-28868 > Project: HBase > Issue Type: Task > Components: rsgroup >Affects Versions: 2.7.0 >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > Labels: pull-request-available > > Found this during HBASE-28867, we do not have security check for > updateRSGroupConfig in branch-2. See > [https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L450] > Same check exists in master > [https://github.com/apache/hbase/blob/52082bc5b80a60406bfaaa630ed5cb23027436c1/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L2279] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28868) Permission check is missing for updateRSGroupConfig in branch-2
[ https://issues.apache.org/jira/browse/HBASE-28868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-28868: --- Component/s: rsgroup > Permission check is missing for updateRSGroupConfig in branch-2 > --- > > Key: HBASE-28868 > URL: https://issues.apache.org/jira/browse/HBASE-28868 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.7.0 >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > > Found this during HBASE-28867, we do not have security check for > updateRSGroupConfig in branch-2. See > [https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L450] > Same check exists in master > [https://github.com/apache/hbase/blob/52082bc5b80a60406bfaaa630ed5cb23027436c1/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L2279] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28868) Permission check is missing for updateRSGroupConfig in branch-2
[
https://issues.apache.org/jira/browse/HBASE-28868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883319#comment-17883319
]
Nihal Jain edited comment on HBASE-28868 at 9/20/24 4:09 PM:
-
Even though the check is missing, the method invocation will still fail as we
do a ADMIN check in {{rpcPreCheck("execMasterService")}}, so technically this
is not a security issue. See
[https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java#L905]
was (Author: nihaljain.cs):
Even though the check is missing, the method invocation will still fail as we
do a ADMIN check in preRpcCheck, so technically this is not a security issue.
See
https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java#L905
> Permission check is missing for updateRSGroupConfig in branch-2
> ---
>
> Key: HBASE-28868
> URL: https://issues.apache.org/jira/browse/HBASE-28868
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
>
> Found this during HBASE-28867, we do not have security check for
> updateRSGroupConfig in branch-2. See
> [https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L450]
> Same check exists in master
> [https://github.com/apache/hbase/blob/52082bc5b80a60406bfaaa630ed5cb23027436c1/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L2279]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28868) Permission check is missing for updateRSGroupConfig in branch-2
[ https://issues.apache.org/jira/browse/HBASE-28868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-28868: --- Issue Type: Task (was: Bug) > Permission check is missing for updateRSGroupConfig in branch-2 > --- > > Key: HBASE-28868 > URL: https://issues.apache.org/jira/browse/HBASE-28868 > Project: HBase > Issue Type: Task > Components: rsgroup >Affects Versions: 2.7.0 >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > > Found this during HBASE-28867, we do not have security check for > updateRSGroupConfig in branch-2. See > [https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L450] > Same check exists in master > [https://github.com/apache/hbase/blob/52082bc5b80a60406bfaaa630ed5cb23027436c1/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L2279] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28868) Add missing permission check for updateRSGroupConfig in branch-2
[ https://issues.apache.org/jira/browse/HBASE-28868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-28868: --- Summary: Add missing permission check for updateRSGroupConfig in branch-2 (was: Permission check is missing for updateRSGroupConfig in branch-2) > Add missing permission check for updateRSGroupConfig in branch-2 > > > Key: HBASE-28868 > URL: https://issues.apache.org/jira/browse/HBASE-28868 > Project: HBase > Issue Type: Task > Components: rsgroup >Affects Versions: 2.7.0 >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > > Found this during HBASE-28867, we do not have security check for > updateRSGroupConfig in branch-2. See > [https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L450] > Same check exists in master > [https://github.com/apache/hbase/blob/52082bc5b80a60406bfaaa630ed5cb23027436c1/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L2279] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28868) Permission check is missing for updateRSGroupConfig in branch-2
[ https://issues.apache.org/jira/browse/HBASE-28868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883319#comment-17883319 ] Nihal Jain commented on HBASE-28868: Even though the check is missing, the method invocation will still fail as we do a ADMIN check in preRpcCheck, so technically this is not a security issue. See https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java#L905 > Permission check is missing for updateRSGroupConfig in branch-2 > --- > > Key: HBASE-28868 > URL: https://issues.apache.org/jira/browse/HBASE-28868 > Project: HBase > Issue Type: Bug >Affects Versions: 2.7.0 >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > > Found this during HBASE-28867, we do not have security check for > updateRSGroupConfig in branch-2. See > [https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L450] > Same check exists in master > [https://github.com/apache/hbase/blob/52082bc5b80a60406bfaaa630ed5cb23027436c1/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L2279] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28868) Permission check is missing for updateRSGroupConfig in branch-2
[ https://issues.apache.org/jira/browse/HBASE-28868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-28868: --- Priority: Minor (was: Major) > Permission check is missing for updateRSGroupConfig in branch-2 > --- > > Key: HBASE-28868 > URL: https://issues.apache.org/jira/browse/HBASE-28868 > Project: HBase > Issue Type: Bug >Affects Versions: 2.7.0 >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > > Found this during HBASE-28867, we do not have security check for > updateRSGroupConfig in branch-2. See > [https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L450] > Same check exists in master > [https://github.com/apache/hbase/blob/52082bc5b80a60406bfaaa630ed5cb23027436c1/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L2279] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28731) Remove the IA.Private annotation on WALEdit's add methods as they have already been used by CP users
[ https://issues.apache.org/jira/browse/HBASE-28731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28731. --- Fix Version/s: 2.7.0 3.0.0-beta-2 2.6.1 2.5.11 Hadoop Flags: Reviewed Resolution: Fixed > Remove the IA.Private annotation on WALEdit's add methods as they have > already been used by CP users > > > Key: HBASE-28731 > URL: https://issues.apache.org/jira/browse/HBASE-28731 > Project: HBase > Issue Type: Task > Components: Coprocessors, wal >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.11 > > > Per the discussion thread here > https://lists.apache.org/thread/b7zfyqmxo9lrt2rpo0lc0m6vsomn217w -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28848) Audit Jira vs. git commit history
[
https://issues.apache.org/jira/browse/HBASE-28848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883309#comment-17883309
]
Duo Zhang commented on HBASE-28848:
---
I've done the first round of aligning.
HBASE-24791 is in changelog but not git commit, it is because we set the wrong
fix version, as there is a backport issue HBASE-28586 for backporting it to
2.6.x, we should remove the fix versions from HBASE-24791.
HBASE-28364 is in changelog but it has a uncommon commit message style so my
script failed to extract the issue number
{noformat}
commit 8852e3a8354b4bc4d479a4904a32924d9bca1754
Author: Nikita Pande <37657012+nikita...@users.noreply.github.com>
Date: Wed Jul 10 14:11:17 2024 +0530
[HBASE-28364] Warn: Cache key had block type null, but was found in L1
cache (#6068)
{noformat}
HBASE-27118 and HBASE-28731 are all committed to branch-2.6 but we forgot to
resolve the issues. I've already resolved them and set fix versions.
So now the jira and git commit are aligned. Will do another round of aligning
after we commit all the necessary issues.
Thanks.
> Audit Jira vs. git commit history
> -
>
> Key: HBASE-28848
> URL: https://issues.apache.org/jira/browse/HBASE-28848
> Project: HBase
> Issue Type: Sub-task
> Components: community
>Reporter: Nick Dimiduk
>Assignee: Duo Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27118) Add security headers to Thrift/HTTP server
[ https://issues.apache.org/jira/browse/HBASE-27118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-27118: -- Fix Version/s: 2.7.0 3.0.0-beta-2 2.6.1 2.5.11 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > Add security headers to Thrift/HTTP server > --- > > Key: HBASE-27118 > URL: https://issues.apache.org/jira/browse/HBASE-27118 > Project: HBase > Issue Type: Improvement > Components: Thrift >Affects Versions: 2.0.6, 2.1.10, 2.2.7, 3.0.0-alpha-2 >Reporter: Andor Molnar >Assignee: Andor Molnar >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.11 > > > In order to further improve HBase exposed service for vulnerability scanners, > would be nice to implement additional security headers for the Thrift server > when HTTP or HTTPS transport is enabled. > Similarly to REST and Web UIs, related tickets are attached. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-24791) Improve HFileOutputFormat2 to avoid always call getTableRelativePath method
[ https://issues.apache.org/jira/browse/HBASE-24791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-24791: -- Fix Version/s: (was: 2.4.18) (was: 2.7.0) (was: 2.6.1) (was: 2.5.9) > Improve HFileOutputFormat2 to avoid always call getTableRelativePath method > --- > > Key: HBASE-24791 > URL: https://issues.apache.org/jira/browse/HBASE-24791 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Affects Versions: 3.0.0-alpha-1 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Critical > Labels: HFileOutputFormat, bulkload, pull-request-available > Fix For: 3.0.0-alpha-1 > > > Bulkload use HFileOutputFormat2 to write HFile > In the HFileOutputFormat2.RecordWriter > in the write method always called the getTableRelativePath method each time > This is unnecessary -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28848) Audit Jira vs. git commit history
[ https://issues.apache.org/jira/browse/HBASE-28848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28848: - Assignee: Duo Zhang > Audit Jira vs. git commit history > - > > Key: HBASE-28848 > URL: https://issues.apache.org/jira/browse/HBASE-28848 > Project: HBase > Issue Type: Sub-task >Reporter: Nick Dimiduk >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-28848) Audit Jira vs. git commit history
[ https://issues.apache.org/jira/browse/HBASE-28848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-28848 started by Duo Zhang. - > Audit Jira vs. git commit history > - > > Key: HBASE-28848 > URL: https://issues.apache.org/jira/browse/HBASE-28848 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Nick Dimiduk >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28848) Audit Jira vs. git commit history
[ https://issues.apache.org/jira/browse/HBASE-28848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28848: -- Component/s: community > Audit Jira vs. git commit history > - > > Key: HBASE-28848 > URL: https://issues.apache.org/jira/browse/HBASE-28848 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Nick Dimiduk >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28874) [hbase-thirdparty] Finish hbase-thirdparty 4.1.9 release
Duo Zhang created HBASE-28874: - Summary: [hbase-thirdparty] Finish hbase-thirdparty 4.1.9 release Key: HBASE-28874 URL: https://issues.apache.org/jira/browse/HBASE-28874 Project: HBase Issue Type: Sub-task Components: community, thirdparty Reporter: Duo Zhang Release the artifacts on repository.apache.org Move the binaries from dist-dev to dist-release Push tag 4.1.9RC0 as tag rel/4.1.9 Release thirdparty-4.1.9 on JIRA https://issues.apache.org/jira/projects/HBASE/versions/12355077 Add release data on https://reporter.apache.org/addrelease.html?hbase Send announcement email -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28873) [hbase-thirdparty] Put up hbase-thirdparty-4.1.9RC0
[ https://issues.apache.org/jira/browse/HBASE-28873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28873: -- Summary: [hbase-thirdparty] Put up hbase-thirdparty-4.1.9RC0 (was: [hbase-thirdparty] Put up hbase-thirdparty-4.1.7RC0) > [hbase-thirdparty] Put up hbase-thirdparty-4.1.9RC0 > --- > > Key: HBASE-28873 > URL: https://issues.apache.org/jira/browse/HBASE-28873 > Project: HBase > Issue Type: Sub-task > Components: community >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28873) [hbase-thirdparty] Put up hbase-thirdparty-4.1.9RC0
[ https://issues.apache.org/jira/browse/HBASE-28873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28873: -- Component/s: thirdparty > [hbase-thirdparty] Put up hbase-thirdparty-4.1.9RC0 > --- > > Key: HBASE-28873 > URL: https://issues.apache.org/jira/browse/HBASE-28873 > Project: HBase > Issue Type: Sub-task > Components: community, thirdparty >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28873) [hbase-thirdparty] Put up hbase-thirdparty-4.1.7RC0
Duo Zhang created HBASE-28873: - Summary: [hbase-thirdparty] Put up hbase-thirdparty-4.1.7RC0 Key: HBASE-28873 URL: https://issues.apache.org/jira/browse/HBASE-28873 Project: HBase Issue Type: Sub-task Components: community Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28871) [hbase-thirdparty] Bump dependency versions before releasing
[ https://issues.apache.org/jira/browse/HBASE-28871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HBASE-28871: --- Labels: pull-request-available (was: ) > [hbase-thirdparty] Bump dependency versions before releasing > > > Key: HBASE-28871 > URL: https://issues.apache.org/jira/browse/HBASE-28871 > Project: HBase > Issue Type: Sub-task > Components: dependencies, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HBASE-28871) [hbase-thirdparty] Bump dependency versions before releasing
[ https://issues.apache.org/jira/browse/HBASE-28871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-28871 started by Duo Zhang. - > [hbase-thirdparty] Bump dependency versions before releasing > > > Key: HBASE-28871 > URL: https://issues.apache.org/jira/browse/HBASE-28871 > Project: HBase > Issue Type: Sub-task > Components: dependencies, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28871) [hbase-thirdparty] Bump dependency versions before releasing
[ https://issues.apache.org/jira/browse/HBASE-28871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28871: - Assignee: Duo Zhang > [hbase-thirdparty] Bump dependency versions before releasing > > > Key: HBASE-28871 > URL: https://issues.apache.org/jira/browse/HBASE-28871 > Project: HBase > Issue Type: Sub-task > Components: dependencies, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28872) HFile.Reader#readAtOffset is not thread safe when not using preads
Mathias Herberts created HBASE-28872: Summary: HFile.Reader#readAtOffset is not thread safe when not using preads Key: HBASE-28872 URL: https://issues.apache.org/jira/browse/HBASE-28872 Project: HBase Issue Type: Bug Affects Versions: 2.5.5 Reporter: Mathias Herberts When not using preads, multiple threads may call readAtOffset from multiple scanners in a concurrent way which may lead to another seek being called before the actual read. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28871) [hbase-thirdparty] Bump dependency versions before releasing
[ https://issues.apache.org/jira/browse/HBASE-28871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28871: -- Release Note: Bump versions for these dependencies netty 4.1.112.Final -> 4.1.113.Final netty-tcnative 2.0.61.Final -> 2.0.66.Final guava 33.2.1-jre -> 33.3.0-jre commons-cli 1.8.0 -> 1.9.0 error_prone_annotations 2.28.0 -> 2.32.0 jetty 9.4.55.v20240627 -> 9.4.56.v20240826 jersey.version 2.43 -> 2.45 > [hbase-thirdparty] Bump dependency versions before releasing > > > Key: HBASE-28871 > URL: https://issues.apache.org/jira/browse/HBASE-28871 > Project: HBase > Issue Type: Sub-task > Components: dependencies, thirdparty >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28869) [hbase-thirdparty] Bump protobuf java to 4.27.5+
[ https://issues.apache.org/jira/browse/HBASE-28869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28869: -- Status: Patch Available (was: Open) > [hbase-thirdparty] Bump protobuf java to 4.27.5+ > > > Key: HBASE-28869 > URL: https://issues.apache.org/jira/browse/HBASE-28869 > Project: HBase > Issue Type: Sub-task > Components: Protobufs, security, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: thirdparty-4.1.9 > > > For addressing CVE-2024-7254 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28869) [hbase-thirdparty] Bump protobuf java to 4.27.5+
[ https://issues.apache.org/jira/browse/HBASE-28869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28869: - Assignee: Duo Zhang > [hbase-thirdparty] Bump protobuf java to 4.27.5+ > > > Key: HBASE-28869 > URL: https://issues.apache.org/jira/browse/HBASE-28869 > Project: HBase > Issue Type: Sub-task > Components: Protobufs, security, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: thirdparty-4.1.9 > > > For addressing CVE-2024-7254 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28871) [hbase-thirdparty] Bump dependency versions before releasing
[ https://issues.apache.org/jira/browse/HBASE-28871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28871: -- Summary: [hbase-thirdparty] Bump dependency versions before releasing (was: Bump dependency versions before releasing) > [hbase-thirdparty] Bump dependency versions before releasing > > > Key: HBASE-28871 > URL: https://issues.apache.org/jira/browse/HBASE-28871 > Project: HBase > Issue Type: Sub-task > Components: dependencies, thirdparty >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28869) [hbase-thirdparty] Bump protobuf java to 4.27.5+
[ https://issues.apache.org/jira/browse/HBASE-28869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28869: -- Release Note: Bump protobuf version from 4.27.3 to 4.28.2. > [hbase-thirdparty] Bump protobuf java to 4.27.5+ > > > Key: HBASE-28869 > URL: https://issues.apache.org/jira/browse/HBASE-28869 > Project: HBase > Issue Type: Sub-task > Components: Protobufs, security, thirdparty >Reporter: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: thirdparty-4.1.9 > > > For addressing CVE-2024-7254 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28871) Bump dependency versions before releasing
Duo Zhang created HBASE-28871: - Summary: Bump dependency versions before releasing Key: HBASE-28871 URL: https://issues.apache.org/jira/browse/HBASE-28871 Project: HBase Issue Type: Sub-task Components: dependencies, thirdparty Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28869) [hbase-thirdparty] Bump protobuf java to 4.27.5+
[ https://issues.apache.org/jira/browse/HBASE-28869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28869: -- Parent: HBASE-28870 Issue Type: Sub-task (was: Task) > [hbase-thirdparty] Bump protobuf java to 4.27.5+ > > > Key: HBASE-28869 > URL: https://issues.apache.org/jira/browse/HBASE-28869 > Project: HBase > Issue Type: Sub-task > Components: Protobufs, security, thirdparty >Reporter: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: thirdparty-4.1.9 > > > For addressing CVE-2024-7254 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28870) [hbase-thirdparty] Release hbase-thirdparty 4.1.9
Duo Zhang created HBASE-28870: - Summary: [hbase-thirdparty] Release hbase-thirdparty 4.1.9 Key: HBASE-28870 URL: https://issues.apache.org/jira/browse/HBASE-28870 Project: HBase Issue Type: Umbrella Components: community, thirdparty Reporter: Duo Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28869) [hbase-thirdparty] Bump protobuf java to 4.27.5+
[ https://issues.apache.org/jira/browse/HBASE-28869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HBASE-28869: --- Labels: pull-request-available (was: ) > [hbase-thirdparty] Bump protobuf java to 4.27.5+ > > > Key: HBASE-28869 > URL: https://issues.apache.org/jira/browse/HBASE-28869 > Project: HBase > Issue Type: Task > Components: Protobufs, security, thirdparty >Reporter: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: thirdparty-4.1.9 > > > For addressing CVE-2024-7254 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28869) [hbase-thirdparty] Bump protobuf java to 4.27.5+
[ https://issues.apache.org/jira/browse/HBASE-28869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28869: -- Fix Version/s: thirdparty-4.1.9 > [hbase-thirdparty] Bump protobuf java to 4.27.5+ > > > Key: HBASE-28869 > URL: https://issues.apache.org/jira/browse/HBASE-28869 > Project: HBase > Issue Type: Task > Components: Protobufs, security, thirdparty >Reporter: Duo Zhang >Priority: Major > Fix For: thirdparty-4.1.9 > > > For addressing CVE-2024-7254 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28869) [hbase-thirdparty] Bump protobuf java to 4.27.5+
Duo Zhang created HBASE-28869: - Summary: [hbase-thirdparty] Bump protobuf java to 4.27.5+ Key: HBASE-28869 URL: https://issues.apache.org/jira/browse/HBASE-28869 Project: HBase Issue Type: Task Components: Protobufs, security, thirdparty Reporter: Duo Zhang For addressing CVE-2024-7254 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28866) Setting `hbase.oldwals.cleaner.thread.size` to negative value will break HMaster and produce hard-to-diagnose logs
[
https://issues.apache.org/jira/browse/HBASE-28866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariadne updated HBASE-28866:
Description:
Problem
-
HBase Master cannot be initialized with the following setting:
hbase.oldwals.cleaner.thread.size
-1
Default is 2
After running the start-hbase.sh, the Master node could not be started due to
an exception:
{code:java}
ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster: Failed to
become active master
java.lang.IllegalArgumentException: Illegal Capacity: -1
at java.util.ArrayList.(ArrayList.java:157)
at
org.apache.hadoop.hbase.master.cleaner.LogCleaner.createOldWalsCleaner(LogCleaner.java:149)
at
org.apache.hadoop.hbase.master.cleaner.LogCleaner.(LogCleaner.java:80)
at
org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1329)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:917)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2081)
at org.apache.hadoop.hbase.master.HMaster.lambda$0(HMaster.java:505)
at java.lang.Thread.run(Thread.java:750){code}
We were really confused and misled by the error log as the 'Illegal Capacity'
of ArrayList seems like an internal code issue.
After we read the source code, we found that
"hbase.oldwals.cleaner.thread.size" is parsed and used in
createOldWalsCleaner() function without checking:
{code:java}
int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE); this.oldWALsCleaner =
createOldWalsCleaner(size); {code}
The value of "hbase.oldwals.cleaner.thread.size" will be served as the
initialCapacity of ArrayList. If the configuration value is negative, an
IllegalArgumentException will be thrown.:
{code:java}
private List createOldWalsCleaner(int size) {
...
List oldWALsCleaner = new ArrayList<>(size);
...
} {code}
Solution (the attached patch)
-
The basic idea of the attached patch is to add a check and relevant logging for
this value during the initialization of the {{LogCleaner}} in the constructor.
This will help users better diagnose the issue. The detailed patch is shown
below.
{code:java}
@@ -78,6 +78,11 @@
public class LogCleaner extends CleanerChore
pool, params, null);
this.pendingDelete = new LinkedBlockingQueue<>();
int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE);
+ if (size <= 0) {
+ LOG.warn("The size of old WALs cleaner thread is {}, which is invalid, "
+ + "the default value will be used.", size);
+ size = DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE;
+ }
this.oldWALsCleaner = createOldWalsCleaner(size);
this.cleanerThreadTimeoutMsec =
conf.getLong(OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC,
DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC);{code}
Thanks!
was:
Problem
-
HBase Master cannot be initialized with the following setting:
hbase.oldwals.cleaner.thread.size
-1
Default is 2
After running the start-hbase.sh, the Master node could not be started due to
an exception:
{code:java}
ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster: Failed to
become active master
java.lang.IllegalArgumentException: Illegal Capacity: -1
at java.util.ArrayList.(ArrayList.java:157)
at
org.apache.hadoop.hbase.master.cleaner.LogCleaner.createOldWalsCleaner(LogCleaner.java:149)
at
org.apache.hadoop.hbase.master.cleaner.LogCleaner.(LogCleaner.java:80)
at
org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1329)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:917)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2081)
at org.apache.hadoop.hbase.master.HMaster.lambda$0(HMaster.java:505)
at java.lang.Thread.run(Thread.java:750){code}
We were really confused and misled by the error log as the 'Illegal Capacity'
of ArrayList seems like an internal code issue.
After we read the source code, we found that
"hbase.oldwals.cleaner.thread.size" is parsed and used in
createOldWalsCleaner() function without checking:
{code:java}
int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE); this.oldWALsCleaner =
createOldWalsCleaner(size); {code}
The value of "hbase.oldwals.cleaner.thread.size" will be served as the
initialCapacity of ArrayList. If the configuration value is negative, an
IllegalArgumentException will be thrown.:
{code:java}
priva
[jira] [Updated] (HBASE-28866) Setting `hbase.oldwals.cleaner.thread.size` to negative value will break HMaster and produce hard-to-diagnose logs
[
https://issues.apache.org/jira/browse/HBASE-28866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariadne updated HBASE-28866:
Attachment: (was: LogCleaner.patch)
> Setting `hbase.oldwals.cleaner.thread.size` to negative value will break
> HMaster and produce hard-to-diagnose logs
> --
>
> Key: HBASE-28866
> URL: https://issues.apache.org/jira/browse/HBASE-28866
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.4.2, 3.0.0-beta-1
>Reporter: Ariadne
>Priority: Critical
> Fix For: 3.0.0-beta-1
>
> Attachments: HBASE-28866-000.patch
>
>
>
> Problem
> -
> HBase Master cannot be initialized with the following setting:
>
> hbase.oldwals.cleaner.thread.size
> -1
> Default is 2
>
>
> After running the start-hbase.sh, the Master node could not be started due to
> an exception:
> {code:java}
> ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster: Failed to
> become active master
> java.lang.IllegalArgumentException: Illegal Capacity: -1
> at java.util.ArrayList.(ArrayList.java:157)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.createOldWalsCleaner(LogCleaner.java:149)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.(LogCleaner.java:80)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1329)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:917)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2081)
> at org.apache.hadoop.hbase.master.HMaster.lambda$0(HMaster.java:505)
> at java.lang.Thread.run(Thread.java:750){code}
> We were really confused and misled by the error log as the 'Illegal Capacity'
> of ArrayList seems like an internal code issue.
>
> After we read the source code, we found that
> "hbase.oldwals.cleaner.thread.size" is parsed and used in
> createOldWalsCleaner() function without checking:
> {code:java}
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE); this.oldWALsCleaner =
> createOldWalsCleaner(size); {code}
> The value of "hbase.oldwals.cleaner.thread.size" will be served as the
> initialCapacity of ArrayList. If the configuration value is negative, an
> IllegalArgumentException will be thrown.:
> {code:java}
> private List createOldWalsCleaner(int size) {
> ...
> List oldWALsCleaner = new ArrayList<>(size);
> ...
> } {code}
>
> Solution (the attached patch)
> -
> The basic idea of the attached patch is to add a check and relevant logging
> for this value during the initialization of the {{LogCleaner}} in the
> constructor. This will help users better diagnose the issue. The detailed
> patch is shown below.
> {code:java}
> @@ -78,6 +78,11 @@
> public class LogCleaner extends CleanerChore
> pool, params, null);
> this.pendingDelete = new LinkedBlockingQueue<>();
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE);
> + if (size <= 0) {
> + LOG.warn("The size of old WALs cleaner thread is {}, which is invalid,
> "
> + + "the default value will be used.", size);
> + size = DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE;
> + }
> this.oldWALsCleaner = createOldWalsCleaner(size);
> this.cleanerThreadTimeoutMsec =
> conf.getLong(OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC,
> DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC);{code}
> Thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28866) Setting `hbase.oldwals.cleaner.thread.size` to negative value will break HMaster and produce hard-to-diagnose logs
[
https://issues.apache.org/jira/browse/HBASE-28866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariadne updated HBASE-28866:
Attachment: HBASE-28866-000.patch
> Setting `hbase.oldwals.cleaner.thread.size` to negative value will break
> HMaster and produce hard-to-diagnose logs
> --
>
> Key: HBASE-28866
> URL: https://issues.apache.org/jira/browse/HBASE-28866
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.4.2, 3.0.0-beta-1
>Reporter: Ariadne
>Priority: Critical
> Fix For: 3.0.0-beta-1
>
> Attachments: HBASE-28866-000.patch
>
>
>
> Problem
> -
> HBase Master cannot be initialized with the following setting:
>
> hbase.oldwals.cleaner.thread.size
> -1
> Default is 2
>
>
> After running the start-hbase.sh, the Master node could not be started due to
> an exception:
> {code:java}
> ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster: Failed to
> become active master
> java.lang.IllegalArgumentException: Illegal Capacity: -1
> at java.util.ArrayList.(ArrayList.java:157)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.createOldWalsCleaner(LogCleaner.java:149)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.(LogCleaner.java:80)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1329)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:917)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2081)
> at org.apache.hadoop.hbase.master.HMaster.lambda$0(HMaster.java:505)
> at java.lang.Thread.run(Thread.java:750){code}
> We were really confused and misled by the error log as the 'Illegal Capacity'
> of ArrayList seems like an internal code issue.
>
> After we read the source code, we found that
> "hbase.oldwals.cleaner.thread.size" is parsed and used in
> createOldWalsCleaner() function without checking:
> {code:java}
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE); this.oldWALsCleaner =
> createOldWalsCleaner(size); {code}
> The value of "hbase.oldwals.cleaner.thread.size" will be served as the
> initialCapacity of ArrayList. If the configuration value is negative, an
> IllegalArgumentException will be thrown.:
> {code:java}
> private List createOldWalsCleaner(int size) {
> ...
> List oldWALsCleaner = new ArrayList<>(size);
> ...
> } {code}
>
> Solution (the attached patch)
> -
> The basic idea of the attached patch is to add a check and relevant logging
> for this value during the initialization of the {{LogCleaner}} in the
> constructor. This will help users better diagnose the issue. The detailed
> patch is shown below.
> {code:java}
> @@ -78,6 +78,11 @@
> public class LogCleaner extends CleanerChore
> pool, params, null);
> this.pendingDelete = new LinkedBlockingQueue<>();
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE);
> + if (size <= 0) {
> + LOG.warn("The size of old WALs cleaner thread is {}, which is invalid,
> "
> + + "the default value will be used.", size);
> + size = DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE;
> + }
> this.oldWALsCleaner = createOldWalsCleaner(size);
> this.cleanerThreadTimeoutMsec =
> conf.getLong(OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC,
> DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC);{code}
> Thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28866) Setting `hbase.oldwals.cleaner.thread.size` to negative value will break HMaster and produce hard-to-diagnose logs
[
https://issues.apache.org/jira/browse/HBASE-28866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariadne updated HBASE-28866:
Summary: Setting `hbase.oldwals.cleaner.thread.size` to negative value will
break HMaster and produce hard-to-diagnose logs (was:
`hbase.hstore.flusher.count` setting to 0 breaks HMaster)
> Setting `hbase.oldwals.cleaner.thread.size` to negative value will break
> HMaster and produce hard-to-diagnose logs
> --
>
> Key: HBASE-28866
> URL: https://issues.apache.org/jira/browse/HBASE-28866
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.4.2, 3.0.0-beta-1
>Reporter: Ariadne
>Priority: Critical
> Fix For: 3.0.0-beta-1
>
> Attachments: LogCleaner.patch
>
>
>
> Problem
> -
> HBase Master cannot be initialized with the following setting:
>
> hbase.oldwals.cleaner.thread.size
> -1
> Default is 2
>
>
> After running the start-hbase.sh, the Master node could not be started due to
> an exception:
> {code:java}
> ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster: Failed to
> become active master
> java.lang.IllegalArgumentException: Illegal Capacity: -1
> at java.util.ArrayList.(ArrayList.java:157)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.createOldWalsCleaner(LogCleaner.java:149)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.(LogCleaner.java:80)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1329)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:917)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2081)
> at org.apache.hadoop.hbase.master.HMaster.lambda$0(HMaster.java:505)
> at java.lang.Thread.run(Thread.java:750){code}
> We were really confused and misled by the error log as the 'Illegal Capacity'
> of ArrayList seems like an internal code issue.
>
> After we read the source code, we found that
> "hbase.oldwals.cleaner.thread.size" is parsed and used in
> createOldWalsCleaner() function without checking:
> {code:java}
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE); this.oldWALsCleaner =
> createOldWalsCleaner(size); {code}
> The value of "hbase.oldwals.cleaner.thread.size" will be served as the
> initialCapacity of ArrayList. If the configuration value is negative, an
> IllegalArgumentException will be thrown.:
> {code:java}
> private List createOldWalsCleaner(int size) {
> ...
> List oldWALsCleaner = new ArrayList<>(size);
> ...
> } {code}
>
> Solution (the attached patch)
> -
> The basic idea of the attached patch is to add a check and relevant logging
> for this value during the initialization of the {{LogCleaner}} in the
> constructor. This will help users better diagnose the issue. The detailed
> patch is shown below.
> {code:java}
> @@ -78,6 +78,11 @@
> public class LogCleaner extends CleanerChore
> pool, params, null);
> this.pendingDelete = new LinkedBlockingQueue<>();
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE);
> + if (size <= 0) {
> + LOG.warn("The size of old WALs cleaner thread is {}, which is invalid,
> "
> + + "the default value will be used.", size);
> + size = DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE;
> + }
> this.oldWALsCleaner = createOldWalsCleaner(size);
> this.cleanerThreadTimeoutMsec =
> conf.getLong(OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC,
> DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC);{code}
> Thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28866) `hbase.hstore.flusher.count` setting to 0 breaks HMaster
[
https://issues.apache.org/jira/browse/HBASE-28866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariadne updated HBASE-28866:
Summary: `hbase.hstore.flusher.count` setting to 0 breaks HMaster (was:
Master will fail to start due to a hard-to-diagnose misconfiguration.)
> `hbase.hstore.flusher.count` setting to 0 breaks HMaster
>
>
> Key: HBASE-28866
> URL: https://issues.apache.org/jira/browse/HBASE-28866
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.4.2, 3.0.0-beta-1
>Reporter: Ariadne
>Priority: Critical
> Fix For: 3.0.0-beta-1
>
> Attachments: LogCleaner.patch
>
>
>
> Problem
> -
> HBase Master cannot be initialized with the following setting:
>
> hbase.oldwals.cleaner.thread.size
> -1
> Default is 2
>
>
> After running the start-hbase.sh, the Master node could not be started due to
> an exception:
> {code:java}
> ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster: Failed to
> become active master
> java.lang.IllegalArgumentException: Illegal Capacity: -1
> at java.util.ArrayList.(ArrayList.java:157)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.createOldWalsCleaner(LogCleaner.java:149)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.(LogCleaner.java:80)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1329)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:917)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2081)
> at org.apache.hadoop.hbase.master.HMaster.lambda$0(HMaster.java:505)
> at java.lang.Thread.run(Thread.java:750){code}
> We were really confused and misled by the error log as the 'Illegal Capacity'
> of ArrayList seems like an internal code issue.
>
> After we read the source code, we found that
> "hbase.oldwals.cleaner.thread.size" is parsed and used in
> createOldWalsCleaner() function without checking:
> {code:java}
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE); this.oldWALsCleaner =
> createOldWalsCleaner(size); {code}
> The value of "hbase.oldwals.cleaner.thread.size" will be served as the
> initialCapacity of ArrayList. If the configuration value is negative, an
> IllegalArgumentException will be thrown.:
> {code:java}
> private List createOldWalsCleaner(int size) {
> ...
> List oldWALsCleaner = new ArrayList<>(size);
> ...
> } {code}
>
> Solution (the attached patch)
> -
> The basic idea of the attached patch is to add a check and relevant logging
> for this value during the initialization of the {{LogCleaner}} in the
> constructor. This will help users better diagnose the issue. The detailed
> patch is shown below.
> {code:java}
> @@ -78,6 +78,11 @@
> public class LogCleaner extends CleanerChore
> pool, params, null);
> this.pendingDelete = new LinkedBlockingQueue<>();
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE);
> + if (size <= 0) {
> + LOG.warn("The size of old WALs cleaner thread is {}, which is invalid,
> "
> + + "the default value will be used.", size);
> + size = DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE;
> + }
> this.oldWALsCleaner = createOldWalsCleaner(size);
> this.cleanerThreadTimeoutMsec =
> conf.getLong(OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC,
> DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC);{code}
> Thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28866) Master will fail to start due to a hard-to-diagnose misconfiguration.
[
https://issues.apache.org/jira/browse/HBASE-28866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariadne updated HBASE-28866:
Summary: Master will fail to start due to a hard-to-diagnose
misconfiguration. (was: Master will be )
> Master will fail to start due to a hard-to-diagnose misconfiguration.
> -
>
> Key: HBASE-28866
> URL: https://issues.apache.org/jira/browse/HBASE-28866
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.4.2, 3.0.0-beta-1
>Reporter: Ariadne
>Priority: Critical
> Fix For: 3.0.0-beta-1
>
> Attachments: LogCleaner.patch
>
>
>
> Problem
> -
> HBase Master cannot be initialized with the following setting:
>
> hbase.oldwals.cleaner.thread.size
> -1
> Default is 2
>
>
> After running the start-hbase.sh, the Master node could not be started due to
> an exception:
> {code:java}
> ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster: Failed to
> become active master
> java.lang.IllegalArgumentException: Illegal Capacity: -1
> at java.util.ArrayList.(ArrayList.java:157)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.createOldWalsCleaner(LogCleaner.java:149)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.(LogCleaner.java:80)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1329)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:917)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2081)
> at org.apache.hadoop.hbase.master.HMaster.lambda$0(HMaster.java:505)
> at java.lang.Thread.run(Thread.java:750){code}
> We were really confused and misled by the error log as the 'Illegal Capacity'
> of ArrayList seems like an internal code issue.
>
> After we read the source code, we found that
> "hbase.oldwals.cleaner.thread.size" is parsed and used in
> createOldWalsCleaner() function without checking:
> {code:java}
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE); this.oldWALsCleaner =
> createOldWalsCleaner(size); {code}
> The value of "hbase.oldwals.cleaner.thread.size" will be served as the
> initialCapacity of ArrayList. If the configuration value is negative, an
> IllegalArgumentException will be thrown.:
> {code:java}
> private List createOldWalsCleaner(int size) {
> ...
> List oldWALsCleaner = new ArrayList<>(size);
> ...
> } {code}
>
> Solution (the attached patch)
> -
> The basic idea of the attached patch is to add a check and relevant logging
> for this value during the initialization of the {{LogCleaner}} in the
> constructor. This will help users better diagnose the issue. The detailed
> patch is shown below.
> {code:java}
> @@ -78,6 +78,11 @@
> public class LogCleaner extends CleanerChore
> pool, params, null);
> this.pendingDelete = new LinkedBlockingQueue<>();
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE);
> + if (size <= 0) {
> + LOG.warn("The size of old WALs cleaner thread is {}, which is invalid,
> "
> + + "the default value will be used.", size);
> + size = DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE;
> + }
> this.oldWALsCleaner = createOldWalsCleaner(size);
> this.cleanerThreadTimeoutMsec =
> conf.getLong(OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC,
> DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC);{code}
> Thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28866) Master will be
[
https://issues.apache.org/jira/browse/HBASE-28866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariadne updated HBASE-28866:
Summary: Master will be (was: Check and add logs for the configuration
settings of log cleaner to prevent runtime errors)
> Master will be
> ---
>
> Key: HBASE-28866
> URL: https://issues.apache.org/jira/browse/HBASE-28866
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.4.2, 3.0.0-beta-1
>Reporter: Ariadne
>Priority: Critical
> Fix For: 3.0.0-beta-1
>
> Attachments: LogCleaner.patch
>
>
>
> Problem
> -
> HBase Master cannot be initialized with the following setting:
>
> hbase.oldwals.cleaner.thread.size
> -1
> Default is 2
>
>
> After running the start-hbase.sh, the Master node could not be started due to
> an exception:
> {code:java}
> ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster: Failed to
> become active master
> java.lang.IllegalArgumentException: Illegal Capacity: -1
> at java.util.ArrayList.(ArrayList.java:157)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.createOldWalsCleaner(LogCleaner.java:149)
> at
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.(LogCleaner.java:80)
> at
> org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1329)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:917)
> at
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2081)
> at org.apache.hadoop.hbase.master.HMaster.lambda$0(HMaster.java:505)
> at java.lang.Thread.run(Thread.java:750){code}
> We were really confused and misled by the error log as the 'Illegal Capacity'
> of ArrayList seems like an internal code issue.
>
> After we read the source code, we found that
> "hbase.oldwals.cleaner.thread.size" is parsed and used in
> createOldWalsCleaner() function without checking:
> {code:java}
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE); this.oldWALsCleaner =
> createOldWalsCleaner(size); {code}
> The value of "hbase.oldwals.cleaner.thread.size" will be served as the
> initialCapacity of ArrayList. If the configuration value is negative, an
> IllegalArgumentException will be thrown.:
> {code:java}
> private List createOldWalsCleaner(int size) {
> ...
> List oldWALsCleaner = new ArrayList<>(size);
> ...
> } {code}
>
> Solution (the attached patch)
> -
> The basic idea of the attached patch is to add a check and relevant logging
> for this value during the initialization of the {{LogCleaner}} in the
> constructor. This will help users better diagnose the issue. The detailed
> patch is shown below.
> {code:java}
> @@ -78,6 +78,11 @@
> public class LogCleaner extends CleanerChore
> pool, params, null);
> this.pendingDelete = new LinkedBlockingQueue<>();
> int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
> DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE);
> + if (size <= 0) {
> + LOG.warn("The size of old WALs cleaner thread is {}, which is invalid,
> "
> + + "the default value will be used.", size);
> + size = DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE;
> + }
> this.oldWALsCleaner = createOldWalsCleaner(size);
> this.cleanerThreadTimeoutMsec =
> conf.getLong(OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC,
> DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC);{code}
> Thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-25140) HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
[
https://issues.apache.org/jira/browse/HBASE-25140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth resolved HBASE-25140.
-
Resolution: Cannot Reproduce
This is known to work to on all Hadoop 3 versions up to 3.3.6. now.
3.4.0 does not work, but that's a different issue tracked elsewhere.
> HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3
> -
>
> Key: HBASE-25140
> URL: https://issues.apache.org/jira/browse/HBASE-25140
> Project: HBase
> Issue Type: Bug
> Components: documentation, hadoop2, test
>Affects Versions: 2.2.3
>Reporter: Miklos Gergely
>Priority: Major
> Attachments: mvn-1.log
>
>
> Running HBaseTestingUtility.startMiniCluster() on HBase 2.2.3 works only with
> hadoop version range 2.8.0 - 3.0.3, for example with 2.4.1 the following
> exception occurs:
>
> {code:java}
> 21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
> Couldn't properly initialize access to HDFS internals. Please update your WAL
> Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
> information.21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] -
> Couldn't properly initialize access to HDFS internals. Please update your WAL
> Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more
> information.java.lang.NoSuchMethodException:
> org.apache.hadoop.hdfs.DFSClient.beginFileLease(long,
> org.apache.hadoop.hdfs.DFSOutputStream) at
> java.lang.Class.getDeclaredMethod(Class.java:2130) ~[?:1.8.0_242] at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createLeaseManager(FanOutOneBlockAsyncDFSOutputHelper.java:198)
> ~[hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:274)
> [hbase-server-2.2.3.jar:2.2.3] at java.lang.Class.forName0(Native Method)
> ~[?:1.8.0_242] at java.lang.Class.forName(Class.java:264) [?:1.8.0_242] at
> org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:136)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:136)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:198)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1871)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1589)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:157)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001)
> [hbase-server-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:184)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:130)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:168)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_242] at
> javax.security.auth.Subject.doAs(Subject.java:360) [?:1.8.0_242] at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1536)
> [hadoop-common-2.4.1.jar:?] at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:341)
> [hbase-common-2.2.3.jar:2.2.3] at
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:165)
> [hbase-server-2.2.3-tests.jar:2.2.3] at
> java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> {code}
> Also upon failure during maven run it would be great if the actual exception
> would be displayed, not just that "Master not initialized after 20ms".
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28868) Permission check is missing for updateRSGroupConfig in branch-2
Nihal Jain created HBASE-28868: -- Summary: Permission check is missing for updateRSGroupConfig in branch-2 Key: HBASE-28868 URL: https://issues.apache.org/jira/browse/HBASE-28868 Project: HBase Issue Type: Bug Affects Versions: 2.7.0 Reporter: Nihal Jain Assignee: Nihal Jain Found this during HBASE-28867, we do not have security check for updateRSGroupConfig in branch-2. See [https://github.com/apache/hbase/blob/0dc334f572329be7eb2455cec3519fc820c04c25/hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java#L450] Same check exists in master [https://github.com/apache/hbase/blob/52082bc5b80a60406bfaaa630ed5cb23027436c1/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L2279] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28867) Backport "HBASE-20653 Add missing observer hooks for region server group to MasterObserver" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-28867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-28867: --- Status: Patch Available (was: Open) > Backport "HBASE-20653 Add missing observer hooks for region server group to > MasterObserver" to branch-2 > --- > > Key: HBASE-28867 > URL: https://issues.apache.org/jira/browse/HBASE-28867 > Project: HBase > Issue Type: Bug >Affects Versions: 2.5.10, 2.6.0, 2.7.0 >Reporter: Ted Yu >Assignee: Nihal Jain >Priority: Major > Labels: pull-request-available > > Currently the following region server group operations don't have > corresponding hook in MasterObserver : > * getRSGroupInfo > * getRSGroupInfoOfServer > * getRSGroupInfoOfTable > * listRSGroup > This JIRA is to > * add them to MasterObserver > * add pre/post hook calls in RSGroupAdminEndpoint thru > master.getMasterCoprocessorHost for the above operations > * add corresponding tests to TestRSGroups (in similar manner to that of > HBASE-20627) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28867) Backport "HBASE-20653 Add missing observer hooks for region server group to MasterObserver" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-28867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HBASE-28867: --- Labels: pull-request-available (was: ) > Backport "HBASE-20653 Add missing observer hooks for region server group to > MasterObserver" to branch-2 > --- > > Key: HBASE-28867 > URL: https://issues.apache.org/jira/browse/HBASE-28867 > Project: HBase > Issue Type: Bug >Affects Versions: 2.6.0, 2.7.0, 2.5.10 >Reporter: Ted Yu >Assignee: Nihal Jain >Priority: Major > Labels: pull-request-available > > Currently the following region server group operations don't have > corresponding hook in MasterObserver : > * getRSGroupInfo > * getRSGroupInfoOfServer > * getRSGroupInfoOfTable > * listRSGroup > This JIRA is to > * add them to MasterObserver > * add pre/post hook calls in RSGroupAdminEndpoint thru > master.getMasterCoprocessorHost for the above operations > * add corresponding tests to TestRSGroups (in similar manner to that of > HBASE-20627) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28867) Backport "HBASE-20653 Add missing observer hooks for region server group to MasterObserver" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-28867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-28867: --- Affects Version/s: 2.5.10 2.6.0 2.7.0 > Backport "HBASE-20653 Add missing observer hooks for region server group to > MasterObserver" to branch-2 > --- > > Key: HBASE-28867 > URL: https://issues.apache.org/jira/browse/HBASE-28867 > Project: HBase > Issue Type: Bug >Affects Versions: 2.6.0, 2.7.0, 2.5.10 >Reporter: Ted Yu >Assignee: Nihal Jain >Priority: Major > > Currently the following region server group operations don't have > corresponding hook in MasterObserver : > * getRSGroupInfo > * getRSGroupInfoOfServer > * getRSGroupInfoOfTable > * listRSGroup > This JIRA is to > * add them to MasterObserver > * add pre/post hook calls in RSGroupAdminEndpoint thru > master.getMasterCoprocessorHost for the above operations > * add corresponding tests to TestRSGroups (in similar manner to that of > HBASE-20627) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28867) Backport "HBASE-20653 Add missing observer hooks for region server group to MasterObserver" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-28867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-28867: --- Fix Version/s: (was: 3.0.0-alpha-1) > Backport "HBASE-20653 Add missing observer hooks for region server group to > MasterObserver" to branch-2 > --- > > Key: HBASE-28867 > URL: https://issues.apache.org/jira/browse/HBASE-28867 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Nihal Jain >Priority: Major > > Currently the following region server group operations don't have > corresponding hook in MasterObserver : > * getRSGroupInfo > * getRSGroupInfoOfServer > * getRSGroupInfoOfTable > * listRSGroup > This JIRA is to > * add them to MasterObserver > * add pre/post hook calls in RSGroupAdminEndpoint thru > master.getMasterCoprocessorHost for the above operations > * add corresponding tests to TestRSGroups (in similar manner to that of > HBASE-20627) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28867) Backport "HBASE-20653 Add missing observer hooks for region server group to MasterObserver" to branch-2
[ https://issues.apache.org/jira/browse/HBASE-28867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883197#comment-17883197 ] Nihal Jain commented on HBASE-28867: Seems these hooks are still missing in branch-2 as HBASE-28867 was merged only in branch-3. I think we should backport this and also put in all active branches. We need this for RANGER-2111. FYI [~zhangduo] , [~ndimiduk] , [~bbeaudreault] > Backport "HBASE-20653 Add missing observer hooks for region server group to > MasterObserver" to branch-2 > --- > > Key: HBASE-28867 > URL: https://issues.apache.org/jira/browse/HBASE-28867 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Nihal Jain >Priority: Major > Fix For: 3.0.0-alpha-1 > > > Currently the following region server group operations don't have > corresponding hook in MasterObserver : > * getRSGroupInfo > * getRSGroupInfoOfServer > * getRSGroupInfoOfTable > * listRSGroup > This JIRA is to > * add them to MasterObserver > * add pre/post hook calls in RSGroupAdminEndpoint thru > master.getMasterCoprocessorHost for the above operations > * add corresponding tests to TestRSGroups (in similar manner to that of > HBASE-20627) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28867) Backport "HBASE-20653 Add missing observer hooks for region server group to MasterObserver" to branch-2
Nihal Jain created HBASE-28867: -- Summary: Backport "HBASE-20653 Add missing observer hooks for region server group to MasterObserver" to branch-2 Key: HBASE-28867 URL: https://issues.apache.org/jira/browse/HBASE-28867 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Nihal Jain Fix For: 3.0.0-alpha-1 Currently the following region server group operations don't have corresponding hook in MasterObserver : * getRSGroupInfo * getRSGroupInfoOfServer * getRSGroupInfoOfTable * listRSGroup This JIRA is to * add them to MasterObserver * add pre/post hook calls in RSGroupAdminEndpoint thru master.getMasterCoprocessorHost for the above operations * add corresponding tests to TestRSGroups (in similar manner to that of HBASE-20627) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28866) Check and add logs for the configuration settings of log cleaner to prevent runtime errors
Chaopeng Luo created HBASE-28866:
Summary: Check and add logs for the configuration settings of log
cleaner to prevent runtime errors
Key: HBASE-28866
URL: https://issues.apache.org/jira/browse/HBASE-28866
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 3.0.0-beta-1, 2.4.2
Reporter: Chaopeng Luo
Fix For: 3.0.0-beta-1
Attachments: LogCleaner.patch
Problem
-
HBase Master cannot be initialized with the following setting:
hbase.oldwals.cleaner.thread.size
-1
Default is 2
After running the start-hbase.sh, the Master node could not be started due to
an exception:
{code:java}
ERROR [master/localhost:16000:becomeActiveMaster] master.HMaster: Failed to
become active master
java.lang.IllegalArgumentException: Illegal Capacity: -1
at java.util.ArrayList.(ArrayList.java:157)
at
org.apache.hadoop.hbase.master.cleaner.LogCleaner.createOldWalsCleaner(LogCleaner.java:149)
at
org.apache.hadoop.hbase.master.cleaner.LogCleaner.(LogCleaner.java:80)
at
org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1329)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:917)
at
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2081)
at org.apache.hadoop.hbase.master.HMaster.lambda$0(HMaster.java:505)
at java.lang.Thread.run(Thread.java:750){code}
We were really confused and misled by the error log as the 'Illegal Capacity'
of ArrayList seems like an internal code issue.
After we read the source code, we found that
"hbase.oldwals.cleaner.thread.size" is parsed and used in
createOldWalsCleaner() function without checking:
{code:java}
int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE); this.oldWALsCleaner =
createOldWalsCleaner(size); {code}
The value of "hbase.oldwals.cleaner.thread.size" will be served as the
initialCapacity of ArrayList. If the configuration value is negative, an
IllegalArgumentException will be thrown.:
{code:java}
private List createOldWalsCleaner(int size) {
...
List oldWALsCleaner = new ArrayList<>(size);
...
} {code}
Solution (the attached patch)
-
The basic idea of the attached patch is to add a check and relevant logging for
this value during the initialization of the {{LogCleaner}} in the constructor.
This will help users better diagnose the issue. The detailed patch is shown
below.
{code:java}
@@ -78,6 +78,11 @@
public class LogCleaner extends CleanerChore
pool, params, null);
this.pendingDelete = new LinkedBlockingQueue<>();
int size = conf.getInt(OLD_WALS_CLEANER_THREAD_SIZE,
DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE);
+ if (size <= 0) {
+ LOG.warn("The size of old WALs cleaner thread is {}, which is invalid, "
+ + "the default value will be used.", size);
+ size = DEFAULT_OLD_WALS_CLEANER_THREAD_SIZE;
+ }
this.oldWALsCleaner = createOldWalsCleaner(size);
this.cleanerThreadTimeoutMsec =
conf.getLong(OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC,
DEFAULT_OLD_WALS_CLEANER_THREAD_TIMEOUT_MSEC);{code}
Thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28864) NoMethodError undefined method assignment_expression?
[
https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883182#comment-17883182
]
Nihal Jain commented on HBASE-28864:
{quote} What version did you test? trunk only? Or also including branch-2?
{quote}
Hi [~apurtell], I had tested both master and branch-2 with JDK17.
I was debugging this issue to understand more on this. Re-tried following on
latest codebase once: with and without patch on master + JDK17.
With patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/book.html#shell
Version 4.0.0-alpha-1-SNAPSHOT, rf0b92778981d77c6e839ee4ea0e4a684757ae19c, Fri
Sep 20 05:27:12 UTC 2024
Took 0.0038 seconds
hbase:001:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.line_no=1
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#>,
@shell=#>,
@debug=false, @interactive=true>>, @binding=#>
hbase:002:0> list
TABLE
0 row(s)
Took 2.7137 seconds
=> []
hbase:003:0> list
TABLE
0 row(s)
Took 0.0092 seconds
=> []
hbase:004:0> quit{code}
Without patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/book.html#shell
Version 4.0.0-alpha-1-SNAPSHOT, r52082bc5b80a60406bfaaa630ed5cb23027436c1, Fri
Sep 20 06:20:05 UTC 2024
Took 0.0053 seconds
hbase:001:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.line_no=1
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#>,
@shell=#>,
@debug=false, @interactive=true>>, @binding=#>
hbase:002:0> list
TABLE
0 row(s)
Took 2.3309 seconds
=> []
hbase:003:0> {code}
Could not reproduce error.
Also built branch-2 + JDK8 without patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.7.0-SNAPSHOT, r0dc334f572329be7eb2455cec3519fc820c04c25, Fri Sep 20
06:48:04 UTC 2024
Took 0.0041 seconds
hbase:001:0> JRUBY_VERSION
=> "9.4.8.0"
hbase:002:0> RUBY_VERSION
hbase:003:0> irb_info
=>
Ruby version: 3.1.4
IRB version: irb 1.4.2 (2022-10-03)
InputMethod: ReidlineInputMethod with Reline 0.4.2
.irbrc path: /usr/local/rvm/rubies/ruby-3.3.4/.irbrc
RUBY_PLATFORM: java
LANG env: C.UTF-8
East Asian Ambiguous Width: 1
hbase:004:0> list
TABLE
0 row(s)
Took 1.1939 seconds
=> []
hbase:005:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.last_value=...
conf.line_no=5
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#,
@connection=#>,
@shell=#,
@connection=#>,
@admin=#,
@admin=#,
@hbck=#,
@connection=#>,
[jira] [Comment Edited] (HBASE-28864) NoMethodError undefined method assignment_expression?
[
https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883182#comment-17883182
]
Nihal Jain edited comment on HBASE-28864 at 9/20/24 7:48 AM:
-
{quote} What version did you test? trunk only? Or also including branch-2?
{quote}
Hi [~apurtell], I had tested both master and branch-2 with JDK17.
I was debugging this issue to understand more on this. Re-tried following on
latest codebase once: with and without patch on master + JDK17.
With patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/book.html#shell
Version 4.0.0-alpha-1-SNAPSHOT, rf0b92778981d77c6e839ee4ea0e4a684757ae19c, Fri
Sep 20 05:27:12 UTC 2024
Took 0.0038 seconds
hbase:001:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.line_no=1
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#>,
@shell=#>,
@debug=false, @interactive=true>>, @binding=#>
hbase:002:0> list
TABLE
0 row(s)
Took 2.7137 seconds
=> []
hbase:003:0> list
TABLE
0 row(s)
Took 0.0092 seconds
=> []
hbase:004:0> quit{code}
Without patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/book.html#shell
Version 4.0.0-alpha-1-SNAPSHOT, r52082bc5b80a60406bfaaa630ed5cb23027436c1, Fri
Sep 20 06:20:05 UTC 2024
Took 0.0053 seconds
hbase:001:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.line_no=1
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#>,
@shell=#>,
@debug=false, @interactive=true>>, @binding=#>
hbase:002:0> list
TABLE
0 row(s)
Took 2.3309 seconds
=> []
hbase:003:0> {code}
Could not reproduce error.
Also built branch-2 + JDK8 without patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.7.0-SNAPSHOT, r0dc334f572329be7eb2455cec3519fc820c04c25, Fri Sep 20
06:48:04 UTC 2024
Took 0.0041 seconds
hbase:001:0> JRUBY_VERSION
=> "9.4.8.0"
hbase:002:0> RUBY_VERSION
=> "3.1.4"
hbase:003:0> irb_info
=>
Ruby version: 3.1.4
IRB version: irb 1.4.2 (2022-10-03)
InputMethod: ReidlineInputMethod with Reline 0.4.2
.irbrc path: /usr/local/rvm/rubies/ruby-3.3.4/.irbrc
RUBY_PLATFORM: java
LANG env: C.UTF-8
East Asian Ambiguous Width: 1
hbase:004:0> list
TABLE
0 row(s)
Took 1.1939 seconds
=> []
hbase:005:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.last_value=...
conf.line_no=5
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#,
@connection=#&
[jira] [Comment Edited] (HBASE-28864) NoMethodError undefined method assignment_expression?
[
https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883182#comment-17883182
]
Nihal Jain edited comment on HBASE-28864 at 9/20/24 7:38 AM:
-
{quote} What version did you test? trunk only? Or also including branch-2?
{quote}
Hi [~apurtell], I had tested both master and branch-2 with JDK17.
I was debugging this issue to understand more on this. Re-tried following on
latest codebase once: with and without patch on master + JDK17.
With patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/book.html#shell
Version 4.0.0-alpha-1-SNAPSHOT, rf0b92778981d77c6e839ee4ea0e4a684757ae19c, Fri
Sep 20 05:27:12 UTC 2024
Took 0.0038 seconds
hbase:001:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.line_no=1
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#>,
@shell=#>,
@debug=false, @interactive=true>>, @binding=#>
hbase:002:0> list
TABLE
0 row(s)
Took 2.7137 seconds
=> []
hbase:003:0> list
TABLE
0 row(s)
Took 0.0092 seconds
=> []
hbase:004:0> quit{code}
Without patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/book.html#shell
Version 4.0.0-alpha-1-SNAPSHOT, r52082bc5b80a60406bfaaa630ed5cb23027436c1, Fri
Sep 20 06:20:05 UTC 2024
Took 0.0053 seconds
hbase:001:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.line_no=1
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#>,
@shell=#>,
@debug=false, @interactive=true>>, @binding=#>
hbase:002:0> list
TABLE
0 row(s)
Took 2.3309 seconds
=> []
hbase:003:0> {code}
Could not reproduce error.
Also built branch-2 + JDK8 without patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.7.0-SNAPSHOT, r0dc334f572329be7eb2455cec3519fc820c04c25, Fri Sep 20
06:48:04 UTC 2024
Took 0.0041 seconds
hbase:001:0> JRUBY_VERSION
=> "9.4.8.0"
hbase:002:0> RUBY_VERSION
hbase:003:0> irb_info
=>
Ruby version: 3.1.4
IRB version: irb 1.4.2 (2022-10-03)
InputMethod: ReidlineInputMethod with Reline 0.4.2
.irbrc path: /usr/local/rvm/rubies/ruby-3.3.4/.irbrc
RUBY_PLATFORM: java
LANG env: C.UTF-8
East Asian Ambiguous Width: 1
hbase:004:0> list
TABLE
0 row(s)
Took 1.1939 seconds
=> []
hbase:005:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.last_value=...
conf.line_no=5
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#,
@connection=#>,
@shell=#,
@connection=#>,
@admin=#
[jira] [Comment Edited] (HBASE-28864) NoMethodError undefined method assignment_expression?
[
https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883182#comment-17883182
]
Nihal Jain edited comment on HBASE-28864 at 9/20/24 7:36 AM:
-
{quote} What version did you test? trunk only? Or also including branch-2?
{quote}
Hi [~apurtell], I had tested both master and branch-2 with JDK17.
I was debugging this issue to understand more on this. Re-tried following on
latest codebase once: with and without patch on master + JDK17.
With patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/book.html#shell
Version 4.0.0-alpha-1-SNAPSHOT, rf0b92778981d77c6e839ee4ea0e4a684757ae19c, Fri
Sep 20 05:27:12 UTC 2024
Took 0.0038 seconds
hbase:001:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.line_no=1
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#>,
@shell=#>,
@debug=false, @interactive=true>>, @binding=#>
hbase:002:0> list
TABLE
0 row(s)
Took 2.7137 seconds
=> []
hbase:003:0> list
TABLE
0 row(s)
Took 0.0092 seconds
=> []
hbase:004:0> quit{code}
Without patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/book.html#shell
Version 4.0.0-alpha-1-SNAPSHOT, r52082bc5b80a60406bfaaa630ed5cb23027436c1, Fri
Sep 20 06:20:05 UTC 2024
Took 0.0053 seconds
hbase:001:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.line_no=1
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#>,
@shell=#>,
@debug=false, @interactive=true>>, @binding=#>
hbase:002:0> list
TABLE
0 row(s)
Took 2.3309 seconds
=> []
hbase:003:0> {code}
Could not reproduce error.
Also built branch-2 + JDK8 without patch:
{code:java}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.7.0-SNAPSHOT, r0dc334f572329be7eb2455cec3519fc820c04c25, Fri Sep 20
06:48:04 UTC 2024
Took 0.0041 seconds
hbase:001:0> JRUBY_VERSION
=> "9.4.8.0"
hbase:002:0> RUBY_VERSION
hbase:003:0> irb_info
=>
Ruby version: 3.1.4
IRB version: irb 1.4.2 (2022-10-03)
InputMethod: ReidlineInputMethod with Reline 0.4.2
.irbrc path: /usr/local/rvm/rubies/ruby-3.3.4/.irbrc
RUBY_PLATFORM: java
LANG env: C.UTF-8
East Asian Ambiguous Width: 1
hbase:004:0> list
TABLE
0 row(s)
Took 1.1939 seconds
=> []
hbase:005:0> puts IRB.CurrentContext
conf.ap_name="hbase"
conf.auto_indent_mode=true
conf.back_trace_limit=0
conf.echo=true
conf.echo_on_assignment=:truncate
conf.extra_doc_dirs=[]
conf.ignore_eof=false
conf.ignore_sigint=true
conf.inspect_method=#,
@init=#>
conf.inspect_mode=true
conf.io=#
conf.irb=#
conf.irb_name="hbase"
conf.irb_path="(hbase)"
conf.last_value=...
conf.line_no=5
conf.load_modules=[]
conf.newline_before_multiline_output=true
conf.prompt_c="%N:%03n:%i* "
conf.prompt_i="%N:%03n:%i> "
conf.prompt_mode=:CUSTOM
conf.prompt_n=nil
conf.prompt_s="%N:%03n:%i%l "
conf.rc=true
conf.return_format="=> %s\n"
conf.thread=#
conf.use_autocomplete=false
conf.use_multiline=nil
conf.use_singleline=nil
conf.verbose=nil
conf.workspace=#,
@connection=#>,
@shell=#,
@connection=#>,
@admin=#,
@ad
[jira] [Comment Edited] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883160#comment-17883160 ] Andrew Kyle Purtell edited comment on HBASE-28864 at 9/20/24 5:56 AM: -- Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition is missing. I can reproduce by launching the shell and tying any command. What version did you test? trunk only? Or also including branch-2? Weird. was (Author: apurtell): Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition is missing. What version did you test? trunk only? Or also including branch-2? > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883160#comment-17883160 ] Andrew Kyle Purtell edited comment on HBASE-28864 at 9/20/24 5:56 AM: -- Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition is missing. I can reproduce by launching the shell and trying any command. What version did you test? trunk only? Or also including branch-2? Weird. was (Author: apurtell): Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition is missing. I can reproduce by launching the shell and tying any command. What version did you test? trunk only? Or also including branch-2? Weird. > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28821) Optimise bucket cache persistence by reusing backmap entry object.
[
https://issues.apache.org/jira/browse/HBASE-28821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883157#comment-17883157
]
Hudson commented on HBASE-28821:
Results for branch branch-2
[build #1149 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Optimise bucket cache persistence by reusing backmap entry object.
> --
>
> Key: HBASE-28821
> URL: https://issues.apache.org/jira/browse/HBASE-28821
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 4.0.0-alpha-1, 3.0.0-beta-2
>Reporter: Janardhan Hungund
>Assignee: Janardhan Hungund
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2
>
>
> During the persistence of backing map entries into the backing map file, we
> create a new BackingMapEntry.Builder for each entry in the backing map. This
> can be optimised by using a single BackingMapEntry.Builder object and using
> it to build each entry during serialisation.
> This Jira tracks the optimisation by avoiding multiple builder objects.
> Thanks,
> Janardhan
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883160#comment-17883160 ] Andrew Kyle Purtell edited comment on HBASE-28864 at 9/20/24 5:55 AM: -- Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition is missing. What version did you test? trunk only? Or also including branch-2? was (Author: apurtell): Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition was missing. Maybe shell in trunk is different enough and the exceptions are swallowed there? I ported HBASE-28250 from branch-2 and am using code that is mostly 2.5, except that hbase-shell is from branch-2 and the jruby, jcodings, and joni definitions have been updated to match. > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883160#comment-17883160 ] Andrew Kyle Purtell commented on HBASE-28864: - Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition was missing. Maybe trunk is different? I ported this change from branch-2 and am using code that is mostly 2.5, except that hbase-shell is from branch-2 and the jruby, jcodings, and joni definitions have been updated to match. > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883160#comment-17883160 ] Andrew Kyle Purtell edited comment on HBASE-28864 at 9/20/24 5:54 AM: -- Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition was missing. Maybe shell in trunk is different enough and the exceptions are swallowed there? I ported HBASE-28250 from branch-2 and am using code that is mostly 2.5, except that hbase-shell is from branch-2 and the jruby, jcodings, and joni definitions have been updated to match. was (Author: apurtell): Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition was missing. Maybe trunk is different? I ported HBASE-28250 from branch-2 and am using code that is mostly 2.5, except that hbase-shell is from branch-2 and the jruby, jcodings, and joni definitions have been updated to match. > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883160#comment-17883160 ] Andrew Kyle Purtell edited comment on HBASE-28864 at 9/20/24 5:54 AM: -- Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition was missing. Maybe trunk is different? I ported HBASE-28250 from branch-2 and am using code that is mostly 2.5, except that hbase-shell is from branch-2 and the jruby, jcodings, and joni definitions have been updated to match. was (Author: apurtell): Linux Centos9 Stream, aarch64, Java 8 It seems this should have been a visible issue, the assignment_expression definition was missing. Maybe trunk is different? I ported this change from branch-2 and am using code that is mostly 2.5, except that hbase-shell is from branch-2 and the jruby, jcodings, and joni definitions have been updated to match. > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28840) Optimise memory utilisation retrieval of bucket-cache from persistence.
[
https://issues.apache.org/jira/browse/HBASE-28840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883158#comment-17883158
]
Hudson commented on HBASE-28840:
Results for branch branch-2
[build #1149 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Optimise memory utilisation retrieval of bucket-cache from persistence.
> ---
>
> Key: HBASE-28840
> URL: https://issues.apache.org/jira/browse/HBASE-28840
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0
>Reporter: Janardhan Hungund
>Assignee: Janardhan Hungund
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2
>
>
> During the persistence of bucket-cache backing map to a file, the backing map
> is divided into multiple smaller chunks and persisted to the file. This
> chunking avoids the high memory utilisation of during persistence, since only
> a small subset of backing map entries need to persisted in one chunk.
> However, during the retrieval of the backing map during the server startup,
> we accumulate all these chunks into a list and then process each chunk to
> recreate the in-memory backing map. Since, all the chunks are fetched from
> the persistence file and then processed, the memory requirement is higher.
> The retrieval of bucket-cache from persistence file can be optimised to
> enable the processing of one chunk at a time to avoid high memory utilisation.
> Thanks,
> Janardhan
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28850) Only return from ReplicationSink.replicationEntries while all background tasks are finished
[
https://issues.apache.org/jira/browse/HBASE-28850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883159#comment-17883159
]
Hudson commented on HBASE-28850:
Results for branch branch-2
[build #1149 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1149/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Only return from ReplicationSink.replicationEntries while all background
> tasks are finished
> ---
>
> Key: HBASE-28850
> URL: https://issues.apache.org/jira/browse/HBASE-28850
> Project: HBase
> Issue Type: Improvement
> Components: Replication, rpc
>Affects Versions: 2.6.0, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.11
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883156#comment-17883156 ] Nihal Jain commented on HBASE-28864: Hi [~apurtell] thanks for reporting this. Is this on a specific OS or with some particular shell setting? Just curious as I had not encountered these during testing for HBASE-28250 on Mac and RHEL. > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28850) Only return from ReplicationSink.replicationEntries while all background tasks are finished
[
https://issues.apache.org/jira/browse/HBASE-28850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883153#comment-17883153
]
Hudson commented on HBASE-28850:
Results for branch branch-2.6
[build #203 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/203/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/203/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/203/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/203/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/203/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/203/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Only return from ReplicationSink.replicationEntries while all background
> tasks are finished
> ---
>
> Key: HBASE-28850
> URL: https://issues.apache.org/jira/browse/HBASE-28850
> Project: HBase
> Issue Type: Improvement
> Components: Replication, rpc
>Affects Versions: 2.6.0, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.11
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28865) Implement proper string builder for MoveRegionRequest in ProtobufUtil.getShortTextFormat
Ranganath Govardhanagiri created HBASE-28865:
Summary: Implement proper string builder for MoveRegionRequest in
ProtobufUtil.getShortTextFormat
Key: HBASE-28865
URL: https://issues.apache.org/jira/browse/HBASE-28865
Project: HBase
Issue Type: Bug
Components: logging
Reporter: Ranganath Govardhanagiri
While looking into an issue where I was trying to debug MoveRegion, came across
this responseTooSlow log where the params aren't properly build for
MoveRegionRequest. Would be good to properly formulate this so that we know the
regionName and other details.
{code:java}
ipc.RpcServer - (responseTooSlow):
{"call":"MoveRegion(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MoveRegionRequest)","starttimems":"<>","responsesize":"0","method":"MoveRegion","param":"TODO:
class
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MoveRegionRequest","processingtimems":830247,"client":"<>","queuetimems":0,"class":"HMaster"}{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-28864: Priority: Minor (was: Major) > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-28864: Status: Patch Available (was: Open) > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28864) NoMethodError undefined method assignment_expression?
[
https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883133#comment-17883133
]
Andrew Kyle Purtell edited comment on HBASE-28864 at 9/20/24 12:10 AM:
---
An example..
{noformat}
hbase:001:0> list
TABLE
TestTable
1 row(s)
Took 0.0877 seconds
Traceback (most recent call last):
NoMethodError (undefined method `assignment_expression?' for
#)
Maybe IRB bug!
{noformat}
was (Author: apurtell):
An example..
{noformat}
The shell is functional but is printing a warning for a NoMethodError after
every statement execution. Filed as a User Story because there is no build yet
for hbase.13.13 so I can't do it as a Bug. Probably a P2.
For example:
{noformat}
hbase:001:0> list
TABLE
TestTable
1 row(s)
Took 0.0877 seconds
Traceback (most recent call last):
NoMethodError (undefined method `assignment_expression?' for
#)
Maybe IRB bug!
{noformat}
> NoMethodError undefined method assignment_expression?
> -
>
> Key: HBASE-28864
> URL: https://issues.apache.org/jira/browse/HBASE-28864
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2
>
>
> After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every
> command the message "NoMethodError undefined method assignment_expression?"
> is printed.
> This is called from code copied from
> https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy
> over the definition of `assignment_expression`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28584) RS SIGSEGV under heavy replication load
[
https://issues.apache.org/jira/browse/HBASE-28584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-28584:
Assignee: (was: Andrew Kyle Purtell)
Resolution: Fixed
Status: Resolved (was: Patch Available)
> RS SIGSEGV under heavy replication load
> ---
>
> Key: HBASE-28584
> URL: https://issues.apache.org/jira/browse/HBASE-28584
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.6.0, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
> Environment: RHEL 7.9
> JDK 11.0.23
> Hadoop 3.2.4
> Hbase 2.5.6
>Reporter: Whitney Jackson
>Priority: Major
> Labels: pull-request-available
> Attachments:
> 0001-Deep-clone-cells-set-to-be-replicated-onto-the-local.patch,
> 0001-Support-configuration-based-selection-of-netty-chann.patch,
> rs_profile_after.html, rs_profile_before.html
>
>
> I'm observing RS crashes under heavy replication load:
>
> {code:java}
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x7f7546873b69, pid=29890, tid=36828
> #
> # JRE version: Java(TM) SE Runtime Environment 18.9 (11.0.23+7) (build
> 11.0.23+7-LTS-222)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM 18.9 (11.0.23+7-LTS-222, mixed
> mode, tiered, compressed oops, g1 gc, linux-amd64)
> # Problematic frame:
> # J 24625 c2
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyBufferToStream(Ljava/io/OutputStream;Ljava/nio/ByteBuffer;II)V
> (75 bytes) @ 0x7f7546873b69 [0x7f7546873960+0x0209]
> {code}
>
> The heavier load comes when a replication peer has been disabled for several
> hours for patching etc. When the peer is re-enabled the replication load is
> high until the peer is all caught up. The crashes happen on the cluster
> receiving the replication edits.
>
> I believe this problem started after upgrading from 2.4.x to 2.5.x.
>
> One possibly relevant non-standard config I run with:
> {code:java}
>
> hbase.region.store.parallel.put.limit
>
> 100
> Added after seeing "failed to accept edits" replication errors
> in the destination region servers indicating this limit was being exceeded
> while trying to process replication edits.
>
> {code}
>
> I understand from other Jiras that the problem is likely around direct memory
> usage by Netty. I haven't yet tried switching the Netty allocator to
> {{unpooled}} or {{{}heap{}}}. I also haven't yet tried any of the
> {{io.netty.allocator.*}} options.
>
> {{MaxDirectMemorySize}} is set to 26g.
>
> Here's the full stack for the relevant thread:
>
> {code:java}
> Stack: [0x7f72e2e5f000,0x7f72e2f6], sp=0x7f72e2f5e450, free
> space=1021k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> J 24625 c2
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyBufferToStream(Ljava/io/OutputStream;Ljava/nio/ByteBuffer;II)V
> (75 bytes) @ 0x7f7546873b69 [0x7f7546873960+0x0209]
> J 26253 c2
> org.apache.hadoop.hbase.ByteBufferKeyValue.write(Ljava/io/OutputStream;Z)I
> (21 bytes) @ 0x7f7545af2d84 [0x7f7545af2d20+0x0064]
> J 22971 c2
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags$KeyValueEncoder.write(Lorg/apache/hadoop/hbase/Cell;)V
> (27 bytes) @ 0x7f754663f700 [0x7f754663f4c0+0x0240]
> J 25251 c2
> org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.write(Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelHandlerContext;Ljava/lang/Object;Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (90 bytes) @ 0x7f7546a53038 [0x7f7546a50e60+0x21d8]
> J 21182 c2
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(Ljava/lang/Object;Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (73 bytes) @ 0x7f7545f4d90c [0x7f7545f4d3a0+0x056c]
> J 21181 c2
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.write(Ljava/lang/Object;ZLorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (149 bytes) @ 0x7f7545fd680c [0x7f7545fd65e0+0x022c]
> J 25389 c2 org.apache.hadoop.hbase.ipc.NettyRpcConnection$$Lambda$247.run()V
> (16 bytes) @ 0x7f7546ade660 [0x7f7546ade140+0x0520]
> J 24098 c2
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(J)Z
> (109 bytes) @ 0x000
[jira] [Commented] (HBASE-28584) RS SIGSEGV under heavy replication load
[
https://issues.apache.org/jira/browse/HBASE-28584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883135#comment-17883135
]
Andrew Kyle Purtell commented on HBASE-28584:
-
Fixed by HBASE-28850
> RS SIGSEGV under heavy replication load
> ---
>
> Key: HBASE-28584
> URL: https://issues.apache.org/jira/browse/HBASE-28584
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.6.0, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
> Environment: RHEL 7.9
> JDK 11.0.23
> Hadoop 3.2.4
> Hbase 2.5.6
>Reporter: Whitney Jackson
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Labels: pull-request-available
> Attachments:
> 0001-Deep-clone-cells-set-to-be-replicated-onto-the-local.patch,
> 0001-Support-configuration-based-selection-of-netty-chann.patch,
> rs_profile_after.html, rs_profile_before.html
>
>
> I'm observing RS crashes under heavy replication load:
>
> {code:java}
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x7f7546873b69, pid=29890, tid=36828
> #
> # JRE version: Java(TM) SE Runtime Environment 18.9 (11.0.23+7) (build
> 11.0.23+7-LTS-222)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM 18.9 (11.0.23+7-LTS-222, mixed
> mode, tiered, compressed oops, g1 gc, linux-amd64)
> # Problematic frame:
> # J 24625 c2
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyBufferToStream(Ljava/io/OutputStream;Ljava/nio/ByteBuffer;II)V
> (75 bytes) @ 0x7f7546873b69 [0x7f7546873960+0x0209]
> {code}
>
> The heavier load comes when a replication peer has been disabled for several
> hours for patching etc. When the peer is re-enabled the replication load is
> high until the peer is all caught up. The crashes happen on the cluster
> receiving the replication edits.
>
> I believe this problem started after upgrading from 2.4.x to 2.5.x.
>
> One possibly relevant non-standard config I run with:
> {code:java}
>
> hbase.region.store.parallel.put.limit
>
> 100
> Added after seeing "failed to accept edits" replication errors
> in the destination region servers indicating this limit was being exceeded
> while trying to process replication edits.
>
> {code}
>
> I understand from other Jiras that the problem is likely around direct memory
> usage by Netty. I haven't yet tried switching the Netty allocator to
> {{unpooled}} or {{{}heap{}}}. I also haven't yet tried any of the
> {{io.netty.allocator.*}} options.
>
> {{MaxDirectMemorySize}} is set to 26g.
>
> Here's the full stack for the relevant thread:
>
> {code:java}
> Stack: [0x7f72e2e5f000,0x7f72e2f6], sp=0x7f72e2f5e450, free
> space=1021k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> J 24625 c2
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyBufferToStream(Ljava/io/OutputStream;Ljava/nio/ByteBuffer;II)V
> (75 bytes) @ 0x7f7546873b69 [0x7f7546873960+0x0209]
> J 26253 c2
> org.apache.hadoop.hbase.ByteBufferKeyValue.write(Ljava/io/OutputStream;Z)I
> (21 bytes) @ 0x7f7545af2d84 [0x7f7545af2d20+0x0064]
> J 22971 c2
> org.apache.hadoop.hbase.codec.KeyValueCodecWithTags$KeyValueEncoder.write(Lorg/apache/hadoop/hbase/Cell;)V
> (27 bytes) @ 0x7f754663f700 [0x7f754663f4c0+0x0240]
> J 25251 c2
> org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.write(Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelHandlerContext;Ljava/lang/Object;Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (90 bytes) @ 0x7f7546a53038 [0x7f7546a50e60+0x21d8]
> J 21182 c2
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(Ljava/lang/Object;Lorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (73 bytes) @ 0x7f7545f4d90c [0x7f7545f4d3a0+0x056c]
> J 21181 c2
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.write(Ljava/lang/Object;ZLorg/apache/hbase/thirdparty/io/netty/channel/ChannelPromise;)V
> (149 bytes) @ 0x7f7545fd680c [0x7f7545fd65e0+0x022c]
> J 25389 c2 org.apache.hadoop.hbase.ipc.NettyRpcConnection$$Lambda$247.run()V
> (16 bytes) @ 0x7f7546ade660 [0x7f7546ade140+0x0520]
> J 24098 c2
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(J)Z
> (109 byte
[jira] [Commented] (HBASE-28864) NoMethodError undefined method assignment_expression?
[
https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883133#comment-17883133
]
Andrew Kyle Purtell commented on HBASE-28864:
-
An example..
{noformat}
The shell is functional but is printing a warning for a NoMethodError after
every statement execution. Filed as a User Story because there is no build yet
for hbase.13.13 so I can't do it as a Bug. Probably a P2.
For example:
{noformat}
hbase:001:0> list
TABLE
TestTable
1 row(s)
Took 0.0877 seconds
Traceback (most recent call last):
NoMethodError (undefined method `assignment_expression?' for
#)
Maybe IRB bug!
{noformat}
> NoMethodError undefined method assignment_expression?
> -
>
> Key: HBASE-28864
> URL: https://issues.apache.org/jira/browse/HBASE-28864
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2
>
>
> After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every
> command the message "NoMethodError undefined method assignment_expression?"
> is printed.
> This is called from code copied from
> https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy
> over the definition of `assignment_expression`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28864) NoMethodError undefined method assignment_expression?
[ https://issues.apache.org/jira/browse/HBASE-28864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HBASE-28864: --- Labels: pull-request-available (was: ) > NoMethodError undefined method assignment_expression? > - > > Key: HBASE-28864 > URL: https://issues.apache.org/jira/browse/HBASE-28864 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 > > > After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every > command the message "NoMethodError undefined method assignment_expression?" > is printed. > This is called from code copied from > https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy > over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28864) NoMethodError undefined method assignment_expression?
Andrew Kyle Purtell created HBASE-28864: --- Summary: NoMethodError undefined method assignment_expression? Key: HBASE-28864 URL: https://issues.apache.org/jira/browse/HBASE-28864 Project: HBase Issue Type: Bug Components: shell Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0 Reporter: Andrew Kyle Purtell Assignee: Andrew Kyle Purtell Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2 After HBASE-28250 Bump jruby to 9.4.8.0 to fix snakeyaml CVE after every command the message "NoMethodError undefined method assignment_expression?" is printed. This is called from code copied from https://github.com/ruby/irb/blob/v1.4.2/lib/irb.rb . The fix is to also copy over the definition of `assignment_expression`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28840) Optimise memory utilisation retrieval of bucket-cache from persistence.
[
https://issues.apache.org/jira/browse/HBASE-28840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883090#comment-17883090
]
Hudson commented on HBASE-28840:
Results for branch master
[build #1166 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1166/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1166/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1166/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Optimise memory utilisation retrieval of bucket-cache from persistence.
> ---
>
> Key: HBASE-28840
> URL: https://issues.apache.org/jira/browse/HBASE-28840
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0
>Reporter: Janardhan Hungund
>Assignee: Janardhan Hungund
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2
>
>
> During the persistence of bucket-cache backing map to a file, the backing map
> is divided into multiple smaller chunks and persisted to the file. This
> chunking avoids the high memory utilisation of during persistence, since only
> a small subset of backing map entries need to persisted in one chunk.
> However, during the retrieval of the backing map during the server startup,
> we accumulate all these chunks into a list and then process each chunk to
> recreate the in-memory backing map. Since, all the chunks are fetched from
> the persistence file and then processed, the memory requirement is higher.
> The retrieval of bucket-cache from persistence file can be optimised to
> enable the processing of one chunk at a time to avoid high memory utilisation.
> Thanks,
> Janardhan
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28850) Only return from ReplicationSink.replicationEntries while all background tasks are finished
[
https://issues.apache.org/jira/browse/HBASE-28850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883092#comment-17883092
]
Hudson commented on HBASE-28850:
Results for branch master
[build #1166 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1166/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1166/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1166/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Only return from ReplicationSink.replicationEntries while all background
> tasks are finished
> ---
>
> Key: HBASE-28850
> URL: https://issues.apache.org/jira/browse/HBASE-28850
> Project: HBase
> Issue Type: Improvement
> Components: Replication, rpc
>Affects Versions: 2.6.0, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.11
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28842) TestRequestAttributes should fail when expected
[
https://issues.apache.org/jira/browse/HBASE-28842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883091#comment-17883091
]
Hudson commented on HBASE-28842:
Results for branch master
[build #1166 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1166/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1166/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1166/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestRequestAttributes should fail when expected
> ---
>
> Key: HBASE-28842
> URL: https://issues.apache.org/jira/browse/HBASE-28842
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 3.0.0
>Reporter: Evelyn Boland
>Assignee: Evelyn Boland
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2, 2.6.1
>
>
> Problem:
> The tests in the TestRequestAttributes class pass even when they should fail.
> I've included an example of a test that should fail but does not below.
> Fix:
> Throw an IOException in the AttributesCoprocessor when the map of expected
> request attributes does not match the map of given request attributes.
>
> Test:
> We set 2+ request attributes on the Get request but always return 0 request
> attributes from AttributesCoprocessor::getRequestAttributesForRowKey method.
> Yet the test passes even though the map of expected request attributes never
> matches the map of given request attributes.
> {code:java}
> @Category({ ClientTests.class, MediumTests.class })
> public class TestRequestAttributes {
> @ClassRule
> public static final HBaseClassTestRule CLASS_RULE =
> HBaseClassTestRule.forClass(TestRequestAttributes.class);
> private static final byte[] ROW_KEY1 = Bytes.toBytes("1");
> private static final Map>
> ROW_KEY_TO_REQUEST_ATTRIBUTES =
> new HashMap<>();
> static {
> CONNECTION_ATTRIBUTES.put("clientId", Bytes.toBytes("foo"));
> ROW_KEY_TO_REQUEST_ATTRIBUTES.put(ROW_KEY1, addRandomRequestAttributes());
> }
> private static final ExecutorService EXECUTOR_SERVICE =
> Executors.newFixedThreadPool(100);
> private static final byte[] FAMILY = Bytes.toBytes("0");
> private static final TableName TABLE_NAME =
> TableName.valueOf("testRequestAttributes");
> private static final HBaseTestingUtil TEST_UTIL = new HBaseTestingUtil();
> private static SingleProcessHBaseCluster cluster;
> @BeforeClass
> public static void setUp() throws Exception {
> cluster = TEST_UTIL.startMiniCluster(1);
> Table table = TEST_UTIL.createTable(TABLE_NAME, new byte[][] { FAMILY },
> 1,
> HConstants.DEFAULT_BLOCKSIZE, AttributesCoprocessor.class.getName());
> table.close();
> }
> @AfterClass
> public static void afterClass() throws Exception {
> cluster.close();
> TEST_UTIL.shutdownMiniCluster();
> }
> @Test
> public void testRequestAttributesGet() throws IOException {
> Configuration conf = TEST_UTIL.getConfiguration();
> try (
> Connection conn = ConnectionFactory.createConnection(conf, null,
> AuthUtil.loginClient(conf),
> CONNECTION_ATTRIBUTES);
> Table table = configureRequestAttributes(conn.getTableBuilder(TABLE_NAME,
> EXECUTOR_SERVICE),
> ROW_KEY_TO_REQUEST_ATTRIBUTES.get(ROW_KEY1)).build()) {
> table.get(new Get(ROW_KEY1));
> }
> }
> private static Map addRandomRequestAttributes() {
> Map requestAttributes = new HashMap<>();
> int j = Math.max(2, (int) (10 * Math.random()));
> for (int i = 0; i < j; i++) {
> requestAttributes.put(String.valueOf(i),
> Bytes.toBytes(UUID.randomUUID().toString()));
> }
> return requestAttributes;
> }
> public static class AttributesCoprocessor implements RegionObserver,
> RegionCoprocessor {
> @Override
> public Optional getRegionObserver() {
> return Optional.of(this);
> }
> @Override
> public void preGetOp(ObserverContext c, Get
> get,
> List result) throws IOEx
[jira] [Created] (HBASE-28863) Create Snapshot V1 reference files using SFT
Prathyusha created HBASE-28863: -- Summary: Create Snapshot V1 reference files using SFT Key: HBASE-28863 URL: https://issues.apache.org/jira/browse/HBASE-28863 Project: HBase Issue Type: Improvement Reporter: Prathyusha HBASE-28564 adds apis to handler Reference files in SFT for create, read, hasReferences. This Jira refactors Snapshot code (V1 manifest) to create reference files using SFT and make the Snapshot layer SFT aware -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28850) Only return from ReplicationSink.replicationEntries while all background tasks are finished
[
https://issues.apache.org/jira/browse/HBASE-28850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883076#comment-17883076
]
Hudson commented on HBASE-28850:
Results for branch branch-3
[build #294 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/294/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/294/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/294/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Only return from ReplicationSink.replicationEntries while all background
> tasks are finished
> ---
>
> Key: HBASE-28850
> URL: https://issues.apache.org/jira/browse/HBASE-28850
> Project: HBase
> Issue Type: Improvement
> Components: Replication, rpc
>Affects Versions: 2.6.0, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.11
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28840) Optimise memory utilisation retrieval of bucket-cache from persistence.
[
https://issues.apache.org/jira/browse/HBASE-28840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883075#comment-17883075
]
Hudson commented on HBASE-28840:
Results for branch branch-3
[build #294 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/294/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/294/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk17 hadoop3 checks{color}
-- For more information [see jdk17
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/294/JDK17_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Optimise memory utilisation retrieval of bucket-cache from persistence.
> ---
>
> Key: HBASE-28840
> URL: https://issues.apache.org/jira/browse/HBASE-28840
> Project: HBase
> Issue Type: Bug
> Components: BucketCache
>Affects Versions: 3.0.0-beta-1, 4.0.0-alpha-1, 2.7.0
>Reporter: Janardhan Hungund
>Assignee: Janardhan Hungund
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2
>
>
> During the persistence of bucket-cache backing map to a file, the backing map
> is divided into multiple smaller chunks and persisted to the file. This
> chunking avoids the high memory utilisation of during persistence, since only
> a small subset of backing map entries need to persisted in one chunk.
> However, during the retrieval of the backing map during the server startup,
> we accumulate all these chunks into a list and then process each chunk to
> recreate the in-memory backing map. Since, all the chunks are fetched from
> the persistence file and then processed, the memory requirement is higher.
> The retrieval of bucket-cache from persistence file can be optimised to
> enable the processing of one chunk at a time to avoid high memory utilisation.
> Thanks,
> Janardhan
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28835) Make connector support for Decimal type
[ https://issues.apache.org/jira/browse/HBASE-28835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HBASE-28835: --- Labels: pull-request-available (was: ) > Make connector support for Decimal type > --- > > Key: HBASE-28835 > URL: https://issues.apache.org/jira/browse/HBASE-28835 > Project: HBase > Issue Type: Improvement > Components: spark >Affects Versions: connector-1.0.0 >Reporter: yan.duan >Priority: Minor > Labels: pull-request-available > Fix For: connector-1.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28850) Only return from ReplicationSink.replicationEntries while all background tasks are finished
[ https://issues.apache.org/jira/browse/HBASE-28850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28850: - Status: Patch Available (was: Open) > Only return from ReplicationSink.replicationEntries while all background > tasks are finished > --- > > Key: HBASE-28850 > URL: https://issues.apache.org/jira/browse/HBASE-28850 > Project: HBase > Issue Type: Improvement > Components: Replication, rpc >Affects Versions: 3.0.0-beta-1, 2.5.6, 2.6.0, 4.0.0-alpha-1 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.6.1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28850) Only return from ReplicationSink.replicationEntries while all background tasks are finished
[ https://issues.apache.org/jira/browse/HBASE-28850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28850: -- Fix Version/s: 2.7.0 3.0.0-beta-2 2.5.11 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to all active branches. Thanks [~apurtell], [~vjasani] and [~ndimiduk]! > Only return from ReplicationSink.replicationEntries while all background > tasks are finished > --- > > Key: HBASE-28850 > URL: https://issues.apache.org/jira/browse/HBASE-28850 > Project: HBase > Issue Type: Improvement > Components: Replication, rpc >Affects Versions: 2.6.0, 2.5.6, 3.0.0-beta-1, 4.0.0-alpha-1 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.11 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28158) Decouple RIT list management from TRSP invocation
[ https://issues.apache.org/jira/browse/HBASE-28158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28158: - Component/s: master Region Assignment > Decouple RIT list management from TRSP invocation > - > > Key: HBASE-28158 > URL: https://issues.apache.org/jira/browse/HBASE-28158 > Project: HBase > Issue Type: Bug > Components: master, Region Assignment >Affects Versions: 2.6.0, 2.5.6, 3.0.0-beta-1 >Reporter: Andrew Kyle Purtell >Priority: Major > > Operators bypassed some in progress TRSPs leading to a state where some > regions were persistently in transition but hidden. Because the master builds > its list of regions in transition by tracking TRSP, the bypass of TRSP > removed the regions from the RIT list. > Although I can see from reading the code this is the expected behavior, it is > surprising for operators and should be changed. Operators expect that regions > that should be open but are not appear the master's RIT list, provided by > /rits.jsp, the output of the shell's 'rit' command, and in ClusterStatus. > We should only remove a region from the RIT map when assignment reaches a > suitable terminal state. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28158) Decouple RIT list management from TRSP invocation
[ https://issues.apache.org/jira/browse/HBASE-28158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28158: - Fix Version/s: (was: 4.0.0-alpha-1) (was: 2.7.0) (was: 3.0.0-beta-2) (was: 2.6.1) (was: 2.5.11) Affects Version/s: 3.0.0-beta-1 2.6.0 folding fix versions into affects versions. > Decouple RIT list management from TRSP invocation > - > > Key: HBASE-28158 > URL: https://issues.apache.org/jira/browse/HBASE-28158 > Project: HBase > Issue Type: Bug >Affects Versions: 2.6.0, 2.5.6, 3.0.0-beta-1 >Reporter: Andrew Kyle Purtell >Priority: Major > Labels: pull-request-available > > Operators bypassed some in progress TRSPs leading to a state where some > regions were persistently in transition but hidden. Because the master builds > its list of regions in transition by tracking TRSP, the bypass of TRSP > removed the regions from the RIT list. > Although I can see from reading the code this is the expected behavior, it is > surprising for operators and should be changed. Operators expect that regions > that should be open but are not appear the master's RIT list, provided by > /rits.jsp, the output of the shell's 'rit' command, and in ClusterStatus. > We should only remove a region from the RIT map when assignment reaches a > suitable terminal state. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28158) Decouple RIT list management from TRSP invocation
[ https://issues.apache.org/jira/browse/HBASE-28158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28158: - Labels: (was: pull-request-available) > Decouple RIT list management from TRSP invocation > - > > Key: HBASE-28158 > URL: https://issues.apache.org/jira/browse/HBASE-28158 > Project: HBase > Issue Type: Bug >Affects Versions: 2.6.0, 2.5.6, 3.0.0-beta-1 >Reporter: Andrew Kyle Purtell >Priority: Major > > Operators bypassed some in progress TRSPs leading to a state where some > regions were persistently in transition but hidden. Because the master builds > its list of regions in transition by tracking TRSP, the bypass of TRSP > removed the regions from the RIT list. > Although I can see from reading the code this is the expected behavior, it is > surprising for operators and should be changed. Operators expect that regions > that should be open but are not appear the master's RIT list, provided by > /rits.jsp, the output of the shell's 'rit' command, and in ClusterStatus. > We should only remove a region from the RIT map when assignment reaches a > suitable terminal state. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28158) Decouple RIT list management from TRSP invocation
[ https://issues.apache.org/jira/browse/HBASE-28158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-28158: - Labels: pull-request-available (was: ) > Decouple RIT list management from TRSP invocation > - > > Key: HBASE-28158 > URL: https://issues.apache.org/jira/browse/HBASE-28158 > Project: HBase > Issue Type: Bug >Affects Versions: 2.5.6 >Reporter: Andrew Kyle Purtell >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.11 > > > Operators bypassed some in progress TRSPs leading to a state where some > regions were persistently in transition but hidden. Because the master builds > its list of regions in transition by tracking TRSP, the bypass of TRSP > removed the regions from the RIT list. > Although I can see from reading the code this is the expected behavior, it is > surprising for operators and should be changed. Operators expect that regions > that should be open but are not appear the master's RIT list, provided by > /rits.jsp, the output of the shell's 'rit' command, and in ClusterStatus. > We should only remove a region from the RIT map when assignment reaches a > suitable terminal state. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28447) New site configuration option "hfile.block.size"
[
https://issues.apache.org/jira/browse/HBASE-28447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-28447:
-
Component/s: regionserver
Fix Version/s: (was: 2.7.0)
(was: 3.0.0-beta-2)
(was: 2.6.1)
(was: 2.5.11)
Affects Version/s: 3.0.0-beta-1
2.6.0
2.5.0
folding fix versions into affects versions.
> New site configuration option "hfile.block.size"
>
>
> Key: HBASE-28447
> URL: https://issues.apache.org/jira/browse/HBASE-28447
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-beta-1
>Reporter: Gourab Taparia
>Assignee: Andrew Kyle Purtell
>Priority: Minor
> Labels: pull-request-available
>
> Right now there is no config attached to the HFile block size by which we can
> override the default. The default is set to 64 KB in
> HConstants.DEFAULT_BLOCKSIZE . We need a global config property that would go
> on hbase-site.xm which can control this value.
> Since the BLOCKSIZE is tracked at the column family level - we will need to
> respect the CFD value first. Also, configuration settings are also something
> that can be set in schema, at the column or table level, and will override
> the relevant values from the site file. Below is the precedence order we can
> use to get the final blocksize value :
> {code:java}
> ColumnFamilyDescriptor.BLOCKSIZE > schema level site configuration overrides
> > site configuration > HConstants.DEFAULT_BLOCKSIZE{code}
> PS: There is one related config “hbase.mapreduce.hfileoutputformat.blocksize”
> however that is specific to map-reduce jobs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27380) RitDuration histogram metric is broken
[ https://issues.apache.org/jira/browse/HBASE-27380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883040#comment-17883040 ] Duo Zhang commented on HBASE-27380: --- Will take a look later. > RitDuration histogram metric is broken > -- > > Key: HBASE-27380 > URL: https://issues.apache.org/jira/browse/HBASE-27380 > Project: HBase > Issue Type: Bug >Affects Versions: 2.5.0, 2.6.0, 3.0.0-beta-1, 4.0.0-alpha-1 >Reporter: Bryan Beaudreault >Priority: Minor > > Looks like the method which updates it in MetricsAssignmentManager, > updateRitDuration, got broken somewhere along the way. It's no longer used. > We should wire this back up. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsAssignmentManager.java#L82 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28221) Introduce table metric for blockedRequestCount
[
https://issues.apache.org/jira/browse/HBASE-28221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-28221:
-
Fix Version/s: (was: 2.7.0)
(was: 3.0.0-beta-2)
(was: 2.6.1)
(was: 2.5.11)
Affects Version/s: 3.0.0-beta-1
2.6.0
Summary: Introduce table metric for blockedRequestCount (was:
Introduce regionserver metric for delayed flushes)
Fixing jira title as per current discussion. Folding fix versions into affects
versions.
> Introduce table metric for blockedRequestCount
> --
>
> Key: HBASE-28221
> URL: https://issues.apache.org/jira/browse/HBASE-28221
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.6.0, 2.4.17, 2.5.6, 3.0.0-beta-1
>Reporter: Viraj Jasani
>Assignee: Rahul Kumar
>Priority: Major
>
> If compaction is disabled temporarily to allow stabilizing hdfs load, we can
> forget re-enabling the compaction. This can result into flushes getting
> delayed for "hbase.hstore.blockingWaitTime" time (90s). While flushes do
> happen eventually after waiting for max blocking time, it is important to
> realize that any cluster cannot function well with compaction disabled for
> significant amount of time.
>
> We would also block any write requests until region is flushed (90+ sec, by
> default):
> {code:java}
> 2023-11-27 20:40:52,124 WARN [,queue=18,port=60020] regionserver.HRegion -
> Region is too busy due to exceeding memstore size limit.
> org.apache.hadoop.hbase.RegionTooBusyException: Above memstore limit,
> regionName=table1,1699923733811.4fd5e52e2133df1e347f32c646f23ab4.,
> server=server-1,60020,1699421714454, memstoreSize=1073820928,
> blockingMemStoreSize=1073741824
> at
> org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:4200)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3264)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:3215)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:967)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:895)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2524)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36812)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2432)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:311)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:291)
> {code}
>
> Delayed flush logs:
> {code:java}
> LOG.warn("{} has too many store files({}); delaying flush up to {} ms",
> region.getRegionInfo().getEncodedName(), getStoreFileCount(region),
> this.blockingWaitTime); {code}
> Suggestion: Introduce regionserver metric (MetricsRegionServerSource) for the
> num of flushes getting delayed due to too many store files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28192) Master should recover if meta region state is inconsistent
[
https://issues.apache.org/jira/browse/HBASE-28192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-28192:
-
Fix Version/s: (was: 2.7.0)
(was: 3.0.0-beta-2)
(was: 2.6.1)
(was: 2.5.11)
Affects Version/s: 3.0.0-beta-1
2.6.0
Folding fix versions into affects versions.
> Master should recover if meta region state is inconsistent
> --
>
> Key: HBASE-28192
> URL: https://issues.apache.org/jira/browse/HBASE-28192
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.6.0, 2.4.17, 2.5.6, 3.0.0-beta-1
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>
> During active master initialization, before we set master as active (i.e.
> {_}setInitialized(true){_}), we need both meta and namespace regions online.
> If the region state of meta or namespace is inconsistent, active master can
> get stuck in the initialization step:
> {code:java}
> private boolean isRegionOnline(RegionInfo ri) {
> RetryCounter rc = null;
> while (!isStopped()) {
> ...
> ...
> ...
> // Check once-a-minute.
> if (rc == null) {
> rc = new RetryCounterFactory(Integer.MAX_VALUE, 1000, 60_000).create();
> }
> Threads.sleep(rc.getBackoffTimeAndIncrementAttempts());
> }
> return false;
> }
> {code}
> In one of the recent outage, we observed that meta was online on a server,
> which was correctly reflected in meta znode, but the server starttime was
> different. This means that as per the latest transition record, meta was
> marked online on old server (same server with old start time). This kept
> active master initialization waiting forever and some SCPs got stuck in
> initial stage where they need to access meta table before getting candidate
> for region moves.
> The only way out of this outage is for operator to schedule recoveries using
> hbck for old server, which triggers SCP for old server address of meta. Since
> many SCPs were stuck, the processing of new SCP too was taking some time and
> manual restart of active master triggered failover, and new master was able
> to complete SCP for old meta server, correcting the meta assignment details,
> which eventually marked master as active and only after this, we were able to
> see real large num of RITs that were hidden so far.
> We need to let master recover from this state to avoid manual intervention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27380) RitDuration histogram metric is broken
[ https://issues.apache.org/jira/browse/HBASE-27380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883038#comment-17883038 ] Nick Dimiduk commented on HBASE-27380: -- [~zhangduo] do you have a suggestion here? > RitDuration histogram metric is broken > -- > > Key: HBASE-27380 > URL: https://issues.apache.org/jira/browse/HBASE-27380 > Project: HBase > Issue Type: Bug >Affects Versions: 2.5.0, 2.6.0, 3.0.0-beta-1, 4.0.0-alpha-1 >Reporter: Bryan Beaudreault >Priority: Minor > > Looks like the method which updates it in MetricsAssignmentManager, > updateRitDuration, got broken somewhere along the way. It's no longer used. > We should wire this back up. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsAssignmentManager.java#L82 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27380) RitDuration histogram metric is broken
[ https://issues.apache.org/jira/browse/HBASE-27380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883037#comment-17883037 ] Nick Dimiduk commented on HBASE-27380: -- Okay, when to update this metric is not obvious to me. The `RegionStates` class has a `regionInTransition` Map, which is updated via delegation to `RegionStateNodes`. I don't think it's sufficient to update the metric upon removal from this map, because it appears that entries come and go from this map as responsibility is passed across procedures. > RitDuration histogram metric is broken > -- > > Key: HBASE-27380 > URL: https://issues.apache.org/jira/browse/HBASE-27380 > Project: HBase > Issue Type: Bug >Affects Versions: 2.5.0, 2.6.0, 3.0.0-beta-1, 4.0.0-alpha-1 >Reporter: Bryan Beaudreault >Priority: Minor > > Looks like the method which updates it in MetricsAssignmentManager, > updateRitDuration, got broken somewhere along the way. It's no longer used. > We should wire this back up. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetricsAssignmentManager.java#L82 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28862) Change the generic type for ObserverContext from 'RegionCoprocessorEnvironment' to '? extends RegionCoprocessorEnvironment' in RegionObserver
[ https://issues.apache.org/jira/browse/HBASE-28862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HBASE-28862: --- Labels: pull-request-available (was: ) > Change the generic type for ObserverContext from > 'RegionCoprocessorEnvironment' to '? extends RegionCoprocessorEnvironment' in > RegionObserver > - > > Key: HBASE-28862 > URL: https://issues.apache.org/jira/browse/HBASE-28862 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, regionserver >Reporter: Duo Zhang >Priority: Major > Labels: pull-request-available > > This will be a breaking change for coprocessor implementation, but the > ability of region observer is not changed, so I think it is OK to include > this in 3.0.0 release, as we have already changed the coprocessor protobuf to > the relocated one, which already breaks lots of coprocessors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28862) Change the generic type for ObserverContext from 'RegionCoprocessorEnvironment' to '? extends RegionCoprocessorEnvironment' in RegionObserver
Duo Zhang created HBASE-28862: - Summary: Change the generic type for ObserverContext from 'RegionCoprocessorEnvironment' to '? extends RegionCoprocessorEnvironment' in RegionObserver Key: HBASE-28862 URL: https://issues.apache.org/jira/browse/HBASE-28862 Project: HBase Issue Type: Sub-task Components: Coprocessors, regionserver Reporter: Duo Zhang This will be a breaking change for coprocessor implementation, but the ability of region observer is not changed, so I think it is OK to include this in 3.0.0 release, as we have already changed the coprocessor protobuf to the relocated one, which already breaks lots of coprocessors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
