Re: [DISCUSS] Remove the need for ADMIN permissions for listDecommissionedRegionServers

[Rushabh Shah]

Thank you Viraj and Andrew for the replies.
I have merged HBASE-28391
  to all the active
branches.

On Tue, Feb 27, 2024 at 9:05 AM Andrew Purtell  wrote:

> We have had this discussion before about other list* methods and we have
> sometimes decided to restrict them to ADMIN. The reason for that was the
> information returned by the method might leak sensitive information. For
> example, listing table descriptors will include all arbitrary and
> potentially sensitive user set attributes in the schema.
> I think here the information listed is not sensitive in the same way. Host
> cluster membership, and especially decommissioned hosts, is not secret.
>
> Compatibility should be fine. Someone granted ADMIN permission will still
> be able to invoke this method if the security check is relaxed.
>
> +1
>
> On Tue, Feb 27, 2024 at 8:32 AM Viraj Jasani  wrote:
>
> > +1 for relaxing the permission. While I haven't gone through the history,
> > it seems that requiring ADMIN for listDecomm operation might be an
> > oversight.
> >
> > Unless it is really big deal from compatibility viewpoint, I think we
> > should be fine relaxing this.
> >
> >
> >
> > On Mon, Feb 26, 2024 at 8:55 PM Rushabh Shah
> >  wrote:
> >
> > > Hi hbase-dev,
> > >
> > > Why do we need ADMIN permissions for
> > > AccessController#preListDecommissionedRegionServers
> > > API ?
> > >
> > > From Phoenix, we are calling Admin#getRegionServers(true) where the
> > > argument excludeDecommissionedRS is set to true. [1]
> > > If excludeDecommissionedRS  is set to true and if we have
> > > AccessController co-proc
> > > attached, it requires ADMIN permissions to execute
> > > listDecommissionedRegionServers RPC. [2]
> > >  Snippet below
> > >
> > >   @Override
> > >   public void
> > >
> >
> preListDecommissionedRegionServers(ObserverContext
> > > ctx)
> > > throws IOException {
> > > requirePermission(ctx, "listDecommissionedRegionServers",
> > > Action.ADMIN);
> > >   }
> > >
> > > I understand that we need ADMIN permissions
> > > for preDecommissionRegionServers and preRecommissionRegionServers
> because
> > > it changes the membership of regionservers but I don’t see any need for
> > > ADMIN permissions for listDecommissionedRegionServers.
> > >
> > > Does anyone have objections if we relax the requirement to READ
> > permissions
> > > instead of ADMIN permissions?
> > >
> > > I have created HBASE-28391
> > > <
> https://urldefense.com/v3/__https://issues.apache.org/jira/browse/HBASE-28391__;!!DCbAVzZNrAf4!BkLK4S-sG0Soms4PFYK9G420e4QQdVUdjjz7x0PfOtSq0cvPOLehISZxTqx2Y1tc-bIayELNPNlK6q08dBVJJ4o$
> > to implement this.
> > > Thank you !
> > >
> > >
> > > 1.
> > >
> > >
> >
> https://urldefense.com/v3/__https://github.com/apache/hbase/blob/branch-2.5/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java*L1721-L1730__;Iw!!DCbAVzZNrAf4!BkLK4S-sG0Soms4PFYK9G420e4QQdVUdjjz7x0PfOtSq0cvPOLehISZxTqx2Y1tc-bIayELNPNlK6q085ZMR_0M$
> > >
> > > 2.
> > >
> > >
> >
> https://urldefense.com/v3/__https://github.com/apache/hbase/blob/branch-2.5/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java*L1205-L1207__;Iw!!DCbAVzZNrAf4!BkLK4S-sG0Soms4PFYK9G420e4QQdVUdjjz7x0PfOtSq0cvPOLehISZxTqx2Y1tc-bIayELNPNlK6q08LXwSMKo$
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Unrest, ignorance distilled, nihilistic imbeciles -
> It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
>- A23, Welcome, Apocalypse
>

[jira] [Resolved] (HBASE-28391) Remove the need for ADMIN permissions for listDecommissionedRegionServers

[Rushabh Shah (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-28391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rushabh Shah resolved HBASE-28391.
--
Fix Version/s: 2.6.0
   2.4.18
   4.0.0-alpha-1
   2.7.0
   2.5.8
   3.0.0-beta-2
   Resolution: Fixed

> Remove the need for ADMIN permissions for listDecommissionedRegionServers
> -
>
> Key: HBASE-28391
> URL: https://issues.apache.org/jira/browse/HBASE-28391
> Project: HBase
>  Issue Type: Bug
>  Components: Admin
>Affects Versions: 2.4.17, 2.5.7
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
>  Labels: pull-request-available
> Fix For: 2.6.0, 2.4.18, 4.0.0-alpha-1, 2.7.0, 2.5.8, 3.0.0-beta-2
>
>
> Why we need {{ADMIN}} permissions for 
> {{AccessController#preListDecommissionedRegionServers}} ?
> From Phoenix, we are calling {{Admin#getRegionServers(true)}} where the 
> argument {{excludeDecommissionedRS}} is set to true. Refer 
> [here|https://github.com/apache/hbase/blob/branch-2.5/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java#L1721-L1730].
> If {{excludeDecommissionedRS}}  is set to true and if we have 
> {{AccessController}} co-proc attached, it requires ADMIN permissions to 
> execute {{listDecommissionedRegionServers}} RPC. Refer 
> [here|https://github.com/apache/hbase/blob/branch-2.5/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L1205-L1207].
>  
> {code:java}
>   @Override
>   public void 
> preListDecommissionedRegionServers(ObserverContext
>  ctx)
> throws IOException {
> requirePermission(ctx, "listDecommissionedRegionServers", Action.ADMIN);
>   }
> {code}
> I understand that we need ADMIN permissions for 
> _preDecommissionRegionServers_ and _preRecommissionRegionServer_ because it 
> changes the membership of regionservers but I don’t see any need for ADMIN 
> permissions for _listDecommissionedRegionServers_.  Do you think we can 
> remove need for ADMIN permissions for  _listDecommissionedRegionServers_ RPC?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] Remove the need for ADMIN permissions for listDecommissionedRegionServers

[Andrew Purtell]

We have had this discussion before about other list* methods and we have
sometimes decided to restrict them to ADMIN. The reason for that was the
information returned by the method might leak sensitive information. For
example, listing table descriptors will include all arbitrary and
potentially sensitive user set attributes in the schema.
I think here the information listed is not sensitive in the same way. Host
cluster membership, and especially decommissioned hosts, is not secret.

Compatibility should be fine. Someone granted ADMIN permission will still
be able to invoke this method if the security check is relaxed.

+1

On Tue, Feb 27, 2024 at 8:32 AM Viraj Jasani  wrote:

> +1 for relaxing the permission. While I haven't gone through the history,
> it seems that requiring ADMIN for listDecomm operation might be an
> oversight.
>
> Unless it is really big deal from compatibility viewpoint, I think we
> should be fine relaxing this.
>
>
>
> On Mon, Feb 26, 2024 at 8:55 PM Rushabh Shah
>  wrote:
>
> > Hi hbase-dev,
> >
> > Why do we need ADMIN permissions for
> > AccessController#preListDecommissionedRegionServers
> > API ?
> >
> > From Phoenix, we are calling Admin#getRegionServers(true) where the
> > argument excludeDecommissionedRS is set to true. [1]
> > If excludeDecommissionedRS  is set to true and if we have
> > AccessController co-proc
> > attached, it requires ADMIN permissions to execute
> > listDecommissionedRegionServers RPC. [2]
> >  Snippet below
> >
> >   @Override
> >   public void
> >
> preListDecommissionedRegionServers(ObserverContext
> > ctx)
> > throws IOException {
> > requirePermission(ctx, "listDecommissionedRegionServers",
> > Action.ADMIN);
> >   }
> >
> > I understand that we need ADMIN permissions
> > for preDecommissionRegionServers and preRecommissionRegionServers because
> > it changes the membership of regionservers but I don’t see any need for
> > ADMIN permissions for listDecommissionedRegionServers.
> >
> > Does anyone have objections if we relax the requirement to READ
> permissions
> > instead of ADMIN permissions?
> >
> > I have created HBASE-28391
> >  to implement this.
> > Thank you !
> >
> >
> > 1.
> >
> >
> https://github.com/apache/hbase/blob/branch-2.5/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java#L1721-L1730
> >
> > 2.
> >
> >
> https://github.com/apache/hbase/blob/branch-2.5/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L1205-L1207
> >
>


-- 
Best regards,
Andrew

Unrest, ignorance distilled, nihilistic imbeciles -
It's what we’ve earned
Welcome, apocalypse, what’s taken you so long?
Bring us the fitting end that we’ve been counting on
   - A23, Welcome, Apocalypse

Re: [DISCUSS] Remove the need for ADMIN permissions for listDecommissionedRegionServers

[Viraj Jasani]

+1 for relaxing the permission. While I haven't gone through the history,
it seems that requiring ADMIN for listDecomm operation might be an
oversight.

Unless it is really big deal from compatibility viewpoint, I think we
should be fine relaxing this.



On Mon, Feb 26, 2024 at 8:55 PM Rushabh Shah
 wrote:

> Hi hbase-dev,
>
> Why do we need ADMIN permissions for
> AccessController#preListDecommissionedRegionServers
> API ?
>
> From Phoenix, we are calling Admin#getRegionServers(true) where the
> argument excludeDecommissionedRS is set to true. [1]
> If excludeDecommissionedRS  is set to true and if we have
> AccessController co-proc
> attached, it requires ADMIN permissions to execute
> listDecommissionedRegionServers RPC. [2]
>  Snippet below
>
>   @Override
>   public void
> preListDecommissionedRegionServers(ObserverContext
> ctx)
> throws IOException {
> requirePermission(ctx, "listDecommissionedRegionServers",
> Action.ADMIN);
>   }
>
> I understand that we need ADMIN permissions
> for preDecommissionRegionServers and preRecommissionRegionServers because
> it changes the membership of regionservers but I don’t see any need for
> ADMIN permissions for listDecommissionedRegionServers.
>
> Does anyone have objections if we relax the requirement to READ permissions
> instead of ADMIN permissions?
>
> I have created HBASE-28391
>  to implement this.
> Thank you !
>
>
> 1.
>
> https://github.com/apache/hbase/blob/branch-2.5/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java#L1721-L1730
>
> 2.
>
> https://github.com/apache/hbase/blob/branch-2.5/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java#L1205-L1207
>

[jira] [Created] (HBASE-28407) [thirdparty] Update release instructions

[Nick Dimiduk (Jira)]

Nick Dimiduk created HBASE-28407:


 Summary: [thirdparty] Update release instructions
 Key: HBASE-28407
 URL: https://issues.apache.org/jira/browse/HBASE-28407
 Project: HBase
  Issue Type: Task
  Components: thirdparty
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk


Our release instructions for hbase-thirdparty are out of date. Update them 
based on recent experience.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[VOTE] hbase-thirdparty 4.1.6 (RC0)

[Nick Dimiduk]

Please vote on this Apache hbase thirdparty release candidate,
hbase-thirdparty-4.1.6RC0

The VOTE will remain open for at least 72 hours.

[ ] +1 Release this package as Apache hbase thirdparty 4.1.6
[ ] -1 Do not release this package because ...

The tag to be voted on is 4.1.6RC0:

  https://github.com/apache/hbase-thirdparty/tree/4.1.6RC0

This tag currently points to git reference

  b1a8d26f5eb9f45d4069d1fdd909d0d22123950c

The release files, including signatures, digests, as well as CHANGES.md
and RELEASENOTES.md included in this RC can be found at:

  https://dist.apache.org/repos/dist/dev/hbase/hbase-thirdparty-4.1.6RC0/

Maven artifacts are available in a staging repository at:

  https://repository.apache.org/content/repositories/orgapachehbase-1533/

Artifacts were signed with the 0xEF4EBF27 key which can be found in:

  https://downloads.apache.org/hbase/KEYS

To learn more about Apache hbase thirdparty, please see

  http://hbase.apache.org/

Thanks,
Your HBase Release Manager

[jira] [Created] (HBASE-28406) Move namespace to RS group can support multiple options to move tables under it

[Rajeshbabu Chintaguntla (Jira)]

Rajeshbabu Chintaguntla created HBASE-28406:
---

 Summary: Move namespace to RS group can support multiple options 
to move tables under it
 Key: HBASE-28406
 URL: https://issues.apache.org/jira/browse/HBASE-28406
 Project: HBase
  Issue Type: Improvement
  Components: rsgroup
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
 Fix For: 4.0.0-alpha-1, 3.0.0-beta-2


As discussed here
[https://github.com/apache/hbase/pull/5661#issuecomment-1949090826]

Move namespaces to rsgroup can have multiple options like:
 # *Move All Tables:* This option involves moving all tables within a namespace 
to the new RS group, regardless of the current RS group they belong to. This 
would effectively consolidate all tables of the namespace into the new RS 
group. This is the same as the current implementation so it can be default 
behavior.

 # {*}Move Tables present in current RS group{*}: With this option, only tables 
belonging to the current RS group of the namespace would be moved to the new RS 
group. This provides more granular control, allowing users to choose specific 
tables to move based on their needs. If the namespace does not belong to any RS 
group, the namespace tables in the default RS group would be moved to the new 
RS group.

 # *Move No Tables:* This option involves changing the RS group of the 
namespace without moving any tables. Existing tables would remain in their 
current RS group. This could be useful if there's a desire to separate the 
namespaces by RS group but without immediately moving the tables.

With this options we can give control to user choose proper option based on the 
requirements. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HBASE-28405) Region open procedure silently does nothing without notifying the parent proc

[Aman Poonia (Jira)]

Aman Poonia created HBASE-28405:
---

 Summary: Region open procedure silently does nothing without 
notifying the parent proc
 Key: HBASE-28405
 URL: https://issues.apache.org/jira/browse/HBASE-28405
 Project: HBase
  Issue Type: Bug
  Components: proc-v2
Affects Versions: 2.5.7
Reporter: Aman Poonia
Assignee: Aman Poonia


We had a scenario in production where a merge operation had failed as below

_2024-02-11 10:53:57,715 ERROR [PEWorker-31] 
assignment.MergeTableRegionsProcedure - Error trying to merge 
[a92008b76ccae47d55c590930b837036, f56752ae9f30fad9de5a80a8ba578e4b] in table1 
(in state=MERGE_TABLE_REGIONS_CLOSE_REGIONS)_
_org.apache.hadoop.hbase.HBaseIOException: The parent region state=MERGING, 
location=rs-229,60020,1707587658182, table=table1, 
region=f56752ae9f30fad9de5a80a8ba578e4b is currently in transition, give up_
_at 
org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120)_
_at 
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.createUnassignProcedures(MergeTableRegionsProcedure.java:648)_
_at 
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.executeFromState(MergeTableRegionsProcedure.java:205)_
_at 
org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.executeFromState(MergeTableRegionsProcedure.java:79)_
_at 
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188)_
_at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922)_
_at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650)_
_at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396)_
_at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75)_
_at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1964)_
_at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)_
_at 
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1991)_

Now when we do rollback of failed merge operation we see a issue where region 
is in state opened until the RS holding it stopped.

Rollback create a TRSP as below

_2024-02-11 10:53:57,719 DEBUG [PEWorker-31] procedure2.ProcedureExecutor - 
Stored [pid=26674602, 
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; 
TransitRegionStateProcedure table=table1, 
region=a92008b76ccae47d55c590930b837036, ASSIGN]_

and rollback finished successfully

_2024-02-11 10:53:57,721 INFO [PEWorker-31] procedure2.ProcedureExecutor - 
Rolled back pid=26673594, state=ROLLEDBACK, 
exception=org.apache.hadoop.hbase.HBaseIOException via 
master-merge-regions:org.apache.hadoop.hbase.HBaseIOException: The parent 
region state=MERGING, location=rs-229,60020,1707587658182, table=table1, 
region=f56752ae9f30fad9de5a80a8ba578e4b is currently in transition, give up; 
MergeTableRegionsProcedure table=table1, 
regions=[a92008b76ccae47d55c590930b837036, f56752ae9f30fad9de5a80a8ba578e4b], 
force=false exec-time=1.4820 sec_

We create a procedure to open the region a92008b76ccae47d55c590930b837036

Intrestingly we didnt close the region as creation of procedure to close 
regions had thrown exception and not execution of procedure.

Now when we run TRSP it sends a OpenRegionProcedure which is handled by 
AssignRegionHandler

This handlers on execution suggests that region is already online

Sequence of events are as follow

_2024-02-11 10:53:58,919 INFO [PEWorker-58] assignment.RegionStateStore - 
pid=26674602 updating hbase:meta row=a92008b76ccae47d55c590930b837036, 
regionState=OPENING, regionLocation=rs-210,60020,1707596461539_

_2024-02-11 10:53:58,920 INFO [PEWorker-58] procedure2.ProcedureExecutor - 
Initialized subprocedures=[\{pid=26675798, ppid=26674602, state=RUNNABLE; 
OpenRegionProcedure a92008b76ccae47d55c590930b837036, 
server=rs-210,60020,1707596461539}]_

_2024-02-11 10:53:59,074 WARN [REGION-regionserver/rs-210:60020-10] 
handler.AssignRegionHandler - Received OPEN for 
table1,r1,1685436252488.a92008b76ccae47d55c590930b837036. which is already 
online_



--
This message was sent by Atlassian Jira
(v8.20.10#820010)