Re: jdk 17 and accumulo testing

[Christopher]

I don't think it's intentional. This might be the source of the problem.

On Thu, Dec 15, 2022 at 3:39 PM Vincent Russell
 wrote:
>
> Also in TserverUtils:270, when the TNonblockingServerSocket is created it
> looks like it ends up using the default frame size.  I am not sure if this
> is intentional or not.
>
> On Thu, Dec 15, 2022 at 3:26 PM Vincent Russell 
> wrote:
>
> > Christopher,
> >
> > I am not sure if this issue is related to 3042 or not.
> >
> > On the client side it does look like TConfiguration ends up being called
> > with the default constructor.  I am not sure if this is intentional or not.
> >
> > On the server side I see this stack, so it also looks like:
> >
> > at org.apache.thrift.TConfiguration.(TConfiguration.java:36)
> > at org.apache.thrift.TConfiguration$Builder.build(TConfiguration.java:99)
> > at org.apache.thrift.TConfiguration.(TConfiguration.java:65)
> > at
> > org.apache.thrift.transport.TNonblockingSocket.(TNonblockingSocket.java:74)
> > at
> > org.apache.thrift.transport.TNonblockingSocket.(TNonblockingSocket.java:68)
> > at
> > org.apache.thrift.transport.TNonblockingServerSocket.accept(TNonblockingServerSocket.java:135)
> > at
> > org.apache.thrift.transport.TNonblockingServerSocket.accept(TNonblockingServerSocket.java:36)
> > at
> > org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.handleAccept(TNonblockingServer.java:218)
> > at
> > org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.select(TNonblockingServer.java:186)
> > at
> > org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.run(TNonblockingServer.java:142)
> >
> >
> > I see this in the server log so it does look like it should be using 1G:
> > 2022-09-01 16:59:41 INFO  [org.apache.accumulo.tserver.TabletServer]
> > ServerUtil:124 - general.server.message.size.max = 1G
> >
> > Thanks,
> > Vincent
> >
> > On Thu, Dec 15, 2022 at 12:26 PM Vincent Russell <
> > vincent.russ...@gmail.com> wrote:
> >
> >>
> >>
> >> I had to make a stack trace with hacking together a remote debug instance:
> >>
> >> at
> >> org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.read(AbstractNonblockingServer.java:334)
> >> at
> >> org.apache.accumulo.server.rpc.CustomNonBlockingServer$CustomFrameBuffer.read(CustomNonBlockingServer.java:134)
> >> at
> >> org.apache.thrift.server.AbstractNonblockingServer$AbstractSelectThread.handleRead(AbstractNonblockingServer.java:187)
> >> at
> >> org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.select(TNonblockingServer.java:189)
> >> at
> >> org.apache.thrift.server.TNonblockingServer$SelectAcceptThread.run(TNonblockingServer.java:142)
> >>
> >> On Thu, Dec 15, 2022 at 12:52 AM Christopher  wrote:
> >>
> >>> From the numbers in the message, it looks like you're sending an 18MB
> >>> payload but something in Thrift is limiting things to 16384000
> >>> (16000KB). I doubt you've overridden the default
> >>> general.server.message.size.max to be anything that low (the default
> >>> is 1G). Unless you're flushing after every mutation, it would not be
> >>> surprising to exceed the 16MB max frame size indicated in the error
> >>> message quite quickly.
> >>>
> >>> This value of 16384000 seemed weird. It looks like it's not using our
> >>> configuration, but using the built-in default value of
> >>> org.apache.thrift.TConfiguration.DEFAULT_MAX_FRAME_SIZE. It looks like
> >>> this can happen whenever `new TConfiguration()` is called without
> >>> parameters... and there's a fair amount of internal code, mostly in
> >>> libthrift itself, that does that. It's a bit tricky to track down the
> >>> one causing this particular issue. If you have a full stack trace, it
> >>> could help.
> >>>
> >>> Also, this might be the same issue seen reported in
> >>> https://github.com/apache/accumulo/issues/3042
> >>>
> >>> On Wed, Dec 14, 2022 at 8:53 PM Vincent Russell
> >>>  wrote:
> >>> >
> >>> > I was able to work out all of my compilation issues; however when I
> >>> run an
> >>> > integration test with the Mini Accumulo Cluster that tests writing
> >>> > mutations with values of 5mb the flush hangs forever
> >>> > and I see  the following logs in the TabletServer logs:
> >>> >
> >>> > 20:41:02.306 [Thread-7] ERROR
> >>> > o.a.a.s.r.CustomNonBlockingServer$CustomFrameBuffer - Read a frame
> >>> size of
> >>> > 18874697, which is bigger than the maximum allowable frame size
> >>> 16384000
> >>> > for ALL connections.
> >>> > 20:41:03.582 [Thread-7] ERROR
> >>> > o.a.a.s.r.CustomNonBlockingServer$CustomFrameBuffer - Read a frame
> >>> size of
> >>> > 18874697, which is bigger than the maximum allowable frame size
> >>> 16384000
> >>> > for ALL connections.
> >>> > 20:41:05.079 [Thread-7] ERROR
> >>> > o.a.a.s.r.CustomNonBlockingServer$CustomFrameBuffer - Read a frame
> >>> size of
> >>> > 18874697, which is bigger than the maximum allowable frame size
> >>> 16384000
> >>> > for ALL connections.
> >>> >
> >>> > Other tests that write smaller amounts of data 

Re: Recommended Permissions

[Christopher]

These could be better documented. It's been awhile since I've thought
about these, but I think:

SystemPermission.SYSTEM - basically permits superuser access to
perform most operations. However, this does not allow automatic access
to tables. A user with this permission, however, can grant themselves
permissions to do so.
SystemPermission.GRANT - this is a special permission that allows the
bearer to grant system permissions to other users. This supports the
case where an admin user is granted system permission, but is not
allowed to share those elevated privileges with other users. At one
point, I think this permission could not be granted... so only the
root user could have this, but I think it can be granted now... but it
must be done so explicitly... either way, the important bit is that it
doesn't come with the SYSTEM permission automatically.
SystemPermission.OBTAIN_DELEGATION_TOKEN - that's a special permission
that allows the user to request delegation tokens, which are
time-limited alternatives to Kerberos credentials when you need to
perform an operation... like MapReduce... across a cluster, and don't
want to (or can't) share your user's authentication credentials. This
is because you probably don't want to share your keytab everywhere, or
run kinit interactively on each MapReduce node in a cluster, in order
to run a MapReduce job. Because these allow impersonation of the
requesting user, they are time-limited, and require explicit
permission to obtain.

For your scenario, the root user is already set up that way. You could
create another user and grant them SYSTEM permissions to do the same.
However, any user you create with SYSTEM permissions will be able to
grant themselves READ and/or WRITE permission on any table they like.
If you need more fine-grained access controls, you could plug in your
own `instance.security.permissionHandler`, but that pluggable
component predates our attempt to create stable SPI endpoints, so it
might not be a stable interface, and you may need to update it for new
releases. Stabilizing that as a proper SPI is a medium term goal of
mine, personally, but I'm not sure when it will happen.

On Thu, Dec 15, 2022 at 1:56 PM Logan Jones  wrote:
>
> Hello:
>
> I'm working on updating our security posture for our various Accumulo
> users. I took a look at the Permissions Page
>  along with the
> Java docs for SystemPermission
> 
> , NamespacePermission
> ,
> and TablePermission
> 
> but
> still have some questions. Specifically, I would like to know what the
> following Permissions are used for:
>
>
>- SystemPermssion.GRANT - My assumption is that this means a user with
>these permissions can grant other users various system permissions.
>Effectively if you have this permission you have the ability to have all
>other system permissions.
>- SystemPermission.SYSTEM
>- SystemPermission.OBTAIN_DELEGATION_TOKEN
>
> Also, I'd be interested in your opinions on what permissions I should set
> up for the following scenario. I would like to create a root user that can
> only manage user/authorizations but cannot read data from any tables, the
> root user would be responsible for creating application users which can do
> everything but create users and alter authorizations. What permissions
> should I set up to make that happen?
>
> Thanks,
>
> - Logan