Re: Multi-Tenancy and shared records
>>would it be best to
use the HBase API for creating the data.
yes, you can use HBase API but you need to ensure that Phoenix Data type
APIs are used to
convert your column values into bytes and also while creating a composite
key(if applicable).
otherwise you would not be able to read data from Phoenix when using
different data types
other than varchar or unsigned_bigint.
>> The sparse nature of the data means
that I will be constantly adding new columns, not sure if Phoenix would
have a problem with that.
Phoenix supports dynamic columns so you should not have a problem with that.
Regards,
Ankit Singhal
On Wed, Sep 4, 2019 at 6:24 PM Simon Mottram
wrote:
> Hi Ankit
>
> Thats very useful, many thanks.
>
> Before I dive into using Phoenix (which has given me a torrid time over
> the last few days!), is using Phoenix the best option given that I'm
> doing some low level access to Cell information, or would it be best to
> use the HBase API for creating the data.
>
> We would of course use Phoenix for querying the tables, I'm just
> wondering if the import of data would be better handled via the native
> HBase API.
>
> I think I only need to set labels or use the ACL system, everything
> else should be straight forward. The sparse nature of the data means
> that I will be constantly adding new columns, not sure if Phoenix would
> have a problem with that.
>
> Best Regards
>
> Simon
>
> On Tue, 2019-09-03 at 16:30 -0700, Ankit Singhal wrote:
> > >> If not possible I guess we have to look at doing something at the
> > HBase
> > level.
> > As Josh said, it's not yet supported in Phoenix, Though you may try
> > using cell-level security of HBase with some Phoenix internal API and
> > let us know if it works for you.
> > Sharing a sample code if you wanna try.
> >
> > /**
> > * Do writes using cell based ACLs
> > **/
> > Properties props = new Properties();
> > //conf = Hbase conf
> > PhoenixConnection conn = (PhoenixConnection)
> > QueryUtil.getConnection(props, conf);
> > conn.setAutoCommit(false);
> > conn.createStatement().executeUpdate("");
> > final Iterator>> iterator =
> > pconn.getMutationState().toMutations(false);
> > while (iterator.hasNext()) {
> > Pair> kvPair = iterator.next();
> > List mutationList = kvPair.getSecond();
> > byte[] tableName = kvPair.getFirst();
> > for (Mutation mutation : mutationList) {
> > //perms is user->permissions map
> > mutation.setACL(perms);
> > }
> > htable.batch(mutationList);
> > }
> > conn.rollback();
> >
> >
> > On Tue, Sep 3, 2019 at 3:19 PM Simon Mottram <
> > simon.mott...@cucumber.co.nz> wrote:
> > > Hi Josh
> > >
> > > Thought as much, thanks very much for taking the time to respond.
> > >
> > > Appreciated
> > >
> > > Simon
> > >
> > > On Tue, 2019-09-03 at 11:19 -0400, Josh Elser wrote:
> > > > Hi Simon,
> > > >
> > > > Phoenix does not provide any authorization/security layers on top
> > > of
> > > > what HBase does (the thread on user@hbase has a suggestion on
> > > cell
> > > > ACLs
> > > > which is good).
> > > >
> > > > I think the question you're ultimately asking is: no, the
> > > TenantID
> > > > is
> > > > not an authorization layer. In a nut-shell, the TenantID is just
> > > an
> > > > extra attribute (column) added to your primary key constraint
> > > > auto-magically. If a user doesn't set a TenantID, then they see
> > > _all_
> > > > data.
> > > >
> > > > Unless you have a layer in-between Phoenix and your end-users
> > > that
> > > > add
> > > > extra guarantees/restrictions, a user could set their own
> > > TenantID
> > > > and
> > > > see other folks' data. I don't think this is a good solution for
> > > > what
> > > > you're trying to accomplish.
> > > >
> > > > On 9/2/19 8:34 PM, Simon Mottram wrote:
> > > > > Hi
> > > > >
> > > > > I'm working on a project where we have a combination of very
> > > sparse
> > > > > data columns with added headaches of multi-tenancy. Hbase
> > > looks
> > > > > great
> > > > > for the back end but I need to check that we can support the
> > > > > customer's
> > > > > multi-tenancy requirements.
> > > > >
> > > > > There are 2 that I'm struggling to find a definitive answer
> > > for.
> > > > > Any
> > > > > info most gratefully received
> > > > >
> > > > > Shared Data
> > > > > ===
> > > > > Each record in the table must be secured but it could be
> > > multiple
> > > > > tenants for a record. Think 'shared' data.
> > > > >
> > > > > So for example if you had 3 records
> > > > >
> > > > > record1, some secret data
> > > > > record2, some other secret data
> > > > > record3, data? what data.
> > > > >
> > > > > We need
> > > > > user1 to be able to see record1 and record2
> > > > > user2 to be able to see record2 and record3
> > > > >
> > > > > From what I see in the mult-tenancy doco, the tenant_id field
> > > is a
> > > > > VARCHAR, can this be multiple values?
> > > > >
> > > > > The actual 'multiple tenant'
Re: Multi-Tenancy and shared records
Hi Ankit
Thats very useful, many thanks.
Before I dive into using Phoenix (which has given me a torrid time over
the last few days!), is using Phoenix the best option given that I'm
doing some low level access to Cell information, or would it be best to
use the HBase API for creating the data.
We would of course use Phoenix for querying the tables, I'm just
wondering if the import of data would be better handled via the native
HBase API.
I think I only need to set labels or use the ACL system, everything
else should be straight forward. The sparse nature of the data means
that I will be constantly adding new columns, not sure if Phoenix would
have a problem with that.
Best Regards
Simon
On Tue, 2019-09-03 at 16:30 -0700, Ankit Singhal wrote:
> >> If not possible I guess we have to look at doing something at the
> HBase
> level.
> As Josh said, it's not yet supported in Phoenix, Though you may try
> using cell-level security of HBase with some Phoenix internal API and
> let us know if it works for you.
> Sharing a sample code if you wanna try.
>
> /**
> * Do writes using cell based ACLs
> **/
> Properties props = new Properties();
> //conf = Hbase conf
> PhoenixConnection conn = (PhoenixConnection)
> QueryUtil.getConnection(props, conf);
> conn.setAutoCommit(false);
> conn.createStatement().executeUpdate("");
> final Iterator>> iterator =
> pconn.getMutationState().toMutations(false);
> while (iterator.hasNext()) {
> Pair> kvPair = iterator.next();
> List mutationList = kvPair.getSecond();
> byte[] tableName = kvPair.getFirst();
> for (Mutation mutation : mutationList) {
> //perms is user->permissions map
> mutation.setACL(perms);
> }
> htable.batch(mutationList);
> }
> conn.rollback();
>
>
> On Tue, Sep 3, 2019 at 3:19 PM Simon Mottram <
> simon.mott...@cucumber.co.nz> wrote:
> > Hi Josh
> >
> > Thought as much, thanks very much for taking the time to respond.
> >
> > Appreciated
> >
> > Simon
> >
> > On Tue, 2019-09-03 at 11:19 -0400, Josh Elser wrote:
> > > Hi Simon,
> > >
> > > Phoenix does not provide any authorization/security layers on top
> > of
> > > what HBase does (the thread on user@hbase has a suggestion on
> > cell
> > > ACLs
> > > which is good).
> > >
> > > I think the question you're ultimately asking is: no, the
> > TenantID
> > > is
> > > not an authorization layer. In a nut-shell, the TenantID is just
> > an
> > > extra attribute (column) added to your primary key constraint
> > > auto-magically. If a user doesn't set a TenantID, then they see
> > _all_
> > > data.
> > >
> > > Unless you have a layer in-between Phoenix and your end-users
> > that
> > > add
> > > extra guarantees/restrictions, a user could set their own
> > TenantID
> > > and
> > > see other folks' data. I don't think this is a good solution for
> > > what
> > > you're trying to accomplish.
> > >
> > > On 9/2/19 8:34 PM, Simon Mottram wrote:
> > > > Hi
> > > >
> > > > I'm working on a project where we have a combination of very
> > sparse
> > > > data columns with added headaches of multi-tenancy. Hbase
> > looks
> > > > great
> > > > for the back end but I need to check that we can support the
> > > > customer's
> > > > multi-tenancy requirements.
> > > >
> > > > There are 2 that I'm struggling to find a definitive answer
> > for.
> > > > Any
> > > > info most gratefully received
> > > >
> > > > Shared Data
> > > > ===
> > > > Each record in the table must be secured but it could be
> > multiple
> > > > tenants for a record. Think 'shared' data.
> > > >
> > > > So for example if you had 3 records
> > > >
> > > > record1, some secret data
> > > > record2, some other secret data
> > > > record3, data? what data.
> > > >
> > > > We need
> > > > user1 to be able to see record1 and record2
> > > > user2 to be able to see record2 and record3
> > > >
> > > > From what I see in the mult-tenancy doco, the tenant_id field
> > is a
> > > > VARCHAR, can this be multiple values?
> > > >
> > > > The actual 'multiple tenant' value would be set at creation and
> > > > very
> > > > rarely (if ever) changed, but I couldn't guarantee immutability
> > > >
> > > >
> > > > Enforced Security
> > > > =
> > > > Can you prevent access without TenantId? Otherwise if someone
> > just
> > > > edits the connection info they can sidestep all the multi-
> > tenancy
> > > > features. Our users include scientific types who will want to
> > > > connect
> > > > directly using JDBC/Python/Other so we need to be sure to lock
> > this
> > > > data down.
> > > >
> > > > Of course they want 'admin' types who CAN see all =) Whether
> > there
> > > > is a
> > > > special connection that allows non-tenanted connections or have
> > a
> > > > multi-tenant key that always contains a master tenantid (yuck)
> > > >
> > > > If not possible I guess we have to look at doing something at
> > the
> > > > HBase
Re: Union PreparedStatement ParameterMetaData Parameter value unbound Issue
Thanks James, I wrote https://issues.apache.org/jira/browse/PHOENIX-5467 -- Sent from: http://apache-phoenix-user-list.1124778.n5.nabble.com/
Re: Union PreparedStatement ParameterMetaData Parameter value unbound Issue
Yes, JIRA please. On Wed, Sep 4, 2019 at 1:24 PM lewjackman wrote: > Does this look like an issue for which I should write a Jira? > > > > -- > Sent from: http://apache-phoenix-user-list.1124778.n5.nabble.com/ >
Re: Any reason for so small phoenix.mutate.batchSize by default?
> On 3 Sep 2019, at 19:45, Alexander Batyrshin <0x62...@gmail.com> wrote: > > I observer that there is some extra mutations in batch for every my UPSERTs > For example if app call executeUpdate() only 5 times then on commit there > will be "DEBUG MutationState:1046 - Sent batch of 10" > Can’t figure out where this extra mutations comes from and why. > > This is mean that “useful” batch size is phoenix.mutate.batchSize / 2. Extra mutation is Mutation.Delete because I have NULL value in UPSERT statement
Re: Union PreparedStatement ParameterMetaData Parameter value unbound Issue
Does this look like an issue for which I should write a Jira? -- Sent from: http://apache-phoenix-user-list.1124778.n5.nabble.com/
