Unsubscribe
unsubscribe
unsubscribe
Unsubscribe
Re: Ranger for standalone hive metastore
Thank you, Vihang. The problem I see is that though the hive plugin is enabled, the service is not listed in the Ranger tables, nor does it show up in the Ranger UI. I will have to debug more to understand the issue. I will update in case I am able to figure this out. Thank you. On Wed, Jul 25, 2018 at 2:02 AM Vihang Karajgaonkar wrote: > I am not an expert in Ranger, but as long as Ranger uses HMS public APIs > it should work. Some of the HMS APIs (get_partitions_by_expr) may not work > if you don't have hive jars in the metastore classpath. However, this API > is only used by Hive so I don't think that could be the cause of your > issue. Also, ACID tables have some limitations in standalone mode. > > One way to debug further would be to run a different client to connect to > HMS and see if it is able to connect. Also check if it works in a > non-secure setup (to rule out if the issue is related to Kerberos). > > On Tue, Jul 24, 2018 at 3:45 AM, Sandhya Agarwal > wrote: > >> Thank you so much for the response. However, I do not see any errors in >> both HMS / Ranger logs. But, just to clarify, I am assuming Ranger is >> supported even with the standalone hive metastore. >> >> On Mon, Jul 23, 2018 at 11:09 PM Vihang Karajgaonkar >> wrote: >> >>> I am not super-familiar with Ranger but do you see any errors in HMS >>> logs. Assuming Ranger is trying to connect to HMS, it should log some >>> exceptions if the connection is not successful. Also would be helpful to >>> look for errors in Ranger logs. >>> >>> On Mon, Jul 23, 2018 at 4:21 AM, Sandhya Agarwal >>> wrote: >>> >>>> Hello, >>>> >>>> I am trying to enable the ranger hive plugin 2.0.0, for standalone hive >>>> metastore 3.0.0. I do not see the link happening, even though both my >>>> ranger admin and hive metastore services are restarted and running without >>>> any errors, after enabling the ranger-hive-plugin. Any pointers ? >>>> >>>> Thank you, >>>> Sandhya >>>> >>> >>> >
Re: Ranger for standalone hive metastore
Thank you so much for the response. However, I do not see any errors in both HMS / Ranger logs. But, just to clarify, I am assuming Ranger is supported even with the standalone hive metastore. On Mon, Jul 23, 2018 at 11:09 PM Vihang Karajgaonkar wrote: > I am not super-familiar with Ranger but do you see any errors in HMS logs. > Assuming Ranger is trying to connect to HMS, it should log some exceptions > if the connection is not successful. Also would be helpful to look for > errors in Ranger logs. > > On Mon, Jul 23, 2018 at 4:21 AM, Sandhya Agarwal > wrote: > >> Hello, >> >> I am trying to enable the ranger hive plugin 2.0.0, for standalone hive >> metastore 3.0.0. I do not see the link happening, even though both my >> ranger admin and hive metastore services are restarted and running without >> any errors, after enabling the ranger-hive-plugin. Any pointers ? >> >> Thank you, >> Sandhya >> > >
Ranger for standalone hive metastore
Hello, I am trying to enable the ranger hive plugin 2.0.0, for standalone hive metastore 3.0.0. I do not see the link happening, even though both my ranger admin and hive metastore services are restarted and running without any errors, after enabling the ranger-hive-plugin. Any pointers ? Thank you, Sandhya
Re: Standalone metastore
Thank you for the revert. We do have a use case to provide tenant / bucket specific keys for a tenant user session, without having to restart the MetaStore server when a new tenant is provisioned. Nevertheless, let me explore a bit around Sentry / Ranger too. Thank you, Sandhya On Thu, Jul 12, 2018 at 3:51 AM Vihang Karajgaonkar wrote: > AFAIK currently, s3 keys cannot be session specific currently. They are > loaded by the metastore server when it starts and it cannot be modified > without a metastore server restart. In order to do this we will have make > some code changes. I had created HIVE-16913 for this long time back but > never got around working on it later (I can take a relook at it if there > are use-cases in practice which would need this). > > The other workaround would be to have one master key configured at the > server level and restrict user access to urls using Sentry or Ranger. > > > On Wed, Jul 11, 2018 at 2:44 AM, Sandhya Agarwal > wrote: > >> Hello, >> >> We want to leverage standalone metastore for our project, for which I >> want to enable access to multiple S3 buckets, each with its own access key >> and secret key. I am trying to access the metastore operations from a Java >> client using HiveMetastoreClient and using the thrift metastore URI to >> connect to the metastore service. For the database location and table >> location, I am using the S3A file system scheme. I am trying to set the >> access key and secret key for the S3 bucket location in the >> HiveMetastoreClient, but the client side settings are never honoured and I >> cannot figure out a way to do this. One way is to provide the per-bucket >> access key and secret key on the server side in metastore-site.xml. >> However, I want this to be dynamic as I want the metastore to be multi >> tenant enabled and based on the current tenant user, I wish to provide >> these keys through my client. I tried looking through the metastore source >> code, but did not find a way to do this. Can this be done ? >> >> I am using apache-hive-metastore-3.0.0 version. >> >> Thank you, >> >> Sandhya >> >> >
Standalone metastore
Hello, We want to leverage standalone metastore for our project, for which I want to enable access to multiple S3 buckets, each with its own access key and secret key. I am trying to access the metastore operations from a Java client using HiveMetastoreClient and using the thrift metastore URI to connect to the metastore service. For the database location and table location, I am using the S3A file system scheme. I am trying to set the access key and secret key for the S3 bucket location in the HiveMetastoreClient, but the client side settings are never honoured and I cannot figure out a way to do this. One way is to provide the per-bucket access key and secret key on the server side in metastore-site.xml. However, I want this to be dynamic as I want the metastore to be multi tenant enabled and based on the current tenant user, I wish to provide these keys through my client. I tried looking through the metastore source code, but did not find a way to do this. Can this be done ? I am using apache-hive-metastore-3.0.0 version. Thank you, Sandhya
