Re: How to apply ranger policies on Spark

Hi Joyan, Spark uses its own metastore. Using Ranger you need to use the Hive Metastore. For this you need to point to Hive Metastore and use HiveContext in your Spark Code. Br, Dennis Von meinem iPhone gesendet > Am 23.11.2020 um 19:04 schrieb joyan sil : > >  > Hi, > > We have ranger

Building High-performance Lake for Spark using OSS, Hudi, Alluxio

Hi Spark Users, Check out this blog on Building High-performance Data Lake using Apache Hudi, Spark and Alluxio at T3Go Cheers - Bin Fan

Re: How to submit a job via REST API?

Hi Yang, I am using Livy Server for submitting jobs. Br, Dennis Von meinem iPhone gesendet > Am 24.11.2020 um 03:34 schrieb Zhou Yang : > >  > Dear experts, > > I found a convenient way to submit job via Rest API at >

how to manage HBase connections in Executors of Spark Streaming ?

Hi, Does any best practices about how to manage Hbase connections with kerberos authentication in Spark Streaming (YARN) environment? Want to now how executors manage the HBase connections,how to create them, close them and refresh Kerberos expires. Thanks.

How to submit a job via REST API?

Dear experts, I found a convenient way to submit job via Rest API at https://gist.github.com/arturmkrtchyan/5d8559b2911ac951d34a#file-submit_job-sh. But I did not know whether can I append `—conf` parameter like what I did in spark-submit. Can someone can help me with this issue? Regards, Yang

How to apply ranger policies on Spark

Hi, We have ranger policies defined on the hive table and authorization works as expected when we use hive cli and beeline. But when we access those hive tables using spark-shell or spark-submit it does not work. Any suggestions to make Ranger work with Spark? Regards Joyan

Re: spark cassandra questiom

Yes, it should be good to use Spark for this use case in my opinion. You can look into using the Cassandra Spark connector for persisting your updated data into Cassandra. Cheers, Sonal Nube Technologies Join me at Data Con LA Oct 23 | Big Data Conference Europe. Nov 24