SparkThriftServer Authorization design

2019-02-15 Thread t4
Goal is to provide ability to deny/allow access to given tables on a per user basis (this is the user connecting via jdbc to spark thrift server. ie with LDAP creds). ie user bob can see table A but not table B. user mary can see table B but not table A. What are folks thoughts on the approach?

Re: Hadoop 3 support

2018-10-16 Thread t4
has anyone got spark jars working with hadoop3.1 that they can share? i am looking to be able to use the latest hadoop-aws fixes from v3.1 -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To

HADOOP-13421 anyone using this with Spark

2018-10-16 Thread t4
https://issues.apache.org/jira/browse/HADOOP-13421 mentions s3a can use s3 v2 api for performance. has anyone been able to use this new hadoop-aws jar with Spark? -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

Re: Spark data quality bug when reading parquet files from hive metastore

2018-08-22 Thread t4
https://issues.apache.org/jira/browse/SPARK-23576 ? -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Apache Airflow (incubator) PMC binding vote needed

2018-08-20 Thread t4
Hi Can any member of Apache Incubator PMC provide a vote for Apache Airflow 1.10 to be released? Thanks https://lists.apache.org/thread.html/fb09a91f1cef4a63df4d5474e2189248aa65a609a6237d8eefcd8eb7@%3Cdev.airflow.apache.org%3E -- Sent from:

Future of SparkSQL - enterprise grade roadmap?

2018-07-14 Thread t4
Are there any plans to make Sparksql have the necessary enterprise grade features to join the ranks of Presto/HiveServer2? Some key things missing from SparkSQL: Role Based Access control authorization (either Sql standard or Apache Ranger) LDAP user/group filters Data masking plugin DoAs