Daniel We looked into Spark a while ago and based on the Spark architecture at that time, similar to Apache Pig, it seemed authorization at the data/compute level seemed to be the only way to enforce authorization.
However, Spark has evolved a long way and it would be good to revisit the recent additions to Spark and see where we can add security. Can we start looking into the use cases where we can add security in Spark? If your team has already have some requirements, then it would be a good start. Regarding contribution to Ranger, there are lot of options. 1. We are aggressively working on enabling security for all Hadoop components. In Ranger 0.5, we added Solr, YARN, KMS and Kafka (dependent on some 0.8.3 release Kafka). We still have few more, Oozie, Zookpeer, Hive MetaStore, etc. 2. We just started the discussion for Ranger 0.6 release. The Wiki is WIP (https://cwiki.apache.org/confluence/display/RANGER/Release+Themes+-+Ranger +0.6), please suggest anything you want to add. 3. We are also working on making our policy engine extendable. In 0.5, we added ability to add custom (dynamic) policy. This will enable us to go beyond static checks (user, group, IPs, etc) and support policies like time of access, location access, tags, external lookup, etc. Please let us what interests you. Thanks Bosco On 9/3/15, 11:14 PM, "Daniel Schulz" <[email protected]> wrote: >Sure: I'd really like to. Is there still some 1) issues,2) solutions to >some issues, and/or3) an architectonial approachto this? Otherwise: I'm >doing currently some investigation on solving this enterprise-scale. >Maybe there is something our R&D has come up with just recently. >Feel free to contact me for coding and architecture support at anytime. >I'm more than happy to assist you guys. > >> Date: Thu, 3 Sep 2015 14:59:23 -0700 >> Subject: Re: Ranger plugin for Spark coming? >> From: [email protected] >> To: [email protected] >> >> I have seen it being discussed in the community, more offline than in >>dev >> email list. >> >> I think it would make sense to have a Ranger plugin for Spark. @Daniel, >> would you be interested to work on this and contribute the relevant >>code? >> >> On Thu, Sep 3, 2015 at 10:32 AM, Daniel Schulz >><[email protected] >> > wrote: >> >> > Hi, >> > >> > Are there plans, a proposal, or an active development of a Ranger >>plugin >> > for Apache Spark yet? >> > >> > Best regards, Daniel. >
