Are you using Spark standalone, YARN, or Mesos? I'm having issues with Zeppelin in a Docker container submitting jobs to Mesos. Zeppelin works fine with Mesos when not in a Docker container.
Thanks! > On Apr 1, 2015, at 8:17 PM, Alex B. <[email protected]> wrote: > > Thank you for keeping us posted! > > We have recently successfully used this kind of approach: run a separate > Zeppelin instances per-user in Docker containers and have a single Nginx that > does reverse-proxy and url re-write (to different ports on the same host > machine) + a cookie-based auth. > > Alex. > >> On Thu, Apr 2, 2015 at 4:31 AM, RJ Nowling <[email protected]> wrote: >> Wanted to follow up again. The 2 docker container approach is proving to be >> a bit unstable and I'm running into problems integration with Mesos. >> >> I'm going to try a different approach using iptables: >> http://unix.stackexchange.com/questions/111906/allowing-users-to-access-certain-ports-on-server >> >> I'll try doing the following: >> * Run a zeppelin instance under each user on separate ports >> * Limit the access to the ports to that user >> * Ask users to use SSH for port forwarding >> >>> On Tue, Mar 31, 2015 at 11:43 AM, RJ Nowling <[email protected]> wrote: >>> Hi all, >>> >>> I was able to get nginx working for basic authentication. I used docker >>> containers for private networking so that the zeppelin ports aren't open to >>> the public. >>> >>> https://github.com/rnowling/zeppelin-authentication >>> >>> The problem right now is that nginx requires authenticating twice -- for >>> the two ports, I'm assuming. >>> >>> If anyone has ideas for how to fix that, please let me know! >>> >>> RJ >>> >>>> On Mon, Mar 30, 2015 at 9:03 AM, Ram Venkatesh >>>> <[email protected]> wrote: >>>> I agree - have also been thinking about the same problem in the context of >>>> running zeppelin against a secure kerberized hadoop cluster. >>>> >>>> There are a couple concerns here: >>>> 1. authentication, both for the zeppelin server and for interpreters that >>>> need it >>>> 2. isolating the code for different users from each other - for example by >>>> running an interpreter for each user >>>> 3. sharing interpreters across note evaluations - this is likely an issue >>>> even without multiple users but multiple tabs for the same user. It is >>>> already taken care of by the interpreter scheduler (by returning a FIFO or >>>> parallel scheduler with the specified max concurrency option), is this >>>> correct? >>>> >>>> Thanks! >>>> Ram >>>> >>>>> On Mar 30, 2015, at 6:35 AM, RJ Nowling <[email protected]> wrote: >>>>> >>>>> Since this may be an oft-requested feature, I'll go ahead and create a >>>>> JIRA to document the interest. >>>>> >>>>>> On Mon, Mar 30, 2015 at 8:33 AM, Corneau Damien <[email protected]> >>>>>> wrote: >>>>>> Actually the branch wasn't merged in the end >>>>>> >>>>>>> On Mon, Mar 30, 2015 at 10:31 PM, RJ Nowling <[email protected]> wrote: >>>>>>> Is there documentation for the simple authentication? >>>>>>> >>>>>>> We'll be creating separate unix users for each zeppelin instance with >>>>>>> appropriate file system permissions. >>>>>>> >>>>>>>> On Fri, Mar 27, 2015 at 9:39 PM, Corneau Damien >>>>>>>> <[email protected]> wrote: >>>>>>>> If I remember well, we merged some kind of simple authentication in >>>>>>>> Zeppelin. But it should be more about accessing the instance than >>>>>>>> having multiple users. One way could be launching multiple zeppelin >>>>>>>> instances, I think Kevin is doing something similar. >>>>>>>> >>>>>>>> One thing to realize however is that it doesnt create separation at >>>>>>>> the data layer >>>>>>>> >>>>>>>>> On Mar 28, 2015 1:53 AM, "RJ Nowling" <[email protected]> wrote: >>>>>>>>> I'll report back if I'm successful with the approach I've mentioned. >>>>>>>>> >>>>>>>>>> On Fri, Mar 27, 2015 at 11:51 AM, RJ Nowling <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> A search of the user list suggests that Zeppelin can support a >>>>>>>>>> separate SparkContext for each notebook but the user who tried it >>>>>>>>>> ran into issues. >>>>>>>>>> >>>>>>>>>>> On Fri, Mar 27, 2015 at 11:32 AM, Denny Lee <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> I haven't done this myself but was recently starting to do research >>>>>>>>>>> on doing this. Perhaps this blog post may be of help: >>>>>>>>>>> http://nginx.com/blog/websocket-nginx/ ? >>>>>>>>>>> >>>>>>>>>>>> On Fri, Mar 27, 2015 at 9:24 AM Silvio Fiorito >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>> I haven’t tried this myself yet but something I’ve been thinking >>>>>>>>>>>> as well. Will the nginx reverse proxy support web sockets as well? >>>>>>>>>>>> >>>>>>>>>>>> Ideally we’d have isolated SparkContexts so users aren’t trampling >>>>>>>>>>>> over each other. Honestly I think it’d be good to have the option >>>>>>>>>>>> of starting a new SparkContext per notebook as well or using the >>>>>>>>>>>> model Databricks has where you “attach” a notebook to a cluster. >>>>>>>>>>>> >>>>>>>>>>>> From: RJ Nowling >>>>>>>>>>>> Reply-To: "[email protected]" >>>>>>>>>>>> Date: Friday, March 27, 2015 at 12:19 PM >>>>>>>>>>>> To: "[email protected]" >>>>>>>>>>>> Subject: Multi-user approach >>>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> I'm looking into ways to support multiple users with Zeppelin. I >>>>>>>>>>>> want to provide isolation between users. >>>>>>>>>>>> >>>>>>>>>>>> I'm considering the following approach: >>>>>>>>>>>> * Run Zeppelin under each user's account with its own set of ports >>>>>>>>>>>> * Use nginx as a reverse proxy for providing authentication >>>>>>>>>>>> >>>>>>>>>>>> Has anyone done anything similar? Any better alternatives? >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> RJ > > > > -- > -- > Kind regards, > Alexander. >
