Hi Tristan- I completely agree with you that it is straight forward when you run notebook themselves. But with little complications in getting the jupyter hub to work in kerborized environment, i would say jupyter hub works awesome (Kudos goes to jupyter engineers Min and Carol to name few), after the setup it reduces the work on the admin side and new users can easily get started. Jupyter hub takes care of all the ticket and negotiations that happen while talking.
Cheers, Gerg On Thursday, October 13, 2016 at 3:38:47 AM UTC-4, Tristan Zajonc wrote: > > Gerg, > > If users run the notebook themselves, as they would on their laptop, this > should be relatively straightforward. Users can just run kinit and their > keytabs should get picked up by most Hadoop tools like Spark. In a > multi-user or containerized setup it may be more complicated. > > Tristan > > On Tuesday, October 4, 2016 at 2:41:44 AM UTC-7, Hani1814 wrote: >> >> Hi Greg, >> >> Is it possible to share the set-up? >> >> On Monday, 27 June 2016 12:46:26 UTC+1, Gerg wrote: >>> >>> Hi Deepak- >>> >>> We are not there yet, but information that min provided will be helpful >>> in setting up HUB. >>> We will update you once we have something >>> >>> Thanks >>> >>> On Friday, June 24, 2016 at 6:38:25 AM UTC-4, Deepak Subhramanian wrote: >>>> >>>> We are also trying to get Jupyter working with Kerberized cluster. Did >>>> anyone managed to get it working ? >>>> >>>> On Thursday, 16 June 2016 05:58:17 UTC-7, Min RK wrote: >>>>> >>>>> I’m afraid that I don’t know too much about Kerberos, but it should be >>>>> doable with the right custom Authenticator and/or Spawner. The JHU >>>>> setup <https://git.acm.jhu.edu/jhuacm/jhuacm_jupyterhub/tree/master> >>>>> Carol linked to is probably the place to start. If you have IT folks who >>>>> know the moving parts for your Kerberos setup, then we should be able to >>>>> work it out. Here are the pieces of information you need: >>>>> >>>>> 1. How do users authenticate? From the looks of the JHU setup, PAM >>>>> authentication will work. >>>>> 2. How do notebook server processes need to be started? In the JHU >>>>> setup, they use sshpass to localhost to start a session for each >>>>> user. >>>>> 3. What information needs to be passed to the notebook server >>>>> and/or kernel processes for a given user? Sometimes environment >>>>> variables >>>>> and/or certificate files need to be loaded. The Spawner can do this, >>>>> as >>>>> long as you tell it what it needs. >>>>> >>>>> With a bit more details, we should be able to have you up and running. >>>>> >>>>> -MinRK >>>>> >>>>> >>>>> >>>>> On Thu, Jun 16, 2016 at 2:31 PM, Gerg <george...@gmail.com> wrote: >>>>> >>>>>> Carol- >>>>>> >>>>>> I really appreciate that Carol. >>>>>> Our data scientists are really waiting on getting there favorite >>>>>> notebook work with our new kerborised setup. Thanks Carol >>>>>> >>>>>> Gerg >>>>>> >>>>>> >>>>>> On Wednesday, June 15, 2016 at 3:14:46 PM UTC-4, Carol Willing wrote: >>>>>>> >>>>>>> Hi Greg, >>>>>>> >>>>>>> Min, the JupyterHub lead dev, is traveling this week. I'm copying >>>>>>> him on this message since he may have more insights on >>>>>>> JupyterHub/Jupyter >>>>>>> with Kerberos. >>>>>>> >>>>>>> Carol >>>>>>> >>>>>>> --- >>>>>>> Carol Willing >>>>>>> >>>>>>> Research Software Engineer >>>>>>> Project Jupyter @ Cal Poly SLO >>>>>>> >>>>>>> Director, Python Software Foundation >>>>>>> >>>>>>> On Jun 15 2016, at 1:22 pm, Gerg <george...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi Carol- >>>>>>>> >>>>>>>> Thanks for your reply. >>>>>>>> It looks like there is no much information for jupyter to work with >>>>>>>> kerberos. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Gerg >>>>>>>> >>>>>>>> On Monday, June 13, 2016 at 10:13:05 AM UTC-4, Carol Willing wrote: >>>>>>>> >>>>>>>> Hello Gerg, >>>>>>>> >>>>>>>> I'm hoping that others will jump in with information about using >>>>>>>> Kerberos. >>>>>>>> >>>>>>>> A thread from earlier this year on Kerberos: >>>>>>>> >>>>>>>> https://groups.google.com/forum/#!searchin/jupyter/kerberos/jupyter/Vfzpx0QstLk/g69c0BpXBwAJ >>>>>>>> >>>>>>>> <https://groups.google.com/forum/#!searchin/jupyter/kerberos/jupyter/Vfzpx0QstLk/g69c0BpXBwAJ&r=anVweXRlckBnb29nbGVncm91cHMuY29t> >>>>>>>> >>>>>>>> Within that thread (Apr 27 message from Mark Rosen), there is a >>>>>>>> configuration that is being used at JHU. >>>>>>>> >>>>>>>> Here's also a message from Min that talks about a custom >>>>>>>> authenticator that would use Authenticator.pre_spawn_start and >>>>>>>> Authenticator.post_spawn_stop that should be helpful as well: >>>>>>>> http://permalink.gmane.org/gmane.comp.programming.jupyter/2545 >>>>>>>> <http://permalink.gmane.org/gmane.comp.programming.jupyter/2545&r=anVweXRlckBnb29nbGVncm91cHMuY29t> >>>>>>>> >>>>>>>> Carol >>>>>>>> >>>>>>>> --- >>>>>>>> Carol Willing >>>>>>>> >>>>>>>> Research Software Engineer >>>>>>>> Project Jupyter @ Cal Poly SLO >>>>>>>> >>>>>>>> Director, Python Software Foundation >>>>>>>> >>>>>>>> On Jun 13 2016, at 5:33 am, Gerg <george...@gmail.com> wrote: >>>>>>>> >>>>>>>> Hello Guys- >>>>>>>> >>>>>>>> We are really happy with jupyter for our data scientists. But >>>>>>>> recently we kerberized our cluster (HDP 2.4.0), and we couldn't find >>>>>>>> any >>>>>>>> article or threads in this group on how to setup jupyter for a >>>>>>>> kerberized >>>>>>>> cluster. >>>>>>>> Basically we will be running pyspark jobs(yarn mode) from our >>>>>>>> notebook. is this ever possible with Jupyter on a kerberized cluster? >>>>>>>> I would really appreciate if someone can help me with the steps in >>>>>>>> configuring jupyter for our kerberized cluster or direct me to any >>>>>>>> article >>>>>>>> which describes that. >>>>>>>> >>>>>>>> Thanks in advance for any help. >>>>>>>> >>>>>>>> Gerg >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Project Jupyter" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to jupyter+u...@googlegroups.com. >>>>>>>> To post to this group, send email to jup...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/jupyter/e641ffcb-36ed-4bfe-b694-a4ec3b044be8%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/jupyter/e641ffcb-36ed-4bfe-b694-a4ec3b044be8%40googlegroups.com?utm_medium=email&utm_source=footer&r=anVweXRlckBnb29nbGVncm91cHMuY29t> >>>>>>>> . >>>>>>>> For more options, visit https://groups.google.com/d/optout >>>>>>>> <https://groups.google.com/d/optout&r=anVweXRlckBnb29nbGVncm91cHMuY29t> >>>>>>>> . >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Project Jupyter" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to jupyter+u...@googlegroups.com. >>>>>>>> To post to this group, send email to jup...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/jupyter/bbd5e7c8-c270-4191-9803-67a041434a4a%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/jupyter/bbd5e7c8-c270-4191-9803-67a041434a4a%40googlegroups.com?utm_medium=email&utm_source=footer&r=anVweXRlckBnb29nbGVncm91cHMuY29t> >>>>>>>> . >>>>>>>> For more options, visit https://groups.google.com/d/optout >>>>>>>> <https://groups.google.com/d/optout&r=anVweXRlckBnb29nbGVncm91cHMuY29t> >>>>>>>> . >>>>>>>> >>>>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "Project Jupyter" group. To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+unsubscr...@googlegroups.com. To post to this group, send email to jupyter@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/c68b4644-9902-4604-ab27-1c2936834c1d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
