Re: [jupyter] JupyterHub on a cluster with local spawning and batch spawning

[Michael Milligan]

Hi Brian -

It is possible, and it sounds like you already mostly have the solution!
The example config at the end of the batchspawner README should be roughly
what you want, except that you will use the SudoSpawner instead of
LocalProcessSpawner. Specifically, you need to install all of the spawners
you are using alongside Jupyterhub (e.g. pip install wrapspawner,
batchspawner, and sudospawner into the same virtualenv as jupyterhub) --
the ProfilesSpawner configuration will tell Jupyterhub which spawner to use
in response to the user's menu selection.

Sorry about the confusing state of the documentation about how to do this.
Batchspawner and wrapspawner were originally distributed in the same
repository, and when we split them apart it sounds like we may not have
included enough context on one side or the other. We're happy to get
suggestions about how to improve the docs if you'd like to give us some
feedback when you get a chance.

Regards,
Michael

On Thu, Mar 15, 2018 at 4:58 PM, Brian Fulton-Howard 
wrote:

> As I understand it, I need to use some combination of wrapspawner,
> localspawner, sudospawner and batchspawner. I want to run jupyterhub as an
> unprivileged user, but the admins are happy to edit the sudoers file to
> make it work. What I'd like to do is give users in my lab a choice to start
> on the LSF cluster (batchspawner) or locally (localspawner). Wrapspawner
> would provide the interface to make a selection.
>
> Is this possible, and how would I go about doing it? I looked in the
> documentation for the various spawners and didn't find any detailed
> information.
>
> The cluster supports PAM, so I don't need any special authenticator.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscr...@googlegroups.com.
> To post to this group, send email to jupyter@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jupyter/a3f21888-0495-4fe0-b598-ca7e226bd61f%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAPuy8gqaTXd9RtQ8hPD4F%2BQdV7ftstfqw0GG9ZN4h6_OrAk9CA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[jupyter] JupyterHub on a cluster with local spawning and batch spawning

[Brian Fulton-Howard]

As I understand it, I need to use some combination of wrapspawner, 
localspawner, sudospawner and batchspawner. I want to run jupyterhub as an 
unprivileged user, but the admins are happy to edit the sudoers file to 
make it work. What I'd like to do is give users in my lab a choice to start 
on the LSF cluster (batchspawner) or locally (localspawner). Wrapspawner 
would provide the interface to make a selection.

Is this possible, and how would I go about doing it? I looked in the 
documentation for the various spawners and didn't find any detailed 
information.

The cluster supports PAM, so I don't need any special authenticator.

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/a3f21888-0495-4fe0-b598-ca7e226bd61f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] Good solution for sharing largish data sets?

[Jason Grout]

There was a guest post on the Jupyter blog the other day about Quilt, which
may be interesting for you to look at:
https://blog.jupyter.org/reproducible-data-dependencies-for-python-guest-post-d0f68293a99

Jason


On Thu, Mar 15, 2018 at 12:34 PM Matthew Turk  wrote:

> Hi!
>
> Great question.  My name's Matt Turk and along with some other folks
> (lurking?) on this list I work on a project called Whole Tale.  We
> just had an overview paper published (gold OA) at
> https://doi.org/10.1016/j.future.2017.12.029 that gives some
> architectural information, but the gist is that we're trying to solve
> that exact problem.  Our website isn't the best, and we're not
> confident of a stable, running instance until early summer (I bet if
> you logged in you could find ways to break it or prickly bits in the
> UI), but you can find a bit more at wholetale.org and
> github.com/whole-tale .  You could even launch your own instance,
> should you want to.
>
> The long and the short of it is that we run docker containers (not
> only Jupyter, but it's currently used as one of the defaults) with
> computational environments and "inject" data through a handcrafted
> FUSE fs.
>
> The ultimate location of the data is not important (can be both local
> or remote), as long as you provide a valid uri containing both
> location and transfer protocol (e.g. 'http://example.com/file',
> 'globus:/endpoint/foo/bar'). There's a couple of additional attributes
> you need to provide (size & name, although over HTTP sometimes we can
> get these). We keep track of all of those using an external db
> (MongoDB via Girder) which is subsequently used by FUSE to resolve
> OS-level IO calls into appropriate requests for data. For example,
> when you open() a file that's registered as a 'http://' url, it will
> (invisibly) locally cache it and present it as though it were local.
>
> Kacper Kowalik, our software architect, recently gave a presentation
> on it that you can see here that might be of interest:
> http://use.yt/upload/c8236396 .
>
> I'd be happy to share more here or offline, too, but this is something
> we're working on pretty hard and while we have a ways to go --
> especially in smoothing things out from a UI/UX perspective and
> getting stability of the platform, we're working hard on it and really
> want to engage much more deeply with folks throughout the community.
>
> -Matt, on behalf of the Whole Tale team
>
> On Thu, Mar 15, 2018 at 1:06 PM, 'Aaron Watters' via Project Jupyter
>  wrote:
> > Hi folks,
> >
> > I'm interested in techniques for sharing data in scientific workflows.
> > Tools like git/github and docker/repo2docker are great for sharing
> > computational
> > environments and moderate sized data, but not good for sharing (say)
> > hundreds of gigabytes of data.  What do people do?
> >
> > I have in mind something like this: a scientist on a good network
> > spins up a jupyter server in a Docker container containing a workflow
> > using github and repo2docker.  In the container s/he provides some
> > authorization credentials and data for the workflow appears in the
> > container if the credentials are valid, maybe with read/write access
> > of some sort if the credentials are really good.
> >
> > If we are interested in provided publically accessible data in read
> > only mode we could just dump the data to a web server anywhere
> > and pull it down using HTTP,
> >
> > I don't know the right way to do this if we want to have limited access
> > to the data and sometime provide the ability to write the data.
> >
> > I'm also interested in the case where the scientist is remote --
> > ie, certain people are allowed to use our compute cluster possible
> > with data they have locally or with other data out there somewhere...
> >
> > Any and all thoughts or pointers appreciated.  Thanks!
> > Sorry if the question is silly or too vague.
> >
> >-- Aaron Watters
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Project Jupyter" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to jupyter+unsubscr...@googlegroups.com.
> > To post to this group, send email to jupyter@googlegroups.com.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/jupyter/003e34fa-a547-40c5-a617-8997ee5db326%40googlegroups.com
> .
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscr...@googlegroups.com.
> To post to this group, send email to jupyter@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/CALO3%3D5HfjR69tOPB37pkCQo4yiWftsGVfVvzAOhyJnaDn6a3cQ%40mail.gmail.com
> .
> For more options, visit 

Re: [jupyter] Adding additional handlers to jupyterhub

[Dan Davis]

I am implementing a MultiAuthenticator for a site that will use Jupyter to 
host training materials.   Since adoption is desired, social authentication 
will be 
desired, and so we will have more than one way to authenticate socially.

That said, I think I've figured out what I did not understand - traitlets.  
 I somehow took traitlets.config.Application to be the 
tornado.web.Application, and that confused thoroughly.
Now it is clear - the web application is initialized rather late, and since 
the init_handlers method calls 
h.extend(self.authenticator.get_handlers(self)), I simply return additional 
handlers that way.

>From the structure of the code, it is pretty clear that each "handler" in 
that list should be a tuple of pattern and handler, and this is confirmed 
by the implementation of jupyterhub.auth.Authenticator.

I've even checked that this basic structure doesn't change with jupyterhub 
0.9.0.dev, which maybe I should be using.

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/6a7dcf06-ac67-4b96-bb70-d7eb841ffc2a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jupyter] Good solution for sharing largish data sets?

['Aaron Watters' via Project Jupyter]

Hi folks,

I'm interested in techniques for sharing data in scientific workflows.
Tools like git/github and docker/repo2docker are great for sharing 
computational
environments and moderate sized data, but not good for sharing (say)
hundreds of gigabytes of data.  What do people do?

I have in mind something like this: a scientist on a good network
spins up a jupyter server in a Docker container containing a workflow
using github and repo2docker.  In the container s/he provides some
authorization credentials and data for the workflow appears in the
container if the credentials are valid, maybe with read/write access
of some sort if the credentials are really good.

If we are interested in provided publically accessible data in read
only mode we could just dump the data to a web server anywhere
and pull it down using HTTP,

I don't know the right way to do this if we want to have limited access
to the data and sometime provide the ability to write the data.

I'm also interested in the case where the scientist is remote --
ie, certain people are allowed to use our compute cluster possible
with data they have locally or with other data out there somewhere...

Any and all thoughts or pointers appreciated.  Thanks!
Sorry if the question is silly or too vague.

   -- Aaron Watters

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/003e34fa-a547-40c5-a617-8997ee5db326%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] how can i elect the browser used by jupyter notebook in linux?

[Thomas Kluyver]

You need to set it to one of the names recognised by the webbrowser module:
https://docs.python.org/3/library/webbrowser.html

You can also use the BROWSER environment variable to set the default.

On 15 March 2018 at 08:48, 唐彬  wrote:

> I have created the config file and add the command
>
>
> c.NotebookApp.browser = '/usr/bin/google-chrome-stable'
>
> and removed the '#' in the beginning of this line.
> But when i launched notebook, what i got was
>
> No web browser found: could not locate runnable browser.
>
> How can i fix this problem?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscr...@googlegroups.com.
> To post to this group, send email to jupyter@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jupyter/489390b6-f270-45b6-8f0a-17183799b418%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAOvn4qjhvzhCn%2Bh454_SS-hKfU%2BFQN7Ut%2B4qK6fFt-JitLu7xw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] Jupyter 4.4.0 tables aren't rendering on Gist

[Thomas Kluyver]

Some content is not rendered on Github for security reasons (so it can't
steal your Github login). I don't know why tables would be affected,
though. Do you have something that displays interactive tables for pandas
output?

On 15 March 2018 at 11:26, Graham Anderson  wrote:

> As of Jupyter 4.4.0, my pandas are not rendering--as a Gist
> The same tables render fine on my local notebook.
>
> Is this a known issue--perhaps the new rendering engine is using
> javascript? To my knowledge, Gist renders html-only.
> Is there a workaround?
>
> Thanks
> G
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscr...@googlegroups.com.
> To post to this group, send email to jupyter@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/jupyter/720e13e6-1596-4d64-a0c7-428963807008%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAOvn4qitf%3DNccFC%3DFg8ajc%2BMjyTT4AB0-1_81_zS4g4ST1m3Wg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[jupyter] Jupyter 4.4.0 tables aren't rendering on Gist

[Graham Anderson]

As of Jupyter 4.4.0, my pandas are not rendering--as a Gist
The same tables render fine on my local notebook.

Is this a known issue--perhaps the new rendering engine is using 
javascript? To my knowledge, Gist renders html-only.
Is there a workaround? 

Thanks
G

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/720e13e6-1596-4d64-a0c7-428963807008%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.