Re: [jupyter] JVM Magics

2019-03-08 Thread Pete Blois 
>From a tooling perspective magics effectively declare a new superset of the
language, which makes it more difficult to reuse existing tools for
functionality such as linting, formatting and static analysis.

I'm curious how magics mesh with proposals like
https://github.com/jupyter/enhancement-proposals/pull/26.

On Wed, Feb 27, 2019 at 1:50 PM Doug Blank  wrote:

> Ryan,
>
> Very nice!
>
> We also have a set of magics that work across languages. It would be great
> to have a group to try to standardize these.
>
> We implemented cross-language magics in metakernel [1], and there are some
> lessons learned that you might want to consider:
>
> 1. We allow languages to define magic prefixes and suffixes for magic,
> shell, and help
> 2. shell is a provided magic
> 3. Some languages allow "?" as a regular character, and so we allow the
> pattern "?object" to trigger help in those languages (eg, Scheme)
> 4. We provide some additional interlanguage support (like function regex)
> to support additional functionality (like the %%pipe magic [2]) for
> chaining function calls
> 5. There is also language-specific start-up code, including additional
> magics
> 6. A general way for one language to call another (%kernel, %kx)
> 7. A general way to use the ipython parallel/cluster tools so that all
> languages can run "in parallel" (%parallel, %px)
> 8. We added some additional magics that are very useful (%download, %get,
> %put, %%dot, etc)
> 9. A general way of using magics (must be at top, can be stacked)
> 10. The amazing %%% triple percent: makes the cell magic "sticky" so it
> applies to the rest of the cells (until you turn it off)
>
> There is more, and probably needs more documentation. Feel free to contact
> me for more information or if you have additional comments. Metakernel was
> developed by me and Steven Sylvester and has had many contributions from
> many people over the years.
>
> -Doug
>
> [1] - https://github.com/calysto/metakernel
> [2] -
> https://github.com/Calysto/metakernel/blob/master/metakernel/magics/README.md#pipe
>
> On Sun, Feb 24, 2019 at 3:04 PM 'Ryan Blue' via Project Jupyter <
> jupyter@googlegroups.com> wrote:
>
>> Thanks for taking a look at this, everyone! I look forward to your
>> feedback on this prototype.
>>
>> It would be great to get this (or something like it) into the Jupyter
>> community as a recommendation so that we can build magics that work in any
>> Kernel. I have a SQL magic implementation that uses Spark that could be
>> packaged this way. It is also a good way to implement a set of built-ins
>> across kernels, like %%timeit.
>>
>> On Thu, Feb 21, 2019 at 5:23 PM Brian Granger 
>> wrote:
>>
>>> Great news, thanks for sharing!
>>>
>>> On Thu, Feb 21, 2019 at 5:20 PM Kyle Kelley  wrote:
>>>
 Hey all,

 A couple weeks ago a bunch of Scala kernel maintainers came together
 during the Notebook Enterprise Summit .
 One of the topics of discussion was making consistent magics across the
 various Scala & JVM kernels. Notes and proposals are here:
 https://github.com/nteract/nes/tree/master/jvm-kernel-magics

 Ryan Blue created jvm-magics as a result of this:
 https://github.com/rdblue/jvm-magics

 Please check it out if you're interested and post issues. When ready,
 we'd love to see this land in the jupyter org to support similarly to
 jvm-repr.

 --
 Kyle Kelley (@rgbkrk ; lambdaops.com)

 --
 You received this message because you are subscribed to the Google
 Groups "Project Jupyter" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to jupyter+unsubscr...@googlegroups.com.
 To post to this group, send email to jupyter@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/jupyter/CA%2BtbMaUOfo5c_ubJzEcA55TYwmTziS4OTs9VT38ndULp-iQZdA%40mail.gmail.com
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>> --
>>> Brian E. Granger
>>> Associate Professor of Physics and Data Science
>>> Cal Poly State University, San Luis Obispo
>>> @ellisonbg on Twitter and GitHub
>>> bgran...@calpoly.edu and elliso...@gmail.com
>>>
>>
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Netflix
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Project Jupyter" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jupyter+unsubscr...@googlegroups.com.
>> To post to this group, send email to jupyter@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jupyter/CAO4re1nLq8pY5LwvPeP9mTBr0s_t4oDJ5mQW06vv3WLz5Ur68Q%40mail.gmail.com
>>

Re: [jupyter] JVM Magics

2019-02-27 Thread Doug Blank 
Ryan,

Very nice!

We also have a set of magics that work across languages. It would be great
to have a group to try to standardize these.

We implemented cross-language magics in metakernel [1], and there are some
lessons learned that you might want to consider:

1. We allow languages to define magic prefixes and suffixes for magic,
shell, and help
2. shell is a provided magic
3. Some languages allow "?" as a regular character, and so we allow the
pattern "?object" to trigger help in those languages (eg, Scheme)
4. We provide some additional interlanguage support (like function regex)
to support additional functionality (like the %%pipe magic [2]) for
chaining function calls
5. There is also language-specific start-up code, including additional
magics
6. A general way for one language to call another (%kernel, %kx)
7. A general way to use the ipython parallel/cluster tools so that all
languages can run "in parallel" (%parallel, %px)
8. We added some additional magics that are very useful (%download, %get,
%put, %%dot, etc)
9. A general way of using magics (must be at top, can be stacked)
10. The amazing %%% triple percent: makes the cell magic "sticky" so it
applies to the rest of the cells (until you turn it off)

There is more, and probably needs more documentation. Feel free to contact
me for more information or if you have additional comments. Metakernel was
developed by me and Steven Sylvester and has had many contributions from
many people over the years.

-Doug

[1] - https://github.com/calysto/metakernel
[2] -
https://github.com/Calysto/metakernel/blob/master/metakernel/magics/README.md#pipe

On Sun, Feb 24, 2019 at 3:04 PM 'Ryan Blue' via Project Jupyter <
jupyter@googlegroups.com> wrote:

> Thanks for taking a look at this, everyone! I look forward to your
> feedback on this prototype.
>
> It would be great to get this (or something like it) into the Jupyter
> community as a recommendation so that we can build magics that work in any
> Kernel. I have a SQL magic implementation that uses Spark that could be
> packaged this way. It is also a good way to implement a set of built-ins
> across kernels, like %%timeit.
>
> On Thu, Feb 21, 2019 at 5:23 PM Brian Granger  wrote:
>
>> Great news, thanks for sharing!
>>
>> On Thu, Feb 21, 2019 at 5:20 PM Kyle Kelley  wrote:
>>
>>> Hey all,
>>>
>>> A couple weeks ago a bunch of Scala kernel maintainers came together
>>> during the Notebook Enterprise Summit .
>>> One of the topics of discussion was making consistent magics across the
>>> various Scala & JVM kernels. Notes and proposals are here:
>>> https://github.com/nteract/nes/tree/master/jvm-kernel-magics
>>>
>>> Ryan Blue created jvm-magics as a result of this:
>>> https://github.com/rdblue/jvm-magics
>>>
>>> Please check it out if you're interested and post issues. When ready,
>>> we'd love to see this land in the jupyter org to support similarly to
>>> jvm-repr.
>>>
>>> --
>>> Kyle Kelley (@rgbkrk ; lambdaops.com)
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Project Jupyter" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to jupyter+unsubscr...@googlegroups.com.
>>> To post to this group, send email to jupyter@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/jupyter/CA%2BtbMaUOfo5c_ubJzEcA55TYwmTziS4OTs9VT38ndULp-iQZdA%40mail.gmail.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>> --
>> Brian E. Granger
>> Associate Professor of Physics and Data Science
>> Cal Poly State University, San Luis Obispo
>> @ellisonbg on Twitter and GitHub
>> bgran...@calpoly.edu and elliso...@gmail.com
>>
>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscr...@googlegroups.com.
> To post to this group, send email to jupyter@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/CAO4re1nLq8pY5LwvPeP9mTBr0s_t4oDJ5mQW06vv3WLz5Ur68Q%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web

Re: [jupyter] JVM Magics

2019-02-24 Thread 'Ryan Blue' via Project Jupyter 
Thanks for taking a look at this, everyone! I look forward to your feedback
on this prototype.

It would be great to get this (or something like it) into the Jupyter
community as a recommendation so that we can build magics that work in any
Kernel. I have a SQL magic implementation that uses Spark that could be
packaged this way. It is also a good way to implement a set of built-ins
across kernels, like %%timeit.

On Thu, Feb 21, 2019 at 5:23 PM Brian Granger  wrote:

> Great news, thanks for sharing!
>
> On Thu, Feb 21, 2019 at 5:20 PM Kyle Kelley  wrote:
>
>> Hey all,
>>
>> A couple weeks ago a bunch of Scala kernel maintainers came together
>> during the Notebook Enterprise Summit .
>> One of the topics of discussion was making consistent magics across the
>> various Scala & JVM kernels. Notes and proposals are here:
>> https://github.com/nteract/nes/tree/master/jvm-kernel-magics
>>
>> Ryan Blue created jvm-magics as a result of this:
>> https://github.com/rdblue/jvm-magics
>>
>> Please check it out if you're interested and post issues. When ready,
>> we'd love to see this land in the jupyter org to support similarly to
>> jvm-repr.
>>
>> --
>> Kyle Kelley (@rgbkrk ; lambdaops.com)
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Project Jupyter" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to jupyter+unsubscr...@googlegroups.com.
>> To post to this group, send email to jupyter@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jupyter/CA%2BtbMaUOfo5c_ubJzEcA55TYwmTziS4OTs9VT38ndULp-iQZdA%40mail.gmail.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
> Brian E. Granger
> Associate Professor of Physics and Data Science
> Cal Poly State University, San Luis Obispo
> @ellisonbg on Twitter and GitHub
> bgran...@calpoly.edu and elliso...@gmail.com
>


-- 
Ryan Blue
Software Engineer
Netflix

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAO4re1nLq8pY5LwvPeP9mTBr0s_t4oDJ5mQW06vv3WLz5Ur68Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] JVM Magics

2019-02-21 Thread Brian Granger 
Great news, thanks for sharing!

On Thu, Feb 21, 2019 at 5:20 PM Kyle Kelley  wrote:

> Hey all,
>
> A couple weeks ago a bunch of Scala kernel maintainers came together
> during the Notebook Enterprise Summit .
> One of the topics of discussion was making consistent magics across the
> various Scala & JVM kernels. Notes and proposals are here:
> https://github.com/nteract/nes/tree/master/jvm-kernel-magics
>
> Ryan Blue created jvm-magics as a result of this:
> https://github.com/rdblue/jvm-magics
>
> Please check it out if you're interested and post issues. When ready, we'd
> love to see this land in the jupyter org to support similarly to jvm-repr.
>
> --
> Kyle Kelley (@rgbkrk ; lambdaops.com)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscr...@googlegroups.com.
> To post to this group, send email to jupyter@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/CA%2BtbMaUOfo5c_ubJzEcA55TYwmTziS4OTs9VT38ndULp-iQZdA%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgran...@calpoly.edu and elliso...@gmail.com

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAH4pYpQk5ypjHsNUpscQG_464Yc1AURCh3VQ4Y0bD22coFp%2B%2BA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[jupyter] JVM Magics

2019-02-21 Thread Kyle Kelley 
Hey all,

A couple weeks ago a bunch of Scala kernel maintainers came together during
the Notebook Enterprise Summit . One of the
topics of discussion was making consistent magics across the various Scala
& JVM kernels. Notes and proposals are here:
https://github.com/nteract/nes/tree/master/jvm-kernel-magics

Ryan Blue created jvm-magics as a result of this:
https://github.com/rdblue/jvm-magics

Please check it out if you're interested and post issues. When ready, we'd
love to see this land in the jupyter org to support similarly to jvm-repr.

-- 
Kyle Kelley (@rgbkrk ; lambdaops.com)

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CA%2BtbMaUOfo5c_ubJzEcA55TYwmTziS4OTs9VT38ndULp-iQZdA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.