[jupyter] Re: Formalizing preparing a release

[Steven Silvester]

I like the idea of handling the tracking and coordination on the 
https://github.com/jupyter/project-mgt and having a snapshot in the weekly 
dev meeting report as well.



On Wednesday, February 8, 2017 at 9:12:24 PM UTC-6, Matthias Bussonnier 
wrote:
>
> Hello all, 
>
> It recently came to the attention to some of us that with the 
> increasing number of projects we have it can be hard to follow when 
> packages are going to be released, which often leads to very short 
> windows of time to give feedback or test the new version with existing 
> software. 
>
> For example, several developers were surprised yesterday with the 
> announcement of an upcoming notebook 5.0 release, and are now 
> struggling to catch up on what is new and to test their 
> plugins/extensions. There are likely others in the community who did 
> not realize the 5.0 release was so close, who would need some time to 
> test their extensions/plugins and give feedback. 
>
> How would the team and everyone else feel if we encouraged Jupyter 
> projects to open an issue when a major release started to take shape 
> which clearly listed the planned schedule for the release and 
> highlighted what was new in the release? The upcoming release and this 
> issue would be announced on the mailing list. People interested in 
> following the release updates could subscribe to this issue. 
>
> That would be of course on a per-project/per-maintainer basis, but the 
> project would try  to encourage it for major releases, or maybe even 
> minor releases. 
>
> Thanks, 
> -- 
> Matthias, with the help of Jamie, Jason, Brian and Fernando. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/15e15697-328a-4903-882b-5c1506bee00a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] Formalizing preparing a release

[Brian Granger]

I am definitely in favor of this.

We have millions of users, with many of them organizations,
universities, non-profits, researchers, etc. who are relying on our
software and building on top of it. So, in addition to our own
developers needing to know about and discuss releases, we also have a
broader set of stakeholders who are affected by upcoming releases.

Many of these stakeholders don't follow our weekly meetings/notes
closely and wouldn't be able to follow all of our repos (even our core
devs working full time are not able to follow these things). Because
of that a post on the mailing list about each major release is about
as good as we can do to let the community know a release is
approaching.

In terms of where to open an issue, I almost think the project
management repo makes sense:

https://github.com/jupyter/project-mgt

That is a great way of keeping Jamie in the loop on these things. But
I am also fine with using other repos for the issues.

On Wed, Feb 8, 2017 at 7:12 PM, Matthias Bussonnier
 wrote:
> Hello all,
>
> It recently came to the attention to some of us that with the
> increasing number of projects we have it can be hard to follow when
> packages are going to be released, which often leads to very short
> windows of time to give feedback or test the new version with existing
> software.
>
> For example, several developers were surprised yesterday with the
> announcement of an upcoming notebook 5.0 release, and are now
> struggling to catch up on what is new and to test their
> plugins/extensions. There are likely others in the community who did
> not realize the 5.0 release was so close, who would need some time to
> test their extensions/plugins and give feedback.
>
> How would the team and everyone else feel if we encouraged Jupyter
> projects to open an issue when a major release started to take shape
> which clearly listed the planned schedule for the release and
> highlighted what was new in the release? The upcoming release and this
> issue would be announced on the mailing list. People interested in
> following the release updates could subscribe to this issue.
>
> That would be of course on a per-project/per-maintainer basis, but the
> project would try  to encourage it for major releases, or maybe even
> minor releases.
>
> Thanks,
> --
> Matthias, with the help of Jamie, Jason, Brian and Fernando.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to jupyter+unsubscr...@googlegroups.com.
> To post to this group, send email to jupyter@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jupyter/CANJQusV%3DVYmhw%2B5mAGYPx6MHLDXyL9tfNqvTvftM%2BOa2g0m9Hg%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.



-- 
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgran...@calpoly.edu and elliso...@gmail.com

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAH4pYpQ7L8MCk%2Bm16nDZE0HqDje-s4ey%2ByH7d-kU4gDoETaQNA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] Formalizing preparing a release

[Carol Willing]

It would be a good idea to include a tentative release calendar at the 
bottom of the weekly development update that Matthias has been helpfully 
sending to this list. Capturing information in a release planning table 
would be beneficial; it would have been hugely helpful last year for 
planning documentation efforts. Information could include:


 * Project name with link to repo
 * Current stable release version and date released
 * In progress release type (major, minor, bugfix), version, and
   projected week of beta/release

I would recommend keeping it simple by incorporating it into our weekly 
workflow for the development meeting.

Jason Grout 
February 9, 2017 at 5:05 AM
I think announcing and coordinating major (and probably even minor) 
releases in the way that Matthias outlines is a great idea. I agree 
with Thomas that bugfix releases should be easier and more frequently 
released.


--
You received this message because you are subscribed to the Google 
Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jupyter+unsubscr...@googlegroups.com 
.
To post to this group, send email to jupyter@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAPDWZHwSPDWDqGJe%3DMd5VJCwMyNEdoZuz39RcrCGfGz3sJVUYw%40mail.gmail.com 
.

For more options, visit https://groups.google.com/d/optout.
Thomas Kluyver 
February 9, 2017 at 3:52 AM
On 9 February 2017 at 03:12, Matthias Bussonnier 
> 
wrote:


For example, several developers were surprised yesterday with the
announcement of an upcoming notebook 5.0 release, and are now
struggling to catch up on what is new and to test their
plugins/extensions. There are likely others in the community who did
not realize the 5.0 release was so close, who would need some time to
test their extensions/plugins and give feedback.


The notes from last week's meeting do say:
"Currently going through issues to try to get 5.0 out as soon as 
possible. Hopefully Beta next month. "


So this shouldn't have come as a total surprise.

How would the team and everyone else feel if we encouraged Jupyter
projects to open an issue when a major release started to take shape
which clearly listed the planned schedule for the release and
highlighted what was new in the release? The upcoming release and this
issue would be announced on the mailing list. People interested in
following the release updates could subscribe to this issue.


I think this makes sense for something like a major release of 
notebook. Not so much for minor releases, or less prominent packages. 
I'd actually like minor releases to involve fewer steps - this has 
been very noticeable doing bugfix releases of IPython, where the 
release process doc lists 14 steps.


Thomas
--
You received this message because you are subscribed to the Google 
Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to jupyter+unsubscr...@googlegroups.com 
.
To post to this group, send email to jupyter@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAOvn4qg5gH%2BDLcFA%3D_TNBwqn0FQna9aD9BMTV_-tdHrWABLTfw%40mail.gmail.com 
.

For more options, visit https://groups.google.com/d/optout.
Matthias Bussonnier 
February 8, 2017 at 7:12 PM
Hello all,

It recently came to the attention to some of us that with the
increasing number of projects we have it can be hard to follow when
packages are going to be released, which often leads to very short
windows of time to give feedback or test the new version with existing
software.

For example, several developers were surprised yesterday with the
announcement of an upcoming notebook 5.0 release, and are now
struggling to catch up on what is new and to test their
plugins/extensions. There are likely others in the community who did
not realize the 5.0 release was so close, who would need some time to
test their extensions/plugins and give feedback.

How would the team and everyone else feel if we encouraged Jupyter
projects to open an issue when a major release started to take shape
which clearly listed the planned schedule for the release and
highlighted what was new in the release? The upcoming release and this
issue would be announced on the 

Re: [jupyter] Formalizing preparing a release

[Jason Grout]

I think announcing and coordinating major (and probably even minor)
releases in the way that Matthias outlines is a great idea. I agree with
Thomas that bugfix releases should be easier and more frequently released.

On Thu, Feb 9, 2017 at 6:53 AM Thomas Kluyver  wrote:

> On 9 February 2017 at 03:12, Matthias Bussonnier <
> bussonniermatth...@gmail.com> wrote:
>
> For example, several developers were surprised yesterday with the
> announcement of an upcoming notebook 5.0 release, and are now
> struggling to catch up on what is new and to test their
> plugins/extensions. There are likely others in the community who did
> not realize the 5.0 release was so close, who would need some time to
> test their extensions/plugins and give feedback.
>
>
> The notes from last week's meeting do say:
> "Currently going through issues to try to get 5.0 out as soon as
> possible. Hopefully Beta next month. "
>
> So this shouldn't have come as a total surprise.
>
> How would the team and everyone else feel if we encouraged Jupyter
> projects to open an issue when a major release started to take shape
> which clearly listed the planned schedule for the release and
> highlighted what was new in the release? The upcoming release and this
> issue would be announced on the mailing list. People interested in
> following the release updates could subscribe to this issue.
>
>
> I think this makes sense for something like a major release of notebook.
> Not so much for minor releases, or less prominent packages. I'd actually
> like minor releases to involve fewer steps - this has been very noticeable
> doing bugfix releases of IPython, where the release process doc lists 14
> steps.
>
> Thomas
>
> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+unsubscr...@googlegroups.com.
> To post to this group, send email to jupyter@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/CAOvn4qg5gH%2BDLcFA%3D_TNBwqn0FQna9aD9BMTV_-tdHrWABLTfw%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAPDWZHwSPDWDqGJe%3DMd5VJCwMyNEdoZuz39RcrCGfGz3sJVUYw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] Formalizing preparing a release

[Thomas Kluyver]

On 9 February 2017 at 03:12, Matthias Bussonnier <
bussonniermatth...@gmail.com> wrote:

> For example, several developers were surprised yesterday with the
> announcement of an upcoming notebook 5.0 release, and are now
> struggling to catch up on what is new and to test their
> plugins/extensions. There are likely others in the community who did
> not realize the 5.0 release was so close, who would need some time to
> test their extensions/plugins and give feedback.
>

The notes from last week's meeting do say:
"Currently going through issues to try to get 5.0 out as soon as possible.
Hopefully Beta next month. "

So this shouldn't have come as a total surprise.

How would the team and everyone else feel if we encouraged Jupyter
> projects to open an issue when a major release started to take shape
> which clearly listed the planned schedule for the release and
> highlighted what was new in the release? The upcoming release and this
> issue would be announced on the mailing list. People interested in
> following the release updates could subscribe to this issue.


I think this makes sense for something like a major release of notebook.
Not so much for minor releases, or less prominent packages. I'd actually
like minor releases to involve fewer steps - this has been very noticeable
doing bugfix releases of IPython, where the release process doc lists 14
steps.

Thomas

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAOvn4qg5gH%2BDLcFA%3D_TNBwqn0FQna9aD9BMTV_-tdHrWABLTfw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] Unicode Greek letters in matplotlib images with nbconvert's python API

[Thomas Kluyver]

On 8 February 2017 at 23:58, Paul Hobson  wrote:

> nbdata = nbformat.read(nbf, as_version=4, encoding='utf-8')


For reference, I'm pretty sure that passing encoding= on this line has no
effect - it's not passed down to anything that takes an encoding argument.
Passing it to open() is the important bit.

Thomas

-- 
You received this message because you are subscribed to the Google Groups 
"Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jupyter+unsubscr...@googlegroups.com.
To post to this group, send email to jupyter@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jupyter/CAOvn4qihBgbE8fOD1Y9JcBmp%2BXLqdg0ebL_hksegkpD28O%2BdWQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [jupyter] Getting the port number in an IPython noteboook

[Johannes Feist]

Hi DG,

I see, I didn't get that your analysis is interactive in the first part.
While I'm not a Jupyter developer, I think what you want could be achieved
with some Javascript more easily. If you make a cell with

%%javascript
var nb = Jupyter.notebook;
nb.save_checkpoint();
nb.kernel.execute("NB_NAME = '" + nb.notebook_name + "'");

the "save_checkpoint" should ensure that your notebook is saved (AFAIK this
is the same javascript that is executed when you click on "save and
checkpoint"), and the last line saves the name of your notebook in the
python variable "NB_NAME".
since shutil and the shell ("!") should use the notebook directory as the
cwd automatically, I don't think you'll ever need the full working path? At
least if your notebook server and kernel are running on the same system.
So then
shutil.copy(NB_NAME,os.path.join(Save_dir,NB_NAME))
!jupyter nbconvert '$NB_NAME' --output '$nfname'

should work. If nb.notebook_name is not enough information, maybe
nb.base_url and nb.notebook_path are also relevant.

Best,
Johannes


Johannes Feist
Departamento de Física Teórica de la Materia Condensada
Universidad Autónoma de Madrid
johannes.fe...@uam.es

On Wed, Feb 8, 2017 at 11:55 PM, DG  wrote:

> Hello Johannes,
> I will take a look at this approach, which looks more oriented to batch
> processing rather the interactive processing I do now.
>
> However, right off the bat there are a few difficulties. The way my
> workflow looks like right now:
> - Open data analysis IPython notebook (from within IPython Jupyter
> notebook home tab)
> - change input variables (from which the path to the source files is
> calculated)
> - run a few cells manually first to see if the output makes sense. Adjust
> some parameters of the analysis
> - run notebook to completion, which includes saving the copy and the HTML
> version
> - repeat from second step for additional data sets
>
> I guess I can create another program or script that generates the needed
> command lines and then executes them. Will I be able to watch the output of
> the notebook in real time?
>
> DG
>
>
>
> On Wednesday, February 8, 2017 at 2:28:57 AM UTC-8, Johannes Feist wrote:
>>
>> Just in case you don't know about it, it sounds to me like this would be
>> easier to do with nbconvert, using the execute preprocessor (which runs the
>> notebook and saves it). E.g., you could do (from the command line)
>> jupyter nbconvert --to notebook --execute mynotebook.ipynb --output
>> Analysis/mynotebook.ipynb
>> and then convert that executed notebook to html (output goes to the same
>> folder):
>> jupyter nbconvert --to html Analysis/mynotebook.ipynb
>>
>> Documentation is at http://nbconvert.readthedocs.i
>> o/en/stable/usage.html#convert-notebook
>>
>> Johannes
>>
>> On Wed, Feb 8, 2017 at 5:40 AM DG  wrote:
>>
>>> OK, here's where it comes from.
>>>
>>> I have a data analysis script, residing in my Documents folder. The
>>> script reads raw data files on a server share. It creates a subfolder
>>> "Analysis" and puts a bunch of tables and plots in that folder. At the end
>>> of the script, I want to:
>>> 1. Save a copy of the executed notebook (.ipynb format) in the same
>>> subfolder, and
>>> 2. Save an HTML version of it, also in the same subfolder
>>>
>>> This should be trivial to do but unfortunately it's not. (It's sad that
>>> such a wonderful tool is so lacking in terms of introspection). I currently
>>> handle those two tasks as follows. To create a copy of the notebook, I use
>>> shutil.copy(nb_path,os.path.join(Save_dir,nb_filename))
>>> To save an HTML copy, I use:
>>>  !jupyter nbconvert '$nb_path' --output '$nfname'
>>>
>>> Both of those require to know the path of the notebook currently
>>> running. After much googling, I settled on this:
>>>
>>> connection_file_path = kernel.get_connection_file()
>>> connection_file = os.path.basename(connection_file_path)
>>> kernel_id = connection_file.split('-', 1)[1].split('.')[0]
>>> sessions = json.load(urllib2.urlopen('http://127.0.0.1:/api/session
>>> s'))
>>> for sess in sessions:
>>> if sess['kernel']['id'] == kernel_id:
>>> nb_rel_path = (sess['notebook']['path'])
>>> break
>>> res = !echo ~
>>> nb_path = os.path.join(res[0],nb_rel_path)
>>>
>>> This normally works, but the other day I had another server running, so
>>> naturally the second server's port was set to 8889 instead of ,
>>> therefore breaking my script.
>>>
>>> There are two other problems. Sometimes the saved .ipynb is not
>>> complete. The last few cells are saved without the output. The sure way to
>>> save a complete notebook is to manually hit the Save button, then re-run
>>> the cell with the file copy command. So I looked for something as simple as
>>> issuing a "Save" command, but that also I could not find.
>>>
>>> The second problem is that code snipped does not seem to work in
>>> Windows...
>>>
>>>
>>> On Tuesday, February 7, 2017 at