[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Terry J. Reedy]

Terry J. Reedy  added the comment:

>  why a C extension apart and not a patch to `readline`?

Search "python ctypes vs c extension".  Multiple SO answers.  I believe speed 
and language competancy are big factors.  I don't think and stdlib modules use 
ctypes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Not surprisingly, I agree with Steven that enhancing IDLE may be a better use 
of core developer time.  It and Tkinter (and turtle) are options for the 
Windows and macOS (and some non-PSF) installers and easily installed  on Linux. 
 IDLE has most of the requested features, and where it does not, I will 
consider well-specified enhancements.  

1. Pasting a compound statement with embedded blank lines.

>>> def f():
a = 1  # following line has no spaces or tabs

b=3
print(a, b)

>>> f()
1, 3

Pasting multiple statements, separated or not by blank lines, and pasting 
non-BMP unicode, are separate issues.

2. Auto-indent: after ':' (see example above) and for syntactic sequences.  
(Currently, the latter only works in editors because '>>> ' prompts and tab 
indents interfere.  There is a issue with PR that will allow this to be fixed.)

strings = ['a',
   'b',  # autoindent
  ]
ardvark = fribble(a,
  b,  # autoindent
 )

3. Code blocks are one history entry.  Yes, where 'code block' means one 
complete statement, possibly multiple lines, as submitted for execution.  The 
example in 1. can be recalled as submitted, with the blank line.

4. Save interactive session.  Yes.  I intend to add an option to only save 
code, without prompts or output, so the code can be pasted into an editor and 
edited and run.  Or maybe the code should be loaded directly into an editor.

5. Add history() builtin.  I believe that on linux this is redundant with 
existing enhanced history mechanisms, which include storing history across 
sessions in a 'standard' place.  I believe my idea for 4. covers must of the 
use cases.

6. Add a. pprint and b. getsource to builtins.  I think this would be rejected. 
 I seldom use either of those two functions, but constantly import other 
things. So I would want other things added.  Startup files are the mechanism 
for personally customizing one's shell.  Simple imports serve for ad hoc 
customizations. Note that getsource(name) needs to be passed to print() to be 
readable.

As for getting source: IDLE displays a popup signature for any function if one 
is available.  Given a traceback or grep output, one can open any source-based 
module in an editor at the line indicated.  Open Module lets one easily open 
any importable source based module with only the import name, not the full 
path.  For long source chunks, a separate window is really much better than 
printed in the shell.  A possible IDLE enhancement would be the ability to add 
an object name and to jump to its definition.

7. Syntax coloring.  Yes, with full customization.

8. Bracket matching for(), {}, and []. Yes, with a couple of options.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Marco Sulla]

Marco Sulla  added the comment:

@Terry:

> Jupyter Console is, I read, QT based

Nope. It's shell based by default. You can open it also as a QT app, like IDLE, 
but by default `jupyter console` is via terminal.

> they must use "‘magic’ commands" entered after the '>>>' prompt 
> instead of code.  Guido specifically vetoed the idea

Indeed I'm against too, and I wrote it. And if you read my proposals, I do not 
suggest any magic word

> the answer to that proposal would be to use a PYTHONSTARTUP module 
> with code such as "import pprint as _; pprint = _.pprint"

I know this, but it should be the default behaviour, IMHO. I mean, you can 
invoke `help()` in REPL but also in a `.py`. It makes no sense, but you can do 
it and you have not to import a separate module before.

> The console feeds pasted lines *1 at a time* to interactive Python

This is fixed by many terminal editors, like `vi`, with bracketed paste mode, 
as I wrote.

> When REPL sends a prompt, everything up to and including a prompt is 
> somehow marked read-only.

A workaround could be simulate input by user. Ugly but effective.

> Syntax-coloring [...] requires full screen editing

???

> [Syntax-coloring] also needs to be configurable

This could be IMHO delayed, or not implemented at all. If you don't like the 
colors, you can always not use it :D It will suffice that the colors will be as 
much as possible readable also by the majority of color-blind person.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Stephen, I think *you* were the one over-anxious to be dismissive.  In the 
title Marco refers to "Jupyter console (IPython)" features and in his opening, 
to "Jupyter console, aka IPython".  Jupyter Console is, I read, QT based.  
IPython/Jupyter Notebooks are GUI-based also.  

However, I agree that single process terminal IPython, such as illustrated near 
the top of
https://ipython.readthedocs.io/en/stable/index.html
is the better reference for comparison.  As pictured, it requires a color 
terminal with full screen edit.

An important constraint is that instead of users talking to the UI program with 
menu clicks, they must use "‘magic’ commands" entered after the '>>>' prompt 
instead of code.  Guido specifically vetoed the idea of allowing these into 
IDLE, so he must not want them in the REPL either.

Skipping the rest of your post, I will just restate why I closed this issue.

1. It introduces too many features not directly related.  The existing 
unix-only completions uses two modules.  I suspect some of the other features 
would also need new modules.  (But Marco, please don't rush to immediately open 
8 new issues.)

Furthest removed, no new module needed: Adding builtins that can be used in 
REPL as it is. I suspect the answer to that proposal would be to use a 
PYTHONSTARTUP module with code such as "import pprint as _; pprint = _.pprint"

2. It introduces policy issues that might require a policy change, and that I 
think should better be discussed on, say, pydev.  Steven and I have obviously 
gotten different policy impressions from Brett and Guido respectively. I will 
try to ask on pydev what the current REPL feature policy is, and what people 
think it should be.  For instance, how far do we want to go is adding features 
that only work on a subset of systems?

3. I believe that some of the concrete proposals have even more implementation 
problems than Marco himself acknowledged.  Difficult does not mean prohibited, 
but coredevs are mostly not interested in working on UIs and it has been 
Guido's stated-to-me policy that 'advanced' UI/IDE features (not defined) 
should be left to other projects.

One communication problem is that once python is running in interactive mode, 
the only thing the user can send to it is lines of Python code.  Consider 
pasting multiline code with blank lines.  The console feeds pasted lines *1 at 
a time* to interactive Python, the same as if the user typed them.  So Python 
does not know if they were typed or pasted.  Nor does it know that there might 
be a next line already waiting, and if so, what it is (dedented?).  If it does 
not execute on receiving '\n', when will it?

There are also communication issues in the other direction.  When REPL sends a 
prompt, everything up to and including a prompt is somehow marked read-only.  
But autoindents must be left erasable so a user can dedent.

If that can be solved, including on Windows, I would like IDLE's autoindent 
code moved to an stdlib module (and polished, if need be) to be used by REPL, 
IDLE, and anyone else who wishes.  The main function would map a sequence of 
lines to a PEP-8 compliant indent for the next line.

Syntax-coloring opens a much bigger can of worms.  It requires full screen 
editing to overwrite existing input.  It also needs to be configurable as part 
of a configuration system.  IPython requires the user to write "a dictionary 
mapping Pygments token types to strings defining the style." in a python file 
with other needed code in a place where IPython can find it.  IDLE lets one 
modify an existing scheme by clicking on an element in sample code and then a 
new color and let IDLE handle the storage.

Marco, you said "I *really* hope that IDLE is simply a GUI wrapper of REPL".  
Nope.  In a single process, python cannot simultaneous execute a file in batch 
mode and input lines in interactive mode.  IDLE uses "exec(code, 
simulated_main_namespace)" and I suspect other simulated shells do something 
similar.  An advantage is that 'code' can represent complete multiline 
statements or even an entire module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Marco Sulla]

Marco Sulla  added the comment:

@Eryk: why a C extension apart and not a patch to `readline`?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Eryk Sun]

Eryk Sun  added the comment:

> For example, I would be shocked if it wasn't absolutely trivial 
> for the current implementation to add auto-indenting following 
> a colon. That feature alone would be a win for usability.

That would be a non-trivial change in Windows. I think it's at least possible 
using the high-level console API. It could be implemented with the 
pInputControl parameter of ReadConsoleW in combination with WriteConsoleW. This 
is how the CMD shell implements tab completion for file paths. That said, many 
of the proposed UI enhancements cannot be implemented in Windows using the 
high-level console API. 

IPython used to depend on readline. (5.0 switched to prompt_toolkit instead.) 
In Windows this was via the pyreadline package, which uses the low-level 
console API via ctypes. pyreadline is apparently abandoned (last updated in 
2015). Maybe CPython could incorporate a fork of pyreadline that fixes bugs 
(Unicode support in particular) and updates it to use a C extension instead of 
ctypes.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Marco Sulla]

Marco Sulla  added the comment:

Steven: currently I'm developing `frozendict` as part of CPython. About IDLE, 
IDLE can't be used on a server without a GUI. Furthermore, I *really* hope that 
IDLE is simply a GUI wrapper of REPL, with some additional features.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Marco Sulla]

Marco Sulla  added the comment:

Well, maybe too much feature requests in a single report. I'll report them 
separately, with more rationale.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Marco: there's no need for these to be "slowly" introduced. If the features are 
worth having in the default REPL, they're worth having as soon as possible, 
without us artificially slowing the process down. It will be hard enough to get 
somebody willing and able to do the work without telling them to dribble the 
features out slowly as well. Trust me on this, the hard part of Python 
development is getting feature requests implemented *at all*, not that they 
come too quickly!

You might like to try building these features on top of the pure-Python 
interactive REPL:

https://docs.python.org/3/library/code.html

or perhaps try adding them to IDLE.

If and when you have something positive to show, you could try re-opening this 
task with a concrete proof-of-concept using the code module, or perhaps a PR 
for IDLE.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Terry, I think you were extremely over-eager, almost aggressively so, to close 
this feature request, especially since your reasons given are rather bogus: 
IPython isn't based on a GUI, it works in a text mode console too, including on 
Windows.

https://ipython.readthedocs.io/en/stable/overview.html

You say "there is no need to completely rewrite current text-based interactive 
mode". You are probably right: there probably is *no need* to completely 
rewrite the current implementation to add at least some, if not all, of the 
requested features.

For example, I would be shocked if it wasn't absolutely trivial for the current 
implementation to add auto-indenting following a colon. That feature alone 
would be a win for usability.

Given that Brett already said that the main obstacle to this feature request 
was lack of somebody interested and able to do the work (as opposed to a policy 
that we want the default REPL to be weak and unfriendly), I think you were 
premature in closing this so quickly. It's not like it has been languishing for 
years.

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Terry J. Reedy]

Terry J. Reedy  added the comment:

In interactive mode, python.exe interacts with a console/(dumb terminal) 
through the std streams using \n as a special character  It gets input from 
stdin, send output to stdout or errors to stderr.  The terminal, not python, 
handles line editing and history retrieval.  Once a statement is entered and 
executed, python has no memory of it.

On Linux, one can use readline and ncurses modules for somewhat enhances 
interaction.

IPython is GUI-based.  Python already come with a GUI-based IDE, IDLE, which 
has many of the features you list - autoindent, statement history, save, line 
numbers in the editor, syntax coloring, and some source inspection.  Code with 
blank lines within statement can be pasted into an editor window and run either 
with or without clearing the shell workspace.

There are other alternatives with similar features, but this is not the place 
to discuss them.  The point is that there is no need to completely rewrite 
current text-based interactive mode.

--
nosy: +terry.reedy
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
components: +Interpreter Core -Demos and Tools
priority: normal -> low
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38747] Slowly introduce a subset of Jupyter console (IPython) features into CPython command line interactive mode

[Marco Sulla]

New submission from Marco Sulla :

Sometimes I’m lazy and I would test code copy - pasted from internet or from 
other sources directly in the interpreter, in interactive mode. But if the code 
contains completely blank lines, the copy - paste fails. For example:

def f():
print("Marco")

print("Sulla")

does not work, but

def f():
print("Marco")

print("Sulla")

yes. Notice that in a script the first code block is perfectly valid and works.

This does not happen with Jupiter console, aka IPython. Jupiter implements 
bracketed paste mode, so it distinguish between normal input and pasted input.

Jupyter offers also:
- autoindent
- save code blocks in one history entry: this way, if you write a function, for 
example, and you press the up key, the whole function will be loaded, and not 
its last line.
- auto-reloading of modules. It should be disabled by default and enabled by a 
flag, and could auto-reload a module if its code changes.
- save code to console. All the code written in the current interactive session 
could be saved to the clipboard. It could be triggered by F12.
- history: it could be a new built-in function. if called without parameters, 
it could show the history, with lines numbered. If called with a number, it 
will paste the corresponding history line to the console
- pretty printing and source inspection. IMHO pprint.pprint() and 
inspect.getsource() are so useful in interactive mode that could be added to 
builtins.
- syntax coloring. It should be disabled by default, and could be enabled by a 
flag or a config.
- bracket matching. See above.

I think that implementing all of this in CPython is really hard. I suppose that 
maybe many things are not possible for compatibility between platforms, or 
can't be available everywhere, like syntax coloring.

--
components: Demos and Tools
messages: 356248
nosy: Marco Sulla
priority: normal
severity: normal
status: open
title: Slowly introduce a subset of Jupyter console (IPython) features into 
CPython command line interactive mode
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com