Re: Pasting code into the cmdline interpreter

[Gregory Ewing]

eryk sun wrote:

if we see
a read that returns less than the buffer size but doesn't end on a
newline, then for a terminal, and only a terminal, I think we can
infer Ctrl+D was typed and handle it as EOF.


I don't think it would be wise to rely on that. There is
no promise that any given read() call will read all the
data that's available.

Even if it worked, I'm not sure that this behaviour would
be desirable. Some programs may rely on the current
behaviour, e.g. shells that do auto-completion when you
type ctrl-D on a non-empty line.

At the very least it would make programs written in
Python behave differently from any other unix program
when reading from a terminal, which I don't think is
a good idea.

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[eryk sun]

On Thu, Sep 22, 2016 at 3:21 PM, Random832  wrote:
> On Thu, Sep 22, 2016, at 09:45, eryk sun wrote:
>
>> Yes, FileIO.readall continues making read() system calls until it sees
>> an empty read. But if we know we're reading from a terminal, we should
>> be able to assume that a read either consumes an entire line up to a
>> newline character or the entire buffer, no? In other words, if we see
>> a read that returns less than the buffer size but doesn't end on a
>> newline, then for a terminal, and only a terminal, I think we can
>> infer Ctrl+D was typed and handle it as EOF.

> I don't see where you're getting that users should even expect ctrl-d at
> the end of a line to be treated as EOF. It doesn't behave that way in
> any other program - try it in cat, dash, etc.

I thought it could be avoided as a convenience by inferring EOF when
reading from a terminal, but clearly that's not the case.

> For example, if the user types the dsusp character (^Y by default) on
> systems that support it, the program will be suspended and will receive a
> partial read when resumed.

I didn't consider this, which means I'm stuck with double tapping
Ctrl+D to get exactly the pasted input without adding an extra
newline.

This is on my mind because I want to add Ctrl+D support for console
I/O in Windows, for consistency with Unix terminals and IDLE. With how
this works on Windows, the read returns with the control character
still in the buffer (e.g. '\x04'), so you know exactly what the user
typed. But I see now that it should retain the default readall()
behavior to only count an empty read (from the function that wraps
ReadConsoleW) as EOF.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Random832]

On Thu, Sep 22, 2016, at 09:45, eryk sun wrote:
> On Thu, Sep 22, 2016 at 12:40 PM, Gregory Ewing
>  wrote:
> > eryk sun wrote:
> >>
> >> Actually in a Unix terminal the cursor can also be at
> >> the end of a line, but a bug in Python requires pressing Ctrl+D twice
> >> in that case.
> >
> > I wouldn't call that a bug, rather it's a consequence of
> > what Ctrl-D does. It doesn't really mean EOF, it means to
> > send whatever the terminal driver has in its input buffer.
> > If the buffer is empty at the time, the process gets a
> > zero-length read which is taken as EOF. But if the buffer
> > is not empty, it just gets whatever is in the buffer.
> >
> > There's nothing Python can do about that, because it
> > never sees the Ctrl-D -- that's handled entirely by the
> > terminal driver.
> 
> Yes, FileIO.readall continues making read() system calls until it sees
> an empty read. But if we know we're reading from a terminal, we should
> be able to assume that a read either consumes an entire line up to a
> newline character or the entire buffer, no? In other words, if we see
> a read that returns less than the buffer size but doesn't end on a
> newline, then for a terminal, and only a terminal, I think we can
> infer Ctrl+D was typed and handle it as EOF.

I don't see where you're getting that users should even expect ctrl-d at
the end of a line to be treated as EOF. It doesn't behave that way in
any other program - try it in cat, dash, etc. There are also other
circumstances that can cause a partial read. For example, if the user
types the dsusp character (^Y by default) on systems that support it,
the program will be suspended and will receive a partial read when
resumed. I wouldn't bet on a guarantee that you can't get a partial read
if input is too long to fit in the driver's edit buffer (which may be
smaller than the program's read buffer) on some platforms (though when
I've tried it it simply stops accepting input until I hit ctrl-D or
enter)... and savvy users may deliberately type ctrl-D for this purpose.

And, of course, in readline, the terminal is really in cbreak mode and
Python receives an actual Ctrl+D rather than having to infer anything.
But here, too, other programs ignore it just like Python: try it in
bash, zsh, etc.

The only "bug" is in some users' simplistic understanding that "ctrl-D
means EOF".
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Veek M]

eryk sun wrote:

> On Thu, Sep 22, 2016 at 12:40 PM, Gregory Ewing
>  wrote:
>> eryk sun wrote:
>>>
>>> Actually in a Unix terminal the cursor can also be at
>>> the end of a line, but a bug in Python requires pressing Ctrl+D
>>> twice in that case.
>>
>> I wouldn't call that a bug, rather it's a consequence of
>> what Ctrl-D does. It doesn't really mean EOF, it means to
>> send whatever the terminal driver has in its input buffer.
>> If the buffer is empty at the time, the process gets a
>> zero-length read which is taken as EOF. But if the buffer
>> is not empty, it just gets whatever is in the buffer.
>>
>> There's nothing Python can do about that, because it
>> never sees the Ctrl-D -- that's handled entirely by the
>> terminal driver.
> 
> Yes, FileIO.readall continues making read() system calls until it sees
> an empty read. But if we know we're reading from a terminal, we should
> be able to assume that a read either consumes an entire line up to a
> newline character or the entire buffer, no? In other words, if we see
> a read that returns less than the buffer size but doesn't end on a
> newline, then for a terminal, and only a terminal, I think we can
> infer Ctrl+D was typed and handle it as EOF.
 
read() changes it behavior intelligently according to the file type 
(pipe, fifo, etc). By default, from a terminal, it reads up to the first 
newline - canonical mode/cooked mode vs non-canonical mode (vi/less). 
Maybe python interpreter is doing the read in non-canonical mode?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[eryk sun]

On Thu, Sep 22, 2016 at 12:40 PM, Gregory Ewing
 wrote:
> eryk sun wrote:
>>
>> Actually in a Unix terminal the cursor can also be at
>> the end of a line, but a bug in Python requires pressing Ctrl+D twice
>> in that case.
>
> I wouldn't call that a bug, rather it's a consequence of
> what Ctrl-D does. It doesn't really mean EOF, it means to
> send whatever the terminal driver has in its input buffer.
> If the buffer is empty at the time, the process gets a
> zero-length read which is taken as EOF. But if the buffer
> is not empty, it just gets whatever is in the buffer.
>
> There's nothing Python can do about that, because it
> never sees the Ctrl-D -- that's handled entirely by the
> terminal driver.

Yes, FileIO.readall continues making read() system calls until it sees
an empty read. But if we know we're reading from a terminal, we should
be able to assume that a read either consumes an entire line up to a
newline character or the entire buffer, no? In other words, if we see
a read that returns less than the buffer size but doesn't end on a
newline, then for a terminal, and only a terminal, I think we can
infer Ctrl+D was typed and handle it as EOF.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Gregory Ewing]

eryk sun wrote:

Actually in a Unix terminal the cursor can also be at
the end of a line, but a bug in Python requires pressing Ctrl+D twice
in that case.


I wouldn't call that a bug, rather it's a consequence of
what Ctrl-D does. It doesn't really mean EOF, it means to
send whatever the terminal driver has in its input buffer.
If the buffer is empty at the time, the process gets a
zero-length read which is taken as EOF. But if the buffer
is not empty, it just gets whatever is in the buffer.

There's nothing Python can do about that, because it
never sees the Ctrl-D -- that's handled entirely by the
terminal driver.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Veek M]

Ben Finney wrote:

> Veek M  writes:
> 
>> Ben Finney wrote:
>>
>> > Since you are writing code into a module file, why not just run the
>> > module from that file with the non-interactive Python interpreter?
>> > 
>> It's part of a hexchat plugin/addon..
> 
> Which tells me that it still isn't appropriate to copy-paste the code
> directly into the *interactive* Python interpreter. Instead, it's a
> module that should be imported from file, by Hexchat.
> 
> Unless I misunderstand what you're talking about?
> 
ah nope, you got it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Veek M]

eryk sun wrote:

> On Thu, Sep 22, 2016 at 5:12 AM, Veek M  wrote:
>> 2. Blank lines in my code within the editor are perfectly acceptable
>> for readability but they act as a block termination on cmd line.
> 
> You can write a simple paste() function. For example:
> 
> import sys
> paste = lambda: exec(sys.stdin.read(), globals())
> 
> >>> paste()
> class Foo:
> """test class"""
> 
> def spam(self):
> '''test method'''
> return 'eggs'
> 
> >>> Foo().spam()
> 'eggs'
> 
> In a Unix terminal and IDLE, stdin.read() is terminated by typing
> Ctrl+D at the start of an empty line. For the Windows console, it's
> Ctrl+Z, Enter. Actually in a Unix terminal the cursor can also be at
> the end of a line, but a bug in Python requires pressing Ctrl+D twice
> in that case.

ah! very clever and many thanks! so, you use .pythonrc.py to 
store/define your cmd-line shortcuts.

It doesn't work in python2.x, because exec is a stmt, so, def paste(
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Lawrence D’Oliveiro]

On Thursday, September 22, 2016 at 5:12:13 PM UTC+12, Veek M wrote:
> How do i deal with this - what's the best way to achieve what I'm trying 
> to do.

If you want a scratchpad for trying out Python code, I can recommend 
Jupyter/IPython .
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Steven D'Aprano]

On Thursday 22 September 2016 17:42, eryk sun wrote:

> On Thu, Sep 22, 2016 at 5:12 AM, Veek M  wrote:
>> 2. Blank lines in my code within the editor are perfectly acceptable for
>> readability but they act as a block termination on cmd line.
> 
> You can write a simple paste() function. For example:
> 
> import sys
> paste = lambda: exec(sys.stdin.read(), globals())

Nice!



-- 
Steven
git gets easier once you get the basic idea that branches are homeomorphic 
endofunctors mapping submanifolds of a Hilbert space.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[eryk sun]

On Thu, Sep 22, 2016 at 5:12 AM, Veek M  wrote:
> 2. Blank lines in my code within the editor are perfectly acceptable for
> readability but they act as a block termination on cmd line.

You can write a simple paste() function. For example:

import sys
paste = lambda: exec(sys.stdin.read(), globals())

>>> paste()
class Foo:
"""test class"""

def spam(self):
'''test method'''
return 'eggs'

>>> Foo().spam()
'eggs'

In a Unix terminal and IDLE, stdin.read() is terminated by typing
Ctrl+D at the start of an empty line. For the Windows console, it's
Ctrl+Z, Enter. Actually in a Unix terminal the cursor can also be at
the end of a line, but a bug in Python requires pressing Ctrl+D twice
in that case.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Ben Finney]

Veek M  writes:

> Ben Finney wrote:
>
> > Since you are writing code into a module file, why not just run the
> > module from that file with the non-interactive Python interpreter?
> > 
> It's part of a hexchat plugin/addon..

Which tells me that it still isn't appropriate to copy-paste the code
directly into the *interactive* Python interpreter. Instead, it's a
module that should be imported from file, by Hexchat.

Unless I misunderstand what you're talking about?

-- 
 \ “Capitalism has destroyed our belief in any effective power but |
  `\  that of self interest backed by force.” —George Bernard Shaw |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Veek M]

Ben Finney wrote:

> Veek M  writes:
> 
>> 1. I had to turn on  highlighting to catch mixed indent (which
>> is a good thing anyways so this was resolved - not sure how tabs got
>> in anyhow)
> 
> The EditorConfig system is a growing consensus for configuring a code
> base to instruct text editors not to mangle it. See the EditorConfig
> site  for more information.
> 
> Sadly, it seems Kate does not yet recognise EditorConfig instructions
> . In the meantime,
> manually configure Kate to only ever insert spaces to indent lines.
> 
>> 2. Blank lines in my code within the editor are perfectly acceptable
>> for readability but they act as a block termination on cmd line.
> 
> Yes. That is a deliberate compromise to make it easy to write code
> interactively at the interactive prompt.
> 
>> I get:
>>  IndentationError: unexpected indent
>>
>> How do i deal with this - what's the best way to achieve what I'm
>> trying to do.
> 
> Since you are writing code into a module file, why not just run the
> module from that file with the non-interactive Python interpreter?
> 
It's part of a hexchat plugin/addon.. like below (basically lots of 
hexchat crud interspersed. 

Anyway, i should hive it off into a separate function, and hide the 
import hexchat - still, bit of a pain.

import hexchat

def send_message(word, word_eol, userdata):
if not(word[0] == "65293"):
return

msg = hexchat.get_info('inputbox')
if msg is None:
return

x = re.match(r'(^\\help)\s+(\w+)', msg)
if x:
filter = x.groups()[1]
for key, value in unicode_tex.tex_to_unicode_map.items():
if filter in key:
print_help(value + ' ' + key)
hexchat.command("settext %s" % '')
return

# findall returns: [(,), (,), ...] or []
tex_matches = re.findall(r'((\\\w+){(.+?)})|(\\\w+)', msg)
if len(tex_matches) == 0: 
  return
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Christian Gollwitzer]

Am 22.09.16 um 07:12 schrieb Veek M:

I wanted to test this piece of code which is Kate (editor) on the cmd
line python >>> prompt:
2. Blank lines in my code within the editor are perfectly acceptable for
readability but they act as a block termination on cmd line. So if i
paste:
tex_matches = re.findall(r'(\\\w+{.+?})|(\\\w+)', msg)
for tex_word in tex_matches:
repl = unicode_tex.tex_to_unicode_map.get(tex_word)

if repl is None:

I get:
 IndentationError: unexpected indent


IPython has a %paste and %cpaste magic to deal with this and other 
problems. It won't resolve the tab issue. However, Python code in 
general (due to indentation) is not very friendly for pasting and 
interactive usage. Better write the code to a file, then (again from 
IPython) you can read this by %run filename.py


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pasting code into the cmdline interpreter

[Ben Finney]

Veek M  writes:

> 1. I had to turn on  highlighting to catch mixed indent (which is a 
> good thing anyways so this was resolved - not sure how tabs got in 
> anyhow)

The EditorConfig system is a growing consensus for configuring a code
base to instruct text editors not to mangle it. See the EditorConfig
site  for more information.

Sadly, it seems Kate does not yet recognise EditorConfig instructions
. In the meantime,
manually configure Kate to only ever insert spaces to indent lines.

> 2. Blank lines in my code within the editor are perfectly acceptable
> for readability but they act as a block termination on cmd line.

Yes. That is a deliberate compromise to make it easy to write code
interactively at the interactive prompt.

> I get:
>  IndentationError: unexpected indent
>
> How do i deal with this - what's the best way to achieve what I'm
> trying to do.

Since you are writing code into a module file, why not just run the
module from that file with the non-interactive Python interpreter?

-- 
 \“The restriction of knowledge to an elite group destroys the |
  `\   spirit of society and leads to its intellectual |
_o__)impoverishment.” —Albert Einstein |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list