Drekin added the comment:
I've also been hit by this issue, see
https://mail.python.org/pipermail/python-list/2015-July/693496.html and the
following thread. I've spent some time trying to find where the problem is, and
just after I found out it's about eof_received() returning None, I could
Changes by Drekin dre...@gmail.com:
--
nosy: +Drekin
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22555
___
___
Python-bugs-list mailing list
Drekin added the comment:
Note that under status quo PyOS_Readline is called from two places: the
tokenizer during an interactive session and the builtin function input. The
tokenizer passes promptstring encoded in sys.stdin.encoding while input()
passes promtstring encoded
Drekin added the comment:
Unfortunately, I have little or no experience with Python C code and I even
don't have a C compiler installed so I cannot experiment. I'll just put my
ideas how to solve this here.
• Add sys.__readlinehook__ attribute, which can be set to a function taking a
prompt
Drekin added the comment:
File redirection has nothing to do with win-unicode-console and this issue.
When stdout is redirected, it is not a tty so win-unicode-console doesn't
replace the stream object, which is the right thing to do. You got
UnicodeEncodeError because Python creates
Drekin added the comment:
Note that win-unicode-console replaces the stdio streams rather than wraps
them. So the desired state would be Unicode stream objects wrapped by PTVS.
There would be no problem if win-unicode-console stream replacement occured
before PTVS wraps them, which should
Drekin added the comment:
I tried the following code:
import pdb
pdb.set_trace()
print(1 + 2)
print(αβγ∫)
When run in vanilla Python it indeed ends with UnicodeEncodeError as soon as it
hits the line with non-ASCII characters. However, the solution via
win_unicode_console package seems
Drekin added the comment:
I understand. I have found
https://mail.python.org/pipermail/python-3000/2007-May/007995.html as a reason
for using NFKC rather than NFC. On the other hand I think one may want these
double-struct mathematical letters to be different from the ordinary ones if
used
New submission from Drekin:
eval(\N{mathematical double-struck capital a})
NameError: name 'A' is not defined
A = 2
eval(\N{mathematical double-struck capital a})
2
\N{mathematical double-struck capital a} == A
False
--
components: Interpreter Core, Unicode
messages: 229653
nosy
Drekin added the comment:
stijn: You are mixing two issues here. One is reading text from a file. There
is no problem with it. You just call open(path,
encoding=the_encoding_of_the_file). Since the encoding of the file depends on
the file, you should provide the information about it.
Another
Drekin added the comment:
I installed wish and tried the demos shipped with Python. The behavior occurs
during the demo basic editable text. So it's an issue of Tk itself. Actually
the same behavior occurs even in the wish shell.
--
___
Python
Drekin added the comment:
Stefan Champailler:
The crash you see is maybe not a crash at all. First it has nothing to do with
printing, the problem is reading of your input line. That explains why Python
exited even before printing the traceback of the SyntaxError. If you try to
read input
Drekin added the comment:
Nick Coghlan: Ok, done.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1602
___
___
Python-bugs-list mailing list
Drekin added the comment:
Idle shell handles Unicode characters well, but one cannot enter them using
deadkey combinations. See http://bugs.python.org/issue22408.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1602
Drekin added the comment:
By modifier I mean a key like Shift, Ctrl, Alt, AltGr; or the corresponding
modifier state. A combination modifier + base key produces either special
action or a character or a dead key. A dead key affects the next character
rather than producing a character itself
Drekin added the comment:
It seems that it's a problem of dead key combinations rather than modifiers.
Unicode characters produced directly (with or without modifiers) work correctly.
Only one deadkey on my keyboard doesn't work at all. AltGr + M is a deadkey,
but any following key
Drekin added the comment:
I have found another problem. PyOS_Readline can be called from two different
places – from Parser/tokenizer.c:tok_nextc (by REPL), which uses
sys.stdin.encoding to encode prompt argument, and from
Python/bltinmodule.c:builtin_input_impl (by input() function), which
New submission from Drekin:
Key events produced on Windows handles Unicode incorrectly when Unicode
character is produced by dead-key combination.
On my keyboard, (AltGr + M, a) produces several key events, last of which
contains char==a, however, it should contain ∀. Also dead-key sequence
Drekin added the comment:
Antoine Pitrou: I understand. It would be nice to have that new Python string
based readline hook. Its default implementation could be to call PyOS_Readline
and decode the bytes using sys.stdin.encoding (as the tokenizer currently
does). Tokenizer then woudn't need
Drekin added the comment:
I realized that the behavior I want can be achieved by setting
PyOS_ReadlineFunctionPointer to a function calling sys.stdin.readline().
However I found another problem: Python REPL just doesn't work, when
sys.stdin.encoding is UTF-16-LE. The tokenizer (Parser
Changes by Drekin dre...@gmail.com:
--
nosy: +Drekin
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22257
___
___
Python-bugs-list mailing list
Drekin added the comment:
I have found another example of where the current interaction between readline
and Python core lead to confussion. It started with following report on my
package: https://github.com/Drekin/win-unicode-console/issues/2 .
Basically, IPython interactive console
Drekin added the comment:
Hello, I wanted to use this feature and found this issue. It is marked as
closed, being a duplicate of another closed issue. However the feature is
actually not present. In 2008 it was planned to be added by Matthew Barnett in
#3825, but the work of improving re
Drekin added the comment:
I think that boxes are ok, it's just missing font. Without active workaroud
there is just UnicodeEncodeError (with cp852 for me). There is problem with
astral characters – I'm getting each box twice. It is possible that Windows
console doesn't handle astral
Drekin added the comment:
I looked to the sourcecode and found the following.
First, the codepath of how interactive loop gets its input follows:
Python/pythonrun.c:PyRun_InteractiveLoopFlags
Python/pythonrun.c:PyRun_InteractiveOneObject
Python/pythonrun.c:PyParser_ASTFromFileObject
Parse
Drekin added the comment:
Sorry for formating in the previous message. Repeating…
I looked to the sourcecode and found the following.
First, the codepath of how interactive loop gets its input follows:
Python/pythonrun.c:PyRun_InteractiveLoopFlags
Python/pythonrun.c:PyRun_InteractiveOneObject
Drekin added the comment:
Shouldn't there be a Ctrl-C check in Modules/_io/fileio.c:fileio_read*
functions? I think some of these are called by standard sys.stdin.readline().
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org
Drekin added the comment:
My naive picture of ideal situation looks like this: When the interactive loop
wants input, it just calls sys.stdin.readline, which delegates to
sys.stdin.buffer.raw.readinto or .read, these can use GNU readline if available
to get the data. May I ask, what's wrong
Changes by Drekin dre...@gmail.com:
Removed file: http://bugs.python.org/file27261/win_unicode_console_2.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1602
Changes by Drekin dre...@gmail.com:
Removed file: http://bugs.python.org/file29563/i.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1602
___
___
Python
Changes by Drekin dre...@gmail.com:
Removed file: http://bugs.python.org/file29564/win_unicode_console_3.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1602
Changes by Drekin dre...@gmail.com:
Removed file: http://bugs.python.org/file31756/streams.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1602
Changes by Drekin dre...@gmail.com:
Removed file: http://bugs.python.org/file31770/win_unicode_console.zip
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1602
Changes by Drekin dre...@gmail.com:
Removed file: http://bugs.python.org/file35990/win_unicode_console.zip
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1602
Drekin added the comment:
I have deleted all my old files and added only my current implementation of the
stream objects as the only relevant part to this issue.
@Mark Summerfield: I have added __init__.py to the new version of
win_unicode_console. If there is any problem, you can start
Changes by Drekin dre...@gmail.com:
Removed file: http://bugs.python.org/file27245/win_unicode_console.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1602
Drekin added the comment:
@Victor Stinner: You are right. So I did it. Here are the links to GitHub and
PyPI: https://github.com/Drekin/win-unicode-console,
https://pypi.python.org/pypi/win_unicode_console.
I also tried to delete the files, but it seems that it is only possible to
unlink
Drekin added the comment:
I have made some updates in the streams code. Better error handling (getting
errno by GetLastError() and raising exception when zero bytes are written on
non-zero input). This prevents the infinite loop in BufferedIOWriter.flush()
when there is odd number of bytes
Drekin added the comment:
There is still the serious inconsistency that the `sys.stdin` is not used for
input by interactive loop but its encoding is. So if I replace `sys.stdin` with
a custom object with its own `encoding` attribute, the standard interactive
loop tries to use this encoding
Drekin added the comment:
Yes, I've done it similarily using a class method. Thank you for help.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19040
Drekin added the comment:
Hello, I have run into this when I wanted to use OrderedEnum and the example in
enum docs seemed too repetitive to me. It's nice to know that it's being worked
on.
--
nosy: +Drekin
___
Python tracker rep...@bugs.python.org
New submission from Drekin:
I tried to implement an Enum variant OptionalEnum such that OptionalEnum(value)
leaves value alone and returnes it (if there is no corresponding enum member)
rather than raising ValueError. My use case is following: I'm trying to parse
some keyboard layout related
Drekin added the comment:
Glenn Linderman: Yes I have built on what the others learned. For your
question, I made it and tested it in Python 3.3, it should also work in 3.4 and
what I've tried, it actually works. As Terry J. Reedy says you can just load
the module and enable the streams. I do
Drekin added the comment:
Hello again. I have rewritten the custom stdio objects and implemented them as
raw io reading and writing bytes in UTF-16-LE encoding. They are then wrapped
in standard BufferedReader/Writer and TextIOWrapper objects. This approach also
solves a bug of wrong string
Drekin added the comment:
Since this issue was fixed, shouldn't it be marked fixed here?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13758
New submission from Drekin:
On Windows, Python 3.3.2, when I run Python as a subprocess via Popen(py -i
somescript.py, stdin=PIPE, stdout=PIPE, stderr=PIPE).communicate() and
somescript.py ends with exception, there is first interactive promt in
stderr output and then the traceback which
Drekin added the comment:
Related stackoverflow question:
http://stackoverflow.com/questions/18419787/where-does-pythons-interactive-prompt-output-to
.
--
nosy: +Drekin
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12869
Drekin added the comment:
Why are there actually more codepaths which may raise this issue? My naive idea
would be that input() is just thin wrapper around sys.stdout.write() (for
prompt) and sys.stdin.readline() which leads to sys.stdin.buffer.raw.read*
where is the place where some low
Drekin added the comment:
There is over year old closely related issue: http://bugs.python.org/issue13758
.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17588
Drekin added the comment:
Hello. Will this be fixed? It's really annoying that you cannot pass valid
unicode filename to compile(). I'm using a workaround: I just pass
placeholder and then “update” the resulting code object recursively to set
the correct co_filename. Afterwards the code
Drekin added the comment:
This patch introduces a bit ugly traceback. Shouldn't there be “raise from
None” or “raise from previous_exc” instead of simple raise?
--
nosy: +Drekin
___
Python tracker rep...@bugs.python.org
http://bugs.python.org
Drekin added the comment:
It's also in __delitem__.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17702
___
___
Python-bugs-list mailing list
New submission from Drekin:
When I run sys.stdin.readline() interactivelly (on Windows and Python 3.3.2)
and hit Ctrl-C, sometimes it returns an empty string just before
KeyboardInterrupt is raised. Sometimes it isn't even raised and instead after
hitting Return some weird SyntaxtError
Drekin added the comment:
I haven't experience with Python C code but I tried to find some clues in the
code. First for input(): it call PyOS_Readline which may call
PyOS_StdioReadline my_fgets fgets in Parser/myreadline.c. There is Windows
related comment on line 56:
“Ctrl-C anywhere
Drekin added the comment:
Is there any chance the API will be added and used by python.exe?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17620
Drekin added the comment:
Is there a chance this will be fixed at least in Python 4?
--
nosy: +Drekin
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11619
New submission from Drekin:
Let's have a simple script test.py:
def f():
return x
x = 2
print(f())
Now if we try to run it via runpy.run_path, we get the following:
import runpy
g = runpy.run_path(test.py)
2
g[f]() is None
True
g[x] is 2
True
g[f].__globals__[x] is None
True
New submission from Drekin:
Currently, there is no way to run code.interact without a banner – empty string
still means to print an empty line. If I want that behaviour, I must subclass
code.InteractiveConsole and reimplement whole .interact method including the
repl logic just not to print
Drekin added the comment:
I encountered it when I changed sys.stdin at runtime (I thought it was a
supported feature) to affect the interactive console, see
http://bugs.python.org/issue1602 .
--
versions: -Python 2.7
___
Python tracker rep
Drekin added the comment:
Sorry for typos.
• interactive console doesn't use sys.stdin for input, why?
• it uses sys.stdin.encoding, shouldn't it rather use sys.__stdin__.encoding if
anything?
• input() and hence code.interact() uses sys.stdin
Drekin added the comment:
I have Windows Vista Business SP2, 64-bit, (6.0.6002). If I start Python any
way (from console, from shortcut, via py.exe or not) the same behavior occurs.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org
Drekin added the comment:
However it raises the exception correctly in IDLE.
The same behavior occured on Windows 7 Home Premium SP1, 64-bit, (6.1.7601).
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17619
Drekin added the comment:
I have no specific use case. I just thought that runpy.run_path should work
similarily as if the file was run directly (which works).
File ∫.py can be created, displayed and run by Python with no problem in
Windows
Drekin added the comment:
There is related weird behavior:
try:
... input()
... except KeyboardInterrupt:
... print(exception occured)
...
# Ctrl-C hit
Traceback (most recent call last):
File stdin, line 2, in module
KeyboardInterrupt
During handling of the above exception, another
New submission from Drekin:
At least on Windows, input() doesn't raise KeyboardInterrupt when Ctrl-C is hit
in Python 3.3 (it does in Python 3.2).
3.3:
x = input() # Ctrl-C hit - no exception raised
x
NameError
3.2:
x = input() # Ctrl-C hit
KeyboardInterrupt
--
messages: 185845
New submission from Drekin:
The Python interactive console actually doesn't use sys.stdin but standard C
stdin for input. Is there any reason for this? Why it then uses its encoding
attribute? (Assigning sys.stdin something, that doesn't have encoding attribute
freezes the interpreter
New submission from Drekin:
runpy.run_path(\u222b.py) raises UnicodeEncodeError when trying to use mbcs
codec on Windows. However opening the file using open() is ok. So why is runpy
trying to encode the name using mbcs encoding when it's not necessary or even
correct? See http://bpaste.net
Drekin added the comment:
Hello. I have made a small upgrade of the workaround.
• win_unicode_console.enable_streams() sets sys.stdin, stdout and stderr to
custom filelike objects which use Windows functions ReadConcoleW and
WriteConsoleW to handle unicode data properly. This can be done
Changes by Drekin dre...@gmail.com:
Added file: http://bugs.python.org/file29564/win_unicode_console_3.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1602
Drekin added the comment:
I have finished a solution working for me. It bypasses standard Python
interactive interpreter and uses its own repl based on code.interact(). This
repl is activated by an ugly hack since PYTHONSTARTUP doesn't apply when some
file is run (python -i somefile.py). Why
Drekin added the comment:
Hello, I'm trying to handle Unicode input and output in Windows console and
found this issue. Will this be solved in 3.3 final? I tried to write a solution
(file attached) based on solution here – rewriting sys.stdin and sys.stdout so
it uses ReadConsoleW
71 matches
Mail list logo
