[issue14929] IDLE crashes on *Edit / Find in files ...* command

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Opened #36058 for followup.

--
resolution:  -> fixed
stage: test needed -> resolved
status: open -> closed
superseder:  -> Improve file decoding before re.search
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14929] IDLE crashes on *Edit / Find in files ...* command

[Terry J. Reedy]

Terry J. Reedy added the comment:

As the 'crash' seems to have been solved, this should probably be closed and a 
new issue about encodings opened. Also, since the re module used to search 
lines within files can work with both bytes and strings, there might be an 
option to search undecoded bytes.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14929
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14929] IDLE crashes on *Edit / Find in files ...* command

[Terry J. Reedy]

Changes by Terry J. Reedy tjre...@udel.edu:


--
stage: needs patch - test needed
versions: +Python 3.4, Python 3.5 -Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14929
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14929] IDLE crashes on *Edit / Find in files ...* command

[Terry J. Reedy]

Terry J. Reedy added the comment:

The 'In files:' field specifies the search directory. I opened #21960 about it 
being more informative when using Find in Files from the Shell window.

Perfect encoding detection is a fantasy; decent encoding detection by 
heuristics is slow and not something to do when searchings 100s, 1000s, or more 
files. What would be reasonable is to check for an encoding cookie. Idle 
already does this when opening a .py file in the editor. That code should be 
encapsulated if it is not and reused.

An encoding field for non-python files would be possible, but is a lesser 
priority to me.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14929
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14929] IDLE crashes on *Edit / Find in files ...* command

[Francisco Gracia]

Francisco Gracia fgragu...@gmail.com added the comment:

While your are at it, here is another suggestion: what the *Find in files ...* 
dialog needs most urgently in my opinion is a field for specifying clearly the 
directory from which the user wants to launch the search.

Also in my modest opinion, having an input field for encondings would be good, 
although detecting the encodings of the handled files is something that any 
self respecting software should reliably do by itself.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14929
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14929] IDLE crashes on *Edit / Find in files ...* command (Python 3.2, Windows XP)

[Francisco Gracia]

New submission from Francisco Gracia fgragu...@gmail.com:

There is little more that I can add to the title statement.

   1. Start IDLE
   2. Go to *Edit* menu option
   3. Select *Find in files...* option
   4. Put some word in the *Find* input box
   5. Press *Search files* button

When the command is issued the disappearance of IDLE's window is so quick that 
almost nothing can be seen. There is however a hint that a new window pops up 
with some message, that I have been unable to read. With some input strings or 
if the search is happening in a directory with many *.py files a quickly moving 
list of messages shows in this window, but the final result is the same: IDLE 
disappears immediately.

--
components: IDLE
messages: 161727
nosy: fgracia
priority: normal
severity: normal
status: open
title: IDLE crashes on *Edit / Find in files ...* command (Python 3.2, Windows 
XP)
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14929
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14929] IDLE crashes on *Edit / Find in files ...* command

[Roger Serwy]

Roger Serwy roger.se...@gmail.com added the comment:

When running IDLE from the terminal on Ubuntu, I get the following error:

Exception in Tkinter callback
Traceback (most recent call last):
  File /home/serwy/python/cpython/Lib/tkinter/__init__.py, line 1442, in 
__call__
return self.func(*args)
  File ./idlelib/GrepDialog.py, line 70, in default_command
self.grep_it(prog, path)
  File ./idlelib/GrepDialog.py, line 90, in grep_it
block = f.readlines(10)
  File /home/serwy/python/cpython/Lib/codecs.py, line 300, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 120: 
invalid continuation byte

The reason why IDLE closes suddenly on Windows is described in issue13582.

--
nosy: +serwy, terry.reedy
title: IDLE crashes on *Edit / Find in files ...* command (Python 3.2, Windows 
XP) - IDLE crashes on *Edit / Find in files ...* command
type:  - behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14929
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14929] IDLE crashes on *Edit / Find in files ...* command

[Roger Serwy]

Roger Serwy roger.se...@gmail.com added the comment:

The GrepDialog opens a file using plain open, without specifying the encoding 
or how to handle errors. The docs for open says that the default encoding is 
platform dependent (whatever locale.getpreferredencoding() returns)... This 
can be problematic, as files can have different encodings and GrepDialog at 
present has no way to detect file encodings.

Attached is a preliminary patch to replace code points if the default decoder 
encounters errors.

--
keywords: +patch
Added file: http://bugs.python.org/file25740/prelim_14929.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14929
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14929] IDLE crashes on *Edit / Find in files ...* command

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 12454f78967b by Terry Jan Reedy in branch '3.2':
Issue14929: Stop Idle 3.x from closing on Unicode decode errors when grepping.
http://hg.python.org/cpython/rev/12454f78967b

New changeset 058c3099e82d by Terry Jan Reedy in branch 'default':
Merge 3.2 #14929
http://hg.python.org/cpython/rev/058c3099e82d

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14929
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14929] IDLE crashes on *Edit / Find in files ...* command

[Terry J. Reedy]

Terry J. Reedy tjre...@udel.edu added the comment:

Interesting -- and nasty. I have used this successfully with both 3.2 and now 
3.3 on Win7 to search idlelib/*.py files and on a directory of my own files, 
all written by Idle and been quite pleased with the speed.

I just tried searching /Lib/*.py for 'itertools' and quickly got 10 results (up 
to decimal.py) before Idle disappeared. OK, import idlelib/idle in console, 
repeat, and I get the same traceback up to

  File C:\Programs\Python33\lib\encodings\cp1252.py, line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 1877: 
character maps to undefined

This is not a problem on 2.7.

Patch fixes problem, so I committed it so it will be in 3.3.0a4 (in a few days).

The default extension is .py. The default encoding for .py files is utf-8. I 
think that is the default for what Idle writes. So I think this should be the 
default encoding (explicitly given) at least for .py files.

--
assignee:  - terry.reedy
versions: +Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14929
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14929] IDLE crashes on *Edit / Find in files ...* command

[Terry J. Reedy]

Terry J. Reedy tjre...@udel.edu added the comment:

Hit send too soon;-). With that much done, we can think about a more complete 
fix. See last paragraph above. Also, perhaps dialog box could have encodings 
field. People should be able to grep python code for any legal identifier, and 
this means proper decoding according to the encoding they actually use.

Francisco, thanks for reporting this, and Roger, thanks for quick fix.

--
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14929
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com