[issue15178] Doctest should handle situations when test files are not readable

[Mark Lawrence]

Change by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
stage: patch review - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Doctest still failed with backtrace if argument is a file name.

$ ./python -m doctest aaa.py
Traceback (most recent call last):
  File /home/serhiy/py/cpython/Lib/runpy.py, line 170, in _run_module_as_main
__main__, mod_spec)
  File /home/serhiy/py/cpython/Lib/runpy.py, line 85, in _run_code
exec(code, run_globals)
  File /home/serhiy/py/cpython/Lib/doctest.py, line 2795, in module
sys.exit(_test())
  File /home/serhiy/py/cpython/Lib/doctest.py, line 2773, in _test
m = __import__(filename[:-3])
ImportError: No module named 'aaa'

It would be good to have tests.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Bohuslav Slavek Kabrda]

Bohuslav Slavek Kabrda added the comment:

Attaching a new version of patch:
- Rebased to latest default branch
- Simplified prints
- Using OSError instead of IOError

Hopefully this is the final version :)

--
Added file: 
http://bugs.python.org/file37421/doctest-dont-end-with-exception-on-unreadable-files-v6.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
stage:  - patch review
versions: +Python 3.4, Python 3.5 -Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Added few nitpicks on Rietveld.

--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Mark Lawrence]

Mark Lawrence added the comment:

@David can you pick this up given your previous involvement?

--
nosy: +BreamoreBoy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Raymond Hettinger]

Changes by Raymond Hettinger raymond.hettin...@gmail.com:


--
nosy: +tim.peters

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Bohuslav Slavek Kabrda]

Bohuslav Slavek Kabrda bkab...@redhat.com added the comment:

Fifth version :)

- On failure in a loaded test, the _test function returns, so this behaviour is 
preserved.
- During _test function, count of both loaded and non-loaded files is kept.
- If a file fails to be loaded, the tests continue, but a non-zero return code 
is returned. Moreover, the results of loaded and non-loaded files are printed 
after each invocation.

This is a sample output:


$ python3 -m doctest -v spam c.py beans
Cannot read 'spam': [Errno 2] No such file or directory: 'spam'
Trying:
2 * 2
Expecting:
4
ok
1 items passed all tests:
   1 tests in c
1 tests in 1 items.
1 passed and 0 failed.
Test passed.
Cannot read 'beans': [Errno 2] No such file or directory: 'beans'
Test files read successfully: 1
Unreadable files: 2


Does this look better?

--
Added file: 
http://bugs.python.org/file26239/doctest-dont-end-with-exception-on-unreadable-files-v5.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

I think that sounds reasonable.  The message should say files in all three 
cases', since the individual  test file summary method will be similar but 
talking about numbers of tests.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Bohuslav Slavek Kabrda]

Bohuslav Slavek Kabrda bkab...@redhat.com added the comment:

So maybe an optimal solution would be to add a note summarizing this to the end 
of output? I mean that this:

X passed and Y failed.

Could be replaced by:

X passed and Y failed, Z files were not loaded.

Then the user will know that the files were not loaded and he can search the 
test output for the reasons.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Terry J. Reedy]

Terry J. Reedy tjre...@udel.edu added the comment:

Failure modes tend to get less attention that successful behavior. If I wrote a 
program that used doctest/unittest to test multiple files, I should like it to 
run an many as possible. If a filename is bad, print name as usual, say 
'aborted', and '0/0 tests' run. Then at the end tell me via return value or 
exception about problems. In any case, having error behavior documented and 
consistent between modules would be good.  I am not sure how much of this would 
be a bugfix versus enhancement.

--
nosy: +terry.reedy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

The  problem with running all the files as things stand is that the errors get 
lost in the other output.  Changing that would definitely be an enhancement.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Bohuslav Slavek Kabrda]

Bohuslav Slavek Kabrda bkab...@redhat.com added the comment:

So I figured it might be best to first agree on the actual behaviour (what the 
patch will look like) and then I can write the tests.

So here is my 3rd version:
- It seems that returning 1 only if last file fails is intentional, as it is 
_inside_ the loop, so unsuccessful test immediately terminates the _test 
function - I thought it might be better to carry on with all of the tests, so 
here is what I did:
- I made a variable failures, which represents if there were any failures 
during tests;
- I took the return 1 one indentation level down, so all files are now 
traversed no matter how many of them fail;
- An error message is printed and failures set to True if a file is 
unopenable.

Does this look acceptable? If not, I will happily work further :) (and provide 
the tests once the behaviour is clear).

Thanks!

--
Added file: 
http://bugs.python.org/file26165/doctest-dont-end-with-exception-on-unreadable-files-v3.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Ah, right, I misread the code when I looked at it.

There is usually a reason why something is done the way it is...though not 
always.  But, running some example command lines, it looks to me like the 
current behavior is there because without it, you won't notice if the tests in 
the middle of your list of files failed.  So, I think your second patch is more 
what we need.  However, rather than calling sys.exit, I think it would be 
better to just print the message and do a return 1.

It is also possible we shouldn't change it at all.  unittest (which has had 
some thought put in to the commnd line interface) just ends in a traceback in a 
similar situation (except that in unittest's case it is an ImportError...)

--
nosy: +michael.foord

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Bohuslav Slavek Kabrda]

Bohuslav Slavek Kabrda bkab...@redhat.com added the comment:

Ok, attaching 4th version :)

I think it is nice when the testing library can react in situations like this 
one and not show a traceback. IMHO it is always nicer to display a pretty 
user-readable message than ending with a traceback.

--
Added file: 
http://bugs.python.org/file26172/doctest-dont-end-with-exception-on-unreadable-files-v4.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Bohuslav Slavek Kabrda]

New submission from Bohuslav Slavek Kabrda bkab...@redhat.com:

Hi,
I think that doctest should be able to handle situations when the file that is 
being tested does not exist/is unreadable/etc...
Currently, doctest raises an IOError exception and the whole Python ends with 
an exception and backtrace.
I'm attaching a patch that fixes this and prints a simple description of what 
has gone wrong.

Thanks for considering!

--
components: Library (Lib)
files: doctest-dont-end-with-exception-on-unreadable-files.patch
keywords: patch
messages: 163936
nosy: bkabrda
priority: normal
severity: normal
status: open
title: Doctest should handle situations when test files are not readable
type: behavior
versions: Python 2.7
Added file: 
http://bugs.python.org/file26153/doctest-dont-end-with-exception-on-unreadable-files.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

In general in Python we let exceptions speak for themselves.  It would, 
however, probably be reasonable to add the try/except and error message in 
_test, which should only be called when the module is run as a script.

--
nosy: +r.david.murray
versions: +Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Bohuslav Slavek Kabrda]

Bohuslav Slavek Kabrda bkab...@redhat.com added the comment:

Thanks for the comment David. I'm attaching second version that does the same 
in the _test function.

--
Added file: 
http://bugs.python.org/file26155/doctest-dont-end-with-exception-on-unreadable-files-v2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Great, thanks.

There's a loop around testfiles.  I think it would be nicer to not call 
sys.exit.  You could at the same time fix what appears to be a bug in the loop 
code.  It looks like 1 is currently returned only if the last file fails.

Hmm, and given that there is a bug, adding tests would be great if you are up 
for it.  I don't think there are currently any tests of the doctest command 
line functionality.  That's more complicated than writing the fix, though :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15178] Doctest should handle situations when test files are not readable

[Bohuslav Slavek Kabrda]

Bohuslav Slavek Kabrda bkab...@redhat.com added the comment:

Sure, if you give me some time, I'll try to do everything as you suggest.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15178
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com