[issue12982] Document that importing .pyo files needs python -O

[Berker Peksag]

Berker Peksag added the comment:

Yes, this can be closed as 'out of date'. I've just updated -O documentation to 
remove the mention of .pyo extension. Thanks!

--
nosy: +berker.peksag
resolution:  -> out of date
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Roundup Robot]

Roundup Robot added the comment:

New changeset 8e9dc3e4ea91 by Berker Peksag in branch '3.5':
Issue #12982: Thanks to PEP 488, Python no longer creates .pyo files
https://hg.python.org/cpython/rev/8e9dc3e4ea91

New changeset 1455851e7332 by Berker Peksag in branch 'default':
Issue #12982: Merge from 3.5
https://hg.python.org/cpython/rev/1455851e7332

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Caleb Hattingh]

Caleb Hattingh added the comment:

Presumably PEP488 (and the 4 years of inactivity) means that this issue could 
be closed?

--
nosy: +cjrh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Eric O. LEBIGOT]

Eric O. LEBIGOT eric.lebi...@normalesup.org added the comment:

Thank you for this lucid account of the situation, Terry.

As for where in the documentation something additional could be said about .pyo 
files and the -O option, I must say that it is already mentioned in some 
relevant places 
(http://docs.python.org/tutorial/modules.html#compiled-python-files),

However, I can see one other place where some additional information would be 
useful: in the documentation for the -O option itself 
(http://docs.python.org/using/cmdline.html#miscellaneous-options). The current 
documentation only mentions the .pyo-producing effect of -O. Mentioning there 
that -O is *required* for interpreting .pyo files would be useful.

Thanks!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Eric O. LEBIGOT]

Eric O. LEBIGOT eric.lebi...@normalesup.org added the comment:

Terry, it seems that the doc I was quoting is for version 1.5.1 
(http://docs.python.org/release/1.5.1p1/tut/node43.html). I can't find it in 
more recent versions of the doc. I should not have quoted an obsolete version 
of the documentation—I'm not sure how this happened. :)

I am not fully sure why -O is essentially required for running .pyo files: why 
not have the Python interpreter handle everything automatically based on the 
extension?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Michael Herrmann]

Michael Herrmann mherrmann...@gmail.com added the comment:

That is *exactly* my point :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Because:

  1) The __debug__ flag is defined to be process-global.  If you test it in one 
module, your code should be able to assume that it has the same value in all 
other modules

  2) python-dev does not support running .pyo code without -O turned on.  In 
the future it might be the case that -O would actually change the behavior of 
the running python interpreter such that .pyo code would fail of -O was not on.

So, you can do the rename or importer trick if you want, and it will work right 
now as long as your program does not depend on __debug__ being globally 
consistent (which would be the case for almost all programs), but we do not 
guarantee it will continue to work in future versions of Python.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Eric Snow]

Eric Snow ericsnowcurren...@gmail.com added the comment:

 I am not fully sure why -O is essentially required for running .pyo
 files: why not have the Python interpreter handle everything
 automatically based on the extension?

In part because it would take work to make it happen and apparently no one 
feels strongly enough about adding that functionality, or convinced enough that 
it's appropriate, to do the work.  If you're interested a good first step would 
be to write a PEP 302 metapath hook (finder that gets inserted to sys.metapath) 
that makes import of .pyo files work even when -O is not used.  Maybe you'd 
also want to have .pyc files work even when -O *is* used.

Keep in mind that there may be other reasons why such functionality would not 
go into the interpreter.  However, the beauty of import hooks is that it 
wouldn't matter once you have yours in hand.

--
nosy: +eric.snow

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Ronan Lamy]

Ronan Lamy ronan.l...@gmail.com added the comment:

Doing it at the interpreter level is trivial (cf. patch), except for an 
annoying bug I noticed (see below). Doing it from user code might require some 
care to avoid disrupting existing import hooks, but AFAICT something like 
sys.path_hooks.append(FileFinder.path_hook(['.pyo', SourcelessFileLoader, 
True])) is supposed to work. 

The bug is that -O has currently no effect on sourceless imports: it seems that 
frozen_importlib actually uses freeze-time __debug__ instead of the current 
interpreter's.

--
keywords: +patch
Added file: http://bugs.python.org/file26008/issue12982.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Terry J. Reedy]

Terry J. Reedy tjre...@udel.edu added the comment:

Eric, can you find a place in the current doc where -O and .pyo are mentioned, 
and where you think a sentence should go. What sentence(s) would you like to 
see.

Other comments:

__debug__ is intended to be a process-global compilation value (implemented as 
a keyword) set on startup

 __debug__
True
 __debug__ = False
SyntaxError: assignment to keyword

The devs are not willing to support having contradictory values in the same 
process. Indeed, since I posted last night, the pydev discussion has moved to 
the question of whether -O, __debug__, and .pyo as now defined are worth the 
nuisance they cause or whether some or all should be deprecated. (Docstring 
stripping for saving space could then be a separate tool.)

---
Python interpreters exist to run Python code. The existence, persistence, and 
other details of compilation caches are version-dependent implementation 
details. Being able to execute from such caches without source present is also 
an implementation detail, and for CPython, it gets secondary support at best. 
(This is a compromise between full support and no support.)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Terry J. Reedy]

Terry J. Reedy tjre...@udel.edu added the comment:

The response from pydev is that running .pyc with -O or running .pyo without is 
not officially supported. Even if mixing usually works now, it does not always 
work properly for code with __debug__, assert, or __doc__. The scope of 
possible malfunctions may increase if ideas for more aggressive optimization 
are implemented. So mixing the two types of cache files is 'do at your own 
risk' and 'don't complain if it does not work now or ceases to work in the 
future'.

A principle reason for the above is that __debug__ == not  -O and that it 
affects complilation, not just execution.

The doc should say the above more clearly. Eric L., where is the doc quote from?

--
components: +Documentation -Interpreter Core

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
title: .pyo file can't be imported unless -O is given - Document that 
importing .pyo files needs python -O

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Michael Herrmann]

Michael Herrmann mherrmann...@gmail.com added the comment:

Hi Eric,

thank you for your quick reply. I'm not the first one who encounters this 
problem and in my opinion it is simply counter-intuitive that you cannot read a 
mixture of .pyo and .pyc files. That is why I think that my proposed change is 
valuable. In the meantime, I will follow your suggestion and try to use the 
imp-module to load the .pyo-files myself. Thank you!

Best regards,
Michael

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Actually it's a lot easier than that, although it is very much a hack: just 
rename the .pyo files to .pyc, and python without -O will happily import them.  
Since the optimization happens when the bytecode is written, this does what you 
want.

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Eric O. LEBIGOT]

Eric O. LEBIGOT eric.lebi...@normalesup.org added the comment:

Hi Michael,

Thank you for your message.

You are mentioning the suggestion of the other Eric (Araujo). My suggestion 
was to rename your .pyo files as .pyc files; it is hackish (according to a 
previous post from Eric Araujo), but might save you some trouble, on the short 
term, as .pyc do not need -O.

Best wishes,

EOL

On Jun 12, 2012, at 21:25, Michael Herrmann wrote:

 
 Michael Herrmann mherrmann...@gmail.com added the comment:
 
 Hi Eric,
 
 thank you for your quick reply. I'm not the first one who encounters this 
 problem and in my opinion it is simply counter-intuitive that you cannot read 
 a mixture of .pyo and .pyc files. That is why I think that my proposed change 
 is valuable. In the meantime, I will follow your suggestion and try to use 
 the imp-module to load the .pyo-files myself. Thank you!
 
 Best regards,
 Michael
 
 --
 
 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue12982
 ___

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Michael Herrmann]

Michael Herrmann mherrmann...@gmail.com added the comment:

Dear Eric OL,

I see - I had read your e-mail but because of the similar names I thought the 
message here was yours too, and thus only replied once. I apologize!

I can of course find a workaround such as renaming .pyo to .pyc. However, I 
would like to avoid having to modify the distribution of the third party 
library I am using in any way. The reason is that if a new version of this 
third party library is released and I have to upgrade to this new version, I 
will again have to manually do all the changes I had to do for the old version 
for the new version, and of course some changes won't work any more and there 
will be problems...

I'm finding it tedious to use the imp-module to read in the .pyo-files by hand 
and might just fall back to your suggestion. Thank you for it. Nevertheless, I 
am still hoping that this may be resolved in a future release.

Best,
Michael

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Terry J. Reedy]

Terry J. Reedy tjre...@udel.edu added the comment:

Michael, you should ask the closed source library distributor to distribute all 
files as .pyc so you have access to docstrings while programming and to avoid 
the problem with reading them. He could also distribute an all-.pyo version. A 
mixture seems accidental.

Is the import restriction currently intended? If a change *is* made, should it 
be regarded as an enhancement rather than a bug fix? I asked both question on 
pydev for more input.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12982] Document that importing .pyo files needs python -O

[Ronan Lamy]

Changes by Ronan Lamy ronan.l...@gmail.com:


--
nosy: +Ronan.Lamy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12982
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com