"Maciej (Matchek) Bliziński" <[email protected]> writes: > 2013/7/29 Peter FELECAN <[email protected]>: >> "Maciej (Matchek) Bliziński" <[email protected]> writes: >>> I did not implement compiling one .py file multiple times for >>> different Python versions. >> >> Do you plan to do it later? > > No, unless we figure out and agree on how to do it. > >>> The main concern for the current CAS is the suitability of pattern >>> matching. Is it good enough to just match a part of the path to know >>> which interpreter to use? It should be good enough for most packages, >>> but I'm not sure what the corner cases / contrived path names could >>> be. >> >> Even if it misses some is not blocking as the code is there. Anyhow, the >> difference between .py and .pyc is not so great (and I don't speak about >> .pyo which seems to me a scam), in my experience and if I understood the >> mechanism of dynamic compilation the overhead is incurred only >> once. This is why supplying the compiled components in the package >> seemed to me a very good solution. > > I'm wondering why Debian's policy is so strict about not shipping > them. The document doesn't explain the reasons, unfortunately, but > there must be some. If we learn what the reasons are and it turns out > they are not relevant for us, we might well do as you're suggesting. I > found a Python packaging FAQ, the question wasn't on the list. > > http://wiki.debian.org/Python/FAQ > > Is anyone onboard a Debian developer and can provide any pointers?
I'm not, but using the following search equation: "pyc portability" on Google and after a cursory exploration of the proposed hits on the first page I think that the reason for which Debian imposes this is raised by network shares between systems running different interpreters, version wise. The only thing that I didn't do is to ask on the good mailing list which, by the way, I don't know. Here follows the salient citations and my observations: ** [[http://stackoverflow.com/questions/2263356/are-python-2-5-pyc-files-compatible-with-python-2-6-pyc-files][stackoverflow]] "In general, .pyc files are specific to one Python version (although portable across different machine architectures, as long as they're running the same version); the files carry the information about the relevant Python version in their headers -- so, if you leave the corresponding .py files next to the .pyc ones, the .pyc will be rebuilt every time a different Python version is used to import those modules." ** [[http://docs.python.org/2/tutorial/modules.html#compiled-python-files][tutorial]] 6.1.3. "Compiled" Python files As an important speed-up of the start-up time for short programs that use a lot of standard modules, if a file called spam.pyc exists in the directory where spam.py is found, this is assumed to contain an already-"byte-compiled" version of the module spam. The modification time of the version of spam.py used to create spam.pyc is recorded in spam.pyc, and the .pyc file is ignored if these don't match. Normally, you don't need to do anything to create the spam.pyc file. Whenever spam.py is successfully compiled, an attempt is made to write the compiled version to spam.pyc. It is not an error if this attempt fails; if for any reason the file is not written completely, the resulting spam.pyc file will be recognized as invalid and thus ignored later. The contents of the spam.pyc file are platform independent, so a Python module directory can be shared by machines of different architectures. Some tips for experts: 1. When the Python interpreter is invoked with the -O flag, optimized code is generated and stored in .pyo files. The optimizer currently doesn't help much; it only removes assert statements. When -O is used, all bytecode is optimized; .pyc files are ignored and .py files are compiled to optimized bytecode. 2. Passing two -O flags to the Python interpreter (-OO) will cause the bytecode compiler to perform optimizations that could in some rare cases result in malfunctioning programs. Currently only __doc__ strings are removed from the bytecode, resulting in more compact .pyo files. Since some programs may rely on having these available, you should only use this option if you know what you're doing. 3. A program doesn't run any faster when it is read from a .pyc or .pyo file than when it is read from a .py file; the only thing that's faster about .pyc or .pyo files is the speed with which they are loaded. 4. When a script is run by giving its name on the command line, the bytecode for the script is never written to a .pyc or .pyo file. Thus, the startup time of a script may be reduced by moving most of its code to a module and having a small bootstrap script that imports that module. It is also possible to name a .pyc or .pyo file directly on the command line. 5. It is possible to have a file called spam.pyc (or spam.pyo when -O is used) without a file spam.py for the same module. This can be used to distribute a library of Python code in a form that is moderately hard to reverse engineer. 6. The module compileall can create .pyc files (or .pyo files when -O is used) for all modules in a directory. Item 6 refers to distributing code. ** [[http://www.network-theory.co.uk/docs/pytut/CompiledPythonfiles.html][An Introduction to Python]] Guido van Rossum himself says: "[...]The contents of the `spam.pyc' file are platform independent, so a Python module directory can be shared by machines of different architectures." ** [[http://www.velocityreviews.com/forums/t643545-are-pyc-files-portable.html][forum discussion]] Discussion related on usage of .pyc on different versions of Solaris. The conclusion is that architecture doesn't matter, interpreter matter. ** [[http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-byte_compilation][Debian Python Policy]] The discussion has more to do with interpreter version changes than with architecture. ** [[http://www.ehow.com/info_12214796_pyc-files.html][random]] "Uses for ".PYC" Files Modules that are imported into user scripts get compiled by the interpreter before execution. Because these modules tend to undergo repeated use, the interpreter compiles the module and stores the ".pyc" file in a directory. This way, when a script imports that module, the bytecode version already exists, ready for use. Furthermore, bytecode ".pyc" files are portable across multiple platforms, making pre-compiling Python scripts useful for distributing Python programs across different operating systems." From my standpoint, here is the exceptional case where Debian is not to be followed. Note that they have a quite similar policy of Emacs Lisp pre-compiled files which is of the same order. What we should do is to supply the compiled files in the package, for each version for which the maintainer thinks that it's adequate. Simpler and leaner, isn't it? -- Peter _______________________________________________ maintainers mailing list [email protected] https://lists.opencsw.org/mailman/listinfo/maintainers .:: This mailing list's archive is public. ::.
