Re: PYDISTUTILS_AUTOPLIST question
On, Sun Jan 26, 2014, Ruslan Makhmatkhanov wrote:
Hi,
while PYDISTUTILS_AUTOPLIST works smoothly for majority of ports, there
is the problem with some of them I come across yesterday. For example,
there is the port devel/py-Products.PasswordResetTool. If I set there
PYDISTUTILS_AUTOPLIST, and try to build, I got packaging error, because
some of the files, that port provides, cannot be compiled (by design).
While they are python code files, they are not supposed to be executed
with python interpreter directly (they are kind of templates).
So, is there any possibility to:
a) Do not add compiled/optimized filenames to packing list, if the
actual file fails to compile?
b) Make a hook, that allows to list file, that should not have
corresponding *.pyc/*.pyo entries in the packing list?
If the port uses distutils, it should not matter, if you do not mess around in
the post-install targets yourself, since the file list will be taken from what
distutils records - we do not implement magic ourselves for distutils-aware
ports (well, a bit :-).
If this does not work properly for you, upstream might do something that
breaks the intended behaviour of distutils.
So, before fixing the symptom, I actually would like to find out, what exactly
they do on installation. Are all files properly recorded in ${_PYTHONPKGLIST}?
What errors do you get, what does the install target complain about, etc.etc.?
Cheers
Marcus
pgpzxWV9qsats.pgp
Description: PGP signature
Re: PYDISTUTILS_AUTOPLIST question
Marcus von Appen wrote on 26.01.2014 15:30:
On, Sun Jan 26, 2014, Ruslan Makhmatkhanov wrote:
Hi,
while PYDISTUTILS_AUTOPLIST works smoothly for majority of ports, there
is the problem with some of them I come across yesterday. For example,
there is the port devel/py-Products.PasswordResetTool. If I set there
PYDISTUTILS_AUTOPLIST, and try to build, I got packaging error, because
some of the files, that port provides, cannot be compiled (by design).
While they are python code files, they are not supposed to be executed
with python interpreter directly (they are kind of templates).
So, is there any possibility to:
a) Do not add compiled/optimized filenames to packing list, if the
actual file fails to compile?
b) Make a hook, that allows to list file, that should not have
corresponding *.pyc/*.pyo entries in the packing list?
If the port uses distutils, it should not matter, if you do not mess around in
the post-install targets yourself, since the file list will be taken from what
distutils records - we do not implement magic ourselves for distutils-aware
ports (well, a bit :-).
If this does not work properly for you, upstream might do something that
breaks the intended behaviour of distutils.
So, before fixing the symptom, I actually would like to find out, what exactly
they do on installation. Are all files properly recorded in ${_PYTHONPKGLIST}?
What errors do you get, what does the install target complain about, etc.etc.?
Cheers
Marcus
For example at install stage it fails with pwreset_constructURL.py file:
byte-compiling
/usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool/work/stage/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.py
to pwreset_constructURL.pyc
File
/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.py,
line 14
return %s/passwordreset/%s % (host, randomstring)
SyntaxError: 'return' outside function
writing byte-compilation script '/tmp/tmpqmG0z8.py'
/usr/local/bin/python2.7 -O /tmp/tmpqmG0z8.py
File
/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.py,
line 14
return %s/passwordreset/%s % (host, randomstring)
SyntaxError: 'return' outside function
removing /tmp/tmpqmG0z8.py
And here is what on package stage:
=== Building package for py27-Products.PasswordResetTool-2.0.15_1
pkg-static:
lstat(/usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool/work/stage/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.pyc):
No such file or directory
pkg-static:
lstat(/usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool/work/stage/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.pyo):
No such file or directory
*** Error code 1
Stop.
make: stopped in
/usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool
As we can see *.pyc/*.pyo pkg-plist entries were generated for
pwreset_constructURL.py despite the fact that byte-compiling had failed.
--
Regards,
Ruslan
T.O.S. Of Reality
___
freebsd-python@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-python
To unsubscribe, send any mail to freebsd-python-unsubscr...@freebsd.org
Re: PYDISTUTILS_AUTOPLIST question
On, Sun Jan 26, 2014, Ruslan Makhmatkhanov wrote:
Marcus von Appen wrote on 26.01.2014 15:30:
On, Sun Jan 26, 2014, Ruslan Makhmatkhanov wrote:
Hi,
while PYDISTUTILS_AUTOPLIST works smoothly for majority of ports, there
is the problem with some of them I come across yesterday. For example,
there is the port devel/py-Products.PasswordResetTool. If I set there
PYDISTUTILS_AUTOPLIST, and try to build, I got packaging error, because
some of the files, that port provides, cannot be compiled (by design).
While they are python code files, they are not supposed to be executed
with python interpreter directly (they are kind of templates).
So, is there any possibility to:
a) Do not add compiled/optimized filenames to packing list, if the
actual file fails to compile?
b) Make a hook, that allows to list file, that should not have
corresponding *.pyc/*.pyo entries in the packing list?
If the port uses distutils, it should not matter, if you do not mess around
in
the post-install targets yourself, since the file list will be taken from
what
distutils records - we do not implement magic ourselves for distutils-aware
ports (well, a bit :-).
If this does not work properly for you, upstream might do something that
breaks the intended behaviour of distutils.
So, before fixing the symptom, I actually would like to find out, what
exactly
they do on installation. Are all files properly recorded in
${_PYTHONPKGLIST}?
What errors do you get, what does the install target complain about,
etc.etc.?
Cheers
Marcus
For example at install stage it fails with pwreset_constructURL.py file:
byte-compiling
/usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool/work/stage/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.py
to pwreset_constructURL.pyc
File
/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.py,
line 14
return %s/passwordreset/%s % (host, randomstring)
SyntaxError: 'return' outside function
writing byte-compilation script '/tmp/tmpqmG0z8.py'
/usr/local/bin/python2.7 -O /tmp/tmpqmG0z8.py
File
/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.py,
line 14
return %s/passwordreset/%s % (host, randomstring)
SyntaxError: 'return' outside function
removing /tmp/tmpqmG0z8.py
And here is what on package stage:
=== Building package for py27-Products.PasswordResetTool-2.0.15_1
pkg-static:
lstat(/usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool/work/stage/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.pyc):
No such file or directory
pkg-static:
lstat(/usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool/work/stage/usr/local/lib/python2.7/site-packages/Products/PasswordResetTool/skins/PasswordReset/pwreset_constructURL.pyo):
No such file or directory
*** Error code 1
Stop.
make: stopped in
/usr/local/poudriere/ports/default/devel/py-Products.PasswordResetTool
As we can see *.pyc/*.pyo pkg-plist entries were generated for
pwreset_constructURL.py despite the fact that byte-compiling had failed.
Thanks - I just checked, what exactly happens within the record magic for
those cases. distutils (and thus setuptools, since it does not do anything
special in that area) just picks up the .py files and adds entries for
matching .pyc and .pyo files without actually checking, if those files
could be compiled successfully.
The code snippet (distutils/install_lib.py, _bytecode_filenames()) actually
needs to verify, if .pyc/.pyo files exist, since the bytecode compiler
function (distutils/util.py, byte_compile() in indirect mode)
does not break on errors, but silently proceeds.
Rather than adding a hack to bsd.python.mk to filter out such entries, I'd
like to see that issue fixed upstream: http://bugs.python.org/issue20397.
My current (quick'n'dirty) idea is to add a simple file existence verification
check into our currently supported versions.
Cheers
Marcus
pgpHyT5o3pfkA.pgp
Description: PGP signature
Re: PYDISTUTILS_AUTOPLIST question
Marcus von Appen wrote on 26.01.2014 18:22: As we can see *.pyc/*.pyo pkg-plist entries were generated for pwreset_constructURL.py despite the fact that byte-compiling had failed. Thanks - I just checked, what exactly happens within the record magic for those cases. distutils (and thus setuptools, since it does not do anything special in that area) just picks up the .py files and adds entries for matching .pyc and .pyo files without actually checking, if those files could be compiled successfully. The code snippet (distutils/install_lib.py, _bytecode_filenames()) actually needs to verify, if .pyc/.pyo files exist, since the bytecode compiler function (distutils/util.py, byte_compile() in indirect mode) does not break on errors, but silently proceeds. Rather than adding a hack to bsd.python.mk to filter out such entries, I'd like to see that issue fixed upstream: http://bugs.python.org/issue20397. My current (quick'n'dirty) idea is to add a simple file existence verification check into our currently supported versions. Cheers Marcus Thank you for this. I though this is something under our control. Let's see what guys at python.org say. -- Regards, Ruslan T.O.S. Of Reality ___ freebsd-python@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-python To unsubscribe, send any mail to freebsd-python-unsubscr...@freebsd.org
