[sage-support] Re: Help on creating packages/modules
Dear Sage-Support,
off-list, Robert pointed me to the source of the problem.
In my original setup.py, I used the optional parameter ext_package,
according to http://docs.python.org/distutils/setupscript.html:
-
If you have a number of extensions all in the same package (or all
under the same base package), use the ext_package keyword argument to
setup(). For example,
setup(...,
ext_package='pkg',
ext_modules=[Extension('foo', ['foo.c']),
Extension('subpkg.bar', ['bar.c'])],
)
will compile foo.c to the extension pkg.foo, and bar.c to
pkg.subpkg.bar.
Robert pointed out that this is not supported by Cython. With
examples, we found that it *is* supported if one only has .pyx-files,
but it fails as soon as one has .pxd-files. One has to provide the
fully qualified package name.
Moreover, in order to get the right package name hardcoded into the
modules, the source directory names have to match the package name.
I changed my sources and setup.py accordingly, removed the build/
directory that was created by python setup.py install, and tried
again -- and still it failed!
Robert then pointed out that it does not suffice to remove (or empty)
the build/ directory --- I was not aware that the Cython-generated c-
files for the extension modules reside in the source directory. They
have to be removed, too, before rebuilding.
Now it works, including pickling.
Thank you, Robert!
Simon
--~--~-~--~~~---~--~~
To post to this group, send email to sage-support@googlegroups.com
To unsubscribe from this group, send email to
sage-support-unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sage-support
URLs: http://www.sagemath.org
-~--~~~~--~~--~--~---
[sage-support] Re: Help on creating packages/modules
On May 1, 2009, at 4:16 AM, Simon King wrote:
Dear Supporters,
I am about to create a sage package pGroupcohomology comprising
several (extension) modules mtx, resolution, ...
From various Python and Cython manual pages, I thought that I ought to
write the following in my setup.py:
packages=[pGroupCohomology],
ext_package=pGroupCohomology, # Here I am not sure: Necessary?
No, I don't think you need that.
ext_modules=[
Extension(pGroupCohomology.mtx, ...),
Extension(pGroupCohomology.resolution,...)], ...
After installation, the following works:
sage: from pGroupCohomolog.mtx import MTX
sage: M=MTX('some data')
I can do computations with M. But M does not know where it belongs to:
sage: M.__class__
mtx.MTX class instance at ...
I am puzzled by the fact that it says mtx.MTX and not
pGroupCohomology.mtx.MTX, which also makes pickling fail.
Can you tell me why the above setup does not yield the desired package
structure?
Is everything in a pGroupCohomology directory with an __init__.py
file?
- Robert
--~--~-~--~~~---~--~~
To post to this group, send email to sage-support@googlegroups.com
To unsubscribe from this group, send email to
sage-support-unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sage-support
URLs: http://www.sagemath.org
-~--~~~~--~~--~--~---
[sage-support] Re: Help on creating packages/modules
Dear Robert, On 1 Mai, 21:14, Robert Bradshaw rober...@math.washington.edu wrote: ... Is everything in a pGroupCohomology directory with an __init__.py file? Yes (and please excuse that I wrote both to Cython-dev and Sage- devel). - In SAGE_LOCAL/lib/python2.5/site-packages/ I have a folder pGroupCohomology. - It contains __init__.py, mtx.so and some other .so - When I wrote my original post, I accidentally had another copy of mtx.so in the site-packages folder (not a subfolder). I removed it, and now the pickling error is a bit different: PicklingError: Can't pickle mtx.MTX_unpickle_class: import of module mtx failed Explanation of the Pickling errors: 1 The module pGroupCohomology.mtx believes that it is simply mtx, without being contained in a package. 2 Hence, when pickling, the MTX_unpickle_class is looked up in mtx (not pGroupCohomology.mtx). 3 When there was an abandoned mtx.so in the site-packages, it was possible to find mtx.MTX_unpickle_class, but of course it was not identic with pGroupCohomology.mtx.MTX_unpickle_class --- this explains the error message from my first post 4 Now, there is no mtx.so in the site-packages (only in a sub-folder). Hence, looking up mtx.MTX_unpickle_class fails, therefore the new error message. Anyway. It seems the key problem is that the modules in the package pGroupCohomology are unaware that they belong to a package. And I have no clue why. Cheers, Simon --~--~-~--~~~---~--~~ To post to this group, send email to sage-support@googlegroups.com To unsubscribe from this group, send email to sage-support-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-support URLs: http://www.sagemath.org -~--~~~~--~~--~--~---
[sage-support] Re: Help on creating packages/modules
Hi! On 1 Mai, 21:46, simon.k...@uni-jena.de wrote: Anyway. It seems the key problem is that the modules in the package pGroupCohomology are unaware that they belong to a package. And I have no clue why. The thread here seems relevant: http://www.mail-archive.com/cython-...@codespeak.net/msg02790.html It starts with the same observation that when a cython module foo with a class Foo is part of a package pkg then one has pkg.foo.Foo ().__module__ == 'foo', while it should be == 'pkg.foo'. It is also said (if I understand correctly) that it is a pyrex vs. cython issue. The difference is that the poster also wants to be able to fix the fully qualified module name at run time, not at compile time. I would be happy to fix the fully qualified module name at compile time --- but I don't know how, since Cython apparently gets it wrong. Meanwhile I wonder how all the extension modules in Sage get their fully qualified names. After Robert's advice off list, I changed the folder names in my sources so that it reflects the intended package- module structure. Now I have spkg-install src/ setup.py pGroupCohomology/ (before, i used CohoSrc/) __init__.py mtx.pyx ... In setup.py, I have name=pGroupCohomology, packages=[pGroupCohomology], ext_modules=[... Extension(pGroupCohomology.mtx, sources = [pGroupCohomology/mtx.pyx], libraries = [mtx], include_dirs = [mtx2.2.3/src/, pGroupCohomology] ), ... ] But still mtx does not now that it is part of the pGroupCohomology package. And I don't see a fundamental difference to all the 100s extensions in Sage. Is setup.py not the right place to determine the package-module structure? Is it needed to hard-code the fully qualified name into mtx.pyx? But how? Cheers, Simon --~--~-~--~~~---~--~~ To post to this group, send email to sage-support@googlegroups.com To unsubscribe from this group, send email to sage-support-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-support URLs: http://www.sagemath.org -~--~~~~--~~--~--~---
