On Wed, Feb 16, 2011 at 8:17 AM, W. Trevor King <wk...@drexel.edu> wrote:
> First, here's a reminder of my end goal:
>
> On Wed, Feb 09, 2011 at 12:23:25PM -0500, W. Trevor King wrote:
>> Now I want to expose those constants in Python, so I have `expose.pyx`:
>>
>> cimport mylib
>>
>> CONST_A = mylib.CONST_A
>> CONST_B = mylib.CONST_B
>> ...
>>
>> But the last part seems pretty silly. I'd like to do something like
>>
>> cimport mylib
>> import sys
>>
>> for name in dir(mylib):
>> setattr(sys.modules[__name__], name, getattr(mylib, name))
>>
>> which compiles fine, but fails to import with...
>
> The discussion eventually led to:
>
> On Wed, Feb 09, 2011 at 12:22:41PM -0800, Robert Bradshaw wrote:
>> cimport * will pollute expose.pyx's C-level namespace. You can use
>> them just fine there. If you need to access these constants from
>> Python, you do have to explicitly expose them.
>
> In an attempt to make it easier to explicitly expose them to other
> Python modules at compile time, I've been working through the Cython
> source, and found that when cythoning expose.pyx, all the items
> declared in mylib.pxd are stored in `Cython.Compiler.Symtab.Entry`s in
> mylib.pxd's `Cython.Compiler.Symtab.ModuleScope` instance.
>
> By tweaking
> Cython.Compiler.Node.CImportStatNode.generate_function_definitions
> to be
> def generate_function_definitions(self, env, code):
> modules = [m for m in env.cimported_modules if m.module_name ==
> self.module_name]
> if len(modules) > 0:
> module = modules[0]
> for name,entry in module.entries.iteritems():
> code.putln('/* %s.%s */' % (self.module_name, name))
> I can add a comment to extern.c listing all the stuff I'd declared in
> `cdef extern` blocks in mylib.pxd. That's nice, but not very useful
> (it does show, however, that Cython is not "forgetting" about the
> definitions as I previously thought).
>
> What I'm missing is a way to bind the ModuleScope namespace to a name
> in expose.pyx so that commands like `dir(mylib)` and `getattr(mylib,
> name)` will work in expose.pyx. If it would be easier to pre-compute
> the result of these commands and hard-code them in `extern.c` at
> compile time, that's fine.
>
> For example, my comment-generating version of
> `generate_function_definitions` is not far from being able to generate
> the results of `dir(mylib)`, it would just need to wrap the results in
> a list and place references to that list in place of the stock `dir()`
> execution code when the dir argument resolved to the cimported module.
> Yuck ;).
Cython modules have two namespaces, the Python level one and the C
level one. This separation is necessary, as the C level one may
contain objects that are unrepresentable in the Python-level layer,
and get statically bound (e.g. one would want to mark them as
read-only at the very least). Many users also like the fact that the
C-level implementation details of their Cython modules do not get
leaked out to their Python namespaces (though use of __all__ could
provide, but not enforce, similar behavior).
You have also hit into the thorny issue that .pxd files are used for
many things. They may be pure C library declarations with no Python
module backing, they may be declarations of (externally implemented)
Python modules (such as numpy.pxd), or they may be declarations for
Cython-implemented modules.
In terms of your specific question, I don't think hijacking the
builtin dir and getattr to blur this line.
> It seems like it would be easier to generate some kind of wrapper
> class (PxdModule?) for mylib when it is cimported (at compile time),
> and then further interactions would take care of themselves (at run
> time).
>
> Does anyone know how I would go about doing this?
Would such an object be created anew for every module that cimports
the declaration file?
I have toyed with the idea of subclassing the module object itself for
better support of C-level attributes from the Python (and Cython)
namespaces.
Here's another idea, what if extern blocks could contain cpdef
declarations, which would automatically generate a Python-level
wrappers for the declared members (if possible, otherwise an error)?
> On an administrative level, since this would seem to require altering
> the Cython source code, should I move this discussion to cython-devel,
> or does splitting the thread across two lists make it too hard to
> follow?
Yes, it would probably be best to move this thread over there. A
succinct summary of what issue you are trying to solve would probably
be helpful now anyways.
- Robert
_______________________________________________
Cython-dev mailing list
Cython-dev@codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev
