[C++-sig] dynamic compile and "to-Python converter ... second conversion method ignored"
Hi, I'm using boost.python to interface with some dynamically generated C++ code, employing some scipy.weave facilities to cache the results. When I force a recompile and import, I get the following message: RuntimeWarning: to-Python converter for __randint already registered; second conversion method ignored. where __randint is the dynamically generated class. As such, the changes to the code (__randint) are not represented in the newly imported module. Is it possible to override this behaviour so that the new class replaces the registration for the old one? Thanks. cheers, Eilif -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 ___ Cplusplus-sig mailing list [email protected] http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] dynamic compile and "to-Python converter ... second conversion method ignored"
On 11/20/2009 05:50 AM, Eilif Mueller wrote: Hi, I'm using boost.python to interface with some dynamically generated C++ code, employing some scipy.weave facilities to cache the results. When I force a recompile and import, I get the following message: RuntimeWarning: to-Python converter for __randint already registered; second conversion method ignored. where __randint is the dynamically generated class. As such, the changes to the code (__randint) are not represented in the newly imported module. Is it possible to override this behaviour so that the new class replaces the registration for the old one? I very much doubt that what you want is possible: All extension modules using boost.python share a single global 'registry' that holds type-conversion information. Thus it is impossible to have that loaded while a separate process is adding new entries to it. (It's a global symbol that is seen by all DSOs. Having two DSOs provide that symbol, but with different content, is illegal.) Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... ___ Cplusplus-sig mailing list [email protected] http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] dynamic compile and "to-Python converter ... second conversion method ignored"
Stefan Seefeld wrote:
On 11/20/2009 05:50 AM, Eilif Mueller wrote:
Hi,
I'm using boost.python to interface with some dynamically generated
C++ code, employing some scipy.weave facilities to cache the
results. When I force a recompile and import, I get the following
message:
RuntimeWarning: to-Python converter for __randint already
registered; second conversion method ignored.
where __randint is the dynamically generated class. As such, the
changes to the code (__randint) are not represented in the newly
imported module.
Is it possible to override this behaviour so that the new class
replaces the registration for the old one?
I very much doubt that what you want is possible: All extension
modules using boost.python share a single global 'registry' that
holds type-conversion information. Thus it is impossible to have that
loaded while a separate process is adding new entries to it.
(It's a global symbol that is seen by all DSOs. Having two DSOs
provide that symbol, but with different content, is illegal.)
I'd like to fix this, now, it is starting to cause me pain.
For instance, it also rears its ugly head if you import two libraries
that both wrap std::vector, for instance if you try to use the
IceTray physics analysis suite together with Paul Kunz's wonderful
Hippodraw.
Another situation that came up recently was that the pair inside
std::map and another container of T,U (that used pair
internally) collided. I think it should be possible to specify that a
type's wrapper is replaceable (if you know, for instance, that two
projects wrap std::pair the same way).
I haven't thought about the interface much yet. Here's something I just
decided against, a per-type global 'replacability' flag, that warns or not:
template
void replacable(bool quietly);
BOOST_PYTHON_MODULE(mymod)
{
replacable >(true);
}
But this doesn't help the situation where you're trying to use two
different python modules together, neither of which specifies that pairs
are replacable. It seems that boost.python should expose some
way to specify replacability at runtime. How about this. A type's
converters are 'locked', 'verbose', and 'nonthrowing' by default (like
they are now).
Boost python modules expose one function
def __lock__(type, lock, verbose, throw): ...
at module scope that allows you to specify whether
- types are locked down or replacable
- collisions should be complained about to stdout
- collision events should throw
e.g.
>>> import M1
# silently replace T's
>>> M1.__lock__(M1.T, lock=False, verbose=False, throw=False)
# verbosely refuse replacement of tees
>>> M1.__lock__(M1.T, lock=True, verbose=True, throw=False)
And that this __lock__ function would also be available on the C++ side
for setting defaults:
BOOST_PYTHON_MODULE(M1)
{
class_("T")
...
lock(false, false, false); // silent, replacable, nonthrowing
}
Just a sketch. Thoughts?
-t
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] dynamic compile and "to-Python converter ... second conversion method ignored"
On 11/20/2009 11:47 AM, troy d. straszheim wrote:
Stefan Seefeld wrote:
(It's a global symbol that is seen by all DSOs. Having two DSOs
provide that symbol, but with different content, is illegal.)
I'd like to fix this, now, it is starting to cause me pain.
For instance, it also rears its ugly head if you import two libraries
that both wrap std::vector, for instance if you try to use the
IceTray physics analysis suite together with Paul Kunz's wonderful
Hippodraw.
Another situation that came up recently was that the pair inside
std::map and another container of T,U (that used pair
internally) collided. I think it should be possible to specify that a
type's wrapper is replaceable (if you know, for instance, that two
projects wrap std::pair the same way).
I haven't thought about the interface much yet.
It's an interesting thought, but it raises a lot of questions, and opens
a huge can of worms. (Welcome in the world of 'DLL hell' !)
I'm not sure what angle to attack this problem from. In particular, I
believe before thinking about interface issues, we should clarify the
problem domain, and in particular, what semantics we want.
For example, what does 'replaceable' mean ? Does it mean two bindings
are identical or equivalent ? Does it merely mean the user allows the
mapping to change ? (This would lead into undefined behavior, if you
don't carefully control in which order things are loaded / initialized.)
A different approach might be to avoid collisions by 'scoping'
conversion operators, so they are only available to specific extension
modules. May a converter registry be explicitly imported into a module,
during initialization ?
That would involve making dependency between extension modules explicit
("module X imports converters provided by module Y"). In general, I
think that would be cleanest. "Explicit is better than implicit." I'm
sure David would like that. ;-)
Regards,
Stefan
--
...ich hab' noch einen Koffer in Berlin...
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] dynamic compile and "to-Python converter ... second conversion method ignored"
Stefan Seefeld wrote:
It's an interesting thought, but it raises a lot of questions, and opens
a huge can of worms. (Welcome in the world of 'DLL hell' !)
I'm not sure what angle to attack this problem from. In particular, I
believe before thinking about interface issues, we should clarify the
problem domain, and in particular, what semantics we want.
I thought this was obvious, sorry. The 'worst' and most common scenario
is, precisely put: more than one python extension module over which we
have no control has wrapped type T. We need to use them simultaneously.
For instance, here are two modules that both wrap vector.
With assertions enabled, this happens:
>>> import converter_collisions1_ext
>>> import converter_collisions2_ext
python:
/home/troy/Projects/boost/src/libs/python/src/converter/registry.cpp:212:
void boost::python::converter::registry::insert(PyObject* (*)(const
void*), boost::python::type_info, const PyTypeObject* (*)()): Assertion
`slot->m_to_python == 0' failed.
zsh: abort python
delightfully, without assertions there is only a warning:
>>> import converter_collisions1_ext
>>> import converter_collisions2_ext
__main__:1: RuntimeWarning: to-Python converter for std::vectorstd::allocator > already registered; second conversion method
ignored.
So one resolution is to remove the 'assert' that triggers this crash and
leave the semantics as-is: second and later converter registrations are
warned about and ignored. I'm simply suggesting we give the user more
control over this behavior, because in a number of situations (it is up
to the user to determine what they are) such control would be useful.
For example, what does 'replaceable' mean ? Does it mean two bindings
are identical or equivalent ?
No it means only that the behavior in the situation above is modified.
Does it merely mean the user allows the
mapping to change? (This would lead into undefined behavior, if you
don't carefully control in which order things are loaded / initialized.)
UB as in what happens when there is an ODR violation? How?
A different approach might be to avoid collisions by 'scoping'
conversion operators, so they are only available to specific extension
modules.
Interesting idea. As I interpret "scoping conversion operators", I'm
not sure the cost in runtime and complexity would be worth it, OTOH I
don't have much to go on here.
> May a converter registry be explicitly imported into a module,
during initialization?
Now I'm sure I've interpreted "scoping conversion operators" differently
than you meant it. Can you clarify?
That would involve making dependency between extension modules explicit
("module X imports converters provided by module Y").
But, you can do that now; "module X imports the converter for T provided
by module Y" just means "module X doesn't wrap T and won't run without
Y", right?
-t
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] dynamic compile and "to-Python converter ... second conversion method ignored"
On 11/20/2009 03:23 PM, troy d. straszheim wrote:
For example, what does 'replaceable' mean ? Does it mean two bindings
are identical or equivalent ?
No it means only that the behavior in the situation above is modified.
Does it merely mean the user allows the mapping to change? (This
would lead into undefined behavior, if you don't carefully control in
which order things are loaded / initialized.)
UB as in what happens when there is an ODR violation? How?
I'm only talking about user-visible behavior, and assuming that it could
be implemented in a way that would avoid ODR violations.
(Whether that's actually possible is a different matter. I'm only
talking about semantics.)
A different approach might be to avoid collisions by 'scoping'
conversion operators, so they are only available to specific
extension modules.
Interesting idea. As I interpret "scoping conversion operators", I'm
not sure the cost in runtime and complexity would be worth it, OTOH I
don't have much to go on here.
> May a converter registry be explicitly imported into a module,
during initialization?
Now I'm sure I've interpreted "scoping conversion operators"
differently than you meant it. Can you clarify?
This is mostly only hand-waving, sorry. I was wondering whether it was
possible to make the registry not an application-global object, but a
per-module object. And if that's possible (and thus, conversions are by
default only available to the defining module), would it be possible to
chain them explicitly, to get both, well-defined semantics, as well as
no ODR violations.
I haven't thought about whether this has to involve runtime overhead,
and if so, how much. At least without the chaining it may actually be
cheap. (Just hide the registry symbol from other modules.)
And even with chaining, this is merely a lookup in a chain of converter
maps, which likely can be reordered / optimized at the point where one
registry is explicitly added to another.
Still, this is just a rough idea. I don't have any implementation ideas yet.
That would involve making dependency between extension modules
explicit ("module X imports converters provided by module Y").
But, you can do that now; "module X imports the converter for T
provided by module Y" just means "module X doesn't wrap T and won't
run without Y", right?
Yes. I'm talking about the case where module X and module Y provide a
converter for T, so the user who wishes to use X and Y needs to specify
the desired behavior, which could be "each uses its own" or "either of
the two will be used globally".
Stefan
--
...ich hab' noch einen Koffer in Berlin...
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
Re: [C++-sig] Boost.Python + OpenGL segmentation faults
Back in 2005/2006 all that worked even with PyOpenGL imported under
Windows.
I'm using boost 1.39 to embed Python 2.6 + PyOpenGL 3 in MSVC 2008 under
Windows XP right now, without any difficulties. I'm not using PyOpenGL
directly from C++, but have all the relevant code in a file pygl.py, and
run
exec("from pygl import *");
in my C++ code. I'm not sure if any of this is helpful, but you're very
welcome to the code samples if they would be of any use.
Best wishes,
Mohan
___
Cplusplus-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/cplusplus-sig
