On Mon, 10 Mar 2008, Andi Vajda wrote:
I got the multi-jcc-built-extension-in-same-process thing to work. There are
good news and bad news:
- the good:
. it works :)
- the bad:
. classpath is set once per VM, there may be some class load tricks
that can be played but it's not something on the radar at the moment.
. it requires some JCC runtime code to be included in a shared library
instead of a python extension (on Mac, a .dylib instead of an .so
bundle).
Python's distutils/setuptools is not so good at knowing how to build shared
libraries. I talked with the person behind setuptools and he said that the
trunk had code to support this but he wasn't sure in what state it was in. He
also said that patches to this were welcome. I'm going to look at this as
I've hit this problem before. Supporting shared libraries between python
extension bundles is a good thing to have in setuptools.
I think that I'm going to check-in something soon where the shared library
building pleasure is left to the user ;) but not making it any harder to
build the current, default way where the JCC runtime is statically linked
into the extension. Later, once I figure out the setuptools thing, I can
update jcc's setup.py to also build the shared library.
I just checked in the changes to support this properly.
initVM() no longer takes an env argument but still needs to be called for
every extension being imported into the process. There is only one env per
process and any extension can return it. The python JCCEnv type is defined
in each extension module but from the _same_ static structure in the shared
library and Python only initializes it once.
Sadly, the classpath additions don't seem to stick so it must be set either
in the environment or via the classpath keyword to initVM().
I made it an error to pass in any other arguments to initVM() on any calls
other than the first one since these would be ignored. In other words, you
get one chance per process to start and configure the Java VM.
To try this out you need:
- a libjcc.dylib shared library
To build this, simply enter this in the shell after building JCC
itself (on leopard intel mac, adapt for other platforms until I
automate this in JCC's setup.py)
g++ -dynamiclib -install_name
/Library/Python/2.5/site-packages/JCC-1.8-py2.5-macosx-10.5-i386.egg/jcc/libjcc.dylib
-o libjcc.dylib build/temp.macosx-10.5-i386-2.5/jcc/sources/jcc.o
build/temp.macosx-10.5-i386-2.5/jcc/sources/JCCEnv.o -undefined dynamic_lookup
cp -p libjcc.dylib
/Library/Python/2.5/site-packages/JCC-1.8-py2.5-macosx-10.5-i386.egg/jcc/
This reuses the .o files built by setup.py (they're compatible). The
g++ statement just links two files together into libjcc.dylib in a way
that forces using libraries to look for it in the installed jcc egg.
You can, of course, change -install_name to suit your install or omit
it altogether and set DYLD_LIBRARY_PATH instead (not so good for
security).
- build all your extensions by invoking jcc with --shared (the default is
unchanged, shared is False)
This tells jcc to link with libjcc.dylib instead of statically
linking with the same JCC runtime code. Again, libjcc.dylib is expected
to be installed into the JCC egg dir (manually).
Then, once you've got everything built and installed, call initVM() on each
of your extension modules, passing in all relevant info into the first call
as follows:
>>> import os, lucene, gs
>>> lucene.initVM(os.pathsep.join([lucene.CLASSPATH, gs.CLASSPATH]),
other args as needed)
>>> gs.initVM()
>>> lucene.Document()
>>> gs.GoodStuff.Frob()
Andi..
_______________________________________________
pylucene-dev mailing list
[email protected]
http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
