On Thu, 18 May 2006, Mike Klaas wrote:
Am I asking in the wrong place, or has no-one really dug into these issues?
I don't know that anyone has run timings of these exact operations.
Since PyLucene got de-swigged, crossing the python<->CNI barrier involves:
python -> C++/Java
- parsing calling args, maybe
- releasing the GIL
- calling C++
- acquiring the GIL
- packaging up return values, maybe
C++/Java -> python
- acquiring the GIL
- creating a calling args tuple, maybe
- calling Python
- extracting return values, maybe
- releasing the GIL
I don't think that calling python from C++ is any slower than calling python
from python. It seems that the GIL operations are pretty fast unless there is
contention.
Getting rid of the GIL operations, even in carefully selected places, is a
recipe for deadlocks or crashes. Keep in mind that the libgcj/gc finalizer
thread acquires the GIL when it calls into Python for releasing wrapped Python
object references.
If you want speed over anything, write the Java class extensions in Java with
extension methods declared native, and provide the native C++ methods via CNI.
If you prefer convenience and the usual python speed is enough, use the
built-in PyLucene extension points.
If you do actual timings, let us know. Thanks !
Andi..
cheers,
-MIke
On 5/15/06, Mike Klaas <[EMAIL PROTECTED]> wrote:
I'm currently investigating using PyLucene for a large scale
distributed search application which is currently in 95% python with a
smattering of c/pyrex for performance. I'm rather impressed with the
package, particularly the ability to "subclass" certain lucene classes
to provide customization hooks.
I was curious if anyone has done heavy-duty lifting using these
reverse bindings. In particulary, I am concerned with subclasses
which are tightly coupled with the core lucene query code -- like
SortComparator. Does the repeated python<->CNI boundary crossing
degrade performance unacceptably? Has anyone had success implementing
these sorts of customizations java-side, recompiling the whole package
with gcj?
Thanks in advance,
-MIke Klaas
_______________________________________________
pylucene-dev mailing list
[email protected]
http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
