Re: [Neo4j] Java segfault using paste and Python bindings
I had no idea that neo4j had a REST interface; I am essentially implementing my own REST interface in order to use neo4j embedded in a desktop app. I will have to look into that more to see if there's an easy way to embed the server and still communicate to it via the REST API. Thanks for the pointer! ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Java segfault using paste and Python bindings
Michael, have you looked at http://docs.neo4j.org/chunked/snapshot/rest-api.html for documentation? We probably should promote the manual much more prominently on the site :/ Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Wed, Nov 2, 2011 at 7:49 AM, Michael Rene Armida m...@marmida.com wrote: I had no idea that neo4j had a REST interface; I am essentially implementing my own REST interface in order to use neo4j embedded in a desktop app. I will have to look into that more to see if there's an easy way to embed the server and still communicate to it via the REST API. Thanks for the pointer! ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Java segfault using paste and Python bindings
I solved my problem by ensuring that I don't 'import neo4j' until I'm in a
dedicated thread. So long as all neo4j interaction is handled within that
thread, I don't crash.
The Python neo4j bindings use JPype; I noticed this note from the JPype
docs (http://jpype.sourceforge.net/doc/user-guide/userguide.html#threading):
For the most part, python threads based on OS level threads (i.e posix
threads), will work without problem. The only thing to remember is to
call jpype.attachThreadToJVM() in the thread body to make the JVM usable
from that thread. For threads that you do not start yourself, you can
call isThreadAttachedToJVM() to check.
I couldn't find any evidence for this, but I think that some of the Java
code in the neo4j bindings may call attachThreadToJVM at import time.
Unfortunately, this means that even though neo4j is thread-safe, it must be
constrained to a single thread when used from Python. But that's not too
disappointing, considering.
On Sat, Oct 29, 2011 at 4:25 PM, Michael Rene Armida m...@marmida.com wrote:
I am attempting to use Paste to serve a small web app that uses neo4j via
the Python bindings. I get a segfault whenever I try to create a new
database via 'neo4j.GraphDatabase'. I have narrowed this down to the
following minimal sample:
http://pastebin.com/MkNRvLR6
The top of the stack trace looks like this:
Stack: [0xb42e7000,0xb4ae8000], sp=0xb4ae44f0, free space=8181k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
code)
C [_jpype.so+0x26497] JPJavaEnv::NewObjectA(_jclass*, _jmethodID*,
jvalue*)+0x37
C [_jpype.so+0x3c0e8] JPMethodOverload::invokeConstructor(_jclass*,
std::vectorHostRef*, std::allocatorHostRef* )+0x178
C [_jpype.so+0x3a417] JPMethod::invokeConstructor(std::vectorHostRef*,
std::allocatorHostRef* )+0x47
C [_jpype.so+0x1beba] JPClass::newInstance(std::vectorHostRef*,
std::allocatorHostRef* )+0x2a
C [_jpype.so+0x67b9c] PyJPClass::newClassInstance(_object*,
_object*)+0xfc
C [python+0x96822] PyEval_EvalFrameEx+0x4332
C [python+0x991e7] PyEval_EvalCodeEx+0x127
I believe that is JPype trying to create an EmbeddedGraphDatabase via
neo4j.GraphDatabase() in python.
I noticed this previous thread:
http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-CPython-Pylons-and-threading-td942435.html
That looks like it might be related, except this problem occurs whenever I
try to create a database, not on shutdown. The issue definitely has
something to do with using neo4j under Paste; here's a Python shell session
which shows neo4j working:
import webob
import app
debug_app = app.DebugApp()
response = debug_app(webob.Request.blank('/'))
response.body
'it worked'
Relevant version info:
- ubuntu 11.04 (same crash under 32- and 64-bit systems)
- Python 2.7.1-0ubuntu5
- neo4j_embedded-1.5.b2
- jpype 0.5.4.1-3
- paste 1.7.5.1
- OpenJDK 6b22-1.10.2 (same crash under the Sun JRE)
Thanks for your time and any clues you might have.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Java segfault using paste and Python bindings
Hey Michael, thanks for the investigative work you put into this, and sorry
for not replying earlier.
I was not aware of the thread limitations of JPype, that's quite
dissapointing.. Overall, it feels like the JPype solution is not a
manageable one in the long term. The embedded use case is cool, because
it's so freaking convenient, but using JPype to embed the JVM is painful.
There are a few other options for JVM-python integration that are starting
to look interesting, perhaps we can look around a little bit and see what
is available. It would be especially cool if we can use something that
works across CPython, Jython as well as PyPy. I know Tobias was thinking
about something that would make that possible.
In the short to medium term though, I'd like to focus on two things (please
note that this is *my opinion*, not a product roadmap):
*Getting cypher and gremlin into the embedded python API*
Eventually, I think we should deprecate the traversal API in the embedded
client in favor of Cypher. Traversals need callbacks and shared state and
so on between the two VMs, and its gonna hurt, I think, if we depend on
that further down the road.
Doing this will also make it simple to keep full 1-1 API compatibility
between the embedded client and REST clients, making it easy for end users
to switch between the two.
*Making the REST API fast enough to make that use case more appealing.*
JPype is really just a cheap way of implementing a form of remoting API.
It's currently worth the pain, since it is a ton faster than our REST API,
but it's a band-aid, not a proper fix. We should focus on speeding up the
REST API.
WDYT?
/Jake
On Tue, Nov 1, 2011 at 5:00 PM, Michael Rene Armida m...@marmida.com wrote:
I solved my problem by ensuring that I don't 'import neo4j' until I'm in a
dedicated thread. So long as all neo4j interaction is handled within that
thread, I don't crash.
The Python neo4j bindings use JPype; I noticed this note from the JPype
docs (http://jpype.sourceforge.net/doc/user-guide/userguide.html#threading
):
For the most part, python threads based on OS level threads (i.e posix
threads), will work without problem. The only thing to remember is to
call jpype.attachThreadToJVM() in the thread body to make the JVM usable
from that thread. For threads that you do not start yourself, you can
call isThreadAttachedToJVM() to check.
I couldn't find any evidence for this, but I think that some of the Java
code in the neo4j bindings may call attachThreadToJVM at import time.
Unfortunately, this means that even though neo4j is thread-safe, it must be
constrained to a single thread when used from Python. But that's not too
disappointing, considering.
On Sat, Oct 29, 2011 at 4:25 PM, Michael Rene Armida m...@marmida.com
wrote:
I am attempting to use Paste to serve a small web app that uses neo4j via
the Python bindings. I get a segfault whenever I try to create a new
database via 'neo4j.GraphDatabase'. I have narrowed this down to the
following minimal sample:
http://pastebin.com/MkNRvLR6
The top of the stack trace looks like this:
Stack: [0xb42e7000,0xb4ae8000], sp=0xb4ae44f0, free space=8181k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
code)
C [_jpype.so+0x26497] JPJavaEnv::NewObjectA(_jclass*, _jmethodID*,
jvalue*)+0x37
C [_jpype.so+0x3c0e8] JPMethodOverload::invokeConstructor(_jclass*,
std::vectorHostRef*, std::allocatorHostRef* )+0x178
C [_jpype.so+0x3a417] JPMethod::invokeConstructor(std::vectorHostRef*,
std::allocatorHostRef* )+0x47
C [_jpype.so+0x1beba] JPClass::newInstance(std::vectorHostRef*,
std::allocatorHostRef* )+0x2a
C [_jpype.so+0x67b9c] PyJPClass::newClassInstance(_object*,
_object*)+0xfc
C [python+0x96822] PyEval_EvalFrameEx+0x4332
C [python+0x991e7] PyEval_EvalCodeEx+0x127
I believe that is JPype trying to create an EmbeddedGraphDatabase via
neo4j.GraphDatabase() in python.
I noticed this previous thread:
http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-CPython-Pylons-and-threading-td942435.html
That looks like it might be related, except this problem occurs whenever
I
try to create a database, not on shutdown. The issue definitely has
something to do with using neo4j under Paste; here's a Python shell
session
which shows neo4j working:
import webob
import app
debug_app = app.DebugApp()
response = debug_app(webob.Request.blank('/'))
response.body
'it worked'
Relevant version info:
- ubuntu 11.04 (same crash under 32- and 64-bit systems)
- Python 2.7.1-0ubuntu5
- neo4j_embedded-1.5.b2
- jpype 0.5.4.1-3
- paste 1.7.5.1
- OpenJDK 6b22-1.10.2 (same crash under the Sun JRE)
Thanks for your time and any clues you might have.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone:
[Neo4j] Java segfault using paste and Python bindings
I am attempting to use Paste to serve a small web app that uses neo4j via
the Python bindings. I get a segfault whenever I try to create a new
database via 'neo4j.GraphDatabase'. I have narrowed this down to the
following minimal sample:
http://pastebin.com/MkNRvLR6
The top of the stack trace looks like this:
Stack: [0xb42e7000,0xb4ae8000], sp=0xb4ae44f0, free space=8181k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
code)
C [_jpype.so+0x26497] JPJavaEnv::NewObjectA(_jclass*, _jmethodID*,
jvalue*)+0x37
C [_jpype.so+0x3c0e8] JPMethodOverload::invokeConstructor(_jclass*,
std::vectorHostRef*, std::allocatorHostRef* )+0x178
C [_jpype.so+0x3a417] JPMethod::invokeConstructor(std::vectorHostRef*,
std::allocatorHostRef* )+0x47
C [_jpype.so+0x1beba] JPClass::newInstance(std::vectorHostRef*,
std::allocatorHostRef* )+0x2a
C [_jpype.so+0x67b9c] PyJPClass::newClassInstance(_object*, _object*)+0xfc
C [python+0x96822] PyEval_EvalFrameEx+0x4332
C [python+0x991e7] PyEval_EvalCodeEx+0x127
I believe that is JPype trying to create an EmbeddedGraphDatabase via
neo4j.GraphDatabase() in python.
I noticed this previous thread:
http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-CPython-Pylons-and-threading-td942435.html
That looks like it might be related, except this problem occurs whenever I
try to create a database, not on shutdown. The issue definitely has
something to do with using neo4j under Paste; here's a Python shell session
which shows neo4j working:
import webob
import app
debug_app = app.DebugApp()
response = debug_app(webob.Request.blank('/'))
response.body
'it worked'
Relevant version info:
- ubuntu 11.04 (same crash under 32- and 64-bit systems)
- Python 2.7.1-0ubuntu5
- neo4j_embedded-1.5.b2
- jpype 0.5.4.1-3
- paste 1.7.5.1
- OpenJDK 6b22-1.10.2 (same crash under the Sun JRE)
Thanks for your time and any clues you might have.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
