Hey Chris, I've just added support for passing in extra JVM args, it's available as of this commit: https://github.com/neo4j/python-embedded/commit/d41950f8c7e6c65d67f60868976e1864a496a2ea
Documentation will pop up in an hour or so here: http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html This feature will be available as of the 1.5 release, due later this week. If you want to get access to it right away, you can build neo4j-embedded directly from source, the code is available here: https://github.com/neo4j/python-embedded See the readme for how to build from source. Thanks for bringing this up! Jake On Fri, Oct 14, 2011 at 7:47 PM, Chris Diehl <[email protected]> wrote: > Hi Michael, > > I added a print statement to my code to output how many nodes have been > inserted. The script zips along until it gets up around 18.6K nodes. Then > the program halts when a Java exception is thrown, as shown below. > > Inserting user 18629 > Inserting user 18630 > Inserting user 18631 > Inserting user 18632 > Inserting user 18633 > Inserting user 18634 > Inserting user 18635 > Traceback (most recent call last): > File "jive_app_db_graph_construction.py", line 53, in <module> > node_idx['userid'][user_node['userid']] = user_node > File > > "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/contextlib.py", > line 34, in __exit__ > self.gen.throw(type, value, traceback) > File "<string>", line 11, in transaction > RuntimeError: An unknown error occured while handling a Java Exception > > Strangely enough, the script returned a more explicit Out of Heap Memory > error once when I reran the script. It's bizarre that the failure output is > not consistent, given I start with a clean DB each time. Increasing the > amount of available heap memory seems like the reasonable place to start. > If > you can let me know how to specify the amount of heap memory for the JVM > instantiated by the bindings, that'd be great. > > I'm running on a MacBook Pro with 8 gigs of memory running OSX 10.6.8. > > Here's the contents of the messages.log file. > > 2011-10-14 10:34:17.336-0700: Creating new db @ /Users/chris.diehl/Graphs/ > JiveCommunityV5.0.ng/neostore > 2011-10-14 10:34:17.419-0700: Opened [/Users/chris.diehl/Graphs/ > JiveCommunityV5.0.ng/nioneo_logical.log.1] clean empty log, version=0, > lastTxId=1 > 2011-10-14 10:34:17.441-0700: Opened [/Users/chris.diehl/Graphs/ > JiveCommunityV5.0.ng/index/lucene.log.1] clean empty log, version=0, > lastTxId=1 > 2011-10-14 10:34:17.443-0700: TM new log: tm_tx_log.1 > 2011-10-14 10:34:17.451-0700: --- CONFIGURATION START --- > 2011-10-14 10:34:17.451-0700: Physical mem: 8192MB, Heap size: 123MB > 2011-10-14 10:34:17.452-0700: Kernel version: Neo4j - Graph Database Kernel > 1.5-SNAPSHOT (revision: 1.5.M02-4-g3d1e9d8) > 2011-10-14 10:34:17.453-0700: Neo4j - Graph Database Kernel 1.5-SNAPSHOT > (revision: 1.5.M02-4-g3d1e9d8) > 2011-10-14 10:34:17.456-0700: Operating System: Mac OS X; version: 10.6.8; > arch: x86_64; cpus: 8 > 2011-10-14 10:34:17.456-0700: VM Name: Java HotSpot(TM) 64-Bit Server VM > 2011-10-14 10:34:17.456-0700: VM Vendor: Apple Inc. > 2011-10-14 10:34:17.456-0700: VM Version: 20.1-b02-384 > 2011-10-14 10:34:17.456-0700: Boot Class Path: > > /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsfd.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Frameworks/JavaRuntimeSupport.framework/Resources/Java/JavaRuntimeSupport.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/ui.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/laf.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/sunrsasign.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsse.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jce.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/charsets.jar > 2011-10-14 10:34:17.457-0700: Class Path: > > /Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/antlr-2.7.7.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-analysis-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-commons-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-tree-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-util-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/blueprints-core-0.9.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/geronimo-jta_1.1_spec-1.1.1.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packag es > > > /neo4j/javalib/gremlin-1.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/groovy-1.8.1.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/jansi-1.5.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/jettison-1.3.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/lucene-core-3.1.0.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-cypher-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-graph-algo-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-graph-matching-1.5-SNAPSHOT.jar:/Users/ch ri > > > s.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-jmx-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-kernel-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-lucene-index-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-udc-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/pipes-0.7.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/scala-library-2.9.0-1.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/stax-api-1.0.1.jar > 2011-10-14 10:34:17.457-0700: Library Path: > > .:/Users/chris.diehl/Library/Java/Extensions:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java > 2011-10-14 10:34:17.459-0700: Garbage Collector: ParNew: [Par Eden Space, > Par Survivor Space] > 2011-10-14 10:34:17.459-0700: Garbage Collector: ConcurrentMarkSweep: [Par > Eden Space, Par Survivor Space, CMS Old Gen, CMS Perm Gen] > 2011-10-14 10:34:17.459-0700: VM Arguments: [] > 2011-10-14 10:34:17.459-0700: > 2011-10-14 10:34:17.460-0700: create=true > 2011-10-14 10:34:17.460-0700: logical_log=/Users/chris.diehl/Graphs/ > JiveCommunityV5.0.ng/nioneo_logical.log > 2011-10-14 10:34:17.460-0700: neo_store=/Users/chris.diehl/Graphs/ > JiveCommunityV5.0.ng/neostore > 2011-10-14 10:34:17.460-0700: neostore.nodestore.db.mapped_memory=20M > 2011-10-14 10:34:17.460-0700: > neostore.propertystore.db.arrays.mapped_memory=130M > 2011-10-14 10:34:17.460-0700: > neostore.propertystore.db.index.keys.mapped_memory=1M > 2011-10-14 10:34:17.460-0700: > neostore.propertystore.db.index.mapped_memory=1M > 2011-10-14 10:34:17.460-0700: neostore.propertystore.db.mapped_memory=90M > 2011-10-14 10:34:17.460-0700: > neostore.propertystore.db.strings.mapped_memory=130M > 2011-10-14 10:34:17.461-0700: > neostore.relationshipstore.db.mapped_memory=100M > 2011-10-14 10:34:17.461-0700: node_auto_indexing=false > 2011-10-14 10:34:17.461-0700: rebuild_idgenerators_fast=true > 2011-10-14 10:34:17.461-0700: relationship_auto_indexing=false > 2011-10-14 10:34:17.461-0700: store_dir=/Users/chris.diehl/Graphs/ > JiveCommunityV5.0.ng > 2011-10-14 10:34:17.461-0700: use_memory_mapped_buffers=true > 2011-10-14 10:34:17.461-0700: --- CONFIGURATION END --- > 2011-10-14 10:34:17.518-0700: Extension > org.neo4j.kernel.KernelExtension[kernel jmx] loaded ok > 2011-10-14 10:34:17.520-0700: Extension > org.neo4j.kernel.KernelExtension[kernel udc] loaded ok > > Chris > > On Thu, Oct 13, 2011 at 8:37 PM, Michael Hunger < > [email protected]> wrote: > > > Hi Chris, > > > > What do you mean by "ground to a halt" ? Is it using all available CPU or > > heavy IO-waits (top, iostat, vmstat)? > > > > A thread dump (via jps + jstack ) would be great. > > > > You can probably pass in the config map into the Graph database call. > > Current config is logged to messages.log, please share that too as well > as > > your system parameters (OS, CPU, RAM, Type of disk). > > > > Michael > > > > Am 13.10.2011 um 23:52 schrieb Chris Diehl: > > > > > Hey All, > > > > > > I just tested a Python script I've been working on to construct a > > > multi-relational graph in Neo4j. I am using the new Python bindings for > > > embedded Neo4j. After creating about 18K nodes, the database ground to > a > > > halt. > > > > > > In the past, when using Gremlin, I would up the heap size for the JVM > > when > > > encountering problems. I'm not sure how to tweak settings for the JVM > now > > > that I'm using the Python bindings to fire up the database. Can I do > that > > > from the GraphDatabase() call? Or do I need to look elsewhere to tweak > > the > > > JVM parameters? > > > > > > If you have any other suggestions about parameters I should change, I'm > > all > > > ears. > > > > > > Thanks! > > > > > > Chris > > > > > > -- > > > View this message in context: > > > http://neo4j-community-discussions.438527.n3.nabble.com/Configuring-Neo4J-via-Python-bindings-tp3419961p3419961.html > > > Sent from the Neo4j Community Discussions mailing list archive at > > Nabble.com. > > > _______________________________________________ > > > Neo4j mailing list > > > [email protected] > > > https://lists.neo4j.org/mailman/listinfo/user > > > > _______________________________________________ > > Neo4j mailing list > > [email protected] > > https://lists.neo4j.org/mailman/listinfo/user > > > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
