Hey Jake, Awesome! Thanks so much for making this happen! On my end, it doesn't look like the docs are up yet. I'll check back...
Thanks again, Chris On Mon, Oct 17, 2011 at 4:12 AM, Jacob Hansson < jacob.hans...@neotechnology.com> wrote: > Hey Chris, > > I've just added support for passing in extra JVM args, it's available as of > this commit: > > https://github.com/neo4j/python-embedded/commit/d41950f8c7e6c65d67f60868976e1864a496a2ea > > Documentation will pop up in an hour or so here: > http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html > > This feature will be available as of the 1.5 release, due later this week. > If you want to get access to it right away, you can build neo4j-embedded > directly from source, the code is available here: > https://github.com/neo4j/python-embedded > > See the readme for how to build from source. > > > Thanks for bringing this up! > Jake > > On Fri, Oct 14, 2011 at 7:47 PM, Chris Diehl <di...@alumni.cmu.edu> wrote: > > > Hi Michael, > > > > I added a print statement to my code to output how many nodes have been > > inserted. The script zips along until it gets up around 18.6K nodes. Then > > the program halts when a Java exception is thrown, as shown below. > > > > Inserting user 18629 > > Inserting user 18630 > > Inserting user 18631 > > Inserting user 18632 > > Inserting user 18633 > > Inserting user 18634 > > Inserting user 18635 > > Traceback (most recent call last): > > File "jive_app_db_graph_construction.py", line 53, in <module> > > node_idx['userid'][user_node['userid']] = user_node > > File > > > > > "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/contextlib.py", > > line 34, in __exit__ > > self.gen.throw(type, value, traceback) > > File "<string>", line 11, in transaction > > RuntimeError: An unknown error occured while handling a Java Exception > > > > Strangely enough, the script returned a more explicit Out of Heap Memory > > error once when I reran the script. It's bizarre that the failure output > is > > not consistent, given I start with a clean DB each time. Increasing the > > amount of available heap memory seems like the reasonable place to start. > > If > > you can let me know how to specify the amount of heap memory for the JVM > > instantiated by the bindings, that'd be great. > > > > I'm running on a MacBook Pro with 8 gigs of memory running OSX 10.6.8. > > > > Here's the contents of the messages.log file. > > > > 2011-10-14 10:34:17.336-0700: Creating new db @ > /Users/chris.diehl/Graphs/ > > JiveCommunityV5.0.ng/neostore > > 2011-10-14 10:34:17.419-0700: Opened [/Users/chris.diehl/Graphs/ > > JiveCommunityV5.0.ng/nioneo_logical.log.1] clean empty log, version=0, > > lastTxId=1 > > 2011-10-14 10:34:17.441-0700: Opened [/Users/chris.diehl/Graphs/ > > JiveCommunityV5.0.ng/index/lucene.log.1] clean empty log, version=0, > > lastTxId=1 > > 2011-10-14 10:34:17.443-0700: TM new log: tm_tx_log.1 > > 2011-10-14 10:34:17.451-0700: --- CONFIGURATION START --- > > 2011-10-14 10:34:17.451-0700: Physical mem: 8192MB, Heap size: 123MB > > 2011-10-14 10:34:17.452-0700: Kernel version: Neo4j - Graph Database > Kernel > > 1.5-SNAPSHOT (revision: 1.5.M02-4-g3d1e9d8) > > 2011-10-14 10:34:17.453-0700: Neo4j - Graph Database Kernel 1.5-SNAPSHOT > > (revision: 1.5.M02-4-g3d1e9d8) > > 2011-10-14 10:34:17.456-0700: Operating System: Mac OS X; version: > 10.6.8; > > arch: x86_64; cpus: 8 > > 2011-10-14 10:34:17.456-0700: VM Name: Java HotSpot(TM) 64-Bit Server VM > > 2011-10-14 10:34:17.456-0700: VM Vendor: Apple Inc. > > 2011-10-14 10:34:17.456-0700: VM Version: 20.1-b02-384 > > 2011-10-14 10:34:17.456-0700: Boot Class Path: > > > > > /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsfd.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Frameworks/JavaRuntimeSupport.framework/Resources/Java/JavaRuntimeSupport.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/ui.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/laf.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/sunrsasign.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsse.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jce.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/charsets.jar > > 2011-10-14 10:34:17.457-0700: Class Path: > > > > > /Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/antlr-2.7.7.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-analysis-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-commons-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-tree-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-util-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/blueprints-core-0.9.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/geronimo-jta_1.1_spec-1.1.1.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packag > es > > > > > > /neo4j/javalib/gremlin-1.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/groovy-1.8.1.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/jansi-1.5.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/jettison-1.3.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/lucene-core-3.1.0.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-cypher-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-graph-algo-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-graph-matching-1.5-SNAPSHOT.jar:/Users/ch > ri > > > > > > s.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-jmx-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-kernel-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-lucene-index-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-udc-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/pipes-0.7.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/scala-library-2.9.0-1.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/stax-api-1.0.1.jar > > 2011-10-14 10:34:17.457-0700: Library Path: > > > > > .:/Users/chris.diehl/Library/Java/Extensions:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java > > 2011-10-14 10:34:17.459-0700: Garbage Collector: ParNew: [Par Eden Space, > > Par Survivor Space] > > 2011-10-14 10:34:17.459-0700: Garbage Collector: ConcurrentMarkSweep: > [Par > > Eden Space, Par Survivor Space, CMS Old Gen, CMS Perm Gen] > > 2011-10-14 10:34:17.459-0700: VM Arguments: [] > > 2011-10-14 10:34:17.459-0700: > > 2011-10-14 10:34:17.460-0700: create=true > > 2011-10-14 10:34:17.460-0700: logical_log=/Users/chris.diehl/Graphs/ > > JiveCommunityV5.0.ng/nioneo_logical.log > > 2011-10-14 10:34:17.460-0700: neo_store=/Users/chris.diehl/Graphs/ > > JiveCommunityV5.0.ng/neostore > > 2011-10-14 10:34:17.460-0700: neostore.nodestore.db.mapped_memory=20M > > 2011-10-14 10:34:17.460-0700: > > neostore.propertystore.db.arrays.mapped_memory=130M > > 2011-10-14 10:34:17.460-0700: > > neostore.propertystore.db.index.keys.mapped_memory=1M > > 2011-10-14 10:34:17.460-0700: > > neostore.propertystore.db.index.mapped_memory=1M > > 2011-10-14 10:34:17.460-0700: neostore.propertystore.db.mapped_memory=90M > > 2011-10-14 10:34:17.460-0700: > > neostore.propertystore.db.strings.mapped_memory=130M > > 2011-10-14 10:34:17.461-0700: > > neostore.relationshipstore.db.mapped_memory=100M > > 2011-10-14 10:34:17.461-0700: node_auto_indexing=false > > 2011-10-14 10:34:17.461-0700: rebuild_idgenerators_fast=true > > 2011-10-14 10:34:17.461-0700: relationship_auto_indexing=false > > 2011-10-14 10:34:17.461-0700: store_dir=/Users/chris.diehl/Graphs/ > > JiveCommunityV5.0.ng > > 2011-10-14 10:34:17.461-0700: use_memory_mapped_buffers=true > > 2011-10-14 10:34:17.461-0700: --- CONFIGURATION END --- > > 2011-10-14 10:34:17.518-0700: Extension > > org.neo4j.kernel.KernelExtension[kernel jmx] loaded ok > > 2011-10-14 10:34:17.520-0700: Extension > > org.neo4j.kernel.KernelExtension[kernel udc] loaded ok > > > > Chris > > > > On Thu, Oct 13, 2011 at 8:37 PM, Michael Hunger < > > michael.hun...@neotechnology.com> wrote: > > > > > Hi Chris, > > > > > > What do you mean by "ground to a halt" ? Is it using all available CPU > or > > > heavy IO-waits (top, iostat, vmstat)? > > > > > > A thread dump (via jps + jstack ) would be great. > > > > > > You can probably pass in the config map into the Graph database call. > > > Current config is logged to messages.log, please share that too as well > > as > > > your system parameters (OS, CPU, RAM, Type of disk). > > > > > > Michael > > > > > > Am 13.10.2011 um 23:52 schrieb Chris Diehl: > > > > > > > Hey All, > > > > > > > > I just tested a Python script I've been working on to construct a > > > > multi-relational graph in Neo4j. I am using the new Python bindings > for > > > > embedded Neo4j. After creating about 18K nodes, the database ground > to > > a > > > > halt. > > > > > > > > In the past, when using Gremlin, I would up the heap size for the JVM > > > when > > > > encountering problems. I'm not sure how to tweak settings for the JVM > > now > > > > that I'm using the Python bindings to fire up the database. Can I do > > that > > > > from the GraphDatabase() call? Or do I need to look elsewhere to > tweak > > > the > > > > JVM parameters? > > > > > > > > If you have any other suggestions about parameters I should change, > I'm > > > all > > > > ears. > > > > > > > > Thanks! > > > > > > > > Chris > > > > > > > > -- > > > > View this message in context: > > > > > > http://neo4j-community-discussions.438527.n3.nabble.com/Configuring-Neo4J-via-Python-bindings-tp3419961p3419961.html > > > > Sent from the Neo4j Community Discussions mailing list archive at > > > Nabble.com. > > > > _______________________________________________ > > > > Neo4j mailing list > > > > User@lists.neo4j.org > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > _______________________________________________ > > > Neo4j mailing list > > > User@lists.neo4j.org > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > _______________________________________________ > > Neo4j mailing list > > User@lists.neo4j.org > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > -- > Jacob Hansson > Phone: +46 (0) 763503395 > Twitter: @jakewins > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
