It looks like our build pipeline is clogged up right now. We're looking into it, but it might take a while..
/jake On Mon, Oct 17, 2011 at 9:56 PM, Chris Diehl <[email protected]> wrote: > Hey Jake, > > Awesome! Thanks so much for making this happen! On my end, it doesn't look > like the docs are up yet. I'll check back... > > Thanks again, Chris > > On Mon, Oct 17, 2011 at 4:12 AM, Jacob Hansson < > [email protected]> wrote: > > > Hey Chris, > > > > I've just added support for passing in extra JVM args, it's available as > of > > this commit: > > > > > https://github.com/neo4j/python-embedded/commit/d41950f8c7e6c65d67f60868976e1864a496a2ea > > > > Documentation will pop up in an hour or so here: > > > http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html > > > > This feature will be available as of the 1.5 release, due later this > week. > > If you want to get access to it right away, you can build neo4j-embedded > > directly from source, the code is available here: > > https://github.com/neo4j/python-embedded > > > > See the readme for how to build from source. > > > > > > Thanks for bringing this up! > > Jake > > > > On Fri, Oct 14, 2011 at 7:47 PM, Chris Diehl <[email protected]> > wrote: > > > > > Hi Michael, > > > > > > I added a print statement to my code to output how many nodes have been > > > inserted. The script zips along until it gets up around 18.6K nodes. > Then > > > the program halts when a Java exception is thrown, as shown below. > > > > > > Inserting user 18629 > > > Inserting user 18630 > > > Inserting user 18631 > > > Inserting user 18632 > > > Inserting user 18633 > > > Inserting user 18634 > > > Inserting user 18635 > > > Traceback (most recent call last): > > > File "jive_app_db_graph_construction.py", line 53, in <module> > > > node_idx['userid'][user_node['userid']] = user_node > > > File > > > > > > > > > "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/contextlib.py", > > > line 34, in __exit__ > > > self.gen.throw(type, value, traceback) > > > File "<string>", line 11, in transaction > > > RuntimeError: An unknown error occured while handling a Java Exception > > > > > > Strangely enough, the script returned a more explicit Out of Heap > Memory > > > error once when I reran the script. It's bizarre that the failure > output > > is > > > not consistent, given I start with a clean DB each time. Increasing the > > > amount of available heap memory seems like the reasonable place to > start. > > > If > > > you can let me know how to specify the amount of heap memory for the > JVM > > > instantiated by the bindings, that'd be great. > > > > > > I'm running on a MacBook Pro with 8 gigs of memory running OSX 10.6.8. > > > > > > Here's the contents of the messages.log file. > > > > > > 2011-10-14 10:34:17.336-0700: Creating new db @ > > /Users/chris.diehl/Graphs/ > > > JiveCommunityV5.0.ng/neostore > > > 2011-10-14 10:34:17.419-0700: Opened [/Users/chris.diehl/Graphs/ > > > JiveCommunityV5.0.ng/nioneo_logical.log.1] clean empty log, version=0, > > > lastTxId=1 > > > 2011-10-14 10:34:17.441-0700: Opened [/Users/chris.diehl/Graphs/ > > > JiveCommunityV5.0.ng/index/lucene.log.1] clean empty log, version=0, > > > lastTxId=1 > > > 2011-10-14 10:34:17.443-0700: TM new log: tm_tx_log.1 > > > 2011-10-14 10:34:17.451-0700: --- CONFIGURATION START --- > > > 2011-10-14 10:34:17.451-0700: Physical mem: 8192MB, Heap size: 123MB > > > 2011-10-14 10:34:17.452-0700: Kernel version: Neo4j - Graph Database > > Kernel > > > 1.5-SNAPSHOT (revision: 1.5.M02-4-g3d1e9d8) > > > 2011-10-14 10:34:17.453-0700: Neo4j - Graph Database Kernel > 1.5-SNAPSHOT > > > (revision: 1.5.M02-4-g3d1e9d8) > > > 2011-10-14 10:34:17.456-0700: Operating System: Mac OS X; version: > > 10.6.8; > > > arch: x86_64; cpus: 8 > > > 2011-10-14 10:34:17.456-0700: VM Name: Java HotSpot(TM) 64-Bit Server > VM > > > 2011-10-14 10:34:17.456-0700: VM Vendor: Apple Inc. > > > 2011-10-14 10:34:17.456-0700: VM Version: 20.1-b02-384 > > > 2011-10-14 10:34:17.456-0700: Boot Class Path: > > > > > > > > > /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsfd.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Frameworks/JavaRuntimeSupport.framework/Resources/Java/JavaRuntimeSupport.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/ui.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/laf.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/sunrsasign.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsse.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jce.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/charsets.jar > > > 2011-10-14 10:34:17.457-0700: Class Path: > > > > > > > > > /Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/antlr-2.7.7.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-analysis-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-commons-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-tree-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-util-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/blueprints-core-0.9.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/geronimo-jta_1.1_spec-1.1.1.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packag > > es > > > > > > > > > > /neo4j/javalib/gremlin-1.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/groovy-1.8.1.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/jansi-1.5.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/jettison-1.3.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/lucene-core-3.1.0.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-cypher-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-graph-algo-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-graph-matching-1.5-SNAPSHOT.jar:/Users/ch > > ri > > > > > > > > > > s.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-jmx-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-kernel-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-lucene-index-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/neo4j-udc-1.5-SNAPSHOT.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/pipes-0.7.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/scala-library-2.9.0-1.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/stax-api-1.0.1.jar > > > 2011-10-14 10:34:17.457-0700: Library Path: > > > > > > > > > .:/Users/chris.diehl/Library/Java/Extensions:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java > > > 2011-10-14 10:34:17.459-0700: Garbage Collector: ParNew: [Par Eden > Space, > > > Par Survivor Space] > > > 2011-10-14 10:34:17.459-0700: Garbage Collector: ConcurrentMarkSweep: > > [Par > > > Eden Space, Par Survivor Space, CMS Old Gen, CMS Perm Gen] > > > 2011-10-14 10:34:17.459-0700: VM Arguments: [] > > > 2011-10-14 10:34:17.459-0700: > > > 2011-10-14 10:34:17.460-0700: create=true > > > 2011-10-14 10:34:17.460-0700: logical_log=/Users/chris.diehl/Graphs/ > > > JiveCommunityV5.0.ng/nioneo_logical.log > > > 2011-10-14 10:34:17.460-0700: neo_store=/Users/chris.diehl/Graphs/ > > > JiveCommunityV5.0.ng/neostore > > > 2011-10-14 10:34:17.460-0700: neostore.nodestore.db.mapped_memory=20M > > > 2011-10-14 10:34:17.460-0700: > > > neostore.propertystore.db.arrays.mapped_memory=130M > > > 2011-10-14 10:34:17.460-0700: > > > neostore.propertystore.db.index.keys.mapped_memory=1M > > > 2011-10-14 10:34:17.460-0700: > > > neostore.propertystore.db.index.mapped_memory=1M > > > 2011-10-14 10:34:17.460-0700: > neostore.propertystore.db.mapped_memory=90M > > > 2011-10-14 10:34:17.460-0700: > > > neostore.propertystore.db.strings.mapped_memory=130M > > > 2011-10-14 10:34:17.461-0700: > > > neostore.relationshipstore.db.mapped_memory=100M > > > 2011-10-14 10:34:17.461-0700: node_auto_indexing=false > > > 2011-10-14 10:34:17.461-0700: rebuild_idgenerators_fast=true > > > 2011-10-14 10:34:17.461-0700: relationship_auto_indexing=false > > > 2011-10-14 10:34:17.461-0700: store_dir=/Users/chris.diehl/Graphs/ > > > JiveCommunityV5.0.ng > > > 2011-10-14 10:34:17.461-0700: use_memory_mapped_buffers=true > > > 2011-10-14 10:34:17.461-0700: --- CONFIGURATION END --- > > > 2011-10-14 10:34:17.518-0700: Extension > > > org.neo4j.kernel.KernelExtension[kernel jmx] loaded ok > > > 2011-10-14 10:34:17.520-0700: Extension > > > org.neo4j.kernel.KernelExtension[kernel udc] loaded ok > > > > > > Chris > > > > > > On Thu, Oct 13, 2011 at 8:37 PM, Michael Hunger < > > > [email protected]> wrote: > > > > > > > Hi Chris, > > > > > > > > What do you mean by "ground to a halt" ? Is it using all available > CPU > > or > > > > heavy IO-waits (top, iostat, vmstat)? > > > > > > > > A thread dump (via jps + jstack ) would be great. > > > > > > > > You can probably pass in the config map into the Graph database > call. > > > > Current config is logged to messages.log, please share that too as > well > > > as > > > > your system parameters (OS, CPU, RAM, Type of disk). > > > > > > > > Michael > > > > > > > > Am 13.10.2011 um 23:52 schrieb Chris Diehl: > > > > > > > > > Hey All, > > > > > > > > > > I just tested a Python script I've been working on to construct a > > > > > multi-relational graph in Neo4j. I am using the new Python bindings > > for > > > > > embedded Neo4j. After creating about 18K nodes, the database ground > > to > > > a > > > > > halt. > > > > > > > > > > In the past, when using Gremlin, I would up the heap size for the > JVM > > > > when > > > > > encountering problems. I'm not sure how to tweak settings for the > JVM > > > now > > > > > that I'm using the Python bindings to fire up the database. Can I > do > > > that > > > > > from the GraphDatabase() call? Or do I need to look elsewhere to > > tweak > > > > the > > > > > JVM parameters? > > > > > > > > > > If you have any other suggestions about parameters I should change, > > I'm > > > > all > > > > > ears. > > > > > > > > > > Thanks! > > > > > > > > > > Chris > > > > > > > > > > -- > > > > > View this message in context: > > > > > > > > > > http://neo4j-community-discussions.438527.n3.nabble.com/Configuring-Neo4J-via-Python-bindings-tp3419961p3419961.html > > > > > Sent from the Neo4j Community Discussions mailing list archive at > > > > Nabble.com. > > > > > _______________________________________________ > > > > > Neo4j mailing list > > > > > [email protected] > > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > _______________________________________________ > > > > Neo4j mailing list > > > > [email protected] > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > _______________________________________________ > > > Neo4j mailing list > > > [email protected] > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > -- > > Jacob Hansson > > Phone: +46 (0) 763503395 > > Twitter: @jakewins > > _______________________________________________ > > Neo4j mailing list > > [email protected] > > https://lists.neo4j.org/mailman/listinfo/user > > > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
