Bill, Wes gave me an idea, so I created a version of the Neo4j server that uses our testing-in-memory graph database, care to try it?
https://github.com/jexp/neo4j-in-memory-server Cheers Michael Am 17.01.2014 um 02:52 schrieb Michael Hunger <[email protected]>: > Good question, not sure about that. > > Indexing uses lucene behind the scenes, and adds some overhead. > > Michael > > Am 17.01.2014 um 02:42 schrieb Bill Scheidel <[email protected]>: > >> So does that mean multiple fsync operations are done? I'm trying to figure >> out what would cause the extra 110ms of delay between no indexing and >> indexing one property. >> >> On Thursday, January 16, 2014 5:39:32 PM UTC-8, Michael Hunger wrote: >> Because it is transactional. >> >> So during your tx whenever things are changed / created that correspond to >> the configured auto-indexing properties they are written to the index. >> >> Michael >> >> Am 17.01.2014 um 02:35 schrieb Bill Scheidel <[email protected]>: >> >>> Hmm... turning off node_auto_indexing drops it from 150ms to 40ms. Why >>> would auto indexing block a request? >>> >>> On Thursday, January 16, 2014 5:07:48 PM UTC-8, Bill Scheidel wrote: >>> My hdparm results are 118 MB/sec which isn't horrible, but it seems like >>> disk latency is the only thing that matters. I guess I'll try going back >>> to the stock settings and moving it over to an SSD and see what happens. >>> >>> On Thursday, January 16, 2014 4:34:30 PM UTC-8, Wes Freeman wrote: >>> My macbook's virtualbox (running centos) got good results too (99% <20ms, >>> 50% <7ms). Was hoping for some weirdness. It is running on an ssd (vintage >>> 2011 macbook pro 13"), hdparm 250MB/sec, so not a great comparison. Only >>> has 800MB allocated for the VM RAM, using Neo4j stock settings. >>> >>> Wes >>> >>> On Thu, Jan 16, 2014 at 7:08 PM, Bill Scheidel <[email protected]> wrote: >>> Yeah, definitely not great. Odd though since I never had a problem when >>> working with Postgres or Mongo and they force things to disk as well. >>> Never had local requests take more than a couple of ms and then I switch >>> over to Neo4j and its almost unusable. There are no flags to change the >>> behavior for dev/test machines? >>> >>> >>> On Thursday, January 16, 2014 4:00:10 PM UTC-8, Michael Hunger wrote: >>> Not so good latency during the test, or? >>> >>> Here is my ioping (cool never heard of that one, nice tool). >>> >>> w/o ab >>> --- /tmp/ (hfs /dev/disk0s2) ioping statistics --- >>> 11 requests completed in 10.2 s, 32.1 k iops, 125.3 mb/s >>> min/avg/max/mdev = 21 us / 31 us / 50 us / 7 us >>> >>> w ab >>> 29 requests completed in 29.0 s, 33.1 k iops, 129.3 mb/s >>> min/avg/max/mdev = 20 us / 30 us / 109 us / 17 us >>> >>> Am 17.01.2014 um 00:54 schrieb Bill Scheidel <[email protected]>: >>> >>> > And this is ioping without the ab test running: >>> > >>> > 31 requests completed in 30.5 s, 3.2 k iops, 12.5 mb/s >>> > min/avg/max/mdev = 190 us / 312 us / 477 us / 63 us >>> > >>> > On Thursday, January 16, 2014 3:51:32 PM UTC-8, Bill Scheidel wrote: >>> > I ran vmstat while running the ab test: >>> > >>> > procs -----------memory---------- ---swap-- -----io---- -system-- >>> > ----cpu---- >>> > r b swpd free buff cache si so bi bo in cs us sy >>> > id wa >>> > 0 1 0 2429284 161572 2133284 0 0 22 75 98 354 9 8 >>> > 80 2 >>> > >>> > I also ran ioping to check disk latency while the ab test was running: >>> > >>> > 190 requests completed in 3.3 min, 37 iops, 151.9 kb/s >>> > min/avg/max/mdev = 192 us / 26.3 ms / 178.1 ms / 26.5 ms >>> > >>> > Results from the ab run: >>> > >>> > Percentage of the requests served within a certain time (ms) >>> > 50% 150 >>> > 66% 158 >>> > 75% 166 >>> > 80% 168 >>> > 90% 209 >>> > 95% 276 >>> > 98% 300 >>> > 99% 324 >>> > 100% 366 (longest request) >>> > >>> > >>> > >>> > On Thursday, January 16, 2014 3:33:55 PM UTC-8, Michael Hunger wrote: >>> > Let's continue this discussion here. >>> > >>> > To collect the other information so far: >>> > http://stackoverflow.com/questions/21145723/neo4j-2-0-0-poor-performance-for-dev-test-in-a-virtual-machine >>> > >>> > The GH issue you raised with Wes' and my answers: >>> > https://github.com/neo4j/neo4j/issues/1829 >>> > >>> > My "ab" tests: https://gist.github.com/jexp/8452037 >>> > Wes' numbers: >>> > https://github.com/neo4j/neo4j/issues/1829#issuecomment-32564561 >>> > >>> > Your messages.log looks good to me. >>> > >>> > So it might be related to disk performance, could you run vmstat or >>> > similar while running the ab test? >>> > >>> > I think it is related to the forced fsync at commit which can be hit by a >>> > higher disk latency? >>> > >>> > Michael >>> > >>> > -- >>> > You received this message because you are subscribed to the Google Groups >>> > "Neo4j" group. >>> > To unsubscribe from this group and stop receiving emails from it, send an >>> > email to [email protected]. >>> > For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Neo4j" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Neo4j" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
