2010/6/11 Atle Prange <atle.pra...@gmail.com>: > Regarding the performance: > > I have tried to tweak Babudb, but could not get any more out of it > than you did. I guess when you put 5M entries at once, there is only > so much tweaking one can do. (At least for babudb). It seems odd > though, since babudb is a less complicated framework, with less > features. We should contact the babudb guys and ask why we see the > results that we see. > > Regarding keys: > > Having one entry in the db for each value will give the fastest update > speed, but will be slower for a large number of nodes with the same > value. Having an entry with all the ids will make a large entry, > causing more bytes to be written to disk for every update.
I tried having keys without the value in them (also making the index only support zero or one value per key) and it was very fast. If this approach would support multiple ids in a value, insertion speed would be even slower (due to lookup for every insert). But it would definately be an interesting implementation to use for one-to-one lookups! Also I noticed that insertion speed degrades quite heavily around 5M entries. Numbers: 1000 lookups (in an index with 5M key/value pairs): 30-40ms (0.03 ms/lookup) that is quite impressive and may be worth implementing as an index provider for such indices. > > > On Fri, Jun 11, 2010 at 1:51 PM, Tobias Ivarsson > <tobias.ivars...@neotechnology.com> wrote: >> On Fri, Jun 11, 2010 at 12:54 PM, Mattias Persson <matt...@neotechnology.com >>> wrote: >> >>> >>> BabuDb has a >>> >>> db.prefixLookup( "key|value|" ) >>> >>> method so that's the one I'm using. >>> >> >> I wonder how well that scales. I'd like to see some performance figures from >> indexing massive volumes and then doing lookup based on prefixLookup. >> >> -- >> Tobias Ivarsson <tobias.ivars...@neotechnology.com> >> Hacker, Neo Technology >> www.neotechnology.com >> Cellphone: +46 706 534857 >> _______________________________________________ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
