wow - i didn't expect that to make such a difference. i may need to look at the code again to see why there is such a big difference considering i dismissed it so easily as impacting performance.
On Fri, Jun 12, 2015 at 8:25 PM, Michael Pollmeier < [email protected]> wrote: > That's a good tip! I just tried with a different example in neo4j and the > graph creation time went from 50s using `sg.addVertex("some > label").setProperty(s"property ${math.random}", math.random)` > to 40s using `sg.addVertex("some label", Map(s"property ${math.random}", > math.random))` > > On 12/06/15 22:06, Stephen Mallette wrote: > >> Not sure how ScalaGraph works, but in reference to your code if this: >> >> sg.addVertex("some label").setProperty("property ${math.random}", >> math.random) >> >> is not doing this >> >> sg.addVertex(label, "some label", "property ${math.random}", math.random) >> >> your getting some extra transactional checks that aren't necessary if you >> know all the properties up-front at the time the vertex is added. >> changing >> that probably isn't going to help your load times all that much, but i >> thought i'd mention it. >> >> After a quick review, I think a "batch trait" might be possible. Recall >> that Neo4jBatchGraph used the Neo4j BatchInserter and not a Neo4jGraphAPI >> instance - again unsure of how that fits here at the moment. I'm guessing >> that BatchInserter is not exposed via apache licensed neo4j interfaces at >> this time so that would be another problem. >> >> >> >> On Thu, Jun 11, 2015 at 7:22 PM, Michael Pollmeier < >> [email protected]> wrote: >> >> Yup that's what I meant. >>> >>> >>> On 12/06/15 10:54, Marko Rodriguez wrote: >>> >>> Hey Michael, >>>> >>>> I'll let Stephen confirm the path forward, but I bet you could use a >>>> Neo4jTrait implementation to do the TinkerPop2-style Neo4jBatchGraph. >>>> Checkout Neo4jTrait in TinkerPop3. >>>> >>>> Marko. >>>> >>>> http://markorodriguez.com >>>> >>>> On Jun 11, 2015, at 4:07 PM, Michael Pollmeier < >>>> [email protected]> wrote: >>>> >>>> Here you go - all pretty straightforward: >>>> >>>>> https://gist.github.com/mpollmeier/108ab8998e3b0321f020 >>>>> >>>>> Without a bulk api for neo4j it takes nearly 70s to create 30k >>>>> vertices. >>>>> >>>>> There is currently no publicly available build artifact of tinkerpop3 >>>>> that contains neo4j-gremlin, so this all depends on custom local >>>>> builds. >>>>> >>>>> The main thing is the obvious stuff I guess: disable indexes and >>>>> integrity constraints. Should be the same as the good old >>>>> Neo4jBatchGraph, >>>>> no? >>>>> >>>>> >>>>> https://github.com/tinkerpop/blueprints/tree/master/blueprints-neo4j-graph/src/main/java/com/tinkerpop/blueprints/impls/neo4j/batch >>>>> I guess for license reasons we can't just copy that but have to >>>>> reimplement it? >>>>> >>>>> Cheers >>>>> Michael >>>>> >>>>> On 12/06/15 01:19, Marko Rodriguez wrote: >>>>> >>>>> Michael, >>>>>> >>>>>> If there are any areas that can be sped up in the Neo4jGraph codebase, >>>>>> please identify. There hasn't been much manual testing of Neo4jGraph >>>>>> so any >>>>>> help/tickets/PRs you could provide would be greatly appreciated. >>>>>> >>>>>> Also, could you share your data loading code via a gist? >>>>>> >>>>>> Thanks, >>>>>> Marko. >>>>>> >>>>>> http://markorodriguez.com >>>>>> >>>>>> On Jun 11, 2015, at 6:37 AM, Stephen Mallette <[email protected]> >>>>>> wrote: >>>>>> >>>>>> General support for bulk loading is still under development and >>>>>> won't >>>>>> >>>>>>> be available for GA. Here's some related issues: >>>>>>> >>>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-319 >>>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-320 >>>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-550 >>>>>>> >>>>>>> As for just straight loading, I'm not sure we've evaluated >>>>>>> performance >>>>>>> of neo4j-gremlin at this point. Not sure what improvements are to >>>>>>> be had, >>>>>>> if any. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 10, 2015 at 8:01 PM, Michael Pollmeier < >>>>>>> [email protected]> wrote: >>>>>>> What's the best way to insert many elements at the same time, is >>>>>>> there >>>>>>> some sort of bulk mode that doesn't check for constraints? I didn't >>>>>>> find >>>>>>> anything fitting in the code/documentation in TP3. >>>>>>> >>>>>>> Creating 25k vertices takes around 40s, and about 50s if I wrap it in >>>>>>> a transaction - that's a bit slow. >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Gremlin-users" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send >>>>>>> an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> >>>>>>> https://groups.google.com/d/msgid/gremlin-users/5578CFDB.5070603%40michaelpollmeier.com >>>>>>> . >>>>>>> >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Gremlin-users" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send >>>>>>> an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> >>>>>>> https://groups.google.com/d/msgid/gremlin-users/CAA-H438DLd-K%2Baj%3DufCx5w3EtY6r32bTTVznTRgoNYwYOMXxCA%40mail.gmail.com >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Gremlin-users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> >>>>> https://groups.google.com/d/msgid/gremlin-users/557A0695.9090807%40michaelpollmeier.com >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Gremlin-users" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> To view this discussion on the web visit >>> >>> https://groups.google.com/d/msgid/gremlin-users/557A1819.2010108%40michaelpollmeier.com >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >> > -- > You received this message because you are subscribed to the Google Groups > "Gremlin-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/gremlin-users/557B786B.2040504%40michaelpollmeier.com > . > For more options, visit https://groups.google.com/d/optout. >
