Not sure how ScalaGraph works, but in reference to your code if this:
sg.addVertex("some label").setProperty("property ${math.random}",
math.random)
is not doing this
sg.addVertex(label, "some label", "property ${math.random}", math.random)
your getting some extra transactional checks that aren't necessary if you
know all the properties up-front at the time the vertex is added. changing
that probably isn't going to help your load times all that much, but i
thought i'd mention it.
After a quick review, I think a "batch trait" might be possible. Recall
that Neo4jBatchGraph used the Neo4j BatchInserter and not a Neo4jGraphAPI
instance - again unsure of how that fits here at the moment. I'm guessing
that BatchInserter is not exposed via apache licensed neo4j interfaces at
this time so that would be another problem.
On Thu, Jun 11, 2015 at 7:22 PM, Michael Pollmeier <
[email protected]> wrote:
> Yup that's what I meant.
>
>
> On 12/06/15 10:54, Marko Rodriguez wrote:
>
>> Hey Michael,
>>
>> I'll let Stephen confirm the path forward, but I bet you could use a
>> Neo4jTrait implementation to do the TinkerPop2-style Neo4jBatchGraph.
>> Checkout Neo4jTrait in TinkerPop3.
>>
>> Marko.
>>
>> http://markorodriguez.com
>>
>> On Jun 11, 2015, at 4:07 PM, Michael Pollmeier <
>> [email protected]> wrote:
>>
>> Here you go - all pretty straightforward:
>>> https://gist.github.com/mpollmeier/108ab8998e3b0321f020
>>>
>>> Without a bulk api for neo4j it takes nearly 70s to create 30k vertices.
>>>
>>> There is currently no publicly available build artifact of tinkerpop3
>>> that contains neo4j-gremlin, so this all depends on custom local builds.
>>>
>>> The main thing is the obvious stuff I guess: disable indexes and
>>> integrity constraints. Should be the same as the good old Neo4jBatchGraph,
>>> no?
>>>
>>> https://github.com/tinkerpop/blueprints/tree/master/blueprints-neo4j-graph/src/main/java/com/tinkerpop/blueprints/impls/neo4j/batch
>>> I guess for license reasons we can't just copy that but have to
>>> reimplement it?
>>>
>>> Cheers
>>> Michael
>>>
>>> On 12/06/15 01:19, Marko Rodriguez wrote:
>>>
>>>> Michael,
>>>>
>>>> If there are any areas that can be sped up in the Neo4jGraph codebase,
>>>> please identify. There hasn't been much manual testing of Neo4jGraph so any
>>>> help/tickets/PRs you could provide would be greatly appreciated.
>>>>
>>>> Also, could you share your data loading code via a gist?
>>>>
>>>> Thanks,
>>>> Marko.
>>>>
>>>> http://markorodriguez.com
>>>>
>>>> On Jun 11, 2015, at 6:37 AM, Stephen Mallette <[email protected]>
>>>> wrote:
>>>>
>>>> General support for bulk loading is still under development and won't
>>>>> be available for GA. Here's some related issues:
>>>>>
>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-319
>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-320
>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-550
>>>>>
>>>>> As for just straight loading, I'm not sure we've evaluated performance
>>>>> of neo4j-gremlin at this point. Not sure what improvements are to be had,
>>>>> if any.
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 10, 2015 at 8:01 PM, Michael Pollmeier <
>>>>> [email protected]> wrote:
>>>>> What's the best way to insert many elements at the same time, is there
>>>>> some sort of bulk mode that doesn't check for constraints? I didn't find
>>>>> anything fitting in the code/documentation in TP3.
>>>>>
>>>>> Creating 25k vertices takes around 40s, and about 50s if I wrap it in
>>>>> a transaction - that's a bit slow.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Gremlin-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/gremlin-users/5578CFDB.5070603%40michaelpollmeier.com
>>>>> .
>>>>>
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Gremlin-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/gremlin-users/CAA-H438DLd-K%2Baj%3DufCx5w3EtY6r32bTTVznTRgoNYwYOMXxCA%40mail.gmail.com
>>>>> .
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>>
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Gremlin-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/gremlin-users/557A0695.9090807%40michaelpollmeier.com
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/557A1819.2010108%40michaelpollmeier.com
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
