Hi Vasia, I agree, Gremlin definitely looks like an interesting API for Flink. I'm not sure how it relates to Gelly. I guess Gelly would (initially) be more tightly integrated with the DataSet API whereas Gremlin would be a connector for other languages. Any ideas on this?
Another question would be whether the connector should to into Flink or Tinkerpop. For example, the Spark, Giraph, and Neo4J connectors are all included in Tinkerpop. This should be discussed with the Tinkerpop community. Best, Fabian 2015-12-14 18:33 GMT+01:00 Vasiliki Kalavri <vasilikikala...@gmail.com>: > Ping squirrels! Any thoughts/opinions on this? > > On 9 December 2015 at 20:40, Vasiliki Kalavri <vasilikikala...@gmail.com> > wrote: > > > Hello squirrels, > > > > I have been discussing with the Apache Tinkerpop [1] community regarding > > an integration with Flink/Gelly. > > You can read our discussion in [2]. > > > > Tinkerpop has a graph traversal machine called Gremlin, which supports > > many high-level graph processing languages and runs on top of different > > systems (e.g. Giraph, Spark, Graph DBs). You can read more in this great > > blog post [3]. > > > > The idea is to provide a FlinkGraphComputer implementation, which will > add > > Gremlin support to Flink. > > > > I believe Tinkerpop is a great project and I would love to see an > > integration with Gelly. > > Before we move forward, I would like your input! > > To me, it seems that this addition would nicely fit in flink-contrib, > > where we also have connectors to other projects. > > If you agree, I will go ahead and open a JIRA about it. > > > > Thank you! > > -Vasia. > > > > [1]: https://tinkerpop.incubator.apache.org/ > > [2]: > > > https://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201511.mbox/%3ccanva_a390l7g169r8sn+ej1-yfkbudlnd4td6atwnp0uza-...@mail.gmail.com%3E > > [3]: > > > http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine > > > > On 25 November 2015 at 16:54, Vasiliki Kalavri < > vasilikikala...@gmail.com> > > wrote: > > > >> Hi James, > >> > >> I've just subscribed to the Tinkerpop dev mailing list. Could you please > >> send a reply to the thread, so then I can reply to it? > >> I'm not sure how I can reply to the thread otherwise... > >> I also saw that there is a grafos.ml project thread. I could also > >> provide some input there :) > >> > >> Thanks! > >> -Vasia. > >> > >> On 25 November 2015 at 15:09, James Thornton <james.thorn...@gmail.com> > >> wrote: > >> > >>> Hi Vasia - > >>> > >>> Yes, a FlinkGraphComputer should be a straight-forward first step. > Also, > >>> on > >>> the Apache Tinkerpop dev mailing list, Marko thought it might be cool > if > >>> there was a "Graph API" similar to the "Table API" -- hooking in > Gremlin > >>> to > >>> Flink's fluent API would give Flink users a full graph query language. > >>> > >>> Stephen Mallette is a TinkerPop core contributor, and he has already > >>> started working on a FlinkGraphComputer. There is a Flink/Tinkerpop > >>> thread > >>> on the TinkerPop dev list -- it would be great to have you part of the > >>> conversation there too as we work on the integration: > >>> > >>> http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/ > >>> > >>> Thanks, Vasia. > >>> > >>> - James > >>> > >>> > >>> On Mon, Nov 23, 2015 at 10:28 AM, Vasiliki Kalavri < > >>> vasilikikala...@gmail.com> wrote: > >>> > >>> > Hi James, > >>> > > >>> > thank you for your e-mail and your interest in Flink :) > >>> > > >>> > I've recently taken a _quick_ look into Apache TinkerPop and I think > >>> it'd > >>> > be very interesting to integrate with Flink/Gelly. > >>> > Are you thinking about something like a Flink GraphComputer, similar > to > >>> > Giraph and Spark GraphComputer's? > >>> > I believe such an integration should be straight-forward to > implement. > >>> You > >>> > can start by looking into Flink iteration operators [1] and Gelly > >>> iteration > >>> > abstractions [2]. > >>> > > >>> > Regarding Apache Geode, I'm not familiar with project, but I'll try > to > >>> take > >>> > a look in the following days! > >>> > > >>> > Cheers, > >>> > -Vasia. > >>> > > >>> > > >>> > [1]: > >>> > > >>> > > >>> > https://ci.apache.org/projects/flink/flink-docs-master/apis/programming_guide.html#iteration-operators > >>> > [2]: > >>> > > >>> > > >>> > https://ci.apache.org/projects/flink/flink-docs-master/libs/gelly_guide.html#iterative-graph-processing > >>> > > >>> > > >>> > On 20 November 2015 at 08:32, James Thornton < > james.thorn...@gmail.com > >>> > > >>> > wrote: > >>> > > >>> > > Hi - > >>> > > > >>> > > This is James Thornton (espeed) from the Apache Tinkerpop project ( > >>> > > http://tinkerpop.incubator.apache.org/). > >>> > > > >>> > > The Flink iterators should pair well with Gremlin's Graph Traversal > >>> > Machine > >>> > > ( > >>> > > > >>> > > > >>> > > >>> > http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine > >>> > > ) > >>> > > -- it would be good to coordinate on creating an integration. > >>> > > > >>> > > Also, Apache Geode made a splash today on HN ( > >>> > > https://news.ycombinator.com/item?id=10596859) -- connecting Geode > >>> and > >>> > > Flink would be killer. Here's the Geode/Spark connector for > >>> refefference: > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > >>> > https://github.com/apache/incubator-geode/tree/develop/gemfire-spark-connector > >>> > > > >>> > > - James > >>> > > > >>> > > >>> > >> > >> > > >
