Updated Gelly Roadmap
Hi all, as promised, I have updated the Gelly roadmap [1]. Below, I am describing and reasoning about the changes I made. Please, let me know whether you agree and if you have any other ideas for further improvements and feature additions. *1. Operators for highly skewed graphs*: I have removed this item completely. It referred to Andra's master thesis which has been completed. We had a discussion back then (see [2]), but no activity after that. *2. Scala API*: I removed this item because it's done *3. Graph Streaming*: I removed this item also. We have built an experimental API for graph streaming with Paris and KTH students. The code is available in [3]. If you think it would be a valuable addition as a library to Flink, we can start a separate discussion thread about it. *4. Library Methods*: - Affinity Propagation: this is WIP in FLINK-1707 - HITS + Adsorption: Removed. They were started by TUB students and not finished. I propose to only revisit these only if someone asks for them. - Strongly CC + DIA: Removed. Nobody has worked on them AFAIK. *5. Graph partitioning*: This is still relevant in my opinion and thus I kept it in the updated roadmap. *6. Partition-centric iterations*: We have created a POC implementation with KTH students [4]. In my opinion, it would be nice to add this to Gelly and most of the work has already been done, so I kept it. *7. Generic Iterations*: This requires caching intermediate results. Anyone has a status update on that? *8. Performance evaluation*: I'm currently working on this, integrating Gelly with the Graphalytics benchmark. My WIP is in [5] in case you want to get involved :) *9. Bipartite support*: This is still relevant in my opinion. Someone had started working on it, but has been inactive for a while. I pinged the JIRA (FLINK-2254). >From the wishlist: - *Neo4j* input/output formats have been implemented as an external project by Martin Junghanns. Shall we go ahead and them to project-flink? We should definitely link to this from the third-party packages. - *TinkerPop*: Discussion started from the TinkerPop community, but there was not much activity from our side [6]. I wrote my opinion then and I am very much in favor. Anyone else wants to share their thoughts? Looking forward to your input, -Vasia. [1]: https://cwiki.apache.org/confluence/display/FLINK/Flink+Gelly [2]: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Proposal-Addition-to-Gelly-td7436.html [3]: https://github.com/vasia/gelly-streaming [4]: https://github.com/vasia/gelly-partition-centric [5]: https://github.com/vasia/graphalytics-platforms-gelly [6]: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Apache-Tinkerpop-amp-Geode-Integration-td9252.html
Re: Gelly Roadmap
On 03 Jun 2015, at 23:51, Stephan Ewen se...@apache.org wrote: It may make sense to link the JIRAs from the roadmap, or add comments what people are working on and what is in the queue. Linking JIRAs from the roadmap would solve all these points. ;) So +1 for issue links.
Re: Gelly Roadmap
Hey, yes, this is kind of an ambitious and long-term roadmap :) As Andra said, tasks 1-2-3-4 are already work in progress. Regarding the rest, we could discuss on the priority or leave it on the personal interest of contributors. Personally, I think bipartite support should be quite easy to add and a very nice feature to have. Then, I would love to see some support for partitioning / partition-centric iterations and generic iterations of course. -Vasia. On 3 June 2015 at 14:58, Andra Lungu lungu.an...@gmail.com wrote: Hi Stephan, Well the operators fro skewed graphs and the Gelly streaming project will be done by the end of the semester. Otherwise Daniel and I won't graduate :); The Scala API is ongoing work (there is a link to the corresponding JIRA in the roadmap); The Library methods are in the making; First two are being written; the second two are potential tasks for a Shivani, an intern I am supervising at DIMA; Graph Partitioning is also programmed for the near future. There is a JIRA there with the corresponding conversations; The generic iterations are still being discussed. We reached a conclusion there, but as far as I know nobody is working on it yet; Performance evaluation will probably be done with graphalytics, but that and the bipartite graph support are future tasks (there is nobody working on them yet); The wish-list is exactly as its name states, a wish-list :) All the other items are either currently addressed or will be addressed in the foreseeable future. The order is more or less the one on the roadmap. What's on the bottom is more of a set of wishes, whereas the tasks at the beginning are considered a priority and are being addressed. Andra On Wed, Jun 3, 2015 at 2:45 PM, Stephan Ewen se...@apache.org wrote: Wow, this is an extremely big and ambitious roadmap. Are all these things realistically possible in the foreseeable future? Or is this more a wish-list? Do you have an idea what issues to approach in what order? On Wed, May 20, 2015 at 8:58 PM, Andra Lungu lungu.an...@gmail.com wrote: The Roadmap is now available as a wiki page. https://cwiki.apache.org/confluence/display/FLINK/Flink+Gelly We're still happy to hear comments/suggestions, if any :) On Wed, May 20, 2015 at 8:43 PM, Andra Lungu lungu.an...@gmail.com wrote: Thanks, Stephan! On Wed, May 20, 2015 at 8:42 PM, Stephan Ewen se...@apache.org wrote: All right, you should have permissions now. On Wed, May 20, 2015 at 8:37 PM, Andra Lungu lungu.an...@gmail.com wrote: Sure, but first I need permissions! :) *NOTE*: Due to spamming, we can not give every confluence user edit permissions to the wiki. Just write to the dev@flink.apache.org (you can also email to rmetzger at apache.org) mailing list to get edit permissions. My user is lungu.andra Thanks! On Wed, May 20, 2015 at 4:27 PM, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Thank you for your feedback and ideas everyone! @Andra, how about moving the roadmap to the wiki? On 20 May 2015 at 15:48, Kostas Tzoumas ktzou...@apache.org wrote: :-D Great! On Tue, May 19, 2015 at 4:00 PM, Andra Lungu lungu.an...@gmail.com wrote: Hi Kostas, We're way ahead of you! The first draft of the blog post is internally reviewed as we speak ;) On Tue, May 19, 2015 at 3:49 PM, Kostas Tzoumas ktzou...@apache.org wrote: This is very cool! Would also love to see a blog post on Gelly at some point, perhaps finishing off with the roadmap On Tue, May 19, 2015 at 10:55 AM, Andra Lungu lungu.an...@gmail.com wrote: Apache TinkerPop made it to the wish-list :) Thanks, Fabian! On Tue, May 19, 2015 at 9:42 AM, Flavio Pompermaier pomperma...@okkam.it wrote: You're definitely right Fabian +1 On Mon, May 18, 2015 at 2:03 PM, Fabian Hueske fhue...@gmail.com wrote: Integration with Apache TinkerPop3 could also be interesting. TinkerPop3 is an API for transactional and analytical graph processing and supported by several Graph engines/databases. It might be interesting to see if/how Gelly's and TinkerPop's concepts match and think about whether it makes sense to integrate the two projects in some way. 2015-05-18 12:20 GMT+02:00 Gianmarco De Francisci Morales g...@apache.org : Hi,
Re: Gelly Roadmap
Wow, this is an extremely big and ambitious roadmap. Are all these things realistically possible in the foreseeable future? Or is this more a wish-list? Do you have an idea what issues to approach in what order? On Wed, May 20, 2015 at 8:58 PM, Andra Lungu lungu.an...@gmail.com wrote: The Roadmap is now available as a wiki page. https://cwiki.apache.org/confluence/display/FLINK/Flink+Gelly We're still happy to hear comments/suggestions, if any :) On Wed, May 20, 2015 at 8:43 PM, Andra Lungu lungu.an...@gmail.com wrote: Thanks, Stephan! On Wed, May 20, 2015 at 8:42 PM, Stephan Ewen se...@apache.org wrote: All right, you should have permissions now. On Wed, May 20, 2015 at 8:37 PM, Andra Lungu lungu.an...@gmail.com wrote: Sure, but first I need permissions! :) *NOTE*: Due to spamming, we can not give every confluence user edit permissions to the wiki. Just write to the dev@flink.apache.org (you can also email to rmetzger at apache.org) mailing list to get edit permissions. My user is lungu.andra Thanks! On Wed, May 20, 2015 at 4:27 PM, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Thank you for your feedback and ideas everyone! @Andra, how about moving the roadmap to the wiki? On 20 May 2015 at 15:48, Kostas Tzoumas ktzou...@apache.org wrote: :-D Great! On Tue, May 19, 2015 at 4:00 PM, Andra Lungu lungu.an...@gmail.com wrote: Hi Kostas, We're way ahead of you! The first draft of the blog post is internally reviewed as we speak ;) On Tue, May 19, 2015 at 3:49 PM, Kostas Tzoumas ktzou...@apache.org wrote: This is very cool! Would also love to see a blog post on Gelly at some point, perhaps finishing off with the roadmap On Tue, May 19, 2015 at 10:55 AM, Andra Lungu lungu.an...@gmail.com wrote: Apache TinkerPop made it to the wish-list :) Thanks, Fabian! On Tue, May 19, 2015 at 9:42 AM, Flavio Pompermaier pomperma...@okkam.it wrote: You're definitely right Fabian +1 On Mon, May 18, 2015 at 2:03 PM, Fabian Hueske fhue...@gmail.com wrote: Integration with Apache TinkerPop3 could also be interesting. TinkerPop3 is an API for transactional and analytical graph processing and supported by several Graph engines/databases. It might be interesting to see if/how Gelly's and TinkerPop's concepts match and think about whether it makes sense to integrate the two projects in some way. 2015-05-18 12:20 GMT+02:00 Gianmarco De Francisci Morales g...@apache.org : Hi, Great job putting together this roadmap! Regarding 1) you may find this work relevant: http://arxiv.org/abs/1503.00626v1 Regarding 2) the semi-streaming model allows polylog(n) passes. If you allow only one pass, it's a streaming model. In addition, I think models such as the turnstile/fully dynamic (with addition and deletion) would be more appropriate than the incremental one. Regarding 5) (when combined with 2) there are some good recent algorithms for partitioning that you might want to look at. http://dl.acm.org/citation.cfm?id=2556213 http://dl.acm.org/citation.cfm?id=2339722 Cheers, -- Gianmarco On 18 May 2015 at 12:42, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Hello everyone, thank you Andra for putting together this nice document and for sharing! Gelly is already in a state that matches the basic functionality of existing distributed graph libraries. We're very happy for this and also very grateful to everyone who has contributed so far! We are also very excited to keep on extending Gelly and our vision is to turn it into something much more than yet another library for running PageRank-like algorithms. The document that Andra has shared contains ongoing work and our ideas on mid- and long-term plans, in order to make this happen :) We would love to get the community's feedback on this, so please feel free to
Re: Gelly Roadmap
:-D Great! On Tue, May 19, 2015 at 4:00 PM, Andra Lungu lungu.an...@gmail.com wrote: Hi Kostas, We're way ahead of you! The first draft of the blog post is internally reviewed as we speak ;) On Tue, May 19, 2015 at 3:49 PM, Kostas Tzoumas ktzou...@apache.org wrote: This is very cool! Would also love to see a blog post on Gelly at some point, perhaps finishing off with the roadmap On Tue, May 19, 2015 at 10:55 AM, Andra Lungu lungu.an...@gmail.com wrote: Apache TinkerPop made it to the wish-list :) Thanks, Fabian! On Tue, May 19, 2015 at 9:42 AM, Flavio Pompermaier pomperma...@okkam.it wrote: You're definitely right Fabian +1 On Mon, May 18, 2015 at 2:03 PM, Fabian Hueske fhue...@gmail.com wrote: Integration with Apache TinkerPop3 could also be interesting. TinkerPop3 is an API for transactional and analytical graph processing and supported by several Graph engines/databases. It might be interesting to see if/how Gelly's and TinkerPop's concepts match and think about whether it makes sense to integrate the two projects in some way. 2015-05-18 12:20 GMT+02:00 Gianmarco De Francisci Morales g...@apache.org : Hi, Great job putting together this roadmap! Regarding 1) you may find this work relevant: http://arxiv.org/abs/1503.00626v1 Regarding 2) the semi-streaming model allows polylog(n) passes. If you allow only one pass, it's a streaming model. In addition, I think models such as the turnstile/fully dynamic (with addition and deletion) would be more appropriate than the incremental one. Regarding 5) (when combined with 2) there are some good recent algorithms for partitioning that you might want to look at. http://dl.acm.org/citation.cfm?id=2556213 http://dl.acm.org/citation.cfm?id=2339722 Cheers, -- Gianmarco On 18 May 2015 at 12:42, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Hello everyone, thank you Andra for putting together this nice document and for sharing! Gelly is already in a state that matches the basic functionality of existing distributed graph libraries. We're very happy for this and also very grateful to everyone who has contributed so far! We are also very excited to keep on extending Gelly and our vision is to turn it into something much more than yet another library for running PageRank-like algorithms. The document that Andra has shared contains ongoing work and our ideas on mid- and long-term plans, in order to make this happen :) We would love to get the community's feedback on this, so please feel free to comment on the doc or discuss individual points here. Cheers, Vasia. On 18 May 2015 at 11:01, Andra Lungu lungu.an...@gmail.com wrote: Dear Squirrels, The following document is intended to give you an insight on where Flink's Graph API, Gelly, is at the moment and especially on the direction in which it's heading in the near future. https://docs.google.com/document/d/11ovK1xZZTl3Qinw6Ud1rqpaDhTvRHDv8PWBzcHITFkM/edit?usp=sharing We would love to hear about your comments/ suggestions/ additions to the wish-list :) Cheers, Andra
Re: Gelly Roadmap
Thanks, Stephan! On Wed, May 20, 2015 at 8:42 PM, Stephan Ewen se...@apache.org wrote: All right, you should have permissions now. On Wed, May 20, 2015 at 8:37 PM, Andra Lungu lungu.an...@gmail.com wrote: Sure, but first I need permissions! :) *NOTE*: Due to spamming, we can not give every confluence user edit permissions to the wiki. Just write to the dev@flink.apache.org (you can also email to rmetzger at apache.org) mailing list to get edit permissions. My user is lungu.andra Thanks! On Wed, May 20, 2015 at 4:27 PM, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Thank you for your feedback and ideas everyone! @Andra, how about moving the roadmap to the wiki? On 20 May 2015 at 15:48, Kostas Tzoumas ktzou...@apache.org wrote: :-D Great! On Tue, May 19, 2015 at 4:00 PM, Andra Lungu lungu.an...@gmail.com wrote: Hi Kostas, We're way ahead of you! The first draft of the blog post is internally reviewed as we speak ;) On Tue, May 19, 2015 at 3:49 PM, Kostas Tzoumas ktzou...@apache.org wrote: This is very cool! Would also love to see a blog post on Gelly at some point, perhaps finishing off with the roadmap On Tue, May 19, 2015 at 10:55 AM, Andra Lungu lungu.an...@gmail.com wrote: Apache TinkerPop made it to the wish-list :) Thanks, Fabian! On Tue, May 19, 2015 at 9:42 AM, Flavio Pompermaier pomperma...@okkam.it wrote: You're definitely right Fabian +1 On Mon, May 18, 2015 at 2:03 PM, Fabian Hueske fhue...@gmail.com wrote: Integration with Apache TinkerPop3 could also be interesting. TinkerPop3 is an API for transactional and analytical graph processing and supported by several Graph engines/databases. It might be interesting to see if/how Gelly's and TinkerPop's concepts match and think about whether it makes sense to integrate the two projects in some way. 2015-05-18 12:20 GMT+02:00 Gianmarco De Francisci Morales g...@apache.org : Hi, Great job putting together this roadmap! Regarding 1) you may find this work relevant: http://arxiv.org/abs/1503.00626v1 Regarding 2) the semi-streaming model allows polylog(n) passes. If you allow only one pass, it's a streaming model. In addition, I think models such as the turnstile/fully dynamic (with addition and deletion) would be more appropriate than the incremental one. Regarding 5) (when combined with 2) there are some good recent algorithms for partitioning that you might want to look at. http://dl.acm.org/citation.cfm?id=2556213 http://dl.acm.org/citation.cfm?id=2339722 Cheers, -- Gianmarco On 18 May 2015 at 12:42, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Hello everyone, thank you Andra for putting together this nice document and for sharing! Gelly is already in a state that matches the basic functionality of existing distributed graph libraries. We're very happy for this and also very grateful to everyone who has contributed so far! We are also very excited to keep on extending Gelly and our vision is to turn it into something much more than yet another library for running PageRank-like algorithms. The document that Andra has shared contains ongoing work and our ideas on mid- and long-term plans, in order to make this happen :) We would love to get the community's feedback on this, so please feel free to comment on the doc or discuss individual points here. Cheers, Vasia. On 18 May 2015 at 11:01, Andra Lungu lungu.an...@gmail.com wrote: Dear Squirrels, The following document is intended to give you an insight on where Flink's Graph API, Gelly, is at the moment and especially on the direction in which it's heading in the near future. https://docs.google.com/document/d/11ovK1xZZTl3Qinw6Ud1rqpaDhTvRHDv8PWBzcHITFkM/edit?usp=sharing
Re: Gelly Roadmap
All right, you should have permissions now. On Wed, May 20, 2015 at 8:37 PM, Andra Lungu lungu.an...@gmail.com wrote: Sure, but first I need permissions! :) *NOTE*: Due to spamming, we can not give every confluence user edit permissions to the wiki. Just write to the dev@flink.apache.org (you can also email to rmetzger at apache.org) mailing list to get edit permissions. My user is lungu.andra Thanks! On Wed, May 20, 2015 at 4:27 PM, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Thank you for your feedback and ideas everyone! @Andra, how about moving the roadmap to the wiki? On 20 May 2015 at 15:48, Kostas Tzoumas ktzou...@apache.org wrote: :-D Great! On Tue, May 19, 2015 at 4:00 PM, Andra Lungu lungu.an...@gmail.com wrote: Hi Kostas, We're way ahead of you! The first draft of the blog post is internally reviewed as we speak ;) On Tue, May 19, 2015 at 3:49 PM, Kostas Tzoumas ktzou...@apache.org wrote: This is very cool! Would also love to see a blog post on Gelly at some point, perhaps finishing off with the roadmap On Tue, May 19, 2015 at 10:55 AM, Andra Lungu lungu.an...@gmail.com wrote: Apache TinkerPop made it to the wish-list :) Thanks, Fabian! On Tue, May 19, 2015 at 9:42 AM, Flavio Pompermaier pomperma...@okkam.it wrote: You're definitely right Fabian +1 On Mon, May 18, 2015 at 2:03 PM, Fabian Hueske fhue...@gmail.com wrote: Integration with Apache TinkerPop3 could also be interesting. TinkerPop3 is an API for transactional and analytical graph processing and supported by several Graph engines/databases. It might be interesting to see if/how Gelly's and TinkerPop's concepts match and think about whether it makes sense to integrate the two projects in some way. 2015-05-18 12:20 GMT+02:00 Gianmarco De Francisci Morales g...@apache.org : Hi, Great job putting together this roadmap! Regarding 1) you may find this work relevant: http://arxiv.org/abs/1503.00626v1 Regarding 2) the semi-streaming model allows polylog(n) passes. If you allow only one pass, it's a streaming model. In addition, I think models such as the turnstile/fully dynamic (with addition and deletion) would be more appropriate than the incremental one. Regarding 5) (when combined with 2) there are some good recent algorithms for partitioning that you might want to look at. http://dl.acm.org/citation.cfm?id=2556213 http://dl.acm.org/citation.cfm?id=2339722 Cheers, -- Gianmarco On 18 May 2015 at 12:42, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Hello everyone, thank you Andra for putting together this nice document and for sharing! Gelly is already in a state that matches the basic functionality of existing distributed graph libraries. We're very happy for this and also very grateful to everyone who has contributed so far! We are also very excited to keep on extending Gelly and our vision is to turn it into something much more than yet another library for running PageRank-like algorithms. The document that Andra has shared contains ongoing work and our ideas on mid- and long-term plans, in order to make this happen :) We would love to get the community's feedback on this, so please feel free to comment on the doc or discuss individual points here. Cheers, Vasia. On 18 May 2015 at 11:01, Andra Lungu lungu.an...@gmail.com wrote: Dear Squirrels, The following document is intended to give you an insight on where Flink's Graph API, Gelly, is at the moment and especially on the direction in which it's heading in the near future. https://docs.google.com/document/d/11ovK1xZZTl3Qinw6Ud1rqpaDhTvRHDv8PWBzcHITFkM/edit?usp=sharing We would love to hear about your comments/ suggestions/ additions to the wish-list :) Cheers, Andra
Re: Gelly Roadmap
The Roadmap is now available as a wiki page. https://cwiki.apache.org/confluence/display/FLINK/Flink+Gelly We're still happy to hear comments/suggestions, if any :) On Wed, May 20, 2015 at 8:43 PM, Andra Lungu lungu.an...@gmail.com wrote: Thanks, Stephan! On Wed, May 20, 2015 at 8:42 PM, Stephan Ewen se...@apache.org wrote: All right, you should have permissions now. On Wed, May 20, 2015 at 8:37 PM, Andra Lungu lungu.an...@gmail.com wrote: Sure, but first I need permissions! :) *NOTE*: Due to spamming, we can not give every confluence user edit permissions to the wiki. Just write to the dev@flink.apache.org (you can also email to rmetzger at apache.org) mailing list to get edit permissions. My user is lungu.andra Thanks! On Wed, May 20, 2015 at 4:27 PM, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Thank you for your feedback and ideas everyone! @Andra, how about moving the roadmap to the wiki? On 20 May 2015 at 15:48, Kostas Tzoumas ktzou...@apache.org wrote: :-D Great! On Tue, May 19, 2015 at 4:00 PM, Andra Lungu lungu.an...@gmail.com wrote: Hi Kostas, We're way ahead of you! The first draft of the blog post is internally reviewed as we speak ;) On Tue, May 19, 2015 at 3:49 PM, Kostas Tzoumas ktzou...@apache.org wrote: This is very cool! Would also love to see a blog post on Gelly at some point, perhaps finishing off with the roadmap On Tue, May 19, 2015 at 10:55 AM, Andra Lungu lungu.an...@gmail.com wrote: Apache TinkerPop made it to the wish-list :) Thanks, Fabian! On Tue, May 19, 2015 at 9:42 AM, Flavio Pompermaier pomperma...@okkam.it wrote: You're definitely right Fabian +1 On Mon, May 18, 2015 at 2:03 PM, Fabian Hueske fhue...@gmail.com wrote: Integration with Apache TinkerPop3 could also be interesting. TinkerPop3 is an API for transactional and analytical graph processing and supported by several Graph engines/databases. It might be interesting to see if/how Gelly's and TinkerPop's concepts match and think about whether it makes sense to integrate the two projects in some way. 2015-05-18 12:20 GMT+02:00 Gianmarco De Francisci Morales g...@apache.org : Hi, Great job putting together this roadmap! Regarding 1) you may find this work relevant: http://arxiv.org/abs/1503.00626v1 Regarding 2) the semi-streaming model allows polylog(n) passes. If you allow only one pass, it's a streaming model. In addition, I think models such as the turnstile/fully dynamic (with addition and deletion) would be more appropriate than the incremental one. Regarding 5) (when combined with 2) there are some good recent algorithms for partitioning that you might want to look at. http://dl.acm.org/citation.cfm?id=2556213 http://dl.acm.org/citation.cfm?id=2339722 Cheers, -- Gianmarco On 18 May 2015 at 12:42, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Hello everyone, thank you Andra for putting together this nice document and for sharing! Gelly is already in a state that matches the basic functionality of existing distributed graph libraries. We're very happy for this and also very grateful to everyone who has contributed so far! We are also very excited to keep on extending Gelly and our vision is to turn it into something much more than yet another library for running PageRank-like algorithms. The document that Andra has shared contains ongoing work and our ideas on mid- and long-term plans, in order to make this happen :) We would love to get the community's feedback on this, so please feel free to comment on the doc or discuss individual points here. Cheers, Vasia. On 18 May 2015 at 11:01, Andra Lungu lungu.an...@gmail.com wrote: Dear Squirrels, The following document is intended to give you an insight on where Flink's Graph API, Gelly, is at the moment and especially on the direction in which
Re: Gelly Roadmap
Apache TinkerPop made it to the wish-list :) Thanks, Fabian! On Tue, May 19, 2015 at 9:42 AM, Flavio Pompermaier pomperma...@okkam.it wrote: You're definitely right Fabian +1 On Mon, May 18, 2015 at 2:03 PM, Fabian Hueske fhue...@gmail.com wrote: Integration with Apache TinkerPop3 could also be interesting. TinkerPop3 is an API for transactional and analytical graph processing and supported by several Graph engines/databases. It might be interesting to see if/how Gelly's and TinkerPop's concepts match and think about whether it makes sense to integrate the two projects in some way. 2015-05-18 12:20 GMT+02:00 Gianmarco De Francisci Morales g...@apache.org : Hi, Great job putting together this roadmap! Regarding 1) you may find this work relevant: http://arxiv.org/abs/1503.00626v1 Regarding 2) the semi-streaming model allows polylog(n) passes. If you allow only one pass, it's a streaming model. In addition, I think models such as the turnstile/fully dynamic (with addition and deletion) would be more appropriate than the incremental one. Regarding 5) (when combined with 2) there are some good recent algorithms for partitioning that you might want to look at. http://dl.acm.org/citation.cfm?id=2556213 http://dl.acm.org/citation.cfm?id=2339722 Cheers, -- Gianmarco On 18 May 2015 at 12:42, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Hello everyone, thank you Andra for putting together this nice document and for sharing! Gelly is already in a state that matches the basic functionality of existing distributed graph libraries. We're very happy for this and also very grateful to everyone who has contributed so far! We are also very excited to keep on extending Gelly and our vision is to turn it into something much more than yet another library for running PageRank-like algorithms. The document that Andra has shared contains ongoing work and our ideas on mid- and long-term plans, in order to make this happen :) We would love to get the community's feedback on this, so please feel free to comment on the doc or discuss individual points here. Cheers, Vasia. On 18 May 2015 at 11:01, Andra Lungu lungu.an...@gmail.com wrote: Dear Squirrels, The following document is intended to give you an insight on where Flink's Graph API, Gelly, is at the moment and especially on the direction in which it's heading in the near future. https://docs.google.com/document/d/11ovK1xZZTl3Qinw6Ud1rqpaDhTvRHDv8PWBzcHITFkM/edit?usp=sharing We would love to hear about your comments/ suggestions/ additions to the wish-list :) Cheers, Andra
Re: Gelly Roadmap
Integration with Apache TinkerPop3 could also be interesting. TinkerPop3 is an API for transactional and analytical graph processing and supported by several Graph engines/databases. It might be interesting to see if/how Gelly's and TinkerPop's concepts match and think about whether it makes sense to integrate the two projects in some way. 2015-05-18 12:20 GMT+02:00 Gianmarco De Francisci Morales g...@apache.org: Hi, Great job putting together this roadmap! Regarding 1) you may find this work relevant: http://arxiv.org/abs/1503.00626v1 Regarding 2) the semi-streaming model allows polylog(n) passes. If you allow only one pass, it's a streaming model. In addition, I think models such as the turnstile/fully dynamic (with addition and deletion) would be more appropriate than the incremental one. Regarding 5) (when combined with 2) there are some good recent algorithms for partitioning that you might want to look at. http://dl.acm.org/citation.cfm?id=2556213 http://dl.acm.org/citation.cfm?id=2339722 Cheers, -- Gianmarco On 18 May 2015 at 12:42, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Hello everyone, thank you Andra for putting together this nice document and for sharing! Gelly is already in a state that matches the basic functionality of existing distributed graph libraries. We're very happy for this and also very grateful to everyone who has contributed so far! We are also very excited to keep on extending Gelly and our vision is to turn it into something much more than yet another library for running PageRank-like algorithms. The document that Andra has shared contains ongoing work and our ideas on mid- and long-term plans, in order to make this happen :) We would love to get the community's feedback on this, so please feel free to comment on the doc or discuss individual points here. Cheers, Vasia. On 18 May 2015 at 11:01, Andra Lungu lungu.an...@gmail.com wrote: Dear Squirrels, The following document is intended to give you an insight on where Flink's Graph API, Gelly, is at the moment and especially on the direction in which it's heading in the near future. https://docs.google.com/document/d/11ovK1xZZTl3Qinw6Ud1rqpaDhTvRHDv8PWBzcHITFkM/edit?usp=sharing We would love to hear about your comments/ suggestions/ additions to the wish-list :) Cheers, Andra
Re: Gelly Roadmap
Hello everyone, thank you Andra for putting together this nice document and for sharing! Gelly is already in a state that matches the basic functionality of existing distributed graph libraries. We're very happy for this and also very grateful to everyone who has contributed so far! We are also very excited to keep on extending Gelly and our vision is to turn it into something much more than yet another library for running PageRank-like algorithms. The document that Andra has shared contains ongoing work and our ideas on mid- and long-term plans, in order to make this happen :) We would love to get the community's feedback on this, so please feel free to comment on the doc or discuss individual points here. Cheers, Vasia. On 18 May 2015 at 11:01, Andra Lungu lungu.an...@gmail.com wrote: Dear Squirrels, The following document is intended to give you an insight on where Flink's Graph API, Gelly, is at the moment and especially on the direction in which it's heading in the near future. https://docs.google.com/document/d/11ovK1xZZTl3Qinw6Ud1rqpaDhTvRHDv8PWBzcHITFkM/edit?usp=sharing We would love to hear about your comments/ suggestions/ additions to the wish-list :) Cheers, Andra
Re: Gelly Roadmap
Hi, Great job putting together this roadmap! Regarding 1) you may find this work relevant: http://arxiv.org/abs/1503.00626v1 Regarding 2) the semi-streaming model allows polylog(n) passes. If you allow only one pass, it's a streaming model. In addition, I think models such as the turnstile/fully dynamic (with addition and deletion) would be more appropriate than the incremental one. Regarding 5) (when combined with 2) there are some good recent algorithms for partitioning that you might want to look at. http://dl.acm.org/citation.cfm?id=2556213 http://dl.acm.org/citation.cfm?id=2339722 Cheers, -- Gianmarco On 18 May 2015 at 12:42, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Hello everyone, thank you Andra for putting together this nice document and for sharing! Gelly is already in a state that matches the basic functionality of existing distributed graph libraries. We're very happy for this and also very grateful to everyone who has contributed so far! We are also very excited to keep on extending Gelly and our vision is to turn it into something much more than yet another library for running PageRank-like algorithms. The document that Andra has shared contains ongoing work and our ideas on mid- and long-term plans, in order to make this happen :) We would love to get the community's feedback on this, so please feel free to comment on the doc or discuss individual points here. Cheers, Vasia. On 18 May 2015 at 11:01, Andra Lungu lungu.an...@gmail.com wrote: Dear Squirrels, The following document is intended to give you an insight on where Flink's Graph API, Gelly, is at the moment and especially on the direction in which it's heading in the near future. https://docs.google.com/document/d/11ovK1xZZTl3Qinw6Ud1rqpaDhTvRHDv8PWBzcHITFkM/edit?usp=sharing We would love to hear about your comments/ suggestions/ additions to the wish-list :) Cheers, Andra
