[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/335 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user vasia commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-73749137 Thanks @balidani. And travis tests now pass :tada: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user vasia commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-73547684 Simple renaming didn't seem to keep the history, so I did the filtering again :) Didn't you have the same problem when moving flink-addons to flink-staging? Let me know if it's fine now. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user vasia commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-73434636 @rmetzger does this mean I need to do the history filtering magic again and open a new pr? @andralungu thanks a lot! @balidani have you submitted yours? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user vasia commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-73434977 Great, thanks ^^ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-73366441 I'm really sorry that I've messed up this pull request by renaming flink-addons to flink-staging :( I was doing it in a rush Really sorry. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user vasia commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-72883525 @balidani that's OK :) @cebe we've got the acknowledgement for you, thanks! @andralungu has submitted hers today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user balidani commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-72851434 Sorry, I will only be able to fill the form on Sunday, my internet connection is horrible here. Is that ok? Daniel On Feb 2, 2015 3:53 PM, Carsten Brandt notificati...@github.com wrote: @cebe https://github.com/cebe: could you please complete and sign this form (if you haven't already)? Thank you! done. â Reply to this email directly or view it on GitHub https://github.com/apache/flink/pull/335#issuecomment-72469660. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-72359922 One more thing ;-) Did we collect ICLAs from all people contributing significant parts to Gelly? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user cebe commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-72366219 @fhueske why is that needed? [Gelly is Apache 2.0](https://github.com/project-flink/flink-graph/blob/master/LICENSE) licensed and Flink too: https://github.com/apache/flink/blob/master/LICENSE --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user fhueske commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-71492121 Sure, XyzTest are unit tests which are executed in Maven's test phase. These should execute rather fast. Everything that brings up a full Flink system is an integration test case (XyzITCase) and executed during mvn verify. These are used for long running tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user vasia commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-71507264 Tests renamed :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons (...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/335#issuecomment-71376640 Thank you. Looks good. History is preserved and you addressed my comments. +1 for merging it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user cebe commented on the pull request: https://github.com/apache/flink/pull/326#issuecomment-71268972 that really sounds like great fun :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/326#issuecomment-71204417 Yes. The package seems to be still wrong. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/326#issuecomment-71157010 I took the pull request and filtered the branch to move the files into the `flink-addons/flink-gelly` directory (that worked better for me than the subtree merge). Also, I prefixed all commit messages with the jira issue and component. You can find it here, it perfectly preserves all commit history. https://github.com/StephanEwen/incubator-flink/commits/gelly Let me know if I should merge it like that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user vasia commented on the pull request: https://github.com/apache/flink/pull/326#issuecomment-70839009 Thank you all for the suggestions regarding preserving the commit history! I'll try them out later today and see whether I can make it :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
GitHub user vasia opened a pull request: https://github.com/apache/flink/pull/326 [FLINK-1201] Add flink-gelly to flink-addons This PR adds an initial version of Gelly, a graph API for Flink, to the flink-addons project. The development of Gelly took place as a collaboration on a separate project, which you can visit here: https://github.com/project-flink/flink-graph. I have kept the commit history and I hope this won't make it too hard to review :) We are currently working on adding documentation, a few more examples and tests and we'll also port remaining issues to JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/vasia/flink flink-gelly Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/326.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #326 commit 1f378eda803783c053adc7b707bd8f938b0616a4 Author: Stephan Ewen stephan.e...@tu-berlin.de Date: 2014-08-12T20:48:59Z Initial commit commit 22b7464cf73497f09a55b26be4d654cc339cf1ec Author: Stephan Ewen se...@apache.org Date: 2014-08-12T20:54:54Z Initial Maven project structure and .gitignore commit 6103cf9dea81c957a5727e351fecc4e32df017ba Author: Stephan Ewen se...@apache.org Date: 2014-08-27T15:56:32Z Add example for parallel dense ID assignment commit f44986d22b5913cba8e11b6e43a77effd925aaa1 Author: Stephan Ewen se...@apache.org Date: 2014-08-30T17:19:21Z First mockup of initial API functions commit 0276986098f18ab18f71b6f81e273fc8c2f86886 Author: Stephan Ewen se...@apache.org Date: 2014-10-08T10:18:04Z Reset repository commit c1336f6950d966220418fcbab241e7e702dcf498 Author: Stephan Ewen se...@apache.org Date: 2014-10-08T10:24:05Z Initial package commit 8083be2d8b34921fd5e80ef6030aee4668ffbb4b Author: Stephan Ewen se...@apache.org Date: 2014-10-08T11:50:11Z Stubs for basic classes commit a708ced256cf6b4176a3d93ef6cb829b2a4eb1d8 Author: Stephan Ewen se...@apache.org Date: 2014-10-08T12:29:46Z fix project setup commit 1ee9b73428602fa37dd8ae86be53aeb79e1af68c Author: vasia vasilikikala...@gmail.com Date: 2014-10-08T13:27:40Z get undirected graph commit 08301093d31b66f553a7c3d3d34c39ea9f1dd683 Author: vasia vasilikikala...@gmail.com Date: 2014-10-09T11:22:24Z edge abd value extend tuple types commit 006a65397b152b08d555899ccedeaa6274a8c48e Author: vasia vasilikikala...@gmail.com Date: 2014-10-09T12:49:34Z reverse and graph input methods commit aa0eb20f5708bdc3e2439651ca7b10436bbde640 Author: Theodore Vasiloudis t...@sics.se Date: 2014-10-09T14:11:03Z Added Push-Gather-Apply, mapVertices, subgraph, outDegrees Added: mapVertices + Test subgraph (Unfinished) outDegrees Push-Gather-Apply Neighborhood Model Junit Dependency Everything is untested as we are blocked on graph creation commit 044d8097b3b4e4b2d60142bb9f6ad04b15d3fb94 Author: Theodore Vasiloudis t...@sics.se Date: 2014-10-09T14:45:03Z Merge branch 'hackathon' of github.com:vasia/flink-graphs into thvasilo Conflicts: src/main/java/flink/graphs/Edge.java src/main/java/flink/graphs/Graph.java src/main/java/flink/graphs/Vertex.java commit 43e8b56f81006a85699c842a44b00b1b395474dd Author: Kostas Tzoumas kostas.tzou...@gmail.com Date: 2014-10-10T11:02:17Z Initial commit commit 0b39cc70130a92d67b908a379fae3347d648dc99 Author: Theodore Vasiloudis t...@sics.se Date: 2014-10-10T14:09:36Z Reverted to using Tuples, added tests. commit 3f2610453c64188e04b5e27b31c2145cfe37cee6 Author: Vasia Kalavri vasilikikala...@gmail.com Date: 2014-10-10T16:14:38Z Merge pull request #1 from thvasilo/thvasilo Merge Theo's Martin's hackathon work. commit 35b865e8b2c87816e813a5966d367910aadd64a8 Author: vasia vasilikikala...@gmail.com Date: 2014-10-10T16:19:37Z keep 1 of the 2 readme files commit fee97bb4e6daa287b7fd8a678395799fbc7905a0 Author: Kostas Tzoumas kostas.tzou...@gmail.com Date: 2014-10-21T06:11:02Z Added Kostas' functions from the hackathon; added ExecutionEnvironment member to Graph commit a405e0973d3157933e78a45fb447fcabb9528e7d Author: vasia vasilikikala...@gmail.com Date: 2014-10-21T15:37:50Z reflect current status in README commit 2c8d859cafcedddb463d895b584a4d29d1be5d26 Author: vasia vasilikikala...@gmail.com Date: 2014-10-25T14:14:42Z type information in Graph and in getUndirected method commit 7ff3f8a463a73495bdc3d2e8f11a204c787b2c9c Author: vasia vasilikikala...@gmail.com Date: 2014-10-25T15:08:53Z typeinfo in reverse and getOutdegrees commit 5af110c10c2301bcca22e19afeb10fff542153e5 Author: vasia vasilikikala...@gmail.com Date: 2014-10-25T16:22:11Z add dependency to
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/326#discussion_r23252081 --- Diff: flink-addons/flink-gelly/src/main/java/org/apache/flink/gelly/Edge.java --- @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * License); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an AS IS BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.gelly; + +import java.io.Serializable; + +import org.apache.flink.api.java.tuple.Tuple3; + +public class EdgeK extends ComparableK Serializable, V extends Serializable --- End diff -- This Edge class, and all the 3 other Edge* classes below are missing javadocs. The rest of the graph api is very well documented, but these classes not. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user vasia commented on the pull request: https://github.com/apache/flink/pull/326#issuecomment-70725539 Hi @rmetzger! Thanks for starting on it so fast ^^ Regarding the name, we thought it'd be nice to have one.. It was actually @ktzoumas that came up with Gelly :) Regarding the history, my intention was to keep it actually, but it seems I failed :disappointed: What should I do to preserve it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/326#issuecomment-70744282 I also prefer flink-graph. After having a quick look at the commits I would suggest to squash a couple of them, e. g. fda6e4c, which is empty or 393902c which deletes 4 lines of unused imports. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user uce commented on the pull request: https://github.com/apache/flink/pull/326#issuecomment-70759342 I'm very exited about this :-) I will have a look later as well. Regarding the name: I like the name gelly, but would also prefer flink-graph (or something along the lines). My past experience with Spargel (which is also a nice name :-)) was that the name was confusing to people new to the system, whereas something like flink-graph directly states what it is. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user ktzoumas commented on the pull request: https://github.com/apache/flink/pull/326#issuecomment-70767467 I think Gelly has way more personality than flink-graph --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/326#issuecomment-70743431 Woot! :) As for merging the streaming back then, it was @StephanEwen who came up with the solution, pasting the important parts here: BTW: I used this way to do it: Have two repositories (clones) - /data/repositories/flink - /data/repositories/flinkbak The do the following for every non-merge commit: - Check out the state after a commit in the backup (detached head) - Remove current streaming directory (physically and from the index) - Add it again (files and index), with the state of the cloned repo - Commit (git recreates the diffs in a way that they reflect the original commit plus any merges) - #!/bin/bash for line in $(cat commits) do cd /data/repositories/flinkbak author=`git --no-pager show -s --format='%an %ae' $line` message=`git --no-pager show -s --format='%s%n' $line` echo picking commit $line from author $author git checkout $line cd /data/repositories/flink rm -rf /data/repositories/flink/flink-addons/flink-streaming git rm -r /data/repositories/flink/flink-addons/flink-streaming cp -r /data/repositories/flinkbak/flink-addons/flink-streaming /data/repositories/flink/flink-addons/flink-streaming git add /data/repositories/flink/flink-addons/flink-streaming git commit --author $author --m $message # read -rsp $'Press any key to continue...\n' -n1 key done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/326#discussion_r23248831
--- Diff:
flink-addons/flink-gelly/src/test/java/org/apache/flink/gelly/test/TestWeaklyConnected.java
---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.gelly.test;
+
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Collection;
+import java.util.LinkedList;
+
+import org.apache.flink.api.java.ExecutionEnvironment;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.gelly.Graph;
+import org.apache.flink.test.util.JavaProgramTestBase;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.junit.runners.Parameterized.Parameters;
+
+@RunWith(Parameterized.class)
+public class TestWeaklyConnected extends JavaProgramTestBase {
--- End diff --
I think its recommended now to use the `MultipleProgramsTestBase` instead
of the `JavaProgramTestBase` because the MultiProgramsTB is
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1201] Add flink-gelly to flink-addons
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/326#issuecomment-70717064 Great, I'm super excited to see the graph API being offered to the main project. I'll start reviewing the code right away, to merge it as soon as possible. One question upfront: How did you come up with the name gelly? Why don't we call the baby by what it is? a graph api ? Should we consider moving the classes while preserving their history? Thats what we did with the streaming system when we merged it. Right now, basically all the code from the graph api has one commit in its history (6c31f8e) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
