Re: Gelly Blog Post

Hi, this is a very nice blog post! I added some minor comments. I am really excited about the future work on partition-centric computation and graph partitioning! And thanks for guiding me to DataSetUtils.zipWithUniqueIds()! I should switch to 0.10-SNAPSHOT :) Best, Martin On

Fast strategy for intersect

Hi all! While working on FLINK-2905, I was wondering what a good (and fast) way to compute the intersect between two data sets (Gelly vertices in my case) with unknown size would be. I came up with three ways to solve this: Consider two sets: DataSet> verticesLeft =

Re: [gelly] Spargel model rework

Hi, At our group, we also moved several algorithms from Giraph to Gelly and ran into some confusing issues (first in understanding, second during implementation) caused by the conceptional differences you described. If there are no concrete advantages (performance mainly) in the Spargel

Re: neo4j - Flink connector

o use neo4j. I think I can help you much better as soon as I have your current pom file + code. [1] http://www.apache.org/legal/resolved.html#category-a [2] https://github.com/s1ck/flink-neo4j On Wed, Nov 11, 2015 at 7:38 PM, Martin Junghanns <m.jungha...@mailbox.org> wrote: Hi, I am a bi

Re: neo4j - Flink connector

Hi, I am a bit stuck with that dependency problem. Any help would be appreciated as I would like to continue working on the formats. Thanks! Best, Martin On 07.11.2015 17:28, Martin Junghanns wrote: Hi Robert, Thank you for the hints. I tried to narrow down the error: Flink version: 0.10

Error when building the docs

Hi, not sure if that's an issue or just a misconfiguration (not familiar with Ruby). I followed the docs/README.md and ran into: s1ck@s1ck-T450s:~/Devel/Java/flink$ docs/build_docs.sh Configuration file: /home/s1ck/Devel/Java/flink/docs/_config.yml

Re: neo4j - Flink connector

may be valuable for an output format. On Sat, Oct 31, 2015 at 9:51 AM, Martin Junghanns < m.jungha...@mailbox.org> wrote: Hi, I wanted to give you a little update. I created a non-parallel InputFormat which reads Cypher results from Neo4j into Tuples [1]. It can be used like the JDBCInputForm

Re: Error when building the docs

remove Jekyll 3.x and install Jekyll 2.x, you can build docs. I’m using Jekyll 2.5.3 to build the docs. Regards, Chiwan Park On November 6, 2015 at 4:58:34 AM, Martin Junghanns (m.jungha...@mailbox.org) wrote: Hi, not sure if that's an issue or just a misconfiguration (not familiar with Ruby

Re: neo4j - Flink connector

s, this may be > valuable for an output format. > > On Sat, Oct 31, 2015 at 9:51 AM, Martin Junghanns <m.jungha...@mailbox.org> > wrote: > >> Hi, >> >> I wanted to give you a little update. I created a non-parallel >> InputFormat which reads Cypher results from

Re: neo4j - Flink connector

Hi, I wanted to give you a little update. I created a non-parallel InputFormat which reads Cypher results from Neo4j into Tuples [1]. It can be used like the JDBCInputFormat: String q = "MATCH (p1:Page)-[:Link]->(p2) RETURN id(p1), id(p2)"; Neo4jInputFormat> neoInput =

Re: Parsing JSON

An example can be found here: Code to read JSON from line: https://github.com/dbs-leipzig/gradoop/blob/master/gradoop-flink/src/main/java/org/gradoop/io/json/JsonReader.java#L57 How it is called:

Re: [gelly] partition-centric iterations

Hi, this would be a very nice addition! I had a glimpse look into the PC implementation and the two library algorithms and when you get the idea, it is easy to follow what's happening. The benchmark results are also very promising. I got some questions about partitions: 1) I was wondering if

Re: Serialization problem in CollectionEnvironment

, Martin Junghanns wrote: Hi, While building IT tests which extend MultipleProgramsTestBase, I encountered a problem with serialization: I posted a minimal example here: https://gist.github.com/s1ck/566796df5f35ee1de6f9 This runs fine with LocalEnvironment. However, when executing

Re: Serialization problem in CollectionEnvironment

objects via serialization as a fallback strategy. That’s a bug in >> these components. I’ll file an issue and open a PR for it. >> >> Thanks a lot for finding this problem Martin! >> >> Cheers, >> Till >> ​ >> >> On Fri, Nov 27, 2015 at 11:59 AM, Mart

Re: Null Pointer Exception in tests but only in COLLECTION mode

Hi Max, fixed in https://github.com/apache/flink/pull/1396 Best, Martin On 24.11.2015 13:46, Maximilian Michels wrote: Hi André, hi Martin, This looks very much like a bug. Martin, I would be happy if you opened a JIRA issue. Thanks, Max On Sun, Nov 22, 2015 at 12:27 PM, Martin Junghanns

Serialization problem in CollectionEnvironment

Hi, While building IT tests which extend MultipleProgramsTestBase, I encountered a problem with serialization: I posted a minimal example here: https://gist.github.com/s1ck/566796df5f35ee1de6f9 This runs fine with LocalEnvironment. However, when executing this in CollectionEnvironment, it

Re: SETUP ISSUES

Hi John, Flink has a nice documentation for all APIs [1]. To get you started with the correct Maven dependencies for the DataSet API, please have a look at "Linking With Flink" [2]. On that site, you will also find a lot of Java and Scala examples. Please let us know, if you need help. Best,

Re: Null Pointer Exception in tests but only in COLLECTION mode

Hi, What he meant was MultipleProgramsTestBase, not FlinkTestBase. I debugged this a bit. The NPE is thrown in https://github.com/apache/flink/blob/master/flink-java/src/main/java/org/apache/flink/api/java/operators/AggregateOperator.java#L296 since current can be null if the input iterator

Type problem in RichFlatMapFunction when using GenericArray type

Hi, I ran into a problem when using generic arrays in a tuple. I wrote a minimal program to reproduce the error [1]. The problem seems to be related to the order of tuple fields. When I switch Tuple2 to Tuple2 and perform the join on field 0, everything works as expected.

Re: Type problem in RichFlatMapFunction when using GenericArray type

:351) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584) at java.lang.Thread.run(Thread.java:745) On 10.10.2016 13:39, Chesnay Schepler wrote: Hello Martin, Could you include the error you are getting? Regards, Chesnay On 10.10.2016 13:31, Martin Junghanns wrote: Hi

Re: Gelly Library. Need an example

Hi all, thanks for reporting the issue! I just looked into it. The VertexGroupReduce outputs two semantically different tuples: one for each vertex without the value (null) and one tuple representing the whole group including the value. As Till pointed out, this crashes if the value has no

Type erasure problem solely on cluster execution

Hi, I am running into a type erasure problem which only occurs when I execute the code using a Flink cluster (1.1.2). I created a Gist [1] which reproduces the problem. I also added a unit test to show that it does not fail in local and collection mode. Maybe it is also interesting to

Re: Type erasure problem solely on cluster execution

oblem and providing code to reproduce it. Would you mind to describe the problem with the forwarding annotations in more detail? I would be interested in the error message and how the semantic annotation is provided (@ForwardFields or withForwardedFields()). Thanks, Fabian 2016-10-19 8:52 GMT

Re: Type problem in RichFlatMapFunction when using GenericArray type

: Hello Martin, Could you include the error you are getting? Regards, Chesnay On 10.10.2016 13:31, Martin Junghanns wrote: Hi, I ran into a problem when using generic arrays in a tuple. I wrote a minimal program to reproduce the error [1]. The problem seems to be related to the order of tuple

[jira] [Created] (FLINK-2411) Add basic graph summarization algorithm

Martin Junghanns created FLINK-2411: --- Summary: Add basic graph summarization algorithm Key: FLINK-2411 URL: https://issues.apache.org/jira/browse/FLINK-2411 Project: Flink Issue Type: New

[jira] [Created] (FLINK-2910) Combine tests for binary graph operators

Martin Junghanns created FLINK-2910: --- Summary: Combine tests for binary graph operators Key: FLINK-2910 URL: https://issues.apache.org/jira/browse/FLINK-2910 Project: Flink Issue Type

[jira] [Created] (FLINK-2981) Update README for building docs

Martin Junghanns created FLINK-2981: --- Summary: Update README for building docs Key: FLINK-2981 URL: https://issues.apache.org/jira/browse/FLINK-2981 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-2590) DataSetUtils.zipWithUniqueID creates duplicate IDs

Martin Junghanns created FLINK-2590: --- Summary: DataSetUtils.zipWithUniqueID creates duplicate IDs Key: FLINK-2590 URL: https://issues.apache.org/jira/browse/FLINK-2590 Project: Flink Issue

[jira] [Created] (FLINK-3118) Check if MessageFunction implements ResultTypeQueryable

Martin Junghanns created FLINK-3118: --- Summary: Check if MessageFunction implements ResultTypeQueryable Key: FLINK-3118 URL: https://issues.apache.org/jira/browse/FLINK-3118 Project: Flink

[jira] [Created] (FLINK-3122) Generalize value type in LabelPropagation

Martin Junghanns created FLINK-3122: --- Summary: Generalize value type in LabelPropagation Key: FLINK-3122 URL: https://issues.apache.org/jira/browse/FLINK-3122 Project: Flink Issue Type

[jira] [Created] (FLINK-3064) Missing size check in GroupReduceOperatorBase leads to NPE

Martin Junghanns created FLINK-3064: --- Summary: Missing size check in GroupReduceOperatorBase leads to NPE Key: FLINK-3064 URL: https://issues.apache.org/jira/browse/FLINK-3064 Project: Flink

[jira] [Created] (FLINK-3272) Generalize vertex value type in ConnectedComponents

Martin Junghanns created FLINK-3272: --- Summary: Generalize vertex value type in ConnectedComponents Key: FLINK-3272 URL: https://issues.apache.org/jira/browse/FLINK-3272 Project: Flink

[jira] [Created] (FLINK-3618) Rename abstract UDF classes in Scatter-Gather implementation

Martin Junghanns created FLINK-3618: --- Summary: Rename abstract UDF classes in Scatter-Gather implementation Key: FLINK-3618 URL: https://issues.apache.org/jira/browse/FLINK-3618 Project: Flink

[jira] [Created] (FLINK-3611) Wrong link in CONTRIBUTING.md

Martin Junghanns created FLINK-3611: --- Summary: Wrong link in CONTRIBUTING.md Key: FLINK-3611 URL: https://issues.apache.org/jira/browse/FLINK-3611 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4872) Type erasure problem exclusively on cluster execution

Martin Junghanns created FLINK-4872: --- Summary: Type erasure problem exclusively on cluster execution Key: FLINK-4872 URL: https://issues.apache.org/jira/browse/FLINK-4872 Project: Flink