Re: [gelly] Spargel model rework

2015-11-03 Thread Andra Lungu
I also think a Giraph-like model could be added, but we shouldn't remove Spargel in favour of it! On Tue, Nov 3, 2015 at 2:35 AM, Stephan Ewen wrote: > When creating the original version of Spargel I was pretty much thinking in > GSA terms, more than in Pregel terms. There are

Re: Caching information from a stream

2015-10-29 Thread Andra Lungu
p(new StatefulMapper()) > > > >.print(); > > > >env.execute(); > > > > } > > > > The output is the following on my machine (discarded the output of the > > print): > > > > Key: 2 Previous state was: -1 Update state to:

[jira] [Created] (FLINK-2926) Add a Strongly Connected Components Library Method

2015-10-27 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2926: -- Summary: Add a Strongly Connected Components Library Method Key: FLINK-2926 URL: https://issues.apache.org/jira/browse/FLINK-2926 Project: Flink Issue Type

Re: Design document for FLINK-2254

2015-10-22 Thread Andra Lungu
Hi Saumitra, As you already noticed, the first version (with duplicates) is highly inefficient and consumes a lot of memory. So, I suggest we drop it for now. The version with the label makes a lot of modifications on the base Graph class, and this, in my opinion would make it more difficult to

[jira] [Created] (FLINK-2899) The groupReduceOn* methods which take types as a parameter fail with TypeErasure

2015-10-22 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2899: -- Summary: The groupReduceOn* methods which take types as a parameter fail with TypeErasure Key: FLINK-2899 URL: https://issues.apache.org/jira/browse/FLINK-2899 Project

Using Flink Streaming to write to multiple output files in HDFS

2015-10-21 Thread Andra Lungu
Hey guys, Long time, no see :). I recently started a new job and it involves performing a set of real-time data analytics using Apache Kafka, Storm and Flume. What happens, on a very high level, is that set of signals is collected, stored into a Kafka topic and then Storm is used to filter

[jira] [Created] (FLINK-2715) Benchmark Triangle Count methods

2015-09-20 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2715: -- Summary: Benchmark Triangle Count methods Key: FLINK-2715 URL: https://issues.apache.org/jira/browse/FLINK-2715 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-2714) Port the Flink DataSet Triangle Count example to the Gelly library

2015-09-20 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2714: -- Summary: Port the Flink DataSet Triangle Count example to the Gelly library Key: FLINK-2714 URL: https://issues.apache.org/jira/browse/FLINK-2714 Project: Flink

[jira] [Created] (FLINK-2661) Add a Node Splitting Technique to Overcome the Limitations of Skewed Graphs

2015-09-12 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2661: -- Summary: Add a Node Splitting Technique to Overcome the Limitations of Skewed Graphs Key: FLINK-2661 URL: https://issues.apache.org/jira/browse/FLINK-2661 Project: Flink

[jira] [Created] (FLINK-2634) Add a Vertex-centric Version of the Tringle Count Library Method

2015-09-08 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2634: -- Summary: Add a Vertex-centric Version of the Tringle Count Library Method Key: FLINK-2634 URL: https://issues.apache.org/jira/browse/FLINK-2634 Project: Flink

[jira] [Created] (FLINK-2570) Add a Triangle Count Library Method

2015-08-25 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2570: -- Summary: Add a Triangle Count Library Method Key: FLINK-2570 URL: https://issues.apache.org/jira/browse/FLINK-2570 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-2563) Gelly's Graph Algorithm Interface is limites

2015-08-23 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2563: -- Summary: Gelly's Graph Algorithm Interface is limites Key: FLINK-2563 URL: https://issues.apache.org/jira/browse/FLINK-2563 Project: Flink Issue Type

Re: Gelly Blog Post

2015-08-22 Thread Andra Lungu
: Wow, this is impressive :-) Amazing work, Gelly folks! On Tue, May 26, 2015 at 10:03 AM, Andra Lungu lungu.an...@gmail.com wrote: Hey everyone, We are very excited to share the first stable draft of the Gelly blog post with you :D https://docs.google.com

Re: Join hints for the Gelly functions

2015-08-22 Thread Andra Lungu
. Best, Martin On 22.08.2015 09:51, Andra Lungu wrote: Hey everyone, When coding for my thesis, I observed that half of the current Gelly functions (the ones that use join operators) fail on a cluster environment with the following exception: java.lang.IllegalArgumentException: Too few

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Andra Lungu
Congrats Chesnay! :) On Thu, Aug 20, 2015 at 8:51 PM, Márton Balassi balassi.mar...@gmail.com wrote: Welcome Chesnay! On Thu, Aug 20, 2015 at 7:29 PM, Henry Saputra henry.sapu...@gmail.com wrote: Welcome Chesnay! On Thu, Aug 20, 2015 at 2:18 AM, Robert Metzger rmetz...@apache.org

[jira] [Created] (FLINK-2546) The webclient does not allow users to save the execution plan

2015-08-19 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2546: -- Summary: The webclient does not allow users to save the execution plan Key: FLINK-2546 URL: https://issues.apache.org/jira/browse/FLINK-2546 Project: Flink

Re: [Proposal] Addition to Gelly

2015-08-12 Thread Andra Lungu
I would love to get some feedback from the guys at data Artisans about this one. So far, the comments originated and spread in the Stockholm area :) On Tue, Aug 11, 2015 at 6:33 PM, Andra Lungu lungu.an...@gmail.com wrote: Hi Samia, A good method to statistically determine skewed vertices

Re: [Proposal] Addition to Gelly

2015-08-11 Thread Andra Lungu
. On Aug 10, 2015 10:47 AM, Andra Lungu lungu.an...@gmail.com wrote: Hey, Before actually opening a PR, I wanted to hear your opinion. So, here goes nothing :). I'd like to add the core of my master thesis to Gelly. That is, a series of operators that take a skewed graph, split its

Re: [Proposal] Addition to Gelly

2015-08-11 Thread Andra Lungu
they will get benefits by using it since node splitting doesn’t come with no cost I guess. I am really eager to see how this will evolve, I think it’s good effort. cheers Paris On 11 Aug 2015, at 14:58, Andra Lungu lungu.an...@gmail.com wrote: Hi Vasia, I shall polish the functions a bit

Re: [Proposal] Addition to Gelly

2015-08-11 Thread Andra Lungu
to separate the high degree vertices from the low degree vertices. Regards, Samia On Tue, Aug 11, 2015 at 3:41 PM, Andra Lungu lungu.an...@gmail.com wrote: Hi Paris, Nice to virtually meet you too :) Maybe it makes sense to share my freshest chart: https://drive.google.com/file

[Proposal] Addition to Gelly

2015-08-10 Thread Andra Lungu
Hey, Before actually opening a PR, I wanted to hear your opinion. So, here goes nothing :). I'd like to add the core of my master thesis to Gelly. That is, a series of operators that take a skewed graph, split its high degree vertices into subvertices and redistribute the edges accordingly (thus

Re: A soft reminder

2015-07-30 Thread Andra Lungu
, I also encountered the EOF exception for a delta iteration with more data. With less data it works ... Best regards, Felix Am 27.07.2015 10:25 vorm. schrieb Andra Lungu lungu.an...@gmail.com: Hi Stephan, I tried to debug a bit around the EOF Exception. It seems that I am pretty

Re: [Gelly] Re-organize / cleanup examples and library

2015-07-29 Thread Andra Lungu
Hi Vasia, We should document the examples in the Gelly guide, I totally agree, perhaps something similar to what Giraph offers. However, if you recall, the examples are also there for test purposes. The library methods contain just the run method; no main method. For the tests, the nicest way to

Re: A soft reminder

2015-07-27 Thread Andra Lungu
, Stephan On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu lungu.an...@gmail.com wrote: Hi guys, I opened these JIRAs a while ago and, unfortunately, no one had time to look at them :|. [1] https://issues.apache.org/jira/browse/FLINK-2361 [2] https://issues.apache.org/jira/browse

A soft reminder

2015-07-24 Thread Andra Lungu
Hi guys, I opened these JIRAs a while ago and, unfortunately, no one had time to look at them :|. [1] https://issues.apache.org/jira/browse/FLINK-2361 [2] https://issues.apache.org/jira/browse/FLINK-2360 [1] could probably be hacked, but [2] is a huge problem that was encountered by other

Theoretical complexity of a coGroup

2015-07-22 Thread Andra Lungu
Hi everyone, I am not 100% sure about this one, so I thought that I could set my thoughts straight via the mailing list. Here's the use case. You coGroup a data set of vertices with a data set of edges. That gives you a complexity of* O(|V| * |E|)*, where |V| is the total number of vertices and

[jira] [Created] (FLINK-2360) EOFException

2015-07-14 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2360: -- Summary: EOFException Key: FLINK-2360 URL: https://issues.apache.org/jira/browse/FLINK-2360 Project: Flink Issue Type: Bug Components: Local Runtime

[jira] [Created] (FLINK-2361) flatMap + distict gives eroneous results for big data sets

2015-07-14 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2361: -- Summary: flatMap + distict gives eroneous results for big data sets Key: FLINK-2361 URL: https://issues.apache.org/jira/browse/FLINK-2361 Project: Flink Issue

Re: The documentation site is cut on the top

2015-07-13 Thread Andra Lungu
The bug is also reproducible in Chrome... The page renders similarly to the way it does in Firefox :( On Mon, Jul 13, 2015 at 10:14 AM, Maximilian Michels m...@apache.org wrote: Hey Henry! Are you on Firefox? This only shows up for me after a recent Firefox update. Safari still shows it

Re: Documentation Webpage Rendering Problem

2015-07-13 Thread Andra Lungu
I just checked for Firefox 38.0.5, still reproducible. But I guess the fix for 39.0 will do the trick for the previous version as well :) On Mon, Jul 13, 2015 at 10:21 AM, Maximilian Michels m...@apache.org wrote: As far as I know this occurs only in the latest Firefox 39.

Passing around huge hash sets

2015-07-10 Thread Andra Lungu
Hey! It appears that my jobs have the same memory issue disguised in different Exceptions. It's expected, I am passing around hash sets of neighbors and for skewed graphs (i.e. with a lot of neighbors) it's bound to fail one way or the other. This time Kryo hates me :( Caused by:

[jira] [Created] (FLINK-2310) Add an Adamic-Adar Similarity example

2015-07-01 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2310: -- Summary: Add an Adamic-Adar Similarity example Key: FLINK-2310 URL: https://issues.apache.org/jira/browse/FLINK-2310 Project: Flink Issue Type: Task

Re: [Runtime] Division by Zero Exception

2015-06-30 Thread Andra Lungu
Sure! FLINK-2293 On Tue, Jun 30, 2015 at 10:22 AM, Fabian Hueske fhue...@gmail.com wrote: That looks like a serious bug. :-( Can you open a JIRA for that? Thanks, Fabian 2015-06-29 23:24 GMT+02:00 Andra Lungu lungu.an...@gmail.com: From the same series of experiments: I am basically

[jira] [Created] (FLINK-2293) Division by Zero Exception

2015-06-30 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2293: -- Summary: Division by Zero Exception Key: FLINK-2293 URL: https://issues.apache.org/jira/browse/FLINK-2293 Project: Flink Issue Type: Bug Components

Re: Monitoring a Flink Job

2015-06-29 Thread Andra Lungu
, 2015 at 4:19 PM, Andra Lungu lungu.an...@gmail.com wrote: Hey guys, Me again :) So now that my wonderful job finishes, I would like to monitor it a bit (i.e. build some charts on the number of messages per vertex, compute the total amount of time elapsed per computation per

Re: Monitoring a Flink Job

2015-06-29 Thread Andra Lungu
at 1:58 PM, Fabian Hueske fhue...@gmail.com wrote: Have you tried to use a custom accumulator that just appends to a list? 2015-06-29 12:59 GMT+02:00 Andra Lungu lungu.an...@gmail.com: Hey Fabian, I am aware of the way open, preSuperstep(), postSuperstep() etc can help me within

[Runtime] Division by Zero Exception

2015-06-29 Thread Andra Lungu
From the same series of experiments: I am basically running an algorithm that simulates a Gather Sum Apply Iteration that performs Traingle Count (Why simulate it? Because you just need a superstep - useless overhead if you use the runGatherSumApply function in Graph). What happens, at a high

Monitoring a Flink Job

2015-06-27 Thread Andra Lungu
Hey guys, Me again :) So now that my wonderful job finishes, I would like to monitor it a bit (i.e. build some charts on the number of messages per vertex, compute the total amount of time elapsed per computation per vertex, etc). The main computational-intensive operation is a coGroup. There,

Re: Flink Runtime Exception

2015-06-19 Thread Andra Lungu
, Till On Fri, Jun 19, 2015 at 1:42 PM Andra Lungu lungu.an...@gmail.com wrote: Hi everyone, I ran a job this morning on 30 wally nodes. DOP 224. Worked like a charm. Then, I ran a similar job, on the exact same configuration, on the same input data set. The only difference between

Re: The correct location for zipWithIndex and zipWithUniqueId

2015-06-12 Thread Andra Lungu
are aware of the DataSetUtils. Cheers, Fabian 2015-06-10 10:45 GMT+02:00 Andra Lungu an...@apache.org: Hey everyone, We needed to assign unique labels as vertex values in Gelly at some point. We got a nice suggestion on how to do that in parallel (Implemented

The correct location for zipWithIndex and zipWithUniqueId

2015-06-10 Thread Andra Lungu
Hey everyone, We needed to assign unique labels as vertex values in Gelly at some point. We got a nice suggestion on how to do that in parallel (Implemented in https://github.com/apache/flink/pull/801#issuecomment-110654447). Now the question is where should these two functions go? Should they

[jira] [Created] (FLINK-2178) groupReduceOnNeighbors throws NoSuchElementException

2015-06-06 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2178: -- Summary: groupReduceOnNeighbors throws NoSuchElementException Key: FLINK-2178 URL: https://issues.apache.org/jira/browse/FLINK-2178 Project: Flink Issue Type

Re: Planning the 0.9 Release

2015-06-05 Thread Andra Lungu
. That is not that big a deal. I am not very attached to the Spargel Stuff, I just want to make sure we do not deprecate something that works well for something that is still work in progress. On Fri, Jun 5, 2015 at 1:46 PM, Andra Lungu lungu.an...@gmail.com wrote: Hi Stephan, I

Re: Planning the 0.9 Release

2015-06-05 Thread Andra Lungu
Hi Stephan, I don't know if I have a saying in this, but I will give it a go :) The two unsolved issues don't affect the functionality at all. Gelly can, at the moment, support anything Spargel could. There is a guide in the documentation explaining how to migrate Spargel code to Gelly. I don't

Re: ALS implementation

2015-06-04 Thread Andra Lungu
Hi Felix, Passing a JoinHint to your function should help. see: http://mail-archives.apache.org/mod_mbox/flink-user/201504.mbox/%3ccanc1h_vffbqyyiktzcdpihn09r4he4oluiursjnci_rwc+c...@mail.gmail.com%3E Cheers, Andra On Thu, Jun 4, 2015 at 7:07 PM, Felix Neutatz neut...@googlemail.com wrote:

[jira] [Created] (FLINK-2140) Access the number of vertices from within the GSA functions

2015-06-03 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2140: -- Summary: Access the number of vertices from within the GSA functions Key: FLINK-2140 URL: https://issues.apache.org/jira/browse/FLINK-2140 Project: Flink Issue

[jira] [Created] (FLINK-2141) Allow GSA's Gather to perform this operation in more than one direction

2015-06-03 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2141: -- Summary: Allow GSA's Gather to perform this operation in more than one direction Key: FLINK-2141 URL: https://issues.apache.org/jira/browse/FLINK-2141 Project: Flink

[jira] [Created] (FLINK-2127) The GSA Documentation has trailing /p s

2015-06-01 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2127: -- Summary: The GSA Documentation has trailing /p s Key: FLINK-2127 URL: https://issues.apache.org/jira/browse/FLINK-2127 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-2113) env.execute() is no longer needed after a call to print()

2015-05-29 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2113: -- Summary: env.execute() is no longer needed after a call to print() Key: FLINK-2113 URL: https://issues.apache.org/jira/browse/FLINK-2113 Project: Flink Issue

Gelly Blog Post

2015-05-26 Thread Andra Lungu
Hey everyone, We are very excited to share the first stable draft of the Gelly blog post with you :D https://docs.google.com/document/d/1FMtpwKSE3kY7RfH082LzQpWrY6o-fdZVxqambIiC_rU/edit?usp=sharing *Feedback* is welcome, as usual! Andra

[jira] [Created] (FLINK-2093) Add a difference method to Gelly's Graph class

2015-05-26 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2093: -- Summary: Add a difference method to Gelly's Graph class Key: FLINK-2093 URL: https://issues.apache.org/jira/browse/FLINK-2093 Project: Flink Issue Type: New

Re: Gelly Roadmap

2015-05-20 Thread Andra Lungu
Thanks, Stephan! On Wed, May 20, 2015 at 8:42 PM, Stephan Ewen se...@apache.org wrote: All right, you should have permissions now. On Wed, May 20, 2015 at 8:37 PM, Andra Lungu lungu.an...@gmail.com wrote: Sure, but first I need permissions! :) *NOTE*: Due to spamming, we can not give

Re: Gelly Roadmap

2015-05-20 Thread Andra Lungu
The Roadmap is now available as a wiki page. https://cwiki.apache.org/confluence/display/FLINK/Flink+Gelly We're still happy to hear comments/suggestions, if any :) On Wed, May 20, 2015 at 8:43 PM, Andra Lungu lungu.an...@gmail.com wrote: Thanks, Stephan! On Wed, May 20, 2015 at 8:42 PM

Re: Gelly Roadmap

2015-05-19 Thread Andra Lungu
this happen :) We would love to get the community's feedback on this, so please feel free to comment on the doc or discuss individual points here. Cheers, Vasia. On 18 May 2015 at 11:01, Andra Lungu lungu.an...@gmail.com wrote: Dear Squirrels

[jira] [Created] (FLINK-2012) addVertices, addEdges, removeVertices, removeEdges methods

2015-05-14 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2012: -- Summary: addVertices, addEdges, removeVertices, removeEdges methods Key: FLINK-2012 URL: https://issues.apache.org/jira/browse/FLINK-2012 Project: Flink Issue

Re: Generat DataSet gaussian distribution

2015-05-08 Thread Andra Lungu
Hi Yi, To my knowledge, there is no simple way to generate this kind of DataSet(i.e. there is no env.generateGaussianSequence()). However, if you look in flink-perf, Till used something like this there:

Re: [Documentation] Broken links

2015-05-07 Thread Andra Lungu
, Stephan On Fri, Apr 24, 2015 at 5:00 PM, Andra Lungu lungu.an...@gmail.com wrote: Hey everyone, I was following the documentation on how to create a new runtime operator and I noticed that all the links to the classes on Github return 404. http://ci.apache.org

[jira] [Created] (FLINK-1976) Add ForwardedFields* hints for the optimizer

2015-05-05 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1976: -- Summary: Add ForwardedFields* hints for the optimizer Key: FLINK-1976 URL: https://issues.apache.org/jira/browse/FLINK-1976 Project: Flink Issue Type: Wish

[jira] [Created] (FLINK-1975) Graph getUndirected improvement

2015-05-05 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1975: -- Summary: Graph getUndirected improvement Key: FLINK-1975 URL: https://issues.apache.org/jira/browse/FLINK-1975 Project: Flink Issue Type: Improvement

Re: Adding a new operator

2015-04-27 Thread Andra Lungu
in the process to make sure that the design is aligned with the system. Otherwise, duplicated work might be necessary before it can be added to the system. Cheers, Fabian 2015-04-26 13:05 GMT+02:00 Andra Lungu lungu.an...@gmail.com: Yes Markus, ds.reduce() - AllReduceDriver

Re: Adding a new operator

2015-04-26 Thread Andra Lungu
enum there is several different ReduceDrivers depending on the strategy the optimizer chooses. best, Markus 2015-04-26 12:26 GMT+02:00 Andra Lungu lungu.an...@gmail.com: Hey guys, I am trying to add a new runtime operator; To this end, I am following the guide here: http

Adding a new operator

2015-04-26 Thread Andra Lungu
Hey guys, I am trying to add a new runtime operator; To this end, I am following the guide here: http://ci.apache.org/projects/flink/flink-docs-master/internals/add_operator.html and the code itself. From what I understood, the run() in ReduceDriver, for instance, should be called every time a

[Documentation] Broken links

2015-04-24 Thread Andra Lungu
Hey everyone, I was following the documentation on how to create a new runtime operator and I noticed that all the links to the classes on Github return 404. http://ci.apache.org/projects/flink/flink-docs-master/internals/add_operator.html Eventually, I started to check the code out directly

Re: Merge Python API

2015-04-20 Thread Andra Lungu
+1 from me too, the presentation of the Python API seemed okay, plus that having one more project in staging will not hurt. On Mon, Apr 20, 2015 at 2:49 PM, Gyula Fóra gyf...@apache.org wrote: +1 On Mon, Apr 20, 2015 at 2:41 PM, Fabian Hueske fhue...@gmail.com wrote: +1 2015-04-20

Re: [Gelly] Vertex-centric iteration updateVertex does not get called

2015-04-16 Thread Andra Lungu
Hello Gabor, Yes, currently updateVertex only gets called when a new message was received. Could you please describe the logic behind your triangle count? The one I know is described at the beginning of page 1643 in this article: http://www.cc.gatech.edu/~bader/papers/GraphBSPonXMT-MTAAP2013.pdf

[jira] [Created] (FLINK-1883) Add Min Vertex ID Propagation Library Method and Example

2015-04-13 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1883: -- Summary: Add Min Vertex ID Propagation Library Method and Example Key: FLINK-1883 URL: https://issues.apache.org/jira/browse/FLINK-1883 Project: Flink Issue

Re: Memory segment error

2015-03-30 Thread Andra Lungu
here. This is regular user code. And the behaviour in the end is definitely not the one expected. Any idea why this might be happening? Thanks! Andra On Fri, Mar 27, 2015 at 12:08 AM, Andra Lungu lungu.an...@gmail.com wrote: Opps! Sorry! Did not know the mailing list does not support attachments

Re: Memory segment error

2015-03-30 Thread Andra Lungu
and the test case that gives the error? Then I have a look at it and try to figure out what's going wrong. Cheers, Fabian 2015-03-30 10:43 GMT+02:00 Andra Lungu lungu.an...@gmail.com: Hello, I went on and did some further debugging on this issue. Even though the exception said that the problem

Re: Memory segment error

2015-03-30 Thread Andra Lungu
of operators in your testcase, maybe by splitting it in half? 2015-03-30 11:01 GMT+02:00 Andra Lungu lungu.an...@gmail.com: Sure, It was in the first mail but that was sent a while ago :) This is the code: https://github.com/andralungu/gelly-partitioning/tree/alphaSplit I also added

Re: Memory segment error

2015-03-30 Thread Andra Lungu
-30 23:25 GMT+02:00 Andra Lungu lungu.an...@gmail.com: Hi Fabian, I'll see what I can do :). I am just a bit shocked. If this set of coGroups and joins was too much for a test case, how come the following worked? https://github.com/andralungu/flink/commit

Re: Memory segment error

2015-03-26 Thread Andra Lungu
org.apache.flink.runtime.taskmanager.TaskManager - Using 1193 MB for Flink managed memory. Can you tell us how much memory Flink is managing in your case? On Thu, Mar 26, 2015 at 4:46 PM, Andra Lungu lungu.an...@gmail.com wrote: Hello everyone, I guess I need to revive this old discussion: http://apache

Re: Queries regarding RDFs with Flink

2015-03-22 Thread Andra Lungu
Hi Flavio, We don't have a specific example for generating RDF graphs using Gelly, but I will try to drop some lines of code here and hope you will find them useful. An RDF statement is formed of Subject - Predicate - Object triples. In Edge notation, the Subject and the Object will be the

Re: [Delta Iterations] The dirty insides(insights)

2015-03-20 Thread Andra Lungu
is. The code you pasted is from the Spargel API - specifically just forwarding registered broadcast variables. What do you mean with the vertex values get reset ? Stephan PS: The delta iterations are based in this paper: http://arxiv.org/pdf/1208.0088.pdf On Thu, Mar 19, 2015 at 9:25 AM, Andra

[Delta Iterations] The dirty insides(insights)

2015-03-19 Thread Andra Lungu
Hello, I've used delta iterations several times up until now, but I just realized that I never fully understood what happens inside. And the documentation only explains things from a user's perspective. Which is why I could really use your help :). Here goes nothing: In Gelly, Graph.java, there

Re: How to test including ITCase using maven?

2015-03-18 Thread Andra Lungu
The way I do it is mvn -e test :) On Wed, Mar 18, 2015 at 11:21 AM, Chiwan Park chiwanp...@icloud.com wrote: Hello. I have a question about test using maven. I tested with `mvn -pl flink-tests test` command to test flink-tests module. I got followed execution logs. (I removed some

[jira] [Created] (FLINK-1726) Add Community Detection Library and Example

2015-03-18 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1726: -- Summary: Add Community Detection Library and Example Key: FLINK-1726 URL: https://issues.apache.org/jira/browse/FLINK-1726 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-1741) Add Jaccard Similarity Metric Example

2015-03-18 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1741: -- Summary: Add Jaccard Similarity Metric Example Key: FLINK-1741 URL: https://issues.apache.org/jira/browse/FLINK-1741 Project: Flink Issue Type: Task

Re: [Multiple Programs Test Base] Consequence of Shading Guava away

2015-03-17 Thread Andra Lungu
IntelliJ is telling me). https://github.com/rmetzger/scratch/commit/203d647086d089575fb27223462d79c87771f1d1 Let me know if this is sufficient or if you need help. Best, Robert On Tue, Mar 17, 2015 at 11:10 AM, Andra Lungu lungu.an...@gmail.com wrote: Good morning, I am using Gelly

Re: [DISCUSS] Deprecate Spargel API for 0.9

2015-03-10 Thread Andra Lungu
Big +1 for deprecating Spargel :D On Tue, Mar 10, 2015 at 10:02 PM, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Hi all, I would like your opinion on whether we should deprecate the Spargel API in 0.9. Gelly doesn't depend on Spargel, it actually contains it -- we have copied the

[MultipleProgramsTestBase][Cluster vs. Collection mode] Inconsistent Behavior

2015-03-04 Thread Andra Lungu
Hello, I have implemented a Bulk Synchronous Version of Triangle Count. The code can be found here: https://github.com/andralungu/gelly-partitioning/tree/triangles In this algorithm, the messages sent differ as the superstep differs. In order to distinguish between superstep numbers, I used the

Re: gelli graph algorithm

2015-02-26 Thread Andra Lungu
Hello, Also, for guidelines on how to implement a graph algorithm in Gelly, you can use the provided examples: https://github.com/apache/flink/tree/master/flink-staging/flink-gelly/src/main/java/org/apache/flink/graph/example Have fun! Andra On Thu, Feb 26, 2015 at 5:31 PM, Fabian Hueske

[DISCUSS] Should we improve the structure of our Gelly test suite?

2015-02-21 Thread Andra Lungu
Hello everyone, The issue reported here: https://issues.apache.org/jira/browse/FLINK-1587 made us -or at least me :) - wonder if the current approach we have towards testing the graph methods is the best one. After implementing the quick fix to the bug(check if the vertex.iterator hasNext and if

[jira] [Created] (FLINK-1576) Change the examples to be consistent with the other Flink examples

2015-02-17 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1576: -- Summary: Change the examples to be consistent with the other Flink examples Key: FLINK-1576 URL: https://issues.apache.org/jira/browse/FLINK-1576 Project: Flink

Memory segment error when migrating functional code from Flink 0.9 to 0.8

2015-02-09 Thread Andra Lungu
Hello everyone, I am implementing a graph algorithm as part of a course and I will also add it to the Flink- Gelly examples. My problem is that I started developing it in the Gelly repository, which runs on flink 0.9. It works like a charm there, but in order to test in on a cluster to see its