I also think a Giraph-like model could be added, but we shouldn't remove
Spargel in favour of it!
On Tue, Nov 3, 2015 at 2:35 AM, Stephan Ewen wrote:
> When creating the original version of Spargel I was pretty much thinking in
> GSA terms, more than in Pregel terms. There are
p(new StatefulMapper())
> >
> >.print();
> >
> >env.execute();
> >
> > }
> >
> > The output is the following on my machine (discarded the output of the
> > print):
> >
> > Key: 2 Previous state was: -1 Update state to:
Andra Lungu created FLINK-2926:
--
Summary: Add a Strongly Connected Components Library Method
Key: FLINK-2926
URL: https://issues.apache.org/jira/browse/FLINK-2926
Project: Flink
Issue Type
Hi Saumitra,
As you already noticed, the first version (with duplicates) is highly
inefficient and consumes a lot of memory. So, I suggest we drop it for now.
The version with the label makes a lot of modifications on the base Graph
class, and this, in my opinion would make it more difficult to
Andra Lungu created FLINK-2899:
--
Summary: The groupReduceOn* methods which take types as a
parameter fail with TypeErasure
Key: FLINK-2899
URL: https://issues.apache.org/jira/browse/FLINK-2899
Project
Hey guys,
Long time, no see :). I recently started a new job and it involves
performing a set of real-time data analytics using Apache Kafka, Storm
and Flume.
What happens, on a very high level, is that set of signals is
collected, stored into a Kafka topic and then Storm is used to filter
Andra Lungu created FLINK-2715:
--
Summary: Benchmark Triangle Count methods
Key: FLINK-2715
URL: https://issues.apache.org/jira/browse/FLINK-2715
Project: Flink
Issue Type: Task
Andra Lungu created FLINK-2714:
--
Summary: Port the Flink DataSet Triangle Count example to the
Gelly library
Key: FLINK-2714
URL: https://issues.apache.org/jira/browse/FLINK-2714
Project: Flink
Andra Lungu created FLINK-2661:
--
Summary: Add a Node Splitting Technique to Overcome the
Limitations of Skewed Graphs
Key: FLINK-2661
URL: https://issues.apache.org/jira/browse/FLINK-2661
Project: Flink
Andra Lungu created FLINK-2634:
--
Summary: Add a Vertex-centric Version of the Tringle Count Library
Method
Key: FLINK-2634
URL: https://issues.apache.org/jira/browse/FLINK-2634
Project: Flink
Andra Lungu created FLINK-2570:
--
Summary: Add a Triangle Count Library Method
Key: FLINK-2570
URL: https://issues.apache.org/jira/browse/FLINK-2570
Project: Flink
Issue Type: Task
Andra Lungu created FLINK-2563:
--
Summary: Gelly's Graph Algorithm Interface is limites
Key: FLINK-2563
URL: https://issues.apache.org/jira/browse/FLINK-2563
Project: Flink
Issue Type
:
Wow, this is impressive :-)
Amazing work, Gelly folks!
On Tue, May 26, 2015 at 10:03 AM, Andra Lungu lungu.an...@gmail.com
wrote:
Hey everyone,
We are very excited to share the first stable draft of the Gelly blog
post
with you :D
https://docs.google.com
.
Best,
Martin
On 22.08.2015 09:51, Andra Lungu wrote:
Hey everyone,
When coding for my thesis, I observed that half of the current Gelly
functions (the ones that use join operators) fail on a cluster environment
with the following exception:
java.lang.IllegalArgumentException: Too few
Congrats Chesnay! :)
On Thu, Aug 20, 2015 at 8:51 PM, Márton Balassi balassi.mar...@gmail.com
wrote:
Welcome Chesnay!
On Thu, Aug 20, 2015 at 7:29 PM, Henry Saputra henry.sapu...@gmail.com
wrote:
Welcome Chesnay!
On Thu, Aug 20, 2015 at 2:18 AM, Robert Metzger rmetz...@apache.org
Andra Lungu created FLINK-2546:
--
Summary: The webclient does not allow users to save the execution
plan
Key: FLINK-2546
URL: https://issues.apache.org/jira/browse/FLINK-2546
Project: Flink
I would love to get some feedback from the guys at data Artisans about this
one.
So far, the comments originated and spread in the Stockholm area :)
On Tue, Aug 11, 2015 at 6:33 PM, Andra Lungu lungu.an...@gmail.com wrote:
Hi Samia,
A good method to statistically determine skewed vertices
.
On Aug 10, 2015 10:47 AM, Andra Lungu lungu.an...@gmail.com wrote:
Hey,
Before actually opening a PR, I wanted to hear your opinion. So, here
goes
nothing :).
I'd like to add the core of my master thesis to Gelly. That is, a series
of
operators that take a skewed graph, split its
they will get benefits by using it since node
splitting doesn’t come with no cost I guess.
I am really eager to see how this will evolve, I think it’s good effort.
cheers
Paris
On 11 Aug 2015, at 14:58, Andra Lungu lungu.an...@gmail.com wrote:
Hi Vasia,
I shall polish the functions a bit
to separate the high degree vertices
from the low degree vertices.
Regards,
Samia
On Tue, Aug 11, 2015 at 3:41 PM, Andra Lungu lungu.an...@gmail.com
wrote:
Hi Paris,
Nice to virtually meet you too :)
Maybe it makes sense to share my freshest chart:
https://drive.google.com/file
Hey,
Before actually opening a PR, I wanted to hear your opinion. So, here goes
nothing :).
I'd like to add the core of my master thesis to Gelly. That is, a series of
operators that take a skewed graph, split its high degree vertices into
subvertices and redistribute the edges accordingly (thus
,
I also encountered the EOF exception for a delta iteration with more
data. With less data it works ...
Best regards,
Felix
Am 27.07.2015 10:25 vorm. schrieb Andra Lungu lungu.an...@gmail.com:
Hi Stephan,
I tried to debug a bit around the EOF Exception. It seems that I am
pretty
Hi Vasia,
We should document the examples in the Gelly guide, I totally agree,
perhaps something similar to what Giraph offers.
However, if you recall, the examples are also there for test purposes. The
library methods contain just the run method; no main method. For the tests,
the nicest way to
,
Stephan
On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu lungu.an...@gmail.com
wrote:
Hi guys,
I opened these JIRAs a while ago and, unfortunately, no one had time to
look at them :|.
[1] https://issues.apache.org/jira/browse/FLINK-2361
[2] https://issues.apache.org/jira/browse
Hi guys,
I opened these JIRAs a while ago and, unfortunately, no one had time to
look at them :|.
[1] https://issues.apache.org/jira/browse/FLINK-2361
[2] https://issues.apache.org/jira/browse/FLINK-2360
[1] could probably be hacked, but [2] is a huge problem that was
encountered by other
Hi everyone,
I am not 100% sure about this one, so I thought that I could set my
thoughts straight via the mailing list.
Here's the use case. You coGroup a data set of vertices with a data set of
edges. That gives you a complexity of* O(|V| * |E|)*, where |V| is the
total number of vertices and
Andra Lungu created FLINK-2360:
--
Summary: EOFException
Key: FLINK-2360
URL: https://issues.apache.org/jira/browse/FLINK-2360
Project: Flink
Issue Type: Bug
Components: Local Runtime
Andra Lungu created FLINK-2361:
--
Summary: flatMap + distict gives eroneous results for big data sets
Key: FLINK-2361
URL: https://issues.apache.org/jira/browse/FLINK-2361
Project: Flink
Issue
The bug is also reproducible in Chrome... The page renders similarly to the
way it does in Firefox :(
On Mon, Jul 13, 2015 at 10:14 AM, Maximilian Michels m...@apache.org wrote:
Hey Henry!
Are you on Firefox? This only shows up for me after a recent Firefox
update. Safari still shows it
I just checked for Firefox 38.0.5, still reproducible. But I guess the fix
for 39.0 will do the trick for the previous version as well :)
On Mon, Jul 13, 2015 at 10:21 AM, Maximilian Michels m...@apache.org wrote:
As far as I know this occurs only in the latest Firefox 39.
Hey!
It appears that my jobs have the same memory issue disguised in different
Exceptions. It's expected, I am passing around hash sets of neighbors and
for skewed graphs (i.e. with a lot of neighbors) it's bound to fail one way
or the other.
This time Kryo hates me :(
Caused by:
Andra Lungu created FLINK-2310:
--
Summary: Add an Adamic-Adar Similarity example
Key: FLINK-2310
URL: https://issues.apache.org/jira/browse/FLINK-2310
Project: Flink
Issue Type: Task
Sure! FLINK-2293
On Tue, Jun 30, 2015 at 10:22 AM, Fabian Hueske fhue...@gmail.com wrote:
That looks like a serious bug. :-(
Can you open a JIRA for that?
Thanks, Fabian
2015-06-29 23:24 GMT+02:00 Andra Lungu lungu.an...@gmail.com:
From the same series of experiments:
I am basically
Andra Lungu created FLINK-2293:
--
Summary: Division by Zero Exception
Key: FLINK-2293
URL: https://issues.apache.org/jira/browse/FLINK-2293
Project: Flink
Issue Type: Bug
Components
, 2015 at 4:19 PM, Andra Lungu lungu.an...@gmail.com
wrote:
Hey guys,
Me again :) So now that my wonderful job finishes, I would like to
monitor
it a bit (i.e. build some charts on the number of messages per vertex,
compute the total amount of time elapsed per computation per
at 1:58 PM, Fabian Hueske fhue...@gmail.com wrote:
Have you tried to use a custom accumulator that just appends to a list?
2015-06-29 12:59 GMT+02:00 Andra Lungu lungu.an...@gmail.com:
Hey Fabian,
I am aware of the way open, preSuperstep(), postSuperstep() etc can help
me
within
From the same series of experiments:
I am basically running an algorithm that simulates a Gather Sum Apply
Iteration that performs Traingle Count (Why simulate it? Because you just
need a superstep - useless overhead if you use the runGatherSumApply
function in Graph).
What happens, at a high
Hey guys,
Me again :) So now that my wonderful job finishes, I would like to monitor
it a bit (i.e. build some charts on the number of messages per vertex,
compute the total amount of time elapsed per computation per vertex, etc).
The main computational-intensive operation is a coGroup. There,
,
Till
On Fri, Jun 19, 2015 at 1:42 PM Andra Lungu lungu.an...@gmail.com wrote:
Hi everyone,
I ran a job this morning on 30 wally nodes. DOP 224. Worked like a charm.
Then, I ran a similar job, on the exact same configuration, on the same
input data set. The only difference between
are aware of the DataSetUtils.
Cheers, Fabian
2015-06-10 10:45 GMT+02:00 Andra Lungu an...@apache.org:
Hey everyone,
We needed to assign unique labels as vertex values in Gelly at some
point.
We got a nice suggestion on how to do that in parallel (Implemented
Hey everyone,
We needed to assign unique labels as vertex values in Gelly at some point.
We got a nice suggestion on how to do that in parallel (Implemented in
https://github.com/apache/flink/pull/801#issuecomment-110654447).
Now the question is where should these two functions go? Should they
Andra Lungu created FLINK-2178:
--
Summary: groupReduceOnNeighbors throws NoSuchElementException
Key: FLINK-2178
URL: https://issues.apache.org/jira/browse/FLINK-2178
Project: Flink
Issue Type
. That is not that big a
deal.
I am not very attached to the Spargel Stuff, I just want to make sure we do
not deprecate something that works well for something that is still work in
progress.
On Fri, Jun 5, 2015 at 1:46 PM, Andra Lungu lungu.an...@gmail.com wrote:
Hi Stephan,
I
Hi Stephan,
I don't know if I have a saying in this, but I will give it a go :)
The two unsolved issues don't affect the functionality at all.
Gelly can, at the moment, support anything Spargel could. There is a guide
in the documentation explaining how to migrate Spargel code to Gelly. I
don't
Hi Felix,
Passing a JoinHint to your function should help.
see:
http://mail-archives.apache.org/mod_mbox/flink-user/201504.mbox/%3ccanc1h_vffbqyyiktzcdpihn09r4he4oluiursjnci_rwc+c...@mail.gmail.com%3E
Cheers,
Andra
On Thu, Jun 4, 2015 at 7:07 PM, Felix Neutatz neut...@googlemail.com
wrote:
Andra Lungu created FLINK-2140:
--
Summary: Access the number of vertices from within the GSA
functions
Key: FLINK-2140
URL: https://issues.apache.org/jira/browse/FLINK-2140
Project: Flink
Issue
Andra Lungu created FLINK-2141:
--
Summary: Allow GSA's Gather to perform this operation in more than
one direction
Key: FLINK-2141
URL: https://issues.apache.org/jira/browse/FLINK-2141
Project: Flink
Andra Lungu created FLINK-2127:
--
Summary: The GSA Documentation has trailing /p s
Key: FLINK-2127
URL: https://issues.apache.org/jira/browse/FLINK-2127
Project: Flink
Issue Type: Bug
Andra Lungu created FLINK-2113:
--
Summary: env.execute() is no longer needed after a call to print()
Key: FLINK-2113
URL: https://issues.apache.org/jira/browse/FLINK-2113
Project: Flink
Issue
Hey everyone,
We are very excited to share the first stable draft of the Gelly blog post
with you :D
https://docs.google.com/document/d/1FMtpwKSE3kY7RfH082LzQpWrY6o-fdZVxqambIiC_rU/edit?usp=sharing
*Feedback* is welcome, as usual!
Andra
Andra Lungu created FLINK-2093:
--
Summary: Add a difference method to Gelly's Graph class
Key: FLINK-2093
URL: https://issues.apache.org/jira/browse/FLINK-2093
Project: Flink
Issue Type: New
Thanks, Stephan!
On Wed, May 20, 2015 at 8:42 PM, Stephan Ewen se...@apache.org wrote:
All right, you should have permissions now.
On Wed, May 20, 2015 at 8:37 PM, Andra Lungu lungu.an...@gmail.com
wrote:
Sure, but first I need permissions! :)
*NOTE*: Due to spamming, we can not give
The Roadmap is now available as a wiki page.
https://cwiki.apache.org/confluence/display/FLINK/Flink+Gelly
We're still happy to hear comments/suggestions, if any :)
On Wed, May 20, 2015 at 8:43 PM, Andra Lungu lungu.an...@gmail.com wrote:
Thanks, Stephan!
On Wed, May 20, 2015 at 8:42 PM
this happen :)
We would love to get the community's feedback on this, so please feel
free
to comment on the doc or discuss individual points here.
Cheers,
Vasia.
On 18 May 2015 at 11:01, Andra Lungu lungu.an...@gmail.com wrote:
Dear Squirrels
Andra Lungu created FLINK-2012:
--
Summary: addVertices, addEdges, removeVertices, removeEdges methods
Key: FLINK-2012
URL: https://issues.apache.org/jira/browse/FLINK-2012
Project: Flink
Issue
Hi Yi,
To my knowledge, there is no simple way to generate this kind of
DataSet(i.e. there is no env.generateGaussianSequence()).
However, if you look in flink-perf, Till used something like this there:
,
Stephan
On Fri, Apr 24, 2015 at 5:00 PM, Andra Lungu lungu.an...@gmail.com
wrote:
Hey everyone,
I was following the documentation on how to create a new runtime
operator
and I noticed that all the links to the classes on Github return 404.
http://ci.apache.org
Andra Lungu created FLINK-1976:
--
Summary: Add ForwardedFields* hints for the optimizer
Key: FLINK-1976
URL: https://issues.apache.org/jira/browse/FLINK-1976
Project: Flink
Issue Type: Wish
Andra Lungu created FLINK-1975:
--
Summary: Graph getUndirected improvement
Key: FLINK-1975
URL: https://issues.apache.org/jira/browse/FLINK-1975
Project: Flink
Issue Type: Improvement
in the process to
make sure that the design is aligned with the system.
Otherwise, duplicated work might be necessary before it can be added to
the
system.
Cheers,
Fabian
2015-04-26 13:05 GMT+02:00 Andra Lungu lungu.an...@gmail.com:
Yes Markus,
ds.reduce() - AllReduceDriver
enum there is several different
ReduceDrivers depending on the strategy the optimizer chooses.
best,
Markus
2015-04-26 12:26 GMT+02:00 Andra Lungu lungu.an...@gmail.com:
Hey guys,
I am trying to add a new runtime operator;
To this end, I am following the guide here:
http
Hey guys,
I am trying to add a new runtime operator;
To this end, I am following the guide here:
http://ci.apache.org/projects/flink/flink-docs-master/internals/add_operator.html
and the code itself.
From what I understood, the run() in ReduceDriver, for instance, should be
called every time a
Hey everyone,
I was following the documentation on how to create a new runtime operator
and I noticed that all the links to the classes on Github return 404.
http://ci.apache.org/projects/flink/flink-docs-master/internals/add_operator.html
Eventually, I started to check the code out directly
+1 from me too, the presentation of the Python API seemed okay, plus that
having one more project in staging will not hurt.
On Mon, Apr 20, 2015 at 2:49 PM, Gyula Fóra gyf...@apache.org wrote:
+1
On Mon, Apr 20, 2015 at 2:41 PM, Fabian Hueske fhue...@gmail.com wrote:
+1
2015-04-20
Hello Gabor,
Yes, currently updateVertex only gets called when a new message was
received.
Could you please describe the logic behind your triangle count? The one I
know is described at the beginning of page 1643 in this article:
http://www.cc.gatech.edu/~bader/papers/GraphBSPonXMT-MTAAP2013.pdf
Andra Lungu created FLINK-1883:
--
Summary: Add Min Vertex ID Propagation Library Method and Example
Key: FLINK-1883
URL: https://issues.apache.org/jira/browse/FLINK-1883
Project: Flink
Issue
here. This is regular user code.
And the behaviour in the end is definitely not the one expected. Any idea
why this might be happening?
Thanks!
Andra
On Fri, Mar 27, 2015 at 12:08 AM, Andra Lungu lungu.an...@gmail.com wrote:
Opps! Sorry! Did not know the mailing list does not support attachments
and the test case that
gives the error?
Then I have a look at it and try to figure out what's going wrong.
Cheers, Fabian
2015-03-30 10:43 GMT+02:00 Andra Lungu lungu.an...@gmail.com:
Hello,
I went on and did some further debugging on this issue. Even though the
exception said that the problem
of operators in your testcase, maybe
by splitting it in half?
2015-03-30 11:01 GMT+02:00 Andra Lungu lungu.an...@gmail.com:
Sure,
It was in the first mail but that was sent a while ago :)
This is the code:
https://github.com/andralungu/gelly-partitioning/tree/alphaSplit
I also added
-30 23:25 GMT+02:00 Andra Lungu lungu.an...@gmail.com:
Hi Fabian,
I'll see what I can do :).
I am just a bit shocked. If this set of coGroups and joins was too much
for
a test case, how come the following worked?
https://github.com/andralungu/flink/commit
org.apache.flink.runtime.taskmanager.TaskManager
- Using 1193 MB for Flink managed memory.
Can you tell us how much memory Flink is managing in your case?
On Thu, Mar 26, 2015 at 4:46 PM, Andra Lungu lungu.an...@gmail.com
wrote:
Hello everyone,
I guess I need to revive this old discussion:
http://apache
Hi Flavio,
We don't have a specific example for generating RDF graphs using Gelly, but
I will try to drop some lines of code here and hope you will find them
useful.
An RDF statement is formed of Subject - Predicate - Object triples. In Edge
notation, the Subject and the Object will be the
is. The code you
pasted is from the Spargel API - specifically just forwarding registered
broadcast variables.
What do you mean with the vertex values get reset ?
Stephan
PS: The delta iterations are based in this paper:
http://arxiv.org/pdf/1208.0088.pdf
On Thu, Mar 19, 2015 at 9:25 AM, Andra
Hello,
I've used delta iterations several times up until now, but I just realized
that I never fully understood what happens inside. And the documentation
only explains things from a user's perspective. Which is why I could really
use your help :).
Here goes nothing:
In Gelly, Graph.java, there
The way I do it is mvn -e test :)
On Wed, Mar 18, 2015 at 11:21 AM, Chiwan Park chiwanp...@icloud.com wrote:
Hello.
I have a question about test using maven.
I tested with `mvn -pl flink-tests test` command to test flink-tests
module. I got followed execution logs. (I removed some
Andra Lungu created FLINK-1726:
--
Summary: Add Community Detection Library and Example
Key: FLINK-1726
URL: https://issues.apache.org/jira/browse/FLINK-1726
Project: Flink
Issue Type: Task
Andra Lungu created FLINK-1741:
--
Summary: Add Jaccard Similarity Metric Example
Key: FLINK-1741
URL: https://issues.apache.org/jira/browse/FLINK-1741
Project: Flink
Issue Type: Task
IntelliJ is telling me).
https://github.com/rmetzger/scratch/commit/203d647086d089575fb27223462d79c87771f1d1
Let me know if this is sufficient or if you need help.
Best,
Robert
On Tue, Mar 17, 2015 at 11:10 AM, Andra Lungu lungu.an...@gmail.com
wrote:
Good morning,
I am using Gelly
Big +1 for deprecating Spargel :D
On Tue, Mar 10, 2015 at 10:02 PM, Vasiliki Kalavri
vasilikikala...@gmail.com wrote:
Hi all,
I would like your opinion on whether we should deprecate the Spargel API in
0.9.
Gelly doesn't depend on Spargel, it actually contains it -- we have copied
the
Hello,
I have implemented a Bulk Synchronous Version of Triangle Count. The code
can be found here:
https://github.com/andralungu/gelly-partitioning/tree/triangles
In this algorithm, the messages sent differ as the superstep differs. In
order to distinguish between superstep numbers, I used the
Hello,
Also, for guidelines on how to implement a graph algorithm in Gelly, you
can
use the provided examples:
https://github.com/apache/flink/tree/master/flink-staging/flink-gelly/src/main/java/org/apache/flink/graph/example
Have fun!
Andra
On Thu, Feb 26, 2015 at 5:31 PM, Fabian Hueske
Hello everyone,
The issue reported here: https://issues.apache.org/jira/browse/FLINK-1587
made us -or at least me :) - wonder if the current approach we have towards
testing the graph methods is the best one.
After implementing the quick fix to the bug(check if the vertex.iterator
hasNext and if
Andra Lungu created FLINK-1576:
--
Summary: Change the examples to be consistent with the other Flink
examples
Key: FLINK-1576
URL: https://issues.apache.org/jira/browse/FLINK-1576
Project: Flink
Hello everyone,
I am implementing a graph algorithm as part of a course and I will also add
it to the Flink- Gelly examples.
My problem is that I started developing it in the Gelly repository, which
runs on flink 0.9. It works like a charm there, but in order to test in on
a cluster to see its
84 matches
Mail list logo