Re: PageRank - 4x slower then Spark?!

Does someone has a current performance test based on PageRank or an idea why Flink lost the comparison? > Am 18.08.2017 um 19:51 schrieb Kaepke, Marc <marc.kae...@haw-hamburg.de>: > > Hi everyone, > > I compared Flink and Spark by using PageRank. I guessed Flink will

PageRank - 4x slower then Spark?!

Hi everyone, I compared Flink and Spark by using PageRank. I guessed Flink will beat Spark or have the same level. But Spark is up to 4x faster then Flink. I hope I did a mistake. So please help me to improve the performance of my cluster and config. The cluster has 4 computers: One JobManager

PageRank iteration

Hi everyone, does PageRank use bulk or delta iteration? I mean the implementation of PageRank of the package: package org.apache.flink.graph.library.link_analysis; Thanks. Best, Marc

Re: Standalone cluster - taskmanager settings ignored

stopped and would reconnect to the > restarted JobManager. > > Greg > > >> On Aug 11, 2017, at 11:25 AM, Kaepke, Marc <marc.kae...@haw-hamburg.de> >> wrote: >> >> Hi, >> >> I have a cluster of 4 dedicated machines (no VMs). My previou

Re: Standalone cluster - taskmanager settings ignored

cluster included additionally slave5. My current cluster has not slave5. But the WebUI shows 4 TM -> master, slave1, slave3 and slave5 Am 11.08.2017 um 17:25 schrieb Kaepke, Marc <marc.kae...@haw-hamburg.de<mailto:marc.kae...@haw-hamburg.de>>: Hi, I have a cluster of 4 ded

Standalone cluster - taskmanager settings ignored

Hi, I have a cluster of 4 dedicated machines (no VMs). My previous config was: 1 master and 3 slaves. Each machine provides a task- or jobmanager. Now I want to reduce my cluster and have 1 master and 3 slaves, but one machine provides a jobmanager and one task manager in parallel. I changed

Re: FileNotFound Exception in Cluster Standalone

l file. HDFS is recommended. If it is already on HDFS then you need to provide an HDFS URL to the file. On 5. Aug 2017, at 14:27, Kaepke, Marc <marc.kae...@haw-hamburg.de<mailto:marc.kae...@haw-hamburg.de>> wrote: Hi there, my really small test job reads an external file and print the i

FileNotFound Exception in Cluster Standalone

Hi there, my really small test job reads an external file and print the input to console. Execute it as standalone with a local cluster, everything is fine. If I execute the same job as standalone with 1 job manager und 1 task manager, I get an FileNotFound Exception. As a real distributed

Re: Gelly PageRank implementations in 1.2 to 1.3

implementation. The last one (without any graph model) works fine. Do you have any idea what I doing wrong? Marc Am 24.07.2017 um 20:56 schrieb Kaepke, Marc <marc.kae...@haw-hamburg.de<mailto:marc.kae...@haw-hamburg.de>>: Thanks for your explanation. The vertex-centric, sg and gsa P

Re: Gelly PageRank implementations in 1.2 to 1.3

tion. On Jul 23, 2017, at 6:38 AM, Kaepke, Marc <marc.kae...@haw-hamburg.de<mailto:marc.kae...@haw-hamburg.de>> wrote: Hi Greg, I do an evaluation between Gelly and GraphX (Spark). Both frameworks implement PageRank and Gelly provides a lot of variants (*thumbs up*). During a re

Re: Gelly PageRank implementations in 1.2 to 1.3

/main/java/org/apache/flink/graph/drivers/PageRank.java): graph.run(new PageRank<K, VV, EV>(dampingFactor, iterations, convergenceThreshold)); Please let us know of any issues or additional questions! Greg On Jul 22, 2017, at 4:33 PM, Kaepke, Marc <marc.kae...@haw-hamburg.de<

Gelly PageRank implementations in 1.2 to 1.3

Hi there, why was the PageRank version (which implements the GraphAlgorithm interface) removed in 1.3? How can I use the new PageRank implementation in 1.3.x? Why PageRank doesn’t use the graph processing models (vertex-centric, sg or gsa) anymore? Thanks! Bests, marc

Gelly - bipartite graph runs vertex-centric

Hi, does Gelly provides a vertex-centric iteration on a bipartite graph? A bipartite graph is using BipartiteEdges and vertex-centric supports regular edges only. Thanks! Best, Marc

Re: coGroup exception or something else in Gelly job

Hi Greg if you have an idea, I'm still interested. In case you didn't, please give me a feedback too. Best, Marc Sent from my iPhone On 15. Jun 2017, at 15:19, Kaepke, Marc <marc.kae...@haw-hamburg.de<mailto:marc.kae...@haw-hamburg.de>> wrote: Hi Greg, I wanna ask if there

Re: coGroup exception or something else in Gelly job

Hi Greg, I wanna ask if there was any news about the implementation or opportunities? Thanks and best regards, Marc Am 12.06.2017 um 19:28 schrieb Kaepke, Marc <marc.kae...@haw-hamburg.de<mailto:marc.kae...@haw-hamburg.de>>: I’m working on an implementation of SemiClustering [1]

Re: coGroup exception or something else in Gelly job

u share a basic description of the algorithm you are working to implement? On Mon, Jun 12, 2017 at 5:47 AM, Kaepke, Marc <marc.kae...@haw-hamburg.de<mailto:marc.kae...@haw-hamburg.de>> wrote: It seems Flink used a different exception graph outside of my IDE (intellij) The job ana

coGroup exception or something else in Gelly job

Hi everyone, I don’t have any exceptions if I execute my Gelly job in my IDE (local) directly. The next step is an execution with a real kubernetes cluster (1 JobManager and 3 TaskManager on dedicated machines). The word count example is running without exceptions. My Gelly job throws

Re: Flink on kubernetes -> shell deployment

e web dashboard, you probably have access to the > Jobmanager in general and can submit jobs from your command line by passing > > flink run --jobmanager ... > > > I've looped in Patrick in case I am missing something kubernetes-specific > here. > > > Nico >

Re: Gelly - generics with custom vertex value

il.ArrayList$SubList does not work out of the box with Kryo / Flink. Try registering a custom serializer for it... On Wed, May 10, 2017 at 4:16 PM, Kaepke, Marc <marc.kae...@haw-hamburg.de<mailto:marc.kae...@haw-hamburg.de>> wrote: Hi, a part of my bachelor thesis is an implementatio

Gelly - generics with custom vertex value

Hi, a part of my bachelor thesis is an implementation of the Semi-Clustering algorithm [1]. I’m using the Scatter-Gather-Iteration. Each vertex has to know its neighbors and the edge-value between of that. Because Gelly’s vertex doesn’t provide both information, I wrote an CustomVertexValue

Re: RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: null

Hi, did some had an answer or solution? Best Marc Am 05.05.2017 um 20:05 schrieb Kaepke, Marc <marc.kae...@haw-hamburg.de<mailto:marc.kae...@haw-hamburg.de>>: Hi everyone, what does mean that following exception, if I run my gelly program? Exception in

RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: null

Hi everyone, what does mean that following exception, if I run my gelly program? Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed. at

Re: The implementation of the CoGroupFunction is not serializable

Hi Greg, thanks! Should all inner classes be static? Best, Marc Am 01.05.2017 um 00:21 schrieb Greg Hogan <c...@greghogan.com<mailto:c...@greghogan.com>>: Hi Marc, These errors are usually an inner class which should be a static. Greg On Apr 30, 2017, at 2:13 PM, Kaepke, Mar

The implementation of the CoGroupFunction is not serializable

Hi guys, I have no errors if I run a Graph.groupReduceOnEdges() in main(String[ ] args). But the same function in its own class throws an exception. DataSet>> vertexDataSet = graph.groupReduceOnEdges(new

Re: gelly scatter/gather

Hi Alieh, I can't solve your problem yet. But I work with gelly and Scatter/Gather and later GSA too. Could you publish or show me your algorithm? Best from Hamburg Marc Sent from my iPhone > On 28. Apr 2017, at 13:58, Alieh wrote: > > Hi all > > I have an

Re: Graph iteration with triplets or access to edges

to summarize my question: Does Flink or Gelly offers an access to the edges of a single vertex? Or: I need a VertexTriplet and not an EdgeTriplet (graph.getTriplets()) Thanks! Best, Marc > Am 27.04.2017 um 20:20 schrieb Kaepke, Marc <marc.kae...@haw-hamburg.de>: > > Hi everyone,

Re: Beginner question - sum multiple edges

().run(networkSumMultiEdges); How can I reduce and combine (1 2 20) and (2 1 10) to one Tuple? Best regards Marc Am 17.04.2017 um 21:47 schrieb Kaepke, Marc <marc.kae...@haw-hamburg.de<mailto:marc.kae...@haw-hamburg.de>>: Hi Gábor, thanks a lot Best, Marc Am 17.04.2017 um 20:32 schrieb

Re: has insufficient permissions to access it - Error

ns.com>: > > Hi Marc, > the file path doesn't look quite right, unless you really have such an > (absolute!) file path. > > Nico > > On Saturday, 8 April 2017 17:41:28 CEST Kaepke, Marc wrote: >> Hi, >> >> if I run my small Gelly application on Intel

Gelly - which partitioning

Hi guys, I can’t found on web which graph partitioning are supported by Gelly. During my search I found this link. But the ticket is still open. https://cwiki.apache.org/confluence/display/FLINK/Flink+Gelly Thanks for help! Best Marc

First Program with WordCount - Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/api/common/functions/FlatMapFunction

Hi guys, I followed this guide (https://ci.apache.org/projects/flink/flink-docs-release-1.2/quickstart/java_api_quickstart.html), but I get an Exception if I run WordCount /usr/lib/jvm/java-8-oracle/bin/java -Didea.launcher.port=7536