Re: Distribute DataSet to subset of nodes

2015-10-19 Thread Stefan Bunk
machine, resulting in no partitioning at all. Is that correct? This happens, even if I run with parallelism 2. Is there some way to control this? Or to run node-local initialization? Cheers Stefan On 21 September 2015 at 21:15, Stefan Bunk <stefan.b...@googlemail.com> wrote: >

Re: Distribute DataSet to subset of nodes

2015-09-21 Thread Stefan Bunk
y 7 > will be send over the network. > On Sep 21, 2015 6:56 PM, "Stefan Bunk" <stefan.b...@googlemail.com> wrote: > >> Hi Fabian, >> >> that sounds good, thank you. >> >> One final question: As I said earlier, this also distributes data in s

Re: Distribute DataSet to subset of nodes

2015-09-16 Thread Stefan Bunk
a cluster and forcing task > scheduling to specific nodes is quite tricky. > Cheers, Fabian > > 2015-09-15 23:15 GMT+02:00 Stefan Bunk <stefan.b...@googlemail.com>: > >> Hi Fabian, >> >> I think we might have a misunderstanding here. I have already copied the &g

Re: Logging in Flink 0.9.0-milestone-1

2015-06-26 Thread Stefan Bunk
/69a400fadd258fe0a1ff0b5670a3611fda4c1cdf/flink-runtime/src/main/java/org/apache/flink/runtime/client/JobClientActor.java#L60 (I'm currently looking up how to control the behavior) If its possible for you, you can switch to 0.9-SNAPSHOT. Best, Robert On Tue, Apr 14, 2015 at 7:02 PM, Stefan Bunk

EOFException when running Flink job

2015-04-17 Thread Stefan Bunk
Hi Squirrels, I have some trouble with a delta-iteration transitive closure program [1]. When I run the program, I get the following error: java.io.EOFException at org.apache.flink.runtime.operators.hash.InMemoryPartition$WriteView.nextSegment(InMemoryPartition.java:333) at

Logging in Flink 0.9.0-milestone-1

2015-04-14 Thread Stefan Bunk
Hi Flinkers, I just switched to 0.9.0-milestone-1, and now I get Flink's logging output again in my console (local execution). I have a log4j.xml under src/main/resources, which says not to log any Flink job progress updates, and which worked fine so far: [...] logger name=org.apache.flink

Re: Logging in Flink 0.9.0-milestone-1

2015-04-14 Thread Stefan Bunk
and it worked). I've used IntelliJ and started the WordCount.java from the quickstart. I think I need more details about your setup to reproduce the issue. On Tue, Apr 14, 2015 at 5:52 PM, Stefan Bunk stefan.b...@googlemail.com wrote: Hi Robert, thanks for the info. Adding the parameter didn't

Re: Logging in Flink 0.9.0-milestone-1

2015-04-14 Thread Stefan Bunk
with - Dlog4j.configuration=log4j.xml? The issue is already fixed in the current master but that doesn't help 0.9.0-milestone-1 users :( Best, Robert On Tue, Apr 14, 2015 at 4:49 PM, Stefan Bunk stefan.b...@googlemail.com wrote: Hi Flinkers, I just switched to 0.9.0-milestone-1, and now I get Flink's

Re: Get 1 element of DataSet

2015-02-05 Thread Stefan Bunk
Hi Vinh, have a look at the first function: http://flink.apache.org/docs/0.8/dataset_transformations.html#first-n Stefan On 5 February 2015 at 15:14, Vinh June hoangthevinh@gmail.com wrote: Hi, Is there any way to get 1 element of a DataSet, for example: