Re: Union of multiple datasets vs Join

2015-03-18 Thread Flavio Pompermaier
I don't know if that could be useful, do you? On Tue, Mar 17, 2015 at 10:29 PM, Stephan Ewen wrote: > Yes, that is the way to do it. > > This makes me think that it would be nice to have a method that builds the > union of a list of data sets. > > DataSet union(DataSet... sets) > > It would be i

Re: Union of multiple datasets vs Join

2015-03-17 Thread Stephan Ewen
Yes, that is the way to do it. This makes me think that it would be nice to have a method that builds the union of a list of data sets. DataSet union(DataSet... sets) It would be implemented like in your loop. Would that be helpful? Stephan On Tue, Mar 17, 2015 at 6:03 PM, Flavio Pompermaier

Re: Union of multiple datasets vs Join

2015-03-17 Thread Flavio Pompermaier
As always one minute after I sent the email I found the problem! It was that I should reassign the initial dataset: ret = ret.union(sourceDs); Bye, Flavio On Tue, Mar 17, 2015 at 5:58 PM, Flavio Pompermaier wrote: > Hi Fabian, > I was trying to use the strategy you suggested with flink 0.8.

Re: Union of multiple datasets vs Join

2015-03-17 Thread Flavio Pompermaier
Hi Fabian, I was trying to use the strategy you suggested with flink 0.8.1 but it seems that the union of the datasets cannot be created programmatically because the union operator gives a name to the generated dataset that is the name of the calling function so that only the first dataset is read