Re: how to split data-sets efficiently?

[Stephan Ewen]

Hi!

"Splitting", in the sense that one function returns two different data
sets, is currently not supported.

I guess you have to go with Ufuk's suggestion. IN your case, I guess it
would look somewhat like this:


DataSet<Tuple2<Long, String>> mapped = ogiginalStrings.map(HashIdMapper());

DataSet<Long> ids = mapped.map(new ProjectTo2());

DataSet<Long> result = ids.runTheGraphAlgorithm(...)

result.join(mapped).where(...).equalTo(...).with(new MapBackToStrings());


Greetings,
Stephan