Hey Jack, thanks for the update.
On 28 Jul 2014, at 03:22, Jack David Galilee <[email protected]> wrote: > What I had to do with my Arrays was to use the string representation of the > array as the ID after a filter operation where I knew it would be unique in > the dataset and across all iterations. This seems to have lead to a weird > case in which I've needed to duplicate the data into Tuple2<String, String[]> > where the 1st column is the string representation of the 2nd. As you mentioned, this is a shortcoming at the moment, but we already have a PR in place, which is waiting to be tested. You posting here, confirms that we need to fix this asap. > Unfortunately the algorithm still doesn't run correctly but I'm not sure if > this is my fault or not yet. I'm going to keep working on it and will let you > know if it turns out to be the iteration operator again. If you feel differently about posting the program in the future, feel free to do so. You shouldn't worry about whether it is your fault or the system's, because the goal is to make the system as easy to use as possible. So if something is not clear for you, we obviously have stuff to improve. ;-) > I feel like this is an interesting problem to solve (non-unique values and > collection types) and would like to know if you agree. I saw in the mailing > list with a survey you did at University of Berlin that the collection data > type issue is one you're aware of. I'd also be quite interested in helping at > the end of the year when I have some more free time. Sounds great! :)
