Re: Weird ClassCastException when using generics from Java

2020-05-24 Thread Stephen Coy
Hi Sean, The fix was to provide explicitly typed versions of the the three generic methods at the bottom of the code: i.e static List createListAndCombine(T v) { List c = new ArrayList<>(); c.add(v); return c; } becomes static List createListAndCombine(Nut v) { List c =

Re: Weird ClassCastException when using generics from Java

2020-05-22 Thread Sean Owen
I don't immediately see what the issue could be - try .count()-ing the individual RDDs to narrow it down? What code change made it work? Also I think this could probably be a few lines of SQL with an aggregate, collect_list(), and joins. On Thu, May 21, 2020 at 11:27 PM Stephen Coy wrote: > >

Weird ClassCastException when using generics from Java

2020-05-21 Thread Stephen Coy
Hi there, This will be a little long so please bear with me. There is a buildable example available at https://github.com/sfcoy/sfcoy-spark-cce-test. Say I have the following three tables: Machines Id,MachineType 11,A 12,B 23,B 24,A 25,B Bolts MachineType,Description