RE: Pattern Matching / Equals on Case Classes in Spark Not Working

2015-01-13 Thread Rosner, Frank (Allianz SE)
. Thanks! Frank From: Matei Zaharia [mailto:matei.zaha...@gmail.com] Sent: Montag, 12. Januar 2015 19:54 To: Rosner, Frank (Allianz SE) Cc: user@spark.apache.org Subject: Re: Pattern Matching / Equals on Case Classes in Spark Not Working Is this in the Spark shell? Case classes don't work correct

Re: Pattern Matching / Equals on Case Classes in Spark Not Working

2015-01-12 Thread Matei Zaharia
Is this in the Spark shell? Case classes don't work correctly in the Spark shell unfortunately (though they do work in the Scala shell) because we change the way lines of code compile to allow shipping functions across the network. The best way to get case classes in there is to compile them int

Pattern Matching / Equals on Case Classes in Spark Not Working

2015-01-12 Thread Rosner, Frank (Allianz SE)
Dear Spark Users, I googled the web for several hours now but I don't find a solution for my problem. So maybe someone from this list can help. I have an RDD of case classes, generated from CSV files with Spark. When I used the distinct operator, there were still duplicates. So I investigated a