The following fails for me in Spark 1.5: https://gist.github.com/nitay/d08cb294ccf00b80c49a Specifically, it returns 1 instead of 100 (in both versions). When I print out the contents (i.e. collect()) I see all 100 items, yet the count returns 1.
This works in 1.3 and 1.4. Any ideas what's going on?