difference between ++ and Union of a RDD

2015-12-29 Thread email2...@gmail.com
Hello All - tried couple of operations by using ++ and union on RDD's but realized that the end results are same. Do you know any differences?. val odd_partA = List(1,3,5,7,9,11,1,3,5,7,9,11,1,3,5,7,9,11) odd_partA: List[Int] = List(1, 3, 5, 7, 9, 11, 1, 3, 5, 7, 9, 11, 1, 3, 5, 7, 9, 11) val

what are the cons/drawbacks of a Spark DataFrames

2015-12-14 Thread email2...@gmail.com
Hello All - I've started using the Spark DataFrames and looks like it provides rich column level operations and functions. In the same time, I would like to understand are there any drawbacks / cons of using a DataFrames?. If so please share your experience on that. Thanks, Gokul -- View