Can you include the output of `explain()` for each of the runs?
On Tue, Sep 1, 2015 at 1:06 AM, patcharee wrote:
> Hi,
>
> I found spark 1.5 sorting is very slow compared to spark 1.4. Below is my
> code snippet
>
> val sqlRDD = sql("select date, u, v, z from fino3_hr3 where zone == 2
> and
Hi,
I found spark 1.5 sorting is very slow compared to spark 1.4. Below is
my code snippet
val sqlRDD = sql("select date, u, v, z from fino3_hr3 where zone ==
2 and z >= 2 and z <= order by date, z")
println("sqlRDD " + sqlRDD.count())
The fino3_hr3 (in the sql command) is a hive ta