Re: How to get a spark sql statement implement duration ?

2017-02-15 Thread ??????????
.@gmail.com>; Cc: "user"<user@spark.apache.org>; Subject: Re: How to get a spark sql statement implement duration ? On 7 Feb 2017 4:17 a.m., "Mars Xu" <xujiao.myc...@gmail.com> wrote: Hello All, Some spark sqls will produce one or more jobs, I have 2 questions,

Re: How to get a spark sql statement implement duration ?

2017-02-07 Thread Jacek Laskowski
On 7 Feb 2017 4:17 a.m., "Mars Xu" wrote: Hello All, Some spark sqls will produce one or more jobs, I have 2 questions, 1, How the cc.sql(“sql statement”) divided into one or more jobs ? It's an implementation detail. You can have zero or more jobs

How to get a spark sql statement implement duration ?

2017-02-06 Thread Mars Xu
Hello All, Some spark sqls will produce one or more jobs, I have 2 questions, 1, How the cc.sql(“sql statement”) divided into one or more jobs ? 2, When I execute spark sql query in spark - shell client, how to get the execution time (Spark 2.1.0) ? if a sql query