The toString method of Dataset.queryExecution includes the various plans. I usually just log that directly.
On Thu, Feb 16, 2017 at 8:26 AM, Muthu Jayakumar <bablo...@gmail.com> wrote: > Hello there, > > I am trying to write to log-line a dataframe/dataset queryExecution and/or > its logical plan. The current code... > > def explain(extended: Boolean): Unit = { > val explain = ExplainCommand(queryExecution.logical, extended = extended) > > sparkSession.sessionState.executePlan(explain).executedPlan.executeCollect().foreach > { > // scalastyle:off println > r => println(r.getString(0)) > // scalastyle:on println > } > } > > sessionState is not accessible if I were to write my own explain(log: > LoggingAdapter). > > Please advice, > Muthu >