Hi, I've just noticed that every time Dataset.hint is used it triggers execution of logical commands, their unions and hint resolution (among other things that analyzer does).
Why? Why does hint trigger hint resolution (through QueryExecution.analyzed)? [1] And moreover why not to use planWithBarrier instead? [2] Looks like an oversight, doesn't it? [1] https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L1219 [2] https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L195 Pozdrawiam, Jacek Laskowski ---- https://about.me/JacekLaskowski Mastering Spark SQL https://bit.ly/mastering-spark-sql Spark Structured Streaming https://bit.ly/spark-structured-streaming Mastering Kafka Streams https://bit.ly/mastering-kafka-streams Follow me at https://twitter.com/jaceklaskowski
