beliefer edited a comment on issue #23841: [SPARK-26936][SQL] Fix bug of insert overwrite local dir and inconsistent behavior with Hive URL: https://github.com/apache/spark/pull/23841#issuecomment-467787546 Then I pull the master branch and compile it and deploy it on my hadoop cluster.I get the inconsistent behavior again. The spark version to test is 3.0.0. ``` ls /tmp/noexistdir ls: cannot access /tmp/noexistdir: No such file or directory ``` ``` Java HotSpot(TM) 64-Bit Server VM warning: Using the ParNew young collector with the Serial old collector is deprecated and will likely be removed in a future release Spark context Web UI available at http://10.198.66.204:55326 Spark context available as 'sc' (master = local[*], app id = local-1551259036573). Spark session available as 'spark'. Welcome to spark version 3.0.0-SNAPSHOT Using Scala version 2.12.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_131) Type in expressions to have them evaluated. Type :help for more information. scala> sql("""select * from t""").show +---+---+ | c0| c1| +---+---+ | 1| 1| +---+---+ scala> sql("""insert overwrite local directory '/tmp/noexistdir/t' select * from t""") res1: org.apache.spark.sql.DataFrame = [] scala> ``` ``` ll /tmp/noexistdir/t -rw-r--r-- 1 xitong xitong 0 Feb 27 17:19 /tmp/noexistdir/t vi /tmp/noexistdir/t 1 ``` The `/tmp/noexistdir/t` is a file too.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
