mcdull-zhang commented on issue #28232: [SPARK-31459][SQL]fix insert overwrite directory target path is an existing file URL: https://github.com/apache/spark/pull/28232#issuecomment-615549251 > Ur, @mcdull-zhang . When I try at Apache Spark 3.0.0-RC1, I get the following. For me, the current behavior looks correct to me. > > ```scala > scala> spark.version > res1: String = 3.0.0 > > scala> sql("INSERT OVERWRITE LOCAL DIRECTORY '/tmp/p1' STORED AS ORC SELECT 1").show > java.lang.RuntimeException: Cannot create staging directory 'file:/tmp/p1/.hive-staging_hive_2020-04-18_03-27-45_173_5810881255552839519-1': Parent path is not a directory: file:/tmp/p1 > ``` > > Could you give me an example how did you get the final operation result? > > > When using the insert overwrite directory syntax, if the target path is an existing file, the final operation result is incorrect. The situation at the time was that the directory was a hdfs directory, so there was no local keyword. If it is a directory on hdfs, no error will be thrown. Please verify it.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
