[ https://issues.apache.org/jira/browse/SPARK-42550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuming Wang resolved SPARK-42550. --------------------------------- Resolution: Cannot Reproduce > table directory will lost on hdfs when `INSERT OVERWRITE` faild > --------------------------------------------------------------- > > Key: SPARK-42550 > URL: https://issues.apache.org/jira/browse/SPARK-42550 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.2.3 > Environment: spark 3.2.3 / HDP 3.1.4 > Reporter: kevinshin > Priority: Critical > Attachments: image-2023-02-24-15-21-55-273.png, > image-2023-02-24-15-23-32-977.png, image-2023-02-24-15-25-57-770.png > > > {color:#4c9aff}when a `{*}INSERT{*} OVERWRITE *TABLE`* statment faild during > execution, the table's directory will be deleted. this is not happen in > spark 3.2.1.{color} > {color:#4c9aff}for example: {color} > *CREATE* *TABLE* *IF* *NOT* *EXISTS* test.spark32_overwrite(amt1 {*}int{*}) > STORED *AS* ORC; > *INSERT* OVERWRITE *TABLE* test.spark32_overwrite *select* 128; > *CREATE* *TABLE* *IF* *NOT* *EXISTS* test.spark32_overwrite2(amt1 long) > STORED *AS* ORC; > *INSERT* OVERWRITE *TABLE* test.spark32_overwrite *select* 6000044164; > *INSERT* OVERWRITE *TABLE* test.spark32_overwrite *select* amt1 *from* > test.spark32_overwrite2; – {color:#de350b}*this will got Casting overflow > exception*{color} > {color:#4c9aff}and then :{color} > *select* * *from* test.spark32_overwrite; > {color:#4c9aff}will got error:{color} > {color:#172b4d}java.io.FileNotFoundException{color} > {color:#172b4d}!image-2023-02-24-15-21-55-273.png!{color} > {color:#172b4d}the table's directory is losted. use `hdfs dfs -ls` cmd to > check:{color} > !image-2023-02-24-15-23-32-977.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org