garyli1019 commented on issue #768: No Space Left On Device for upsert URL: https://github.com/apache/incubator-hudi/issues/768#issuecomment-511510001 Hello @vinothchandar , sure I will use email to ask future questions. This issue was caused by `/tmp` folder was too small. ``` [2019-07-10 15:18:48,515] {bash_operator.py:127} INFO - Caused by: java.io.IOException: No space left on device [2019-07-10 15:18:48,515] {bash_operator.py:127} INFO - at java.io.FileOutputStream.writeBytes(Native Method) [2019-07-10 15:18:48,515] {bash_operator.py:127} INFO - at java.io.FileOutputStream.write(FileOutputStream.java:326) ``` Based on my understanding, hudi is using `java.io` package to write parquet file and the default `java.io.tmpdir` was `/tmp`. I attended to redirect `java.io.tmpdir` to somewhere else, but it doesn't work. Because in Spark on YARN mode, any user-specified `java.io.tmpdir` will be overriden(source: https://community.hortonworks.com/content/supportkb/186401/why-is-spark-filling-the-tmp-folder.html) Do you know if there is any way to redirect `java.io.tmpdir` for Hudi jobs?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
