If it is the issue with file size, you can rotate the output file. writer.setAlwaysWriteToTmp(true); writer.setRotationWindows(240);
Thanks,Samba Surabhi. From: [email protected] To: [email protected] Subject: HDFS Write File Operator struggling to start Date: Thu, 4 Aug 2016 14:49:16 +0000 Hi I have a HDFS file write operator in my DAG, which is failing to start a new operator and keep on trying to start one!! It created approx. 800 temporary files in the destination hdfs directory!! How can I fix this issue…. And debug the root cause…. ? All I can see in container log is, File corrupted message…!! Can someone please help me fix this ? Regards, Raja.
