Hi Ashwin, It happens when the application is running!!
Regards, Raja. From: Ashwin Chandra Putta <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday, August 4, 2016 at 4:42 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: HDFS Write File Operator struggling to start Raja, When do you see this issue? Does it happen while the application is running? Does this happen while restarting a failed application? Or does this happen while starting a new application? Regards, Ashwin. On Thu, Aug 4, 2016 at 11:25 AM, Samba Surabhi <[email protected]<mailto:[email protected]>> wrote: If it is the issue with file size, you can rotate the output file. writer.setAlwaysWriteToTmp(true); writer.setRotationWindows(240); Thanks, Samba Surabhi. ________________________________ From: [email protected]<mailto:[email protected]> To: [email protected]<mailto:[email protected]> Subject: HDFS Write File Operator struggling to start Date: Thu, 4 Aug 2016 14:49:16 +0000 Hi I have a HDFS file write operator in my DAG, which is failing to start a new operator and keep on trying to start one!! It created approx. 800 temporary files in the destination hdfs directory!! How can I fix this issue…. And debug the root cause…. ? All I can see in container log is, File corrupted message…!! Can someone please help me fix this ? Regards, Raja. -- Regards, Ashwin.
