Hi,
I finally got it working.
I was using the updateStateByKey() function to maintain the previous
value of the state, and I found that the event list was empty. Hence
handling the empty event list by using event.isEmtpy() sort out the
problem.
On Sun, May 22, 2016 at 7:59 PM, Ted Yu
You mean when rdd.isEmpty() returned false, saveAsTextFile still produced
empty file ?
Can you show code snippet that demonstrates this ?
Cheers
On Sun, May 22, 2016 at 5:17 AM, Yogesh Vyas wrote:
> Hi,
> I am reading files using textFileStream, performing some action
Hi,
I am reading files using textFileStream, performing some action onto
it and then saving it to HDFS using saveAsTextFile.
But whenever there is no file to read, Spark will write and empty RDD(
[] ) to HDFS.
So, how to handle the empty RDD.
I checked rdd.isEmpty() and rdd.count>0, but both of