I have a log collection application that writes .avro files within HDFS. Ideally I would like to include the current days (open for append) file as one of the input files for a periodic M/R job.
I tried this but the Map job exited in error with the dreaded "Invalid Sync!" IOException. I guess I should have expected this, but is there a reasonable way around it? Can I catch the exception and just exit the map at that point? All suggestions appreciated. -Terry
