I will have to try and make some lower-level way to validate and repair corrupted .avro files and/or append them correctly, since this is killing my M/R jobs. And it takes a long time digging to find the offending file (it would be nice if the 'Invalid Sync!' exception listed this).
I'll let you know if I come up with anything useful. Too many other things to do now.... On 01/15/2013 01:39 PM, Alan Miller wrote: > Just an idea but... > I thought there were some low level methods available that you could use to > get the sync markers. Maybe then you could sequentially step through the > orig file and try to write each record to a new file. > > Alan. > > > Sent from my iPhone > > On Jan 15, 2013, at 18:59, Terry Healy <[email protected]> wrote: > >> I have an .avro file that I'm trying to use within a Map/Reduce job. I >> believe it was corrupted when I appended one file to another by mistake. >> >> Are there any tools to repair this? >> >> >>
