Yep, just keep reading until you you can't or you get a valid record. I like to increment a "bad record" counter when that happens., for sanity checks.
-----Original Message----- From: "Mark Tozzi" <[email protected]> To: [email protected] Sent: 12/31/2010 4:59 AM Subject: discarding bad rows in load UDF Hi all, I'm working on a custom load UDF. Part of the motivation is to be able to filter out lines in my input data which are not well formed, as this is easy to detect during the load. What should the UDF do when it encounters such a line though? I have tried returning null, and that seems to terminate reading from that split. Should I just loop through the RecordReader until I find a good row or run out of data? Thanks, --Mark Tozzi
