[
https://issues.apache.org/jira/browse/PIG-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239270#comment-13239270
]
Jonathan Coveney commented on PIG-2614:
---------------------------------------
Right now there is no plan to make it any more prominent than the fact that it
is a Hadoop counter. I'm not sure how hard it is to print various stuff at the
end... it's an interesting thought (to make it so functions can register
information they want printed at the end). Either way, the counters + the fact
that any threshholds are set by the user feel sufficient to me.
There's a thread going around about the RC. There are a handful open...if you
need this patch to be in the RC, just post in the thread (with any others). My
guess would be sometime next week, to give you a chance to write tests, Bill to
finish up some of his fixes, Julien a chance to look over JRuby stuff, etc.
Right now am trying to get a final list of what will go into the RC so we stop
stuffing the ballot box, as it were :)
> AvroStorage crashes on LOADING a single bad error
> -------------------------------------------------
>
> Key: PIG-2614
> URL: https://issues.apache.org/jira/browse/PIG-2614
> Project: Pig
> Issue Type: Bug
> Components: piggybank
> Affects Versions: 0.10, 0.11
> Reporter: Russell Jurney
> Labels: avro, avrostorage, bad, book, cutting, doug, for, my,
> pig, sadism
> Fix For: 0.10, 0.11
>
> Attachments: PIG-2614_0.patch
>
>
> AvroStorage dies when a single bad record exists, such as one with missing
> fields. This is very bad on 'big data,' where bad records are inevitable.
> See discussion at
> http://www.quora.com/Big-Data/In-Big-Data-ETL-how-many-records-are-an-acceptable-loss
> for more theory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
