On Fri, 02 Mar 2012, Thorsten S wrote: > however, why are individual bad input records not simply skipped *and > logged as such for later inspection* and the queue continues otherwise > unaffected? a single bad edit shouldn't stall everything afterwards, > unless there is a convincing reason that I am not aware of
Briefly put, upload jobs are usually run sequentially, since in general a future upload job may depend on the result of a previous upload job. So, if an upload job ends in an error, manual inspection and resolution may be required. Historically, in this case, the upload queue was stopped and an SMS alert was sent to the admin support person, who then logged in and inspected the problem and unblocked the queue. This `easy technical solution' works, but requires human intervention, which is definitely sub-optimal. We worked on `harder solution' that would require less human intervention and that is basically doing exactly what you pictured above. It is known under the term `quarantined records' facility for BibSched and BibUpload, see the detailed description at: <http://invenio-software.org/ticket/152> There was a good progress done at the time, but the facility is not ready yet, unfortunately. I've just bumped the ticket priority. P.S. There are more tickets touching the uploading comfort for INSPIRE operations, especially the smart uploader facility will come handy, see <http://invenio-software.org/ticket/816>. It is virtually finished, we are still tweaking with Vasanth the treatment of deleted fields a bit. P.S. CC-ing Invenio developers mailing list. Best regards -- Tibor Simko
