[
https://issues.apache.org/jira/browse/OODT-931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365292#comment-15365292
]
Chris A. Mattmann commented on OODT-931:
----------------------------------------
tested in both workflow crawling for processing and stand-alone ingest crawling:
h2. Ingest crawling
{noformat}
INFO: Crawling /usr/local/bigtranslate/data/staging
Jul 06, 2016 3:30:05 PM org.apache.oodt.cas.crawl.ProductCrawler handleFile
INFO: Handling file
/usr/local/bigtranslate/data/staging/computrabajo-ar-20121106.tsv
Jul 06, 2016 3:30:05 PM
org.apache.oodt.cas.metadata.preconditions.PreCondEvalUtils eval
{noformat}
h2. Processing crawling
{noformat}
INFO: Handling file
/usr/local/bigtranslate/data/jobs/splitter/computrabajo-ar-20121106.tsv_1467844206549/output/computrabajo-ar-20121106.tsv.aaaa.met
Jul 06, 2016 3:30:07 PM org.apache.oodt.cas.crawl.StdProductCrawler
passesPreconditions
INFO: Failed precondition comparator id MetFileExistsCheck file:
/usr/local/bigtranslate/data/jobs/splitter/computrabajo-ar-20121106.tsv_1467844206549/output/computrabajo-ar-20121106.tsv.aaaa.met\
.met does not exist.
Jul 06, 2016 3:30:07 PM org.apache.oodt.cas.crawl.ProductCrawler handleFile
WARNING: Failed to pass preconditions for ingest of product:
[/usr/local/bigtranslate/data/jobs/splitter/computrabajo-ar-20121106.tsv_1467844206549/output/computrabajo-ar-20121106.tsv.aaaa.met]
Jul 06, 2016 3:30:07 PM org.apache.oodt.cas.crawl.ProductCrawler handleFile
{noformat}
Going to submit the code shortly.
> CAS Crawler and metadata preconditions should say what preconditions failed
> ---------------------------------------------------------------------------
>
> Key: OODT-931
> URL: https://issues.apache.org/jira/browse/OODT-931
> Project: OODT
> Issue Type: Bug
> Components: crawler, metadata container
> Reporter: Chris A. Mattmann
> Assignee: Chris A. Mattmann
> Priority: Critical
> Fix For: 1.1
>
>
> It's immensely frustrating to receive messages such as:
> {noformat}
> INFO: Crawling
> /usr/local/bigtranslate/data/jobs/splitter/computrabajo-ar-20121106.tsv_1467043421232/output
> Jun 27, 2016 9:03:41 AM org.apache.oodt.cas.crawl.ProductCrawler handleFile
> INFO: Handling file
> /usr/local/bigtranslate/data/jobs/splitter/computrabajo-ar-20121106.tsv_1467043421232/output/computrabajo-ar-20121106.tsv.aaaa
> Jun 27, 2016 9:03:41 AM org.apache.oodt.cas.crawl.ProductCrawler handleFile
> WARNING: Failed to pass preconditions for ingest of product:
> [/usr/local/bigtranslate/data/jobs/splitter/computrabajo-ar-20121106.tsv_1467043421232/output/computrabajo-ar-20121106.tsv.aaaa]
> {noformat}
> Which indicate that preconditions failed while checking them during crawling.
> But, which ones failed??! We should report that during crawling.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
