[
https://issues.apache.org/jira/browse/OODT-150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13001516#comment-13001516
]
Chris A. Mattmann commented on OODT-150:
----------------------------------------
Hi Brian:
bq. just register a checksummer to a mime-type or whatnot and the crawler would
automatically determine the checksum for the file
Can't you already do this with the MimeType action, that calls a checksummer?
bq. i need the crawler stuff to be instant (milliseconds) – using the workflow
manager to do checksumming is going to slow down the crawler
Maybe we could actually benchmark this before we claim that it will
unequivocally slow it down without having any hard # to say how much or why.
bq. and i had a bunch of checksum stuff in the crawler2 initially (MD5, PGP, .
. .) – you make me remove them because of you wanted to keep the dependency
tree small
Keeping the dependency tree small was one part of it. The main part was that
all of your actions had a bunch of PEATE specific nomenclature in them (e.g.,
the policy to configure them, including environment variable refs, etc.). That
was the main thing I objected to.
bq. i still have them on PEATE if you want them . . . i also have an example on
PEATE which uses the new ExternAction to perform a checksum.
Yah, probably some baked in example config with the ExternAction would probably
make a lot of sense. Let's think about doing that and what a patch would look
like.
> ToggleAction addition to crawler
> --------------------------------
>
> Key: OODT-150
> URL: https://issues.apache.org/jira/browse/OODT-150
> Project: OODT
> Issue Type: New Feature
> Components: crawler
> Affects Versions: 0.2
> Environment: none
> Reporter: Brian Foster
> Assignee: Brian Foster
> Priority: Minor
> Fix For: 0.3
>
> Attachments: crawler.OODT-150.2011-03-01.patch.txt
>
>
> Allows for cases where there are one of two (or more) actions which needs to
> run. For example: sometimes the same file types have different checksum files
> associated with it, this allows for either case.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
