[
https://issues.apache.org/jira/browse/NUTCH-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500935
]
Doğacan Güney commented on NUTCH-392:
-------------------------------------
Perhaps we can allow a user to configure this on a per-structure basis by
adding new properties: compression.type.{parse_text,crawldb,parse_data,linkdb}
or whatever. Then we can make such a property take one of the 4 valid values:
BLOCK, NONE, RECORD, DEFAULT where DEFAULT is the value of
io.sequence.file.compression.
> OutputFormat implementations should pass on Progressable
> --------------------------------------------------------
>
> Key: NUTCH-392
> URL: https://issues.apache.org/jira/browse/NUTCH-392
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher
> Reporter: Doug Cutting
> Assignee: Andrzej Bialecki
> Fix For: 1.0.0
>
> Attachments: NUTCH-392.patch
>
>
> OutputFormat implementations should pass the Progressable they are passed to
> underlying SequenceFile implementations. This will keep reduce tasks from
> timing out when block writes are slow. This issue depends on
> http://issues.apache.org/jira/browse/HADOOP-636.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
