[
https://issues.apache.org/jira/browse/NUTCH-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giuseppe Totaro updated NUTCH-1998:
-----------------------------------
Attachment: NUTCH-1998.patch
[~Lukeliush], you can find in attachment the patch to add a user-defined
extension to output documents. For example, you can use the
CommonCrawlDataDumper as follows:
{noformat}
bin/nutch commoncrawldump -segment /path/to/data/segments -outputDir
/path/to/output -SimpleDateFormat -epochFilename -jsonArray -reverseKey
-extension cbor
{noformat}
The {{-extension}} command-line option allows to use the given file extension
as extension of each output document. This patch does not include
[NUTCH-1997|https://issues.apache.org/jira/browse/NUTCH-1997].
Thanks a lot for supporting me on this work.
> Add support for user-defined file extension to CommonCrawlDataDumper
> --------------------------------------------------------------------
>
> Key: NUTCH-1998
> URL: https://issues.apache.org/jira/browse/NUTCH-1998
> Project: Nutch
> Issue Type: Improvement
> Components: tool
> Reporter: Giuseppe Totaro
> Priority: Minor
> Attachments: NUTCH-1998.patch
>
>
> {{CommonCrawlDataDumper}} tool is able to generate CBOR-encoded files,
> extracted from Nutch crawled data, using the Common Crawl format. By default,
> {{CommonCrawlDataDumper}} uses the original file extension.
> We are going to add support for a command-line option (e.g., {{-extension}})
> that allows the user to provide a file extension to use in place of the
> original one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
