[ http://issues.apache.org/jira/browse/NUTCH-383?page=all ]
Andrzej Bialecki updated NUTCH-383:
------------------------------------
Attachment: patch-v2.txt
This patch uses Hadoop 0.7.1. New changes:
* upgrade to use Hadoop 0.7.1
* upgrade to use Lucene 2.0.0 jars
* use Hadoop's ToolBase instead of Nutch ToolBase
This patch also provides a limited backward-compatibility, namely:
* existing crawldb-s can be converted to new format using CrawlDbConverter tool
* existing segments can be partially converted, using SegmentMerger. However,
segment parts related to parsing will NOT be converted and have to be removed
prior to converting - i.e. crawl_parse, parse_data and parse_text needs to be
removed, and after conversion re-created with 'nutch parse'.
* other Nutch data, such as linkdb and indexes, need to be re-created.
> Upgrade Nutch to Hadoop 0.7
> ---------------------------
>
> Key: NUTCH-383
> URL: http://issues.apache.org/jira/browse/NUTCH-383
> Project: Nutch
> Issue Type: Improvement
> Affects Versions: 0.9.0
> Reporter: Andrzej Bialecki
> Assigned To: Andrzej Bialecki
> Attachments: patch-v2.txt, patch.txt
>
>
> Upgrade Nutch to Hadoop 0.7, and replace all occurences of UTF8 with Text.
> UTF8 is deprecated and its use is discouraged due to its limitations.
> This change will break API, in the sense that all third-party additions will
> have to be updated to use new APIs that use Text instead of UTF8 in method
> parameters.
> This change also breaks backward compatibility of data in CrawlDb, LinkDb and
> segments. A tool to upgrade CrawlDb, LinkDb and segments can be created to
> facilitate the upgrade path.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
