[ https://issues.apache.org/jira/browse/NUTCH-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebastian Nagel resolved NUTCH-2254. ------------------------------------ Resolution: Fixed Committed, r6d2bfa9. Thanks, [~fedechicco]! > Charset issues when using -addBinaryContent and -base64 options > --------------------------------------------------------------- > > Key: NUTCH-2254 > URL: https://issues.apache.org/jira/browse/NUTCH-2254 > Project: Nutch > Issue Type: Bug > Components: indexer > Affects Versions: 1.11 > Reporter: Federico Bonelli > Assignee: Sebastian Nagel > Priority: Minor > Attachments: base64-nutch.patch > > > The bug is reproducible with these steps: > # find a site with cp1252 encoded pages like "http://www.ilsole24ore.com/" > and characters with accents (byte representation >127, like [àèéìòù]) > # start a crawl on that site indexing on Solr with options -addBinaryContent > -base64 > # find a document inside the newly indexed Solr collection with those > accented characters > # get the base64 binary representation for said html page and decode it back > to raw binary, save it > The file obtained will have invalid characters, which are neither UTF-8 nor > cp1252. -- This message was sent by Atlassian JIRA (v6.3.4#6332)