[
https://issues.apache.org/jira/browse/NIFI-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837291#comment-16837291
]
ASF subversion and git services commented on NIFI-6288:
-------------------------------------------------------
Commit dc38a6bdceec0095ae322f225401b3f505af1da5 in nifi's branch
refs/heads/master from Endre Zoltan Kovacs
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=dc38a6b ]
NIFI-6288: hooking in & using charset to HTTP based ES processors
This closes #3467
Signed-off-by: Mike Thomsen <[email protected]>
> character set encoding issue in FetchElasticsearchHttp processor
> ----------------------------------------------------------------
>
> Key: NIFI-6288
> URL: https://issues.apache.org/jira/browse/NIFI-6288
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.9.2
> Reporter: Endre Kovacs
> Assignee: Endre Kovacs
> Priority: Major
> Labels: easyfix
> Attachments: after-fix-encoded-message.png, bad-encoded-message.png,
> message-generator.png, simple-flow-overview.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
> which have special UTF-8 chars, eg.: characters of foreign languages:
> accented chars or Japanese/Chinese chars.
> It was working as expected on platforms that have UTF-8 as a default
> _file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
> turned to "?" in the fetched, output flow files.
>
> Taking a look at the source code showed:
> - AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but
> it was not added to
> AbstractElasticsearchHttpProcessor in the static initializer block.
> - and in the place where the content of the document is written to the
> flowfile, :
> [https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
> it uses
> {code}
> out.write(source.toString().getBytes());
> {code}
>
> which will only work if the JVM's _file.encoding_ is UTF-8.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
