Endre Kovacs created NIFI-6288:
----------------------------------
Summary: character set encoding issue in FetchElasticsearchHttp
processor
Key: NIFI-6288
URL: https://issues.apache.org/jira/browse/NIFI-6288
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.9.2
Reporter: Endre Kovacs
Assignee: Endre Kovacs
I used FetchElasticsearchHttp processor to fetch documents in Elasticsearch
which have special UTF-8 chars, eg.: characters of foreign languages: accented
chars or Japanese/Chinese chars.
It was working as expected on platforms that have UTF-8 as a default
_file.encoding._ But on e.g.: SLES12 VM, the special chars in the document,
turned to "?" in the fetched, output flow files.
Taking a look at the source code showed:
- AbstractElasticsearchProcessor declares *CHARSET* property descriptor, but it
was not added to
AbstractElasticsearchHttpProcessor in the static initializer block.
- and in the place where the content of the document is written to the
flowfile, :
[https://github.com/apache/nifi/blob/65c41ab917d7b5f323aa71d841cc03b29e12d480/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/FetchElasticsearchHttp.java#L237]
it uses
{code}
out.write(source.toString().getBytes());
{code}
which will only work if the JVM's _file.encoding_ is UTF-8.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
