[
https://issues.apache.org/jira/browse/FLUME-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Paliwal reassigned FLUME-2476:
-------------------------------------
Assignee: Ashish Paliwal
> ContentBuilderUtil.appendField incorrectly manages json-like data
> -----------------------------------------------------------------
>
> Key: FLUME-2476
> URL: https://issues.apache.org/jira/browse/FLUME-2476
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.5.0.1
> Environment: elasticsearch 1.1.0, elasticsearch 0.90.13
> Reporter: Pavel Zalunin
> Assignee: Ashish Paliwal
> Priority: Blocker
> Fix For: v1.6.0
>
>
> There is a problem in
> org.apache.flume.sink.elasticsearch.ElasticSearchDynamicSerializer.getContentBuilder
> returns incorrect value, in case when Event contains well-formed json
> (possible as sub-string).
> Example:
> {code}
> ElasticSearchDynamicSerializer ser = new ElasticSearchDynamicSerializer();
> Event event = new SimpleEvent();
> event.setBody("{\"true\":\"false\"}".getBytes());
> System.out.println(ser.getContentBuilder(event).string());
> //prints:
> //{"body":"org.elasticsearch.common.xcontent.XContentBuilder@31a5fdb9"}
> {code}
> I tried to find origins of the problem, and found this chunk of code
> (flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ContentBuilderUtil.java):
> {code}
> public static void addComplexField(XContentBuilder builder, String fieldName,
> XContentType contentType, byte[] data) throws IOException {
> XContentParser parser = null;
> try {
> XContentBuilder tmp = jsonBuilder();
> parser = XContentFactory.xContent(contentType).createParser(data);
> parser.nextToken();
> tmp.copyCurrentStructure(parser);
> builder.field(fieldName, tmp); //here field (String, String) called,
> because there is no method(String,XContentBuilder)
> //maybe tmp.string() should be here instead?
> {code}
> It makes impossible to send any string, which contains json to elasticsearch
> sink using this serializer.
> Maybe I'm wrong, but what are benefits from decoding, then encoding chunk of
> json data?
> If it really needed, maybe it is possible to add some option to
> ElasticSearchDynamicSerializer which forces to use
> ContentBuilderUtil.addSimpleField instead of ContentBuilderUtil.appendField ?
> I can volunteer to create patch when will be decided what is the best way to
> avoid this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
