[
https://issues.apache.org/jira/browse/EAGLE-286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261349#comment-15261349
]
Edward Zhang commented on EAGLE-286:
------------------------------------
In PartitionManager, seems it directly acks this message if no tuples are
generated from this message
public EmitState next(SpoutOutputCollector collector) {
if (_waitingToEmit.isEmpty()) {
fill();
}
while (true) {
MessageAndRealOffset toEmit = _waitingToEmit.pollFirst();
if (toEmit == null) {
return EmitState.NO_EMITTED;
}
Iterable<List<Object>> tups =
KafkaUtils.generateTuples(_spoutConfig, toEmit.msg);
if (tups != null) {
for (List<Object> tup : tups) {
collector.emit(tup, new KafkaMessageId(_partition,
toEmit.offset));
}
break;
} else {
ack(toEmit.offset);
}
}
if (!_waitingToEmit.isEmpty()) {
return EmitState.EMITTED_MORE_LEFT;
} else {
return EmitState.EMITTED_END;
}
}
> Failed to deserialize json
> --------------------------
>
> Key: EAGLE-286
> URL: https://issues.apache.org/jira/browse/EAGLE-286
> Project: Eagle
> Issue Type: Bug
> Affects Versions: v0.3.0
> Environment: Hortonworks Sandbox 2.2.4
> Reporter: Huizhi Lu
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When Eagle detects one failed message such as invalid json, it may not ack
> the message back to Kafka. So Kafka offset may not be updated to the latest
> one. This would become a big problem when more and more message are coming.
> Please see the following invalid json message. It is invalid because there is
> a special character (space) before "timestamp".
> After the special character is deleted, it is good.
> NOP [ERROR] Failed to deserialize json from: {"host":
> "/192.168.6.227","source": "/192.168.6.227","user": "root", "timestamp":
> 1455574202866}
> om.fasterxml.jackson.core.JsonParseException: Unexpected character ('Â' (code
> 194)): was expecting double-quote to start field name
> at [Source: [B@49a298b0; line: 1, column: 70]
> at
> com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1369)
> ~[stormjar.jar:na]
> at
> com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:532)
> ~[stormjar.jar:na]
> at
> com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:453)
> ~[stormjar.jar:na]
> at
> com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnusualFieldName(UTF8StreamJsonParser.java:1720)
> ~[stormjar.jar:na]
> at
> com.fasterxml.jackson.core.json.UTF8StreamJsonParser._parseFieldName(UTF8StreamJsonParser.java:1410)
> ~[stormjar.jar:na]
> at
> com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:668)
> ~[stormjar.jar:na]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
