[
https://issues.apache.org/jira/browse/KAFKA-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17053077#comment-17053077
]
Weichu Liu edited comment on KAFKA-9669 at 3/6/20, 8:11 AM:
------------------------------------------------------------
-With 2.4.1 RC0, the Kafka does provider prettier error logs, but still cannot
produce:-
Update:
Sorry for the misleading, so I double checked the result. Despite for the ERROR
log, the messages were actually stored to the topic.
On the other hand, throwing one ERROR level message per produced record is
kinda not acceptable for production usage, because the Kafka log would flood
out the disk.
{noformat}
[2020-03-06 06:36:37,159] ERROR [ReplicaManager broker=0] Error processing
append operation on partition test-3-0 (kafka.server.ReplicaManager)
org.apache.kafka.common.InvalidRecordException: Inner record
LegacyRecordBatch(offset=0, Record(magic=1, attributes=0, compression=NONE,
crc=1478844555, CreateTime=1583476596257, key=0 bytes, value=199 bytes)) inside
the compressed record batch does not have incremental offsets, expected offset
is 1 in topic partition test-3-0.
[2020-03-06 06:36:37,177] ERROR [ReplicaManager broker=0] Error processing
append operation on partition test-3-0 (kafka.server.ReplicaManager)
org.apache.kafka.common.InvalidRecordException: Inner record
LegacyRecordBatch(offset=0, Record(magic=1, attributes=0, compression=NONE,
crc=4137384030, CreateTime=1583476596257, key=0 bytes, value=199 bytes)) inside
the compressed record batch does not have incremental offsets, expected offset
is 1 in topic partition test-3-0.
[2020-03-06 06:36:37,181] ERROR [ReplicaManager broker=0] Error processing
append operation on partition test-3-0 (kafka.server.ReplicaManager)
org.apache.kafka.common.InvalidRecordException: Inner record
LegacyRecordBatch(offset=0, Record(magic=1, attributes=0, compression=NONE,
crc=4137384030, CreateTime=1583476596257, key=0 bytes, value=199 bytes)) inside
the compressed record batch does not have incremental offsets, expected offset
is 1 in topic partition test-3-0.
[2020-03-06 06:36:37,194] ERROR [ReplicaManager broker=0] Error processing
append operation on partition test-3-0 (kafka.server.ReplicaManager)
org.apache.kafka.common.InvalidRecordException: Inner record
LegacyRecordBatch(offset=0, Record(magic=1, attributes=0, compression=NONE,
crc=3885590418, CreateTime=1583476596257, key=0 bytes, value=199 bytes)) inside
the compressed record batch does not have incremental offsets, expected offset
is 1 in topic partition test-3-0.
{noformat}
was (Author: weichu):
With 2.4.1 RC0, the Kafka does provider prettier error logs, but still cannot
produce:
{noformat}
[2020-03-06 06:36:37,159] ERROR [ReplicaManager broker=0] Error processing
append operation on partition test-3-0 (kafka.server.ReplicaManager)
org.apache.kafka.common.InvalidRecordException: Inner record
LegacyRecordBatch(offset=0, Record(magic=1, attributes=0, compression=NONE,
crc=1478844555, CreateTime=1583476596257, key=0 bytes, value=199 bytes)) inside
the compressed record batch does not have incremental offsets, expected offset
is 1 in topic partition test-3-0.
[2020-03-06 06:36:37,177] ERROR [ReplicaManager broker=0] Error processing
append operation on partition test-3-0 (kafka.server.ReplicaManager)
org.apache.kafka.common.InvalidRecordException: Inner record
LegacyRecordBatch(offset=0, Record(magic=1, attributes=0, compression=NONE,
crc=4137384030, CreateTime=1583476596257, key=0 bytes, value=199 bytes)) inside
the compressed record batch does not have incremental offsets, expected offset
is 1 in topic partition test-3-0.
[2020-03-06 06:36:37,181] ERROR [ReplicaManager broker=0] Error processing
append operation on partition test-3-0 (kafka.server.ReplicaManager)
org.apache.kafka.common.InvalidRecordException: Inner record
LegacyRecordBatch(offset=0, Record(magic=1, attributes=0, compression=NONE,
crc=4137384030, CreateTime=1583476596257, key=0 bytes, value=199 bytes)) inside
the compressed record batch does not have incremental offsets, expected offset
is 1 in topic partition test-3-0.
[2020-03-06 06:36:37,194] ERROR [ReplicaManager broker=0] Error processing
append operation on partition test-3-0 (kafka.server.ReplicaManager)
org.apache.kafka.common.InvalidRecordException: Inner record
LegacyRecordBatch(offset=0, Record(magic=1, attributes=0, compression=NONE,
crc=3885590418, CreateTime=1583476596257, key=0 bytes, value=199 bytes)) inside
the compressed record batch does not have incremental offsets, expected offset
is 1 in topic partition test-3-0.
{noformat}
> Kafka 2.4.0 Chokes on Filebeat 5.6 Produced Data
> ------------------------------------------------
>
> Key: KAFKA-9669
> URL: https://issues.apache.org/jira/browse/KAFKA-9669
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 2.4.0
> Reporter: Weichu Liu
> Priority: Major
>
> Hi
> In our environment, after upgrading to Kafka 2.4.0, we discovered the broker
> was not compatible with filebeat 5.
> Here is how to reproduce:
> 1. Startup Kafka 2.4.0, all configurations are vanilla:
> {code}
> $ kafka_2.13-2.4.0/bin/zookeeper-server-start.sh
> kafka_2.13-2.4.0/config/zookeeper.properties
> $ kafka_2.13-2.4.0/bin/kafka-server-start.sh
> kafka_2.13-2.4.0/config/server.properties
> {code}
> 2. Startup filebeat 5.6.16 with the following configuration. (downloaded from
> https://www.elastic.co/jp/downloads/past-releases/filebeat-5-6-16)
> {code}
> $ cat /tmp/filebeat.yml
> name: test
> output.kafka:
> enabled: true
> hosts:
> - localhost:9092
> topic: test-3
> version: 0.10.0
> compression: gzip
> filebeat:
> prospectors:
> - input_type: log
> paths:
> - /tmp/filebeat-in
> encoding: plain
> {code}
> {code}
> $ filebeat-5.6.16-linux-x86_64/filebeat -e -c /tmp/filebeat.yml
> {code}
> 3. Write some lines to file {{/tmp/filebeat-in}}. Looks like single line
> won't trigger the issue, but 30 lines are enough.
> {code}
> seq 30 >> /tmp/filebeat-in
> {code}
> 4. Kafka throws the following error chunk, like, per produced record.
> {noformat}
> [2020-03-06 05:17:40,129] ERROR [ReplicaManager broker=0] Error processing
> append operation on partition test-3-0 (kafka.server.ReplicaManager)
> org.apache.kafka.common.InvalidRecordException: Inner record
> LegacyRecordBatch(offset=0, Record(magic=1, attributes=0, compression=NONE,
> crc=1453875406, CreateTime=1583471854475, key=0 bytes, value=202 bytes))
> inside the compressed record batch does not have incremental offsets,
> expected offset is 1 in topic partition test-3-0.
> [2020-03-06 05:17:40,129] ERROR [KafkaApi-0] Error when handling request:
> clientId=beats, correlationId=102, api=PRODUCE, version=2,
> body={acks=1,timeout=10000,partitionSizes=[test-3-0=272]}
> (kafka.server.KafkaApis)
> java.lang.NullPointerException: `field` must be non-null
> at java.base/java.util.Objects.requireNonNull(Objects.java:246)
> at
> org.apache.kafka.common.protocol.types.Struct.validateField(Struct.java:474)
> at
> org.apache.kafka.common.protocol.types.Struct.instance(Struct.java:418)
> at
> org.apache.kafka.common.protocol.types.Struct.instance(Struct.java:436)
> at
> org.apache.kafka.common.requests.ProduceResponse.toStruct(ProduceResponse.java:281)
> at
> org.apache.kafka.common.requests.AbstractResponse.toSend(AbstractResponse.java:35)
> at
> org.apache.kafka.common.requests.RequestContext.buildResponse(RequestContext.java:80)
> at kafka.server.KafkaApis.sendResponse(KafkaApis.scala:2892)
> at kafka.server.KafkaApis.sendResponseCallback$2(KafkaApis.scala:554)
> at
> kafka.server.KafkaApis.$anonfun$handleProduceRequest$11(KafkaApis.scala:576)
> at
> kafka.server.KafkaApis.$anonfun$handleProduceRequest$11$adapted(KafkaApis.scala:576)
> at kafka.server.ReplicaManager.appendRecords(ReplicaManager.scala:546)
> at kafka.server.KafkaApis.handleProduceRequest(KafkaApis.scala:577)
> at kafka.server.KafkaApis.handle(KafkaApis.scala:126)
> at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:70)
> at java.base/java.lang.Thread.run(Thread.java:835)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
