sypeng created ROCKETMQ-329:
-------------------------------
Summary: Some messages was skipped by consumerOffset when rocketMQ
consuming a large number of messages.
Key: ROCKETMQ-329
URL: https://issues.apache.org/jira/browse/ROCKETMQ-329
Project: Apache RocketMQ
Issue Type: Bug
Components: rocketmq-broker, rocketmq-client
Affects Versions: 4.1.0-incubating
Environment: centos 7
one mqnamesvr, one mqbroker, with default settings.
Reporter: sypeng
Assignee: yukon
Attachments: 0A3A66281DB342EB89871F8DE01865DB.jpg,
343007284451C411C424FABCC52F352B.jpg, 85C2383276ECDDD9C573913186B746C8.jpg
I started one mqnamesrv and one mqbroker process on centos7 with default
settings:
{{nohup bin/mqname &
nohup bin/mqbroker -n localhost:9876 &}}
and produced a large number of messages with 5 threads continuously(total
1000000 messages)，found that each consumerOffset skipped about 4500 messages
when persist (as shown in the attached picture), which cause them lost while
consumer consuming. The consumer lost about 16000 messages eventually.
The speed of message production is about 5000 per second, and the speed of
message consumption is 5 per second (the consumer's logic is sleep 200
millseconds and return consume success.)
The info "the message queue not be able to consume, because it's dropped" and
the warning "the pull request offset illegal" are printed in the mq's log when
this condition happened. (as shown in the attached picture)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
