[
https://issues.apache.org/jira/browse/KAFKA-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474456#comment-13474456
]
John Fung commented on KAFKA-567:
---------------------------------
Uploaded mirror_maker_12.log for a similar test but with the following changes:
11. After Mirror Maker is bounced and wait 15 seconds, Producer (sending data
in the source cluster) is stopped. Then wait for 60 seconds more to make sure
that Mirror Maker is able to catch up.
12. kafka.consumer.ConsumerIterator is configured to be at TRACE level in
log4j.properties
13. kafka.producer.Producer is also configured to be at TRACE level.
14. The attached mirror maker log file shows that there are 400 messages each
showing in the TRACE level messages which is matching the Producer data count
but Target consumer is still missing some messages.
> Replication Data Loss in Mirror Maker Bouncing testcase
> -------------------------------------------------------
>
> Key: KAFKA-567
> URL: https://issues.apache.org/jira/browse/KAFKA-567
> Project: Kafka
> Issue Type: Bug
> Reporter: John Fung
> Attachments: kafka-mirror-maker-bouncing-data-loss.patch,
> mirror_maker_12.log, system_test_1349971807_testcase_5002.tar
>
>
> * Test Description:
> 1. Start a 3-broker cluster as source
> 2. Start a 3-broker cluster as target
> 3. Start 1 instance of Mirror Maker to replicate data from source to target
> 4. While producer is sending data into source cluster, stop Mirror Maker with
> "kill -15". Start Mirror Maker again after 1 second.
> 5. Start a consumer to consume data from target cluster.
> 6. Compare the MessageID in the data between producer log and consumer log.
> * To reproduce this issue, please do the followings:
> 1. Download the latest 0.8 branch
> 2. Apply the patch attached to this JIRA
> 3. Build kafka by running "./sbt update package"
> 4. Execute the test in directory "system_test" : "python -B
> system_test_runner.py"
> * The test result may look like the following:
> _test_case_name : testcase_5002
> _test_class_name : MirrorMakerTest
> arg : bounce_leader : false
> arg : bounce_mirror_maker : true
> arg : message_producing_free_time_sec : 15
> arg : num_iteration : 1
> arg : num_messages_to_produce_per_producer_call : 50
> arg : num_partition : 1
> arg : replica_factor : 3
> arg : sleep_seconds_between_producer_calls : 1
> validation_status :
> Log segment checksum matching across all replicas : FAILED
> Unique messages from consumer on [test_1] : 355
> Unique messages from producer on [test_1] : 400
> Validate for data matched on topic [test_1] : FAILED
> * Attached a tar file for the system test output log, the brokers' log4j
> files and data log segment files.
> * There are no unusual Exception / Error found in the logs. However, there
> are consistently data loss in this Mirror Maker bouncing test case. Not sure
> if this is related to KAFKA-552.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
