curcur commented on pull request #11725: URL: https://github.com/apache/flink/pull/11725#issuecomment-621721146
Updates to commit "Kafka Shuffle and Test Case" 1. add log4j2 file for better testing 2. I found the original tests are occasionally failing when testing for a large number of data. The reason is the default property "auto.offset.reset" is set to "latest". That means if the consumer starts later than the producer starting to writing data, then I am going to lose some data occasionally. So, I switched to use KafkaTestEnvironment.getStandardProperties which set the auto.offset.reset = earliest ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
