rymarm opened a new pull request #2456: URL: https://github.com/apache/drill/pull/2456
# [DRILL-8122](https://issues.apache.org/jira/browse/DRILL-8122): Change kafka metadata obtaining due to KAFKA-5697 ## Description [`Consumer#poll(long)`](https://javadoc.io/static/org.apache.kafka/kafka-clients/3.1.0/org/apache/kafka/clients/consumer/Consumer.html#poll-long-) is deprecated starting from kafka 2.0. In Drill, `Consumer#poll` is used in 2 places: 1. [By its direct purpose ](https://github.com/apache/drill/blob/15b2f52260e4f0026f2dfafa23c5d32e0fb66502/contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/MessageIterator.java#L82) 2. As an only one way to make a Kafka consumer [update metadata ](https://github.com/apache/drill/blob/15b2f52260e4f0026f2dfafa23c5d32e0fb66502/contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaGroupScan.java#L185) Kafka [hasn't implemented](https://cwiki.apache.org/confluence/display/KAFKA/KIP-505%3A+Add+new+public+method+to+only+update+assignment+metadata+in+consumer) a separate method to update metadata. And new implementation [Consumer#poll(Duration)](https://javadoc.io/static/org.apache.kafka/kafka-clients/3.1.0/org/apache/kafka/clients/consumer/Consumer.html#poll-java.time.Duration-) doesn't work with a hack that Drill use: `poll(0)`, due to changed logic: https://github.com/apache/kafka/pull/4855 . That is why I had to use a loop with a timeout to workaround the absent separate method. ## Documentation \- ## Testing Unit tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
