[GitHub] [spark] dongjoon-hyun commented on pull request #36517: Revert "[SPARK-36837][BUILD] Upgrade Kafka to 3.1.0"
dongjoon-hyun commented on PR #36517: URL: https://github.com/apache/spark/pull/36517#issuecomment-1125525626 For master branch, we are starting to pursue Apache Kafka 3.2.x in Apache Spark 3.4 timeframe. - https://github.com/apache/spark/pull/36526 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #36517: Revert "[SPARK-36837][BUILD] Upgrade Kafka to 3.1.0"
dongjoon-hyun commented on PR #36517: URL: https://github.com/apache/spark/pull/36517#issuecomment-1125272185 Thank you, @HeartSaVioR and all. - I've been using Kafka 3.1.0 internally to consume the latest updates and there was no serious issue from our side with our setup so far. On top of that, we are going to use Apache Kafka 3.1.1 (with RC1) and catch up 3.2.0 internally in separate Spark versions because we can adapt easily internally if something happens. - Apache Spark community has enough discussions from the initial Kafka 3.1 PR and on the mailing list (Thanks to you). I have been agreed all colleagues' valid concerns and grateful to have this trial together. This reverting has been also one of options which was proposed by me from the beginning. - This PR adopts all your concerns and tries to choose the best combination of AS-OF-TODAY in terms of the new features, stability and maintainability not only Apache Spark 3.3.0, but also upcoming maintenance versions like Apache Spark 3.3.1, 3.3.2, and 3.3.4. - AFAIK, the latest Kafka 3.2 is worth of re-trying in `master` branch (Apache Spark 3.4 timeframe) because Apache Kafka community is really moving fast toward. Merged to branch-3.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #36517: Revert "[SPARK-36837][BUILD] Upgrade Kafka to 3.1.0"
dongjoon-hyun commented on PR #36517: URL: https://github.com/apache/spark/pull/36517#issuecomment-1124595674 Thank you, @HyukjinKwon , @viirya , @martin-g , @huaxingao , @sunchao . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #36517: Revert "[SPARK-36837][BUILD] Upgrade Kafka to 3.1.0"
dongjoon-hyun commented on PR #36517: URL: https://github.com/apache/spark/pull/36517#issuecomment-1124546435 cc @viirya , @sunchao , @huaxingao , @srowen , @HyukjinKwon , @MaxGekk , @HeartSaVioR for the discussion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
