Hi everyone, It's been almost a year since the last thread sent out [1]. As of current status, we have completed milestone switching CI to Java11 ( https://github.com/apache/beam/issues/31677), and the infrastructure that requiring certain Beam components to higher Java version has been setup ( https://github.com/apache/beam/pull/34858).
It's time to consider phase 2 in [1]. Here is my plan 1. Move Java 8 clients submitted pipeline will run on Java 11 container This would actually extend Java8 support and make the transirion smoothier. We already see some Beam's dependencies dropped Java8 (tnotable ones including Apache Avro 1.12, Apache Iceberg 1.7+, Apache Arrow 18+). This would allow Java8 users submitting pipelines that containing transient dependencies resolved to newer versions and do not need to pin them to older versions 2. Stop releasing apache/beam_java8_sdk container This would make custom image build requiring Java11+. 3. Upgrade buildSrc to require Java11, and update Gradle plugins originally blocked by Java11 adoption. For the next step Phase 3, I suggest it be added to Beam 3.0 milestone [2]. An implication is that certain components that do not support Java11 by the time of Beam 3.0 release have to be removed (see [3]). Current status: - HCatalogIO: upgrade should be feasible: https://github.com/apache/beam/pull/33750 if we drop support for Hadoop 2.x at the same time - Samza runner: call for volunteers to solve https://github.com/apache/beam/issues/32208. Otherwise, Samza runner will be effectively deprecated as part of Beam Java8 deprecation. Regards, [1] https://lists.apache.org/thread/w5lf2tg1nlk77so2p39cqphr1gqbh72s [2] https://github.com/apache/beam/milestone/31 [3] https://github.com/apache/beam/issues/31678#issuecomment-2344621740 -- Yi Hu, (he/him/his) Software Engineer
