gortiz commented on issue #15343: URL: https://github.com/apache/pinot/issues/15343#issuecomment-3744510338
> We cannot upgrade a lot of dependencies as more and more of them start dropping support for older java versions. Which dependencies are we talking about? The most problematic one I know is Apache Datasketches, which controversially decided to require Java 25 as a runtime. > I still don't quite understand what is changing in practice. We're saying this won't affect drivers/connectors, but also drivers/connectors depend on common code that would be getting compiled with Java 21. I understand your confusion. Right now, we have the following logical modules: 1. Pinot processes (servers, brokers, controllers & minions) 2. Pinot drivers (mainly the Java driver) 3. External connectors (plugins that are installed on external projects like Flink or Spark so they can connect to Pinot) 4. Pinot internal libraries. Most of them are used only by Pinot processes, but some, like Pinot common. Today, all modules are built using Java 11. But that wasn't always the case. Until Pinot 1.0, the drivers and connectors were built using Java 8, while the rest of Pinot was using Java 11. How did we do that? In a hackish way. We have a GHA pipeline that compiled Pinot processes, including `pinot-common`, using Java 11 while another compiled the drivers, including `pinot-common`, using Java 8. That means we were forced to write `pinot-common` using Java 8 features. What I propose is similar, but using a tool we didn't have at the time: [Maven Toolchains](https://www.baeldung.com/maven-toolchains). This opens the door to compile the whole Pinot repository with a single Maven command, even if we use different JDKs for each project. Specifically: 1. Pinot drivers, connectors and their dependencies (including `pinot-common`) should compile with Java 8. 2. Pinot processes and the rest of the dependencies should compile with Java 21. Eventually, we should try to isolate the driver and connectors to avoid using `pinot-common`, which will open the door to moving all Pinot backend code to Java 21 (or even higher). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
