LuciferYang commented on PR #51144: URL: https://github.com/apache/spark/pull/51144#issuecomment-3074439496
> Hi folks @LuciferYang, @HyukjinKwon, @yaooqinn, and @dongjoon-hyun, I think that this commit broke the Maven build. If you attempt a full clean build (`./build/mvn -DskipTests clean package`), after of course invalidating all caches and deleting all temp files (for example .m2, intellij temp files, etc.) - then the build actually fails with a rather obscure error message: > > ``` > [INFO] ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] ------------------------------------------------------------------------ > [INFO] Total time: 4.733 s > [INFO] Finished at: 2025-07-15T18:43:39+02:00 > [INFO] ------------------------------------------------------------------------ > [ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.9.5:compile (scala-compile-first) on project spark-tags_2.13: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:4.9.5:compile failed: org.apache.commons.exec.ExecuteException: Process exited with an error: 255 (Exit value: 255) -> [Help 1] > [ERROR] > ``` > > This might prove to be difficult for regular developers to pick up, because some stuff may be already cached, and incremental compile through Maven may go through in some cases. However, if you attempt a fresh build - it will definitely fail, which we've confirmed on three separate machines and several fresh spark clones. Furthermore, note that the issue is not detected in CI, which is why PRs are seemingly uninterrupted. However, development experience is still severly impacted by this issue, as developers are not able to build spark using Maven (which is the official recommended way to build Spark) and run test suites locally through UI in some IDEs such as IntelliJ IDEA. > > I identified this as the offending commit through manual bisection and fresh Maven build attempts. Simply reverting the changes from this PR (#51144, corresponding to https://issues.apache.org/jira/browse/SPARK-52437) seems to resolve the issue at latest master. I'd like to confirm the steps to reproduce the issue: 1. Clone a fresh copy of the code repository; 2. Delete the `.m2` directory; 3. Execute `./build/mvn -DskipTests clean package` with Java 17 ?. Is that correct? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
