Re: maven build failing in spark sql w/BouncyCastleProvider CNFE

[Yang,Jie(INF)]

I think we can try scala-maven-plugin 4.8.0

发件人: Steve Loughran 
日期: 2022年12月6日 星期二 18:19
收件人: "Yang,Jie(INF)" 
抄送: Hyukjin Kwon , Apache Spark Dev 
主题: Re: maven build failing in spark sql w/BouncyCastleProvider CNFE



On Tue, 6 Dec 2022 at 04:10, Yang,Jie(INF) 
mailto:yangji...@baidu.com>> wrote:
Steve, did compile failed happen when mvn build Spark master with hadoop 
3.4.0-SNAPSHOT?

yes. doesn't happen with
* branch-3.3 snapshot (3.3.9-SNAPSHOT)
* branch-3.3.5 RC0 "pre-rc" in asf staging.

maybe trying the 4.8.0 plugin would be worth trying...not something i'll do 
this week as i'm really trying to get the RC0 out rather than anything else


发件人: Hyukjin Kwon mailto:gurwls...@gmail.com>>
日期: 2022年12月6日 星期二 10:27
抄送: Apache Spark Dev mailto:dev@spark.apache.org>>
主题: Re: maven build failing in spark sql w/BouncyCastleProvider CNFE

Steve, does the lower version of scala plugin work for you? If that solves, we 
could temporary downgrade for now.

On Mon, 5 Dec 2022 at 22:23, Steve Loughran  wrote:
 trying to build spark master w/ hadoop trunk and the maven sbt plugin is 
failing. This doesn't happen with the 3.3.5 RC0;

I note that the only mention of this anywhere was me in march.

clearly something in hadoop trunk has changed in a way which is incompatible.

Has anyone else tried such a build/seen this problem? any suggestions of a fix?

Created SPARK-41392 to cover this...

[INFO] 
[ERROR] Failed to execute goal 
net.alchim31.maven:scala-maven-plugin:4.7.2:testCompile 
(scala-test-compile-first) on project spark-sql_2.12: Execution 
scala-test-compile-first of goal 
net.alchim31.maven:scala-maven-plugin:4.7.2:testCompile failed: A required 
class was missing while executing 
net.alchim31.maven:scala-maven-plugin:4.7.2:testCompile: 
org/bouncycastle/jce/provider/BouncyCastleProvider
[

Re: maven build failing in spark sql w/BouncyCastleProvider CNFE

[Steve Loughran]

On Tue, 6 Dec 2022 at 04:10, Yang,Jie(INF)  wrote:

> Steve, did compile failed happen when mvn build Spark master with hadoop
> 3.4.0-SNAPSHOT?
>

yes. doesn't happen with
* branch-3.3 snapshot (3.3.9-SNAPSHOT)
* branch-3.3.5 RC0 "pre-rc" in asf staging.

maybe trying the 4.8.0 plugin would be worth trying...not something i'll do
this week as i'm really trying to get the RC0 out rather than anything else


>
>
> *发件人**: *Hyukjin Kwon 
> *日期**: *2022年12月6日 星期二 10:27
> *抄送**: *Apache Spark Dev 
> *主题**: *Re: maven build failing in spark sql w/BouncyCastleProvider CNFE
>
>
>
> Steve, does the lower version of scala plugin work for you? If that
> solves, we could temporary downgrade for now.
>
>
>
> On Mon, 5 Dec 2022 at 22:23, Steve Loughran 
> wrote:
>
>  trying to build spark master w/ hadoop trunk and the maven sbt plugin is
> failing. This doesn't happen with the 3.3.5 RC0;
>
>
>
> I note that the only mention of this anywhere was me in march.
>
>
>
> clearly something in hadoop trunk has changed in a way which is
> incompatible.
>
>
>
> Has anyone else tried such a build/seen this problem? any suggestions of a
> fix?
>
>
>
> Created SPARK-41392 to cover this...
>
>
>
> [INFO]
> 
> [ERROR] Failed to execute goal
> net.alchim31.maven:scala-maven-plugin:4.7.2:testCompile
> (scala-test-compile-first) on project spark-sql_2.12: Execution
> scala-test-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:4.7.2:testCompile failed: A required
> class was missing while executing
> net.alchim31.maven:scala-maven-plugin:4.7.2:testCompile:
> org/bouncycastle/jce/provider/BouncyCastleProvider
> [
>
>