Re: ASF policy violation and Scala version issues

2023-06-07 Thread Grisha Weintraub
oundation/marks/downstream.html#source (ASF > Website) > > Dongjoon > > > On Wed, Jun 7, 2023 at 12:16 PM Grisha Weintraub < > grisha.weintr...@gmail.com> wrote: > >> Yes, in Spark UI you have it as "3.1.2-amazon", but when you create a >>

Re: ASF policy violation and Scala version issues

2023-06-07 Thread Grisha Weintraub
Yes, in Spark UI you have it as "3.1.2-amazon", but when you create a cluster it's just Spark 3.1.2. On Wed, Jun 7, 2023 at 10:05 PM Nan Zhu wrote: > > for EMR, I think they show 3.1.2-amazon in Spark UI, no? > > > On Wed, Jun 7, 2023 at 11:30 Grisha Weintraub >

Re: ASF policy violation and Scala version issues

2023-06-07 Thread Grisha Weintraub
Hi, I am not taking sides here, but just for fairness, I think it should be noted that AWS EMR does exactly the same thing. We choose the EMR version (e.g., 6.4.0) and it has an associated Spark version (e.g., 3.1.2). The Spark version here is not the original Apache version but AWS Spark