Re: Offline elastic index creation

2022-11-10 Thread Debasish Das
Hi Vibhor, We worked on a project to create lucene indexes using spark but the project has not been managed for some time now. If there is interest we can resurrect it

Spark Scala Contract Opportunity @USA

2022-11-10 Thread sri hari kali charan Tummala
Hi All, Is anyone looking for a spark scala contract role inside the USA? A company called Maxonic has an open spark scala contract position (100% remote) inside the USA if anyone is interested, please send your CV to kali.tumm...@gmail.com. Thanks & Regards Sri Tummala

Re: [Spark Core] Adaptive dynamic partition pruning

2022-11-10 Thread Jie Han
Which version are you using? I test it in spark 3.2.1 and sure that dynamic pruning works in queries with multi joins. BTW, could you execute ‘explain extended your sql’? > 2022年11月10日 02:10,hajyoussef amine 写道: > > Hello everyone, > > Let me take the following spark sql example to

Re: cannot write spark log to s3a

2022-11-10 Thread Chris Nauroth
The Spark event log writer expects to use a Hadoop-compatible file system that supports the ability to sync [1] previously written data, immediately making it durable and visible to other clients. The log message is warning that the S3A file system does not provide this capability. Even if it's

Re: Spark Scala Contract Opportunity @USA

2022-11-10 Thread Stephen Boesch
Please do not send advertisements on this channel. On Thu, 10 Nov 2022 at 13:40, sri hari kali charan Tummala < kali.tumm...@gmail.com> wrote: > Hi All, > > Is anyone looking for a spark scala contract role inside the USA? A > company called Maxonic has an open spark scala contract position