Hi Ismael, We have upgraded the spark to 2.4. In our setup we had run few basic tests and found it to be pretty stable.
*Thanks & Regards,* *Vishwas * On Fri, Dec 7, 2018 at 2:53 PM Ismaël Mejía <[email protected]> wrote: > Hello Vishwas, > > The spark dependency in the spark runner is provided so you can > already pass the dependencies of spark 2.4 and it should work out of > the box. > > JB did a PR to upgrade the version of Spark in the runner, but maybe > it is worth to wait a bit before merging it, at least until some of > the Big Data distributions has spark 2.4.x support available, so far > nobody has upgraded it (well apart of databricks). > > What do others think, should we move ahead or are you aware of any > issue introduced by version 2.4.0? (Notice that the PR just updates > the version so code compatibility should be ok). > > Ismaël > > On Thu, Dec 6, 2018 at 12:14 PM Jean-Baptiste Onofré <[email protected]> > wrote: > > > > Hi Vishwas > > > > Yes, I already started the update. > > > > Regards > > JB > > > > On 06/12/2018 07:39, Vishwas Bm wrote: > > > Hi, > > > > > > Currently I see that the spark version dependency used in Beam is > > > //"2.3.2". > > > As spark 2.4 is released now, is there a plan to upgrade Beam spark > > > dependency ? > > > > > > > > > *Thanks & Regards,* > > > *Vishwas > > > * > > > *Mob : 9164886653* > > > > -- > > Jean-Baptiste Onofré > > [email protected] > > http://blog.nanthrax.net > > Talend - http://www.talend.com >
