Hello Vishwas, The spark dependency in the spark runner is provided so you can already pass the dependencies of spark 2.4 and it should work out of the box.
JB did a PR to upgrade the version of Spark in the runner, but maybe it is worth to wait a bit before merging it, at least until some of the Big Data distributions has spark 2.4.x support available, so far nobody has upgraded it (well apart of databricks). What do others think, should we move ahead or are you aware of any issue introduced by version 2.4.0? (Notice that the PR just updates the version so code compatibility should be ok). Ismaël On Thu, Dec 6, 2018 at 12:14 PM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > > Hi Vishwas > > Yes, I already started the update. > > Regards > JB > > On 06/12/2018 07:39, Vishwas Bm wrote: > > Hi, > > > > Currently I see that the spark version dependency used in Beam is > > //"2.3.2". > > As spark 2.4 is released now, is there a plan to upgrade Beam spark > > dependency ? > > > > > > *Thanks & Regards,* > > *Vishwas > > * > > *Mob : 9164886653* > > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com
