Hi Gary, Beam 2.3.0 and the Spark runner use Scala 2.11.
I can help you to have a smooth transition by creating a local branch using Scala 2.10. However, I strongly advice to upgrade to 2.11 as some other part of Beam (other runners and IOs) use 2.11 already. Regards JB On 02/22/2018 05:55 PM, Gary Dusbabek wrote: > Hi, > > My apologies if this belongs on the dev list. If it does, let me know and I'll > shoot things over that way... > > For the last day or so, I've been trying to create a Spark Runner that will > work > on older deployments using Scala 2.10. I've taken a few approaches: > > 1. selectively changing a few dependencies in beam-runners-spark.pom (and a > few > other places in the parent) > 2. updating every dependency that references *_2.11 to be *_2.10 > > In the former case the sticking point in both cases is that there is a library > incompatibility with jackson-module-scala_2.xx. In the latter case there is a > problem with SourceRDD.SourcePartitioning not [correctly] implementing > `equals(...)` from the parent trait. > > Posts on the mailing list made me think that the move to Scala 2.11 started > only > last fall, so I figured it should be easy to make the switch back. > > However, I have a feeling that it could be the case that I just don't > understand > the Beam build system well enough to produce the right outcome (a custom > version > that can be used with older Scala). > > Is there a correct or better way of achieving this? > > Kind Regards, > > Gary Dusbabek -- Jean-Baptiste Onofré [email protected] http://blog.nanthrax.net Talend - http://www.talend.com
