Hi Jia, I think we should support spark 2.4, a lot of users still use it. More than that I think more users still have jobs written in spark 2.4 than 3.0. We will use an additional branch for that use case ? I mean Spark 2.4 with scala 2.12 is important one. Regards, Paweł
pon., 9 lis 2020 o 20:44 Jia Yu <ji...@apache.org> napisał(a): > Dear all, > > In Sedona 1.0, we definitely will support Spark 3.0. But I wonder whether > we should support Spark 2.4. > > In order to support Spark 2.4, we need to do the following > > 1. Compile the source using Scala 2.11. Sedona master branch currently is > compiled by Scala 2.12 and Java 1.8 > 2. For the Scala code of Sedona-SQL and Viz-SQL, I need to change the (1) > UDF registration hook (2) the SQL aggregation function format > 3. In the future releases of Sedona, use git cherry-pick to pick important > features back to the Spark 2.4 branch. This is what I did in GeoSpark to > support Spark 2.1, 2.2, 2.3 > > GeoSpark 1.2.0 - 1.3.1 support Spark 2.4 already. We can simply leave it > that way and just support Spark 3.0. > > Do you think we should support Spark 2.4 in the future release? > > Thanks, > Jia Yu >
