Noticed on the now released 1.5.0, sedona-common depends on it.geosolutions.jaiext.jiffle:jt-jiffle-language:1.1.24 which is only in the osgeo maven repo, so just trying to use pyspark --packages org.apache.sedona:sedona-spark-3.4_2.12:1.5.0 will fail. I know there are workarounds (add additional repositories, use the shaded jar, use excludes for maven builds), but does it make sense to not include things not in maven central by default? Should that dependency also have the geotools scope (and maybe be included in the geotools wrapper?).
On a related note, does it make sense/is it possible to publish artifacts to an Apache staging maven repo during the RC process? It would make things like this (and previous issues with Scala 2.13 POM problems) easier to discover during the RC process. On Thu, Oct 12, 2023 at 3:19 AM Jia Yu <ji...@apache.org> wrote: > Thanks, Martin. We have logged a JIRA ticket for this: > https://issues.apache.org/jira/browse/SEDONA-409 > > And we have collected enough votes so I will close this thread. > > Thanks, > Jia > > On Tue, Oct 10, 2023 at 10:50 AM Martin Desruisseaux > <martin.desruisse...@geomatys.com> wrote: > > > > Hello all > > > > Note that shaded JAR files are incompatible with Java Platform Module > > System (JPMS), because we cannot have multiple module-info.class files > > in a single JAR file. This is not an issue as long as Sedona does not > > have dependencies that are JPMS modules, or otherwise as long as the > > modular dependencies apply some workaround for making possible to run on > > the class-path (e.g. duplicating module-info.class information into > > META-INF/services). But there is a possibility that some days, it will > > not work anymore or would be very hard (e.g. merging all > > module-info.class files into a single one may be difficult). It may be > > safe to plan a transition from shaded JAR to unshaded ones, not > > necessarily in this release but for the future. > > > > Martin > > > > > > Le 2023-10-10 à 19 h 29, Jia Yu a écrit : > > > > > The unshaded jars created lots of confusion for the users. People who > > > directly use the precompiled jars (due to no external internet > > > connection / no Maven resolvers) should use the shaded jars, rather > > > than the unshaded jars. A couple of users in the past just put all > > > shaded/unshaded jars in SPARK_HOME/jars which will break the > > > environment. > > > > > > Therefore, I decided to remove all unshaded jars in the released > > > binary. If someone really wants to use the unshaded jars, they should > > > use the Maven coordinate together with a Maven dependency resolver. We > > > will still release those unshaded jars to Maven Central but just not > > > to ASF release binary. > > > > > > In addition, ASF's voting process is mainly focused on voting the > > > source code. Binary is just a convenience release for users, no hard > > > requirements on it. > > > > > > Please let me know if this makes sense to you. Or, if you have > > > suggestions, please also advise. > > > > > > Thanks, > > > Jia > > > -- Adam Binford
