Hi Percy, jai_core:1.1.3 issue was very annoying. Maven Central does not have jai_core:1.1.3 jar but it keeps a POM.xml file there. This will mess up many resolvers because most resolvers will by default first try Maven Central. If they find a pom there, then they will assume jars are there and then stop searching in the next repo. So the order of searching different maven repos matters.
What will be the best way to help you address this issue? One thing I can do for you is that, I can add the additional geotools jars you need to geotools-wrapper (https://github.com/jiayuasu/geotools-wrapper) and publish a new geotools-wrapper version to Maven Central. This can be very quick since the release of geotools-wrapper is not within the release cycle of Apache Sedona. Please let me know what you think. Thanks, Jia On Wed, Mar 29, 2023 at 7:56 PM PERSIE WANG <[email protected]> wrote: > > Hey Jia, > > Thanks a lot for your response, there are two issues to update: > > 1) EMR. I think that is the issue with EMR itself, the steps mentioned in > tutorial is supposed to be right, but currently it is not working while EMR > Spark cannot add those jars in classPath during runtime; Generally, we use > spark.jars.packages in the notebook cell for downloading dependencies. > > 2) gt-referencing. If you try sedona-shaded(or Sedona-Core&SQL) with > gt-ferenceing-28.2(not geotools-wrapper), it is not working, the problem is > with jai_core:1.1.3 not found from the central maven repo. > > Best, > Percy > > On Tue, Mar 28, 2023 at 1:28 AM Jia Yu <[email protected]> wrote: >> >> Hi Percy, >> >> I just tried Sedona 1.4.0 + geotools-wrapper-1.4.0-28.2 on AWS EMR >> 6.9.0 with ST_Transform from epsg:4326 to epsg:3857 and it works fine. >> Can you please let me know what CRS system you are using? So I can >> reproduce the problem. >> >> Thanks, >> Jia >> >> >> On Mon, Mar 27, 2023 at 3:07 PM PERSIE WANG <[email protected]> wrote: >> > >> > To whom it may concern, >> > >> > I'm following your tutorial to run the spark on EMR notebook, it seems like >> > there exists a dependency issue between Sedona and Geotools. See details as >> > following, >> > >> > In order to use ST_transform function, I need both Sedona and >> > Geotools-wrapper (for gt-referencing), so I follow the the tutorial as >> > following: >> > https://sedona.apache.org/latest-snapshot/setup/emr/#add-software-configuration >> > to setup cluster. The spark application can be initiated, but when i import >> > the following dependencies, it does not work. "*<console>:45: error: object >> > sedona is not a member of package org"* >> > >> > *"* >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > *import org.apache.spark.SparkContextimport >> > org.apache.spark.sql.SparkSessionimport >> > org.apache.spark.serializer.KryoSerializerimport >> > org.apache.sedona.core.formatMapper.GeoJsonReaderimport >> > org.apache.sedona.sql.utils.{Adapter, SedonaSQLRegistrator}import >> > org.apache.sedona.core.serde.SedonaKryoRegistratorimport >> > java.net.URLDecoderimport >> > org.opengis.referencing.NoSuchAuthorityCodeExceptionimport >> > org.locationtech.jts.geomimport org.wololo.geojson.GeoJSON* >> > *"* >> > >> > I also tried use SparkMagic in my cells to use >> > https://mvnrepository.com/artifact/org.apache.sedona/sedona-spark-shaded-3.0_2.12/1.4.0 >> > and >> > https://mvnrepository.com/artifact/org.datasyslab/geotools-wrapper/1.4.0-28.2, >> > but got the same error. But if i use sedona-shaded only, all imports works >> > except *org.opengis.referencing.NoSuchAuthorityCodeException.* >> > >> > >> > Please help take a look, and let me know if there is any tracking issue >> > related. >> > >> > Thanks >> > Percy
