dabruehl opened a new issue, #2521: URL: https://github.com/apache/sedona/issues/2521
Databricks launched their own spatial functions with DBR 17.1 and above https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-st-geospatial-functions It is still possible to install (init_script) Sedona on a DBR 17.3 LTS cluster as described on Sedona setup page https://sedona.apache.org/1.8.0/setup/databricks/ https://mvnrepository.com/artifact/org.apache.sedona/sedona-spark-shaded-4.0_2.13/1.8.0 However, it is not possibe to configure and use Sedona over the SQL ``` spark.sql.extensions org.apache.sedona.viz.sql.SedonaVizExtensions,org.apache.sedona.sql.SedonaSqlExtensions spark.serializer org.apache.spark.serializer.KryoSerializer spark.kryo.registrator org.apache.sedona.core.serde.SedonaKryoRegistrator spark.sedona.enableParserExtension false ``` This result into following cluster startup failure ``` { "reason": { "code": "INVALID_ARGUMENT", "type": "CLIENT_ERROR", "parameters": { "databricks_error_message": "Spark Driver was down due to misconfiguration, please check your config. [details] SparkMisconfiguration: java.lang.ClassNotFoundException: org.apache.spark.sql.catalyst.expressions.FoldableUnevaluable", "azure_error_code": "SparkMisconfiguration", "azure_error_message": "java.lang.ClassNotFoundException: org.apache.spark.sql.catalyst.expressions.FoldableUnevaluable" } }, "add_node_failure_details": { "failure_count": 1, "resource_type": "container", "will_retry": false } } ``` Is it possible to create a configuration which prefix all Sedona SQL functions (vector, raster)? * RS_FromGeoTiff -> SEDONA_RS_FromGeoTiff * ST_GeomFromWKT -> SEDONA_ST_GeomFromWKT * ... Or is there another solution? ... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
