This is an automated email from the ASF dual-hosted git repository. jiayu pushed a commit to branch fix-fabric in repository https://gitbox.apache.org/repos/asf/sedona.git
commit e10cb7a63267167384ca877216d77cb1f6c89eaf Author: Jia Yu <[email protected]> AuthorDate: Tue Apr 30 01:46:26 2024 -0700 Add the spark properties --- docs/image/fabric/{fabric-9.png => fabric-10.png} | Bin docs/image/fabric/fabric-5.png | Bin 103507 -> 192084 bytes docs/image/fabric/fabric-6.png | Bin 189504 -> 103507 bytes docs/image/fabric/fabric-7.png | Bin 123955 -> 189504 bytes docs/image/fabric/fabric-8.png | Bin 146759 -> 123955 bytes docs/image/fabric/fabric-9.png | Bin 150114 -> 146759 bytes docs/setup/fabric.md | 30 ++++++++++++++-------- 7 files changed, 20 insertions(+), 10 deletions(-) diff --git a/docs/image/fabric/fabric-9.png b/docs/image/fabric/fabric-10.png similarity index 100% copy from docs/image/fabric/fabric-9.png copy to docs/image/fabric/fabric-10.png diff --git a/docs/image/fabric/fabric-5.png b/docs/image/fabric/fabric-5.png index f4f3b7bc0..0c1127a55 100644 Binary files a/docs/image/fabric/fabric-5.png and b/docs/image/fabric/fabric-5.png differ diff --git a/docs/image/fabric/fabric-6.png b/docs/image/fabric/fabric-6.png index 00b250cf2..f4f3b7bc0 100644 Binary files a/docs/image/fabric/fabric-6.png and b/docs/image/fabric/fabric-6.png differ diff --git a/docs/image/fabric/fabric-7.png b/docs/image/fabric/fabric-7.png index 2162e33b1..00b250cf2 100644 Binary files a/docs/image/fabric/fabric-7.png and b/docs/image/fabric/fabric-7.png differ diff --git a/docs/image/fabric/fabric-8.png b/docs/image/fabric/fabric-8.png index eb0ac3c2c..2162e33b1 100644 Binary files a/docs/image/fabric/fabric-8.png and b/docs/image/fabric/fabric-8.png differ diff --git a/docs/image/fabric/fabric-9.png b/docs/image/fabric/fabric-9.png index 45effa93a..eb0ac3c2c 100644 Binary files a/docs/image/fabric/fabric-9.png and b/docs/image/fabric/fabric-9.png differ diff --git a/docs/setup/fabric.md b/docs/setup/fabric.md index 1db3bacc7..5a40f8fd0 100644 --- a/docs/setup/fabric.md +++ b/docs/setup/fabric.md @@ -24,19 +24,29 @@ In the `Environment` page, click the `Public libraries` tab and then type in `ap  -## Step 5: Save and publish the environment +## Step 5: Set Spark properties -Click the `Save` button and then click the `Publish` button to save and publish the environment. This will create the environment with the Apache Sedona Python package installed. The publishing process will take about 10 minutes. +In the `Environment` page, click the `Spark properties` tab, then create the following 3 properties: + +- `spark.sql.extensions`: `org.apache.sedona.viz.sql.SedonaVizExtensions,org.apache.sedona.sql.SedonaSqlExtensions` +- `spark.serializer`: `org.apache.spark.serializer.KryoSerializer` +- `spark.kryo.registrator`: `org.apache.sedona.core.serde.SedonaKryoRegistrator`  -## Step 6: Download Sedona jars +## Step 6: Save and publish the environment + +Click the `Save` button and then click the `Publish` button to save and publish the environment. This will create the environment with the Apache Sedona Python package installed. The publishing process will take about 10 minutes. + + + +## Step 7: Download Sedona jars 1. Learn the Sedona jars you need from our [Sedona maven coordinate](maven-coordinates.md) 2. Download the `sedona-spark-shaded` jars from [Maven Central](https://search.maven.org/search?q=g:org.apache.sedona). Please pay attention to the Spark version and Scala version of the jars. If you select Spark 3.4 in the Fabric environment, you should download the Sedona jars with Spark 3.4 and Scala 2.12 and the jar name should be like `sedona-spark-shaded-3.4_2.12-1.5.1.jar`. 3. Download the `geotools-wrapper` jars from [Maven Central](https://search.maven.org/search?q=g:org.datasyslab). Please pay attention to the Sedona versions of the jar. If you select Sedona 1.5.1, you should download the `geotools-wrapper` jar with version 1.5.1 and the jar name should be like `geotools-wrapper-1.5.1-28.2.jar`. -## Step 7: Upload Sedona jars to the Fabric environment LakeHouse storage +## Step 8: Upload Sedona jars to the Fabric environment LakeHouse storage In the notebook page, choose the `Explorer` and click the `LakeHouses` option. If you don't have a LakeHouse, you can create one. Then choose `Files` and upload the 2 jars you downloaded in the previous step. @@ -48,15 +58,15 @@ abfss://9e9d4196-870a-4901-8fa5-e24841492...@onelake.dfs.fabric.microsoft.com/e1 abfss://9e9d4196-870a-4901-8fa5-e24841492...@onelake.dfs.fabric.microsoft.com/e15f3695-af7e-47de-979e-473c3caa9f5b/Files/geotools-wrapper-1.5.1-28.2.jar ``` - -  -## Step 8: Start the notebook with the Sedona environment and install the jars + + +## Step 9: Start the notebook with the Sedona environment and install the jars In the notebook page, select the `ApacheSedona` environment you created before. - + In the notebook, you can install the jars by running the following code. Please replace the `spark.jars` with the `ABFS` paths of the 2 jars you uploaded in the previous step. @@ -69,7 +79,7 @@ In the notebook, you can install the jars by running the following code. Please } ``` -## Step 9: Verify the installation +## Step 10: Verify the installation You can verify the installation by running the following code in the notebook. @@ -85,4 +95,4 @@ sedona.sql("SELECT ST_GeomFromEWKT('SRID=4269;POINT(40.7128 -74.0060)')").show() If you see the output of the point, then the installation is successful. - +
