paleolimbot commented on PR #1751:
URL: https://github.com/apache/sedona/pull/1751#issuecomment-2607790654
Ok! This seems to work:
<details>
```python
import os
import pyspark
from sedona.spark import SedonaContext
if "SPARK_HOME" in os.environ:
del os.environ["SPARK_HOME"]
pyspark_version = pyspark.__version__[:pyspark.__version__.rfind(".")]
config = (
SedonaContext.builder()
.config(
"spark.jars",
"spark-shaded/target/sedona-spark-shaded-3.3_2.12-1.7.1-SNAPSHOT.jar",
)
.config(
"spark.jars.packages",
"org.datasyslab:geotools-wrapper:1.7.0-28.5",
)
.config(
"spark.jars.repositories",
"https://artifacts.unidata.ucar.edu/repository/unidata-all";,
)
.getOrCreate()
)
sedona = SedonaContext.create(config)
```
</details>
```python
from sedona.spark import Adapter, GridType
!rm -rf cities_maybe_partitioned
df = sedona.read.format("geoparquet").load("cities.parquet")
rdd = Adapter.toSpatialRdd(df, "geometry")
rdd.analyze()
rdd.spatialPartitioning(GridType.KDBTREE, num_partitions=6)
df2 = Adapter.toDfPartitioned(rdd, sedona)
df2.write.format("geoparquet").save("cities_maybe_partitioned")
!ls cities_maybe_partitioned/*.parquet
#>
cities_maybe_partitioned/part-00000-809113f8-6f63-4763-bbd0-3ba609efcdfd-c000.snappy.parquet
#>
cities_maybe_partitioned/part-00001-809113f8-6f63-4763-bbd0-3ba609efcdfd-c000.snappy.parquet
#>
cities_maybe_partitioned/part-00002-809113f8-6f63-4763-bbd0-3ba609efcdfd-c000.snappy.parquet
#>
cities_maybe_partitioned/part-00003-809113f8-6f63-4763-bbd0-3ba609efcdfd-c000.snappy.parquet
#>
cities_maybe_partitioned/part-00004-809113f8-6f63-4763-bbd0-3ba609efcdfd-c000.snappy.parquet
#>
cities_maybe_partitioned/part-00005-809113f8-6f63-4763-bbd0-3ba609efcdfd-c000.snappy.parquet
#>
cities_maybe_partitioned/part-00006-809113f8-6f63-4763-bbd0-3ba609efcdfd-c000.snappy.parquet
#>
cities_maybe_partitioned/part-00007-809113f8-6f63-4763-bbd0-3ba609efcdfd-c000.snappy.parquet
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
