[ 
https://issues.apache.org/jira/browse/SPARK-61895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated SPARK-61895:
-----------------------------------
    Labels: pull-request-available  (was: )

> Load the SRS registry PySpark ships in GeometryType and GeographyType
> ---------------------------------------------------------------------
>
>                 Key: SPARK-61895
>                 URL: https://issues.apache.org/jira/browse/SPARK-61895
>             Project: Spark
>          Issue Type: Improvement
>          Components: Connect
>    Affects Versions: 4.2.0, 4.3.0, 5.0.0, 4.2.1, 4.4.0, 4.2.2, 4.3.1, 4.5.0
>            Reporter: James Willis
>            Priority: Major
>              Labels: pull-request-available
>
> SPARK-55790 committed the PROJ-based SRS registry to both {{sql/api}} 
> resources and {{python/pyspark/sql/srs_registry.csv}}, but only the JVM reads 
> it. PySpark's {{SpatialReferenceSystemCache}} ({{geo_utils.py}}) still 
> hard-codes {{SRID:0}}, {{EPSG:3857}} and {{OGC:CRS84}}, so 
> {{GeometryType(32610)}} raises {{ST_INVALID_SRID_VALUE}} while the server 
> accepts the SRID. For a Spark Connect client a result schema with 
> {{geometry(32610)}} therefore fails to decode; every released 
> {{pyspark-client}} is affected.
> {{python/pyspark/sql/srs_registry.csv}} already exists, written by the same 
> generator as the JVM copy. Load it in the Python cache with 
> {{importlib.resources}} (as {{pyspark.errors}} loads its JSON), parsing rows 
> like the Java class and adding {{SRID:0}} and the OGC overrides on top; add 
> {{**/*.csv}} to {{package_data}} so the classic and client wheels ship it; 
> mirror {{SpatialReferenceSystemMapperSuite}} in Python.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to