jiayuasu opened a new issue, #1214:
URL: https://github.com/apache/sedona-db/issues/1214
## Summary
`lit()` (`python/sedonadb/python/sedonadb/expr/literal.py`) resolves Python
values through a class-name → handler table plus a `pa.array([obj])` fallback.
While building the `sedonadb-geopandas` assignment surface (#1195) a number of
common scalar values turned out to need normalization *before* they can reach
`lit()`, and the workarounds currently live in that package. Most of them would
be more useful in `lit()` itself, where every SedonaDB user benefits. Listing
them here so they can be picked off independently.
## GeoArrow scalars
- `_lit_from_geoarrow_scalar` rebuilds the type as `ga.wkb().with_crs(crs)`,
which **drops the edge type**: a spherical (geography) WKB scalar becomes
planar geometry.
```python
sph =
ga.wkb().with_edge_type(ga.EdgeType.SPHERICAL).with_crs(crs.to_json())
df.select(lit(pa.scalar(point.wkb, sph))) # -> geometry<Wkb(epsg:4267)>,
edge type lost
```
- Only `geoarrow.pyarrow._scalar.WkbScalar` is registered, so scalars of
other GeoArrow types (`WktScalar`, native point/linestring/polygon scalars)
fall through to `pa.array([obj])` and fail with `Can't create SedonaDB literal
from object of type geoarrow.pyarrow._scalar.WktScalar`. They all expose
`.wkb`, so the same WKB path could serve them.
## Missing-value sentinels and temporal scalars
| value | today |
|---|---|
| `pandas.NA` | `ValueError` (unrecognized type) |
| `pandas.NaT` | `ValueError` — it is neither a `Timestamp` nor a
`Timedelta` instance |
| `pandas.Timestamp` / `pandas.Timedelta` | resolve at **microsecond**
resolution; nanoseconds are silently dropped (`Timedelta(1)` becomes `0`) |
| `numpy.datetime64` / `numpy.timedelta64` in non-Arrow units (`D`, `W`,
`Y`, …) | `ValueError` — a lossless conversion to `s` exists for these |
| `numpy.ma.masked` | `ValueError` — it means "missing" |
| 0-d `numpy.ndarray` | `ValueError` — it is one value |
| structured `numpy.void` | `ValueError` — representable as a typed Arrow
struct |
| typed-null nested Arrow scalar (`pa.scalar(None, pa.list_(...))`) |
`ValueError` — the one-element array spelling `pa.array([None], type)` works |
The pandas cases can be handled the way the table already handles pandas
frames — by qualified class name — without importing pandas.
## Related
- #1195 carries the current workarounds in `sedonadb_geopandas/_series.py`
(`normalize_scalar`) and `_temporal.py`; each item above that lands here lets
that code shrink to a pass-through.
- LargeBinary-backed WKB is a separate engine-side issue (WKB import rejects
LargeBinary storage) and is tracked on its own.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
