jiayuasu opened a new issue, #3110:
URL: https://github.com/apache/sedona/issues/3110
## Describe the bug
Calling the STAC Python `Client.search()` (or
`CollectionClient.get_dataframe()` / `get_items()`) with a `datetime` argument
in `YYYY-mm` form fails.
```python
Client.open("https://planetarycomputer.microsoft.com/api/stac/v1";).search(
collection_id="aster-l1t",
datetime="2020-05",
)
```
raises:
```
RuntimeError: Failed to get filtered dataframe
```
which wraps:
```
TypeError: 'NoneType' object is not callable
```
The `YYYY-mm` form is documented as supported (the tutorial states
*"`YYYY-mm` expands to `["YYYY-mm-01T00:00:00Z",
"YYYY-mm-<last_day>T23:59:59Z"]`"*), so this is a functional gap, not just a
docs mismatch.
## Root cause
`python/sedona/spark/stac/collection_client.py` imports (line 24):
```python
from pyspark.sql.types import dt
```
`pyspark.sql.types` exposes no meaningful `dt`; the imported name resolves
to `None`. In `_expand_date`, the `YYYY-mm` branch then does (line 315):
```python
elif len(date_str) == 7: # YYYY-mm
year, month = date_str.split("-")
last_day = (dt(int(year), int(month) + 1, 1) - dt.timedelta(days=1)).day
return [f"{date_str}-01T00:00:00Z", f"{date_str}-{last_day}T23:59:59Z"]
```
Because `dt is None`, `dt(...)` raises `TypeError: 'NoneType' object is not
callable`. Note also that even if `dt` had resolved to the `datetime` module,
`dt(...)` would still be wrong (a module is not callable), and `int(month) + 1`
overflows for December (`month == 12` produces month `13`). The module already
imports `datetime as python_datetime` on line 22, which looks like the intended
reference.
Only the `YYYY-mm` form is affected. `YYYY`, `YYYY-mm-dd`,
`YYYY-mm-ddTHH:MM:SSZ`, and explicit interval lists all work.
## To Reproduce
Minimal, no network required:
```python
from sedona.spark.stac.collection_client import CollectionClient
CollectionClient._expand_date("2020-05")
# TypeError: 'NoneType' object is not callable
```
## Expected behavior
`_expand_date("2020-05")` should return:
```python
["2020-05-01T00:00:00Z", "2020-05-31T23:59:59Z"]
```
consistent with the docstring and the STAC tutorial docs.
## Suggested fix
Remove the erroneous `from pyspark.sql.types import dt` and use the standard
library (the module already imports `datetime as python_datetime`), computing
the last day with `calendar.monthrange` to also fix the December overflow:
```python
import calendar
...
elif len(date_str) == 7: # YYYY-mm
year, month = (int(x) for x in date_str.split("-"))
last_day = calendar.monthrange(year, month)[1]
return [f"{date_str}-01T00:00:00Z",
f"{date_str}-{last_day:02d}T23:59:59Z"]
```
## Environment
- Sedona: `master` (also present in 1.8.1)
- PySpark: 3.4.4
- Python: 3.12
- File: `python/sedona/spark/stac/collection_client.py` — `_expand_date`
(erroneous import at line 24, failure at line 315)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
