jiayuasu opened a new issue, #3126:
URL: https://github.com/apache/sedona/issues/3126
## Description
`StacUtils.getFilterTemporal` computes a bounding interval for compound
temporal filters before sending the `datetime` parameter to a STAC catalog.
When an `OrFilter` child has an unbounded endpoint, `calculateUnionTemporal`
discards that unbounded endpoint and can emit a finite interval that is
narrower than the original predicate.
Because the Spark filter remains as a residual predicate, the remote
interval must always be a superset. A narrower interval can drop matching STAC
items before Spark evaluates the residual filter.
This behavior predates #3122 and is not introduced by that PR.
## Reproducer
```scala
val filter = TemporalFilter.OrFilter(
TemporalFilter.GreaterThanFilter(
"datetime",
LocalDateTime.parse("2025-03-06T00:00:00")),
TemporalFilter.LessThanFilter(
"datetime",
LocalDateTime.parse("2025-03-07T00:00:00")))
StacUtils.getFilterTemporal(filter)
```
The predicate is a tautology: every timestamp is either after March 6 or
before March 7. It currently serializes to a finite interval similar to:
```text
datetime=2025-03-06T00:00:00.000000000Z/2025-03-07T00:00:00.000000999Z
```
The STAC catalog therefore excludes timestamps outside that interval even
though the residual Spark predicate would retain them.
Mixed `OR` expressions such as equality combined with a one-sided inequality
have the same problem. The existing `getFilterTemporal with OrFilter` test
currently codifies the unsafe finite interval.
## Expected behavior
Temporal pushdown must not narrow the result set accepted by the residual
Spark predicate.
- Preserve an unbounded endpoint when the convex hull of an `OR` expression
is unbounded.
- Skip temporal pushdown when no useful safe interval can be produced, such
as a tautology.
- Do not emit reversed intervals for contradictory compound predicates; let
the residual filter determine the empty result if necessary.
## Suggested tests
- `datetime > a OR datetime < b` where `a < b`
- equality combined with a one-sided inequality
- disjoint bounded ranges whose safe convex hull is finite
- contradictory compound ranges that could otherwise produce a reversed
interval
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
