jiayuasu opened a new issue, #3089:
URL: https://github.com/apache/sedona/issues/3089
The raster distance-join support added in #2980 (GH-2809) deserializes
`GridCoverage2D` rasters in several hot paths **without calling
`dispose(true)`**, leaking off-heap native raster memory on long-running
executors. Sibling raster paths (`RS_Predicate.evaluator`,
`TraitJoinQueryBase.toWGS84EnvelopeRDD`) already dispose via `try/finally`.
### Resource leaks (3 sites)
1. `RS_DWithin.eval` (`RasterPredicates.scala`) — deserializes raster(s),
calls `RasterPredicates.rsDWithin`, never disposes.
2. `TraitJoinQueryBase.toExpandedWGS84EnvelopeRDD` — deserializes a raster
for its WGS84 envelope, no dispose.
3. `BroadcastIndexJoinExec` raster distance stream-side branch — same
pattern.
Fix: wrap each deserialization in `try/finally { raster.dispose(true) }`,
mirroring the sibling paths.
### Related, lower severity
- `expandRasterFilterEnvelope` expands an empty / `NullGeometry` `baseShape`
(the substitute for NULL inputs in `toExpandedWGS84EnvelopeRDD`) into a
non-empty filter envelope, which can spuriously match rows that should never
join. It should return the empty geometry unchanged.
- The raster distance-join physical node's `simpleString` renders
`RS_Distance(...) < r`, but there is no `RS_Distance` SQL function — the join
key is `RS_DWithin`, so `EXPLAIN` output is misleading.
I have fixes for all of these and can open a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
