jiayuasu opened a new issue, #3306: URL: https://github.com/apache/sedona/issues/3306
## Problem When `GeoSeries.fillna(limit=...)` receives an independently constructed distributed GeoSeries replacement, it runs `positional_join.agg(...).first()` before returning in order to choose between positional and label alignment. The eventual result action then recomputes the uncached alignment subtree, and the missing/non-missing union references that subtree twice. A Spark 3.5 probe confirmed that constructing this result starts Spark jobs while scalar and same-frame replacements remain lazy. The final plan repeats both inputs and does not reuse the alignment exchange. Exact job counts depend on the Spark configuration, but the eager pass and repeated scans are structural. ## Desired behavior Build a lazy distributed plan for independent GeoSeries replacements while preserving the current GeoPandas-compatible rules: - exact duplicate axes pair positionally; - a unique replacement index broadcasts to duplicate left labels; - a non-identical duplicate replacement index raises the pandas-compatible error; - mixed Index and MultiIndex shapes do not match; - the exact left axis and natural order are preserved. A simple unconditional left join would break the duplicate-axis cases. Persisting inside `fillna` would also require a safe cache lifecycle. This likely needs a plan-level alignment strategy rather than either shortcut. Follow-up to #2068 and #3302. PR #3302 includes a lazy same-frame fast path; this issue covers arbitrary independent GeoSeries replacements. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
