paleolimbot opened a new issue, #1092: URL: https://github.com/apache/sedona-db/issues/1092
Currently we implement GDAL reading via OGR using pyogrio's Python interface. This leads to some Python calls that would be nice to avoid...among other things, when we have Python in multiple places in the plan (e.g., udfs, writes that use Python) we can get a hang. Even though we can probably fix those hangs otherwise, moving GDAL reads and/or writes into Rust means we can get them for free in SQL and R. Roughly, we need to implement the Arrow interface here: https://github.com/apache/sedona-db/blob/e3f57ff4714706a5839cce33537d0deccbcf6e7d/c/sedona-gdal/src/vector/layer.rs#L30-L35 ...and wire up constructing the dataset and layer like pyogrio does. This is pretty involved, and handles the predicate pushdown (bbox, "ignored fields", limit). https://github.com/geopandas/pyogrio/blob/main/pyogrio/_io.pyx#L1908-L1929 We currently implement the ExternalFormatSpec via Python...we just need to implement the ExternalFormatSpec directly in Rust. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
