james-willis commented on PR #1074: URL: https://github.com/apache/sedona-db/pull/1074#issuecomment-5072073564
We need special treatment for raster because we don't want to load the same image from the network many times. Thats the idea of this PR: 1. first we load so when theres 0 dupes of rasters we've already created the binary view 2. then we join so when we copy the raster from the input to the output we reuse the backing buffer of the binary view. We use the probe side so rasters sharing a buffer backing their binary view tend to wind up in the same batch of downstream 3. finally run zonal stats. all copies of the same raster come through the same batch and thus all references to the same buffer are released. This keeps peak ram usage down. The alternative is a cache. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
