clintropolis commented on PR #18176: URL: https://github.com/apache/druid/pull/18176#issuecomment-3076046102
> Initially our thinking was to focus on partial fetches from S3, so we only have to fetch the columns actually needed for a query. From my PoV (@clintropolis may have his own opinion) from there it would make sense to have the Druid servers able to fetch from each other. It would make it more feasible to burst up compute for a short period of time (~seconds even would make sense). Thanks for chiming in on this @gianm, I agree the next big area I'm probably going to be looking at is partial downloads, which will build on top of some of the stuff here (with some modifications) and introduce some new machinery to do downloads at a finer granularity than the whole segment that this requires. After that I agree it makes sense to allow pulling 'locally' from other servers if available instead of deep storage after we get all the foundations in place. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
