abhishekagarwal87 commented on issue #12262:
URL: https://github.com/apache/druid/issues/12262#issuecomment-1183016198
Awesome. with this capability, we can do shuffle joins, window aggregations,
etc without bottlenecking on the broker. have you also thought about
elasticity? Right now, scaling historicals up and down is tricky for large
clusters as segment balancing takes time and historicals need to download
segments from deep storage. with elasticity in mind, (1) makes much more sense
than (2) because
- comparing druid architecture to storage-compute, I see brokers as compute
and historicals as storage. To get more concurrency, brokers are users would
want to scale up.
- scaling brokers up and down does not require shuffling of local data.
This is really the important reason behind going with (2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
