mentasm commented on PR #42352: URL: https://github.com/apache/spark/pull/42352#issuecomment-1894518160
I will happily add my experiences to that email thread as a consumer of SSS if I can work out how. Our prod environment runs around 40 SSS apps and traffic is determined by banking transactions so it is varied throughout the day, we would see significant savings if we could scale based on load. As it is we have to run 24x7 at resource levels that can deal with peak. As the batch centric scaling that currently exists within spark does not provide any benefit to us, except in non-prod environments where traffic can be zero, we have actually had to completely disable DRA. In my opinion getting something in place for SSS, even if rudimentary in nature would be a huge step forward for consumers of SSS. On Wed, 17 Jan 2024, 3:23 am Krystal Mitchell, ***@***.***> wrote: > Hi @mridulm <https://github.com/mridulm>. We appreciate you support on > this. DRA is essential to auto scaling up and back down. Can you please > confirm if this proposal was ever dropped in the dev list for discussion - > to ensure better visibility to the proposal and soliciting feedback in > parallel. Is that what this thread > <https://lists.apache.org/thread/9yx0jnk9h1234joymwlzfx2gh2m8b9bo> was > meant to support? > > Please let us know what the best steps are to help get this work > prioritized. Thank you! > > CC: @mridulm <https://github.com/mridulm>, @vitgorbunov > <https://github.com/vitgorbunov> , @krymitch <https://github.com/krymitch> > , @mentasm <https://github.com/mentasm> , @pky-c > <https://github.com/pky-c>, @pkotikalapudi > <https://github.com/pkotikalapudi> > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/42352#issuecomment-1894134932>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/ADX7NTBZSFWLXMTJM56NFGDYO2WBHAVCNFSM6AAAAAA3EZQX6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUGEZTIOJTGI> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
