+1 on splitting the large message sending between Shim and Core and processing them in a parallel way. Look forward to the performance improvement!
On Tue, Sep 28, 2021 at 1:16 AM Wilfred Spiegelenburg <[email protected]> wrote: > Oops: something went wrong with this sentence: > > * k8shim: the proxy is pushed down all the way into the applications and > > nodes. The tasks (allocations) leverage the context to call the api. We > > thus have a widespread impact in the cache code as the call is not only > > direct from the cache but the objects are generated in > > > * k8shim: the proxy is pushed down all the way into the applications and > nodes. The tasks (allocations) leverage the context to call the api. We > thus have a widespread impact in the cache code as the call is not only > direct from the cache context but from the objects that are part of the > cache. > > On Tue, 28 Sept 2021 at 18:11, Wilfred Spiegelenburg <[email protected]> > wrote: > > > Hi Weiwei, > > > > Mani and I had started thinking about this work a while ago. This is just > > the start to make sure we have the basics covered. > > There is further detail that needs to be worked out that is exactly why > > the document was created and the mail send. > > > > On Tue, 28 Sept 2021 at 10:07, Weiwei Yang <[email protected]> wrote: > > > >> Hi Mani > >> > >> Thank you for starting this. Really nice to see proposals like this to > >> further improve the efficiency and readability. > >> > >> As this is going to change the scheduler-interface pretty dramatically, > I > >> think we need to bring this to all dev's attention. Please help to > review > >> the doc and share your thoughts. Doc: > >> > >> > https://docs.google.com/document/d/1qNGz7JgA5ArfFy5gLC_uhCsP-w9ii-pJoV-Fbh01awU/edit > >> . > >> > >> Starting from some high-level questions > >> > >> 1. Is this planned for 1.0.0 release? 1.0.0 will be released sometime > >> in > >> Nov or Dec this year. > >> > > > > Yes updated the target release in the jira > > > > > >> 2. Have we done PoC to verify the size of this change? I assume we > will > >> need changes in both repos. > >> > > > > There will be changes in 3 repos: > > * scheduler interface > > * core > > * k8shim > > > > The impact on each repository is different. Splitting this into the > > interface change and message changes. > > Interface changes: > > * scheduler interface: as per the doc > > * core: the si.UpdateRequest objects are unpacked and packed in the > > context and RMproxy, The interface is implemented in the RMProxy only. > > There should be no impact outside of that code of the interface change. > > * k8shim: the proxy is pushed down all the way into the applications and > > nodes. The tasks (allocations) leverage the context to call the api. We > > thus have a widespread impact in the cache code as the call is not only > > direct from the cache but the objects are generated in > > > > Message changes: > > * core: the individual messages are unwrapped and passed down into the > > partition at that point everything is converted into core objects. The > > conversion routines will need updating but that is it. The "si" objects > > were factored out as part of the cache removal. > > * k8shim: all code uses the si_helper for the object creation except for > > Allocations. That also seem to have filtered into the app manager > > > > 3. Do we need an umbrella JIRA and a feature branch for this work? > >> > > > > YUNIKORN-337 is the umbrella, changed it > > > > > >> 4. We need to cover the doc changes as well, such as > >> http://yunikorn.apache.org/docs/next/design/scheduler_core_design > >> > > > > Jira is logged and added as a task > > > > > >> Apart from that, I have one more ask for this. Today, we have a callback > >> interface: "ResourceManagerCallback". The shim registers a callback to > the > >> core via: > >> > >> > https://github.com/apache/incubator-yunikorn-k8shim/blob/2507b8b68b1c385e67ac2afcd88626e890d2e268/pkg/shim/scheduler.go#L227 > >> . > >> Is it possible to change this to an event handling model? I have some > >> thoughts, could we do something like this: > >> https://play.golang.org/p/Kibbq4QtRUf. This should work for both local > vs > >> gRPC communications (currently we are using the local, aka inner-process > >> communication mode). The gRPC mode just requires us to implement a > >> client-side lib to receive gRPC messages from the core, and trigger > event > >> handling based on the messages. This way I think it is clearer than the > >> current approach. Would this make sense? I have also commented in the > >> design doc. > >> > > > > The gRPC interface has a separate definition. > > For using gRPC there has to be a server in both the k8shim and the core. > > The core can send messages to the shim and the other way around. The > server > > code cannot be used to send a new message, just a response to an incoming > > message. The server and client code are generated using the "service > > Scheduler" section with the rpc tags. So yes we need to still cover that > in > > the document. It does not affect the implementation as that is a far > larger > > task. Beside updating the text in the interface document for the service > we > > need to update the scheduler client and SimpleScheduler code in the core. > > This has no impact on the k8shim as there has never been any attempt to > > implement this on the shim side. > > Added a task to make sure that is handled. > > > > Wilfred > > > > > >> Thanks > >> > >> On Mon, Sep 27, 2021 at 6:59 AM Manikandan R <[email protected]> > >> wrote: > >> > >> > Hi Everyone, > >> > > >> > I would like to bring the discussions going on > >> > https://issues.apache.org/jira/browse/YUNIKORN-337 to the community > >> due to > >> > changes in the way shim and core communicate for allocation, > >> application, > >> > node etc. Please go through the WIP proposal doc specified in JIRA and > >> > share your thoughts. > >> > > >> > Thanks, > >> > Mani > >> > > >> > > >
