Oops: something went wrong with this sentence: * k8shim: the proxy is pushed down all the way into the applications and > nodes. The tasks (allocations) leverage the context to call the api. We > thus have a widespread impact in the cache code as the call is not only > direct from the cache but the objects are generated in
* k8shim: the proxy is pushed down all the way into the applications and nodes. The tasks (allocations) leverage the context to call the api. We thus have a widespread impact in the cache code as the call is not only direct from the cache context but from the objects that are part of the cache. On Tue, 28 Sept 2021 at 18:11, Wilfred Spiegelenburg <[email protected]> wrote: > Hi Weiwei, > > Mani and I had started thinking about this work a while ago. This is just > the start to make sure we have the basics covered. > There is further detail that needs to be worked out that is exactly why > the document was created and the mail send. > > On Tue, 28 Sept 2021 at 10:07, Weiwei Yang <[email protected]> wrote: > >> Hi Mani >> >> Thank you for starting this. Really nice to see proposals like this to >> further improve the efficiency and readability. >> >> As this is going to change the scheduler-interface pretty dramatically, I >> think we need to bring this to all dev's attention. Please help to review >> the doc and share your thoughts. Doc: >> >> https://docs.google.com/document/d/1qNGz7JgA5ArfFy5gLC_uhCsP-w9ii-pJoV-Fbh01awU/edit >> . >> >> Starting from some high-level questions >> >> 1. Is this planned for 1.0.0 release? 1.0.0 will be released sometime >> in >> Nov or Dec this year. >> > > Yes updated the target release in the jira > > >> 2. Have we done PoC to verify the size of this change? I assume we will >> need changes in both repos. >> > > There will be changes in 3 repos: > * scheduler interface > * core > * k8shim > > The impact on each repository is different. Splitting this into the > interface change and message changes. > Interface changes: > * scheduler interface: as per the doc > * core: the si.UpdateRequest objects are unpacked and packed in the > context and RMproxy, The interface is implemented in the RMProxy only. > There should be no impact outside of that code of the interface change. > * k8shim: the proxy is pushed down all the way into the applications and > nodes. The tasks (allocations) leverage the context to call the api. We > thus have a widespread impact in the cache code as the call is not only > direct from the cache but the objects are generated in > > Message changes: > * core: the individual messages are unwrapped and passed down into the > partition at that point everything is converted into core objects. The > conversion routines will need updating but that is it. The "si" objects > were factored out as part of the cache removal. > * k8shim: all code uses the si_helper for the object creation except for > Allocations. That also seem to have filtered into the app manager > > 3. Do we need an umbrella JIRA and a feature branch for this work? >> > > YUNIKORN-337 is the umbrella, changed it > > >> 4. We need to cover the doc changes as well, such as >> http://yunikorn.apache.org/docs/next/design/scheduler_core_design >> > > Jira is logged and added as a task > > >> Apart from that, I have one more ask for this. Today, we have a callback >> interface: "ResourceManagerCallback". The shim registers a callback to the >> core via: >> >> https://github.com/apache/incubator-yunikorn-k8shim/blob/2507b8b68b1c385e67ac2afcd88626e890d2e268/pkg/shim/scheduler.go#L227 >> . >> Is it possible to change this to an event handling model? I have some >> thoughts, could we do something like this: >> https://play.golang.org/p/Kibbq4QtRUf. This should work for both local vs >> gRPC communications (currently we are using the local, aka inner-process >> communication mode). The gRPC mode just requires us to implement a >> client-side lib to receive gRPC messages from the core, and trigger event >> handling based on the messages. This way I think it is clearer than the >> current approach. Would this make sense? I have also commented in the >> design doc. >> > > The gRPC interface has a separate definition. > For using gRPC there has to be a server in both the k8shim and the core. > The core can send messages to the shim and the other way around. The server > code cannot be used to send a new message, just a response to an incoming > message. The server and client code are generated using the "service > Scheduler" section with the rpc tags. So yes we need to still cover that in > the document. It does not affect the implementation as that is a far larger > task. Beside updating the text in the interface document for the service we > need to update the scheduler client and SimpleScheduler code in the core. > This has no impact on the k8shim as there has never been any attempt to > implement this on the shim side. > Added a task to make sure that is handled. > > Wilfred > > >> Thanks >> >> On Mon, Sep 27, 2021 at 6:59 AM Manikandan R <[email protected]> >> wrote: >> >> > Hi Everyone, >> > >> > I would like to bring the discussions going on >> > https://issues.apache.org/jira/browse/YUNIKORN-337 to the community >> due to >> > changes in the way shim and core communicate for allocation, >> application, >> > node etc. Please go through the WIP proposal doc specified in JIRA and >> > share your thoughts. >> > >> > Thanks, >> > Mani >> > >> >
