cameronlee314 commented on pull request #1506: URL: https://github.com/apache/samza/pull/1506#issuecomment-870003549
> > My initial thoughts are that if we think it is useful for the general TaskModel API to separate out the side inputs (for use by actual applications), then that could help the implementation here > > That is one of the benefits of having the divide exposed in the TaskModel. While that requires concrete use cases from the application perspective, the other purpose was to simplify a lot of flow within ContainerStorageManager and the newly introduced classes for state restoration which right now has needs to know which SSPs are side inputs vs changelog SSPs vs input SSPs. @dxichen do you have any inputs here to see if this would benefit/simplify the existing implementation? > > If we decide against it, then the changes looks good to me. It is useful to understand the long term path and potential clean up options if we want to remove this hack. Do we need to put all of this information in the main `TaskModel` API layer though? I would imagine that the state restoration layer can have multiple implementations, so it could be helpful to expose this granular SSP information somewhere, but if we put the info in `TaskModel`, would that be overloading the scope of `TaskModel` too much? Maybe not, because this is all I/O that a task is dealing with. What do others think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
