Re: [DISCUSS] SEP: Host affinity in standalone.

Boris, Thanks for your review. Responses are inline. >> I think we also need MetadataStorage one (details may be worked out later) to hide the locality storage implementation details. - Agree, my initial proposal had LocalityManager interface which was logically similar to

Re: [DISCUSS] SEP: Host affinity in standalone.

Yi, Thanks for taking time to review this and providing your feedback. Responses inline. >> 1) ContainerInfo: (containerId, physicalResourceId): this mapping is the reported mapping from container processes in standalone, and preferred mapping in YARN >> 2) TaskLocality: (taskId,

Re: [DISCUSS] SEP: Host affinity in standalone.

Hi Yi, Thanks for your feedback. Responses inline. > It seems like we can deprecate the whole BalancingTaskNameGrouper altogether. - Yes, that’s part of the proposed interface changes. > That also means that you will somehow store the task-to-container mapping info in the locality

Re: [DISCUSS] SEP: Host affinity in standalone.

Linking Boris' earlier comments to the correct [DISCUSSION] thread: http://mail-archives.apache.org/mod_mbox/samza-dev/201801.mbox/%3CCAPAaT%2BtH2H5TEvFQUn9jw6iR%3DyvVEu46rDLJsqexpwKz0CAH1g%40mail.gmail.com%3E On Thu, Feb 1, 2018 at 5:27 PM, Yi Pan wrote: > Hi, Santhoosh, >

Re: [DISCUSS] SEP: Host affinity in standalone.

Hi, Santhoosh, Thanks for the SEP and the latest revisions. Here are some of my comments based on the latest proposal: - The basic idea for implementing state-aware task-to-physical-process assignment in JobModel is not quite clear. ContainerAllocator is solving a different problem in

[DISCUSS] SEP: Host affinity in standalone.

Hi, I created SEP for SAMZA-1554 : Host affinity in standalone. The link to the SEP is here: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957309 Please review and comments are welcome. Thanks.