dnishimura opened a new pull request #1027: SAMZA-2046: Startpoint fan out implementation URL: https://github.com/apache/samza/pull/1027 This PR implements the fan out logic. During fan out, Startpoints written to the metadata store keyed by SSP and/or SSP+TaskName are moved to a seperate namespace with structure: `Map<TaskName, Map<SSP, Startpoint>>`, similar to how checkpoints are structured. The intent is for the job coordinators perform the fan out upon startup. This allows for Startpoints to be written while the job is running and applied to the starting offsets only upon job start time. The fan out isolation allows for Startpoints to be written at anytime. A follow on PR after this PR will integrate the fan out with the various job coordinators. Another follow on PR(#1026) that is in flight and is required for this to work is the integration of the Startpoint-to-starting offset resolver. @shanthoosh @cameronlee314 @prateekm
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
