johnyangk opened a new pull request #159: [NEMO-216,251,259] Implement ReadyCheckingSideInputReader URL: https://github.com/apache/incubator-nemo/pull/159 JIRA: [NEMO-216: Support side inputs and windowing](https://issues.apache.org/jira/projects/NEMO/issues/NEMO-216) (+ NEMO-251, NEMO-259) **Major changes:** - MaterializedViewReader (implements ReadyCheckingSideInputReader) and appropriate push-back logics - SideInputTransform: Wraps materialized views with an index in SideInputElement to distinguish multiple sideinputs destined to the same DoFn (replaces the previous usage of BroadcastVariable ids) - In the runtime, treat a SideInput as just another DataFetcher (do not use BroadcastManagerWorker): all sideinput-specific logics stay in the Beam frontend **Minor changes to note:** - Clean up watermark propagation flow for DataFetchers - In MultiThreadParentTaskDataFetcher, let the task thread handle watermarks just like other elements and not data fetcher threads **Tests for the changes:** - WindowedBroadcastITCase **Other comments:** - This is the last sub-issue of the umbrella issue: [NEMO-229] Support watermark, so once this PR is merged we should be ready to mark that as resolved
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
