gaoyunhaii opened a new pull request #6: URL: https://github.com/apache/flink-ml/pull/6
Iteration requires to broadcast the epoch-watermark inside iteration no matter what partitioner is used. Currently the functionality is implementation via reflection. As a whole, the output of the an operator might be 1. ChainingOutput: if the following operators is chained with this one. In this case, we could direct output the event, which id equivalent to broadcast. 2. RecordWriterOutput: the records are written to an external task. The output wraps an `RecordWriter`, which provides the functionality of broadcast to support watermark / latencymark, and we need to proxy broadcasting to this object via reflection. 3. BroadcastingOutput: The operator itself have multiple downstream operators / tasks. `BroadcastingOutput` has a list of internal outputs, which must be 1 or 2. We should also maintain the metric of `numOutputRecords` by counting the broadcasted event. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
