[GitHub] jgrier commented on issue #7099: [FLINK-10887] [jobmaster] Add source watermark tracking to the JobMaster
jgrier commented on issue #7099: [FLINK-10887] [jobmaster] Add source watermark tracking to the JobMaster URL: https://github.com/apache/flink/pull/7099#issuecomment-459420616 @tweise @aljoscha @StephanEwen I think this is in a good state for final review and merge. Take a look when you get the chance please. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jgrier commented on issue #7099: [FLINK-10887] [jobmaster] Add source watermark tracking to the JobMaster
jgrier commented on issue #7099: [FLINK-10887] [jobmaster] Add source watermark tracking to the JobMaster URL: https://github.com/apache/flink/pull/7099#issuecomment-457430460 @tweise Have a look.. Also, I didn't ignore the input about "idleness" issues but that will be handled by the particular aggregation function used. For the example of event time source sync we will want to do this but not in general. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jgrier commented on issue #7099: [FLINK-10887] [jobmaster] Add source watermark tracking to the JobMaster
jgrier commented on issue #7099: [FLINK-10887] [jobmaster] Add source watermark tracking to the JobMaster URL: https://github.com/apache/flink/pull/7099#issuecomment-445316366 > Can we always assume that the user-jar/class loader will be available where the `AggregateFunction` is needed? If yes, I think this is a nice approach! (We can probably use concrete types in the interface, though) @aljoscha I'm actually not sure if the user code classloader is available from the JobMaster but I would think that's reasonable since there's a 1:1 relationship between the JobMaster and a single job. WRT concrete types in the RPC interface I'm not sure what you're thinking there. The concrete types are not known in this approach. The types are up to the user/client and can be different for each named aggregate. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jgrier commented on issue #7099: [FLINK-10887] [jobmaster] Add source watermark tracking to the JobMaster
jgrier commented on issue #7099: [FLINK-10887] [jobmaster] Add source watermark tracking to the JobMaster URL: https://github.com/apache/flink/pull/7099#issuecomment-444907282 @aljoscha @tweise Can you guys comment on the above proposal? I'd like to keep this moving forward. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jgrier commented on issue #7099: [FLINK-10887] [jobmaster] Add source watermark tracking to the JobMaster
jgrier commented on issue #7099: [FLINK-10887] [jobmaster] Add source watermark tracking to the JobMaster URL: https://github.com/apache/flink/pull/7099#issuecomment-443739811 Sorry I haven't responded to this. We had a baby boy this week so that has kept me pretty busy ;) Okay, so I'm on board with generifying this further. @StephanEwen if we're to do a generic transient aggregator do you mean to allow the client to provide the aggregation function? In this case the API would look something like this: `CompletableFuture updateAggregate( String aggregateName, T aggregand, BiFunction aggregationFunction);` Is this what you had in mind? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
