Baunsgaard commented on PR #1726: URL: https://github.com/apache/systemds/pull/1726#issuecomment-1315411665
> > Currently we only have the federated monitoring tool, > > we could potentially add features to this to make it more general purpose, for the main system as well. > > I think it would be best if we consolidate such monitoring / control interfaces in the same tool. > > It could potentially be a very useful feature similar to the SparkUI. > > It makes sense to want to "unify" all monitoring in a single monitoring service. It would be amazing to have something similar to Spark/Flink UI eventually. > > Re your original point, we have only 2 options: > > 1. Synchronous calls to a `/health` endpoint (polling) > > 2. Async call using something like `java.util.concurrent.Future` (callback) > > > In either case, we will need a timeout. After all, we don't want to wait indefinitely for the service to be up and running. > > [1] is easier to implement and maintain IMO. [2] is equally valid though, and I think it really depends on whether somebody wants to implement it. In the latter case, it might make sense to create a JIRA story and tag it appropriately so people can discover it easily and work on it. I am in general more a fan of the callback strategy, since this would allow the started processes to call, once ready rather than us repeatedly asking "are you ready" with polling :P. I guess an easy design now that i think about it is to add a flag to make SystemDS send a HTTP message to an address once it is up. Then we would only need to make a REST API on the testing side that listens for ready workers, coordinators, monitoring services, etc. All that said, i am about to close this PR, and i like the discussion so we should perhaps move it to the dev mailing list. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
