HeartSaVioR commented on pull request #35548: URL: https://github.com/apache/spark/pull/35548#issuecomment-1047398979
I'm not heavily involved on Spark UI - I'm afraid I'm the right person to make a decision about exposing REST APIs which are not used directly in Spark. My general concern is that whether we consider REST APIs as public API or not. If we consider it as public API, we will have to worry about compatibility on `StreamingQueryData` and `StreamingQueryProcess`, since they are directly exposed to the REST API without any versioning. We wouldn't like to get blocked to change these models just because we expose them via REST API. Versioning would add non-trivial additional load to us - whenever we change the models, we have to retain the all models with versions so that we can serve all clients without having them to change their logic. The realistic approach might be considering the REST APIs as developer API, meaning the output would be subject to change according to Spark version. I'd like to hear the actual use case of new REST APIs to see whether this could be acceptable or not, and also see we have alternative (like dropwizard metrics) or not. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
