Github user squito commented on the pull request:
https://github.com/apache/spark/pull/4435#issuecomment-85712794
This is mostly ready to go. I need to add a test for the security of the
history server, but I'm not sure how to do that. any suggestions?
I believe this addresses all the previous concerns. I jammed some tests of
the json into `UISeleniumSuite`, b/c it was convenient for reusing the test
cases. If you think this makes that test too confusing I can refactor into two
independent test suites. Also those tests didn't currently have any tests for
the behavior when jobs & stages fell off the "retained" set, so I added some
tests, and noticed some slightly inconsistent behavior in the UI which I
updated. The behavior is still a little strange if some of the stages for a
job get purged, the job displays with limited info, but I guess that's OK.
I think its also worth thinking a bit about the structure of the routes --
do we have a reasonable set of endpoints, with good query parameters, etc. (at
least as a starting point)? Eg., how much detail goes into the call to list
all stages, vs. the details of one stage? Do we need a separate call to get
all attempts for one stage, or is just stage list + stage attempt details
enough? etc.
The failing binary compatibility is b/c of `ExecutorSummaryInfo`, which was
`private[org.apache.spark.ui]`. I suppose this could leak in java code -- does
this matter? Or can this be ignored as a false-positive?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
