Github user squito commented on the pull request:
https://github.com/apache/spark/pull/4435#issuecomment-77946570
Hi @JoshRosen , thanks for the review. Sorry for all the style details
wrong -- I will take a closer look at all the code, just wanted to get some
validation of the overall approach. Some answers to specific questions:
* It works almost exactly the same in the HistoryServer & the live UI,
though to be honest I've only done very light testing of the live UI. Also I
mentioned one issue on the JIRA -- I'd like the URLs the same in the live UI &
the history server, but this does mean you need to put in the `app_id` even
when you're dealing w/ the live UI.
* I have a few test cases cover missing resources & response codes for the
history server (looks like I didn't push them all, sorry ...), but I suppose
there should probably be more for the live UI. I certainly hadn't thought
about the live UI purging old jobs & stages.
* Its a good question about jackson -- the dependency was already
introduced here: https://github.com/apache/spark/commit/1390e56f , but it might
still be worth considering. Just anecdotally, I've mostly been able to upgrade
jackson in the past but I've occasionally run into issues when upgrading
jackson, and having different versions of the various jackson modules.
* Good point on the case classes -- I will change that
* also agreed on the other style issues, I will clean those things up.
Also I've got another version which uses Jersey for the internal routing,
it cleans things up. I'll update with your feedback and ping you here -- will
be a few days as I am in the weeds on another issue.
thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
