Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/19270
@vanzin I haven't followed the prs from SPARK-18085 closely, you keep
mentioning it here but don't give anymore context as to what that has to do
with this change. Did you change the format of tables in there? Please point
to specifics.
Yes there is obviously a trade off here since its loading all the data at
first. This is useful in many situations but not in others. the memory usage
can be an issue but isn't worse then anyone hitting the rest api directly
(other then your browser side). Personally I hate the tables as is now, they
are not friendly at all when looking for things in the task tables so we need
to improve them somehow.
If you have other design ideas for this perhaps you can talk about those.
There are definitely other approaches/frameworks, we can definitely improve
upon this to make things happen more on the server side if its needed and we
support in the rest api. We could also change back away from the rest api, but
I thought that is the way we were wanting to go but maybe that has changed
again?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
