Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/19270
> from my understanding SPARK-20657 is proposing to not using the rest api
Not really; I just don't know enough JS to actually make the change to use
the REST api, but I do believe using it is a better approach.
The main thing about that change is to generate the view the client wants
without having the whole data set in memory (either in the client or server
side).
> the question is do we want to do it now with this pr, or...
I'm not really against the idea of the changes in this PR, but if we're
changing things, I'd like at least a little more thought put into how to do
things right. I don't think the "transfer everything from the server to the
client" approach is the right thing to do in the long term; it will make
looking at small stages better, yes, but it will make looking at larger stages
worse. Even if afterwards it's quicker to resort columns, I have little
patience to wait 1min for a page to load. And so do not others who have filed
bugs about that kind of thing.
Another point I'd like to make, and this is selfish me talking, is that
large changes in these areas really slow down progress in SPARK-18085.
Everybody tries to make local fixes to the problems I've tried to holistically
fix in that project, and very few people seem interested in reviewing the code
I've been putting out. I believe the changes in that project will make changes
like this a lot easier, so I'd really appreciate getting those changes reviewed
and committed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
