Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/19270
ok trying this out it is taking a lot longer then I expected. retrieving
the data is relatively quick (2 seconds for 40000 tasks) but then is taking a
lot longer to parse and then display. Total time for 40000 was 17 seconds.
Looking at the code itself we seem to be doing a lot of loops over the
data. If we can fix that to only make one pass over it, it would be a lot
better. Or we change to do server side.
Either way it sounds like this needs more work and if we have other jira
with large changes perhaps we should hold off on this until some of that gets
in.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
