Github user rekhajoshm commented on the pull request:
https://github.com/apache/spark/pull/7602#issuecomment-127066526
Thanks @andrewor14 for your review comments. done. What you say is correct.
That said, in stack recursion it is simply bad practice to create more objects
than absolutely necessary., as it is loophole for trouble.As its not just heap,
even stack overflow caused due to recursion with job depth can camouflage as
heap/OOM issue.I did some initial changes, and post great review comments from
@srowen even removed the original extra string builder created on every
recursion.Functionality being constant, if you do a load test with different
complexity of jobs that can be tested across possible configurations and check
profiler, you would see the improvement.
Please review approve @andrewor14 thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
