Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/2742#issuecomment-58728319
Note: this is reqd since there are heap and vm limits enforced, so we
juggle available memory around so that jobs can run to completion!
On 11-Oct-2014 4:56 am, "Mridul Muralidharan" <[email protected]> wrote:
> With 1.1, in expts, we have done both : depending on whether our user code
> is mmap'ing too much data (and so we pull things into heap .. using
> libraries not in our control :-) ); decreasing it when heap is at premium.
> On 11-Oct-2014 4:42 am, "Aaron Davidson" <[email protected]> wrote:
>
>> @mridulm <https://github.com/mridulm> Could you give an example of which
>> way you would want to shift it via config? Map more or less often?
>>
>> â
>> Reply to this email directly or view it on GitHub
>> <https://github.com/apache/spark/pull/2742#issuecomment-58727337>.
>>
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
