Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Hi,
No, I havent test this with out the ExternalAppendOnlyMap fix if this is
what you are asking. I tested it with changes to both the classes:
TaskMemoryManager and ExternalAppendOnlyMap.
-Sameer.
On Tue, Jan 24, 2017 at 3:17 AM, Marcelo Vanzin <[email protected]>
wrote:
> @samkum <https://github.com/samkum> can you reply to the questions above?
> This PR is basically blocked on your feedback that it makes things worse.
>
> @samkum <https://github.com/samkum> did you try this patch without the
> "spill map on every call" behavior (basically returning "false" instead of
> spilling the in-memory map when the iterator doesn't exist)?
>
> If that case also results in increased gc, maybe your app just needs more
> memory. That would be the case where the assert would trigger before.
>
> â
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/16387#issuecomment-274627803>, or
mute
> the thread
>
<https://github.com/notifications/unsubscribe-auth/AARBhqyMAq1IRIMLHFwU5atf2IYdWs88ks5rVSALgaJpZM4LUlyt>
> .
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
