Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
I can't commit on the timelines. I don't want to block things at my end.
-Sameer.
On Thu, Feb 16, 2017 at 1:23 AM, Marcelo Vanzin
wrote:
> @samkum <ht
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Hi Marcelo,
Are you referring to following commit.
5645533(
https://github.com/apache/spark/pull/16387/commits/5645533ede8bb37aa316afbc0d003d042bf5d52d
)
-Sameer
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Please share the commit id...i think that will make it clear. I can't share
the code as its propriety to organization,. but i am just reducing a map phase,
reduce by key and reducebykeyandw
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Hi liang,
I got pulled into another project. Will need few more days to catch up on
bandwidth.
Just for confirmation, I will testing with changes just to TaskManager
class
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
@mridulm <https://github.com/mridulm> - need one clarification: if I test
it with out ut @viirya <https://github.com/viirya> commit b1ef9ec,wouldnt
the commit fail at the assert
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
ok..I will get back to you in next couple of days.
-Sameer.
On Thu, Jan 26, 2017 at 3:56 AM, Marcelo Vanzin
wrote:
> Nope, I didn't tested it in i
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Nope, I didn't tested it in isolation.
-Sameer.
On Jan 24, 2017 10:09 PM, "Marcelo Vanzin" wrote:
> No the question is whether you tested without @v
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
Hi,
No, I havent test this with out the ExternalAppendOnlyMap fix if this is
what you are asking. I tested it with changes to both the classes:
TaskMemoryManager and
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16603
Combined with 16387.
On Jan 18, 2017 3:54 PM, "Liang-Chi Hsieh" wrote:
> @samkum <https://github.com/samkum> Just this fix or combined with #16387
> <
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16603
This fix also has GC issues. Please find the screenshot attached.
![gc](https://cloud.githubusercontent.com/assets/278918/22057886/b90224e6-dd8c-11e6-91e1-e3b34f461c68.png)
---
If your
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
The fix is better but still produces visible GC issues. After removing the
fix, GC is no longer a problem.
![gc](https://cloud.githubusercontent.com/assets/278918/22021367/e81f96a0-dce2
Github user samkum commented on the issue:
https://github.com/apache/spark/pull/16387
I have tested this, but I found a very strange observation. GC frequency
has increased many folds...and majority of the time is spend in GC.
---
If your project is set up for it, you can reply to
12 matches
Mail list logo