[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-10-14 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951165#comment-16951165 ] Min Shen commented on SPARK-21492: -- Want to further clarify the scope of the fix in PR 

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-10-14 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951154#comment-16951154 ] Min Shen commented on SPARK-21492: -- We have deployed the latest version of the PR in 

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-09-21 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935176#comment-16935176 ] Min Shen commented on SPARK-21492: -- We also saw this issue happening in our cluster. Based on the

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-09-11 Thread zhoukang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928204#comment-16928204 ] zhoukang commented on SPARK-21492: -- Any progress of this issue? [~jiangxb1987] We also encountered this

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-05-20 Thread Xingbo Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844313#comment-16844313 ] Xingbo Jiang commented on SPARK-21492: -- I'm working on this, will post a simple design doc in a few

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-04-29 Thread Tao Luo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829545#comment-16829545 ] Tao Luo commented on SPARK-21492: - The problem is that the task won't complete because of memory being

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-03-18 Thread Xiaoju Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794962#comment-16794962 ] Xiaoju Wu commented on SPARK-21492: --- Any updates? Do you have any discussion on the general fix

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-02-27 Thread Guruprasad (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779460#comment-16779460 ] Guruprasad commented on SPARK-21492: i ran the sample code provided on spark 2.0.2 and it used

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-02-21 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773861#comment-16773861 ] Wenchen Fan commented on SPARK-21492: - [~taoluo] Thanks for the detailed explanation! I kind of

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-02-12 Thread Tao Luo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766598#comment-16766598 ] Tao Luo commented on SPARK-21492: - cc [~tejasp], [~kiszk] for input on code generation to address the

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-02-11 Thread Tao Luo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765378#comment-16765378 ] Tao Luo commented on SPARK-21492: - I'll take a stab at this jira, should have something to review today

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-02-08 Thread Tao Luo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763825#comment-16763825 ] Tao Luo commented on SPARK-21492: - Can someone add 'affects version' 2.4.0 as well?  > Memory leak in

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-02-08 Thread Tao Luo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763820#comment-16763820 ] Tao Luo commented on SPARK-21492: - If SortMergeJoinScanner doesn't consume the iterator from

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2017-10-20 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212522#comment-16212522 ] Bartosz Mścichowski commented on SPARK-21492: - Here's a script that exposes memory leak

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2017-07-20 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095425#comment-16095425 ] Zhan Zhang commented on SPARK-21492: root cause: In the SortMergeJoin, inner/leftOuter/rightOuter,

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2017-07-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095418#comment-16095418 ] Apache Spark commented on SPARK-21492: -- User 'zhzhan' has created a pull request for this issue: