[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528038#comment-15528038 ] Apache Spark commented on SPARK-17666: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527994#comment-15527994 ] Reynold Xin commented on SPARK-17666: - This still needs to be backported to branch-2.0. > take() or

[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524108#comment-15524108 ] Apache Spark commented on SPARK-17666: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523967#comment-15523967 ] Sean Owen commented on SPARK-17666: --- Agree, it might happen to help some cases or even this one but

[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-26 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523921#comment-15523921 ] Josh Rosen commented on SPARK-17666: I think that one problem with that approach is that any

[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523799#comment-15523799 ] Sean Owen commented on SPARK-17666: --- Hm, I wonder if a couple problems of this form could be solved by

[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-26 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523742#comment-15523742 ] Josh Rosen commented on SPARK-17666: My hunch is that there's cleanup which is performed in a

[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-26 Thread Igor Berman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523303#comment-15523303 ] Igor Berman commented on SPARK-17666: - sorry, still no idea. What's the difference between take and

[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523290#comment-15523290 ] Sean Owen commented on SPARK-17666: --- No, I mean what about take() do you believe leaks a connection? is

[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-26 Thread Igor Berman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523286#comment-15523286 ] Igor Berman commented on SPARK-17666: - [~sowen], yes, I believe so rdd.isEmpty() uses take inside as

[jira] [Commented] (SPARK-17666) take() or isEmpty() on dataset leaks s3a connections

2016-09-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523280#comment-15523280 ] Sean Owen commented on SPARK-17666: --- Where are you saying the leak is, from take()? > take() or