[jira] [Commented] (SPARK-23970) pyspark - simple filter/select doesn't use all tasks when coalesce is set

2018-04-13 Thread Matthew Anthony (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438188#comment-16438188 ] Matthew Anthony commented on SPARK-23970: - Understood but this isnhorribly inefficient. In the

[jira] [Updated] (SPARK-23970) pyspark - simple filter/select doesn't use all tasks when coalesce is set

2018-04-12 Thread Matthew Anthony (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Anthony updated SPARK-23970: Description: Running in (py)spark 2.2.  Marking this as PySpark, but have not confirmed

[jira] [Updated] (SPARK-23970) pyspark - simple filter/select doesn't use all tasks when coalesce is set

2018-04-12 Thread Matthew Anthony (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Anthony updated SPARK-23970: Description: Running in (py)spark 2.2.  Marking this as PySpark, but have not confirmed

[jira] [Created] (SPARK-23970) pyspark - simple filter/select doesn't use all tasks when coalesce is set

2018-04-12 Thread Matthew Anthony (JIRA)
Matthew Anthony created SPARK-23970: --- Summary: pyspark - simple filter/select doesn't use all tasks when coalesce is set Key: SPARK-23970 URL: https://issues.apache.org/jira/browse/SPARK-23970