[jira] [Commented] (SPARK-25491) pandas_udf(GROUPED_MAP) fails when using ArrayType(ArrayType(DoubleType()))

[ https://issues.apache.org/jira/browse/SPARK-25491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624977#comment-16624977 ] Ofer Fridman commented on SPARK-25491: -- [~hyukjin.kwon], here is both the exception trace and the

[jira] [Resolved] (SPARK-25512) Using RowNumbers in SparkR Dataframe

[ https://issues.apache.org/jira/browse/SPARK-25512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25512. -- Resolution: Invalid Questions should go to mailing list (see

[jira] [Commented] (SPARK-25511) Map with "null" key not working in spark 2.3

[ https://issues.apache.org/jira/browse/SPARK-25511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624972#comment-16624972 ] Hyukjin Kwon commented on SPARK-25511: -- ping [~ravi_b_shankar], I would resolve this ticket if

[jira] [Updated] (SPARK-25491) pandas_udf(GROUPED_MAP) fails when using ArrayType(ArrayType(DoubleType()))

[ https://issues.apache.org/jira/browse/SPARK-25491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25491: - Summary: pandas_udf(GROUPED_MAP) fails when using ArrayType(ArrayType(DoubleType()))(was:

[jira] [Commented] (SPARK-25491) pandas_udf fails when using ArrayType(ArrayType(DoubleType()))

[ https://issues.apache.org/jira/browse/SPARK-25491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624971#comment-16624971 ] Hyukjin Kwon commented on SPARK-25491: -- Please specify Pandas version as well. Also mind if I ask

[jira] [Resolved] (SPARK-25506) Spark CSV multiline with CRLF

[ https://issues.apache.org/jira/browse/SPARK-25506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25506. -- Resolution: Duplicate > Spark CSV multiline with CRLF > - > >

[jira] [Assigned] (SPARK-25473) PySpark ForeachWriter test fails on Python 3.6 and macOS High Serria

[ https://issues.apache.org/jira/browse/SPARK-25473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25473: Assignee: Hyukjin Kwon > PySpark ForeachWriter test fails on Python 3.6 and macOS High

[jira] [Resolved] (SPARK-25473) PySpark ForeachWriter test fails on Python 3.6 and macOS High Serria

[ https://issues.apache.org/jira/browse/SPARK-25473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25473. -- Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22480

[jira] [Comment Edited] (SPARK-25480) Dynamic partitioning + saveAsTable with multiple partition columns create empty directory

[ https://issues.apache.org/jira/browse/SPARK-25480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624913#comment-16624913 ] Christopher Burns edited comment on SPARK-25480 at 9/23/18 2:54 AM:

[jira] [Commented] (SPARK-25480) Dynamic partitioning + saveAsTable with multiple partition columns create empty directory

[ https://issues.apache.org/jira/browse/SPARK-25480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624913#comment-16624913 ] Christopher Burns commented on SPARK-25480: --- I can confirm this happens with Spark 2.3 / HDFS

[jira] [Commented] (SPARK-25460) DataSourceV2: Structured Streaming does not respect SessionConfigSupport

[ https://issues.apache.org/jira/browse/SPARK-25460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624909#comment-16624909 ] Apache Spark commented on SPARK-25460: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-25460) DataSourceV2: Structured Streaming does not respect SessionConfigSupport

[ https://issues.apache.org/jira/browse/SPARK-25460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624908#comment-16624908 ] Apache Spark commented on SPARK-25460: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-25460) DataSourceV2: Structured Streaming does not respect SessionConfigSupport

[ https://issues.apache.org/jira/browse/SPARK-25460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624907#comment-16624907 ] Apache Spark commented on SPARK-25460: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-25513) Read zipped CSV and JSON files

[ https://issues.apache.org/jira/browse/SPARK-25513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25513: Assignee: (was: Apache Spark) > Read zipped CSV and JSON files >

[jira] [Commented] (SPARK-25513) Read zipped CSV and JSON files

[ https://issues.apache.org/jira/browse/SPARK-25513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624818#comment-16624818 ] Apache Spark commented on SPARK-25513: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25513) Read zipped CSV and JSON files

[ https://issues.apache.org/jira/browse/SPARK-25513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25513: Assignee: Apache Spark > Read zipped CSV and JSON files > --

[jira] [Created] (SPARK-25513) Read zipped CSV and JSON files

Maxim Gekk created SPARK-25513: -- Summary: Read zipped CSV and JSON files Key: SPARK-25513 URL: https://issues.apache.org/jira/browse/SPARK-25513 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-17952) SparkSession createDataFrame method throws exception for nested JavaBeans

[ https://issues.apache.org/jira/browse/SPARK-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17952: Assignee: Apache Spark > SparkSession createDataFrame method throws exception for nested

[jira] [Assigned] (SPARK-17952) SparkSession createDataFrame method throws exception for nested JavaBeans

[ https://issues.apache.org/jira/browse/SPARK-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17952: Assignee: (was: Apache Spark) > SparkSession createDataFrame method throws exception

[jira] [Commented] (SPARK-17952) SparkSession createDataFrame method throws exception for nested JavaBeans

[ https://issues.apache.org/jira/browse/SPARK-17952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624791#comment-16624791 ] Apache Spark commented on SPARK-17952: -- User 'michalsenkyr' has created a pull request for this

[jira] [Commented] (SPARK-24794) DriverWrapper should have both master addresses in -Dspark.master

[ https://issues.apache.org/jira/browse/SPARK-24794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624790#comment-16624790 ] Behroz Sikander commented on SPARK-24794: - Can someone please have a look at this PR? >

[jira] [Resolved] (SPARK-25465) Refactor Parquet test suites in project Hive

[ https://issues.apache.org/jira/browse/SPARK-25465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25465. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 2.5.0 > Refactor Parquet test

[jira] [Commented] (SPARK-24486) Slow performance reading ArrayType columns

[ https://issues.apache.org/jira/browse/SPARK-24486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624723#comment-16624723 ] Yuming Wang commented on SPARK-24486: - I can reproduce this issue. I'm working on. > Slow

[jira] [Updated] (SPARK-25512) Using RowNumbers in SparkR Dataframe

[ https://issues.apache.org/jira/browse/SPARK-25512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif Khan updated SPARK-25512: -- Issue Type: Question (was: Bug) > Using RowNumbers in SparkR Dataframe >

[jira] [Created] (SPARK-25512) Using RowNumbers in SparkR Dataframe

Asif Khan created SPARK-25512: - Summary: Using RowNumbers in SparkR Dataframe Key: SPARK-25512 URL: https://issues.apache.org/jira/browse/SPARK-25512 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-25502) [Spark Job History] Empty Page when page number exceeds the reatinedTask size

[ https://issues.apache.org/jira/browse/SPARK-25502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624696#comment-16624696 ] Apache Spark commented on SPARK-25502: -- User 'shahidki31' has created a pull request for this

[jira] [Commented] (SPARK-25502) [Spark Job History] Empty Page when page number exceeds the reatinedTask size

[ https://issues.apache.org/jira/browse/SPARK-25502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624695#comment-16624695 ] shahid commented on SPARK-25502: I have raised PR https://github.com/apache/spark/pull/22502 > [Spark

[jira] [Assigned] (SPARK-25502) [Spark Job History] Empty Page when page number exceeds the reatinedTask size

[ https://issues.apache.org/jira/browse/SPARK-25502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25502: Assignee: Apache Spark > [Spark Job History] Empty Page when page number exceeds the

[jira] [Assigned] (SPARK-25502) [Spark Job History] Empty Page when page number exceeds the reatinedTask size

[ https://issues.apache.org/jira/browse/SPARK-25502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25502: Assignee: (was: Apache Spark) > [Spark Job History] Empty Page when page number

[jira] [Commented] (SPARK-25503) [Spark Job History] Total task message in stage page is ambiguous

[ https://issues.apache.org/jira/browse/SPARK-25503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624685#comment-16624685 ] Apache Spark commented on SPARK-25503: -- User 'shahidki31' has created a pull request for this

[jira] [Commented] (SPARK-25503) [Spark Job History] Total task message in stage page is ambiguous

[ https://issues.apache.org/jira/browse/SPARK-25503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624684#comment-16624684 ] Apache Spark commented on SPARK-25503: -- User 'shahidki31' has created a pull request for this

[jira] [Commented] (SPARK-25503) [Spark Job History] Total task message in stage page is ambiguous

[ https://issues.apache.org/jira/browse/SPARK-25503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624683#comment-16624683 ] shahid commented on SPARK-25503: I have raised the PR https://github.com/apache/spark/pull/22525 >

[jira] [Assigned] (SPARK-25503) [Spark Job History] Total task message in stage page is ambiguous

[ https://issues.apache.org/jira/browse/SPARK-25503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25503: Assignee: Apache Spark > [Spark Job History] Total task message in stage page is

[jira] [Assigned] (SPARK-25503) [Spark Job History] Total task message in stage page is ambiguous

[ https://issues.apache.org/jira/browse/SPARK-25503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25503: Assignee: (was: Apache Spark) > [Spark Job History] Total task message in stage page

[jira] [Commented] (SPARK-25511) Map with "null" key not working in spark 2.3

[ https://issues.apache.org/jira/browse/SPARK-25511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624642#comment-16624642 ] Michal Šenkýř commented on SPARK-25511: --- Null keys in MapType should have been disallowed in 2.1.x

[jira] [Commented] (SPARK-24018) Spark-without-hadoop package fails to create or read parquet files with snappy compression

[ https://issues.apache.org/jira/browse/SPARK-24018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624628#comment-16624628 ] Yuming Wang commented on SPARK-24018: - It may be fixed by

[jira] [Commented] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624579#comment-16624579 ] Apache Spark commented on SPARK-25497: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25497: Assignee: (was: Apache Spark) > limit operation within whole stage codegen should

[jira] [Assigned] (SPARK-25497) limit operation within whole stage codegen should not consume all the inputs

[ https://issues.apache.org/jira/browse/SPARK-25497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25497: Assignee: Apache Spark > limit operation within whole stage codegen should not consume

[jira] [Commented] (SPARK-25501) Kafka delegation token support

[ https://issues.apache.org/jira/browse/SPARK-25501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624511#comment-16624511 ] Jungtaek Lim commented on SPARK-25501: -- +1 for this. This should help remedy the needs to pass

[jira] [Comment Edited] (SPARK-10816) EventTime based sessionization

[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623749#comment-16623749 ] Jungtaek Lim edited comment on SPARK-10816 at 9/22/18 6:00 AM: ---