[jira] [Updated] (SPARK-26113) TypeError: object of type 'NoneType' has no len() in authenticate_and_accum_updates of pyspark/accumulators.py

2018-11-19 Thread Sai Varun Reddy Daram (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Varun Reddy Daram updated SPARK-26113: -- Priority: Critical (was: Major) > TypeError: object of type 'NoneType' has no

[jira] [Updated] (SPARK-26113) TypeError: object of type 'NoneType' has no len() in authenticate_and_accum_updates of pyspark/accumulators.py

2018-11-19 Thread Sai Varun Reddy Daram (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Varun Reddy Daram updated SPARK-26113: -- Priority: Major (was: Critical) > TypeError: object of type 'NoneType' has no

[jira] [Commented] (SPARK-26113) TypeError: object of type 'NoneType' has no len() in authenticate_and_accum_updates of pyspark/accumulators.py

2018-11-19 Thread Sai Varun Reddy Daram (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691381#comment-16691381 ] Sai Varun Reddy Daram commented on SPARK-26113: --- [~hyukjin.kwon] sorry for

[jira] [Updated] (SPARK-26113) TypeError: object of type 'NoneType' has no len() in authenticate_and_accum_updates of pyspark/accumulators.py

2018-11-19 Thread Sai Varun Reddy Daram (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Varun Reddy Daram updated SPARK-26113: -- Description: Machine OS: Ubuntu 16.04. Kubernetes: Minikube  Kubernetes Vers

[jira] [Created] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Sergey Zhemzhitsky (JIRA)
Sergey Zhemzhitsky created SPARK-26114: -- Summary: Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions Key: SPARK-26114 URL: https://issues.apache.org/jira/browse/SPAR

[jira] [Updated] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Sergey Zhemzhitsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-26114: --- Attachment: run1-noparams-dominator-tree.png > Memory leak of PartitionedPairBuffer

[jira] [Updated] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Sergey Zhemzhitsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-26114: --- Attachment: run1-noparams-dominator-tree-externalsorter.png > Memory leak of Partiti

[jira] [Comment Edited] (SPARK-26113) TypeError: object of type 'NoneType' has no len() in authenticate_and_accum_updates of pyspark/accumulators.py

2018-11-19 Thread Sai Varun Reddy Daram (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691330#comment-16691330 ] Sai Varun Reddy Daram edited comment on SPARK-26113 at 11/19/18 9:33 AM: -

[jira] [Issue Comment Deleted] (SPARK-20236) Overwrite a partitioned data source table should only overwrite related partitions

2018-11-19 Thread Mikalai Surta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikalai Surta updated SPARK-20236: -- Comment: was deleted (was: Could you please explain why it doesn't work as I expect in my case

[jira] [Commented] (SPARK-20236) Overwrite a partitioned data source table should only overwrite related partitions

2018-11-19 Thread Mikalai Surta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691472#comment-16691472 ] Mikalai Surta commented on SPARK-20236: --- [~cloud_fan] It was my fault, works fine,

[jira] [Updated] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Sergey Zhemzhitsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-26114: --- Attachment: run1-noparams-dominator-tree-externalsorter-gc-root.png > Memory leak of

[jira] [Updated] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Sergey Zhemzhitsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-26114: --- Description: Trying to use _coalesce_ after shuffle-oriented transformations leads t

[jira] [Updated] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Sergey Zhemzhitsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-26114: --- Description: Trying to use _coalesce_ after shuffle-oriented transformations leads t

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Sai Varun Reddy Daram (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691462#comment-16691462 ] Sai Varun Reddy Daram commented on SPARK-26019: --- Any help with https://iss

[jira] [Updated] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Sergey Zhemzhitsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-26114: --- Description: Trying to use _coalesce_ after shuffle-oriented transformations leads t

[jira] [Commented] (SPARK-25380) Generated plans occupy over 50% of Spark driver memory

2018-11-19 Thread Dave DeCaprio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691519#comment-16691519 ] Dave DeCaprio commented on SPARK-25380: --- I would like to comment that we are also

[jira] [Resolved] (SPARK-26113) TypeError: object of type 'NoneType' has no len() in authenticate_and_accum_updates of pyspark/accumulators.py

2018-11-19 Thread Sai Varun Reddy Daram (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Varun Reddy Daram resolved SPARK-26113. --- Resolution: Invalid > TypeError: object of type 'NoneType' has no len() in

[jira] [Assigned] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26114: Assignee: Apache Spark > Memory leak of PartitionedPairBuffer when coalescing after > re

[jira] [Commented] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691575#comment-16691575 ] Apache Spark commented on SPARK-26114: -- User 'szhem' has created a pull request for

[jira] [Assigned] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26114: Assignee: (was: Apache Spark) > Memory leak of PartitionedPairBuffer when coalescing

[jira] [Created] (SPARK-26115) Fix deprecated warnings related to scala 2.12 in core module

2018-11-19 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-26115: -- Summary: Fix deprecated warnings related to scala 2.12 in core module Key: SPARK-26115 URL: https://issues.apache.org/jira/browse/SPARK-26115 Project: Spark

[jira] [Resolved] (SPARK-26115) Fix deprecated warnings related to scala 2.12 in core module

2018-11-19 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-26115. Resolution: Duplicate Duplicated to SPARK-2609 > Fix deprecated warnings related to scala

[jira] [Created] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
Pierre Lienhart created SPARK-26116: --- Summary: Spark SQL - Sort when writing partitioned parquet lead to OOM errors Key: SPARK-26116 URL: https://issues.apache.org/jira/browse/SPARK-26116 Project: S

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {code:java}partitionBy{code}, it looks

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Summary: Spark SQL - Sort when writing partitioned parquet leads to OOM errors (was: Spar

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet lead to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

2018-11-19 Thread Pierre Lienhart (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Updated] (SPARK-26114) Memory leak of PartitionedPairBuffer when coalescing after repartitionAndSortWithinPartitions

2018-11-19 Thread Sergey Zhemzhitsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zhemzhitsky updated SPARK-26114: --- Description: Trying to use _coalesce_ after shuffle-oriented transformations leads t

[jira] [Created] (SPARK-26117) use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception

2018-11-19 Thread caoxuewen (JIRA)
caoxuewen created SPARK-26117: - Summary: use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception Key: SPARK-26117 URL: https://issues.apache.org/jira/browse/SPARK-26117 Project: Spark

[jira] [Assigned] (SPARK-26117) use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26117: Assignee: Apache Spark > use SparkOutOfMemoryError instead of OutOfMemoryError when catch

[jira] [Assigned] (SPARK-26117) use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26117: Assignee: (was: Apache Spark) > use SparkOutOfMemoryError instead of OutOfMemoryError

[jira] [Resolved] (SPARK-26026) Published Scaladoc jars missing from Maven Central

2018-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26026. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23069 [https://github.c

[jira] [Resolved] (SPARK-26043) Make SparkHadoopUtil private to Spark

2018-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26043. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23066 [https://github.c

[jira] [Assigned] (SPARK-26043) Make SparkHadoopUtil private to Spark

2018-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26043: - Assignee: Sean Owen > Make SparkHadoopUtil private to Spark > -

[jira] [Commented] (SPARK-26117) use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691725#comment-16691725 ] Apache Spark commented on SPARK-26117: -- User 'heary-cao' has created a pull request

[jira] [Resolved] (SPARK-26068) ChunkedByteBufferInputStream is truncated by empty chunk

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26068. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23040 [https://gith

[jira] [Assigned] (SPARK-26068) ChunkedByteBufferInputStream is truncated by empty chunk

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26068: --- Assignee: Liu, Linhong > ChunkedByteBufferInputStream is truncated by empty chunk > ---

[jira] [Assigned] (SPARK-26112) Update since versions of new built-in functions.

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26112: --- Assignee: Takuya Ueshin > Update since versions of new built-in functions. > --

[jira] [Resolved] (SPARK-26024) Dataset API: repartitionByRange(...) has inconsistent behaviour

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26024. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23025 [https://gith

[jira] [Assigned] (SPARK-26024) Dataset API: repartitionByRange(...) has inconsistent behaviour

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26024: --- Assignee: Julien Peloton > Dataset API: repartitionByRange(...) has inconsistent behaviour

[jira] [Resolved] (SPARK-26112) Update since versions of new built-in functions.

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26112. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23082 [https://gith

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2018-11-19 Thread Andrew Otto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691787#comment-16691787 ] Andrew Otto commented on SPARK-14492: - I found my issue: we were loading some Hive 1

[jira] [Resolved] (SPARK-26071) disallow map as map key

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26071. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23045 [https://gith

[jira] [Commented] (SPARK-25528) data source V2 API refactoring (batch read)

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691799#comment-16691799 ] Apache Spark commented on SPARK-25528: -- User 'cloud-fan' has created a pull request

[jira] [Updated] (SPARK-25528) data source V2 API refactoring (batch read)

2018-11-19 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25528: Summary: data source V2 API refactoring (batch read) (was: data source V2 read side API refactori

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2018-11-19 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691811#comment-16691811 ] Sunil Rangwani commented on SPARK-14492: [~srowen] Please reopen this. It is n

[jira] [Commented] (SPARK-14492) Spark SQL 1.6.0 does not work with external Hive metastore version lower than 1.2.0; its not backwards compatible with earlier version

2018-11-19 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691828#comment-16691828 ] Sunil Rangwani commented on SPARK-14492: [~ottomata] you are using a CDH version

[jira] [Resolved] (SPARK-26090) Resolve most miscellaneous deprecation and build warnings for Spark 3

2018-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26090. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23065 [https://github.c

[jira] [Commented] (SPARK-23886) update query.status

2018-11-19 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691881#comment-16691881 ] Gabor Somogyi commented on SPARK-23886: --- Thanks [~efim.poberezkin]! May I ask you

[jira] [Commented] (SPARK-26026) Published Scaladoc jars missing from Maven Central

2018-11-19 Thread Long Cao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16691965#comment-16691965 ] Long Cao commented on SPARK-26026: -- [~srowen] thanks for the expedient resolution! It's

[jira] [Created] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Attila Zsolt Piros (JIRA)
Attila Zsolt Piros created SPARK-26118: -- Summary: Make Jetty's requestHeaderSize configurable in Spark Key: SPARK-26118 URL: https://issues.apache.org/jira/browse/SPARK-26118 Project: Spark

[jira] [Updated] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Attila Zsolt Piros (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Zsolt Piros updated SPARK-26118: --- Affects Version/s: (was: 2.3.2) (was: 2.4.0)

[jira] [Commented] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Attila Zsolt Piros (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692018#comment-16692018 ] Attila Zsolt Piros commented on SPARK-26118: I am working on a PR. > Make J

[jira] [Created] (SPARK-26119) Task metrics summary in the stage page should contain only successful tasks metrics

2018-11-19 Thread ABHISHEK KUMAR GUPTA (JIRA)
ABHISHEK KUMAR GUPTA created SPARK-26119: Summary: Task metrics summary in the stage page should contain only successful tasks metrics Key: SPARK-26119 URL: https://issues.apache.org/jira/browse/SPARK-2611

[jira] [Commented] (SPARK-26119) Task metrics summary in the stage page should contain only successful tasks metrics

2018-11-19 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692093#comment-16692093 ] shahid commented on SPARK-26119: Thanks. I am working on it > Task metrics summary in t

[jira] [Updated] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26120: - Component/s: Structured Streaming SparkR > Fix a streaming query leak in Struct

[jira] [Commented] (SPARK-26119) Task metrics summary in the stage page should contain only successful tasks metrics

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692175#comment-16692175 ] Apache Spark commented on SPARK-26119: -- User 'shahidki31' has created a pull reques

[jira] [Updated] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26120: - Priority: Minor (was: Major) > Fix a streaming query leak in Structured Streaming R tests > ---

[jira] [Assigned] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26120: Assignee: Apache Spark (was: Shixiong Zhu) > Fix a streaming query leak in Structured St

[jira] [Assigned] (SPARK-26119) Task metrics summary in the stage page should contain only successful tasks metrics

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26119: Assignee: (was: Apache Spark) > Task metrics summary in the stage page should contain

[jira] [Commented] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692176#comment-16692176 ] Apache Spark commented on SPARK-26120: -- User 'zsxwing' has created a pull request f

[jira] [Assigned] (SPARK-26119) Task metrics summary in the stage page should contain only successful tasks metrics

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26119: Assignee: Apache Spark > Task metrics summary in the stage page should contain only succe

[jira] [Commented] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692178#comment-16692178 ] Apache Spark commented on SPARK-26120: -- User 'zsxwing' has created a pull request f

[jira] [Created] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-26120: Summary: Fix a streaming query leak in Structured Streaming R tests Key: SPARK-26120 URL: https://issues.apache.org/jira/browse/SPARK-26120 Project: Spark Is

[jira] [Assigned] (SPARK-26120) Fix a streaming query leak in Structured Streaming R tests

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26120: Assignee: Shixiong Zhu (was: Apache Spark) > Fix a streaming query leak in Structured St

[jira] [Created] (SPARK-26121) [Structured Streaming] Allow users to define prefix of Kafka's consumer group (group.id)

2018-11-19 Thread Anastasios Zouzias (JIRA)
Anastasios Zouzias created SPARK-26121: -- Summary: [Structured Streaming] Allow users to define prefix of Kafka's consumer group (group.id) Key: SPARK-26121 URL: https://issues.apache.org/jira/browse/SPARK-261

[jira] [Reopened] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Dautkhanov reopened SPARK-26019: --- Reproduced myself > pyspark/accumulators.py: "TypeError: object of type 'NoneType' has

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692219#comment-16692219 ] Ruslan Dautkhanov commented on SPARK-26019: --- [~hyukjin.kwon] today I reproduce

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692226#comment-16692226 ] Ruslan Dautkhanov commented on SPARK-26019: --- Might be broken by https://github

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692233#comment-16692233 ] Ruslan Dautkhanov commented on SPARK-26019: --- Sorry, nope it was broken by this

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692237#comment-16692237 ] Ruslan Dautkhanov commented on SPARK-26019: --- cc [~lucacanali] > pyspark/accu

[jira] [Assigned] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26118: Assignee: (was: Apache Spark) > Make Jetty's requestHeaderSize configurable in Spark

[jira] [Commented] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692246#comment-16692246 ] Apache Spark commented on SPARK-26118: -- User 'attilapiros' has created a pull reque

[jira] [Commented] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692248#comment-16692248 ] Apache Spark commented on SPARK-26118: -- User 'attilapiros' has created a pull reque

[jira] [Assigned] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26118: Assignee: Apache Spark > Make Jetty's requestHeaderSize configurable in Spark > -

[jira] [Created] (SPARK-26122) Support encoding for multiLine in CSV datasource

2018-11-19 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26122: -- Summary: Support encoding for multiLine in CSV datasource Key: SPARK-26122 URL: https://issues.apache.org/jira/browse/SPARK-26122 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-26122) Support encoding for multiLine in CSV datasource

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692305#comment-16692305 ] Apache Spark commented on SPARK-26122: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-26122) Support encoding for multiLine in CSV datasource

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26122: Assignee: Apache Spark > Support encoding for multiLine in CSV datasource > -

[jira] [Commented] (SPARK-26122) Support encoding for multiLine in CSV datasource

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692306#comment-16692306 ] Apache Spark commented on SPARK-26122: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-26122) Support encoding for multiLine in CSV datasource

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26122: Assignee: (was: Apache Spark) > Support encoding for multiLine in CSV datasource > --

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692355#comment-16692355 ] Imran Rashid commented on SPARK-26019: -- [~hyukjin.kwon] you think maybe these two l

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692377#comment-16692377 ] Ruslan Dautkhanov commented on SPARK-26019: --- [~irashid] thanks a lot for looki

[jira] [Commented] (SPARK-26094) Streaming WAL should create parent dirs

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692380#comment-16692380 ] Apache Spark commented on SPARK-26094: -- User 'squito' has created a pull request fo

[jira] [Assigned] (SPARK-26094) Streaming WAL should create parent dirs

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26094: Assignee: Imran Rashid (was: Apache Spark) > Streaming WAL should create parent dirs > -

[jira] [Assigned] (SPARK-26094) Streaming WAL should create parent dirs

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26094: Assignee: Apache Spark (was: Imran Rashid) > Streaming WAL should create parent dirs > -

[jira] [Commented] (SPARK-26094) Streaming WAL should create parent dirs

2018-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692382#comment-16692382 ] Apache Spark commented on SPARK-26094: -- User 'squito' has created a pull request fo

[jira] [Created] (SPARK-26123) Make all MLlib/ML Models public

2018-11-19 Thread Nicholas Resnick (JIRA)
Nicholas Resnick created SPARK-26123: Summary: Make all MLlib/ML Models public Key: SPARK-26123 URL: https://issues.apache.org/jira/browse/SPARK-26123 Project: Spark Issue Type: Wish

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692495#comment-16692495 ] Hyukjin Kwon commented on SPARK-26019: -- I don't understand how the swapping two lin

[jira] [Resolved] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26019. -- Resolution: Cannot Reproduce > pyspark/accumulators.py: "TypeError: object of type 'NoneType'

[jira] [Updated] (SPARK-25919) Date value corrupts when tables are "ParquetHiveSerDe" formatted and target table is Partitioned

2018-11-19 Thread Pawan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pawan updated SPARK-25919: -- Affects Version/s: 2.1.0 > Date value corrupts when tables are "ParquetHiveSerDe" formatted and target > tabl

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692498#comment-16692498 ] Hyukjin Kwon commented on SPARK-26019: -- Please reopen if this is still being reprod

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-19 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692504#comment-16692504 ] Liang-Chi Hsieh commented on SPARK-26019: - Isn't the server beginning to handle

  1   2   >