[jira] [Resolved] (SPARK-24101) MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-24101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24101. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 17086

[jira] [Assigned] (SPARK-25998) TorrentBroadcast holds strong reference to broadcast object

[ https://issues.apache.org/jira/browse/SPARK-25998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25998: Assignee: Apache Spark > TorrentBroadcast holds strong reference to broadcast object >

[jira] [Commented] (SPARK-25998) TorrentBroadcast holds strong reference to broadcast object

[ https://issues.apache.org/jira/browse/SPARK-25998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682017#comment-16682017 ] Apache Spark commented on SPARK-25998: -- User 'bkrieger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25998) TorrentBroadcast holds strong reference to broadcast object

[ https://issues.apache.org/jira/browse/SPARK-25998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25998: Assignee: (was: Apache Spark) > TorrentBroadcast holds strong reference to broadcast

[jira] [Assigned] (SPARK-25999) make-distribution.sh failure with --r and -Phadoop-provided

[ https://issues.apache.org/jira/browse/SPARK-25999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25999: Assignee: (was: Apache Spark) > make-distribution.sh failure with --r and

[jira] [Commented] (SPARK-25999) make-distribution.sh failure with --r and -Phadoop-provided

[ https://issues.apache.org/jira/browse/SPARK-25999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682129#comment-16682129 ] Apache Spark commented on SPARK-25999: -- User 'shanyu' has created a pull request for this issue:

[jira] [Commented] (SPARK-25999) make-distribution.sh failure with --r and -Phadoop-provided

[ https://issues.apache.org/jira/browse/SPARK-25999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682126#comment-16682126 ] Apache Spark commented on SPARK-25999: -- User 'shanyu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25999) make-distribution.sh failure with --r and -Phadoop-provided

[ https://issues.apache.org/jira/browse/SPARK-25999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25999: Assignee: Apache Spark > make-distribution.sh failure with --r and -Phadoop-provided >

[jira] [Updated] (SPARK-26000) Missing block when reading HDFS Data from Cloudera Manager

[ https://issues.apache.org/jira/browse/SPARK-26000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] john updated SPARK-26000: - Description: I am able to write to Cloudera Manager HDFS through Open Source Spark which runs separately. but

[jira] [Commented] (SPARK-24229) Upgrade to the latest Apache Thrift 0.10.0 release

[ https://issues.apache.org/jira/browse/SPARK-24229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682137#comment-16682137 ] Dongjoon Hyun commented on SPARK-24229: --- I also agree with [~vanzin] 's opinion. Since this is

[jira] [Commented] (SPARK-26000) Missing block when reading HDFS Data from Cloudera Manager

[ https://issues.apache.org/jira/browse/SPARK-26000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682151#comment-16682151 ] Yuming Wang commented on SPARK-26000: - Could you provide more information? > Missing block when

[jira] [Assigned] (SPARK-26001) Reduce memory copy when writing decimal

[ https://issues.apache.org/jira/browse/SPARK-26001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26001: Assignee: (was: Apache Spark) > Reduce memory copy when writing decimal >

[jira] [Resolved] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25102. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 3.0.0 > Write Spark version to

[jira] [Commented] (SPARK-26000) Missing block when reading HDFS Data from Cloudera Manager

[ https://issues.apache.org/jira/browse/SPARK-26000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682259#comment-16682259 ] Yuming Wang commented on SPARK-26000: - It is not a Spark issue, Maybe you need to increase 

[jira] [Commented] (SPARK-25999) make-distribution.sh failure with --r and -Phadoop-provided

[ https://issues.apache.org/jira/browse/SPARK-25999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682122#comment-16682122 ] Yuming Wang commented on SPARK-25999: - Please create pull request at:

[jira] [Commented] (SPARK-26000) Missing block when reading HDFS Data from Cloudera Manager

[ https://issues.apache.org/jira/browse/SPARK-26000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682163#comment-16682163 ] john commented on SPARK-26000: -- I have Cloudera Manager in Environment A which has HDFS component and Spark

[jira] [Created] (SPARK-26001) Reduce memory copy when writing decimal

caoxuewen created SPARK-26001: - Summary: Reduce memory copy when writing decimal Key: SPARK-26001 URL: https://issues.apache.org/jira/browse/SPARK-26001 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-24229) Upgrade to the latest Apache Thrift 0.10.0 release

[ https://issues.apache.org/jira/browse/SPARK-24229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682137#comment-16682137 ] Dongjoon Hyun edited comment on SPARK-24229 at 11/10/18 1:40 AM: - I also

[jira] [Resolved] (SPARK-24229) Upgrade to the latest Apache Thrift 0.10.0 release

[ https://issues.apache.org/jira/browse/SPARK-24229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-24229. --- Resolution: Not A Problem > Upgrade to the latest Apache Thrift 0.10.0 release >

[jira] [Updated] (SPARK-26001) Reduce memory copy when writing decimal

[ https://issues.apache.org/jira/browse/SPARK-26001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-26001: Affects Version/s: (was: 2.5.0) 3.0.0 > Reduce memory copy when

[jira] [Commented] (SPARK-25993) Add test cases for resolution of ORC table location

[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682141#comment-16682141 ] kevin yu commented on SPARK-25993: -- I am looking into it now. Kevin > Add test cases for resolution of

[jira] [Created] (SPARK-26000) Missing block when reading HDFS Data from Cloudera Manager

john created SPARK-26000: Summary: Missing block when reading HDFS Data from Cloudera Manager Key: SPARK-26000 URL: https://issues.apache.org/jira/browse/SPARK-26000 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-26001) Reduce memory copy when writing decimal

[ https://issues.apache.org/jira/browse/SPARK-26001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682238#comment-16682238 ] Apache Spark commented on SPARK-26001: -- User 'heary-cao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26001) Reduce memory copy when writing decimal

[ https://issues.apache.org/jira/browse/SPARK-26001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26001: Assignee: Apache Spark > Reduce memory copy when writing decimal >

[jira] [Commented] (SPARK-21542) Helper functions for custom Python Persistence

[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681891#comment-16681891 ] John Bauer commented on SPARK-21542: {code} from pyspark.sql import SparkSession from

[jira] [Resolved] (SPARK-25958) error: [Errno 97] Address family not supported by protocol in dataframe.take()

[ https://issues.apache.org/jira/browse/SPARK-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Dautkhanov resolved SPARK-25958. --- Resolution: Not A Problem > error: [Errno 97] Address family not supported by

[jira] [Assigned] (SPARK-24101) MulticlassClassificationEvaluator should use sample weight data

[ https://issues.apache.org/jira/browse/SPARK-24101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24101: - Assignee: Ilya Matiach > MulticlassClassificationEvaluator should use sample weight data >

[jira] [Comment Edited] (SPARK-21542) Helper functions for custom Python Persistence

[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681895#comment-16681895 ] John Bauer edited comment on SPARK-21542 at 11/9/18 7:54 PM: - This is a)

[jira] [Commented] (SPARK-21542) Helper functions for custom Python Persistence

[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681895#comment-16681895 ] John Bauer commented on SPARK-21542: This is a) much more minimal, b) genuinely useful, and c)

[jira] [Created] (SPARK-25996) Agregaciones no retornan los valores correctos con rows con timestamps iguales

Ignacio Gómez created SPARK-25996: - Summary: Agregaciones no retornan los valores correctos con rows con timestamps iguales Key: SPARK-25996 URL: https://issues.apache.org/jira/browse/SPARK-25996

[jira] [Created] (SPARK-25998) TorrentBroadcast holds strong reference to broadcast object

Brandon Krieger created SPARK-25998: --- Summary: TorrentBroadcast holds strong reference to broadcast object Key: SPARK-25998 URL: https://issues.apache.org/jira/browse/SPARK-25998 Project: Spark

[jira] [Commented] (SPARK-25999) make-distribution.sh failure with --r and -Phadoop-provided

[ https://issues.apache.org/jira/browse/SPARK-25999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682089#comment-16682089 ] shanyu zhao commented on SPARK-25999: - patch attached. Basically it creates an optional project that

[jira] [Created] (SPARK-25997) Python example code for Power Iteration Clustering in spark.ml

Huaxin Gao created SPARK-25997: -- Summary: Python example code for Power Iteration Clustering in spark.ml Key: SPARK-25997 URL: https://issues.apache.org/jira/browse/SPARK-25997 Project: Spark

[jira] [Created] (SPARK-25999) Spark make-distribution failure with --r and -Phadoop-provided

shanyu zhao created SPARK-25999: --- Summary: Spark make-distribution failure with --r and -Phadoop-provided Key: SPARK-25999 URL: https://issues.apache.org/jira/browse/SPARK-25999 Project: Spark

[jira] [Updated] (SPARK-25996) Agregaciones no retornan los valores correctos con rows con timestamps iguales

[ https://issues.apache.org/jira/browse/SPARK-25996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ignacio Gómez updated SPARK-25996: -- Description: Hi all, When using pyspark I perform a count operation prior to the previous

[jira] [Commented] (SPARK-25958) error: [Errno 97] Address family not supported by protocol in dataframe.take()

[ https://issues.apache.org/jira/browse/SPARK-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681892#comment-16681892 ] Ruslan Dautkhanov commented on SPARK-25958: --- Yep, the pyspark job completes fine afetr we

[jira] [Assigned] (SPARK-25997) Python example code for Power Iteration Clustering in spark.ml

[ https://issues.apache.org/jira/browse/SPARK-25997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25997: Assignee: (was: Apache Spark) > Python example code for Power Iteration Clustering

[jira] [Assigned] (SPARK-25997) Python example code for Power Iteration Clustering in spark.ml

[ https://issues.apache.org/jira/browse/SPARK-25997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25997: Assignee: Apache Spark > Python example code for Power Iteration Clustering in spark.ml

[jira] [Updated] (SPARK-25999) make-distribution.sh failure with --r and -Phadoop-provided

[ https://issues.apache.org/jira/browse/SPARK-25999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shanyu zhao updated SPARK-25999: Summary: make-distribution.sh failure with --r and -Phadoop-provided (was: Spark

[jira] [Comment Edited] (SPARK-21542) Helper functions for custom Python Persistence

[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681895#comment-16681895 ] John Bauer edited comment on SPARK-21542 at 11/9/18 8:07 PM: - Compared to

[jira] [Updated] (SPARK-25996) Agregaciones no retornan los valores correctos con rows con timestamps iguales

[ https://issues.apache.org/jira/browse/SPARK-25996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ignacio Gómez updated SPARK-25996: -- Description: Qué, tal? Al utilizar pyspark realizo una operación de conteo de registros

[jira] [Updated] (SPARK-25996) Agregaciones no retornan los valores correctos con rows con timestamps iguales

[ https://issues.apache.org/jira/browse/SPARK-25996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ignacio Gómez updated SPARK-25996: -- Description: Qué, tal? Al utilizar pyspark realizo una operación de conteo de registros

[jira] [Updated] (SPARK-25999) make-distribution.sh failure with --r and -Phadoop-provided

[ https://issues.apache.org/jira/browse/SPARK-25999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shanyu zhao updated SPARK-25999: Attachment: SPARK-25999.patch > make-distribution.sh failure with --r and -Phadoop-provided >

[jira] [Comment Edited] (SPARK-21542) Helper functions for custom Python Persistence

[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681895#comment-16681895 ] John Bauer edited comment on SPARK-21542 at 11/9/18 7:56 PM: - This is a)

[jira] [Commented] (SPARK-25997) Python example code for Power Iteration Clustering in spark.ml

[ https://issues.apache.org/jira/browse/SPARK-25997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682023#comment-16682023 ] Apache Spark commented on SPARK-25997: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Commented] (SPARK-24421) Accessing sun.misc.Cleaner in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681151#comment-16681151 ] Alan commented on SPARK-24421: -- If I understanding correctly, the high-level need is

[jira] [Commented] (SPARK-25966) "EOF Reached the end of stream with bytes left to read" while reading/writing to Parquets

[ https://issues.apache.org/jira/browse/SPARK-25966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681297#comment-16681297 ] Steve Loughran commented on SPARK-25966: bq. Hadoop 3.1.x is not yet officially supported in

[jira] [Commented] (SPARK-25991) Update binary for 2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-25991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681300#comment-16681300 ] Yuming Wang commented on SPARK-25991: - Sorry. I do not understand what you mean. > Update binary

[jira] [Commented] (SPARK-25991) Update binary for 2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-25991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681306#comment-16681306 ] Vladimir Tsvetkov commented on SPARK-25991: --- [~yumwang] sounds strange, but I run spark-submit

[jira] [Resolved] (SPARK-22737) Simplity OneVsRest transform

[ https://issues.apache.org/jira/browse/SPARK-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-22737. -- Resolution: Not A Problem > Simplity OneVsRest transform > > >

[jira] [Created] (SPARK-25991) Update binary for 2.4.0 release

Vladimir Tsvetkov created SPARK-25991: - Summary: Update binary for 2.4.0 release Key: SPARK-25991 URL: https://issues.apache.org/jira/browse/SPARK-25991 Project: Spark Issue Type: New

[jira] [Assigned] (SPARK-25973) Spark History Main page performance improvement

[ https://issues.apache.org/jira/browse/SPARK-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25973: - Assignee: William Montaz > Spark History Main page performance improvement >

[jira] [Updated] (SPARK-25989) OneVsRestModel handle empty outputCols incorrectly

[ https://issues.apache.org/jira/browse/SPARK-25989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-25989: - Priority: Minor (was: Major) > OneVsRestModel handle empty outputCols incorrectly >

[jira] [Created] (SPARK-25989) OneVsRestModel handle empty outputCols incorrectly

zhengruifeng created SPARK-25989: Summary: OneVsRestModel handle empty outputCols incorrectly Key: SPARK-25989 URL: https://issues.apache.org/jira/browse/SPARK-25989 Project: Spark Issue

[jira] [Assigned] (SPARK-25989) OneVsRestModel handle empty outputCols incorrectly

[ https://issues.apache.org/jira/browse/SPARK-25989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25989: Assignee: Apache Spark > OneVsRestModel handle empty outputCols incorrectly >

[jira] [Commented] (SPARK-25989) OneVsRestModel handle empty outputCols incorrectly

[ https://issues.apache.org/jira/browse/SPARK-25989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681018#comment-16681018 ] Apache Spark commented on SPARK-25989: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-25989) OneVsRestModel handle empty outputCols incorrectly

[ https://issues.apache.org/jira/browse/SPARK-25989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681020#comment-16681020 ] Apache Spark commented on SPARK-25989: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-25990) TRANSFORM should handle different data types correctly

Wenchen Fan created SPARK-25990: --- Summary: TRANSFORM should handle different data types correctly Key: SPARK-25990 URL: https://issues.apache.org/jira/browse/SPARK-25990 Project: Spark Issue

[jira] [Updated] (SPARK-25852) we should filter the workOffers with freeCores>=CPUS_PER_TASK at first for better performance

[ https://issues.apache.org/jira/browse/SPARK-25852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zuotingbing updated SPARK-25852: Priority: Trivial (was: Major) > we should filter the workOffers with freeCores>=CPUS_PER_TASK

[jira] [Assigned] (SPARK-24229) Upgrade to the latest Apache Thrift 0.10.0 release

[ https://issues.apache.org/jira/browse/SPARK-24229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24229: Assignee: (was: Apache Spark) > Upgrade to the latest Apache Thrift 0.10.0 release >

[jira] [Commented] (SPARK-24229) Upgrade to the latest Apache Thrift 0.10.0 release

[ https://issues.apache.org/jira/browse/SPARK-24229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681099#comment-16681099 ] Apache Spark commented on SPARK-24229: -- User 'Fokko' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24229) Upgrade to the latest Apache Thrift 0.10.0 release

[ https://issues.apache.org/jira/browse/SPARK-24229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24229: Assignee: Apache Spark > Upgrade to the latest Apache Thrift 0.10.0 release >

[jira] [Commented] (SPARK-25973) Spark History Main page performance improvement

[ https://issues.apache.org/jira/browse/SPARK-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681054#comment-16681054 ] Apache Spark commented on SPARK-25973: -- User 'Willymontaz' has created a pull request for this

[jira] [Assigned] (SPARK-25973) Spark History Main page performance improvement

[ https://issues.apache.org/jira/browse/SPARK-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25973: Assignee: Apache Spark > Spark History Main page performance improvement >

[jira] [Assigned] (SPARK-25989) OneVsRestModel handle empty outputCols incorrectly

[ https://issues.apache.org/jira/browse/SPARK-25989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25989: Assignee: (was: Apache Spark) > OneVsRestModel handle empty outputCols incorrectly >

[jira] [Commented] (SPARK-12216) Spark failed to delete temp directory

[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681103#comment-16681103 ] Deej commented on SPARK-12216: -- This issue has *NOT* been fixed, so marking it as Resolved is plain silly.

[jira] [Assigned] (SPARK-25973) Spark History Main page performance improvement

[ https://issues.apache.org/jira/browse/SPARK-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25973: Assignee: (was: Apache Spark) > Spark History Main page performance improvement >

[jira] [Comment Edited] (SPARK-12216) Spark failed to delete temp directory

[ https://issues.apache.org/jira/browse/SPARK-12216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681103#comment-16681103 ] Deej edited comment on SPARK-12216 at 11/9/18 9:08 AM: --- This issue has *NOT* been

[jira] [Created] (SPARK-25992) Accumulators giving KeyError in pyspark

Abdeali Kothari created SPARK-25992: --- Summary: Accumulators giving KeyError in pyspark Key: SPARK-25992 URL: https://issues.apache.org/jira/browse/SPARK-25992 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25991) Update binary for 2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-25991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681371#comment-16681371 ] Yuming Wang commented on SPARK-25991: - Please check your SPARK_HOME:

[jira] [Updated] (SPARK-25991) Update binary for 2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-25991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25991: Attachment: image-2018-11-09-20-12-47-245.png > Update binary for 2.4.0 release >

[jira] [Resolved] (SPARK-25991) Update binary for 2.4.0 release

[ https://issues.apache.org/jira/browse/SPARK-25991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-25991. - Resolution: Invalid > Update binary for 2.4.0 release > --- > >

[jira] [Commented] (SPARK-23814) Couldn't read file with colon in name and new line character in one of the field.

[ https://issues.apache.org/jira/browse/SPARK-23814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681712#comment-16681712 ] Julia commented on SPARK-23814: --- [~hyukjin.kwon] Still got the same error with Spark 2.3.1. > Couldn't

[jira] [Commented] (SPARK-24244) Parse only required columns of CSV file

[ https://issues.apache.org/jira/browse/SPARK-24244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681786#comment-16681786 ] Maxim Gekk commented on SPARK-24244: > is this new option available in PySpark too? Yes, it is as

[jira] [Created] (SPARK-25994) SPIP: DataFrame-based graph queries and algorithms

Xiangrui Meng created SPARK-25994: - Summary: SPIP: DataFrame-based graph queries and algorithms Key: SPARK-25994 URL: https://issues.apache.org/jira/browse/SPARK-25994 Project: Spark Issue

[jira] [Commented] (SPARK-25696) The storage memory displayed on spark Application UI is incorrect.

[ https://issues.apache.org/jira/browse/SPARK-25696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681500#comment-16681500 ] Sean Owen commented on SPARK-25696: --- Per the pull request -- the error is actually slightly different.

[jira] [Resolved] (SPARK-25973) Spark History Main page performance improvement

[ https://issues.apache.org/jira/browse/SPARK-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25973. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22982

[jira] [Created] (SPARK-25995) sparkR should ensure user args are after the argument used for the port

Thomas Graves created SPARK-25995: - Summary: sparkR should ensure user args are after the argument used for the port Key: SPARK-25995 URL: https://issues.apache.org/jira/browse/SPARK-25995 Project:

[jira] [Resolved] (SPARK-25979) Window function: allow parentheses around window reference

[ https://issues.apache.org/jira/browse/SPARK-25979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25979. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 2.4.1

[jira] [Assigned] (SPARK-24421) Accessing sun.misc.Cleaner in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24421: Assignee: (was: Apache Spark) > Accessing sun.misc.Cleaner in JDK11 >

[jira] [Commented] (SPARK-24421) Accessing sun.misc.Cleaner in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681722#comment-16681722 ] Apache Spark commented on SPARK-24421: -- User 'srowen' has created a pull request for this issue:

[jira] [Updated] (SPARK-24421) Accessing sun.misc.Cleaner in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24421: -- Labels: release-notes (was: ) > Accessing sun.misc.Cleaner in JDK11 >

[jira] [Assigned] (SPARK-24421) Accessing sun.misc.Cleaner in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24421: Assignee: Apache Spark > Accessing sun.misc.Cleaner in JDK11 >

[jira] [Commented] (SPARK-24244) Parse only required columns of CSV file

[ https://issues.apache.org/jira/browse/SPARK-24244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681735#comment-16681735 ] Ruslan Dautkhanov commented on SPARK-24244: --- [~maxgekk] great improvement  is this new option

[jira] [Resolved] (SPARK-25988) Keep names unchanged when deduplicating the column names in Analyzer

[ https://issues.apache.org/jira/browse/SPARK-25988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25988. - Resolution: Fixed Fix Version/s: 3.0.0 2.4.1 > Keep names unchanged when

[jira] [Created] (SPARK-25993) Add test cases for resolution of ORC table location

Xiao Li created SPARK-25993: --- Summary: Add test cases for resolution of ORC table location Key: SPARK-25993 URL: https://issues.apache.org/jira/browse/SPARK-25993 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25993) Add test cases for resolution of ORC table location

[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25993: Labels: starter (was: ) > Add test cases for resolution of ORC table location >