[jira] [Resolved] (SPARK-26140) Enable custom shuffle metrics implementation in shuffle reader

[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26140. - Resolution: Fixed Fix Version/s: 3.0.0 > Enable custom shuffle metrics implementation in shuffle

[jira] [Commented] (SPARK-26146) CSV wouln't be ingested in Spark 2.4.0 with Scala 2.12

[ https://issues.apache.org/jira/browse/SPARK-26146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697533#comment-16697533 ] Anders Eriksson commented on SPARK-26146: - I also run into this bug. I too could avoid it by

[jira] [Assigned] (SPARK-26142) Implement shuffle read metrics in SQL

[ https://issues.apache.org/jira/browse/SPARK-26142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26142: Assignee: (was: Apache Spark) > Implement shuffle read metrics in SQL >

[jira] [Assigned] (SPARK-26142) Implement shuffle read metrics in SQL

[ https://issues.apache.org/jira/browse/SPARK-26142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-26142: --- Assignee: Yuanjian Li > Implement shuffle read metrics in SQL >

[jira] [Commented] (SPARK-26142) Implement shuffle read metrics in SQL

[ https://issues.apache.org/jira/browse/SPARK-26142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697525#comment-16697525 ] Apache Spark commented on SPARK-26142: -- User 'xuanyuanking' has created a pull request for this

[jira] [Assigned] (SPARK-26142) Implement shuffle read metrics in SQL

[ https://issues.apache.org/jira/browse/SPARK-26142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26142: Assignee: Apache Spark > Implement shuffle read metrics in SQL >

[jira] [Assigned] (SPARK-26139) Support passing shuffle metrics to exchange operator

[ https://issues.apache.org/jira/browse/SPARK-26139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26139: Assignee: Reynold Xin (was: Apache Spark) > Support passing shuffle metrics to exchange

[jira] [Commented] (SPARK-26139) Support passing shuffle metrics to exchange operator

[ https://issues.apache.org/jira/browse/SPARK-26139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697513#comment-16697513 ] Apache Spark commented on SPARK-26139: -- User 'xuanyuanking' has created a pull request for this

[jira] [Assigned] (SPARK-26139) Support passing shuffle metrics to exchange operator

[ https://issues.apache.org/jira/browse/SPARK-26139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26139: Assignee: Apache Spark (was: Reynold Xin) > Support passing shuffle metrics to exchange

[jira] [Commented] (SPARK-26159) Codegen for LocalTableScanExec

[ https://issues.apache.org/jira/browse/SPARK-26159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697486#comment-16697486 ] Apache Spark commented on SPARK-26159: -- User 'juliuszsompolski' has created a pull request for this

[jira] [Assigned] (SPARK-26038) Decimal toScalaBigInt/toJavaBigInteger not work for decimals not fitting in long

[ https://issues.apache.org/jira/browse/SPARK-26038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-26038: - Assignee: Juliusz Sompolski > Decimal toScalaBigInt/toJavaBigInteger not work

[jira] [Resolved] (SPARK-26038) Decimal toScalaBigInt/toJavaBigInteger not work for decimals not fitting in long

[ https://issues.apache.org/jira/browse/SPARK-26038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-26038. --- Resolution: Fixed Fix Version/s: 3.0.0 > Decimal

[jira] [Commented] (SPARK-26159) Codegen for LocalTableScanExec

[ https://issues.apache.org/jira/browse/SPARK-26159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697485#comment-16697485 ] Apache Spark commented on SPARK-26159: -- User 'juliuszsompolski' has created a pull request for this

[jira] [Assigned] (SPARK-26159) Codegen for LocalTableScanExec

[ https://issues.apache.org/jira/browse/SPARK-26159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26159: Assignee: (was: Apache Spark) > Codegen for LocalTableScanExec >

[jira] [Assigned] (SPARK-26159) Codegen for LocalTableScanExec

[ https://issues.apache.org/jira/browse/SPARK-26159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26159: Assignee: Apache Spark > Codegen for LocalTableScanExec > --

[jira] [Comment Edited] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697410#comment-16697410 ] Dongjoon Hyun edited comment on SPARK-20144 at 11/23/18 5:56 PM: - Sorry,

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697410#comment-16697410 ] Dongjoon Hyun commented on SPARK-20144: --- Sorry, [~darabos]. IMHO, the proposed way is not

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697319#comment-16697319 ] Daniel Darabos commented on SPARK-20144: So where do we go from here? Should I try to find a

[jira] [Created] (SPARK-26159) Codegen for LocalTableScanExec

Juliusz Sompolski created SPARK-26159: - Summary: Codegen for LocalTableScanExec Key: SPARK-26159 URL: https://issues.apache.org/jira/browse/SPARK-26159 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-26108) Support custom lineSep in CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-26108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26108: - Fix Version/s: 3.0.0 > Support custom lineSep in CSV datasource >

[jira] [Resolved] (SPARK-21098) Set lineseparator csv multiline and csv write to \n

[ https://issues.apache.org/jira/browse/SPARK-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21098. -- Resolution: Not A Problem > Set lineseparator csv multiline and csv write to \n >

[jira] [Resolved] (SPARK-21289) Text based formats do not support custom end-of-line delimiters

[ https://issues.apache.org/jira/browse/SPARK-21289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21289. -- Resolution: Done CSV, Text and JSON support this option now. Should be resolvable. > Text

[jira] [Updated] (SPARK-26108) Support custom lineSep in CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-26108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26108: - Issue Type: Sub-task (was: New Feature) Parent: SPARK-21289 > Support custom lineSep

[jira] [Resolved] (SPARK-26108) Support custom lineSep in CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-26108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26108. -- Resolution: Fixed Assignee: Maxim Gekk fixed in

[jira] [Assigned] (SPARK-26158) Enhance the accuracy of covariance in RowMatrix for DenseVector

[ https://issues.apache.org/jira/browse/SPARK-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26158: Assignee: (was: Apache Spark) > Enhance the accuracy of covariance in RowMatrix for

[jira] [Commented] (SPARK-26158) Enhance the accuracy of covariance in RowMatrix for DenseVector

[ https://issues.apache.org/jira/browse/SPARK-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697242#comment-16697242 ] Apache Spark commented on SPARK-26158: -- User 'KyleLi1985' has created a pull request for this

[jira] [Assigned] (SPARK-26158) Enhance the accuracy of covariance in RowMatrix for DenseVector

[ https://issues.apache.org/jira/browse/SPARK-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26158: Assignee: Apache Spark > Enhance the accuracy of covariance in RowMatrix for DenseVector

[jira] [Commented] (SPARK-26158) Enhance the accuracy of covariance in RowMatrix for DenseVector

[ https://issues.apache.org/jira/browse/SPARK-26158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697241#comment-16697241 ] Apache Spark commented on SPARK-26158: -- User 'KyleLi1985' has created a pull request for this

[jira] [Created] (SPARK-26158) Enhance the accuracy of covariance in RowMatrix for DenseVector

Liang Li created SPARK-26158: Summary: Enhance the accuracy of covariance in RowMatrix for DenseVector Key: SPARK-26158 URL: https://issues.apache.org/jira/browse/SPARK-26158 Project: Spark

[jira] [Commented] (SPARK-25433) Add support for PEX in PySpark

[ https://issues.apache.org/jira/browse/SPARK-25433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697213#comment-16697213 ] Anderson de Andrade commented on SPARK-25433: - [~fhoering] Care to share your example? >

[jira] [Assigned] (SPARK-26117) use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception

[ https://issues.apache.org/jira/browse/SPARK-26117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26117: --- Assignee: caoxuewen > use SparkOutOfMemoryError instead of OutOfMemoryError when catch

[jira] [Resolved] (SPARK-26117) use SparkOutOfMemoryError instead of OutOfMemoryError when catch exception

[ https://issues.apache.org/jira/browse/SPARK-26117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26117. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23084

[jira] [Updated] (SPARK-26157) Asynchronous execution of stored procedure

[ https://issues.apache.org/jira/browse/SPARK-26157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaime de Roque Martínez updated SPARK-26157: Description: I am executing a jar file with spark-submit. This jar file

[jira] [Created] (SPARK-26157) Asynchronous execution of stored procedure

Jaime de Roque Martínez created SPARK-26157: --- Summary: Asynchronous execution of stored procedure Key: SPARK-26157 URL: https://issues.apache.org/jira/browse/SPARK-26157 Project: Spark

[jira] [Commented] (SPARK-23410) Unable to read jsons in charset different from UTF-8

[ https://issues.apache.org/jira/browse/SPARK-23410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696638#comment-16696638 ] xuqianjin commented on SPARK-23410: --- hi [~hyukjin.kwon] At present, most isuses of flink are SQL Table

[jira] [Commented] (SPARK-23410) Unable to read jsons in charset different from UTF-8

[ https://issues.apache.org/jira/browse/SPARK-23410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696593#comment-16696593 ] Hyukjin Kwon commented on SPARK-23410: -- That's not even merged yet. > Unable to read jsons in

[jira] [Commented] (SPARK-26156) Revise summary section of stage page

[ https://issues.apache.org/jira/browse/SPARK-26156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696551#comment-16696551 ] Apache Spark commented on SPARK-26156: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-26156) Revise summary section of stage page

[ https://issues.apache.org/jira/browse/SPARK-26156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696549#comment-16696549 ] Apache Spark commented on SPARK-26156: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-26156) Revise summary section of stage page

[ https://issues.apache.org/jira/browse/SPARK-26156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26156: Assignee: (was: Apache Spark) > Revise summary section of stage page >

[jira] [Assigned] (SPARK-26156) Revise summary section of stage page

[ https://issues.apache.org/jira/browse/SPARK-26156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26156: Assignee: Apache Spark > Revise summary section of stage page >

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696541#comment-16696541 ] Adrian Wang commented on SPARK-26155: - It seems the performance downgrade is related to CPU cache,

[jira] [Created] (SPARK-26156) Revise summary section of stage page

Gengliang Wang created SPARK-26156: -- Summary: Revise summary section of stage page Key: SPARK-26156 URL: https://issues.apache.org/jira/browse/SPARK-26156 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23410) Unable to read jsons in charset different from UTF-8

[ https://issues.apache.org/jira/browse/SPARK-23410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696542#comment-16696542 ] xuqianjin commented on SPARK-23410: --- hi [~hyukjin.kwon]  this the PR

[jira] [Updated] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Jia updated SPARK-26155: --- Attachment: q19.sql Q19 analysis in Spark2.3 without L486 & 487.pdf Q19

[jira] [Commented] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696526#comment-16696526 ] Pierre Lienhart commented on SPARK-26116: - I just enhanced the ticket description. > Spark SQL

[jira] [Updated] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Lienhart updated SPARK-26116: Description: When writing partitioned parquet using {{partitionBy}}, it looks like Spark

[jira] [Comment Edited] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696476#comment-16696476 ] Ke Jia edited comment on SPARK-26155 at 11/23/18 7:58 AM: -- *Cluster info:* | 

[jira] [Updated] (SPARK-26059) Spark standalone mode, does not correctly record a failed Spark Job.

[ https://issues.apache.org/jira/browse/SPARK-26059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-26059: Description: In order to reproduce submit a failing job to spark standalone master. The