[jira] [Created] (SPARK-46399) Add exit status to the Application End event for the use of Spark Listener

Reza Safi created SPARK-46399: - Summary: Add exit status to the Application End event for the use of Spark Listener Key: SPARK-46399 URL: https://issues.apache.org/jira/browse/SPARK-46399 Project: Spark

[jira] [Commented] (SPARK-30064) Incomplete Graphical SQL Plan in Spark application UI's SQL tab for SaveIntoDataSourceCommand

[ https://issues.apache.org/jira/browse/SPARK-30064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984030#comment-16984030 ] Reza Safi commented on SPARK-30064: --- I am working on a fix for this and hopefully will send a PR

[jira] [Commented] (SPARK-30064) Incomplete Graphical SQL Plan in Spark application UI's SQL tab for SaveIntoDataSourceCommand

[ https://issues.apache.org/jira/browse/SPARK-30064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984016#comment-16984016 ] Reza Safi commented on SPARK-30064: --- I also attached the incorrect query plan. > Incomplete

[jira] [Updated] (SPARK-30064) Incomplete Graphical SQL Plan in Spark application UI's SQL tab for SaveIntoDataSourceCommand

[ https://issues.apache.org/jira/browse/SPARK-30064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-30064: -- Attachment: Screen Shot 2019-11-21 at 3.42.16 PM.png > Incomplete Graphical SQL Plan in Spark

[jira] [Created] (SPARK-30064) Incomplete Graphical SQL Plan in Spark application UI's SQL tab for SaveIntoDataSourceCommand

Reza Safi created SPARK-30064: - Summary: Incomplete Graphical SQL Plan in Spark application UI's SQL tab for SaveIntoDataSourceCommand Key: SPARK-30064 URL: https://issues.apache.org/jira/browse/SPARK-30064

[jira] [Commented] (SPARK-25139) PythonRunner#WriterThread released block after TaskRunner finally block which invoke BlockManager#releaseAllLocksForTask

[ https://issues.apache.org/jira/browse/SPARK-25139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845422#comment-16845422 ] Reza Safi commented on SPARK-25139: --- Sure, I will send a pr soon. Thanks. > PythonRunner#WriterThread

[jira] [Commented] (SPARK-25139) PythonRunner#WriterThread released block after TaskRunner finally block which invoke BlockManager#releaseAllLocksForTask

[ https://issues.apache.org/jira/browse/SPARK-25139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845136#comment-16845136 ] Reza Safi commented on SPARK-25139: --- Any reason that this bug fix wasn't merged to 2.3 line? >

[jira] [Commented] (SPARK-24935) Problem with Executing Hive UDF's from Spark 2.2 Onwards

[ https://issues.apache.org/jira/browse/SPARK-24935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831250#comment-16831250 ] Reza Safi commented on SPARK-24935: --- Is there any reason that this wasn't merged into 2.3 line? >

[jira] [Updated] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

[ https://issues.apache.org/jira/browse/SPARK-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-27194: -- Issue Type: Bug (was: Improvement) > Job failures when task attempts do not clean up spark-staging

[jira] [Updated] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

[ https://issues.apache.org/jira/browse/SPARK-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-27194: -- Description: When a container fails for some reason (for example when killed by yarn for exceeding

[jira] [Updated] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

[ https://issues.apache.org/jira/browse/SPARK-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-27194: -- Description: When a container fails for some reason (for example when killed by yarn for exceeding

[jira] [Updated] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

[ https://issues.apache.org/jira/browse/SPARK-27194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-27194: -- Description: When a container fails for some reason (for example when killed by yarn for exceeding

[jira] [Created] (SPARK-27194) Job failures when task attempts do not clean up spark-staging parquet files

Reza Safi created SPARK-27194: - Summary: Job failures when task attempts do not clean up spark-staging parquet files Key: SPARK-27194 URL: https://issues.apache.org/jira/browse/SPARK-27194 Project: Spark

[jira] [Commented] (SPARK-26357) Expose executors' procfs metrics to Metrics system

[ https://issues.apache.org/jira/browse/SPARK-26357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782046#comment-16782046 ] Reza Safi commented on SPARK-26357: --- I think the primary usecase is for those users who are depending

[jira] [Commented] (SPARK-26357) Expose executors' procfs metrics to Metrics system

[ https://issues.apache.org/jira/browse/SPARK-26357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719788#comment-16719788 ] Reza Safi commented on SPARK-26357: --- I will send a pr soon > Expose executors' procfs metrics to

[jira] [Created] (SPARK-26357) Expose executors' procfs metrics to Metrics system

Reza Safi created SPARK-26357: - Summary: Expose executors' procfs metrics to Metrics system Key: SPARK-26357 URL: https://issues.apache.org/jira/browse/SPARK-26357 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19526) Spark should raise an exception when it tries to read a Hive view but it doesn't have read access on the corresponding table(s)

[ https://issues.apache.org/jira/browse/SPARK-19526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713368#comment-16713368 ] Reza Safi commented on SPARK-19526: --- It seems that this can be resolved since we can't reproduce the

[jira] [Updated] (SPARK-24958) Add executors' process tree total memory information to heartbeat signals

[ https://issues.apache.org/jira/browse/SPARK-24958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-24958: -- Summary: Add executors' process tree total memory information to heartbeat signals (was: Report

[jira] [Updated] (SPARK-25318) Add exception handling when wrapping the input stream during the the fetch or stage retry in response to a corrupted block

[ https://issues.apache.org/jira/browse/SPARK-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-25318: -- Description: SPARK-4105 provided a solution to block corruption issue by retrying the fetch or the

[jira] [Commented] (SPARK-25318) Add exception handling when wrapping the input stream during the the fetch or stage retry in response to a corrupted block

[ https://issues.apache.org/jira/browse/SPARK-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16602565#comment-16602565 ] Reza Safi commented on SPARK-25318: --- I will send a pr for this shortly > Add exception handling when

[jira] [Created] (SPARK-25318) Add exception handling when wrapping the input stream during the the fetch or stage retry in response to a corrupted block

Reza Safi created SPARK-25318: - Summary: Add exception handling when wrapping the input stream during the the fetch or stage retry in response to a corrupted block Key: SPARK-25318 URL:

[jira] [Created] (SPARK-25233) Give the user the option of specifying a fixed minimum message per partition per batch when using kafka direct API with backpressure

Reza Safi created SPARK-25233: - Summary: Give the user the option of specifying a fixed minimum message per partition per batch when using kafka direct API with backpressure Key: SPARK-25233 URL:

[jira] [Commented] (SPARK-25233) Give the user the option of specifying a fixed minimum message per partition per batch when using kafka direct API with backpressure

[ https://issues.apache.org/jira/browse/SPARK-25233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591889#comment-16591889 ] Reza Safi commented on SPARK-25233: --- I will send a PR shortly for this. > Give the user the option of

[jira] [Commented] (SPARK-24958) Report executors' process tree total memory information to heartbeat signals

[ https://issues.apache.org/jira/browse/SPARK-24958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560648#comment-16560648 ] Reza Safi commented on SPARK-24958: --- Need to figure out how to create a PR on top of another open PR

[jira] [Commented] (SPARK-24958) Report executors' process tree total memory information to heartbeat signals

[ https://issues.apache.org/jira/browse/SPARK-24958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560646#comment-16560646 ] Reza Safi commented on SPARK-24958: --- I will send a WIP on top of the PR for SPARK-23429 shortly. >

[jira] [Created] (SPARK-24958) Report executors' process tree total memory information to heartbeat signals

Reza Safi created SPARK-24958: - Summary: Report executors' process tree total memory information to heartbeat signals Key: SPARK-24958 URL: https://issues.apache.org/jira/browse/SPARK-24958 Project:

[jira] [Updated] (SPARK-22162) Executors and the driver should use inconsistent Job IDs during the new RDD commit protocol

[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-22162: -- Summary: Executors and the driver should use inconsistent Job IDs during the new RDD commit protocol

[jira] [Updated] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-22162: -- Summary: Executors and the driver use inconsistent Job IDs during the new RDD commit protocol (was:

[jira] [Created] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

Reza Safi created SPARK-22162: - Summary: Executors and the driver use inconsistent Job IDs during the new RDD commit protocol Key: SPARK-22162 URL: https://issues.apache.org/jira/browse/SPARK-22162

[jira] [Commented] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185036#comment-16185036 ] Reza Safi commented on SPARK-22162: --- I will send a pull request shortly for this issue. > Executors

[jira] [Commented] (SPARK-20926) Exposure to Guava libraries by directly accessing tableRelationCache in SessionCatalog caused failures

[ https://issues.apache.org/jira/browse/SPARK-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029996#comment-16029996 ] Reza Safi commented on SPARK-20926: --- I will post a pull request for this issue soon, by tonight at the

[jira] [Created] (SPARK-20926) Exposure to Guava libraries by directly accessing tableRelationCache in SessionCatalog caused failures

Reza Safi created SPARK-20926: - Summary: Exposure to Guava libraries by directly accessing tableRelationCache in SessionCatalog caused failures Key: SPARK-20926 URL: https://issues.apache.org/jira/browse/SPARK-20926

[jira] [Commented] (SPARK-19392) Throw an exception "NoSuchElementException: key not found: scale" in OracleDialect

[ https://issues.apache.org/jira/browse/SPARK-19392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15886108#comment-15886108 ] Reza Safi commented on SPARK-19392: --- As [~srowen] mentioned the problem occurred in a 1.6.x maintenance

[jira] [Created] (SPARK-19526) Spark should rise an exception when it tries to read a Hive view but it doesn't have read access on the corresponding table(s)

Reza Safi created SPARK-19526: - Summary: Spark should rise an exception when it tries to read a Hive view but it doesn't have read access on the corresponding table(s) Key: SPARK-19526 URL:

[jira] [Comment Edited] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837424#comment-15837424 ] Reza Safi edited comment on SPARK-19340 at 1/25/17 9:24 AM: As I mentioned in

[jira] [Comment Edited] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837424#comment-15837424 ] Reza Safi edited comment on SPARK-19340 at 1/25/17 9:23 AM: As I mentioned in

[jira] [Commented] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837424#comment-15837424 ] Reza Safi commented on SPARK-19340: --- As I mentioned in an earlier comment the exception only occurs if

[jira] [Comment Edited] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836629#comment-15836629 ] Reza Safi edited comment on SPARK-19340 at 1/24/17 9:03 PM: [~jayadevan.m]

[jira] [Commented] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836629#comment-15836629 ] Reza Safi commented on SPARK-19340: --- [~jayadevan.m] You can have does file names in hadoop. In fact the

[jira] [Updated] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-19340: -- Description: If you want to open a file that its name is like {noformat} "*{*}*.*" {noformat} or

[jira] [Updated] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-19340: -- Priority: Minor (was: Major) > Opening a file in CSV format will result in an exception if the

[jira] [Updated] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-19340: -- Description: If you want to open a file that its name is like {noformat} "*{*}*.*" {noformat} or

[jira] [Commented] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836489#comment-15836489 ] Reza Safi commented on SPARK-19340: --- [~hyukjin.kwon] I updated the description and the way to

[jira] [Updated] (SPARK-19340) Opening a file in CSV format will result in an exception if the filename contains special characters

[ https://issues.apache.org/jira/browse/SPARK-19340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated SPARK-19340: -- Description: If you want to open a file that its name is like {noformat} "*{*}*.*" {noformat} or

[jira] [Commented] (SPARK-15621) BatchEvalPythonExec fails with OOM

[ https://issues.apache.org/jira/browse/SPARK-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566752#comment-15566752 ] Reza Safi commented on SPARK-15621: --- Hi [~davies], can the fix be backported to branch-2.0 as well,