[jira] [Updated] (SPARK-24896) Uuid expression should produce different values in each execution under streaming query

[ https://issues.apache.org/jira/browse/SPARK-24896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-24896: Component/s: Structured Streaming > Uuid expression should produce different values in

[jira] [Assigned] (SPARK-24896) Uuid expression should produce different values in each execution under streaming query

[ https://issues.apache.org/jira/browse/SPARK-24896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24896: Assignee: Apache Spark > Uuid expression should produce different values in each

[jira] [Assigned] (SPARK-24896) Uuid expression should produce different values in each execution under streaming query

[ https://issues.apache.org/jira/browse/SPARK-24896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24896: Assignee: (was: Apache Spark) > Uuid expression should produce different values in

[jira] [Commented] (SPARK-24896) Uuid expression should produce different values in each execution under streaming query

[ https://issues.apache.org/jira/browse/SPARK-24896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553803#comment-16553803 ] Apache Spark commented on SPARK-24896: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-24896) Uuid expression should produce different values in each execution under streaming query

Liang-Chi Hsieh created SPARK-24896: --- Summary: Uuid expression should produce different values in each execution under streaming query Key: SPARK-24896 URL: https://issues.apache.org/jira/browse/SPARK-24896

[jira] [Resolved] (SPARK-24857) required the sample code test the spark steaming job in kubernates and write the data in remote hdfs file system

[ https://issues.apache.org/jira/browse/SPARK-24857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24857. -- Resolution: Cannot Reproduce Please reopen if you happen to have a fix or provide

[jira] [Updated] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24895: - Description: Spark 2.4.0 has maven build errors because artifacts uploaded to apache maven

[jira] [Commented] (SPARK-21097) Dynamic allocation will preserve cached data

[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553768#comment-16553768 ] James commented on SPARK-21097: --- Hi [~bradkaiser] whats the progress of this project or problem now? i'd

[jira] [Commented] (SPARK-24760) Pandas UDF does not handle NaN correctly

[ https://issues.apache.org/jira/browse/SPARK-24760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553756#comment-16553756 ] Hyukjin Kwon commented on SPARK-24760: -- +1 for not a problem resolution for now. > Pandas UDF does

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553700#comment-16553700 ] Carson Wang commented on SPARK-23128: - [~tgraves], We have split the code into 7 PRs based on Spark

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553679#comment-16553679 ] Saisai Shao commented on SPARK-24615: - [~Tagar] yes! > Accelerator-aware task scheduling for Spark

[jira] [Commented] (SPARK-15694) Implement ScriptTransformation in sql/core

[ https://issues.apache.org/jira/browse/SPARK-15694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553677#comment-16553677 ] Abdulrahman Alfozan commented on SPARK-15694: - I'm currently working on this. > Implement

[jira] [Resolved] (SPARK-24594) Introduce metrics for YARN executor allocation problems

[ https://issues.apache.org/jira/browse/SPARK-24594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24594. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21635

[jira] [Assigned] (SPARK-24594) Introduce metrics for YARN executor allocation problems

[ https://issues.apache.org/jira/browse/SPARK-24594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24594: --- Assignee: Attila Zsolt Piros > Introduce metrics for YARN executor allocation problems >

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553656#comment-16553656 ] Hyukjin Kwon commented on SPARK-24895: -- FWIW, I hit this too. Thanks for analysis. > Spark 2.4.0

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553654#comment-16553654 ] Hyukjin Kwon commented on SPARK-24895: -- Yes please! don't mind at all. > Spark 2.4.0 Snapshot

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553610#comment-16553610 ] Yin Huai commented on SPARK-24895: -- [~kiszk] [~hyukjin.kwon] since this thing is pretty tricky to test

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553568#comment-16553568 ] Yin Huai commented on SPARK-24895: --   [~kiszk] and [~hyukjin.kwon] we hit this issue today. Per

[jira] [Updated] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-24895: - Target Version/s: 2.4.0 > Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553566#comment-16553566 ] Xiao Li commented on SPARK-24288: - [~TomaszGaweda] Based on my understanding, when the predicates fail

[jira] [Created] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

Eric Chang created SPARK-24895: -- Summary: Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames Key: SPARK-24895 URL: https://issues.apache.org/jira/browse/SPARK-24895 Project:

[jira] [Created] (SPARK-24894) Invalid DNS name due to hostname truncation

Dharmesh Kakadia created SPARK-24894: Summary: Invalid DNS name due to hostname truncation Key: SPARK-24894 URL: https://issues.apache.org/jira/browse/SPARK-24894 Project: Spark Issue

[jira] [Commented] (SPARK-24882) separate responsibilities of the data source v2 read API

[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553460#comment-16553460 ] Ryan Blue commented on SPARK-24882: --- [~cloud_fan], just to confirm: this is *not intended to go into

[jira] [Commented] (SPARK-23957) Sorts in subqueries are redundant and can be removed

[ https://issues.apache.org/jira/browse/SPARK-23957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553434#comment-16553434 ] Apache Spark commented on SPARK-23957: -- User 'dilipbiswal' has created a pull request for this

[jira] [Updated] (SPARK-24814) Relationship between catalog and datasources

[ https://issues.apache.org/jira/browse/SPARK-24814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-24814: -- Description: This is somewhat related, though not identical to, [~rdblue]'s SPIP on

[jira] [Resolved] (SPARK-24339) spark sql can not prune column in transform/map/reduce query

[ https://issues.apache.org/jira/browse/SPARK-24339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24339. - Resolution: Fixed Assignee: Li Yuanjian Fix Version/s: 2.4.0 > spark sql can not prune

[jira] [Resolved] (SPARK-24699) Watermark / Append mode should work with Trigger.Once

[ https://issues.apache.org/jira/browse/SPARK-24699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-24699. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21746

[jira] [Assigned] (SPARK-24699) Watermark / Append mode should work with Trigger.Once

[ https://issues.apache.org/jira/browse/SPARK-24699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-24699: - Assignee: Tathagata Das > Watermark / Append mode should work with Trigger.Once >

[jira] [Commented] (SPARK-24814) Relationship between catalog and datasources

[ https://issues.apache.org/jira/browse/SPARK-24814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553336#comment-16553336 ] Bruce Robbins commented on SPARK-24814: --- [~rdblue] Your parquet example is a compelling one. If

[jira] [Commented] (SPARK-6459) Warn when Column API is constructing trivially true equality

[ https://issues.apache.org/jira/browse/SPARK-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553330#comment-16553330 ] Michael Armbrust commented on SPARK-6459: - [~tenstriker] this will never happen from a SQL query. 

[jira] [Commented] (SPARK-24814) Relationship between catalog and datasources

[ https://issues.apache.org/jira/browse/SPARK-24814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553324#comment-16553324 ] Ryan Blue commented on SPARK-24814: --- I've been implementing more logical plans (AppendData,

[jira] [Commented] (SPARK-24760) Pandas UDF does not handle NaN correctly

[ https://issues.apache.org/jira/browse/SPARK-24760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553315#comment-16553315 ] Wes McKinney commented on SPARK-24760: -- If data comes to Spark from pandas, any "NaN" values should

[jira] [Assigned] (SPARK-24893) Remove the entire CaseWhen if all the outputs are semantic equivalence

[ https://issues.apache.org/jira/browse/SPARK-24893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24893: Assignee: Apache Spark (was: DB Tsai) > Remove the entire CaseWhen if all the outputs

[jira] [Assigned] (SPARK-24893) Remove the entire CaseWhen if all the outputs are semantic equivalence

[ https://issues.apache.org/jira/browse/SPARK-24893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24893: Assignee: DB Tsai (was: Apache Spark) > Remove the entire CaseWhen if all the outputs

[jira] [Commented] (SPARK-24893) Remove the entire CaseWhen if all the outputs are semantic equivalence

[ https://issues.apache.org/jira/browse/SPARK-24893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553286#comment-16553286 ] Apache Spark commented on SPARK-24893: -- User 'dbtsai' has created a pull request for this issue:

[jira] [Created] (SPARK-24893) Remove the entire CaseWhen if all the outputs are semantic equivalence

DB Tsai created SPARK-24893: --- Summary: Remove the entire CaseWhen if all the outputs are semantic equivalence Key: SPARK-24893 URL: https://issues.apache.org/jira/browse/SPARK-24893 Project: Spark

[jira] [Assigned] (SPARK-24891) Fix HandleNullInputsForUDF rule

[ https://issues.apache.org/jira/browse/SPARK-24891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24891: Assignee: Apache Spark > Fix HandleNullInputsForUDF rule >

[jira] [Assigned] (SPARK-24891) Fix HandleNullInputsForUDF rule

[ https://issues.apache.org/jira/browse/SPARK-24891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24891: Assignee: (was: Apache Spark) > Fix HandleNullInputsForUDF rule >

[jira] [Commented] (SPARK-24891) Fix HandleNullInputsForUDF rule

[ https://issues.apache.org/jira/browse/SPARK-24891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553271#comment-16553271 ] Apache Spark commented on SPARK-24891: -- User 'maryannxue' has created a pull request for this

[jira] [Updated] (SPARK-24891) Fix HandleNullInputsForUDF rule

[ https://issues.apache.org/jira/browse/SPARK-24891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maryann Xue updated SPARK-24891: Summary: Fix HandleNullInputsForUDF rule (was: Fix HandleNullInputsForUDF rule.) > Fix

[jira] [Assigned] (SPARK-24890) Short circuiting the `if` condition when `trueValue` and `falseValue` are the same

[ https://issues.apache.org/jira/browse/SPARK-24890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-24890: --- Assignee: DB Tsai > Short circuiting the `if` condition when `trueValue` and `falseValue` are the

[jira] [Assigned] (SPARK-24892) Simplify `CaseWhen` to `If` when there is only one branch

[ https://issues.apache.org/jira/browse/SPARK-24892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-24892: --- Assignee: DB Tsai > Simplify `CaseWhen` to `If` when there is only one branch >

[jira] [Assigned] (SPARK-24892) Simplify `CaseWhen` to `If` when there is only one branch

[ https://issues.apache.org/jira/browse/SPARK-24892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24892: Assignee: Apache Spark > Simplify `CaseWhen` to `If` when there is only one branch >

[jira] [Assigned] (SPARK-24892) Simplify `CaseWhen` to `If` when there is only one branch

[ https://issues.apache.org/jira/browse/SPARK-24892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24892: Assignee: (was: Apache Spark) > Simplify `CaseWhen` to `If` when there is only one

[jira] [Commented] (SPARK-24892) Simplify `CaseWhen` to `If` when there is only one branch

[ https://issues.apache.org/jira/browse/SPARK-24892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553245#comment-16553245 ] Apache Spark commented on SPARK-24892: -- User 'dbtsai' has created a pull request for this issue:

[jira] [Created] (SPARK-24892) Simplify `CaseWhen` to `If` when there is only one branch

DB Tsai created SPARK-24892: --- Summary: Simplify `CaseWhen` to `If` when there is only one branch Key: SPARK-24892 URL: https://issues.apache.org/jira/browse/SPARK-24892 Project: Spark Issue Type:

[jira] [Created] (SPARK-24891) Fix HandleNullInputsForUDF rule.

Maryann Xue created SPARK-24891: --- Summary: Fix HandleNullInputsForUDF rule. Key: SPARK-24891 URL: https://issues.apache.org/jira/browse/SPARK-24891 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-24243) Expose exceptions from InProcessAppHandle

[ https://issues.apache.org/jira/browse/SPARK-24243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24243: Assignee: Apache Spark > Expose exceptions from InProcessAppHandle >

[jira] [Assigned] (SPARK-24243) Expose exceptions from InProcessAppHandle

[ https://issues.apache.org/jira/browse/SPARK-24243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24243: Assignee: (was: Apache Spark) > Expose exceptions from InProcessAppHandle >

[jira] [Assigned] (SPARK-24243) Expose exceptions from InProcessAppHandle

[ https://issues.apache.org/jira/browse/SPARK-24243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24243: Assignee: Apache Spark > Expose exceptions from InProcessAppHandle >

[jira] [Assigned] (SPARK-24243) Expose exceptions from InProcessAppHandle

[ https://issues.apache.org/jira/browse/SPARK-24243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24243: Assignee: (was: Apache Spark) > Expose exceptions from InProcessAppHandle >

[jira] [Commented] (SPARK-24243) Expose exceptions from InProcessAppHandle

[ https://issues.apache.org/jira/browse/SPARK-24243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553155#comment-16553155 ] Apache Spark commented on SPARK-24243: -- User 'sahilTakiar' has created a pull request for this

[jira] [Assigned] (SPARK-24890) Short circuiting the `if` condition when `trueValue` and `falseValue` are the same

[ https://issues.apache.org/jira/browse/SPARK-24890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24890: Assignee: (was: Apache Spark) > Short circuiting the `if` condition when `trueValue`

[jira] [Commented] (SPARK-24890) Short circuiting the `if` condition when `trueValue` and `falseValue` are the same

[ https://issues.apache.org/jira/browse/SPARK-24890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553151#comment-16553151 ] Apache Spark commented on SPARK-24890: -- User 'dbtsai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24890) Short circuiting the `if` condition when `trueValue` and `falseValue` are the same

[ https://issues.apache.org/jira/browse/SPARK-24890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24890: Assignee: Apache Spark > Short circuiting the `if` condition when `trueValue` and

[jira] [Created] (SPARK-24890) Short circuiting the `if` condition when `trueValue` and `falseValue` are the same

DB Tsai created SPARK-24890: --- Summary: Short circuiting the `if` condition when `trueValue` and `falseValue` are the same Key: SPARK-24890 URL: https://issues.apache.org/jira/browse/SPARK-24890 Project:

[jira] [Commented] (SPARK-24835) col function ignores drop

[ https://issues.apache.org/jira/browse/SPARK-24835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553136#comment-16553136 ] Michael Souder commented on SPARK-24835: Liang-Chi, your first block actually fails for me with

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553121#comment-16553121 ] Ruslan Dautkhanov commented on SPARK-24615: --- Is this work related to Spark's project

[jira] [Resolved] (SPARK-24850) Query plan string representation grows exponentially on queries with recursive cached datasets

[ https://issues.apache.org/jira/browse/SPARK-24850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24850. - Resolution: Fixed Assignee: Onur Satici Fix Version/s: 2.4.0 > Query plan string

[jira] [Commented] (SPARK-23874) Upgrade apache/arrow to 0.10.0

[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553077#comment-16553077 ] Xiao Li commented on SPARK-23874: - [~bryanc] Thank you for driving this. > Upgrade apache/arrow to

[jira] [Assigned] (SPARK-24855) Built-in AVRO support should support specified schema on write

[ https://issues.apache.org/jira/browse/SPARK-24855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24855: Assignee: Brian Lindblom (was: Apache Spark) > Built-in AVRO support should support

[jira] [Assigned] (SPARK-24855) Built-in AVRO support should support specified schema on write

[ https://issues.apache.org/jira/browse/SPARK-24855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24855: Assignee: Apache Spark (was: Brian Lindblom) > Built-in AVRO support should support

[jira] [Commented] (SPARK-24855) Built-in AVRO support should support specified schema on write

[ https://issues.apache.org/jira/browse/SPARK-24855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553040#comment-16553040 ] Apache Spark commented on SPARK-24855: -- User 'lindblombr' has created a pull request for this

[jira] [Resolved] (SPARK-24887) Use SerializableConfiguration in Spark util

[ https://issues.apache.org/jira/browse/SPARK-24887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24887. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 2.4.0 > Use

[jira] [Resolved] (SPARK-24802) Optimization Rule Exclusion

[ https://issues.apache.org/jira/browse/SPARK-24802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24802. - Resolution: Fixed Assignee: Maryann Xue Fix Version/s: 2.4.0 > Optimization Rule

[jira] [Commented] (SPARK-24869) SaveIntoDataSourceCommand's input Dataset does not use Cached Data

[ https://issues.apache.org/jira/browse/SPARK-24869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552965#comment-16552965 ] Takeshi Yamamuro commented on SPARK-24869: -- yea, I made the branch just for this test purpose

[jira] [Updated] (SPARK-24889) dataset.unpersist() doesn't update storage memory stats

[ https://issues.apache.org/jira/browse/SPARK-24889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuri Bogomolov updated SPARK-24889: --- Description: Steps to reproduce: 1) Start a Spark cluster, and check the storage memory

[jira] [Updated] (SPARK-24889) dataset.unpersist() doesn't update storage memory stats

[ https://issues.apache.org/jira/browse/SPARK-24889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuri Bogomolov updated SPARK-24889: --- Attachment: image-2018-07-23-10-53-58-474.png > dataset.unpersist() doesn't update storage

[jira] [Created] (SPARK-24889) dataset.unpersist() doesn't update storage memory stats

Yuri Bogomolov created SPARK-24889: -- Summary: dataset.unpersist() doesn't update storage memory stats Key: SPARK-24889 URL: https://issues.apache.org/jira/browse/SPARK-24889 Project: Spark

[jira] [Commented] (SPARK-21097) Dynamic allocation will preserve cached data

[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552847#comment-16552847 ] Brad commented on SPARK-21097: -- No, I did not consider cpu utilization when redistributing the cache. It

[jira] [Updated] (SPARK-24888) spark-submit --master spark://host:port --status driver-id does not work

[ https://issues.apache.org/jira/browse/SPARK-24888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] srinivasan updated SPARK-24888: --- Description: spark-submit --master spark://host:port --status driver-id does not return anything.

[jira] [Created] (SPARK-24888) spark-submit --master spark://host:port --status driver-id does not work

srinivasan created SPARK-24888: -- Summary: spark-submit --master spark://host:port --status driver-id does not work Key: SPARK-24888 URL: https://issues.apache.org/jira/browse/SPARK-24888 Project: Spark

[jira] [Commented] (SPARK-24887) Use SerializableConfiguration in Spark util

[ https://issues.apache.org/jira/browse/SPARK-24887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552558#comment-16552558 ] Apache Spark commented on SPARK-24887: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-24887) Use SerializableConfiguration in Spark util

[ https://issues.apache.org/jira/browse/SPARK-24887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24887: Assignee: (was: Apache Spark) > Use SerializableConfiguration in Spark util >

[jira] [Assigned] (SPARK-24887) Use SerializableConfiguration in Spark util

[ https://issues.apache.org/jira/browse/SPARK-24887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24887: Assignee: Apache Spark > Use SerializableConfiguration in Spark util >

[jira] [Created] (SPARK-24887) Use SerializableConfiguration in Spark util

Gengliang Wang created SPARK-24887: -- Summary: Use SerializableConfiguration in Spark util Key: SPARK-24887 URL: https://issues.apache.org/jira/browse/SPARK-24887 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-24883) Remove implicit class AvroDataFrameWriter/AvroDataFrameReader

[ https://issues.apache.org/jira/browse/SPARK-24883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24883. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21841

[jira] [Assigned] (SPARK-24883) Remove implicit class AvroDataFrameWriter/AvroDataFrameReader

[ https://issues.apache.org/jira/browse/SPARK-24883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24883: Assignee: Gengliang Wang > Remove implicit class AvroDataFrameWriter/AvroDataFrameReader

[jira] [Assigned] (SPARK-24883) Remove implicit class AvroDataFrameWriter/AvroDataFrameReader

[ https://issues.apache.org/jira/browse/SPARK-24883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24883: Assignee: (was: Apache Spark) > Remove implicit class

[jira] [Commented] (SPARK-24883) Remove implicit class AvroDataFrameWriter/AvroDataFrameReader

[ https://issues.apache.org/jira/browse/SPARK-24883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552410#comment-16552410 ] Apache Spark commented on SPARK-24883: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-24883) Remove implicit class AvroDataFrameWriter/AvroDataFrameReader

[ https://issues.apache.org/jira/browse/SPARK-24883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24883: Assignee: Apache Spark > Remove implicit class AvroDataFrameWriter/AvroDataFrameReader >