[jira] [Created] (SPARK-30992) Arrange scattered config for streaming

jiaan.geng created SPARK-30992: -- Summary: Arrange scattered config for streaming Key: SPARK-30992 URL: https://issues.apache.org/jira/browse/SPARK-30992 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-27750) Standalone scheduler - ability to prioritize applications over drivers, many drivers act like Denial of Service

[ https://issues.apache.org/jira/browse/SPARK-27750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048120#comment-17048120 ] Dongjoon Hyun commented on SPARK-27750: --- According to the review and discussion on the proposed

[jira] [Resolved] (SPARK-27750) Standalone scheduler - ability to prioritize applications over drivers, many drivers act like Denial of Service

[ https://issues.apache.org/jira/browse/SPARK-27750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27750. --- Resolution: Not A Problem > Standalone scheduler - ability to prioritize applications over

[jira] [Closed] (SPARK-27750) Standalone scheduler - ability to prioritize applications over drivers, many drivers act like Denial of Service

[ https://issues.apache.org/jira/browse/SPARK-27750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-27750. - > Standalone scheduler - ability to prioritize applications over drivers, many > drivers act like

[jira] [Resolved] (SPARK-30977) ResourceProfile and Builder should be private in spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-30977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30977. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27737

[jira] [Assigned] (SPARK-30977) ResourceProfile and Builder should be private in spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-30977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30977: - Assignee: Thomas Graves > ResourceProfile and Builder should be private in spark 3.0 >

[jira] [Created] (SPARK-30991) Refactor AQE readers

Wei Xue created SPARK-30991: --- Summary: Refactor AQE readers Key: SPARK-30991 URL: https://issues.apache.org/jira/browse/SPARK-30991 Project: Spark Issue Type: Bug Components: SQL

[jira] [Updated] (SPARK-30991) Refactor AQE readers and RDDs

[ https://issues.apache.org/jira/browse/SPARK-30991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Xue updated SPARK-30991: Summary: Refactor AQE readers and RDDs (was: Refactor AQE readers) > Refactor AQE readers and RDDs >

[jira] [Resolved] (SPARK-29969) parse_url function result in incorrect result

[ https://issues.apache.org/jira/browse/SPARK-29969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29969. -- Resolution: Not A Problem > parse_url function result in incorrect result >

[jira] [Comment Edited] (SPARK-29969) parse_url function result in incorrect result

[ https://issues.apache.org/jira/browse/SPARK-29969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047970#comment-17047970 ] YoungGyu Chun edited comment on SPARK-29969 at 2/28/20 9:54 PM:

[jira] [Comment Edited] (SPARK-29969) parse_url function result in incorrect result

[ https://issues.apache.org/jira/browse/SPARK-29969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047970#comment-17047970 ] YoungGyu Chun edited comment on SPARK-29969 at 2/28/20 9:54 PM:

[jira] [Comment Edited] (SPARK-29969) parse_url function result in incorrect result

[ https://issues.apache.org/jira/browse/SPARK-29969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047970#comment-17047970 ] YoungGyu Chun edited comment on SPARK-29969 at 2/28/20 9:48 PM:

[jira] [Commented] (SPARK-29969) parse_url function result in incorrect result

[ https://issues.apache.org/jira/browse/SPARK-29969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047970#comment-17047970 ] YoungGyu Chun commented on SPARK-29969: --- [~xiaoxigua] [~srowen] [~hyukjin.kwon] [~dongjoon] I

[jira] [Resolved] (SPARK-29149) Update YARN cluster manager For Stage Level Scheduling

[ https://issues.apache.org/jira/browse/SPARK-29149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-29149. --- Fix Version/s: 3.1.0 Resolution: Fixed > Update YARN cluster manager For Stage Level

[jira] [Commented] (SPARK-30990) Binary incompatibility changes

[ https://issues.apache.org/jira/browse/SPARK-30990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047953#comment-17047953 ] Huaxin Gao commented on SPARK-30990: cc [~smilegator] I audited all the MiMa exclusions  added for

[jira] [Created] (SPARK-30990) Binary incompatibility changes

Huaxin Gao created SPARK-30990: -- Summary: Binary incompatibility changes Key: SPARK-30990 URL: https://issues.apache.org/jira/browse/SPARK-30990 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-29969) parse_url function result in incorrect result

[ https://issues.apache.org/jira/browse/SPARK-29969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047946#comment-17047946 ] YoungGyu Chun commented on SPARK-29969: --- [~xiaoxigua]. Thank you for answering the question. I

[jira] [Resolved] (SPARK-30987) ResourceDiscoveryPluginSuite sometimes fails

[ https://issues.apache.org/jira/browse/SPARK-30987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-30987. -- Fix Version/s: 3.0.0 Target Version/s: 3.0.0 Resolution: Fixed Resolved by

[jira] [Assigned] (SPARK-30987) ResourceDiscoveryPluginSuite sometimes fails

[ https://issues.apache.org/jira/browse/SPARK-30987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang reassigned SPARK-30987: Assignee: Thomas Graves > ResourceDiscoveryPluginSuite sometimes fails >

[jira] [Updated] (SPARK-30970) Fix NPE in resolving k8s master url

[ https://issues.apache.org/jira/browse/SPARK-30970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30970: -- Fix Version/s: 2.4.6 > Fix NPE in resolving k8s master url >

[jira] [Resolved] (SPARK-30947) Log better message when accelerate resource is empty

[ https://issues.apache.org/jira/browse/SPARK-30947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-30947. -- Fix Version/s: 3.1.0 Resolution: Fixed Resolved by

[jira] [Assigned] (SPARK-30947) Log better message when accelerate resource is empty

[ https://issues.apache.org/jira/browse/SPARK-30947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang reassigned SPARK-30947: Assignee: wuyi > Log better message when accelerate resource is empty >

[jira] [Created] (SPARK-30989) TABLE.COLUMN reference doesn't work with new columns created by UDF

Chris Suchanek created SPARK-30989: -- Summary: TABLE.COLUMN reference doesn't work with new columns created by UDF Key: SPARK-30989 URL: https://issues.apache.org/jira/browse/SPARK-30989 Project:

[jira] [Commented] (SPARK-30986) Structured Streaming: mapGroupsWithState UDT serialization does not work

[ https://issues.apache.org/jira/browse/SPARK-30986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047854#comment-17047854 ] Bryan Jeffrey commented on SPARK-30986: --- I validated this same issue exists on Spark 2.4.0 >

[jira] [Updated] (SPARK-30986) Structured Streaming: mapGroupsWithState UDT serialization does not work

[ https://issues.apache.org/jira/browse/SPARK-30986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Jeffrey updated SPARK-30986: -- Affects Version/s: 2.4.0 > Structured Streaming: mapGroupsWithState UDT serialization does

[jira] [Created] (SPARK-30988) Add more edge-case exercising values to stats tests

Maxim Gekk created SPARK-30988: -- Summary: Add more edge-case exercising values to stats tests Key: SPARK-30988 URL: https://issues.apache.org/jira/browse/SPARK-30988 Project: Spark Issue Type:

[jira] [Commented] (SPARK-30962) Document ALTER TABLE statement in SQL Reference [Phase 2]

[ https://issues.apache.org/jira/browse/SPARK-30962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047806#comment-17047806 ] kevin yu commented on SPARK-30962: -- I work on it. > Document ALTER TABLE statement in SQL Reference

[jira] [Commented] (SPARK-30987) ResourceDiscoveryPluginSuite sometimes fails

[ https://issues.apache.org/jira/browse/SPARK-30987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047783#comment-17047783 ] Thomas Graves commented on SPARK-30987: --- all of these fail in just starting up the local-cluster.

[jira] [Created] (SPARK-30987) ResourceDiscoveryPluginSuite sometimes fails

Thomas Graves created SPARK-30987: - Summary: ResourceDiscoveryPluginSuite sometimes fails Key: SPARK-30987 URL: https://issues.apache.org/jira/browse/SPARK-30987 Project: Spark Issue Type:

[jira] [Created] (SPARK-30986) Structured Streaming: mapGroupsWithState UDT serialization does not work

Bryan Jeffrey created SPARK-30986: - Summary: Structured Streaming: mapGroupsWithState UDT serialization does not work Key: SPARK-30986 URL: https://issues.apache.org/jira/browse/SPARK-30986 Project:

[jira] [Resolved] (SPARK-30842) Adjust abstraction structure for join operators

[ https://issues.apache.org/jira/browse/SPARK-30842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30842. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27595

[jira] [Created] (SPARK-30985) Propagate SPARK_CONF_DIR files to driver and exec pods.

Prashant Sharma created SPARK-30985: --- Summary: Propagate SPARK_CONF_DIR files to driver and exec pods. Key: SPARK-30985 URL: https://issues.apache.org/jira/browse/SPARK-30985 Project: Spark

[jira] [Commented] (SPARK-30365) When deploy mode is a client, why doesn't it support remote "spark.files" download?

[ https://issues.apache.org/jira/browse/SPARK-30365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047459#comment-17047459 ] wangzhun commented on SPARK-30365: -- [~hyukjin.kwon] Thank you for your advice > When deploy mode is a

[jira] [Resolved] (SPARK-30945) Preserve JVM information when executor heartbeat timeout

[ https://issues.apache.org/jira/browse/SPARK-30945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhun resolved SPARK-30945. -- Resolution: Done > Preserve JVM information when executor heartbeat timeout >

[jira] [Assigned] (SPARK-30842) Adjust abstraction structure for join operators

[ https://issues.apache.org/jira/browse/SPARK-30842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30842: --- Assignee: Xin Wu > Adjust abstraction structure for join operators >

[jira] [Updated] (SPARK-30908) Add version information to the configuration of Kryo

[ https://issues.apache.org/jira/browse/SPARK-30908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-30908: --- Summary: Add version information to the configuration of Kryo (was: Arrange version info of Kryo)

[jira] [Resolved] (SPARK-30938) BinaryClassificationMetrics optimization

[ https://issues.apache.org/jira/browse/SPARK-30938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-30938. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27682

[jira] [Assigned] (SPARK-30938) BinaryClassificationMetrics optimization

[ https://issues.apache.org/jira/browse/SPARK-30938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-30938: Assignee: zhengruifeng > BinaryClassificationMetrics optimization >

[jira] [Updated] (SPARK-30970) Fix NPE in resolving k8s master url

[ https://issues.apache.org/jira/browse/SPARK-30970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30970: -- Affects Version/s: (was: 3.1.0) 2.3.4 > Fix NPE in resolving k8s

[jira] [Resolved] (SPARK-30970) Fix NPE in resolving k8s master url

[ https://issues.apache.org/jira/browse/SPARK-30970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30970. --- Fix Version/s: 3.0.0 Assignee: Kent Yao Resolution: Fixed This is resolved

[jira] [Created] (SPARK-30984) Add UI test for Structured Streaming UI

Shixiong Zhu created SPARK-30984: Summary: Add UI test for Structured Streaming UI Key: SPARK-30984 URL: https://issues.apache.org/jira/browse/SPARK-30984 Project: Spark Issue Type: Test