[jira] [Resolved] (SPARK-32948) Optimize Json expression chain

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32948. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29828

[jira] [Resolved] (SPARK-33020) Add nth_value in PySpark API

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33020. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29899

[jira] [Assigned] (SPARK-33020) Add nth_value in PySpark API

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33020: - Assignee: Hyukjin Kwon > Add nth_value in PySpark API > >

[jira] [Resolved] (SPARK-33015) Compute the current date only once

2020-09-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33015. - Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-33015) Compute the current date only once

2020-09-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33015: --- Assignee: Maxim Gekk > Compute the current date only once >

[jira] [Updated] (SPARK-33019) Use spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=1 by default

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33019: -- Target Version/s: 3.0.2, 3.1.0 > Use

[jira] [Updated] (SPARK-33019) Use spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=1 by default

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33019: -- Labels: correctness (was: ) > Use

[jira] [Updated] (SPARK-33019) Use spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=1 by default

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33019: -- Priority: Blocker (was: Critical) > Use

[jira] [Resolved] (SPARK-33021) Move functions related test cases into test_functions.py

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33021. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29898

[jira] [Assigned] (SPARK-33021) Move functions related test cases into test_functions.py

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33021: - Assignee: Hyukjin Kwon > Move functions related test cases into test_functions.py >

[jira] [Assigned] (SPARK-28895) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28895: Assignee: (was: Apache Spark) > Spark client process is unable to upload jars to

[jira] [Assigned] (SPARK-28895) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28895: Assignee: Apache Spark > Spark client process is unable to upload jars to hdfs while

[jira] [Commented] (SPARK-28895) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203608#comment-17203608 ] Apache Spark commented on SPARK-28895: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-28895) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203609#comment-17203609 ] Apache Spark commented on SPARK-28895: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Updated] (SPARK-28992) Support update dependencies from hdfs when task run on executor pods

2020-09-28 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-28992: - Parent: SPARK-33005 Issue Type: Sub-task (was: Improvement) > Support update dependencies from

[jira] [Updated] (SPARK-28895) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2020-09-28 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-28895: - Parent: SPARK-33005 Issue Type: Sub-task (was: Bug) > Spark client process is unable to upload

[jira] [Assigned] (SPARK-33020) Add nth_value in PySpark API

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33020: Assignee: (was: Apache Spark) > Add nth_value in PySpark API >

[jira] [Assigned] (SPARK-33020) Add nth_value in PySpark API

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33020: Assignee: Apache Spark > Add nth_value in PySpark API > > >

[jira] [Commented] (SPARK-33020) Add nth_value in PySpark API

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203605#comment-17203605 ] Apache Spark commented on SPARK-33020: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-33021) Move functions related test cases into test_functions.py

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33021: Assignee: Apache Spark > Move functions related test cases into test_functions.py >

[jira] [Assigned] (SPARK-33021) Move functions related test cases into test_functions.py

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33021: Assignee: (was: Apache Spark) > Move functions related test cases into

[jira] [Commented] (SPARK-33021) Move functions related test cases into test_functions.py

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203600#comment-17203600 ] Apache Spark commented on SPARK-33021: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-09-28 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203598#comment-17203598 ] Dustin Smith commented on SPARK-32046: -- [~planga82] the fact that the dataframes are named

[jira] [Created] (SPARK-33021) Move functions related test cases into test_functions.py

2020-09-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33021: Summary: Move functions related test cases into test_functions.py Key: SPARK-33021 URL: https://issues.apache.org/jira/browse/SPARK-33021 Project: Spark

[jira] [Created] (SPARK-33020) Add nth_value in PySpark API

2020-09-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33020: Summary: Add nth_value in PySpark API Key: SPARK-33020 URL: https://issues.apache.org/jira/browse/SPARK-33020 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-33018) Fix compute statistics issue

2020-09-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33018: Affects Version/s: 3.0.0 3.0.1 > Fix compute statistics issue >

[jira] [Updated] (SPARK-32999) TreeNode.nodeName should not throw malformed class name error

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32999: -- Fix Version/s: 2.4.8 > TreeNode.nodeName should not throw malformed class name error >

[jira] [Commented] (SPARK-33006) Add dynamic PVC usage example into K8s doc

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203552#comment-17203552 ] Apache Spark commented on SPARK-33006: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-32999) TreeNode.nodeName should not throw malformed class name error

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203547#comment-17203547 ] Apache Spark commented on SPARK-32999: -- User 'rednaxelafx' has created a pull request for this

[jira] [Commented] (SPARK-32999) TreeNode.nodeName should not throw malformed class name error

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203546#comment-17203546 ] Apache Spark commented on SPARK-32999: -- User 'rednaxelafx' has created a pull request for this

[jira] [Updated] (SPARK-33019) Use spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=1 by default

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33019: -- Priority: Critical (was: Major) > Use

[jira] [Commented] (SPARK-24266) Spark client terminates while driver is still running

2020-09-28 Thread Martin Tapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203461#comment-17203461 ] Martin Tapp commented on SPARK-24266: - Could we target 3.0.2 as a release for this major fix? >

[jira] [Commented] (SPARK-33019) Use spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=1 by default

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203456#comment-17203456 ] Apache Spark commented on SPARK-33019: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33019) Use spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=1 by default

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33019: Assignee: Apache Spark > Use

[jira] [Assigned] (SPARK-33019) Use spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=1 by default

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33019: Assignee: (was: Apache Spark) > Use

[jira] [Created] (SPARK-33019) Use spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=1 by default

2020-09-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33019: - Summary: Use spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=1 by default Key: SPARK-33019 URL: https://issues.apache.org/jira/browse/SPARK-33019

[jira] [Commented] (SPARK-32996) Handle Option.empty v1.ExecutorSummary#peakMemoryMetrics

2020-09-28 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203404#comment-17203404 ] L. C. Hsieh commented on SPARK-32996: - Resolved by https://github.com/apache/spark/pull/29872. >

[jira] [Resolved] (SPARK-32996) Handle Option.empty v1.ExecutorSummary#peakMemoryMetrics

2020-09-28 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-32996. - Fix Version/s: 3.1.0 Assignee: Shruti Gumma Resolution: Fixed > Handle

[jira] [Commented] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-09-28 Thread Pablo Langa Blanco (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203345#comment-17203345 ] Pablo Langa Blanco commented on SPARK-32046: [~smilegator] why are we computing current time

[jira] [Comment Edited] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-09-28 Thread Pablo Langa Blanco (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203040#comment-17203040 ] Pablo Langa Blanco edited comment on SPARK-32046 at 9/28/20, 4:20 PM:

[jira] [Updated] (SPARK-33018) Fix compute statistics issue

2020-09-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33018: Attachment: SPARK-33018.jpg > Fix compute statistics issue > > >

[jira] [Updated] (SPARK-33012) Upgrade fabric8 to 4.10.3 to support k8s 1.18.0

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33012: -- Component/s: Build > Upgrade fabric8 to 4.10.3 to support k8s 1.18.0 >

[jira] [Updated] (SPARK-33012) Upgrade fabric8 to 4.10.3 to support k8s 1.18.0

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33012: -- Parent: SPARK-33005 Issue Type: Sub-task (was: Improvement) > Upgrade fabric8 to

[jira] [Updated] (SPARK-33012) Upgrade fabric8 to 4.10.3 to support k8s 1.18.0

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33012: -- Affects Version/s: (was: 3.0.1) 3.1.0 > Upgrade fabric8 to 4.10.3

[jira] [Commented] (SPARK-33018) Fix compute statistics issue

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203322#comment-17203322 ] Apache Spark commented on SPARK-33018: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-33018) Fix compute statistics issue

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203319#comment-17203319 ] Apache Spark commented on SPARK-33018: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33018) Fix compute statistics issue

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33018: Assignee: (was: Apache Spark) > Fix compute statistics issue >

[jira] [Assigned] (SPARK-33018) Fix compute statistics issue

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33018: Assignee: Apache Spark > Fix compute statistics issue > > >

[jira] [Commented] (SPARK-32691) Test org.apache.spark.DistributedSuite failed on arm64 jenkins

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203315#comment-17203315 ] Dongjoon Hyun commented on SPARK-32691: --- [~podongfeng], it failed again. It seems that they are

[jira] [Updated] (SPARK-32691) Test org.apache.spark.DistributedSuite failed on arm64 jenkins

2020-09-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32691: -- Attachment: Screen Shot 2020-09-28 at 8.49.04 AM.png > Test org.apache.spark.DistributedSuite

[jira] [Created] (SPARK-33018) Fix compute statistics issue

2020-09-28 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33018: --- Summary: Fix compute statistics issue Key: SPARK-33018 URL: https://issues.apache.org/jira/browse/SPARK-33018 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-33017) PySpark Context should have getCheckpointDir() method

2020-09-28 Thread Nicholas Chammas (Jira)
Nicholas Chammas created SPARK-33017: Summary: PySpark Context should have getCheckpointDir() method Key: SPARK-33017 URL: https://issues.apache.org/jira/browse/SPARK-33017 Project: Spark

[jira] [Updated] (SPARK-33000) cleanCheckpoints config does not clean all checkpointed RDDs on shutdown

2020-09-28 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-33000: - Description: Maybe it's just that the documentation needs to be updated, but I found

[jira] [Resolved] (SPARK-31900) Client memory passed unvalidated to the JVM Xmx

2020-09-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31900. -- Resolution: Won't Fix > Client memory passed unvalidated to the JVM Xmx >

[jira] [Assigned] (SPARK-32976) Support column list in INSERT statement

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32976: Assignee: (was: Apache Spark) > Support column list in INSERT statement >

[jira] [Commented] (SPARK-32976) Support column list in INSERT statement

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203225#comment-17203225 ] Apache Spark commented on SPARK-32976: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32976) Support column list in INSERT statement

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32976: Assignee: Apache Spark > Support column list in INSERT statement >

[jira] [Commented] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-09-28 Thread Pablo Langa Blanco (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203159#comment-17203159 ] Pablo Langa Blanco commented on SPARK-32046: Ok, I see, it make sense. I'll think about a

[jira] [Updated] (SPARK-33016) Potential SQLMetrics missed which might cause WEB UI display issue while AQE is on.

2020-09-28 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33016: Affects Version/s: (was: 3.0.1) 3.0.0 > Potential SQLMetrics missed

[jira] [Created] (SPARK-33016) Potential SQLMetrics missed which might cause WEB UI display issue while AQE is on.

2020-09-28 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33016: --- Summary: Potential SQLMetrics missed which might cause WEB UI display issue while AQE is on. Key: SPARK-33016 URL: https://issues.apache.org/jira/browse/SPARK-33016

[jira] [Commented] (SPARK-30796) Add parameter position for REGEXP_REPLACE

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203106#comment-17203106 ] Apache Spark commented on SPARK-30796: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-30796) Add parameter position for REGEXP_REPLACE

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30796: Assignee: (was: Apache Spark) > Add parameter position for REGEXP_REPLACE >

[jira] [Assigned] (SPARK-30796) Add parameter position for REGEXP_REPLACE

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-30796: Assignee: Apache Spark > Add parameter position for REGEXP_REPLACE >

[jira] [Updated] (SPARK-32113) Avoid coalescing shuffle partitions if joining event-based table

2020-09-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-32113: Attachment: join expansion.jpeg > Avoid coalescing shuffle partitions if joining event-based

[jira] [Updated] (SPARK-32113) Avoid coalescing shuffle partitions if joining event-based table

2020-09-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-32113: Attachment: join expansion.jpg > Avoid coalescing shuffle partitions if joining event-based table

[jira] [Commented] (SPARK-33014) Multiple bucket column not works in DataSourceV2 table

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203097#comment-17203097 ] Apache Spark commented on SPARK-33014: -- User 'LantaoJin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33014) Multiple bucket column not works in DataSourceV2 table

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33014: Assignee: (was: Apache Spark) > Multiple bucket column not works in DataSourceV2

[jira] [Assigned] (SPARK-33014) Multiple bucket column not works in DataSourceV2 table

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33014: Assignee: Apache Spark > Multiple bucket column not works in DataSourceV2 table >

[jira] [Commented] (SPARK-33015) Compute the current date only once

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203092#comment-17203092 ] Apache Spark commented on SPARK-33015: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33015) Compute the current date only once

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33015: Assignee: (was: Apache Spark) > Compute the current date only once >

[jira] [Assigned] (SPARK-33015) Compute the current date only once

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33015: Assignee: Apache Spark > Compute the current date only once >

[jira] [Commented] (SPARK-33015) Compute the current date only once

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203091#comment-17203091 ] Apache Spark commented on SPARK-33015: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Created] (SPARK-33015) Compute the current date only once

2020-09-28 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33015: -- Summary: Compute the current date only once Key: SPARK-33015 URL: https://issues.apache.org/jira/browse/SPARK-33015 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-33015) Compute the current date only once

2020-09-28 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203084#comment-17203084 ] Maxim Gekk commented on SPARK-33015: I am working on this > Compute the current date only once >

[jira] [Created] (SPARK-33014) Multiple bucket column not works in DataSourceV2 table

2020-09-28 Thread Lantao Jin (Jira)
Lantao Jin created SPARK-33014: -- Summary: Multiple bucket column not works in DataSourceV2 table Key: SPARK-33014 URL: https://issues.apache.org/jira/browse/SPARK-33014 Project: Spark Issue

[jira] [Updated] (SPARK-33013) The constrains may grow exponentially in sql optimizer 'InferFiltersFromConstraints', which leads to driver oom

2020-09-28 Thread zhou xiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhou xiang updated SPARK-33013: --- Description:      Consider the case below: {code:java}

[jira] [Created] (SPARK-33013) The constrains may grow exponentially in sql optimizer 'InferFiltersFromConstraints', which leads to driver oom

2020-09-28 Thread zhou xiang (Jira)
zhou xiang created SPARK-33013: -- Summary: The constrains may grow exponentially in sql optimizer 'InferFiltersFromConstraints', which leads to driver oom Key: SPARK-33013 URL:

[jira] [Assigned] (SPARK-33012) Upgrade fabric8 to 4.10.3 to support k8s 1.18.0

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33012: Assignee: (was: Apache Spark) > Upgrade fabric8 to 4.10.3 to support k8s 1.18.0 >

[jira] [Updated] (SPARK-33012) Upgrade fabric8 to 4.10.3 to support k8s 1.18.0

2020-09-28 Thread Jonathan Lafleche (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Lafleche updated SPARK-33012: -- Description: According to [fabric8's compatibility

[jira] [Commented] (SPARK-33012) Upgrade fabric8 to 4.10.3 to support k8s 1.18.0

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203057#comment-17203057 ] Apache Spark commented on SPARK-33012: -- User 'laflechejonathan' has created a pull request for this

[jira] [Assigned] (SPARK-33012) Upgrade fabric8 to 4.10.3 to support k8s 1.18.0

2020-09-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33012: Assignee: Apache Spark > Upgrade fabric8 to 4.10.3 to support k8s 1.18.0 >

[jira] [Created] (SPARK-33012) Upgrade fabric8 to 4.10.3 to support k8s 1.18.0

2020-09-28 Thread Jonathan Lafleche (Jira)
Jonathan Lafleche created SPARK-33012: - Summary: Upgrade fabric8 to 4.10.3 to support k8s 1.18.0 Key: SPARK-33012 URL: https://issues.apache.org/jira/browse/SPARK-33012 Project: Spark

[jira] [Comment Edited] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-09-28 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203043#comment-17203043 ] Dustin Smith edited comment on SPARK-32046 at 9/28/20, 7:12 AM:

[jira] [Commented] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-09-28 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203043#comment-17203043 ] Dustin Smith commented on SPARK-32046: -- [~planga82] the use case is related to a working

[jira] [Commented] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-09-28 Thread Pablo Langa Blanco (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203040#comment-17203040 ] Pablo Langa Blanco commented on SPARK-32046: I have been looking at the problem and I think