[jira] [Commented] (SPARK-33958) spark sql DoubleType(0 * (-1)) return "-0.0"

2021-01-04 Thread Zhang Jianguo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258109#comment-17258109 ] Zhang Jianguo commented on SPARK-33958: --- [~yumwang] Gauss and Oracle return 0. And it looks mathe

[jira] [Updated] (SPARK-33985) Transform with clusterby/orderby/sortby

2021-01-04 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-33985: -- Summary: Transform with clusterby/orderby/sortby (was: Support transform with

[jira] [Created] (SPARK-33988) Add an option to enable CBO in TPCDSQueryBenchmark

2021-01-04 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-33988: Summary: Add an option to enable CBO in TPCDSQueryBenchmark Key: SPARK-33988 URL: https://issues.apache.org/jira/browse/SPARK-33988 Project: Spark

[jira] [Commented] (SPARK-33977) Add doc for "'like any' and 'like all' operators"

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258121#comment-17258121 ] Apache Spark commented on SPARK-33977: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33977) Add doc for "'like any' and 'like all' operators"

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33977: Assignee: (was: Apache Spark) > Add doc for "'like any' and 'like all' operators" >

[jira] [Assigned] (SPARK-33977) Add doc for "'like any' and 'like all' operators"

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33977: Assignee: Apache Spark > Add doc for "'like any' and 'like all' operators" >

[jira] [Commented] (SPARK-33976) Add a dedicated SQL document page for the TRANSFORM-related functionality,

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258143#comment-17258143 ] Apache Spark commented on SPARK-33976: -- User 'AngersZh' has created a pull request for this

[jira] [Created] (SPARK-33987) v2 ALTER TABLE .. DROP PARTITION does not refresh cached table

2021-01-04 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33987: -- Summary: v2 ALTER TABLE .. DROP PARTITION does not refresh cached table Key: SPARK-33987 URL: https://issues.apache.org/jira/browse/SPARK-33987 Project: Spark

[jira] [Assigned] (SPARK-33988) Add an option to enable CBO in TPCDSQueryBenchmark

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33988: Assignee: Apache Spark > Add an option to enable CBO in TPCDSQueryBenchmark >

[jira] [Commented] (SPARK-33988) Add an option to enable CBO in TPCDSQueryBenchmark

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258155#comment-17258155 ] Apache Spark commented on SPARK-33988: -- User 'maropu' has created a pull request for this issue:

[jira] [Created] (SPARK-33989) Strip auto-generated cast when resolving UnresolvedAlias

2021-01-04 Thread ulysses you (Jira)
ulysses you created SPARK-33989: --- Summary: Strip auto-generated cast when resolving UnresolvedAlias Key: SPARK-33989 URL: https://issues.apache.org/jira/browse/SPARK-33989 Project: Spark Issue

[jira] [Commented] (SPARK-33987) v2 ALTER TABLE .. DROP PARTITION does not refresh cached table

2021-01-04 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258179#comment-17258179 ] Maxim Gekk commented on SPARK-33987: I am working on this > v2 ALTER TABLE .. DROP PARTITION does

[jira] [Created] (SPARK-33990) v2 ALTER TABLE .. DROP PARTITION does not remove data from dropped partition

2021-01-04 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33990: -- Summary: v2 ALTER TABLE .. DROP PARTITION does not remove data from dropped partition Key: SPARK-33990 URL: https://issues.apache.org/jira/browse/SPARK-33990 Project:

[jira] [Commented] (SPARK-33950) ALTER TABLE .. DROP PARTITION doesn't refresh cache

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258075#comment-17258075 ] Apache Spark commented on SPARK-33950: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33983) Update cloudpickle to v1.6.0

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258104#comment-17258104 ] Apache Spark commented on SPARK-33983: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Comment Edited] (SPARK-33982) Sparksql does not support when the inserted table is a read table

2021-01-04 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258052#comment-17258052 ] hao edited comment on SPARK-33982 at 1/4/21, 11:19 AM: --- 我认为sparksql应该得到支持insert

[jira] [Commented] (SPARK-33949) Make approx_count_distinct result consistent whether Optimize rule exists or not

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258074#comment-17258074 ] Apache Spark commented on SPARK-33949: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-33949) Make approx_count_distinct result consistent whether Optimize rule exists or not

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33949: Assignee: Apache Spark > Make approx_count_distinct result consistent whether Optimize

[jira] [Assigned] (SPARK-33949) Make approx_count_distinct result consistent whether Optimize rule exists or not

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33949: Assignee: (was: Apache Spark) > Make approx_count_distinct result consistent whether

[jira] [Assigned] (SPARK-33983) Update cloudpickle to v1.6.0

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33983: Assignee: (was: Apache Spark) > Update cloudpickle to v1.6.0 >

[jira] [Assigned] (SPARK-33983) Update cloudpickle to v1.6.0

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33983: Assignee: Apache Spark > Update cloudpickle to v1.6.0 > > >

[jira] [Comment Edited] (SPARK-33958) spark sql DoubleType(0 * (-1)) return "-0.0"

2021-01-04 Thread Zhang Jianguo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258109#comment-17258109 ] Zhang Jianguo edited comment on SPARK-33958 at 1/4/21, 9:50 AM:

[jira] [Created] (SPARK-33984) Upgrade to Py4J 0.10.9.1

2021-01-04 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33984: Summary: Upgrade to Py4J 0.10.9.1 Key: SPARK-33984 URL: https://issues.apache.org/jira/browse/SPARK-33984 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-33985) Support transform with clusterby/orderby/sortby

2021-01-04 Thread angerszhu (Jira)
angerszhu created SPARK-33985: - Summary: Support transform with clusterby/orderby/sortby Key: SPARK-33985 URL: https://issues.apache.org/jira/browse/SPARK-33985 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33990) v2 ALTER TABLE .. DROP PARTITION does not remove data from dropped partition

2021-01-04 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258183#comment-17258183 ] Maxim Gekk commented on SPARK-33990: I am working on the issue. > v2 ALTER TABLE .. DROP PARTITION

[jira] [Commented] (SPARK-33005) Kubernetes GA Preparation

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258047#comment-17258047 ] Dongjoon Hyun commented on SPARK-33005: --- Sure, [~hyukjin.kwon]. > Kubernetes GA Preparation >

[jira] [Commented] (SPARK-33984) Upgrade to Py4J 0.10.9.1

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258134#comment-17258134 ] Apache Spark commented on SPARK-33984: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-33984) Upgrade to Py4J 0.10.9.1

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33984: Assignee: (was: Apache Spark) > Upgrade to Py4J 0.10.9.1 >

[jira] [Assigned] (SPARK-33984) Upgrade to Py4J 0.10.9.1

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33984: Assignee: Apache Spark > Upgrade to Py4J 0.10.9.1 > > >

[jira] [Commented] (SPARK-33984) Upgrade to Py4J 0.10.9.1

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258135#comment-17258135 ] Apache Spark commented on SPARK-33984: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-33976) Add a dedicated SQL document page for the TRANSFORM-related functionality,

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33976: Assignee: Apache Spark > Add a dedicated SQL document page for the TRANSFORM-related

[jira] [Created] (SPARK-33986) Spark handle always return LOST status in standalone cluster mode with Spark launcher

2021-01-04 Thread ZhongyuWang (Jira)
ZhongyuWang created SPARK-33986: --- Summary: Spark handle always return LOST status in standalone cluster mode with Spark launcher Key: SPARK-33986 URL: https://issues.apache.org/jira/browse/SPARK-33986

[jira] [Assigned] (SPARK-33976) Add a dedicated SQL document page for the TRANSFORM-related functionality,

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33976: Assignee: (was: Apache Spark) > Add a dedicated SQL document page for the

[jira] [Updated] (SPARK-33986) Spark handle always return LOST status in standalone cluster mode with Spark launcher

2021-01-04 Thread ZhongyuWang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhongyuWang updated SPARK-33986: Description: I can use it to submit spark app successfully in standalone client/yarn client/yarn

[jira] [Commented] (SPARK-33976) Add a dedicated SQL document page for the TRANSFORM-related functionality,

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258141#comment-17258141 ] Apache Spark commented on SPARK-33976: -- User 'AngersZh' has created a pull request for this

[jira] [Updated] (SPARK-33985) Transform with clusterby/orderby/sortby

2021-01-04 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-33985: -- Description: Need to add UT to make sure data same with Hive > Transform with

[jira] [Created] (SPARK-33983) Update cloudpickle to v1.6.0

2021-01-04 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-33983: Summary: Update cloudpickle to v1.6.0 Key: SPARK-33983 URL: https://issues.apache.org/jira/browse/SPARK-33983 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-33988) Add an option to enable CBO in TPCDSQueryBenchmark

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33988: Assignee: (was: Apache Spark) > Add an option to enable CBO in TPCDSQueryBenchmark >

[jira] [Resolved] (SPARK-33005) Kubernetes GA Preparation

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33005. --- Resolution: Done > Kubernetes GA Preparation > - > >

[jira] [Updated] (SPARK-33711) Race condition in Spark k8s Pod lifecycle manager that leads to shutdowns

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33711: -- Parent: (was: SPARK-33005) Issue Type: Bug (was: Sub-task) > Race condition in

[jira] [Commented] (SPARK-33982) Sparksql does not support when the inserted table is a read table

2021-01-04 Thread hao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258052#comment-17258052 ] hao commented on SPARK-33982: - 我认为sparksql应该得到支持 > Sparksql does not support when the inserted table is a

[jira] [Created] (SPARK-33982) Sparksql does not support when the inserted table is a read table

2021-01-04 Thread hao (Jira)
hao created SPARK-33982: --- Summary: Sparksql does not support when the inserted table is a read table Key: SPARK-33982 URL: https://issues.apache.org/jira/browse/SPARK-33982 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-33965) CACHE TABLE does not support `spark_catalog` in Hive table names

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33965: --- Assignee: Maxim Gekk > CACHE TABLE does not support `spark_catalog` in Hive table names >

[jira] [Resolved] (SPARK-33965) CACHE TABLE does not support `spark_catalog` in Hive table names

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33965. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30997

[jira] [Commented] (SPARK-33005) Kubernetes GA Preparation

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258051#comment-17258051 ] Hyukjin Kwon commented on SPARK-33005: -- Awesome! > Kubernetes GA Preparation >

[jira] [Updated] (SPARK-33005) Kubernetes GA Preparation

2021-01-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33005: - Fix Version/s: 3.1.0 > Kubernetes GA Preparation > - > >

[jira] [Assigned] (SPARK-33978) Support ZSTD compression in ORC data source

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33978: - Assignee: Dongjoon Hyun > Support ZSTD compression in ORC data source >

[jira] [Resolved] (SPARK-33978) Support ZSTD compression in ORC data source

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33978. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31002

[jira] [Updated] (SPARK-33276) Fix K8s IT Flakiness

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33276: -- Parent: (was: SPARK-33005) Issue Type: Bug (was: Sub-task) > Fix K8s IT

[jira] [Updated] (SPARK-28895) Spark client process is unable to upload jars to hdfs while using ConfigMap not HADOOP_CONF_DIR

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28895: -- Parent: (was: SPARK-33005) Issue Type: Bug (was: Sub-task) > Spark client

[jira] [Updated] (SPARK-33349) ExecutorPodsWatchSnapshotSource: Kubernetes client has been closed

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33349: -- Parent: (was: SPARK-33005) Issue Type: Bug (was: Sub-task) >

[jira] [Updated] (SPARK-28992) Support update dependencies from hdfs when task run on executor pods

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28992: -- Parent: (was: SPARK-33005) Issue Type: Improvement (was: Sub-task) > Support

[jira] [Commented] (SPARK-33952) Python-friendly dtypes for pyspark dataframes

2021-01-04 Thread Marc de Lignie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258046#comment-17258046 ] Marc de Lignie commented on SPARK-33952: @[~hyukjin.kwon] Thanks for asking. When you write a

[jira] [Created] (SPARK-33992) resolveOperatorsUpWithNewOutput should wrap allowInvokingTransformsInAnalyzer

2021-01-04 Thread Kent Yao (Jira)
Kent Yao created SPARK-33992: Summary: resolveOperatorsUpWithNewOutput should wrap allowInvokingTransformsInAnalyzer Key: SPARK-33992 URL: https://issues.apache.org/jira/browse/SPARK-33992 Project: Spark

[jira] [Commented] (SPARK-33992) resolveOperatorsUpWithNewOutput should wrap allowInvokingTransformsInAnalyzer

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258192#comment-17258192 ] Apache Spark commented on SPARK-33992: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33992) resolveOperatorsUpWithNewOutput should wrap allowInvokingTransformsInAnalyzer

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33992: Assignee: Apache Spark > resolveOperatorsUpWithNewOutput should wrap

[jira] [Assigned] (SPARK-33990) v2 ALTER TABLE .. DROP PARTITION does not remove data from dropped partition

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33990: Assignee: Apache Spark > v2 ALTER TABLE .. DROP PARTITION does not remove data from

[jira] [Assigned] (SPARK-33990) v2 ALTER TABLE .. DROP PARTITION does not remove data from dropped partition

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33990: Assignee: (was: Apache Spark) > v2 ALTER TABLE .. DROP PARTITION does not remove

[jira] [Commented] (SPARK-33991) Repair enumeration conversion error for AllJobsPage

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258228#comment-17258228 ] Apache Spark commented on SPARK-33991: -- User 'FelixYik' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33991) Repair enumeration conversion error for AllJobsPage

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33991: Assignee: Apache Spark > Repair enumeration conversion error for AllJobsPage >

[jira] [Assigned] (SPARK-33991) Repair enumeration conversion error for AllJobsPage

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33991: Assignee: (was: Apache Spark) > Repair enumeration conversion error for AllJobsPage

[jira] [Created] (SPARK-33991) Repair enumeration conversion error for page showing list

2021-01-04 Thread kaif Yi (Jira)
kaif Yi created SPARK-33991: --- Summary: Repair enumeration conversion error for page showing list Key: SPARK-33991 URL: https://issues.apache.org/jira/browse/SPARK-33991 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33991) Repair enumeration conversion error for page showing list

2021-01-04 Thread kaif Yi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kaif Yi updated SPARK-33991: Description: For AllJobsPage class, AllJobsPage gets the schedulingMode of enumerated type by loading the

[jira] [Updated] (SPARK-33991) Repair enumeration conversion error for AllJobsPage

2021-01-04 Thread Felix Yi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Yi updated SPARK-33991: - Description: For AllJobsPage class, AllJobsPage gets the schedulingMode of enumerated type by loading

[jira] [Comment Edited] (SPARK-33948) branch-3.1 jenkins test failed in Scala 2.13

2021-01-04 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258217#comment-17258217 ] Yang Jie edited comment on SPARK-33948 at 1/4/21, 2:00 PM: --- *Sync:*

[jira] [Updated] (SPARK-33991) Repair enumeration conversion error for page showing list of all ongoing and recently finished jobs

2021-01-04 Thread kaif Yi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kaif Yi updated SPARK-33991: Summary: Repair enumeration conversion error for page showing list of all ongoing and recently finished

[jira] [Updated] (SPARK-33991) Repair enumeration conversion error for AllJobsPage

2021-01-04 Thread kaif Yi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kaif Yi updated SPARK-33991: Summary: Repair enumeration conversion error for AllJobsPage (was: Repair enumeration conversion error

[jira] [Commented] (SPARK-33990) v2 ALTER TABLE .. DROP PARTITION does not remove data from dropped partition

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258195#comment-17258195 ] Apache Spark commented on SPARK-33990: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33736) Handle MERGE in ReplaceNullWithFalseInPredicate

2021-01-04 Thread Anton Okolnychyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258222#comment-17258222 ] Anton Okolnychyi commented on SPARK-33736: -- Sorry, I was on holidays. Will get back to the PR

[jira] [Created] (SPARK-33994) ORC encryption interop

2021-01-04 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created SPARK-33994: Summary: ORC encryption interop Key: SPARK-33994 URL: https://issues.apache.org/jira/browse/SPARK-33994 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-33991) Repair enumeration conversion error for AllJobsPage

2021-01-04 Thread Felix Yi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Yi updated SPARK-33991: - Component/s: (was: Web UI) Spark Core > Repair enumeration conversion error for

[jira] [Commented] (SPARK-33991) Repair enumeration conversion error for AllJobsPage

2021-01-04 Thread Felix Yi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258201#comment-17258201 ] Felix Yi commented on SPARK-33991: -- I saw that the #org.apache.spark.scheduler.TaskSchedulerImpl class

[jira] [Commented] (SPARK-25075) Build and test Spark against Scala 2.13

2021-01-04 Thread Guillaume Martres (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258218#comment-17258218 ] Guillaume Martres commented on SPARK-25075: --- Now that 2.13 support is basically complete,

[jira] [Commented] (SPARK-33948) branch-3.1 jenkins test failed in Scala 2.13

2021-01-04 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258217#comment-17258217 ] Yang Jie commented on SPARK-33948: -- *Sync:* {code:java} commit 58583f7c3fdcac1232607a7ab4b0d052320ac3ea

[jira] [Assigned] (SPARK-33992) resolveOperatorsUpWithNewOutput should wrap allowInvokingTransformsInAnalyzer

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33992: Assignee: (was: Apache Spark) > resolveOperatorsUpWithNewOutput should wrap

[jira] [Created] (SPARK-33993) Parquet encryption interop

2021-01-04 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created SPARK-33993: Summary: Parquet encryption interop Key: SPARK-33993 URL: https://issues.apache.org/jira/browse/SPARK-33993 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-33875) Implement DESCRIBE COLUMN for v2 catalog

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33875: --- Assignee: Terry Kim > Implement DESCRIBE COLUMN for v2 catalog >

[jira] [Assigned] (SPARK-33984) Upgrade to Py4J 0.10.9.1

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33984: - Assignee: Hyukjin Kwon > Upgrade to Py4J 0.10.9.1 > > >

[jira] [Resolved] (SPARK-33984) Upgrade to Py4J 0.10.9.1

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33984. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31009

[jira] [Resolved] (SPARK-33875) Implement DESCRIBE COLUMN for v2 catalog

2021-01-04 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33875. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 30881

[jira] [Commented] (SPARK-31786) Exception on submitting Spark-Pi to Kubernetes 1.17.3

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258383#comment-17258383 ] Dongjoon Hyun commented on SPARK-31786: --- Did you do `export HTTP2_DISABLE=true` before

[jira] [Resolved] (SPARK-33990) v2 ALTER TABLE .. DROP PARTITION does not remove data from dropped partition

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33990. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31014

[jira] [Assigned] (SPARK-33990) v2 ALTER TABLE .. DROP PARTITION does not remove data from dropped partition

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33990: - Assignee: Maxim Gekk > v2 ALTER TABLE .. DROP PARTITION does not remove data from

[jira] [Resolved] (SPARK-33988) Add an option to enable CBO in TPCDSQueryBenchmark

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33988. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31011

[jira] [Assigned] (SPARK-33988) Add an option to enable CBO in TPCDSQueryBenchmark

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33988: - Assignee: Takeshi Yamamuro > Add an option to enable CBO in TPCDSQueryBenchmark >

[jira] [Commented] (SPARK-31786) Exception on submitting Spark-Pi to Kubernetes 1.17.3

2021-01-04 Thread Sachit Murarka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258345#comment-17258345 ] Sachit Murarka commented on SPARK-31786: [~maver1ck] / [~dongjoon] : I am facing this issue. I

[jira] [Updated] (SPARK-33988) Add an option to enable CBO in TPCDSQueryBenchmark

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33988: -- Parent: (was: SPARK-33828) Issue Type: Improvement (was: Sub-task) > Add an

[jira] [Updated] (SPARK-33988) Add an option to enable CBO in TPCDSQueryBenchmark

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33988: -- Parent: SPARK-33828 Issue Type: Sub-task (was: Test) > Add an option to enable CBO

[jira] [Assigned] (SPARK-33983) Update cloudpickle to v1.6.0

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33983: - Assignee: Hyukjin Kwon > Update cloudpickle to v1.6.0 > >

[jira] [Resolved] (SPARK-33983) Update cloudpickle to v1.6.0

2021-01-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33983. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31007

[jira] [Comment Edited] (SPARK-31786) Exception on submitting Spark-Pi to Kubernetes 1.17.3

2021-01-04 Thread Sachit Murarka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258411#comment-17258411 ] Sachit Murarka edited comment on SPARK-31786 at 1/4/21, 6:41 PM: -

[jira] [Commented] (SPARK-33908) Refact SparkSubmitUtils.resolveMavenCoordinates return parameter

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258454#comment-17258454 ] Apache Spark commented on SPARK-33908: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-33987) v2 ALTER TABLE .. DROP PARTITION does not refresh cached table

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258479#comment-17258479 ] Apache Spark commented on SPARK-33987: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33987) v2 ALTER TABLE .. DROP PARTITION does not refresh cached table

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33987: Assignee: Apache Spark > v2 ALTER TABLE .. DROP PARTITION does not refresh cached table

[jira] [Assigned] (SPARK-33987) v2 ALTER TABLE .. DROP PARTITION does not refresh cached table

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33987: Assignee: (was: Apache Spark) > v2 ALTER TABLE .. DROP PARTITION does not refresh

[jira] [Assigned] (SPARK-33894) Word2VecSuite failed for Scala 2.13

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33894: Assignee: (was: Apache Spark) > Word2VecSuite failed for Scala 2.13 >

[jira] [Commented] (SPARK-33894) Word2VecSuite failed for Scala 2.13

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258485#comment-17258485 ] Apache Spark commented on SPARK-33894: -- User 'koertkuipers' has created a pull request for this

[jira] [Assigned] (SPARK-33894) Word2VecSuite failed for Scala 2.13

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33894: Assignee: Apache Spark > Word2VecSuite failed for Scala 2.13 >

[jira] [Commented] (SPARK-33894) Word2VecSuite failed for Scala 2.13

2021-01-04 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258486#comment-17258486 ] Apache Spark commented on SPARK-33894: -- User 'koertkuipers' has created a pull request for this

[jira] [Commented] (SPARK-31786) Exception on submitting Spark-Pi to Kubernetes 1.17.3

2021-01-04 Thread Sachit Murarka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258411#comment-17258411 ] Sachit Murarka commented on SPARK-31786: [~dongjoon] -> Yes I have used `export

<    1   2   3   >