[jira] [Commented] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2020-10-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218823#comment-17218823 ] Yuming Wang commented on SPARK-33207: - But it still launch #-of-buckets tasks. May be we can change 

[jira] [Assigned] (SPARK-33218) Update misleading log messages for removed shuffle blocks

2020-10-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33218: - Assignee: Dongjoon Hyun > Update misleading log messages for removed shuffle blocks >

[jira] [Created] (SPARK-33218) Update misleading log messages for removed shuffle blocks

2020-10-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33218: - Summary: Update misleading log messages for removed shuffle blocks Key: SPARK-33218 URL: https://issues.apache.org/jira/browse/SPARK-33218 Project: Spark

[jira] [Assigned] (SPARK-32354) Fix & re-enable failing R K8s tests

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32354: Assignee: Apache Spark > Fix & re-enable failing R K8s tests >

[jira] [Assigned] (SPARK-32354) Fix & re-enable failing R K8s tests

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32354: Assignee: (was: Apache Spark) > Fix & re-enable failing R K8s tests >

[jira] [Commented] (SPARK-32354) Fix & re-enable failing R K8s tests

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218838#comment-17218838 ] Apache Spark commented on SPARK-32354: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33218) Update misleading log messages for removed shuffle blocks

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33218: Assignee: (was: Apache Spark) > Update misleading log messages for removed shuffle

[jira] [Commented] (SPARK-33218) Update misleading log messages for removed shuffle blocks

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218798#comment-17218798 ] Apache Spark commented on SPARK-33218: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-30796) Add parameter position for REGEXP_REPLACE

2020-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30796. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29891

[jira] [Assigned] (SPARK-30796) Add parameter position for REGEXP_REPLACE

2020-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30796: --- Assignee: jiaan.geng > Add parameter position for REGEXP_REPLACE >

[jira] [Resolved] (SPARK-33218) Update misleading log messages for removed shuffle blocks

2020-10-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33218. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30129

[jira] [Commented] (SPARK-33218) Update misleading log messages for removed shuffle blocks

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218799#comment-17218799 ] Apache Spark commented on SPARK-33218: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33218) Update misleading log messages for removed shuffle blocks

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33218: Assignee: Apache Spark > Update misleading log messages for removed shuffle blocks >

[jira] [Commented] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2020-10-22 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218827#comment-17218827 ] Cheng Su commented on SPARK-33207: -- If you filter out empty `FilePartition` here, then `FileScanRDD`

[jira] [Updated] (SPARK-32085) Migrate to NumPy documentation style

2020-10-22 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-32085: --- Description: https://github.com/numpy/numpydoc For example, Before:

[jira] [Updated] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2020-10-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33207: Attachment: image-2020-10-22-15-17-01-389.png > Reduce the number of tasks launched after bucket

[jira] [Updated] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2020-10-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33207: Attachment: image-2020-10-22-15-17-26-956.png > Reduce the number of tasks launched after bucket

[jira] [Updated] (SPARK-33219) Disable GetPrimaryKeys and GetCrossReference APIs explicitly for Spark ThriftServer

2020-10-22 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-33219: - Priority: Minor (was: Major) > Disable GetPrimaryKeys and GetCrossReference APIs explicitly for Spark

[jira] [Created] (SPARK-33219) Disable GetPrimaryKeys and GetCrossReference APIs explicitly for Spark ThriftServer

2020-10-22 Thread Kent Yao (Jira)
Kent Yao created SPARK-33219: Summary: Disable GetPrimaryKeys and GetCrossReference APIs explicitly for Spark ThriftServer Key: SPARK-33219 URL: https://issues.apache.org/jira/browse/SPARK-33219 Project:

[jira] [Updated] (SPARK-33219) Disable GetPrimaryKeys and GetCrossReference APIs explicitly for Spark ThriftServer

2020-10-22 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-33219: - Issue Type: Improvement (was: Bug) > Disable GetPrimaryKeys and GetCrossReference APIs explicitly for

[jira] [Created] (SPARK-33220) `scheduleAtFixedRate` change to `schedulerWithFixedDelay` to avoid repeated unecessary scheduling for a short time

2020-10-22 Thread angerszhu (Jira)
angerszhu created SPARK-33220: - Summary: `scheduleAtFixedRate` change to `schedulerWithFixedDelay` to avoid repeated unecessary scheduling for a short time Key: SPARK-33220 URL:

[jira] [Resolved] (SPARK-26533) Support query auto cancel on thriftserver

2020-10-22 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-26533. -- Fix Version/s: 3.1.0 Assignee: Xuedong Luan Resolution: Fixed

[jira] [Assigned] (SPARK-33217) Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4

2020-10-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33217: Assignee: Hyukjin Kwon > Set upper bound of Pandas and PyArrow version in GitHub Actions

[jira] [Created] (SPARK-33221) UDF in when operation applied in all the rows regardless the condition

2020-10-22 Thread Marco De Nadai (Jira)
Marco De Nadai created SPARK-33221: -- Summary: UDF in when operation applied in all the rows regardless the condition Key: SPARK-33221 URL: https://issues.apache.org/jira/browse/SPARK-33221 Project:

[jira] [Commented] (SPARK-33222) Expose missing information (graphs) on SS UI

2020-10-22 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218974#comment-17218974 ] Gabor Somogyi commented on SPARK-33222: --- I've started to implement it. > Expose missing

[jira] [Created] (SPARK-33224) Expose watermark information on SS UI

2020-10-22 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-33224: - Summary: Expose watermark information on SS UI Key: SPARK-33224 URL: https://issues.apache.org/jira/browse/SPARK-33224 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-33220) `scheduleAtFixedRate` change to `schedulerWithFixedDelay` to avoid repeated unecessary scheduling for a short time

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33220: Assignee: Apache Spark > `scheduleAtFixedRate` change to `schedulerWithFixedDelay` to

[jira] [Commented] (SPARK-33220) `scheduleAtFixedRate` change to `schedulerWithFixedDelay` to avoid repeated unecessary scheduling for a short time

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218882#comment-17218882 ] Apache Spark commented on SPARK-33220: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-33160) Allow saving/loading INT96 in parquet w/o rebasing

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218969#comment-17218969 ] Apache Spark commented on SPARK-33160: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Created] (SPARK-33222) Expose missing information (graphs) on SS UI

2020-10-22 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-33222: - Summary: Expose missing information (graphs) on SS UI Key: SPARK-33222 URL: https://issues.apache.org/jira/browse/SPARK-33222 Project: Spark Issue Type:

[jira] [Created] (SPARK-33223) Expose state information on SS UI

2020-10-22 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-33223: - Summary: Expose state information on SS UI Key: SPARK-33223 URL: https://issues.apache.org/jira/browse/SPARK-33223 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-33217) Set upper bound of Pandas and PyArrow version in GitHub Actions in branch-2.4

2020-10-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33217. -- Fix Version/s: 2.4.8 Resolution: Fixed Issue resolved by pull request 30128

[jira] [Commented] (SPARK-33220) `scheduleAtFixedRate` change to `schedulerWithFixedDelay` to avoid repeated unecessary scheduling for a short time

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218883#comment-17218883 ] Apache Spark commented on SPARK-33220: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-33220) `scheduleAtFixedRate` change to `schedulerWithFixedDelay` to avoid repeated unecessary scheduling for a short time

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33220: Assignee: (was: Apache Spark) > `scheduleAtFixedRate` change to

[jira] [Commented] (SPARK-33207) Reduce the number of tasks launched after bucket pruning

2020-10-22 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219458#comment-17219458 ] Cheng Su commented on SPARK-33207: -- [~yumwang] - I recently added the functionality to disable bucketed

[jira] [Assigned] (SPARK-32991) RESET can clear StaticSQLConfs

2020-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32991: --- Assignee: Kent Yao > RESET can clear StaticSQLConfs > -- > >

[jira] [Assigned] (SPARK-33226) Upgrade to SBT 1.4.1

2020-10-22 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-33226: --- Assignee: Dongjoon Hyun > Upgrade to SBT 1.4.1 > > >

[jira] [Resolved] (SPARK-32991) RESET can clear StaticSQLConfs

2020-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32991. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30045

[jira] [Resolved] (SPARK-33226) Upgrade to SBT 1.4.1

2020-10-22 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-33226. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30137

[jira] [Commented] (SPARK-33226) Upgrade to SBT 1.4.1

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219421#comment-17219421 ] Apache Spark commented on SPARK-33226: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33226) Upgrade to SBT 1.4.1

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33226: Assignee: Apache Spark > Upgrade to SBT 1.4.1 > > >

[jira] [Created] (SPARK-33226) Upgrade to SBT 1.4.1

2020-10-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33226: - Summary: Upgrade to SBT 1.4.1 Key: SPARK-33226 URL: https://issues.apache.org/jira/browse/SPARK-33226 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-33226) Upgrade to SBT 1.4.1

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33226: Assignee: (was: Apache Spark) > Upgrade to SBT 1.4.1 > > >

[jira] [Assigned] (SPARK-33104) Fix `YarnClusterSuite.yarn-cluster should respect conf overrides in SparkHadoopUtil`

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33104: Assignee: (was: Apache Spark) > Fix `YarnClusterSuite.yarn-cluster should respect

[jira] [Created] (SPARK-33225) Extract AliasHelper trait

2020-10-22 Thread Tanel Kiis (Jira)
Tanel Kiis created SPARK-33225: -- Summary: Extract AliasHelper trait Key: SPARK-33225 URL: https://issues.apache.org/jira/browse/SPARK-33225 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-32978) Incorrect number of dynamic part metric

2020-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32978. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30026

[jira] [Resolved] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

2020-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33095. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30025

[jira] [Assigned] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

2020-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33095: --- Assignee: Prashant Sharma > Support ALTER TABLE in JDBC v2 Table Catalog: add, update type

[jira] [Assigned] (SPARK-32978) Incorrect number of dynamic part metric

2020-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32978: --- Assignee: Yang Jie > Incorrect number of dynamic part metric >

[jira] [Assigned] (SPARK-32852) spark.sql.hive.metastore.jars support HDFS location

2020-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32852: --- Assignee: angerszhu > spark.sql.hive.metastore.jars support HDFS location >

[jira] [Resolved] (SPARK-32852) spark.sql.hive.metastore.jars support HDFS location

2020-10-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32852. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29881

[jira] [Commented] (SPARK-33104) Fix `YarnClusterSuite.yarn-cluster should respect conf overrides in SparkHadoopUtil`

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219083#comment-17219083 ] Apache Spark commented on SPARK-33104: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-33104) Fix `YarnClusterSuite.yarn-cluster should respect conf overrides in SparkHadoopUtil`

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33104: Assignee: Apache Spark > Fix `YarnClusterSuite.yarn-cluster should respect conf

[jira] [Updated] (SPARK-33145) In Execution web page, when `Succeeded Job` has many child url elements,they will extend over the edge of the page.

2020-10-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-33145: - Fix Version/s: (was: 3.1.0) > In Execution web page, when `Succeeded Job` has many child

[jira] [Updated] (SPARK-33204) `Event Timeline` in Spark Job UI sometimes cannot be opened

2020-10-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-33204: - Fix Version/s: (was: 3.1.0) > `Event Timeline` in Spark Job UI sometimes cannot be opened

[jira] [Updated] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2020-10-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33212: -- Labels: releasenotes (was: ) > Move to shaded clients for Hadoop 3.x profile >

[jira] [Commented] (SPARK-29250) Upgrade to Hadoop 3.3.0

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219255#comment-17219255 ] Apache Spark commented on SPARK-29250: -- User 'sunchao' has created a pull request for this issue:

[jira] [Updated] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2020-10-22 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-33212: - Description: Hadoop 3.x+ offers shaded client jars: hadoop-client-api and hadoop-client-runtime, which

[jira] [Updated] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2020-10-22 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-33212: - Description: Hadoop 3.x+ offers shaded client jars: hadoop-client-api and hadoop-client-runtime, which

[jira] [Updated] (SPARK-33212) Move to shaded clients for Hadoop 3.x profile

2020-10-22 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-33212: - Description: Hadoop 3.x+ offers shaded client jars: hadoop-client-api and hadoop-client-runtime, which

[jira] [Assigned] (SPARK-33225) Extract AliasHelper trait

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33225: Assignee: Apache Spark > Extract AliasHelper trait > - > >

[jira] [Commented] (SPARK-33225) Extract AliasHelper trait

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219257#comment-17219257 ] Apache Spark commented on SPARK-33225: -- User 'tanelk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33225) Extract AliasHelper trait

2020-10-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33225: Assignee: (was: Apache Spark) > Extract AliasHelper trait >

[jira] [Updated] (SPARK-29250) Upgrade to Hadoop 3.3.0

2020-10-22 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-29250: - Summary: Upgrade to Hadoop 3.3.0 (was: Upgrade to Hadoop 3.2.2) > Upgrade to Hadoop 3.3.0 >

[jira] [Updated] (SPARK-32461) Shuffled hash join improvement

2020-10-22 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-32461: Priority: Major (was: Minor) > Shuffled hash join improvement > -- > >

[jira] [Updated] (SPARK-30602) SPIP: Support push-based shuffle to improve shuffle efficiency

2020-10-22 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30602: Labels: release-notes (was: ) > SPIP: Support push-based shuffle to improve shuffle efficiency >

[jira] [Updated] (SPARK-32461) Shuffled hash join improvement

2020-10-22 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-32461: Labels: release-notes (was: ) > Shuffled hash join improvement > -- > >

[jira] [Comment Edited] (SPARK-33221) UDF in when operation applied in all the rows regardless the condition

2020-10-22 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219390#comment-17219390 ] Aoyuan Liao edited comment on SPARK-33221 at 10/23/20, 12:28 AM: -

[jira] [Resolved] (SPARK-33221) UDF in when operation applied in all the rows regardless the condition

2020-10-22 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aoyuan Liao resolved SPARK-33221. - Resolution: Not A Problem > UDF in when operation applied in all the rows regardless the

[jira] [Assigned] (SPARK-32998) Add ability to override default remote repos with internal repos only

2020-10-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32998: - Assignee: Ankit Srivastava > Add ability to override default remote repos with

[jira] [Resolved] (SPARK-32998) Add ability to override default remote repos with internal repos only

2020-10-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32998. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29874

[jira] [Commented] (SPARK-33221) UDF in when operation applied in all the rows regardless the condition

2020-10-22 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219390#comment-17219390 ] Aoyuan Liao commented on SPARK-33221: - [~marcodena] It is not a bug but intended. In