[jira] [Updated] (SPARK-32751) Add code-gen for ObjectHashAggregateExec

2021-01-28 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32751: - Parent: SPARK-34287 Issue Type: Sub-task (was: Improvement) > Add code-gen for ObjectHashAggreg

[jira] [Updated] (SPARK-32750) Add code-gen for SortAggregateExec

2021-01-28 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-32750: - Parent: SPARK-34287 Issue Type: Sub-task (was: Improvement) > Add code-gen for SortAggregateExe

[jira] [Commented] (SPARK-34287) Object hash and sort aggregation execution improvement

2021-01-28 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274191#comment-17274191 ] Cheng Su commented on SPARK-34287: -- Just FYI - I am working on each sub-task. Thanks.

[jira] [Updated] (SPARK-34286) Memory-size-based object hash aggregation fallback mechanism

2021-01-28 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-34286: - Parent: SPARK-34287 Issue Type: Sub-task (was: New Feature) > Memory-size-based object hash agg

[jira] [Updated] (SPARK-34253) Object hash aggregate should not fallback if no more input rows

2021-01-28 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-34253: - Parent: SPARK-34287 Issue Type: Sub-task (was: Improvement) > Object hash aggregate should not

[jira] [Updated] (SPARK-34237) Add more metrics (fallback, spill) for object hash aggregate

2021-01-28 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-34237: - Parent: SPARK-34287 Issue Type: Sub-task (was: Improvement) > Add more metrics (fallback, spill

[jira] [Updated] (SPARK-34287) Object hash and sort aggregation execution improvement

2021-01-28 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Su updated SPARK-34287: - Issue Type: Umbrella (was: Improvement) > Object hash and sort aggregation execution improvement >

[jira] [Created] (SPARK-34287) Object hash and sort aggregation execution improvement

2021-01-28 Thread Cheng Su (Jira)
Cheng Su created SPARK-34287: Summary: Object hash and sort aggregation execution improvement Key: SPARK-34287 URL: https://issues.apache.org/jira/browse/SPARK-34287 Project: Spark Issue Type: Im

[jira] [Resolved] (SPARK-34269) simplify view resolution

2021-01-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34269. - Fix Version/s: 3.2.0 Resolution: Fixed > simplify view resolution > -

[jira] [Created] (SPARK-34286) Memory-size-based object hash aggregation fallback mechanism

2021-01-28 Thread Cheng Su (Jira)
Cheng Su created SPARK-34286: Summary: Memory-size-based object hash aggregation fallback mechanism Key: SPARK-34286 URL: https://issues.apache.org/jira/browse/SPARK-34286 Project: Spark Issue T

[jira] [Commented] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274153#comment-17274153 ] Apache Spark commented on SPARK-33212: -- User 'bozhang2820' has created a pull reque

[jira] [Commented] (SPARK-33212) Upgrade to Hadoop 3.2.2 and move to shaded clients for Hadoop 3.x profile

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274154#comment-17274154 ] Apache Spark commented on SPARK-33212: -- User 'bozhang2820' has created a pull reque

[jira] [Updated] (SPARK-34285) Implement Parquet StringEndsWith、StringContains Filter

2021-01-28 Thread Xudingyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xudingyu updated SPARK-34285: - Summary: Implement Parquet StringEndsWith、StringContains Filter (was: Implement StringEndWith、StringCon

[jira] [Created] (SPARK-34285) Implement StringEndWith、StringContains Filter

2021-01-28 Thread Xudingyu (Jira)
Xudingyu created SPARK-34285: Summary: Implement StringEndWith、StringContains Filter Key: SPARK-34285 URL: https://issues.apache.org/jira/browse/SPARK-34285 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-10388) Public dataset loader interface

2021-01-28 Thread louischoi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274151#comment-17274151 ] louischoi commented on SPARK-10388: --- Anyone is working on this? look it is discussed l

[jira] [Resolved] (SPARK-34237) Add more metrics (fallback, spill) for object hash aggregate

2021-01-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34237. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31340 [https://gith

[jira] [Assigned] (SPARK-34237) Add more metrics (fallback, spill) for object hash aggregate

2021-01-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34237: --- Assignee: Cheng Su > Add more metrics (fallback, spill) for object hash aggregate > ---

[jira] [Commented] (SPARK-28123) String Functions: Add support btrim

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274141#comment-17274141 ] Apache Spark commented on SPARK-28123: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-34284) Fix deprecated API usage of commons-io

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34284: Assignee: (was: Apache Spark) > Fix deprecated API usage of commons-io >

[jira] [Assigned] (SPARK-34284) Fix deprecated API usage of commons-io

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34284: Assignee: Apache Spark > Fix deprecated API usage of commons-io > ---

[jira] [Commented] (SPARK-34284) Fix deprecated API usage of commons-io

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274139#comment-17274139 ] Apache Spark commented on SPARK-34284: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274135#comment-17274135 ] Zhichao Zhang commented on SPARK-34283: I will handle this issue > Combines al

[jira] [Updated] (SPARK-34260) UnresolvedException when creating temp view twice

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34260: -- Affects Version/s: 2.4.7 > UnresolvedException when creating temp view twice > ---

[jira] [Updated] (SPARK-34260) UnresolvedException when creating temp view twice

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34260: -- Fix Version/s: 2.4.8 > UnresolvedException when creating temp view twice > ---

[jira] [Created] (SPARK-34284) Fix deprecated API usage of commons-io

2021-01-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-34284: Summary: Fix deprecated API usage of commons-io Key: SPARK-34284 URL: https://issues.apache.org/jira/browse/SPARK-34284 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Description: Problem: Currently when using 'Dataset.union.distinct' to union some datasets,

[jira] [Updated] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Description: Problem: Currently when using 'Dataset.union.distinct' to union some datasets,

[jira] [Updated] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Description: Problem: Currently when using 'Dataset.union.distinct' to union some datasets,

[jira] [Updated] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Description: Problem: Currently when using 'Dataset.union.distinct' to union some datasets,

[jira] [Updated] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Attachment: image-2021-01-29-11-14-42-700.png > Combines all adjacent 'Union' operators into

[jira] [Updated] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Attachment: image-2021-01-29-11-14-08-822.png > Combines all adjacent 'Union' operators into

[jira] [Updated] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Attachment: image-2021-01-29-11-13-42-055.png > Combines all adjacent 'Union' operators into

[jira] [Updated] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Attachment: image-2021-01-29-11-12-44-112.png > Combines all adjacent 'Union' operators into

[jira] [Updated] (SPARK-34283) Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Summary: Combines all adjacent 'Union' operators into a single 'Union' when using 'Dataset.u

[jira] [Updated] (SPARK-34283) all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Description: Problem: Currently when using 'Dataset.union.distinct' to union some datasets,

[jira] [Updated] (SPARK-34283) all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.distinct'

2021-01-28 Thread Zhichao Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated SPARK-34283: --- Summary: all adjacent 'Union' operators into a single 'Union' when using 'Dataset.union.dist

[jira] [Created] (SPARK-34283) Combine 'Deduplicate(columns, Union)' operators into a single union

2021-01-28 Thread Zhichao Zhang (Jira)
Zhichao Zhang created SPARK-34283: -- Summary: Combine 'Deduplicate(columns, Union)' operators into a single union Key: SPARK-34283 URL: https://issues.apache.org/jira/browse/SPARK-34283 Project: Spar

[jira] [Commented] (SPARK-34260) UnresolvedException when creating temp view twice

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274113#comment-17274113 ] Apache Spark commented on SPARK-34260: -- User 'linhongliu-db' has created a pull req

[jira] [Commented] (SPARK-34260) UnresolvedException when creating temp view twice

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274112#comment-17274112 ] Apache Spark commented on SPARK-34260: -- User 'linhongliu-db' has created a pull req

[jira] [Updated] (SPARK-33867) java.time.Instant and java.time.LocalDate not handled in org.apache.spark.sql.jdbc.JdbcDialect#compileValue

2021-01-28 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-33867: - Fix Version/s: 3.0.2 > java.time.Instant and java.time.LocalDate not handled in > org.a

[jira] [Resolved] (SPARK-34261) Avoid side effect if create exists temporary function

2021-01-28 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-34261. -- Fix Version/s: 3.2.0 Assignee: ulysses you Resolution: Fixed Resolved

[jira] [Commented] (SPARK-34276) Check the unreleased/unresolved JIRAs/PRs of Parquet 1.11

2021-01-28 Thread Michael Heuer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274065#comment-17274065 ] Michael Heuer commented on SPARK-34276: --- This downstream pull request builds on th

[jira] [Updated] (SPARK-26346) Upgrade parquet to 1.11.1

2021-01-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-26346: Description: Parquet 1.11 new features: PARQUET-1201 - Column indexes PARQUET-1253 - Support for

[jira] [Updated] (SPARK-26346) Upgrade parquet to 1.11.1

2021-01-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-26346: Description: Parquet 1.11.1 new features: PARQUET-1201 - Column indexes PARQUET-1253 - Support fo

[jira] [Resolved] (SPARK-26346) Upgrade parquet to 1.11.1

2021-01-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-26346. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 26804 [https://gith

[jira] [Assigned] (SPARK-26346) Upgrade parquet to 1.11.1

2021-01-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-26346: --- Assignee: Yuming Wang > Upgrade parquet to 1.11.1 > - > >

[jira] [Updated] (SPARK-34273) Do not reregister BlockManager when SparkContext is stopped

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34273: -- Fix Version/s: (was: 3.1.0) 3.1.1 > Do not reregister BlockManager when

[jira] [Resolved] (SPARK-34273) Do not reregister BlockManager when SparkContext is stopped

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34273. --- Fix Version/s: 2.4.8 3.0.2 3.1.0 Resolution: Fix

[jira] [Assigned] (SPARK-34273) Do not reregister BlockManager when SparkContext is stopped

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34273: - Assignee: Dongjoon Hyun > Do not reregister BlockManager when SparkContext is stopped >

[jira] [Updated] (SPARK-34281) Promote spark.kubernetes.executor.podNamePrefix to the public conf

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34281: -- Fix Version/s: (was: 3.1.0) 3.1.1 > Promote spark.kubernetes.executor.p

[jira] [Updated] (SPARK-34281) Promote spark.kubernetes.executor.podNamePrefix to the public conf

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34281: -- Affects Version/s: (was: 3.2.0) > Promote spark.kubernetes.executor.podNamePrefix to the p

[jira] [Assigned] (SPARK-34281) Promote spark.kubernetes.executor.podNamePrefix to the public conf

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34281: - Assignee: Dongjoon Hyun > Promote spark.kubernetes.executor.podNamePrefix to the public

[jira] [Resolved] (SPARK-34281) Promote spark.kubernetes.executor.podNamePrefix to the public conf

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34281. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 31386 [https://

[jira] [Commented] (SPARK-34282) Unify v1 and v2 TRUNCATE TABLE tests

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273979#comment-17273979 ] Apache Spark commented on SPARK-34282: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-34282) Unify v1 and v2 TRUNCATE TABLE tests

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34282: Assignee: Maxim Gekk (was: Apache Spark) > Unify v1 and v2 TRUNCATE TABLE tests > --

[jira] [Assigned] (SPARK-34282) Unify v1 and v2 TRUNCATE TABLE tests

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34282: Assignee: Apache Spark (was: Maxim Gekk) > Unify v1 and v2 TRUNCATE TABLE tests > --

[jira] [Commented] (SPARK-34282) Unify v1 and v2 TRUNCATE TABLE tests

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273978#comment-17273978 ] Apache Spark commented on SPARK-34282: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-34281) Promote spark.kubernetes.executor.podNamePrefix to the public conf

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273972#comment-17273972 ] Apache Spark commented on SPARK-34281: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-34281) Promote spark.kubernetes.executor.podNamePrefix to the public conf

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34281: Assignee: Apache Spark > Promote spark.kubernetes.executor.podNamePrefix to the public co

[jira] [Assigned] (SPARK-34281) Promote spark.kubernetes.executor.podNamePrefix to the public conf

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34281: Assignee: (was: Apache Spark) > Promote spark.kubernetes.executor.podNamePrefix to th

[jira] [Commented] (SPARK-34281) Promote spark.kubernetes.executor.podNamePrefix to the public conf

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273971#comment-17273971 ] Apache Spark commented on SPARK-34281: -- User 'dongjoon-hyun' has created a pull req

[jira] [Updated] (SPARK-34282) Unify v1 and v2 TRUNCATE TABLE tests

2021-01-28 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34282: --- Description: Extract TRUNCATE TABLE tests to the common place to run them for V1 and v2 datasources.

[jira] [Created] (SPARK-34282) Unify v1 and v2 TRUNCATE TABLE tests

2021-01-28 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34282: -- Summary: Unify v1 and v2 TRUNCATE TABLE tests Key: SPARK-34282 URL: https://issues.apache.org/jira/browse/SPARK-34282 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-34281) Promote spark.kubernetes.executor.podNamePrefix to the public conf

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-34281: -- Summary: Promote spark.kubernetes.executor.podNamePrefix to the public conf (was: Promote spa

[jira] [Created] (SPARK-34281) Promote spark.kubernetes.executor.podNamePrefix to the official conf

2021-01-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-34281: - Summary: Promote spark.kubernetes.executor.podNamePrefix to the official conf Key: SPARK-34281 URL: https://issues.apache.org/jira/browse/SPARK-34281 Project: Spark

[jira] [Created] (SPARK-34280) Avoid migrating un-needed shuffle files

2021-01-28 Thread Holden Karau (Jira)
Holden Karau created SPARK-34280: Summary: Avoid migrating un-needed shuffle files Key: SPARK-34280 URL: https://issues.apache.org/jira/browse/SPARK-34280 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-34193) Potential race condition during decommissioning with TorrentBroadcast

2021-01-28 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-34193: - Fix Version/s: 3.1.1 > Potential race condition during decommissioning with TorrentBroadcast > -

[jira] [Resolved] (SPARK-32866) Docker buildx now requires --push

2021-01-28 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-32866. -- Fix Version/s: 3.1.1 3.2.0 Assignee: Holden Karau Resolution

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

2021-01-28 Thread Patrick Grandjean (Jira)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273955#comment-17273955 ] Patrick Grandjean commented on SPARK-7768: -- Fine with me :) (y) > Make user-def

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

2021-01-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273935#comment-17273935 ] Sean R. Owen commented on SPARK-7768: - I'm not aware of an argument against opening t

[jira] [Commented] (SPARK-33763) Add metrics for better tracking of dynamic allocation

2021-01-28 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273923#comment-17273923 ] Attila Zsolt Piros commented on SPARK-33763: I started working on this. >

[jira] [Assigned] (SPARK-34278) Make BlockManagerMaster driver heartbeat timeout configurable

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34278: - Assignee: Dongjoon Hyun > Make BlockManagerMaster driver heartbeat timeout configurable

[jira] [Resolved] (SPARK-34278) Make BlockManagerMaster driver heartbeat timeout configurable

2021-01-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-34278. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31383 [https://

[jira] [Resolved] (SPARK-34253) Object hash aggregate should not fallback if no more input rows

2021-01-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34253. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 31353 [https://gith

[jira] [Assigned] (SPARK-34253) Object hash aggregate should not fallback if no more input rows

2021-01-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34253: --- Assignee: Cheng Su > Object hash aggregate should not fallback if no more input rows >

[jira] [Assigned] (SPARK-34165) Add countDistinct option to Dataset#summary

2021-01-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-34165: Assignee: Matthew Powers > Add countDistinct option to Dataset#summary >

[jira] [Resolved] (SPARK-34165) Add countDistinct option to Dataset#summary

2021-01-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-34165. -- Fix Version/s: 3.2.0 Resolution: Fixed Resolved by https://github.com/apache/spark/pull

[jira] [Commented] (SPARK-34126) SQL running error, spark does not exit, resulting in data quality problems

2021-01-28 Thread Attila Zsolt Piros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273761#comment-17273761 ] Attila Zsolt Piros commented on SPARK-34126: [~shikui] are you using the " -

[jira] [Assigned] (SPARK-34279) Upgrade Junit from 4.x to 5.x

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34279: Assignee: (was: Apache Spark) > Upgrade Junit from 4.x to 5.x > -

[jira] [Commented] (SPARK-34279) Upgrade Junit from 4.x to 5.x

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273646#comment-17273646 ] Apache Spark commented on SPARK-34279: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-34279) Upgrade Junit from 4.x to 5.x

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273645#comment-17273645 ] Apache Spark commented on SPARK-34279: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-34279) Upgrade Junit from 4.x to 5.x

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34279: Assignee: Apache Spark > Upgrade Junit from 4.x to 5.x > - >

[jira] [Created] (SPARK-34279) Upgrade Junit from 4.x to 5.x

2021-01-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-34279: Summary: Upgrade Junit from 4.x to 5.x Key: SPARK-34279 URL: https://issues.apache.org/jira/browse/SPARK-34279 Project: Spark Issue Type: Improvement Compo

[jira] [Assigned] (SPARK-31816) Create high level description about JDBC connection providers for users/developers

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31816: Assignee: Apache Spark > Create high level description about JDBC connection providers fo

[jira] [Assigned] (SPARK-31816) Create high level description about JDBC connection providers for users/developers

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31816: Assignee: (was: Apache Spark) > Create high level description about JDBC connection p

[jira] [Assigned] (SPARK-31816) Create high level description about JDBC connection providers for users/developers

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31816: Assignee: (was: Apache Spark) > Create high level description about JDBC connection p

[jira] [Commented] (SPARK-31816) Create high level description about JDBC connection providers for users/developers

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273556#comment-17273556 ] Apache Spark commented on SPARK-31816: -- User 'gaborgsomogyi' has created a pull req

[jira] [Resolved] (SPARK-34275) Replaces filter and size with count

2021-01-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34275. -- Fix Version/s: 3.1.2 2.4.8 3.2.0 3.0.2

[jira] [Assigned] (SPARK-34275) Replaces filter and size with count

2021-01-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-34275: Assignee: Yang Jie > Replaces filter and size with count > --

[jira] [Resolved] (SPARK-34262) ALTER TABLE .. SET LOCATION doesn't refresh v1 table cache

2021-01-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-34262. -- Fix Version/s: (was: 3.1.1) 3.1.2 Resolution: Fixed Issue resolv

[jira] [Assigned] (SPARK-34278) Make BlockManagerMaster driver heartbeat timeout configurable

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34278: Assignee: (was: Apache Spark) > Make BlockManagerMaster driver heartbeat timeout conf

[jira] [Assigned] (SPARK-34278) Make BlockManagerMaster driver heartbeat timeout configurable

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34278: Assignee: Apache Spark > Make BlockManagerMaster driver heartbeat timeout configurable >

[jira] [Commented] (SPARK-34278) Make BlockManagerMaster driver heartbeat timeout configurable

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273443#comment-17273443 ] Apache Spark commented on SPARK-34278: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-34278) Make BlockManagerMaster driver heartbeat timeout configurable

2021-01-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-34278: - Summary: Make BlockManagerMaster driver heartbeat timeout configurable Key: SPARK-34278 URL: https://issues.apache.org/jira/browse/SPARK-34278 Project: Spark

[jira] [Commented] (SPARK-34144) java.time.Instant and java.time.LocalDate not handled when writing to tables

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273435#comment-17273435 ] Apache Spark commented on SPARK-34144: -- User 'cristichircu' has created a pull requ

[jira] [Commented] (SPARK-34144) java.time.Instant and java.time.LocalDate not handled when writing to tables

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273434#comment-17273434 ] Apache Spark commented on SPARK-34144: -- User 'cristichircu' has created a pull requ

[jira] [Commented] (SPARK-33867) java.time.Instant and java.time.LocalDate not handled in org.apache.spark.sql.jdbc.JdbcDialect#compileValue

2021-01-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273420#comment-17273420 ] Apache Spark commented on SPARK-33867: -- User 'cristichircu' has created a pull requ