[jira] [Commented] (SPARK-35143) Add default log config for spark-sql

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325527#comment-17325527 ] Apache Spark commented on SPARK-35143: -- User 'hddong' has created a pull request fo

[jira] [Commented] (SPARK-35143) Add default log config for spark-sql

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325525#comment-17325525 ] Apache Spark commented on SPARK-35143: -- User 'hddong' has created a pull request fo

[jira] [Commented] (SPARK-35143) Add default log config for spark-sql

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325524#comment-17325524 ] Apache Spark commented on SPARK-35143: -- User 'ChenDou2021' has created a pull reque

[jira] [Assigned] (SPARK-35143) Add default log config for spark-sql

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35143: Assignee: Apache Spark > Add default log config for spark-sql > -

[jira] [Commented] (SPARK-35143) Add default log config for spark-sql

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325523#comment-17325523 ] Apache Spark commented on SPARK-35143: -- User 'ChenDou2021' has created a pull reque

[jira] [Assigned] (SPARK-35143) Add default log config for spark-sql

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35143: Assignee: (was: Apache Spark) > Add default log config for spark-sql > --

[jira] [Assigned] (SPARK-35075) Migrate to transformWithPruning or resolveWithPruning for subquery related rules

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35075: Assignee: (was: Apache Spark) > Migrate to transformWithPruning or resolveWithPruning

[jira] [Commented] (SPARK-35075) Migrate to transformWithPruning or resolveWithPruning for subquery related rules

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325499#comment-17325499 ] Apache Spark commented on SPARK-35075: -- User 'sigmod' has created a pull request fo

[jira] [Assigned] (SPARK-35075) Migrate to transformWithPruning or resolveWithPruning for subquery related rules

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35075: Assignee: Apache Spark > Migrate to transformWithPruning or resolveWithPruning for subque

[jira] [Created] (SPARK-35143) Add default log config for spark-sql

2021-04-19 Thread hong dongdong (Jira)
hong dongdong created SPARK-35143: - Summary: Add default log config for spark-sql Key: SPARK-35143 URL: https://issues.apache.org/jira/browse/SPARK-35143 Project: Spark Issue Type: Improvemen

[jira] [Resolved] (SPARK-35018) Test transferring year-month interval via Hive Thrift server

2021-04-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35018. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32240 [https://github.com

[jira] [Assigned] (SPARK-35018) Test transferring year-month interval via Hive Thrift server

2021-04-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35018: Assignee: Max Gekk > Test transferring year-month interval via Hive Thrift server > -

[jira] [Assigned] (SPARK-35134) setup.py will copy different version netty jars into deps/jars when execute PySpark pip packaging tests

2021-04-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35134: Assignee: Yang Jie > setup.py will copy different version netty jars into deps/jars when

[jira] [Resolved] (SPARK-35134) setup.py will copy different version netty jars into deps/jars when execute PySpark pip packaging tests

2021-04-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35134. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32230 [https://gi

[jira] [Assigned] (SPARK-35052) Use static treePatternBitSet for Leaf expressions like AttributeReference and Literal

2021-04-19 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35052: -- Assignee: Yingyi Bu > Use static treePatternBitSet for Leaf expressions like Attribut

[jira] [Resolved] (SPARK-35052) Use static treePatternBitSet for Leaf expressions like AttributeReference and Literal

2021-04-19 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35052. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32157 [https:

[jira] [Updated] (SPARK-35080) Correlated subqueries with equality predicates can return wrong results

2021-04-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-35080: Fix Version/s: 3.0.3 2.4.8 > Correlated subqueries with equality predicates can

[jira] [Resolved] (SPARK-35015) Add type hints of pyspark.sql.column.* and pyspark.sql.types.*

2021-04-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35015. -- Resolution: Won't Fix > Add type hints of pyspark.sql.column.* and pyspark.sql.types.* > -

[jira] [Assigned] (SPARK-35080) Correlated subqueries with equality predicates can return wrong results

2021-04-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35080: --- Assignee: Allison Wang > Correlated subqueries with equality predicates can return wrong re

[jira] [Resolved] (SPARK-35080) Correlated subqueries with equality predicates can return wrong results

2021-04-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35080. - Fix Version/s: 3.1.2 3.2.0 Resolution: Fixed Issue resolved by pull re

[jira] [Resolved] (SPARK-35117) UI progress bar no longer highlights in progress tasks

2021-04-19 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35117. Fix Version/s: 3.2.0 3.1.2 Assignee: Adam Binford Resolu

[jira] [Commented] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325428#comment-17325428 ] Apache Spark commented on SPARK-35142: -- User 'harupy' has created a pull request fo

[jira] [Commented] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325427#comment-17325427 ] Apache Spark commented on SPARK-35142: -- User 'harupy' has created a pull request fo

[jira] [Assigned] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35142: Assignee: Apache Spark > `OneVsRest` classifier uses incorrect data type for `rawPredicti

[jira] [Assigned] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35142: Assignee: (was: Apache Spark) > `OneVsRest` classifier uses incorrect data type for `

[jira] [Assigned] (SPARK-35060) Group exception messages in sql/types

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35060: Assignee: Apache Spark > Group exception messages in sql/types >

[jira] [Commented] (SPARK-35060) Group exception messages in sql/types

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325417#comment-17325417 ] Apache Spark commented on SPARK-35060: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-35060) Group exception messages in sql/types

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35060: Assignee: (was: Apache Spark) > Group exception messages in sql/types > -

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: `OneVsRest` classifier uses an incorrect data type for the `rawPrediction

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: Code to reproduce the issue// code placeholder     {code:java} from pys

[jira] [Updated] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harutaka Kawamura updated SPARK-35142: -- Description: Code to reproduce the issue: from pyspark.ml.classification import Logist

[jira] [Created] (SPARK-35142) `OneVsRest` classifier uses incorrect data type for `rawPrediction` column

2021-04-19 Thread Harutaka Kawamura (Jira)
Harutaka Kawamura created SPARK-35142: - Summary: `OneVsRest` classifier uses incorrect data type for `rawPrediction` column Key: SPARK-35142 URL: https://issues.apache.org/jira/browse/SPARK-35142

[jira] [Commented] (SPARK-35141) Support two level map for final hash aggregation

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325391#comment-17325391 ] Apache Spark commented on SPARK-35141: -- User 'c21' has created a pull request for t

[jira] [Assigned] (SPARK-35141) Support two level map for final hash aggregation

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35141: Assignee: Apache Spark > Support two level map for final hash aggregation > -

[jira] [Assigned] (SPARK-35141) Support two level map for final hash aggregation

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35141: Assignee: (was: Apache Spark) > Support two level map for final hash aggregation > --

[jira] [Created] (SPARK-35141) Support two level map for final hash aggregation

2021-04-19 Thread Cheng Su (Jira)
Cheng Su created SPARK-35141: Summary: Support two level map for final hash aggregation Key: SPARK-35141 URL: https://issues.apache.org/jira/browse/SPARK-35141 Project: Spark Issue Type: Improvem

[jira] [Commented] (SPARK-35140) Establish error message guidelines

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325319#comment-17325319 ] Apache Spark commented on SPARK-35140: -- User 'karenfeng' has created a pull request

[jira] [Assigned] (SPARK-35140) Establish error message guidelines

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35140: Assignee: (was: Apache Spark) > Establish error message guidelines >

[jira] [Assigned] (SPARK-35140) Establish error message guidelines

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35140: Assignee: Apache Spark > Establish error message guidelines > ---

[jira] [Created] (SPARK-35140) Establish error message guidelines

2021-04-19 Thread Karen Feng (Jira)
Karen Feng created SPARK-35140: -- Summary: Establish error message guidelines Key: SPARK-35140 URL: https://issues.apache.org/jira/browse/SPARK-35140 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-35103) Improve type coercion rule performance

2021-04-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-35103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-35103. --- Fix Version/s: 3.2.0 Resolution: Fixed > Improve type coercion rule performan

[jira] [Assigned] (SPARK-35103) Improve type coercion rule performance

2021-04-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-35103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-35103: - Assignee: Yingyi Bu (was: Herman van Hövell) > Improve type coercion rule perf

[jira] [Assigned] (SPARK-35103) Improve type coercion rule performance

2021-04-19 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-35103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-35103: - Assignee: Herman van Hövell > Improve type coercion rule performance >

[jira] [Updated] (SPARK-35139) Support ANSI intervals as Arrow Column vectors

2021-04-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35139: - Summary: Support ANSI intervals as Arrow Column vectors (was: Support ANSI intervals as Arrow Column ve

[jira] [Created] (SPARK-35139) Support ANSI intervals as Arrow Column verctors

2021-04-19 Thread Max Gekk (Jira)
Max Gekk created SPARK-35139: Summary: Support ANSI intervals as Arrow Column verctors Key: SPARK-35139 URL: https://issues.apache.org/jira/browse/SPARK-35139 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-35131) Support early driver service clean-up during app termination

2021-04-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35131. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32226 [https://

[jira] [Assigned] (SPARK-35131) Support early driver service clean-up during app termination

2021-04-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35131: - Assignee: Dongjoon Hyun > Support early driver service clean-up during app termination

[jira] [Commented] (SPARK-35068) Add tests for ANSI intervals to HiveThriftBinaryServerSuite

2021-04-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325260#comment-17325260 ] Max Gekk commented on SPARK-35068: -- After the PR https://github.com/apache/spark/pull/3

[jira] [Commented] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

2021-04-19 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325252#comment-17325252 ] Thomas Graves commented on SPARK-35108: --- seems like a correctness issue in some ca

[jira] [Updated] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

2021-04-19 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-35108: -- Labels: correctness (was: ) > Pickle produces incorrect key labels for GenericRowWithSchema (

[jira] [Updated] (SPARK-35108) Pickle produces incorrect key labels for GenericRowWithSchema (data corruption)

2021-04-19 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-35108: -- Priority: Blocker (was: Major) > Pickle produces incorrect key labels for GenericRowWithSchem

[jira] [Assigned] (SPARK-35018) Test transferring year-month interval via Hive Thrift server

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35018: Assignee: (was: Apache Spark) > Test transferring year-month interval via Hive Thrift

[jira] [Assigned] (SPARK-35018) Test transferring year-month interval via Hive Thrift server

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35018: Assignee: Apache Spark > Test transferring year-month interval via Hive Thrift server > -

[jira] [Commented] (SPARK-35018) Test transferring year-month interval via Hive Thrift server

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325240#comment-17325240 ] Apache Spark commented on SPARK-35018: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-35124) Local mode fails to start cluster due to configuration value escape issue

2021-04-19 Thread Malthe Borch (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325228#comment-17325228 ] Malthe Borch commented on SPARK-35124: -- The issue seems to stem from the way Python

[jira] [Resolved] (SPARK-35120) Guide users to sync branch and enable GitHub Actions in their forked repository

2021-04-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35120. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32235 [https://

[jira] [Assigned] (SPARK-35120) Guide users to sync branch and enable GitHub Actions in their forked repository

2021-04-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35120: - Assignee: Hyukjin Kwon > Guide users to sync branch and enable GitHub Actions in their

[jira] [Resolved] (SPARK-35138) Remove ANTLR 4.7 workaround

2021-04-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35138. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32238 [https://

[jira] [Assigned] (SPARK-35138) Remove ANTLR 4.7 workaround

2021-04-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35138: - Assignee: Cheng Pan > Remove ANTLR 4.7 workaround > --- > >

[jira] [Commented] (SPARK-35138) Remove ANTLR 4.7 workaround

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325221#comment-17325221 ] Apache Spark commented on SPARK-35138: -- User 'pan3793' has created a pull request f

[jira] [Assigned] (SPARK-35138) Remove ANTLR 4.7 workaround

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35138: Assignee: (was: Apache Spark) > Remove ANTLR 4.7 workaround > ---

[jira] [Assigned] (SPARK-35138) Remove ANTLR 4.7 workaround

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35138: Assignee: Apache Spark > Remove ANTLR 4.7 workaround > --- > >

[jira] [Commented] (SPARK-35138) Remove ANTLR 4.7 workaround

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325220#comment-17325220 ] Apache Spark commented on SPARK-35138: -- User 'pan3793' has created a pull request f

[jira] [Created] (SPARK-35138) Remove ANTLR 4.7 workaround

2021-04-19 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35138: - Summary: Remove ANTLR 4.7 workaround Key: SPARK-35138 URL: https://issues.apache.org/jira/browse/SPARK-35138 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-35129) Construct year-month interval column from integral fields

2021-04-19 Thread Matthew Powers (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325219#comment-17325219 ] Matthew Powers commented on SPARK-35129: [~maxgekk] - yep, thanks for pinging me

[jira] [Updated] (SPARK-34999) Consolidate PySpark testing utils

2021-04-19 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-34999: - Description: `python/pyspark/pandas/testing` hold test utilites for pandas-on-spark, and `python

[jira] [Updated] (SPARK-34999) Consolidate PySpark testing utils

2021-04-19 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-34999: - Description: `python/pyspark/pandas/testing` and `python/pyspark/testing` share the same goal. M

[jira] [Commented] (SPARK-34639) always remove unnecessary Alias in Analyzer.resolveExpression

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325179#comment-17325179 ] Apache Spark commented on SPARK-34639: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-34639) always remove unnecessary Alias in Analyzer.resolveExpression

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325178#comment-17325178 ] Apache Spark commented on SPARK-34639: -- User 'cloud-fan' has created a pull request

[jira] [Resolved] (SPARK-35126) Execute jdbc cancellation method when jdbc load job is interrupted

2021-04-19 Thread zhangrenhua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangrenhua resolved SPARK-35126. - Resolution: Not A Problem I found that by setting FetchSize, can respond quickly and return the

[jira] [Commented] (SPARK-35126) Execute jdbc cancellation method when jdbc load job is interrupted

2021-04-19 Thread zhangrenhua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325172#comment-17325172 ] zhangrenhua commented on SPARK-35126: - I found that by setting FetchSize, can respon

[jira] [Commented] (SPARK-34198) Add RocksDB StateStore as external module

2021-04-19 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325162#comment-17325162 ] L. C. Hsieh commented on SPARK-34198: - Thanks [~XuanYuan] and Databricks for the dec

[jira] [Updated] (SPARK-35136) Initial null value of LiveStage.info can lead to NPE

2021-04-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-35136: Fix Version/s: 3.1.2 > Initial null value of LiveStage.info can lead to NPE >

[jira] [Resolved] (SPARK-35136) Initial null value of LiveStage.info can lead to NPE

2021-04-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35136. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32233 [https://gith

[jira] [Assigned] (SPARK-35136) Initial null value of LiveStage.info can lead to NPE

2021-04-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35136: --- Assignee: Sander Goos > Initial null value of LiveStage.info can lead to NPE >

[jira] [Assigned] (SPARK-35102) Make spark.sql.hive.version meaningful and not deprecated

2021-04-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35102: --- Assignee: Kent Yao > Make spark.sql.hive.version meaningful and not deprecated > --

[jira] [Resolved] (SPARK-35102) Make spark.sql.hive.version meaningful and not deprecated

2021-04-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35102. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32200 [https://gith

[jira] [Resolved] (SPARK-35125) Upgrade K8s client to 5.3.0 to support K8s 1.20

2021-04-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35125. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32221 [https://

[jira] [Assigned] (SPARK-35125) Upgrade K8s client to 5.3.0 to support K8s 1.20

2021-04-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35125: - Assignee: Dongjoon Hyun > Upgrade K8s client to 5.3.0 to support K8s 1.20 > ---

[jira] [Commented] (SPARK-34198) Add RocksDB StateStore as external module

2021-04-19 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325031#comment-17325031 ] Yuanjian Li commented on SPARK-34198: - Thanks all for joining the discussion! Datab

[jira] [Resolved] (SPARK-34715) Add round trip tests for period <-> month and duration <-> micros

2021-04-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-34715. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32234 [https://github.com

[jira] [Assigned] (SPARK-34715) Add round trip tests for period <-> month and duration <-> micros

2021-04-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-34715: Assignee: jiaan.geng > Add round trip tests for period <-> month and duration <-> micros > --

[jira] [Resolved] (SPARK-35107) Parse unit-to-unit interval literals to ANSI intervals

2021-04-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35107. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32209 [https://github.com

[jira] [Resolved] (SPARK-34837) Support ANSI SQL intervals by the aggregate function `avg`

2021-04-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-34837. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32229 [https://github.com

[jira] [Assigned] (SPARK-34837) Support ANSI SQL intervals by the aggregate function `avg`

2021-04-19 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-34837: Assignee: jiaan.geng > Support ANSI SQL intervals by the aggregate function `avg` > -

[jira] [Assigned] (SPARK-35137) Revise outputpartitioning number in some SparkPlan

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35137: Assignee: Apache Spark > Revise outputpartitioning number in some SparkPlan > ---

[jira] [Assigned] (SPARK-35120) Guide users to sync branch and enable GitHub Actions in their forked repository

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35120: Assignee: (was: Apache Spark) > Guide users to sync branch and enable GitHub Actions

[jira] [Assigned] (SPARK-35137) Revise outputpartitioning number in some SparkPlan

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35137: Assignee: (was: Apache Spark) > Revise outputpartitioning number in some SparkPlan >

[jira] [Assigned] (SPARK-35120) Guide users to sync branch and enable GitHub Actions in their forked repository

2021-04-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35120: Assignee: Apache Spark > Guide users to sync branch and enable GitHub Actions in their fo

  1   2   >