[jira] [Commented] (SPARK-33133) History server fails when loading invalid rolling event logs

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213629#comment-17213629 ] Hyukjin Kwon commented on SPARK-33133: -- cc [~kabhwan] FYI > History server fails when loading

[jira] [Commented] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213612#comment-17213612 ] Jungtaek Lim commented on SPARK-33136: -- Note that AppendData in branch-2.4 is also broken as same,

[jira] [Comment Edited] (SPARK-29536) PySpark does not work with Python 3.8.0

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213558#comment-17213558 ] Dongjoon Hyun edited comment on SPARK-29536 at 10/14/20, 3:18 AM: -- Hi,

[jira] [Updated] (SPARK-29536) PySpark does not work with Python 3.8.0

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29536: -- Affects Version/s: 2.4.7 > PySpark does not work with Python 3.8.0 >

[jira] [Commented] (SPARK-29536) PySpark does not work with Python 3.8.0

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213558#comment-17213558 ] Dongjoon Hyun commented on SPARK-29536: --- Hi, [~hyukjin.kwon]. Apache Spark 2.4.7 also fails. I

[jira] [Resolved] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33134. -- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33134: Assignee: Maxim Gekk > Incorrect nested complex JSON fields raise an exception >

[jira] [Commented] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Thejdeep (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213550#comment-17213550 ] Thejdeep commented on SPARK-33138: -- Can I look into this please ? > unify temp view and permanent view

[jira] [Created] (SPARK-33142) SQL temp view should store SQL text as well

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33142: --- Summary: SQL temp view should store SQL text as well Key: SPARK-33142 URL: https://issues.apache.org/jira/browse/SPARK-33142 Project: Spark Issue Type:

[jira] [Created] (SPARK-33141) capture SQL configs when creating permanent views

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33141: --- Summary: capture SQL configs when creating permanent views Key: SPARK-33141 URL: https://issues.apache.org/jira/browse/SPARK-33141 Project: Spark Issue Type:

[jira] [Created] (SPARK-33140) make Analyzer and Optimizer rules using SQLConf.get

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33140: --- Summary: make Analyzer and Optimizer rules using SQLConf.get Key: SPARK-33140 URL: https://issues.apache.org/jira/browse/SPARK-33140 Project: Spark Issue

[jira] [Created] (SPARK-33139) protect setActiveSession and clearActiveSession

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33139: --- Summary: protect setActiveSession and clearActiveSession Key: SPARK-33139 URL: https://issues.apache.org/jira/browse/SPARK-33139 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33138: Description: Currently, temp view store mapping of temp view name and its logicalPlan, and

[jira] [Created] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33138: --- Summary: unify temp view and permanent view behaviors Key: SPARK-33138 URL: https://issues.apache.org/jira/browse/SPARK-33138 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33138: Description: Currently, temp view store mapping of temp view name and its logicalPlan, and

[jira] [Created] (SPARK-33137) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (PostgreSQL dialect)

2020-10-13 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-33137: -- Summary: Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (PostgreSQL dialect) Key: SPARK-33137 URL:

[jira] [Commented] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213461#comment-17213461 ] Apache Spark commented on SPARK-33136: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33136: Assignee: Apache Spark > Handling nullability for complex types is broken during

[jira] [Assigned] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33136: Assignee: (was: Apache Spark) > Handling nullability for complex types is broken

[jira] [Commented] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213448#comment-17213448 ] Jungtaek Lim commented on SPARK-33136: -- will submit a PR soon. > Handling nullability for complex

[jira] [Created] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-33136: Summary: Handling nullability for complex types is broken during resolution of V2 write command Key: SPARK-33136 URL: https://issues.apache.org/jira/browse/SPARK-33136

[jira] [Commented] (SPARK-29625) Spark Structure Streaming Kafka Wrong Reset Offset twice

2020-10-13 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213336#comment-17213336 ] L. C. Hsieh commented on SPARK-29625: - How did you specify the starting offset? > Spark Structure

[jira] [Updated] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33134: --- Description: The code below: {code:scala} val pokerhand_raw = Seq("""[{"cards": [19], "playerId":

[jira] [Commented] (SPARK-33071) Join with ambiguous column succeeding but giving wrong output

2020-10-13 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213312#comment-17213312 ] Aoyuan Liao commented on SPARK-33071: - The master branch has the same bug. > Join with ambiguous

[jira] [Commented] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213224#comment-17213224 ] Apache Spark commented on SPARK-33134: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33132: - Assignee: akiyamaneko > The 'Shuffle Read Size / Records' field in Stage Summary

[jira] [Resolved] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33132. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30030

[jira] [Commented] (SPARK-33071) Join with ambiguous column succeeding but giving wrong output

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213205#comment-17213205 ] Wenchen Fan commented on SPARK-33071: - To confirm: is this a long-standing bug that 2.4, 3.0, and

[jira] [Assigned] (SPARK-33135) Use listLocatedStatus from FileSystem implementations

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33135: Assignee: Apache Spark > Use listLocatedStatus from FileSystem implementations >

[jira] [Resolved] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33129. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30028

[jira] [Assigned] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33129: - Assignee: Prashant Sharma > Since the sbt version is now upgraded, old `test-only`

[jira] [Assigned] (SPARK-33135) Use listLocatedStatus from FileSystem implementations

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33135: Assignee: (was: Apache Spark) > Use listLocatedStatus from FileSystem

[jira] [Commented] (SPARK-33135) Use listLocatedStatus from FileSystem implementations

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213203#comment-17213203 ] Apache Spark commented on SPARK-33135: -- User 'sunchao' has created a pull request for this issue:

[jira] [Created] (SPARK-33135) Use listLocatedStatus from FileSystem implementations

2020-10-13 Thread Chao Sun (Jira)
Chao Sun created SPARK-33135: Summary: Use listLocatedStatus from FileSystem implementations Key: SPARK-33135 URL: https://issues.apache.org/jira/browse/SPARK-33135 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33134: --- Description: The code below: {code:scala} val pokerhand_raw = Seq("""[{"cards": [11],

[jira] [Commented] (SPARK-25547) Pluggable jdbc connection factory

2020-10-13 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213152#comment-17213152 ] Gabor Somogyi commented on SPARK-25547: --- [~fsauer65] JDBC connection provider API is added here:

[jira] [Commented] (SPARK-33128) mismatched input since Spark 3.0

2020-10-13 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213137#comment-17213137 ] Yang Jie commented on SPARK-33128: -- set `spark.sql.ansi.enabled` to true can work with Spark 3.0,I'm

[jira] [Assigned] (SPARK-13860) TPCDS query 39 returns wrong results compared to TPC official result set

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-13860: --- Assignee: Leanken.Lin > TPCDS query 39 returns wrong results compared to TPC official

[jira] [Reopened] (SPARK-13860) TPCDS query 39 returns wrong results compared to TPC official result set

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-13860: - > TPCDS query 39 returns wrong results compared to TPC official result set >

[jira] [Resolved] (SPARK-13860) TPCDS query 39 returns wrong results compared to TPC official result set

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-13860. - Fix Version/s: 3.1.0 Resolution: Fixed > TPCDS query 39 returns wrong results compared

[jira] [Assigned] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33134: Assignee: (was: Apache Spark) > Incorrect nested complex JSON fields raise an

[jira] [Assigned] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33134: Assignee: Apache Spark > Incorrect nested complex JSON fields raise an exception >

[jira] [Commented] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213109#comment-17213109 ] Apache Spark commented on SPARK-33134: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213108#comment-17213108 ] Apache Spark commented on SPARK-33134: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Updated] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33132: Fix Version/s: (was: 3.0.2) > The 'Shuffle Read Size / Records' field in Stage Summary

[jira] [Assigned] (SPARK-33125) Improve the error when Lead and Lag are not allowed to specify window frame

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33125: --- Assignee: jiaan.geng > Improve the error when Lead and Lag are not allowed to specify

[jira] [Resolved] (SPARK-33125) Improve the error when Lead and Lag are not allowed to specify window frame

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33125. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30021

[jira] [Created] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33134: -- Summary: Incorrect nested complex JSON fields raise an exception Key: SPARK-33134 URL: https://issues.apache.org/jira/browse/SPARK-33134 Project: Spark Issue

[jira] [Resolved] (SPARK-33081) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect)

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33081. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29972

[jira] [Assigned] (SPARK-33081) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect)

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33081: --- Assignee: Huaxin Gao > Support ALTER TABLE in JDBC v2 Table Catalog: update type and

[jira] [Updated] (SPARK-33133) History server fails when loading invalid rolling event logs

2020-10-13 Thread Adam Binford (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Binford updated SPARK-33133: - Description: We have run into an issue where our history server fails to load new

[jira] [Resolved] (SPARK-32858) UnwrapCastInBinaryComparison: support other numeric types

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32858. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29792

[jira] [Assigned] (SPARK-32858) UnwrapCastInBinaryComparison: support other numeric types

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32858: --- Assignee: Chao Sun > UnwrapCastInBinaryComparison: support other numeric types >

[jira] [Created] (SPARK-33133) History server fails when loading invalid rolling event logs

2020-10-13 Thread Adam Binford (Jira)
Adam Binford created SPARK-33133: Summary: History server fails when loading invalid rolling event logs Key: SPARK-33133 URL: https://issues.apache.org/jira/browse/SPARK-33133 Project: Spark

[jira] [Resolved] (SPARK-33115) `kvstore` and `unsafe` doc tasks fail

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33115. -- Fix Version/s: 3.1.0 3.0.2 Assignee: Denis Pyshev

[jira] [Assigned] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33132: Assignee: Apache Spark > The 'Shuffle Read Size / Records' field in Stage Summary

[jira] [Commented] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213073#comment-17213073 ] Apache Spark commented on SPARK-33132: -- User 'akiyamaneko' has created a pull request for this

[jira] [Assigned] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33132: Assignee: (was: Apache Spark) > The 'Shuffle Read Size / Records' field in Stage

[jira] [Updated] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread echohlne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] echohlne updated SPARK-33132: - Summary: The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

[jira] [Updated] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefind'

2020-10-13 Thread echohlne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] echohlne updated SPARK-33132: - Description: Spark Version: 3.0.1 Description: The 'Shuffle Read Size / Records' field in Stage

[jira] [Updated] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefind'

2020-10-13 Thread echohlne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] echohlne updated SPARK-33132: - Attachment: Stage Summary shows NaN undefined.png > The 'Shuffle Read Size / Records' field in Stage

[jira] [Created] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefind'

2020-10-13 Thread echohlne (Jira)
echohlne created SPARK-33132: Summary: The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefind' Key: SPARK-33132 URL: https://issues.apache.org/jira/browse/SPARK-33132

[jira] [Commented] (SPARK-33131) Fix grouping sets with having clause can not resolve qualified col name

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213068#comment-17213068 ] Apache Spark commented on SPARK-33131: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-33131) Fix grouping sets with having clause can not resolve qualified col name

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33131: Assignee: Apache Spark > Fix grouping sets with having clause can not resolve qualified

[jira] [Assigned] (SPARK-33131) Fix grouping sets with having clause can not resolve qualified col name

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33131: Assignee: (was: Apache Spark) > Fix grouping sets with having clause can not resolve

[jira] [Issue Comment Deleted] (SPARK-32681) PySpark type hints support

2020-10-13 Thread echohlne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] echohlne updated SPARK-32681: - Comment: was deleted (was: (y)) > PySpark type hints support > -- > >

[jira] [Created] (SPARK-33131) Fix grouping sets with having clause can not resolve qualified col name

2020-10-13 Thread ulysses you (Jira)
ulysses you created SPARK-33131: --- Summary: Fix grouping sets with having clause can not resolve qualified col name Key: SPARK-33131 URL: https://issues.apache.org/jira/browse/SPARK-33131 Project: Spark

[jira] [Commented] (SPARK-32681) PySpark type hints support

2020-10-13 Thread echohlne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213065#comment-17213065 ] echohlne commented on SPARK-32681: -- (y) > PySpark type hints support > -- > >

[jira] [Resolved] (SPARK-32295) Add not null and size > 0 filters before inner explode to benefit from predicate pushdown

2020-10-13 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32295. -- Fix Version/s: 3.1.0 Assignee: Tanel Kiis Resolution: Fixed Resolved

[jira] [Created] (SPARK-33130) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MsSqlServer dialect)

2020-10-13 Thread Prashant Sharma (Jira)
Prashant Sharma created SPARK-33130: --- Summary: Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MsSqlServer dialect) Key: SPARK-33130 URL:

[jira] [Commented] (SPARK-21708) Migrate build to sbt 1.3.13

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213033#comment-17213033 ] Apache Spark commented on SPARK-21708: -- User 'ScrapCodes' has created a pull request for this

[jira] [Commented] (SPARK-21708) Migrate build to sbt 1.3.13

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213032#comment-17213032 ] Apache Spark commented on SPARK-21708: -- User 'ScrapCodes' has created a pull request for this

[jira] [Commented] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213031#comment-17213031 ] Apache Spark commented on SPARK-33129: -- User 'ScrapCodes' has created a pull request for this

[jira] [Assigned] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33129: Assignee: (was: Apache Spark) > Since the sbt version is now upgraded, old

[jira] [Assigned] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33129: Assignee: Apache Spark > Since the sbt version is now upgraded, old `test-only` needs to

[jira] [Updated] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-33129: Description: Follow up to SPARK-21708, updating the references to test-only with

[jira] [Created] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Prashant Sharma (Jira)
Prashant Sharma created SPARK-33129: --- Summary: Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly` Key: SPARK-33129 URL: https://issues.apache.org/jira/browse/SPARK-33129

[jira] [Assigned] (SPARK-32069) Improve error message on reading unexpected directory which is not a table partition

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32069: Assignee: (was: Apache Spark) > Improve error message on reading unexpected

[jira] [Assigned] (SPARK-32069) Improve error message on reading unexpected directory which is not a table partition

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32069: Assignee: Apache Spark > Improve error message on reading unexpected directory which is

[jira] [Commented] (SPARK-32069) Improve error message on reading unexpected directory which is not a table partition

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213012#comment-17213012 ] Apache Spark commented on SPARK-32069: -- User 'AngersZh' has created a pull request for this

[jira] [Commented] (SPARK-27733) Upgrade to Avro 1.10.0

2020-10-13 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213010#comment-17213010 ] Ismaël Mejía commented on SPARK-27733: -- [~csun] This is excellent news! We will get this done and

[jira] [Commented] (SPARK-32978) Incorrect number of dynamic part metric

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213002#comment-17213002 ] Apache Spark commented on SPARK-32978: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-32978) Incorrect number of dynamic part metric

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32978: Assignee: (was: Apache Spark) > Incorrect number of dynamic part metric >

[jira] [Assigned] (SPARK-32978) Incorrect number of dynamic part metric

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32978: Assignee: Apache Spark > Incorrect number of dynamic part metric >

[jira] [Commented] (SPARK-32978) Incorrect number of dynamic part metric

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213001#comment-17213001 ] Apache Spark commented on SPARK-32978: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-33128) mismatched input since Spark 3.0

2020-10-13 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33128: --- Summary: mismatched input since Spark 3.0 Key: SPARK-33128 URL: https://issues.apache.org/jira/browse/SPARK-33128 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33095: Assignee: Apache Spark > Support ALTER TABLE in JDBC v2 Table Catalog: add, update type

[jira] [Assigned] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33095: Assignee: (was: Apache Spark) > Support ALTER TABLE in JDBC v2 Table Catalog: add,

[jira] [Commented] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212989#comment-17212989 ] Apache Spark commented on SPARK-33095: -- User 'ScrapCodes' has created a pull request for this

[jira] [Updated] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

2020-10-13 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-33095: Description: Override the default SQL strings for: ALTER TABLE UPDATE COLUMN TYPE ALTER

[jira] [Assigned] (SPARK-32229) Application entry parsing fails because DriverWrapper registered instead of the normal driver

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32229: Assignee: (was: Apache Spark) > Application entry parsing fails because

[jira] [Commented] (SPARK-32229) Application entry parsing fails because DriverWrapper registered instead of the normal driver

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212968#comment-17212968 ] Apache Spark commented on SPARK-32229: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Assigned] (SPARK-32229) Application entry parsing fails because DriverWrapper registered instead of the normal driver

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32229: Assignee: Apache Spark > Application entry parsing fails because DriverWrapper

[jira] [Created] (SPARK-33127) Change Executor filesystem.* metrics to type Counter

2020-10-13 Thread Wolfgang Buchner (Jira)
Wolfgang Buchner created SPARK-33127: Summary: Change Executor filesystem.* metrics to type Counter Key: SPARK-33127 URL: https://issues.apache.org/jira/browse/SPARK-33127 Project: Spark

[jira] [Assigned] (SPARK-33119) ScalarSubquery should returns the first two rows to avoid Driver OOM

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33119: Assignee: Yuming Wang > ScalarSubquery should returns the first two rows to avoid Driver

[jira] [Resolved] (SPARK-33119) ScalarSubquery should returns the first two rows to avoid Driver OOM

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33119. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30016

[jira] [Updated] (SPARK-33126) Simplify offset window function(Remove direction field)

2020-10-13 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-33126: --- Summary: Simplify offset window function(Remove direction field) (was: Improve offset window

[jira] [Commented] (SPARK-33126) Improve offset window function(Remove direction field)

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212935#comment-17212935 ] Apache Spark commented on SPARK-33126: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33126) Improve offset window function(Remove direction field)

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33126: Assignee: (was: Apache Spark) > Improve offset window function(Remove direction

[jira] [Assigned] (SPARK-33126) Improve offset window function(Remove direction field)

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33126: Assignee: Apache Spark > Improve offset window function(Remove direction field) >

  1   2   >