[jira] [Created] (SPARK-36595) Document window & session_window function in SQL API doc

2021-08-25 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-36595: Summary: Document window & session_window function in SQL API doc Key: SPARK-36595 URL: https://issues.apache.org/jira/browse/SPARK-36595 Project: Spark

[jira] [Commented] (SPARK-36457) Review and fix issues in API docs

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404931#comment-17404931 ] Apache Spark commented on SPARK-36457: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-36506) Improve test coverage for series.py and indexes/*.py.

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404924#comment-17404924 ] Apache Spark commented on SPARK-36506: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36506) Improve test coverage for series.py and indexes/*.py.

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36506: Assignee: (was: Apache Spark) > Improve test coverage for series.py and

[jira] [Commented] (SPARK-36506) Improve test coverage for series.py and indexes/*.py.

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404923#comment-17404923 ] Apache Spark commented on SPARK-36506: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36506) Improve test coverage for series.py and indexes/*.py.

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36506: Assignee: Apache Spark > Improve test coverage for series.py and indexes/*.py. >

[jira] [Commented] (SPARK-36594) ORC vectorized reader should properly check maximal number of fields

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404916#comment-17404916 ] Apache Spark commented on SPARK-36594: -- User 'c21' has created a pull request for this issue:

[jira] [Commented] (SPARK-36594) ORC vectorized reader should properly check maximal number of fields

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404914#comment-17404914 ] Apache Spark commented on SPARK-36594: -- User 'c21' has created a pull request for this issue:

[jira] [Commented] (SPARK-36594) ORC vectorized reader should properly check maximal number of fields

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404896#comment-17404896 ] Apache Spark commented on SPARK-36594: -- User 'c21' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36594) ORC vectorized reader should properly check maximal number of fields

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36594: Assignee: (was: Apache Spark) > ORC vectorized reader should properly check maximal

[jira] [Assigned] (SPARK-36594) ORC vectorized reader should properly check maximal number of fields

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36594: Assignee: Apache Spark > ORC vectorized reader should properly check maximal number of

[jira] [Commented] (SPARK-36594) ORC vectorized reader should properly check maximal number of fields

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404895#comment-17404895 ] Apache Spark commented on SPARK-36594: -- User 'c21' has created a pull request for this issue:

[jira] [Created] (SPARK-36594) ORC vectorized reader should properly check maximal number of fields

2021-08-25 Thread Cheng Su (Jira)
Cheng Su created SPARK-36594: Summary: ORC vectorized reader should properly check maximal number of fields Key: SPARK-36594 URL: https://issues.apache.org/jira/browse/SPARK-36594 Project: Spark

[jira] [Resolved] (SPARK-36488) "Invalid usage of '*' in expression" error due to the feature of 'quotedRegexColumnNames' in some scenarios.

2021-08-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36488. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33802

[jira] [Assigned] (SPARK-36488) "Invalid usage of '*' in expression" error due to the feature of 'quotedRegexColumnNames' in some scenarios.

2021-08-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36488: --- Assignee: Pablo Langa Blanco > "Invalid usage of '*' in expression" error due to the

[jira] [Assigned] (SPARK-36568) Missed broadcast join in V2 plan

2021-08-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36568: --- Assignee: Peter Toth > Missed broadcast join in V2 plan >

[jira] [Resolved] (SPARK-36568) Missed broadcast join in V2 plan

2021-08-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36568. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33825

[jira] [Commented] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-08-25 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404844#comment-17404844 ] 胡振宇 commented on SPARK-36584: - any one can verify this question  [~holden.ka...@gmail.com]  [~holdenkarau] ?

[jira] [Resolved] (SPARK-36590) Special timestamp_ntz value should be converted in the session tz

2021-08-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36590. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33838

[jira] [Created] (SPARK-36593) Support the Volcano Job API

2021-08-25 Thread Holden Karau (Jira)
Holden Karau created SPARK-36593: Summary: Support the Volcano Job API Key: SPARK-36593 URL: https://issues.apache.org/jira/browse/SPARK-36593 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36058) Support replicasets/job API

2021-08-25 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-36058: - Fix Version/s: 3.3.0 Affects Version/s: 3.3.0 > Support replicasets/job API >

[jira] [Commented] (SPARK-36058) Support replicasets/job API

2021-08-25 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404810#comment-17404810 ] Holden Karau commented on SPARK-36058: -- So I've added support for this, which makes the executor

[jira] [Resolved] (SPARK-36058) Support replicasets/job API

2021-08-25 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-36058. -- Assignee: Holden Karau Resolution: Fixed > Support replicasets/job API >

[jira] [Resolved] (SPARK-36592) Test bug

2021-08-25 Thread Tarun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tarun resolved SPARK-36592. --- Resolution: Not A Problem > Test bug > > > Key: SPARK-36592 > URL:

[jira] [Created] (SPARK-36592) Test bug

2021-08-25 Thread Tarun (Jira)
Tarun created SPARK-36592: - Summary: Test bug Key: SPARK-36592 URL: https://issues.apache.org/jira/browse/SPARK-36592 Project: Spark Issue Type: Bug Components: Java API Affects

[jira] [Closed] (SPARK-36591) test bug

2021-08-25 Thread Tarun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tarun closed SPARK-36591. - > test bug > > > Key: SPARK-36591 > URL:

[jira] [Resolved] (SPARK-36591) test bug

2021-08-25 Thread Tarun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tarun resolved SPARK-36591. --- Resolution: Later > test bug > > > Key: SPARK-36591 > URL:

[jira] [Created] (SPARK-36591) test bug

2021-08-25 Thread Tarun (Jira)
Tarun created SPARK-36591: - Summary: test bug Key: SPARK-36591 URL: https://issues.apache.org/jira/browse/SPARK-36591 Project: Spark Issue Type: Bug Components: docs Affects Versions:

[jira] [Commented] (SPARK-36567) Support foldable special datetime values in CAST

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404709#comment-17404709 ] Apache Spark commented on SPARK-36567: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-35581) Casting special strings to DATE/TIMESTAMP returns inconsistent results

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404707#comment-17404707 ] Apache Spark commented on SPARK-35581: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Updated] (SPARK-36577) Spark on K8s: Driver pod keeps running when executor allocator terminates with Fatal exception

2021-08-25 Thread Vikas Garg (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Garg updated SPARK-36577: --- Description: In Spark on Kubernetes, the class ExecutorPodsSnapshotsStoreImpl creates a thread

[jira] [Commented] (SPARK-36291) Refactor second set of 20 query execution errors to use error classes

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404604#comment-17404604 ] Apache Spark commented on SPARK-36291: -- User 'dgd-contributor' has created a pull request for this

[jira] [Assigned] (SPARK-36291) Refactor second set of 20 query execution errors to use error classes

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36291: Assignee: Apache Spark > Refactor second set of 20 query execution errors to use error

[jira] [Assigned] (SPARK-36291) Refactor second set of 20 query execution errors to use error classes

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36291: Assignee: (was: Apache Spark) > Refactor second set of 20 query execution errors to

[jira] [Commented] (SPARK-36291) Refactor second set of 20 query execution errors to use error classes

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404603#comment-17404603 ] Apache Spark commented on SPARK-36291: -- User 'dgd-contributor' has created a pull request for this

[jira] [Commented] (SPARK-36590) Special timestamp_ntz value should be converted in the session tz

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404600#comment-17404600 ] Apache Spark commented on SPARK-36590: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36590) Special timestamp_ntz value should be converted in the session tz

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36590: Assignee: Apache Spark (was: Max Gekk) > Special timestamp_ntz value should be

[jira] [Assigned] (SPARK-36590) Special timestamp_ntz value should be converted in the session tz

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36590: Assignee: Max Gekk (was: Apache Spark) > Special timestamp_ntz value should be

[jira] [Commented] (SPARK-36590) Special timestamp_ntz value should be converted in the session tz

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404596#comment-17404596 ] Apache Spark commented on SPARK-36590: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Created] (SPARK-36590) Special timestamp_ntz value should be converted in the session tz

2021-08-25 Thread Max Gekk (Jira)
Max Gekk created SPARK-36590: Summary: Special timestamp_ntz value should be converted in the session tz Key: SPARK-36590 URL: https://issues.apache.org/jira/browse/SPARK-36590 Project: Spark

[jira] [Commented] (SPARK-36227) Remove TimestampNTZ type support in Spark 3.2

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404574#comment-17404574 ] Apache Spark commented on SPARK-36227: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-36227) Remove TimestampNTZ type support in Spark 3.2

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404573#comment-17404573 ] Apache Spark commented on SPARK-36227: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-36588) Use v2 commands by default

2021-08-25 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404569#comment-17404569 ] Terry Kim commented on SPARK-36588: --- Yes. Thanks [~cloud_fan]. > Use v2 commands by default >

[jira] [Created] (SPARK-36589) Creating a value in json which represents decimal

2021-08-25 Thread Anuradha Bhan (Jira)
Anuradha Bhan created SPARK-36589: - Summary: Creating a value in json which represents decimal Key: SPARK-36589 URL: https://issues.apache.org/jira/browse/SPARK-36589 Project: Spark Issue

[jira] [Commented] (SPARK-36293) Refactor fourth set of 20 query execution errors to use error classes

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404522#comment-17404522 ] Apache Spark commented on SPARK-36293: -- User 'dgd-contributor' has created a pull request for this

[jira] [Assigned] (SPARK-36293) Refactor fourth set of 20 query execution errors to use error classes

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36293: Assignee: (was: Apache Spark) > Refactor fourth set of 20 query execution errors to

[jira] [Assigned] (SPARK-36293) Refactor fourth set of 20 query execution errors to use error classes

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36293: Assignee: Apache Spark > Refactor fourth set of 20 query execution errors to use error

[jira] [Commented] (SPARK-36565) "Unscaled value too large for precision" while reading simple parquet file readable with parquet-tools and pandas

2021-08-25 Thread Lorenzo Martini (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404519#comment-17404519 ] Lorenzo Martini commented on SPARK-36565: - The file was given to us by a customer who had it

[jira] [Resolved] (SPARK-36585) Support setting "since" version in FunctionRegistry

2021-08-25 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36585. Fix Version/s: 3.2.0 Resolution: Fixed The issue is resolved in

[jira] [Comment Edited] (SPARK-16087) Spark Hangs When Using Union With Persisted Hadoop RDD

2021-08-25 Thread Melitta Dragaschnig (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404510#comment-17404510 ] Melitta Dragaschnig edited comment on SPARK-16087 at 8/25/21, 2:26 PM:

[jira] [Comment Edited] (SPARK-16087) Spark Hangs When Using Union With Persisted Hadoop RDD

2021-08-25 Thread Melitta Dragaschnig (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404510#comment-17404510 ] Melitta Dragaschnig edited comment on SPARK-16087 at 8/25/21, 2:26 PM:

[jira] [Commented] (SPARK-16087) Spark Hangs When Using Union With Persisted Hadoop RDD

2021-08-25 Thread Melitta Dragaschnig (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404510#comment-17404510 ] Melitta Dragaschnig commented on SPARK-16087: - I ran into this problem as well, using spark

[jira] [Commented] (SPARK-36588) Use v2 commands by default

2021-08-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404499#comment-17404499 ] Wenchen Fan commented on SPARK-36588: - [~imback82] do you want to drive this project? > Use v2

[jira] [Created] (SPARK-36588) Use v2 commands by default

2021-08-25 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-36588: --- Summary: Use v2 commands by default Key: SPARK-36588 URL: https://issues.apache.org/jira/browse/SPARK-36588 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-36586) Migrate all ParsedStatement to the new v2 command framework

2021-08-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404488#comment-17404488 ] Wenchen Fan commented on SPARK-36586: - [~imback82] do you want do drive this? > Migrate all

[jira] [Comment Edited] (SPARK-36586) Migrate all ParsedStatement to the new v2 command framework

2021-08-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404488#comment-17404488 ] Wenchen Fan edited comment on SPARK-36586 at 8/25/21, 2:11 PM: ---

[jira] [Assigned] (SPARK-36587) Migrate CreateNamespaceStatement to v2 command framework

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36587: Assignee: (was: Apache Spark) > Migrate CreateNamespaceStatement to v2 command

[jira] [Commented] (SPARK-36587) Migrate CreateNamespaceStatement to v2 command framework

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404480#comment-17404480 ] Apache Spark commented on SPARK-36587: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36587) Migrate CreateNamespaceStatement to v2 command framework

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36587: Assignee: Apache Spark > Migrate CreateNamespaceStatement to v2 command framework >

[jira] [Created] (SPARK-36587) Migrate CreateNamespaceStatement to v2 command framework

2021-08-25 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-36587: --- Summary: Migrate CreateNamespaceStatement to v2 command framework Key: SPARK-36587 URL: https://issues.apache.org/jira/browse/SPARK-36587 Project: Spark Issue

[jira] [Created] (SPARK-36586) Migrate all ParsedStatement to the new v2 command framework

2021-08-25 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-36586: --- Summary: Migrate all ParsedStatement to the new v2 command framework Key: SPARK-36586 URL: https://issues.apache.org/jira/browse/SPARK-36586 Project: Spark

[jira] [Updated] (SPARK-36581) Add back transformAllExpressions to AnalysisHelper

2021-08-25 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-36581: --- Priority: Minor (was: Major) > Add back transformAllExpressions to AnalysisHelper >

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-08-25 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description:  When driver broadcast object, it will send the

[jira] [Commented] (SPARK-36585) Support setting "since" version in FunctionRegistry

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404415#comment-17404415 ] Apache Spark commented on SPARK-36585: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-36585) Support setting "since" version in FunctionRegistry

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36585: Assignee: Gengliang Wang (was: Apache Spark) > Support setting "since" version in

[jira] [Assigned] (SPARK-36585) Support setting "since" version in FunctionRegistry

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36585: Assignee: Apache Spark (was: Gengliang Wang) > Support setting "since" version in

[jira] [Commented] (SPARK-36585) Support setting "since" version in FunctionRegistry

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404414#comment-17404414 ] Apache Spark commented on SPARK-36585: -- User 'gengliangwang' has created a pull request for this

[jira] [Updated] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36398: --- Fix Version/s: 3.1.3 3.2.0 > Redact sensitive information in Spark

[jira] [Resolved] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36398. Target Version/s: 3.2.0, 3.1.3 Resolution: Fixed Issue resolved in

[jira] [Updated] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36398: --- Target Version/s: (was: 3.2.0, 3.1.3) > Redact sensitive information in Spark Thrift

[jira] [Created] (SPARK-36585) Support setting "since" version in FunctionRegistry

2021-08-25 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-36585: -- Summary: Support setting "since" version in FunctionRegistry Key: SPARK-36585 URL: https://issues.apache.org/jira/browse/SPARK-36585 Project: Spark

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-08-25 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description: When driver broadcast object, it will send the

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-08-25 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description: When driver broadcast object, it will send the [SparkListenerBlockUpdated|#L228] event.

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-08-25 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description: When driver broadcast object, it will send the [SparkListenerBlockUpdated|#L228] event.

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-08-25 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description: When driver broadcast object, it will send the [SparkListenerBlockUpdated|#L228]] event.

[jira] [Updated] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-08-25 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-36584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 胡振宇 updated SPARK-36584: Description: When driver broadcast object, it will send the

[jira] [Created] (SPARK-36584) ExecutorMonitor#onBlockUpdated will receive event from driver

2021-08-25 Thread Jira
胡振宇 created SPARK-36584: --- Summary: ExecutorMonitor#onBlockUpdated will receive event from driver Key: SPARK-36584 URL: https://issues.apache.org/jira/browse/SPARK-36584 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36582) Spark HistoryPage may show 'NotFound' in some multiple attempts cases

2021-08-25 Thread Sungpeo Kook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sungpeo Kook updated SPARK-36582: - Description: Current historypage show a attemptId column in case of hasMultipleAttempts is

[jira] [Updated] (SPARK-36582) Spark HistoryPage may show 'NotFound' in some multiple attempts cases

2021-08-25 Thread Sungpeo Kook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sungpeo Kook updated SPARK-36582: - Attachment: Screen Shot 2021-08-25 at 2.35.30 PM.png > Spark HistoryPage may show 'NotFound' in

[jira] [Updated] (SPARK-36582) Spark HistoryPage may show 'NotFound' in some multiple attempts cases

2021-08-25 Thread Sungpeo Kook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sungpeo Kook updated SPARK-36582: - Attachment: Screen Shot 2021-08-25 at 2.35.30 PM.png > Spark HistoryPage may show 'NotFound' in

[jira] [Updated] (SPARK-36582) Spark HistoryPage may show 'NotFound' in some multiple attempts cases

2021-08-25 Thread Sungpeo Kook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sungpeo Kook updated SPARK-36582: - Attachment: (was: Screen Shot 2021-08-25 at 2.35.30 PM.png) > Spark HistoryPage may show

[jira] [Assigned] (SPARK-36505) Improve test coverage for frame.py

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36505: Assignee: Apache Spark > Improve test coverage for frame.py >

[jira] [Assigned] (SPARK-36505) Improve test coverage for frame.py

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36505: Assignee: (was: Apache Spark) > Improve test coverage for frame.py >

[jira] [Commented] (SPARK-36505) Improve test coverage for frame.py

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404337#comment-17404337 ] Apache Spark commented on SPARK-36505: -- User 'itholic' has created a pull request for this issue:

[jira] [Updated] (SPARK-36582) Spark HistoryPage may show 'NotFound' in some multiple attempts cases

2021-08-25 Thread Sungpeo Kook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sungpeo Kook updated SPARK-36582: - Description: Current historypage show a attemptId column in case of hasMultipleAttempts is

[jira] [Assigned] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36398: Assignee: Kousuke Saruta (was: Apache Spark) > Redact sensitive information in Spark

[jira] [Assigned] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36398: Assignee: Apache Spark (was: Kousuke Saruta) > Redact sensitive information in Spark

[jira] [Commented] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404333#comment-17404333 ] Apache Spark commented on SPARK-36398: -- User 'sarutak' has created a pull request for this issue:

[jira] [Reopened] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta reopened SPARK-36398: > Redact sensitive information in Spark Thrift Server log >

[jira] [Commented] (SPARK-36400) Redact sensitive information in Spark Thrift Server UI

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404322#comment-17404322 ] Kousuke Saruta commented on SPARK-36400: Issue resolved in

[jira] [Updated] (SPARK-36400) Redact sensitive information in Spark Thrift Server UI

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36400: --- Fix Version/s: (was: 3.1.2) 3.1.3 > Redact sensitive information in

[jira] [Updated] (SPARK-36400) Redact sensitive information in Spark Thrift Server UI

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36400: --- Fix Version/s: (was: 3.3.0) 3.1.2 > Redact sensitive information in

[jira] [Updated] (SPARK-36400) Redact sensitive information in Spark Thrift Server UI

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36400: --- Fix Version/s: (was: 3.1.3) 3.3.0 > Redact sensitive information in

[jira] [Updated] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36398: --- Fix Version/s: (was: 3.1.3) (was: 3.2.0) > Redact sensitive

[jira] [Commented] (SPARK-36583) Upgrade commons-pool2 from 2.6.2 to 2.11.1

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404306#comment-17404306 ] Apache Spark commented on SPARK-36583: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-36583) Upgrade commons-pool2 from 2.6.2 to 2.11.1

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36583: Assignee: (was: Apache Spark) > Upgrade commons-pool2 from 2.6.2 to 2.11.1 >

[jira] [Commented] (SPARK-36583) Upgrade commons-pool2 from 2.6.2 to 2.11.1

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404305#comment-17404305 ] Apache Spark commented on SPARK-36583: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-36583) Upgrade commons-pool2 from 2.6.2 to 2.11.1

2021-08-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36583: Assignee: Apache Spark > Upgrade commons-pool2 from 2.6.2 to 2.11.1 >

[jira] [Created] (SPARK-36583) Upgrade commons-pool2 from 2.6.2 to 2.11.1

2021-08-25 Thread Yang Jie (Jira)
Yang Jie created SPARK-36583: Summary: Upgrade commons-pool2 from 2.6.2 to 2.11.1 Key: SPARK-36583 URL: https://issues.apache.org/jira/browse/SPARK-36583 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-36582) Spark HistoryPage may show 'NotFound' in some multiple attempts cases

2021-08-25 Thread Sungpeo Kook (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sungpeo Kook updated SPARK-36582: - Description: Current historypage show a attemptId column in case of hasMultipleAttempts is

  1   2   >