[jira] [Commented] (SPARK-35383) Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343017#comment-17343017 ] Apache Spark commented on SPARK-35383: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-35383) Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35383: Assignee: Apache Spark > Improve s3a magic committer support by inferring missing

[jira] [Assigned] (SPARK-35383) Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35383: Assignee: (was: Apache Spark) > Improve s3a magic committer support by inferring

[jira] [Updated] (SPARK-35383) Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35383: -- Description: {code} Exception in thread "main"

[jira] [Created] (SPARK-35383) Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35383: - Summary: Improve s3a magic committer support by inferring missing configs Key: SPARK-35383 URL: https://issues.apache.org/jira/browse/SPARK-35383 Project: Spark

[jira] [Commented] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343007#comment-17343007 ] Apache Spark commented on SPARK-35381: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35381: Assignee: (was: Apache Spark) > Fix lambda variable name issues in nested DataFrame

[jira] [Commented] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343006#comment-17343006 ] Apache Spark commented on SPARK-35381: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35381: Assignee: Apache Spark > Fix lambda variable name issues in nested DataFrame functions

[jira] [Commented] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343003#comment-17343003 ] Apache Spark commented on SPARK-35364: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35364: Assignee: Apache Spark > Renaming the existing Koalas related codes. >

[jira] [Assigned] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35364: Assignee: (was: Apache Spark) > Renaming the existing Koalas related codes. >

[jira] [Commented] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343002#comment-17343002 ] Apache Spark commented on SPARK-35364: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-35380) Support loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342998#comment-17342998 ] Apache Spark commented on SPARK-35380: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-35380) Support loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342996#comment-17342996 ] Apache Spark commented on SPARK-35380: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35380) Support loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35380: Assignee: (was: Apache Spark) > Support loading SparkSessionExtensions from

[jira] [Assigned] (SPARK-35380) Support loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35380: Assignee: Apache Spark > Support loading SparkSessionExtensions from ServiceLoader >

[jira] [Created] (SPARK-35382) Fix lambda variable name issues in nested DataFrame functions in Python APIs

2021-05-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-35382: Summary: Fix lambda variable name issues in nested DataFrame functions in Python APIs Key: SPARK-35382 URL: https://issues.apache.org/jira/browse/SPARK-35382

[jira] [Created] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

2021-05-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-35381: Summary: Fix lambda variable name issues in nested DataFrame functions in R APIs Key: SPARK-35381 URL: https://issues.apache.org/jira/browse/SPARK-35381 Project:

[jira] [Created] (SPARK-35380) Support loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread Kent Yao (Jira)
Kent Yao created SPARK-35380: Summary: Support loading SparkSessionExtensions from ServiceLoader Key: SPARK-35380 URL: https://issues.apache.org/jira/browse/SPARK-35380 Project: Spark Issue

[jira] [Assigned] (SPARK-35379) Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35379: Assignee: (was: Apache Spark) > Improve InferFiltersFromConstraints rule performance

[jira] [Assigned] (SPARK-35379) Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35379: Assignee: Apache Spark > Improve InferFiltersFromConstraints rule performance when

[jira] [Commented] (SPARK-35379) Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342992#comment-17342992 ] Apache Spark commented on SPARK-35379: -- User 'wankunde' has created a pull request for this issue:

[jira] [Created] (SPARK-35379) Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread Wan Kun (Jira)
Wan Kun created SPARK-35379: --- Summary: Improve InferFiltersFromConstraints rule performance when parsing spark sql Key: SPARK-35379 URL: https://issues.apache.org/jira/browse/SPARK-35379 Project: Spark

[jira] [Commented] (SPARK-35378) Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342984#comment-17342984 ] Apache Spark commented on SPARK-35378: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35378) Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35378: Assignee: (was: Apache Spark) > Convert LeafRunnableCommand to LocalRelation when

[jira] [Assigned] (SPARK-35378) Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35378: Assignee: Apache Spark > Convert LeafRunnableCommand to LocalRelation when query with

[jira] [Commented] (SPARK-35378) Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342983#comment-17342983 ] Apache Spark commented on SPARK-35378: -- User 'beliefer' has created a pull request for this issue:

[jira] [Updated] (SPARK-35378) Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35378: --- Summary: Convert LeafRunnableCommand to LocalRelation when query with CTE (was: Support

[jira] [Commented] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342981#comment-17342981 ] Yuming Wang commented on SPARK-35365: - {noformat} -- 2.4

[jira] [Updated] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yao updated SPARK-35365: Attachment: spark3.1.1_report_originalsql > spark3.1.1 use too long time to analyze table fields >

[jira] [Updated] (SPARK-35378) Support LeafRunnableCommand as sub query

2021-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35378: --- Description: Currently, Spark doesn't support LeafRunnableCommand as sub query. Because the

[jira] [Commented] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342978#comment-17342978 ] yao commented on SPARK-35365: - [~yumwang] I have upload two files,  I removed some columns and adjust the

[jira] [Comment Edited] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342978#comment-17342978 ] yao edited comment on SPARK-35365 at 5/12/21, 3:04 AM: --- [~yumwang] I have uploaded

[jira] [Updated] (SPARK-35378) Support LeafRunnableCommand as sub query

2021-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35378: --- Summary: Support LeafRunnableCommand as sub query (was: Support LeafRunnableCommand as sub query

[jira] [Updated] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yao updated SPARK-35365: Attachment: spark3.11report > spark3.1.1 use too long time to analyze table fields >

[jira] [Updated] (SPARK-35378) Support LeafRunnableCommand as sub query so as supports query command of CTE

2021-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35378: --- Summary: Support LeafRunnableCommand as sub query so as supports query command of CTE (was:

[jira] [Updated] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yao updated SPARK-35365: Attachment: spark2.4report > spark3.1.1 use too long time to analyze table fields >

[jira] [Created] (SPARK-35378) Support LeafRunnableCommand as subQuery so as support query command with CTE

2021-05-11 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-35378: -- Summary: Support LeafRunnableCommand as subQuery so as support query command with CTE Key: SPARK-35378 URL: https://issues.apache.org/jira/browse/SPARK-35378 Project:

[jira] [Commented] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342975#comment-17342975 ] Apache Spark commented on SPARK-35377: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35377: Assignee: Apache Spark (was: Kousuke Saruta) > Add JS linter to GA >

[jira] [Assigned] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35377: Assignee: Kousuke Saruta (was: Apache Spark) > Add JS linter to GA >

[jira] [Commented] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342974#comment-17342974 ] Apache Spark commented on SPARK-35377: -- User 'sarutak' has created a pull request for this issue:

[jira] [Updated] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35377: --- Issue Type: Improvement (was: Bug) > Add JS linter to GA > --- > >

[jira] [Created] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35377: -- Summary: Add JS linter to GA Key: SPARK-35377 URL: https://issues.apache.org/jira/browse/SPARK-35377 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yao updated SPARK-35365: Summary: spark3.1.1 use too long time to analyze table fields (was: spark3.1.1 use too long to analyze table

[jira] [Commented] (SPARK-35365) spark3.1.1 use too long to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342962#comment-17342962 ] yao commented on SPARK-35365: - [~yumwang] thanks, I will have a try, then give the results > spark3.1.1 use

[jira] [Assigned] (SPARK-35376) Fallback config should override defaultValue

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35376: Assignee: Apache Spark > Fallback config should override defaultValue >

[jira] [Assigned] (SPARK-35376) Fallback config should override defaultValue

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35376: Assignee: (was: Apache Spark) > Fallback config should override defaultValue >

[jira] [Commented] (SPARK-35376) Fallback config should override defaultValue

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342959#comment-17342959 ] Apache Spark commented on SPARK-35376: -- User 'ulysses-you' has created a pull request for this

[jira] [Resolved] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35361. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32507

[jira] [Assigned] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35361: Assignee: Chao Sun > Improve performance for ApplyFunctionExpression >

[jira] [Created] (SPARK-35376) Fallback config should override defaultValue

2021-05-11 Thread XiDuo You (Jira)
XiDuo You created SPARK-35376: - Summary: Fallback config should override defaultValue Key: SPARK-35376 URL: https://issues.apache.org/jira/browse/SPARK-35376 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35356: - Description: In the case of externalCatalog is InMemoryCatalog, Using a different SparkSession will report the

[jira] [Updated] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35356: - Description: In the case of externalCatalog is InMemoryCatalog, Using a different SparkSession will report the

[jira] [Commented] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342949#comment-17342949 ] yikf commented on SPARK-35356: -- Sorry, my version of Spark is 3.0.0, Please visit

[jira] [Commented] (SPARK-35375) Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342947#comment-17342947 ] Hyukjin Kwon commented on SPARK-35375: -- Fixed in https://github.com/apache/spark/pull/32509 > Use

[jira] [Resolved] (SPARK-35375) Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35375. -- Fix Version/s: 3.2.0 3.1.2 Resolution: Fixed > Use Jinja2 < 3.0.0

[jira] [Commented] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342938#comment-17342938 ] L. C. Hsieh commented on SPARK-35356: - Is the affect version wrong? Do you mean 3.0.2? > Fix issue

[jira] [Updated] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35356: Fix Version/s: (was: 3.0.0) > Fix issue of the createTable when externalCatalog is

[jira] [Commented] (SPARK-35371) Scala UDF returning string or complex type applied to array members returns wrong data

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342937#comment-17342937 ] L. C. Hsieh commented on SPARK-35371: - Oh, I think it was fixed by SPARK-34829. > Scala UDF

[jira] [Commented] (SPARK-35371) Scala UDF returning string or complex type applied to array members returns wrong data

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342936#comment-17342936 ] L. C. Hsieh commented on SPARK-35371: - I just ran the example in both current master branch, and

[jira] [Updated] (SPARK-35375) Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35375: --- Summary: Use Jinja2 < 3.0.0 for Python linter dependency in GA (was: Use Jinja2 < 3.0.0

[jira] [Updated] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35375: --- Description: >From a few hours ago, Python linter fails in GA. The latest Jinja 3.0.0 seems

[jira] [Assigned] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35375: Assignee: Kousuke Saruta (was: Apache Spark) > Use Jinja2 < 3.0.0 for linter dependency

[jira] [Assigned] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35375: Assignee: Apache Spark (was: Kousuke Saruta) > Use Jinja2 < 3.0.0 for linter dependency

[jira] [Commented] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342926#comment-17342926 ] Apache Spark commented on SPARK-35375: -- User 'sarutak' has created a pull request for this issue:

[jira] [Updated] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35375: --- Description: >From a few hours ago, Python linter fails in GA. The latest Jinja 3.0.0 seems

[jira] [Created] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35375: -- Summary: Use Jinja2 < 3.0.0 for linter dependency in GA Key: SPARK-35375 URL: https://issues.apache.org/jira/browse/SPARK-35375 Project: Spark Issue

[jira] [Commented] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342793#comment-17342793 ] Apache Spark commented on SPARK-35361: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35361: Assignee: (was: Apache Spark) > Improve performance for ApplyFunctionExpression >

[jira] [Assigned] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35361: Assignee: Apache Spark > Improve performance for ApplyFunctionExpression >

[jira] [Commented] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342792#comment-17342792 ] Apache Spark commented on SPARK-35361: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-34205) Add pipe API to Dataset

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342788#comment-17342788 ] L. C. Hsieh commented on SPARK-34205: - This is basically driven by our internal customer use-case.

[jira] [Resolved] (SPARK-34205) Add pipe API to Dataset

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-34205. - Resolution: Won't Fix > Add pipe API to Dataset > --- > >

[jira] [Resolved] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35372. Fix Version/s: 3.2.0 Assignee: Hyukjin Kwon Resolution: Fixed This issue

[jira] [Commented] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342733#comment-17342733 ] Apache Spark commented on SPARK-35374: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35374: Assignee: Kousuke Saruta (was: Apache Spark) > Add string-to-number conversion support

[jira] [Assigned] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35374: Assignee: Apache Spark (was: Kousuke Saruta) > Add string-to-number conversion support

[jira] [Commented] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342732#comment-17342732 ] Apache Spark commented on SPARK-35374: -- User 'sarutak' has created a pull request for this issue:

[jira] [Created] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35374: -- Summary: Add string-to-number conversion support to JacksonParser Key: SPARK-35374 URL: https://issues.apache.org/jira/browse/SPARK-35374 Project: Spark

[jira] [Assigned] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35373: Assignee: Apache Spark (was: Sean R. Owen) > Verify checksums of downloaded artifacts

[jira] [Commented] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342709#comment-17342709 ] Apache Spark commented on SPARK-35373: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35373: Assignee: Apache Spark (was: Sean R. Owen) > Verify checksums of downloaded artifacts

[jira] [Assigned] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35373: Assignee: Sean R. Owen (was: Apache Spark) > Verify checksums of downloaded artifacts

[jira] [Created] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-11 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-35373: Summary: Verify checksums of downloaded artifacts in build/mvn Key: SPARK-35373 URL: https://issues.apache.org/jira/browse/SPARK-35373 Project: Spark Issue

[jira] [Commented] (SPARK-35013) Spark allows to set spark.driver.cores=0

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342645#comment-17342645 ] Apache Spark commented on SPARK-35013: -- User 'shahidki31' has created a pull request for this

[jira] [Assigned] (SPARK-35013) Spark allows to set spark.driver.cores=0

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35013: Assignee: (was: Apache Spark) > Spark allows to set spark.driver.cores=0 >

[jira] [Assigned] (SPARK-35013) Spark allows to set spark.driver.cores=0

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35013: Assignee: Apache Spark > Spark allows to set spark.driver.cores=0 >

[jira] [Commented] (SPARK-35370) IllegalArgumentException when loading a PipelineModel with Spark 3

2021-05-11 Thread Avenash Kabeera (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342559#comment-17342559 ] Avenash Kabeera commented on SPARK-35370: - Some additional details.  My model was saved as 

[jira] [Resolved] (SPARK-35229) Spark Job web page is extremely slow while there are more than 1500 events in timeline

2021-05-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35229. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32381

[jira] [Assigned] (SPARK-35229) Spark Job web page is extremely slow while there are more than 1500 events in timeline

2021-05-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35229: -- Assignee: Kousuke Saruta > Spark Job web page is extremely slow while there are more

[jira] [Assigned] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35372: Assignee: (was: Apache Spark) > JDK 11 compilation failure due to StackOverflowError

[jira] [Commented] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342503#comment-17342503 ] Apache Spark commented on SPARK-35372: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342504#comment-17342504 ] Apache Spark commented on SPARK-35372: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35372: Assignee: Apache Spark > JDK 11 compilation failure due to StackOverflowError >

[jira] [Created] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-35372: Summary: JDK 11 compilation failure due to StackOverflowError Key: SPARK-35372 URL: https://issues.apache.org/jira/browse/SPARK-35372 Project: Spark Issue

[jira] [Updated] (SPARK-35371) Scala UDF returning string or complex type applied to array members returns wrong data

2021-05-11 Thread David Benedeki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Benedeki updated SPARK-35371: --- Description: When using an UDF returning string or complex type (Struct) on array members

[jira] [Updated] (SPARK-35371) Scala UDF returning string or complex type applied to array members returns wrong data

2021-05-11 Thread David Benedeki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Benedeki updated SPARK-35371: --- Description: When using an UDF returning string or complex type (Struct) on array members

[jira] [Updated] (SPARK-35371) Scala UDF returning string or complex type applied to array members returns wrong data

2021-05-11 Thread David Benedeki (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Benedeki updated SPARK-35371: --- Description: When using an UDF returning string or complex type (Struct) on array members

  1   2   >