[jira] [Commented] (SPARK-37285) Add Weight of Evidence and Information value to ml.feature

[ https://issues.apache.org/jira/browse/SPARK-37285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489301#comment-17489301 ] zhengruifeng commented on SPARK-37285: -- were these metrics or algorithms implemente

[jira] [Updated] (SPARK-38120) HiveExternalCatalog.listPartitions is failing when partition column name is upper case and dot in partition value

[ https://issues.apache.org/jira/browse/SPARK-38120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38120: -- Affects Version/s: 3.0.3 > HiveExternalCatalog.listPartitions is failing when partition column

[jira] [Updated] (SPARK-38120) HiveExternalCatalog.listPartitions is failing when partition column name is upper case and dot in partition value

[ https://issues.apache.org/jira/browse/SPARK-38120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38120: -- Affects Version/s: 3.1.2 > HiveExternalCatalog.listPartitions is failing when partition column

[jira] [Resolved] (SPARK-38153) Remove option newlines.topLevelStatements in scalafmt.conf

[ https://issues.apache.org/jira/browse/SPARK-38153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38153. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35455 [https://

[jira] [Commented] (SPARK-38033) The structured streaming processing cannot be started because the commitId and offsetId are inconsistent

[ https://issues.apache.org/jira/browse/SPARK-38033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489290#comment-17489290 ] Apache Spark commented on SPARK-38033: -- User 'LLiu' has created a pull request

[jira] [Assigned] (SPARK-38033) The structured streaming processing cannot be started because the commitId and offsetId are inconsistent

[ https://issues.apache.org/jira/browse/SPARK-38033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38033: Assignee: (was: Apache Spark) > The structured streaming processing cannot be started

[jira] [Assigned] (SPARK-38033) The structured streaming processing cannot be started because the commitId and offsetId are inconsistent

[ https://issues.apache.org/jira/browse/SPARK-38033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38033: Assignee: Apache Spark > The structured streaming processing cannot be started because th

[jira] [Commented] (SPARK-38033) The structured streaming processing cannot be started because the commitId and offsetId are inconsistent

[ https://issues.apache.org/jira/browse/SPARK-38033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489289#comment-17489289 ] Apache Spark commented on SPARK-38033: -- User 'LLiu' has created a pull request

[jira] [Assigned] (SPARK-36553) KMeans fails with NegativeArraySizeException for K = 50000 after issue #27758 was introduced

[ https://issues.apache.org/jira/browse/SPARK-36553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36553: Assignee: (was: Apache Spark) > KMeans fails with NegativeArraySizeException for K =

[jira] [Commented] (SPARK-36553) KMeans fails with NegativeArraySizeException for K = 50000 after issue #27758 was introduced

[ https://issues.apache.org/jira/browse/SPARK-36553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489281#comment-17489281 ] Apache Spark commented on SPARK-36553: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-36553) KMeans fails with NegativeArraySizeException for K = 50000 after issue #27758 was introduced

[ https://issues.apache.org/jira/browse/SPARK-36553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36553: Assignee: Apache Spark > KMeans fails with NegativeArraySizeException for K = 5 after

[jira] [Updated] (SPARK-38145) Address the 'pyspark' tagged tests when "spark.sql.ansi.enabled" is True

[ https://issues.apache.org/jira/browse/SPARK-38145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-38145: Parent: SPARK-38154 Issue Type: Sub-task (was: Improvement) > Address the 'pyspark' tagge

[jira] [Commented] (SPARK-36553) KMeans fails with NegativeArraySizeException for K = 50000 after issue #27758 was introduced

[ https://issues.apache.org/jira/browse/SPARK-36553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489278#comment-17489278 ] zhengruifeng commented on SPARK-36553: -- it is a overflow:   {code:java} scala> val

[jira] [Updated] (SPARK-37960) A new framework to represent catalyst expressions in DS v2 APIs

[ https://issues.apache.org/jira/browse/SPARK-37960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37960: --- Description: Spark need a new framework to represent catalyst expressions in DS v2 APIs. CASE ... W

[jira] [Updated] (SPARK-37960) A new framework to represent catalyst expressions in DS v2 APIs

[ https://issues.apache.org/jira/browse/SPARK-37960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37960: --- Summary: A new framework to represent catalyst expressions in DS v2 APIs (was: Support aggregate pu

[jira] [Commented] (SPARK-30661) KMeans blockify input vectors

[ https://issues.apache.org/jira/browse/SPARK-30661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489271#comment-17489271 ] zhengruifeng commented on SPARK-30661: -- ok, I will skip .mllib calling .ml here. We

[jira] [Comment Edited] (SPARK-31007) KMeans optimization based on triangle-inequality

[ https://issues.apache.org/jira/browse/SPARK-31007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489270#comment-17489270 ] zhengruifeng edited comment on SPARK-31007 at 2/9/22, 6:05 AM: ---

[jira] [Commented] (SPARK-31007) KMeans optimization based on triangle-inequality

[ https://issues.apache.org/jira/browse/SPARK-31007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489270#comment-17489270 ] zhengruifeng commented on SPARK-31007: --   this case is not OOM, but the overflow:

[jira] [Comment Edited] (SPARK-38157) Fix /sql/hive-thriftserver/org.apache.spark.sql.hive.thriftserver.ThriftServerQueryTestSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489259#comment-17489259 ] Xinyi Yu edited comment on SPARK-38157 at 2/9/22, 6:00 AM: --- I

[jira] [Commented] (SPARK-38157) Fix /sql/hive-thriftserver/org.apache.spark.sql.hive.thriftserver.ThriftServerQueryTestSuite under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-38157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489259#comment-17489259 ] Xinyi Yu commented on SPARK-38157: -- I will take this one. Kinda my first OSS PR! > Fix

[jira] [Updated] (SPARK-38158) Rdd and shuffle blocks not migrated to new executors when decommission feature is enabled

[ https://issues.apache.org/jira/browse/SPARK-38158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohan Patidar updated SPARK-38158: -- Attachment: sparkConf_Decommission_to_alternate_storage.conf sparkConf_Decommis

[jira] [Created] (SPARK-38158) Rdd and shuffle blocks not migrated to new executors when decommission feature is enabled

Mohan Patidar created SPARK-38158: - Summary: Rdd and shuffle blocks not migrated to new executors when decommission feature is enabled Key: SPARK-38158 URL: https://issues.apache.org/jira/browse/SPARK-38158

[jira] [Commented] (SPARK-36714) bugs in MIniLSH

[ https://issues.apache.org/jira/browse/SPARK-36714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489258#comment-17489258 ] zhengruifeng commented on SPARK-36714: -- [~sheng_1992] Since you had investigate thi

[jira] [Created] (SPARK-38157) Fix /sql/hive-thriftserver/org.apache.spark.sql.hive.thriftserver.ThriftServerQueryTestSuite under ANSI mode

Xinyi Yu created SPARK-38157: Summary: Fix /sql/hive-thriftserver/org.apache.spark.sql.hive.thriftserver.ThriftServerQueryTestSuite under ANSI mode Key: SPARK-38157 URL: https://issues.apache.org/jira/browse/SPARK-38

[jira] [Commented] (SPARK-38156) Support CREATE EXTERNAL TABLE LIKE syntax

[ https://issues.apache.org/jira/browse/SPARK-38156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489250#comment-17489250 ] Apache Spark commented on SPARK-38156: -- User 'yeshengm' has created a pull request

[jira] [Commented] (SPARK-38156) Support CREATE EXTERNAL TABLE LIKE syntax

[ https://issues.apache.org/jira/browse/SPARK-38156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489249#comment-17489249 ] Apache Spark commented on SPARK-38156: -- User 'yeshengm' has created a pull request

[jira] [Assigned] (SPARK-38156) Support CREATE EXTERNAL TABLE LIKE syntax

[ https://issues.apache.org/jira/browse/SPARK-38156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38156: Assignee: (was: Apache Spark) > Support CREATE EXTERNAL TABLE LIKE syntax > -

[jira] [Assigned] (SPARK-38156) Support CREATE EXTERNAL TABLE LIKE syntax

[ https://issues.apache.org/jira/browse/SPARK-38156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38156: Assignee: Apache Spark > Support CREATE EXTERNAL TABLE LIKE syntax >

[jira] [Created] (SPARK-38156) Support CREATE EXTERNAL TABLE LIKE syntax

Yesheng Ma created SPARK-38156: -- Summary: Support CREATE EXTERNAL TABLE LIKE syntax Key: SPARK-38156 URL: https://issues.apache.org/jira/browse/SPARK-38156 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38155) Disallow distinct aggregate in lateral subqueries with unsupported correlated predicates

[ https://issues.apache.org/jira/browse/SPARK-38155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-38155: - Description: Block lateral subqueries in CheckAnalysis that contains DISTINCT aggregate and cor

[jira] [Created] (SPARK-38155) Disallow distinct aggregate in lateral subqueries with unsupported correlated predicates

Allison Wang created SPARK-38155: Summary: Disallow distinct aggregate in lateral subqueries with unsupported correlated predicates Key: SPARK-38155 URL: https://issues.apache.org/jira/browse/SPARK-38155

[jira] [Created] (SPARK-38154) Set up a new GA job to run tests with ANSI mode

Wenchen Fan created SPARK-38154: --- Summary: Set up a new GA job to run tests with ANSI mode Key: SPARK-38154 URL: https://issues.apache.org/jira/browse/SPARK-38154 Project: Spark Issue Type: Imp

[jira] [Resolved] (SPARK-38149) Upgrade joda-time to 2.10.13

[ https://issues.apache.org/jira/browse/SPARK-38149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38149. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35452 [https://

[jira] [Resolved] (SPARK-38147) Upgrade shapeless to 2.3.7

[ https://issues.apache.org/jira/browse/SPARK-38147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38147. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35450 [https://

[jira] [Assigned] (SPARK-37507) Add the TO_BINARY() function

[ https://issues.apache.org/jira/browse/SPARK-37507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37507: Assignee: (was: Apache Spark) > Add the TO_BINARY() function > --

[jira] [Commented] (SPARK-37507) Add the TO_BINARY() function

[ https://issues.apache.org/jira/browse/SPARK-37507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489238#comment-17489238 ] Apache Spark commented on SPARK-37507: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-37507) Add the TO_BINARY() function

[ https://issues.apache.org/jira/browse/SPARK-37507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37507: Assignee: Apache Spark > Add the TO_BINARY() function > > >

[jira] [Assigned] (SPARK-38153) Remove option newlines.topLevelStatements in scalafmt.conf

[ https://issues.apache.org/jira/browse/SPARK-38153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38153: Assignee: Apache Spark (was: Gengliang Wang) > Remove option newlines.topLevelStatements

[jira] [Assigned] (SPARK-38153) Remove option newlines.topLevelStatements in scalafmt.conf

[ https://issues.apache.org/jira/browse/SPARK-38153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38153: Assignee: Gengliang Wang (was: Apache Spark) > Remove option newlines.topLevelStatements

[jira] [Commented] (SPARK-38153) Remove option newlines.topLevelStatements in scalafmt.conf

[ https://issues.apache.org/jira/browse/SPARK-38153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489236#comment-17489236 ] Apache Spark commented on SPARK-38153: -- User 'gengliangwang' has created a pull req

[jira] [Created] (SPARK-38153) Remove option newlines.topLevelStatements in scalafmt.conf

Gengliang Wang created SPARK-38153: -- Summary: Remove option newlines.topLevelStatements in scalafmt.conf Key: SPARK-38153 URL: https://issues.apache.org/jira/browse/SPARK-38153 Project: Spark

[jira] [Commented] (SPARK-31007) KMeans optimization based on triangle-inequality

[ https://issues.apache.org/jira/browse/SPARK-31007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489235#comment-17489235 ] Sean R. Owen commented on SPARK-31007: -- Hm, can you use a simpler "unpacked" repres

[jira] [Assigned] (SPARK-38145) Address the 'pyspark' tagged tests when "spark.sql.ansi.enabled" is True

[ https://issues.apache.org/jira/browse/SPARK-38145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38145: Assignee: (was: Apache Spark) > Address the 'pyspark' tagged tests when "spark.sql.an

[jira] [Commented] (SPARK-38145) Address the 'pyspark' tagged tests when "spark.sql.ansi.enabled" is True

[ https://issues.apache.org/jira/browse/SPARK-38145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489234#comment-17489234 ] Apache Spark commented on SPARK-38145: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-38145) Address the 'pyspark' tagged tests when "spark.sql.ansi.enabled" is True

[ https://issues.apache.org/jira/browse/SPARK-38145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38145: Assignee: Apache Spark > Address the 'pyspark' tagged tests when "spark.sql.ansi.enabled"

[jira] [Assigned] (SPARK-38086) Make ArrowColumnVector Extendable

[ https://issues.apache.org/jira/browse/SPARK-38086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38086: - Assignee: Kazuyuki Tanimura > Make ArrowColumnVector Extendable > -

[jira] [Resolved] (SPARK-38086) Make ArrowColumnVector Extendable

[ https://issues.apache.org/jira/browse/SPARK-38086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38086. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35393 [https://

[jira] [Commented] (SPARK-36714) bugs in MIniLSH

[ https://issues.apache.org/jira/browse/SPARK-36714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489232#comment-17489232 ] zhengruifeng commented on SPARK-36714: -- could you please provide a simple script to

[jira] [Updated] (SPARK-38145) Address the 'pyspark' tagged tests when "spark.sql.ansi.enabled" is True

[ https://issues.apache.org/jira/browse/SPARK-38145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-38145: Summary: Address the 'pyspark' tagged tests when "spark.sql.ansi.enabled" is True (was: Address t

[jira] [Commented] (SPARK-31007) KMeans optimization based on triangle-inequality

[ https://issues.apache.org/jira/browse/SPARK-31007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489230#comment-17489230 ] zhengruifeng commented on SPARK-31007: -- [~srowen]  This optimization needs an array

[jira] [Created] (SPARK-38150) Update comment of RelationConversions

angerszhu created SPARK-38150: - Summary: Update comment of RelationConversions Key: SPARK-38150 URL: https://issues.apache.org/jira/browse/SPARK-38150 Project: Spark Issue Type: Task Co

[jira] [Commented] (SPARK-38037) Spark MLlib FPGrowth not working with 40+ items in Frequent Item set

[ https://issues.apache.org/jira/browse/SPARK-38037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489225#comment-17489225 ] zhengruifeng commented on SPARK-38037: -- could you please provide a simple script to

[jira] [Updated] (SPARK-38152) Upgrade Jetty version from 9.4.40 to 9.4.44

[ https://issues.apache.org/jira/browse/SPARK-38152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-38152: --- Description: Upgrade Jetty version to 9.4.44 in Spark 2.4.8 and 3.0.3 in order to fix vulner

[jira] [Created] (SPARK-38152) Upgrade Jetty version from 9.4.40 to 9.4.44

SHOBHIT SHUKLA created SPARK-38152: -- Summary: Upgrade Jetty version from 9.4.40 to 9.4.44 Key: SPARK-38152 URL: https://issues.apache.org/jira/browse/SPARK-38152 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-38150) Update comment of RelationConversions

[ https://issues.apache.org/jira/browse/SPARK-38150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489219#comment-17489219 ] Apache Spark commented on SPARK-38150: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-38150) Update comment of RelationConversions

[ https://issues.apache.org/jira/browse/SPARK-38150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38150: Assignee: (was: Apache Spark) > Update comment of RelationConversions > -

[jira] [Assigned] (SPARK-38150) Update comment of RelationConversions

[ https://issues.apache.org/jira/browse/SPARK-38150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38150: Assignee: Apache Spark > Update comment of RelationConversions >

[jira] [Updated] (SPARK-38030) Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1

[ https://issues.apache.org/jira/browse/SPARK-38030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38030: -- Fix Version/s: 3.1.3 > Query with cast containing non-nullable columns fails with AQE on Spark

[jira] [Updated] (SPARK-38030) Query with cast containing non-nullable columns fails with AQE on Spark 3.1.1

[ https://issues.apache.org/jira/browse/SPARK-38030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38030: -- Fix Version/s: 3.2.2 > Query with cast containing non-nullable columns fails with AQE on Spark

[jira] [Updated] (SPARK-38151) Flaky Test: DateTimeUtilsSuite.`daysToMicros and microsToDays`

[ https://issues.apache.org/jira/browse/SPARK-38151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38151: -- Description: - https://github.com/dongjoon-hyun/spark/runs/5119322349?check_suite_focus=true {

[jira] [Created] (SPARK-38151) Flaky Test: DateTimeUtilsSuite.`daysToMicros and microsToDays`

Dongjoon Hyun created SPARK-38151: - Summary: Flaky Test: DateTimeUtilsSuite.`daysToMicros and microsToDays` Key: SPARK-38151 URL: https://issues.apache.org/jira/browse/SPARK-38151 Project: Spark

[jira] [Assigned] (SPARK-38149) Upgrade joda-time to 2.10.13

[ https://issues.apache.org/jira/browse/SPARK-38149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38149: Assignee: Apache Spark (was: Kousuke Saruta) > Upgrade joda-time to 2.10.13 > --

[jira] [Commented] (SPARK-38149) Upgrade joda-time to 2.10.13

[ https://issues.apache.org/jira/browse/SPARK-38149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489214#comment-17489214 ] Apache Spark commented on SPARK-38149: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-38149) Upgrade joda-time to 2.10.13

[ https://issues.apache.org/jira/browse/SPARK-38149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38149: Assignee: Kousuke Saruta (was: Apache Spark) > Upgrade joda-time to 2.10.13 > --

[jira] [Created] (SPARK-38149) Upgrade joda-time to 2.10.13

Kousuke Saruta created SPARK-38149: -- Summary: Upgrade joda-time to 2.10.13 Key: SPARK-38149 URL: https://issues.apache.org/jira/browse/SPARK-38149 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-38148) Do not add dynamic partition pruning if there exists static partition pruning

[ https://issues.apache.org/jira/browse/SPARK-38148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489212#comment-17489212 ] Apache Spark commented on SPARK-38148: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-38148) Do not add dynamic partition pruning if there exists static partition pruning

[ https://issues.apache.org/jira/browse/SPARK-38148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38148: Assignee: (was: Apache Spark) > Do not add dynamic partition pruning if there exists

[jira] [Commented] (SPARK-38148) Do not add dynamic partition pruning if there exists static partition pruning

[ https://issues.apache.org/jira/browse/SPARK-38148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489211#comment-17489211 ] Apache Spark commented on SPARK-38148: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-38147) Upgrade shapeless to 2.3.7

[ https://issues.apache.org/jira/browse/SPARK-38147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38147: - Assignee: Dongjoon Hyun > Upgrade shapeless to 2.3.7 > -- > >

[jira] [Assigned] (SPARK-38148) Do not add dynamic partition pruning if there exists static partition pruning

[ https://issues.apache.org/jira/browse/SPARK-38148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38148: Assignee: Apache Spark > Do not add dynamic partition pruning if there exists static part

[jira] [Assigned] (SPARK-38147) Upgrade shapeless to 2.3.7

[ https://issues.apache.org/jira/browse/SPARK-38147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38147: Assignee: Apache Spark > Upgrade shapeless to 2.3.7 > -- > >

[jira] [Assigned] (SPARK-38147) Upgrade shapeless to 2.3.7

[ https://issues.apache.org/jira/browse/SPARK-38147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38147: Assignee: (was: Apache Spark) > Upgrade shapeless to 2.3.7 >

[jira] [Commented] (SPARK-38147) Upgrade shapeless to 2.3.7

[ https://issues.apache.org/jira/browse/SPARK-38147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489210#comment-17489210 ] Apache Spark commented on SPARK-38147: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-38148) Do not add dynamic partition pruning if there exists static partition pruning

XiDuo You created SPARK-38148: - Summary: Do not add dynamic partition pruning if there exists static partition pruning Key: SPARK-38148 URL: https://issues.apache.org/jira/browse/SPARK-38148 Project: Spar

[jira] [Created] (SPARK-38147) Upgrade shapeless to 2.3.7

Dongjoon Hyun created SPARK-38147: - Summary: Upgrade shapeless to 2.3.7 Key: SPARK-38147 URL: https://issues.apache.org/jira/browse/SPARK-38147 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-38146) UDAF fails with unsafe rows containing a TIMESTAMP_NTZ column

[ https://issues.apache.org/jira/browse/SPARK-38146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489207#comment-17489207 ] Bruce Robbins edited comment on SPARK-38146 at 2/9/22, 2:23 AM: --

[jira] [Commented] (SPARK-38146) UDAF fails with unsafe rows containing a TIMESTAMP_NTZ column

[ https://issues.apache.org/jira/browse/SPARK-38146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489207#comment-17489207 ] Bruce Robbins commented on SPARK-38146: --- This affects master only and has a simple

[jira] [Assigned] (SPARK-37414) Inline type hints for python/pyspark/ml/tuning.py

[ https://issues.apache.org/jira/browse/SPARK-37414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37414: -- Assignee: Maciej Szymkiewicz > Inline type hints for python/pyspark/ml/tuning

[jira] [Resolved] (SPARK-37414) Inline type hints for python/pyspark/ml/tuning.py

[ https://issues.apache.org/jira/browse/SPARK-37414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37414. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35406

[jira] [Created] (SPARK-38146) UDAF fails with unsafe rows containing a TIMESTAMP_NTZ column

Bruce Robbins created SPARK-38146: - Summary: UDAF fails with unsafe rows containing a TIMESTAMP_NTZ column Key: SPARK-38146 URL: https://issues.apache.org/jira/browse/SPARK-38146 Project: Spark

[jira] [Updated] (SPARK-38094) Parquet: enable matching schema columns by field id

[ https://issues.apache.org/jira/browse/SPARK-38094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackie Zhang updated SPARK-38094: - Description: Field Id is a native field in the Parquet schema ([https://github.com/apache/parqu

[jira] [Commented] (SPARK-38139) ml.recommendation.ALS doctests failures

[ https://issues.apache.org/jira/browse/SPARK-38139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489194#comment-17489194 ] Hyukjin Kwon commented on SPARK-38139: -- Yeah, I think we should fix as so! > ml.re

[jira] [Updated] (SPARK-37980) Extend METADATA column to support row indices for file based data sources

[ https://issues.apache.org/jira/browse/SPARK-37980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37980: - Affects Version/s: 3.3.0 (was: 3.3) > Extend METADATA column to suppo

[jira] [Updated] (SPARK-37923) Generate partition transforms for BucketSpec inside parser

[ https://issues.apache.org/jira/browse/SPARK-37923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37923: - Affects Version/s: 3.3.0 (was: 3.3) > Generate partition transforms f

[jira] [Updated] (SPARK-38094) Parquet: enable matching schema columns by field id

[ https://issues.apache.org/jira/browse/SPARK-38094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38094: - Affects Version/s: 3.3.0 (was: 3.3) > Parquet: enable matching schema

[jira] [Updated] (SPARK-38032) Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL

[ https://issues.apache.org/jira/browse/SPARK-38032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38032: - Affects Version/s: 3.3.0 (was: 3.3) > Upgrade Arrow version < 7.0.0 f

[jira] [Updated] (SPARK-38025) Improve test suite ExternalCatalogSuite

[ https://issues.apache.org/jira/browse/SPARK-38025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38025: - Affects Version/s: 3.3.0 (was: 3.3) > Improve test suite ExternalCata

[jira] [Updated] (SPARK-38114) Spark build fails in Windows

[ https://issues.apache.org/jira/browse/SPARK-38114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38114: - Affects Version/s: 3.3.0 (was: 3.3) > Spark build fails in Windows >

[jira] [Updated] (SPARK-38138) Materialize QueryPlan subqueries

[ https://issues.apache.org/jira/browse/SPARK-38138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38138: - Affects Version/s: 3.3.0 (was: 3.3) > Materialize QueryPlan subquerie

[jira] [Updated] (SPARK-38100) Remove unused method in `Decimal`

[ https://issues.apache.org/jira/browse/SPARK-38100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38100: - Fix Version/s: 3.3.0 (was: 3.3) > Remove unused method in `Decimal` > ---

[jira] [Updated] (SPARK-38145) Address the tests when "spark.sql.ansi.enabled" is True

[ https://issues.apache.org/jira/browse/SPARK-38145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38145: - Affects Version/s: 3.3.0 (was: 3.3) > Address the tests when "spark.s

[jira] [Created] (SPARK-38145) Address the tests when "spark.sql.ansi.enabled" is True

Haejoon Lee created SPARK-38145: --- Summary: Address the tests when "spark.sql.ansi.enabled" is True Key: SPARK-38145 URL: https://issues.apache.org/jira/browse/SPARK-38145 Project: Spark Issue T

[jira] [Resolved] (SPARK-37404) Inline type hints for python/pyspark/ml/evaluation.py

[ https://issues.apache.org/jira/browse/SPARK-37404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37404. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35403

[jira] [Assigned] (SPARK-37404) Inline type hints for python/pyspark/ml/evaluation.py

[ https://issues.apache.org/jira/browse/SPARK-37404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37404: -- Assignee: Maciej Szymkiewicz > Inline type hints for python/pyspark/ml/evalua

[jira] [Comment Edited] (SPARK-38042) Encoder cannot be found when a tuple component is a type alias for an Array

[ https://issues.apache.org/jira/browse/SPARK-38042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489167#comment-17489167 ] Johan Nyström-Persson edited comment on SPARK-38042 at 2/8/22, 11:50 PM: -

[jira] [Comment Edited] (SPARK-38042) Encoder cannot be found when a tuple component is a type alias for an Array

[ https://issues.apache.org/jira/browse/SPARK-38042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489167#comment-17489167 ] Johan Nyström-Persson edited comment on SPARK-38042 at 2/8/22, 11:49 PM: -

[jira] [Commented] (SPARK-38042) Encoder cannot be found when a tuple component is a type alias for an Array

[ https://issues.apache.org/jira/browse/SPARK-38042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489167#comment-17489167 ] Johan Nyström-Persson commented on SPARK-38042: --- My initial idea above was

[jira] [Resolved] (SPARK-38069) improve structured streaming window of calculated

[ https://issues.apache.org/jira/browse/SPARK-38069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38069. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35362 [https://gith

[jira] [Assigned] (SPARK-38069) improve structured streaming window of calculated

[ https://issues.apache.org/jira/browse/SPARK-38069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38069: --- Assignee: nyingping > improve structured streaming window of calculated > -

[jira] [Resolved] (SPARK-38144) Remove unused `spark.storage.safetyFraction` config

[ https://issues.apache.org/jira/browse/SPARK-38144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38144. --- Fix Version/s: 3.3.0 3.2.2 3.1.3 Resolution: Fix

  1   2   >