[jira] [Created] (SPARK-33409) Spark job can not be killed in BoradcastNestedLoopJoin

zhou xiang created SPARK-33409: -- Summary: Spark job can not be killed in BoradcastNestedLoopJoin Key: SPARK-33409 URL: https://issues.apache.org/jira/browse/SPARK-33409 Project: Spark Issue

[jira] [Commented] (SPARK-33388) Merge In and InSet predicate

[ https://issues.apache.org/jira/browse/SPARK-33388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229043#comment-17229043 ] L. C. Hsieh commented on SPARK-33388: - Oh, almost forget this. > Merge In and InSet predicate >

[jira] [Commented] (SPARK-32917) Add support for executors to push shuffle blocks after successful map task completion

[ https://issues.apache.org/jira/browse/SPARK-32917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229037#comment-17229037 ] Apache Spark commented on SPARK-32917: -- User 'otterc' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32917) Add support for executors to push shuffle blocks after successful map task completion

[ https://issues.apache.org/jira/browse/SPARK-32917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32917: Assignee: Apache Spark > Add support for executors to push shuffle blocks after

[jira] [Commented] (SPARK-32917) Add support for executors to push shuffle blocks after successful map task completion

[ https://issues.apache.org/jira/browse/SPARK-32917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229036#comment-17229036 ] Apache Spark commented on SPARK-32917: -- User 'otterc' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32917) Add support for executors to push shuffle blocks after successful map task completion

[ https://issues.apache.org/jira/browse/SPARK-32917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32917: Assignee: (was: Apache Spark) > Add support for executors to push shuffle blocks

[jira] [Resolved] (SPARK-33391) element_at with CreateArray not respect one based index

[ https://issues.apache.org/jira/browse/SPARK-33391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33391. - Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-33391) element_at with CreateArray not respect one based index

[ https://issues.apache.org/jira/browse/SPARK-33391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33391: --- Assignee: Leanken.Lin > element_at with CreateArray not respect one based index >

[jira] [Commented] (SPARK-33383) Improve performance of Column.isin Expression

[ https://issues.apache.org/jira/browse/SPARK-33383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229024#comment-17229024 ] L. C. Hsieh commented on SPARK-33383: - When codegen doesn't work, it might not only in expression

[jira] [Commented] (SPARK-12567) Add aes_encrypt and aes_decrypt UDFs

[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229023#comment-17229023 ] Takeshi Yamamuro commented on SPARK-12567: -- Reopened. > Add aes_encrypt and aes_decrypt UDFs >

[jira] [Updated] (SPARK-12567) Add aes_encrypt and aes_decrypt UDFs

[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12567: - Labels: (was: bulk-closed) > Add aes_encrypt and aes_decrypt UDFs >

[jira] [Updated] (SPARK-12567) Add aes_encrypt and aes_decrypt UDFs

[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12567: - Affects Version/s: 3.1.0 > Add aes_encrypt and aes_decrypt UDFs >

[jira] [Reopened] (SPARK-12567) Add aes_encrypt and aes_decrypt UDFs

[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro reopened SPARK-12567: -- Assignee: (was: Kai Jiang) > Add aes_encrypt and aes_decrypt UDFs >

[jira] [Commented] (SPARK-33400) Reduce unneeded sorts between two SortMergeJoins

[ https://issues.apache.org/jira/browse/SPARK-33400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229016#comment-17229016 ] Takeshi Yamamuro commented on SPARK-33400: -- Is this different from SPARK-33399 ? > Reduce

[jira] [Commented] (SPARK-33401) Vector type column is not possible to create using spark SQL

[ https://issues.apache.org/jira/browse/SPARK-33401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229015#comment-17229015 ] Takeshi Yamamuro commented on SPARK-33401: -- I think this is not a bug but an expected behaviour

[jira] [Assigned] (SPARK-32354) Fix & re-enable failing R K8s tests

[ https://issues.apache.org/jira/browse/SPARK-32354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32354: - Assignee: Dongjoon Hyun > Fix & re-enable failing R K8s tests >

[jira] [Assigned] (SPARK-33408) Use R 4.0 in K8s R image

[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33408: - Assignee: Dongjoon Hyun > Use R 4.0 in K8s R image > > >

[jira] [Updated] (SPARK-33187) Add a check on the number of returned partitions in the HiveShim#getPartitionsByFilter method

[ https://issues.apache.org/jira/browse/SPARK-33187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-33187: - Affects Version/s: (was: 3.0.1) 3.1.0

[jira] [Commented] (SPARK-33408) Use R 4.0 in K8s R image

[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229001#comment-17229001 ] Apache Spark commented on SPARK-33408: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-33408) Use R 4.0 in K8s R image

[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33408: -- Affects Version/s: 3.0.0 3.0.1 > Use R 4.0 in K8s R image >

[jira] [Commented] (SPARK-33408) Use R 4.0 in K8s R image

[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228996#comment-17228996 ] Apache Spark commented on SPARK-33408: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-33244) Unify the code paths for spark.table and spark.read.table

[ https://issues.apache.org/jira/browse/SPARK-33244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33244. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30148

[jira] [Assigned] (SPARK-33244) Unify the code paths for spark.table and spark.read.table

[ https://issues.apache.org/jira/browse/SPARK-33244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33244: --- Assignee: Yuanjian Li > Unify the code paths for spark.table and spark.read.table >

[jira] [Commented] (SPARK-33408) Use R 4.0 in K8s R image

[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228992#comment-17228992 ] Apache Spark commented on SPARK-33408: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-33408) Use R 4.0 in K8s R image

[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228991#comment-17228991 ] Apache Spark commented on SPARK-33408: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33408) Use R 4.0 in K8s R image

[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33408: Assignee: (was: Apache Spark) > Use R 4.0 in K8s R image >

[jira] [Assigned] (SPARK-33408) Use R 4.0 in K8s R image

[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33408: Assignee: Apache Spark > Use R 4.0 in K8s R image > > >

[jira] [Updated] (SPARK-33408) Use R 4.0 in K8s R image

[ https://issues.apache.org/jira/browse/SPARK-33408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33408: -- Summary: Use R 4.0 in K8s R image (was: Use K8s R image to use R 4.0) > Use R 4.0 in K8s R

[jira] [Created] (SPARK-33408) Use K8s R image to use R 4.0

Dongjoon Hyun created SPARK-33408: - Summary: Use K8s R image to use R 4.0 Key: SPARK-33408 URL: https://issues.apache.org/jira/browse/SPARK-33408 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-33366) Migrate LOAD DATA to new resolution framework

[ https://issues.apache.org/jira/browse/SPARK-33366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33366: --- Assignee: Terry Kim > Migrate LOAD DATA to new resolution framework >

[jira] [Resolved] (SPARK-33366) Migrate LOAD DATA to new resolution framework

[ https://issues.apache.org/jira/browse/SPARK-33366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33366. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30270

[jira] [Commented] (SPARK-33407) Simplify the exception message from Python UDFs

[ https://issues.apache.org/jira/browse/SPARK-33407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228982#comment-17228982 ] Apache Spark commented on SPARK-33407: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-33407) Simplify the exception message from Python UDFs

[ https://issues.apache.org/jira/browse/SPARK-33407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228981#comment-17228981 ] Apache Spark commented on SPARK-33407: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-33407) Simplify the exception message from Python UDFs

[ https://issues.apache.org/jira/browse/SPARK-33407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33407: Assignee: (was: Apache Spark) > Simplify the exception message from Python UDFs >

[jira] [Assigned] (SPARK-33407) Simplify the exception message from Python UDFs

[ https://issues.apache.org/jira/browse/SPARK-33407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33407: Assignee: Apache Spark > Simplify the exception message from Python UDFs >

[jira] [Commented] (SPARK-33372) Fix InSet bucket pruning

[ https://issues.apache.org/jira/browse/SPARK-33372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228976#comment-17228976 ] Apache Spark commented on SPARK-33372: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-33407) Simplify the exception message from Python UDFs

Hyukjin Kwon created SPARK-33407: Summary: Simplify the exception message from Python UDFs Key: SPARK-33407 URL: https://issues.apache.org/jira/browse/SPARK-33407 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-33369) Skip schema inference in DataframeWriter.save() if table provider supports external metadata

[ https://issues.apache.org/jira/browse/SPARK-33369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33369. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30273

[jira] [Updated] (SPARK-33405) Upgrade commons-compress to 1.20

[ https://issues.apache.org/jira/browse/SPARK-33405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33405: -- Fix Version/s: 2.4.8 > Upgrade commons-compress to 1.20 > >

[jira] [Resolved] (SPARK-33213) Upgrade Apache Arrow to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33213. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30306

[jira] [Assigned] (SPARK-33213) Upgrade Apache Arrow to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33213: - Assignee: Chao Sun > Upgrade Apache Arrow to 2.0.0 > - > >

[jira] [Assigned] (SPARK-32691) Update commons-crypto to v1.1.0

[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-32691: Assignee: huangtianhua (was: zhengruifeng) > Update commons-crypto to v1.1.0 >

[jira] [Updated] (SPARK-32691) Update commons-crypto to v1.1.0

[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32691: -- Affects Version/s: 2.4.7 3.0.0 3.0.1 > Update

[jira] [Commented] (SPARK-33405) Upgrade commons-compress to 1.20

[ https://issues.apache.org/jira/browse/SPARK-33405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228925#comment-17228925 ] Apache Spark commented on SPARK-33405: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-33223) Expose state information on SS UI

[ https://issues.apache.org/jira/browse/SPARK-33223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-33223. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30151

[jira] [Assigned] (SPARK-33223) Expose state information on SS UI

[ https://issues.apache.org/jira/browse/SPARK-33223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-33223: Assignee: Gabor Somogyi > Expose state information on SS UI >

[jira] [Updated] (SPARK-33363) Add prompt information related to the current task when pyspark starts

[ https://issues.apache.org/jira/browse/SPARK-33363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33363: - Component/s: SparkR > Add prompt information related to the current task when pyspark starts >

[jira] [Assigned] (SPARK-33363) Add prompt information related to the current task when pyspark starts

[ https://issues.apache.org/jira/browse/SPARK-33363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33363: Assignee: akiyamaneko > Add prompt information related to the current task when pyspark

[jira] [Resolved] (SPARK-33363) Add prompt information related to the current task when pyspark starts

[ https://issues.apache.org/jira/browse/SPARK-33363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33363. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30266

[jira] [Resolved] (SPARK-33405) Upgrade commons-compress to 1.20

[ https://issues.apache.org/jira/browse/SPARK-33405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33405. -- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-33405) Upgrade commons-compress to 1.20

[ https://issues.apache.org/jira/browse/SPARK-33405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33405: Assignee: Dongjoon Hyun > Upgrade commons-compress to 1.20 >

[jira] [Resolved] (SPARK-33397) mistakenly generate markdown to html for available-patterns-for-shs-custom-executor-log-ur

[ https://issues.apache.org/jira/browse/SPARK-33397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-33397. -- Fix Version/s: 3.1.0 3.0.2 Assignee: Kent Yao

[jira] [Commented] (SPARK-33213) Upgrade Apache Arrow to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228893#comment-17228893 ] Apache Spark commented on SPARK-33213: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-33213) Upgrade Apache Arrow to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228892#comment-17228892 ] Chao Sun commented on SPARK-33213: -- Thanks. I think there are still a few improvements on the Java side

[jira] [Assigned] (SPARK-33213) Upgrade Apache Arrow to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33213: Assignee: Apache Spark > Upgrade Apache Arrow to 2.0.0 > - >

[jira] [Assigned] (SPARK-33213) Upgrade Apache Arrow to 2.0.0

[ https://issues.apache.org/jira/browse/SPARK-33213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33213: Assignee: (was: Apache Spark) > Upgrade Apache Arrow to 2.0.0 >

[jira] [Created] (SPARK-33406) k8s pyspark shell doesn't honor pyspark memory setting

Thomas Graves created SPARK-33406: - Summary: k8s pyspark shell doesn't honor pyspark memory setting Key: SPARK-33406 URL: https://issues.apache.org/jira/browse/SPARK-33406 Project: Spark

[jira] [Assigned] (SPARK-33251) Migration to NumPy documentation style in ML (pyspark.ml.*)

[ https://issues.apache.org/jira/browse/SPARK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33251: Assignee: Maciej Szymkiewicz > Migration to NumPy documentation style in ML

[jira] [Resolved] (SPARK-33251) Migration to NumPy documentation style in ML (pyspark.ml.*)

[ https://issues.apache.org/jira/browse/SPARK-33251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33251. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30285

[jira] [Commented] (SPARK-33405) Upgrade commons-compress to 1.20

[ https://issues.apache.org/jira/browse/SPARK-33405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228879#comment-17228879 ] Apache Spark commented on SPARK-33405: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-33405) Upgrade commons-compress to 1.20

[ https://issues.apache.org/jira/browse/SPARK-33405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228878#comment-17228878 ] Apache Spark commented on SPARK-33405: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33405) Upgrade commons-compress to 1.20

[ https://issues.apache.org/jira/browse/SPARK-33405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33405: Assignee: (was: Apache Spark) > Upgrade commons-compress to 1.20 >

[jira] [Assigned] (SPARK-33405) Upgrade commons-compress to 1.20

[ https://issues.apache.org/jira/browse/SPARK-33405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33405: Assignee: Apache Spark > Upgrade commons-compress to 1.20 >

[jira] [Commented] (SPARK-33254) Migration to NumPy documentation style in Core (pyspark.*, pyspark.resource.*, etc.)

[ https://issues.apache.org/jira/browse/SPARK-33254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228871#comment-17228871 ] Hyukjin Kwon commented on SPARK-33254: -- Thank you [~zero323]!! > Migration to NumPy documentation

[jira] [Commented] (SPARK-33405) Upgrade commons-compress to 1.20

[ https://issues.apache.org/jira/browse/SPARK-33405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228870#comment-17228870 ] Apache Spark commented on SPARK-33405: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-33405) Upgrade commons-compress to 1.20

Dongjoon Hyun created SPARK-33405: - Summary: Upgrade commons-compress to 1.20 Key: SPARK-33405 URL: https://issues.apache.org/jira/browse/SPARK-33405 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-33254) Migration to NumPy documentation style in Core (pyspark.*, pyspark.resource.*, etc.)

[ https://issues.apache.org/jira/browse/SPARK-33254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228867#comment-17228867 ] Maciej Szymkiewicz commented on SPARK-33254: I am working on this one. > Migration to NumPy

[jira] [Assigned] (SPARK-33404) "date_trunc" expression returns incorrect results

[ https://issues.apache.org/jira/browse/SPARK-33404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33404: Assignee: (was: Apache Spark) > "date_trunc" expression returns incorrect results >

[jira] [Commented] (SPARK-33404) "date_trunc" expression returns incorrect results

[ https://issues.apache.org/jira/browse/SPARK-33404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228864#comment-17228864 ] Apache Spark commented on SPARK-33404: -- User 'utkarsh39' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33404) "date_trunc" expression returns incorrect results

[ https://issues.apache.org/jira/browse/SPARK-33404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33404: Assignee: Apache Spark > "date_trunc" expression returns incorrect results >

[jira] [Created] (SPARK-33404) "date_trunc" expression returns incorrect results

Utkarsh Agarwal created SPARK-33404: --- Summary: "date_trunc" expression returns incorrect results Key: SPARK-33404 URL: https://issues.apache.org/jira/browse/SPARK-33404 Project: Spark

[jira] [Commented] (SPARK-33383) Improve performance of Column.isin Expression

[ https://issues.apache.org/jira/browse/SPARK-33383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228851#comment-17228851 ] Felix Wollschläger commented on SPARK-33383: I can see on the console that codegen doesnt

[jira] [Updated] (SPARK-32691) Update commons-crypto to v1.1.0

[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32691: -- Summary: Update commons-crypto to v1.1.0 (was: Test org.apache.spark.DistributedSuite failed

[jira] [Resolved] (SPARK-32691) Test org.apache.spark.DistributedSuite failed on arm64 jenkins

[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32691. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30275

[jira] [Created] (SPARK-33403) DSv2 SHOW TABLES doesn't show `default`

Maxim Gekk created SPARK-33403: -- Summary: DSv2 SHOW TABLES doesn't show `default` Key: SPARK-33403 URL: https://issues.apache.org/jira/browse/SPARK-33403 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-33402) SparkHadoopWriter to pass job UUID down in spark.sql.sources.writeJobUUID

[ https://issues.apache.org/jira/browse/SPARK-33402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228788#comment-17228788 ] Steve Loughran commented on SPARK-33402: The good news, this surfaced in a test I have, now that

[jira] [Created] (SPARK-33402) SparkHadoopWriter to pass job UUID down in spark.sql.sources.writeJobUUID

Steve Loughran created SPARK-33402: -- Summary: SparkHadoopWriter to pass job UUID down in spark.sql.sources.writeJobUUID Key: SPARK-33402 URL: https://issues.apache.org/jira/browse/SPARK-33402

[jira] [Commented] (SPARK-33230) FileOutputWriter jobs have duplicate JobIDs if launched in same second

[ https://issues.apache.org/jira/browse/SPARK-33230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228783#comment-17228783 ] Steve Loughran commented on SPARK-33230: Related issue: classic FileOutputCommitter has the same

[jira] [Updated] (SPARK-33329) Pluggable API to fetch shuffle merger locations with Push based shuffle

[ https://issues.apache.org/jira/browse/SPARK-33329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Shen updated SPARK-33329: - Parent: SPARK-33235 Issue Type: Sub-task (was: New Feature) > Pluggable API to fetch shuffle

[jira] [Updated] (SPARK-33329) Pluggable API to fetch shuffle merger locations with Push based shuffle

[ https://issues.apache.org/jira/browse/SPARK-33329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Shen updated SPARK-33329: - Parent: (was: SPARK-30602) Issue Type: New Feature (was: Sub-task) > Pluggable API to

[jira] [Commented] (SPARK-31911) Using S3A staging committer, pending uploads are committed more than once and listed incorrectly in _SUCCESS data

[ https://issues.apache.org/jira/browse/SPARK-31911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228775#comment-17228775 ] Steve Loughran commented on SPARK-31911: yeah, its the uuid. see SPARK-33230 for the problem: if

[jira] [Commented] (SPARK-33383) Improve performance of Column.isin Expression

[ https://issues.apache.org/jira/browse/SPARK-33383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228743#comment-17228743 ] L. C. Hsieh commented on SPARK-33383: - You can show up the query plan difference between 1 and

[jira] [Assigned] (SPARK-32915) RPC implementation to support pushing and merging shuffle blocks

[ https://issues.apache.org/jira/browse/SPARK-32915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-32915: --- Assignee: Min Shen > RPC implementation to support pushing and merging

[jira] [Created] (SPARK-33401) Vector type column is not possible to create using spark SQL

Pavlo Borshchenko created SPARK-33401: - Summary: Vector type column is not possible to create using spark SQL Key: SPARK-33401 URL: https://issues.apache.org/jira/browse/SPARK-33401 Project:

[jira] [Resolved] (SPARK-32916) Add support for external shuffle service in YARN deployment mode to leverage push-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-32916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-32916. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-32916) Add support for external shuffle service in YARN deployment mode to leverage push-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-32916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-32916: --- Assignee: Chandni Singh > Add support for external shuffle service in YARN

[jira] [Updated] (SPARK-30010) Remove deprecated SparkConf.setAll(Traversable)

[ https://issues.apache.org/jira/browse/SPARK-30010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30010: - Docs Text: The deprecated method SparkConf.setAll(Traversable) has been removed in Spark 3.1.

[jira] [Commented] (SPARK-30010) Remove deprecated SparkConf.setAll(Traversable)

[ https://issues.apache.org/jira/browse/SPARK-30010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228698#comment-17228698 ] Sean R. Owen commented on SPARK-30010: -- Right, both are in 3.0.x, but the old one is in 2.x only,

[jira] [Commented] (SPARK-30010) Remove deprecated SparkConf.setAll(Traversable)

[ https://issues.apache.org/jira/browse/SPARK-30010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228694#comment-17228694 ] Steve Loughran commented on SPARK-30010: This is a public API and breaks applications which use

[jira] [Assigned] (SPARK-33400) Reduce unneeded sorts between two SortMergeJoins

[ https://issues.apache.org/jira/browse/SPARK-33400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33400: Assignee: Apache Spark > Reduce unneeded sorts between two SortMergeJoins >

[jira] [Commented] (SPARK-33400) Reduce unneeded sorts between two SortMergeJoins

[ https://issues.apache.org/jira/browse/SPARK-33400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228680#comment-17228680 ] Apache Spark commented on SPARK-33400: -- User 'prakharjain09' has created a pull request for this

[jira] [Assigned] (SPARK-33400) Reduce unneeded sorts between two SortMergeJoins

[ https://issues.apache.org/jira/browse/SPARK-33400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33400: Assignee: (was: Apache Spark) > Reduce unneeded sorts between two SortMergeJoins >

[jira] [Created] (SPARK-33400) Reduce unneeded sorts between two SortMergeJoins

Prakhar Jain created SPARK-33400: Summary: Reduce unneeded sorts between two SortMergeJoins Key: SPARK-33400 URL: https://issues.apache.org/jira/browse/SPARK-33400 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-33396) Spark SQL CLI not print application id in processing file mode

[ https://issues.apache.org/jira/browse/SPARK-33396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33396: Assignee: (was: Apache Spark) > Spark SQL CLI not print application id in processing

[jira] [Assigned] (SPARK-33396) Spark SQL CLI not print application id in processing file mode

[ https://issues.apache.org/jira/browse/SPARK-33396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33396: Assignee: Apache Spark > Spark SQL CLI not print application id in processing file mode

[jira] [Commented] (SPARK-33396) Spark SQL CLI not print application id in processing file mode

[ https://issues.apache.org/jira/browse/SPARK-33396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228666#comment-17228666 ] Apache Spark commented on SPARK-33396: -- User 'artiship' has created a pull request for this issue:

[jira] [Commented] (SPARK-33399) Reduce unneeded exchanges after SortMerge joins

[ https://issues.apache.org/jira/browse/SPARK-33399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228664#comment-17228664 ] Apache Spark commented on SPARK-33399: -- User 'prakharjain09' has created a pull request for this

[jira] [Commented] (SPARK-33399) Reduce unneeded exchanges after SortMerge joins

[ https://issues.apache.org/jira/browse/SPARK-33399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228663#comment-17228663 ] Apache Spark commented on SPARK-33399: -- User 'prakharjain09' has created a pull request for this

[jira] [Assigned] (SPARK-33399) Reduce unneeded exchanges after SortMerge joins

[ https://issues.apache.org/jira/browse/SPARK-33399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33399: Assignee: (was: Apache Spark) > Reduce unneeded exchanges after SortMerge joins >

[jira] [Assigned] (SPARK-33399) Reduce unneeded exchanges after SortMerge joins

[ https://issues.apache.org/jira/browse/SPARK-33399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33399: Assignee: Apache Spark > Reduce unneeded exchanges after SortMerge joins >

  1   2   >