[jira] [Comment Edited] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

2022-12-12 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646477#comment-17646477 ] Mridul Muralidharan edited comment on SPARK-41497 at 12/13/22 7:41 AM:

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

2022-12-12 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646477#comment-17646477 ] Mridul Muralidharan commented on SPARK-41497: - Agree - there appears to be a bunch of

[jira] [Commented] (SPARK-38217) insert overwrite failed for external table with dynamic partition table

2022-12-12 Thread Jiayi Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646464#comment-17646464 ] Jiayi Liu commented on SPARK-38217: --- This is because spark deletes the overwrite partition, but hive

[jira] [Created] (SPARK-41505) Assign name to _LEGACY_ERROR_TEMP_1179

2022-12-12 Thread Yang Jie (Jira)
Yang Jie created SPARK-41505: Summary: Assign name to _LEGACY_ERROR_TEMP_1179 Key: SPARK-41505 URL: https://issues.apache.org/jira/browse/SPARK-41505 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-41481) Reuse `INVALID_TYPED_LITERAL` instead of `_LEGACY_ERROR_TEMP_0020`

2022-12-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41481. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39025

[jira] [Assigned] (SPARK-41481) Reuse `INVALID_TYPED_LITERAL` instead of `_LEGACY_ERROR_TEMP_0020`

2022-12-12 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41481: Assignee: Yang Jie > Reuse `INVALID_TYPED_LITERAL` instead of `_LEGACY_ERROR_TEMP_0020` >

[jira] [Resolved] (SPARK-33782) Place spark.files, spark.jars and spark.files under the current working directory on the driver in K8S cluster mode

2022-12-12 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-33782. -- Fix Version/s: 3.4.0 Assignee: Pralabh Kumar Resolution: Fixed > Place

[jira] [Resolved] (SPARK-41499) Upgrade protobuf version to 3.21.11

2022-12-12 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41499. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39042

[jira] [Resolved] (SPARK-41504) Update R version to 4.1.2 in Dockerfile comment

2022-12-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41504. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39044

[jira] [Assigned] (SPARK-41504) Update R version to 4.1.2 in Dockerfile comment

2022-12-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41504: - Assignee: Dongjoon Hyun > Update R version to 4.1.2 in Dockerfile comment >

[jira] [Commented] (SPARK-41504) Update R version to 4.1.2 in Dockerfile comment

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646410#comment-17646410 ] Apache Spark commented on SPARK-41504: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-41504) Update R version to 4.1.2 in Dockerfile comment

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41504: Assignee: (was: Apache Spark) > Update R version to 4.1.2 in Dockerfile comment >

[jira] [Assigned] (SPARK-41504) Update R version to 4.1.2 in Dockerfile comment

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41504: Assignee: Apache Spark > Update R version to 4.1.2 in Dockerfile comment >

[jira] [Commented] (SPARK-41504) Update R version to 4.1.2 in Dockerfile comment

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646409#comment-17646409 ] Apache Spark commented on SPARK-41504: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-41504) Update R version to 4.1.2 in Dockerfile comment

2022-12-12 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-41504: - Summary: Update R version to 4.1.2 in Dockerfile comment Key: SPARK-41504 URL: https://issues.apache.org/jira/browse/SPARK-41504 Project: Spark Issue

[jira] [Updated] (SPARK-41504) Update R version to 4.1.2 in Dockerfile comment

2022-12-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41504: -- Priority: Minor (was: Major) > Update R version to 4.1.2 in Dockerfile comment >

[jira] [Resolved] (SPARK-41502) Upgrade the minimum Minikube version to 1.28.0

2022-12-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41502. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39043

[jira] [Created] (SPARK-41503) Implement Partition Transformation Functions

2022-12-12 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41503: - Summary: Implement Partition Transformation Functions Key: SPARK-41503 URL: https://issues.apache.org/jira/browse/SPARK-41503 Project: Spark Issue Type:

[jira] [Closed] (SPARK-41501) auto generate concat as Double when string minus an INTERVAL type

2022-12-12 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng closed SPARK-41501. -- Network issues lead to duplicate creation > auto generate concat as Double when string minus an INTERVAL

[jira] [Resolved] (SPARK-41501) auto generate concat as Double when string minus an INTERVAL type

2022-12-12 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JacobZheng resolved SPARK-41501. Resolution: Invalid > auto generate concat as Double when string minus an INTERVAL type >

[jira] [Assigned] (SPARK-41502) Upgrade the minimum Minikube version to 1.28.0

2022-12-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41502: - Assignee: Dongjoon Hyun > Upgrade the minimum Minikube version to 1.28.0 >

[jira] [Commented] (SPARK-41502) Upgrade the minimum Minikube version to 1.28.0

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646397#comment-17646397 ] Apache Spark commented on SPARK-41502: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-41502) Upgrade the minimum Minikube version to 1.28.0

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41502: Assignee: Apache Spark > Upgrade the minimum Minikube version to 1.28.0 >

[jira] [Assigned] (SPARK-41502) Upgrade the minimum Minikube version to 1.28.0

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41502: Assignee: (was: Apache Spark) > Upgrade the minimum Minikube version to 1.28.0 >

[jira] [Created] (SPARK-41502) Upgrade the minimum Minikube version to 1.28.0

2022-12-12 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-41502: - Summary: Upgrade the minimum Minikube version to 1.28.0 Key: SPARK-41502 URL: https://issues.apache.org/jira/browse/SPARK-41502 Project: Spark Issue Type:

[jira] [Created] (SPARK-41501) auto generate concat as Double when string minus an INTERVAL type

2022-12-12 Thread JacobZheng (Jira)
JacobZheng created SPARK-41501: -- Summary: auto generate concat as Double when string minus an INTERVAL type Key: SPARK-41501 URL: https://issues.apache.org/jira/browse/SPARK-41501 Project: Spark

[jira] [Created] (SPARK-41500) auto generate concat as Double when string minus an INTERVAL type

2022-12-12 Thread JacobZheng (Jira)
JacobZheng created SPARK-41500: -- Summary: auto generate concat as Double when string minus an INTERVAL type Key: SPARK-41500 URL: https://issues.apache.org/jira/browse/SPARK-41500 Project: Spark

[jira] [Assigned] (SPARK-41499) Upgrade protobuf version to 3.21.11

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41499: Assignee: Apache Spark (was: Gengliang Wang) > Upgrade protobuf version to 3.21.11 >

[jira] [Commented] (SPARK-41499) Upgrade protobuf version to 3.21.11

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646386#comment-17646386 ] Apache Spark commented on SPARK-41499: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-41499) Upgrade protobuf version to 3.21.11

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41499: Assignee: Gengliang Wang (was: Apache Spark) > Upgrade protobuf version to 3.21.11 >

[jira] [Created] (SPARK-41499) Upgrade protobuf version to 3.21.11

2022-12-12 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-41499: -- Summary: Upgrade protobuf version to 3.21.11 Key: SPARK-41499 URL: https://issues.apache.org/jira/browse/SPARK-41499 Project: Spark Issue Type: Task

[jira] [Assigned] (SPARK-41495) Implement `collection` functions: P~Z

2022-12-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41495: Assignee: Ruifeng Zheng > Implement `collection` functions: P~Z >

[jira] [Resolved] (SPARK-41495) Implement `collection` functions: P~Z

2022-12-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41495. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39033

[jira] [Commented] (SPARK-27561) Support "lateral column alias references" to allow column aliases to be used within SELECT clauses

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646368#comment-17646368 ] Apache Spark commented on SPARK-27561: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Updated] (SPARK-41498) Union does not propagate Metadata output

2022-12-12 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-41498: Fix Version/s: (was: 3.4.0) > Union does not propagate Metadata output >

[jira] [Commented] (SPARK-27561) Support "lateral column alias references" to allow column aliases to be used within SELECT clauses

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646367#comment-17646367 ] Apache Spark commented on SPARK-27561: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40776) Add documentation (similar to Avro functions).

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40776: Assignee: Apache Spark > Add documentation (similar to Avro functions). >

[jira] [Assigned] (SPARK-40776) Add documentation (similar to Avro functions).

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40776: Assignee: (was: Apache Spark) > Add documentation (similar to Avro functions). >

[jira] [Commented] (SPARK-40776) Add documentation (similar to Avro functions).

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646358#comment-17646358 ] Apache Spark commented on SPARK-40776: -- User 'SandishKumarHN' has created a pull request for this

[jira] [Resolved] (SPARK-41360) Avoid BlockManager re-registration if the executor has been lost

2022-12-12 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-41360. - Fix Version/s: 3.2.4 3.3.2 3.4.0

[jira] [Assigned] (SPARK-41360) Avoid BlockManager re-registration if the executor has been lost

2022-12-12 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-41360: --- Assignee: wuyi > Avoid BlockManager re-registration if the executor has

[jira] [Commented] (SPARK-41378) Support Column Stats in DS V2

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646295#comment-17646295 ] Apache Spark commented on SPARK-41378: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-41378) Support Column Stats in DS V2

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646296#comment-17646296 ] Apache Spark commented on SPARK-41378: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Resolved] (SPARK-41491) Update postgres 15.1

2022-12-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41491. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39031

[jira] [Assigned] (SPARK-41491) Update postgres 15.1

2022-12-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41491: - Assignee: William Hyun > Update postgres 15.1 > > >

[jira] [Updated] (SPARK-41491) Update postgres docker image to 15.1

2022-12-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41491: -- Summary: Update postgres docker image to 15.1 (was: Update postgres 15.1) > Update postgres

[jira] [Commented] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

2022-12-12 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646097#comment-17646097 ] wuyi commented on SPARK-41497: -- [~mridulm80] [~tgraves] [~attilapiros] [~ivoson] any good ideas? >

[jira] [Updated] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

2022-12-12 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-41497: - Description: Accumulator could be undercounted when the retried task has rdd cache.  See the example below and

[jira] [Commented] (SPARK-41214) SQL metrics are missing from Spark UI when AQE for Cached DataFrame is enabled

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646094#comment-17646094 ] Apache Spark commented on SPARK-41214: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-41214) SQL metrics are missing from Spark UI when AQE for Cached DataFrame is enabled

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646093#comment-17646093 ] Apache Spark commented on SPARK-41214: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-41461) protoc-3.21.9-linux-x86_64.exe requires GLIBC_2.14

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646086#comment-17646086 ] Apache Spark commented on SPARK-41461: -- User 'WolverineJiang' has created a pull request for this

[jira] [Assigned] (SPARK-41498) Union does not propagate Metadata output

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41498: Assignee: Apache Spark > Union does not propagate Metadata output >

[jira] [Assigned] (SPARK-41498) Union does not propagate Metadata output

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41498: Assignee: (was: Apache Spark) > Union does not propagate Metadata output >

[jira] [Commented] (SPARK-41498) Union does not propagate Metadata output

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646070#comment-17646070 ] Apache Spark commented on SPARK-41498: -- User 'fred-db' has created a pull request for this issue:

[jira] [Created] (SPARK-41498) Union does not propagate Metadata output

2022-12-12 Thread Jira
Fredrik Klauß created SPARK-41498: - Summary: Union does not propagate Metadata output Key: SPARK-41498 URL: https://issues.apache.org/jira/browse/SPARK-41498 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-41485) Unify the environment variable of *_PROTOC_EXEC_PATH

2022-12-12 Thread Haonan Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Jiang updated SPARK-41485: - Description: At present, there are 3 similar environment variable of *_PROTOC_EXEC_PATH, but

[jira] [Updated] (SPARK-41485) Unify the environment variable of *_PROTOC_EXEC_PATH

2022-12-12 Thread Haonan Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Jiang updated SPARK-41485: - Description: At present, there are 3 similar environment variable of *_PROTOC_EXEC_PATH, but

[jira] [Updated] (SPARK-41485) Unify the environment variable of *_PROTOC_EXEC_PATH

2022-12-12 Thread Haonan Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Jiang updated SPARK-41485: - Description: At present, there are 3 similar ones, but they use the same pb version. If they

[jira] [Created] (SPARK-41497) Accumulator undercounting in the case of retry task with rdd cache

2022-12-12 Thread wuyi (Jira)
wuyi created SPARK-41497: Summary: Accumulator undercounting in the case of retry task with rdd cache Key: SPARK-41497 URL: https://issues.apache.org/jira/browse/SPARK-41497 Project: Spark Issue

[jira] [Updated] (SPARK-41448) Make consistent MR job IDs in FileBatchWriter and FileFormatWriter

2022-12-12 Thread Hui An (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui An updated SPARK-41448: --- Description: [SPARK-26873|https://issues.apache.org/jira/browse/SPARK-26873] fix the consistent issue for

[jira] (SPARK-41438) Implement DataFrame. colRegex

2022-12-12 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41438 ] jiaan.geng deleted comment on SPARK-41438: was (Author: beliefer): I'm working on. > Implement DataFrame. colRegex > - > > Key: SPARK-41438 >

[jira] [Commented] (SPARK-41438) Implement DataFrame. colRegex

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646045#comment-17646045 ] Apache Spark commented on SPARK-41438: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41438) Implement DataFrame. colRegex

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41438: Assignee: (was: Apache Spark) > Implement DataFrame. colRegex >

[jira] [Commented] (SPARK-41438) Implement DataFrame. colRegex

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646044#comment-17646044 ] Apache Spark commented on SPARK-41438: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41438) Implement DataFrame. colRegex

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41438: Assignee: Apache Spark > Implement DataFrame. colRegex > - >

[jira] [Commented] (SPARK-40171) Fix flaky tests in ImageFileFormatSuite

2022-12-12 Thread Ahmed Mahran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646027#comment-17646027 ] Ahmed Mahran commented on SPARK-40171: -- What's common in these 3 tests and different from the rest:

[jira] [Assigned] (SPARK-41448) Make consistent MR job IDs in FileBatchWriter and FileFormatWriter

2022-12-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41448: --- Assignee: Hui An > Make consistent MR job IDs in FileBatchWriter and FileFormatWriter >

[jira] [Resolved] (SPARK-41448) Make consistent MR job IDs in FileBatchWriter and FileFormatWriter

2022-12-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41448. - Fix Version/s: 3.3.2 3.2.3 3.4.0 Resolution: Fixed

[jira] [Assigned] (SPARK-40775) V2 file scans have duplicative descriptions

2022-12-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-40775: --- Assignee: Adam Binford > V2 file scans have duplicative descriptions >

[jira] [Resolved] (SPARK-40775) V2 file scans have duplicative descriptions

2022-12-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-40775. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38229

[jira] [Resolved] (SPARK-41468) Fix PlanExpression handling in EquivalentExpressions

2022-12-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41468. - Fix Version/s: 3.3.2 3.4.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-41468) Fix PlanExpression handling in EquivalentExpressions

2022-12-12 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41468: --- Assignee: Peter Toth > Fix PlanExpression handling in EquivalentExpressions >

[jira] [Updated] (SPARK-41496) Spark doesn't support multiple documents in one pod template file

2022-12-12 Thread Yanzhe Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanzhe Xu updated SPARK-41496: -- Description: When defining multiple documents in the pod template file: {code:java} apiVersion: v1

[jira] [Updated] (SPARK-41496) Spark doesn't support multiple documents in one pod template file

2022-12-12 Thread Yanzhe Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanzhe Xu updated SPARK-41496: -- Description: When defining multiple documents in the pod template file: {code:java} apiVersion: v1

[jira] [Updated] (SPARK-41496) Spark doesn't support multiple documents in one pod template file

2022-12-12 Thread Yanzhe Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanzhe Xu updated SPARK-41496: -- Description: When defining multiple documents in the pod template file: {code:java} apiVersion: v1

[jira] [Updated] (SPARK-41496) Spark doesn't support multiple documents in one pod template file

2022-12-12 Thread Yanzhe Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanzhe Xu updated SPARK-41496: -- Description: When defining multiple documents in the pod template file: ```yaml apiVersion: v1 kind:

[jira] [Created] (SPARK-41496) Spark doesn't support multiple documents in one pod template file

2022-12-12 Thread Yanzhe Xu (Jira)
Yanzhe Xu created SPARK-41496: - Summary: Spark doesn't support multiple documents in one pod template file Key: SPARK-41496 URL: https://issues.apache.org/jira/browse/SPARK-41496 Project: Spark

[jira] [Resolved] (SPARK-41492) implement MISC function

2022-12-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41492. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39032

[jira] [Assigned] (SPARK-41492) implement MISC function

2022-12-12 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41492: - Assignee: Ruifeng Zheng > implement MISC function > --- > >

[jira] [Commented] (SPARK-41412) Implement `Cast`

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645978#comment-17645978 ] Apache Spark commented on SPARK-41412: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-41412) Implement `Cast`

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645977#comment-17645977 ] Apache Spark commented on SPARK-41412: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-41495) Implement `collection` functions: P~Z

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41495: Assignee: Apache Spark > Implement `collection` functions: P~Z >

[jira] [Commented] (SPARK-41495) Implement `collection` functions: P~Z

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645975#comment-17645975 ] Apache Spark commented on SPARK-41495: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41495) Implement `collection` functions: P~Z

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41495: Assignee: (was: Apache Spark) > Implement `collection` functions: P~Z >

[jira] [Commented] (SPARK-41495) Implement `collection` functions: P~Z

2022-12-12 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645974#comment-17645974 ] Apache Spark commented on SPARK-41495: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-41495) Implement `collection` functions: P~Z

2022-12-12 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41495: - Summary: Implement `collection` functions: P~Z Key: SPARK-41495 URL: https://issues.apache.org/jira/browse/SPARK-41495 Project: Spark Issue Type: Sub-task