[
https://issues.apache.org/jira/browse/SPARK-44670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750959#comment-17750959
]
Madhukar commented on SPARK-44670:
--
Raised a PR for using openpyxl instead of xlrd -
[
[
https://issues.apache.org/jira/browse/SPARK-44670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhukar updated SPARK-44670:
-
Description:
With python3.7 and openpyxl installed got error:
=
[
https://issues.apache.org/jira/browse/SPARK-44582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-44582.
--
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42206
[https://gi
[
https://issues.apache.org/jira/browse/SPARK-44582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-44582:
Assignee: Wan Kun
> JVM crash caused by SMJ and WindowExec
>
Hyukjin Kwon created SPARK-44671:
Summary: Retry ExecutePlan in case initial request didn't reach
server in Python client
Key: SPARK-44671
URL: https://issues.apache.org/jira/browse/SPARK-44671
Projec
[
https://issues.apache.org/jira/browse/SPARK-44009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-44009:
-
Summary: Support profiler for Python UDTFs (was: Support memory_profiler
for UDTFs )
> Suppor
[
https://issues.apache.org/jira/browse/SPARK-44663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-44663:
-
Summary: Disable arrow optimization by default for Python UDTFs (was:
Disable arrow optimizatio
[
https://issues.apache.org/jira/browse/SPARK-44670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhukar updated SPARK-44670:
-
Description:
With python3.7 and openpyxl installed got error:
=
[
https://issues.apache.org/jira/browse/SPARK-44670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhukar updated SPARK-44670:
-
Affects Version/s: 3.4.1
(was: 3.4.0)
> Fix the `test_to_excel` tests for pyt
Madhukar created SPARK-44670:
Summary: Fix the `test_to_excel` tests for python3.7
Key: SPARK-44670
URL: https://issues.apache.org/jira/browse/SPARK-44670
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-44670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhukar updated SPARK-44670:
-
Description: (was: So far, we've been skipping the `read_excel` test in
pandas API on Spark:
https:/
[
https://issues.apache.org/jira/browse/SPARK-44668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jia Fan closed SPARK-44668.
---
> ObjectMapper are threadsafe, we can reuse it in Object
> -
[
https://issues.apache.org/jira/browse/SPARK-44668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jia Fan resolved SPARK-44668.
-
Resolution: Invalid
> ObjectMapper are threadsafe, we can reuse it in Object
> -
Cheng Pan created SPARK-44669:
-
Summary: Parquet/ORC files written using Hive Serde should has
file extension
Key: SPARK-44669
URL: https://issues.apache.org/jira/browse/SPARK-44669
Project: Spark
Jia Fan created SPARK-44668:
---
Summary: ObjectMapper are threadsafe, we can reuse it in Object
Key: SPARK-44668
URL: https://issues.apache.org/jira/browse/SPARK-44668
Project: Spark
Issue Type: Impr
Ruifeng Zheng created SPARK-44667:
-
Summary: Uninstall large ML libraries for non-ML jobs
Key: SPARK-44667
URL: https://issues.apache.org/jira/browse/SPARK-44667
Project: Spark
Issue Type: Su
Ruifeng Zheng created SPARK-44666:
-
Summary: Uninstall CodeQL/Go/Node in non-container jobs
Key: SPARK-44666
URL: https://issues.apache.org/jira/browse/SPARK-44666
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-44653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-44653.
-
Fix Version/s: 3.3.3
3.5.0
3.4.2
Resolution: Fixed
[
https://issues.apache.org/jira/browse/SPARK-44653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-44653:
---
Assignee: Wenchen Fan
> non-trivial DataFrame unions should not break caching
> ---
[
https://issues.apache.org/jira/browse/SPARK-44624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-44624:
Assignee: Juliusz Sompolski
> Spark Connect reattachable Execute when initial ExecutePlan
[
https://issues.apache.org/jira/browse/SPARK-44624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-44624.
--
Fix Version/s: 3.5.0
4.0.0
Resolution: Fixed
Issue resolved by pull
[
https://issues.apache.org/jira/browse/SPARK-44664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-44664.
--
Fix Version/s: 3.5.0
4.0.0
Assignee: Hyukjin Kwon
Resolution
[
https://issues.apache.org/jira/browse/SPARK-44619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruifeng Zheng resolved SPARK-44619.
---
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42253
[https://
[
https://issues.apache.org/jira/browse/SPARK-43562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-43562:
Assignee: Haejoon Lee
> Enable DataFrameTests.test_append for pandas 2.0.0.
> ---
[
https://issues.apache.org/jira/browse/SPARK-43870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-43870.
--
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42268
[https://gi
[
https://issues.apache.org/jira/browse/SPARK-43870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-43870:
Assignee: Haejoon Lee
> Enable SeriesTests for pandas 2.0.0.
> --
[
https://issues.apache.org/jira/browse/SPARK-43562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-43562.
--
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42268
[https://gi
[
https://issues.apache.org/jira/browse/SPARK-43873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-43873.
--
Fix Version/s: 4.0.0
Assignee: Haejoon Lee
Resolution: Fixed
Fixed in https://
[
https://issues.apache.org/jira/browse/SPARK-44640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-44640.
--
Fix Version/s: 4.0.0
Assignee: Allison Wang
Resolution: Fixed
Fixed in https:/
[
https://issues.apache.org/jira/browse/SPARK-44548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amanda Liu updated SPARK-44548:
---
Summary: Add support for pandas-on-Spark DataFrame assertDataFrameEqual
(was: Add support for panda
Amanda Liu created SPARK-44665:
--
Summary: Add support for pandas DataFrame assertDataFrameEqual
Key: SPARK-44665
URL: https://issues.apache.org/jira/browse/SPARK-44665
Project: Spark
Issue Type:
Hyukjin Kwon created SPARK-44664:
Summary: Release the execute when closing the iterator in Python
client
Key: SPARK-44664
URL: https://issues.apache.org/jira/browse/SPARK-44664
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-44642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-44642:
Assignee: Juliusz Sompolski
> ExecutePlanResponseReattachableIterator should release all
[
https://issues.apache.org/jira/browse/SPARK-44642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-44642.
--
Fix Version/s: 3.5.0
4.0.0
Resolution: Fixed
Issue resolved by pull
[
https://issues.apache.org/jira/browse/SPARK-44652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-44652.
--
Fix Version/s: 3.5.0
4.0.0
Resolution: Fixed
Issue resolved by pull
[
https://issues.apache.org/jira/browse/SPARK-44652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-44652:
Assignee: Amanda Liu
> Raise error when only one df is None
> ---
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
Allison Wang created SPARK-44663:
Summary: Disable arrow optimization by default
Key: SPARK-44663
URL: https://issues.apache.org/jira/browse/SPARK-44663
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-44661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-44661:
--
Priority: Minor (was: Major)
> getMapOutputLocation should not throw NPE
> --
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-44661.
---
Fix Version/s: 3.5.0
4.0.0
3.4.2
Resolution: Fix
[
https://issues.apache.org/jira/browse/SPARK-44662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Asif updated SPARK-44662:
-
Description:
h2. *Q1. What are you trying to do? Articulate your objectives using absolutely
no jargon.*
On th
[
https://issues.apache.org/jira/browse/SPARK-44661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-44661:
-
Assignee: Dongjoon Hyun
> getMapOutputLocation should not throw NPE
> -
[
https://issues.apache.org/jira/browse/SPARK-44646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750895#comment-17750895
]
L. C. Hsieh commented on SPARK-44646:
-
I have not used it but maybe you can try
htt
Asif created SPARK-44662:
Summary: SPIP: Improving performance of BroadcastHashJoin queries
with stream side join key on non partition columns
Key: SPARK-44662
URL: https://issues.apache.org/jira/browse/SPARK-44662
[
https://issues.apache.org/jira/browse/SPARK-44646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750891#comment-17750891
]
Yu Tian commented on SPARK-44646:
-
Hi [~viirya]
Could you please check this thread? It
[
https://issues.apache.org/jira/browse/SPARK-44658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-44658:
-
Assignee: Dongjoon Hyun
> ShuffleStatus.getMapStatus should return None instead of Some
[
https://issues.apache.org/jira/browse/SPARK-44658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-44658.
---
Fix Version/s: 3.5.0
4.0.0
Resolution: Fixed
Issue resolved by pul
[
https://issues.apache.org/jira/browse/SPARK-44660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750881#comment-17750881
]
Chao Sun commented on SPARK-44660:
--
In fact the check is necessary, but it seems
{code
[
https://issues.apache.org/jira/browse/SPARK-44641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-44641:
--
Affects Version/s: 3.4.0
> SPJ: Results duplicated when SPJ partial-cluster and pushdown enabl
[
https://issues.apache.org/jira/browse/SPARK-44641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-44641:
-
Priority: Blocker (was: Major)
> SPJ: Results duplicated when SPJ partial-cluster and pushdown enabled
Dongjoon Hyun created SPARK-44661:
-
Summary: getMapOutputLocation should not throw NPE
Key: SPARK-44661
URL: https://issues.apache.org/jira/browse/SPARK-44661
Project: Spark
Issue Type: Test
Chao Sun created SPARK-44660:
Summary: Relax constraint for columnar shuffle check in AQE
Key: SPARK-44660
URL: https://issues.apache.org/jira/browse/SPARK-44660
Project: Spark
Issue Type: Improv
[
https://issues.apache.org/jira/browse/SPARK-44658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-44658:
--
Summary: ShuffleStatus.getMapStatus should return None instead of
Some(null) (was: ShuffleSta
[
https://issues.apache.org/jira/browse/SPARK-44659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-44659:
-
Summary: SPJ: Include keyGroupedPartitioning in StoragePartitionJoinParams
equality check (was: Include
[
https://issues.apache.org/jira/browse/SPARK-44641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-44641:
-
Summary: SPJ: Results duplicated when SPJ partial-cluster and pushdown
enabled but conditions unmet (wa
Chao Sun created SPARK-44659:
Summary: Include keyGroupedPartitioning in
StoragePartitionJoinParams equality check
Key: SPARK-44659
URL: https://issues.apache.org/jira/browse/SPARK-44659
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-44641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun updated SPARK-44641:
-
Parent: SPARK-37375
Issue Type: Sub-task (was: Bug)
> Results duplicated when SPJ partial-clust
Dongjoon Hyun created SPARK-44658:
-
Summary: ShuffleStatus.getMapStatus should return None
Key: SPARK-44658
URL: https://issues.apache.org/jira/browse/SPARK-44658
Project: Spark
Issue Type: B
[
https://issues.apache.org/jira/browse/SPARK-43496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750791#comment-17750791
]
Laurenceau Julien commented on SPARK-43496:
---
Hi,
I would like to suggest to g
[
https://issues.apache.org/jira/browse/SPARK-44654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750743#comment-17750743
]
7mming7 commented on SPARK-44654:
-
[~yumwang] This is also possible, but if it is the ca
[
https://issues.apache.org/jira/browse/SPARK-44654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750739#comment-17750739
]
Yuming Wang commented on SPARK-44654:
-
Another way is convert join to filter if maxi
Venkata Sai Akhil Gudesa created SPARK-44657:
Summary: Incorrect limit handling and config parsing in Arrow
collect
Key: SPARK-44657
URL: https://issues.apache.org/jira/browse/SPARK-44657
[
https://issues.apache.org/jira/browse/SPARK-44656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Juliusz Sompolski updated SPARK-44656:
--
Epic Link: SPARK-43754
> Close dangling iterators in SparkResult too (Spark Connect Sc
Alice Sayutina created SPARK-44656:
--
Summary: Close dangling iterators in SparkResult too (Spark
Connect Scala)
Key: SPARK-44656
URL: https://issues.apache.org/jira/browse/SPARK-44656
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-44619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruifeng Zheng updated SPARK-44619:
--
Summary: Free up disk space for container jobs (was: Free up disk space
for pyspark container
Wenchen Fan created SPARK-44655:
---
Summary: make the code cleaner about static and dynamc
data/partition filters
Key: SPARK-44655
URL: https://issues.apache.org/jira/browse/SPARK-44655
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-44654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
7mming7 updated SPARK-44654:
Description:
The following SQL cannot perform partition pruning
{code:java}
SELECT * FROM parquet_part WHE
[
https://issues.apache.org/jira/browse/SPARK-40927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750660#comment-17750660
]
Iain Morrison commented on SPARK-40927:
---
In our case I found the following setting
[
https://issues.apache.org/jira/browse/SPARK-44654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
7mming7 updated SPARK-44654:
Attachment: image-2023-08-03-17-22-53-981.png
> In subquery cannot perform partition pruning
> ---
7mming7 created SPARK-44654:
---
Summary: In subquery cannot perform partition pruning
Key: SPARK-44654
URL: https://issues.apache.org/jira/browse/SPARK-44654
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-44575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750659#comment-17750659
]
ASF GitHub Bot commented on SPARK-44575:
User 'heyihong' has created a pull requ
[
https://issues.apache.org/jira/browse/SPARK-44619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750657#comment-17750657
]
ASF GitHub Bot commented on SPARK-44619:
User 'zhengruifeng' has created a pull
[
https://issues.apache.org/jira/browse/SPARK-44581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750656#comment-17750656
]
ASF GitHub Bot commented on SPARK-44581:
User 'liangyu-1' has created a pull req
[
https://issues.apache.org/jira/browse/SPARK-44649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750652#comment-17750652
]
ASF GitHub Bot commented on SPARK-44649:
User 'beliefer' has created a pull requ
[
https://issues.apache.org/jira/browse/SPARK-44649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750653#comment-17750653
]
ASF GitHub Bot commented on SPARK-44649:
User 'beliefer' has created a pull requ
87 matches
Mail list logo