exmy created SPARK-34987:
Summary: AQE improve: change shuffle hash join to sort merge join
when skewed shuffle hash join exists
Key: SPARK-34987
URL: https://issues.apache.org/jira/browse/SPARK-34987
Project
[
https://issues.apache.org/jira/browse/SPARK-34986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
angerszhu updated SPARK-34986:
--
Description:
Aggregate ordinal should judge is it contains agg function.
before this change it's
```
angerszhu created SPARK-34986:
-
Summary: Aggregate ordinal should judge is it contains agg function
Key: SPARK-34986
URL: https://issues.apache.org/jira/browse/SPARK-34986
Project: Spark
Issue Ty
[
https://issues.apache.org/jira/browse/SPARK-34985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lianjunzhi updated SPARK-34985:
---
Description:
Hive has two non-partitioned tables, trade_order and trade_order_goods.
Trade_order co
[
https://issues.apache.org/jira/browse/SPARK-34985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lianjunzhi updated SPARK-34985:
---
Description:
Hive has two non-partitioned tables, trade_order and trade_order_goods.
Trade_order co
[
https://issues.apache.org/jira/browse/SPARK-34985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lianjunzhi updated SPARK-34985:
---
Description:
Hive has two non-partitioned tables, trade_order and trade_order_goods.
Trade_order co
lianjunzhi created SPARK-34985:
--
Summary: Different execution plans under jdbc and hdfs
Key: SPARK-34985
URL: https://issues.apache.org/jira/browse/SPARK-34985
Project: Spark
Issue Type: Bug
Max Gekk created SPARK-34984:
Summary: ANSI intervals formatting in hive results
Key: SPARK-34984
URL: https://issues.apache.org/jira/browse/SPARK-34984
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-34970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang updated SPARK-34970:
---
Affects Version/s: (was: 3.2.0)
3.0.0
3.0.1
[
https://issues.apache.org/jira/browse/SPARK-34970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang updated SPARK-34970:
---
Fix Version/s: 3.0.3
3.2.0
> Redact map-type options in the output of exp
[
https://issues.apache.org/jira/browse/SPARK-34970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316839#comment-17316839
]
Apache Spark commented on SPARK-34970:
--
User 'gengliangwang' has created a pull req
[
https://issues.apache.org/jira/browse/SPARK-34970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316838#comment-17316838
]
Apache Spark commented on SPARK-34970:
--
User 'gengliangwang' has created a pull req
[
https://issues.apache.org/jira/browse/SPARK-34980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316833#comment-17316833
]
Apache Spark commented on SPARK-34980:
--
User 'ulysses-you' has created a pull reque
[
https://issues.apache.org/jira/browse/SPARK-34980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34980:
Assignee: Apache Spark
> Support coalesce partition through union
> -
[
https://issues.apache.org/jira/browse/SPARK-34980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34980:
Assignee: (was: Apache Spark)
> Support coalesce partition through union
> --
Haejoon Lee created SPARK-34983:
---
Summary: Renaming the package alias from pp to ps
Key: SPARK-34983
URL: https://issues.apache.org/jira/browse/SPARK-34983
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-34886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34886:
Assignee: Apache Spark
> Port/integrate Koalas DataFrame unit test into PySpark
> ---
[
https://issues.apache.org/jira/browse/SPARK-34886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34886:
Assignee: (was: Apache Spark)
> Port/integrate Koalas DataFrame unit test into PySpar
[
https://issues.apache.org/jira/browse/SPARK-34886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316824#comment-17316824
]
Apache Spark commented on SPARK-34886:
--
User 'xinrong-databricks' has created a pul
[
https://issues.apache.org/jira/browse/SPARK-34886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinrong Meng updated SPARK-34886:
-
Description:
This JIRA aims to port [Koalas DataFrame
test|https://github.com/databricks/koalas
[
https://issues.apache.org/jira/browse/SPARK-34886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinrong Meng updated SPARK-34886:
-
Summary: Port/integrate Koalas DataFrame unit test into PySpark (was:
Port/integrate Koalas uni
[
https://issues.apache.org/jira/browse/SPARK-34922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takeshi Yamamuro updated SPARK-34922:
-
Fix Version/s: 3.0.3
3.1.2
> Use better CBO cost function
> -
[
https://issues.apache.org/jira/browse/SPARK-34981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316787#comment-17316787
]
Apache Spark commented on SPARK-34981:
--
User 'sunchao' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-34981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34981:
Assignee: (was: Apache Spark)
> Implement V2 function resolution and evaluation
> --
[
https://issues.apache.org/jira/browse/SPARK-34981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34981:
Assignee: Apache Spark
> Implement V2 function resolution and evaluation
> -
[
https://issues.apache.org/jira/browse/SPARK-34982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kumaresh AK updated SPARK-34982:
Environment:
Tested with EMR 6.2.0. spark v3.0.2, python v3.8.5
Also Tested with local pyspark on
[
https://issues.apache.org/jira/browse/SPARK-34928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314428#comment-17314428
]
Supun De Silva edited comment on SPARK-34928 at 4/7/21, 11:02 PM:
[
https://issues.apache.org/jira/browse/SPARK-34982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kumaresh AK updated SPARK-34982:
Priority: Minor (was: Major)
> Pyspark asDict() returns wrong child field for nested dataframe
>
[
https://issues.apache.org/jira/browse/SPARK-34982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kumaresh AK updated SPARK-34982:
Summary: Pyspark asDict() returns wrong child field for nested dataframe
(was: Pyspark asDict() r
[
https://issues.apache.org/jira/browse/SPARK-34982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kumaresh AK updated SPARK-34982:
Description:
Hello! I upgraded a job to Spark 3.0.1 (from 2.4.4) and encountered this issue.
The
[
https://issues.apache.org/jira/browse/SPARK-34982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kumaresh AK updated SPARK-34982:
Attachment: SPARK-34982.py
> Pyspark asDict() returns wrong fields for a nested dataframe
> --
Kumaresh AK created SPARK-34982:
---
Summary: Pyspark asDict() returns wrong fields for a nested
dataframe
Key: SPARK-34982
URL: https://issues.apache.org/jira/browse/SPARK-34982
Project: Spark
I
[
https://issues.apache.org/jira/browse/SPARK-34780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316714#comment-17316714
]
Chao Sun commented on SPARK-34780:
--
Hi [~mikechen] (and sorry for the late reply again)
[
https://issues.apache.org/jira/browse/SPARK-34674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316697#comment-17316697
]
Sergey Kotlov commented on SPARK-34674:
---
The fix that I currently use: [https://g
[
https://issues.apache.org/jira/browse/SPARK-34674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316694#comment-17316694
]
Apache Spark commented on SPARK-34674:
--
User 'kotlovs' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-34674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316695#comment-17316695
]
Apache Spark commented on SPARK-34674:
--
User 'kotlovs' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-32634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316683#comment-17316683
]
Cheng Su commented on SPARK-32634:
--
[~Thomas Liu] - Implement fallback mechanism for wh
[
https://issues.apache.org/jira/browse/SPARK-34955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-34955.
---
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 32052
[https://
[
https://issues.apache.org/jira/browse/SPARK-34970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang resolved SPARK-34970.
Fix Version/s: 3.1.2
Resolution: Fixed
Issue resolved by pull request 32079
[https:
[
https://issues.apache.org/jira/browse/SPARK-34981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316477#comment-17316477
]
Chao Sun commented on SPARK-34981:
--
Will submit a PR soon.
> Implement V2 function res
Chao Sun created SPARK-34981:
Summary: Implement V2 function resolution and evaluation
Key: SPARK-34981
URL: https://issues.apache.org/jira/browse/SPARK-34981
Project: Spark
Issue Type: Improvem
[
https://issues.apache.org/jira/browse/SPARK-34493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk reassigned SPARK-34493:
Assignee: Haejoon Lee
> Create "TEXT Files" page for Data Source documents.
> ---
[
https://issues.apache.org/jira/browse/SPARK-34493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk resolved SPARK-34493.
--
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 32053
[https://github.com
[
https://issues.apache.org/jira/browse/SPARK-34064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316361#comment-17316361
]
Aaron Wang commented on SPARK-34064:
May I ask, when will this be resolved?
> Broad
[
https://issues.apache.org/jira/browse/SPARK-34668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-34668:
---
Assignee: Max Gekk
> Support casting of day-time intervals to strings
> ---
[
https://issues.apache.org/jira/browse/SPARK-34668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-34668.
-
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 32070
[https://gith
[
https://issues.apache.org/jira/browse/SPARK-34976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-34976:
---
Assignee: angerszhu
> Rename GroupingSet to GroupingAnalytic
>
[
https://issues.apache.org/jira/browse/SPARK-34976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-34976.
-
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 32073
[https://gith
[
https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang updated SPARK-34979:
Summary: Failed to install pyspark[sql] (due to pyarrow error) on aarch64
(was: Failed to install
[
https://issues.apache.org/jira/browse/SPARK-32634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316327#comment-17316327
]
Lietong Liu commented on SPARK-32634:
-
[~chengsu] I have a question about implementi
[
https://issues.apache.org/jira/browse/SPARK-34674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34674:
Assignee: Apache Spark
> Spark app on k8s doesn't terminate without call to sparkContext.
[
https://issues.apache.org/jira/browse/SPARK-34674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34674:
Assignee: (was: Apache Spark)
> Spark app on k8s doesn't terminate without call to sp
[
https://issues.apache.org/jira/browse/SPARK-34674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316319#comment-17316319
]
Apache Spark commented on SPARK-34674:
--
User 'KarlManong' has created a pull reques
[
https://issues.apache.org/jira/browse/SPARK-34674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316286#comment-17316286
]
KarlManong commented on SPARK-34674:
I have the same problem, and I submitted a PR:
[
https://issues.apache.org/jira/browse/SPARK-34972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-34972.
--
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 32069
[https://gi
[
https://issues.apache.org/jira/browse/SPARK-34972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-34972:
Assignee: Takuya Ueshin
> Make doctests work in Spark.
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-34970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316218#comment-17316218
]
Apache Spark commented on SPARK-34970:
--
User 'gengliangwang' has created a pull req
ulysses you created SPARK-34980:
---
Summary: Support coalesce partition through union
Key: SPARK-34980
URL: https://issues.apache.org/jira/browse/SPARK-34980
Project: Spark
Issue Type: Improvemen
[
https://issues.apache.org/jira/browse/SPARK-27658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-27658:
---
Assignee: Ryan Blue
> Catalog API to load functions
> -
>
>
[
https://issues.apache.org/jira/browse/SPARK-27658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-27658.
-
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 24559
[https://gith
[
https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang updated SPARK-34979:
Description:
~^$ pip install pyspark[sql]^~
~^Collecting pyarrow>=1.0.0^~
~^Using cached pyarrow
[
https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang updated SPARK-34979:
Description:
~^$ pip install pyspark[sql] --prefer-binary^~
~^Collecting pyarrow>=1.0.0^~
~^Usin
[
https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316131#comment-17316131
]
Yikun Jiang edited comment on SPARK-34979 at 4/7/21, 8:56 AM:
[
https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yikun Jiang updated SPARK-34979:
Description:
$ pip install pyspark[sql] --prefer-binary
Collecting pyarrow>=1.0.0
Using cached p
[
https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316135#comment-17316135
]
Yikun Jiang commented on SPARK-34979:
-
Looks like we have 2 potential options in sp
[
https://issues.apache.org/jira/browse/SPARK-34979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316131#comment-17316131
]
Yikun Jiang commented on SPARK-34979:
-
As the tmp workground, you could install nigh
Yikun Jiang created SPARK-34979:
---
Summary: Failed to install pyspark[sql] (due to pyarrow error)
Key: SPARK-34979
URL: https://issues.apache.org/jira/browse/SPARK-34979
Project: Spark
Issue Typ
[
https://issues.apache.org/jira/browse/SPARK-34975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315926#comment-17315926
]
Junqing Cai edited comment on SPARK-34975 at 4/7/21, 7:53 AM:
[
https://issues.apache.org/jira/browse/SPARK-34969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Herman van Hövell resolved SPARK-34969.
---
Fix Version/s: 3.2.0
Assignee: Ali Afroozeh
Resolution: Fixed
> Foll
[
https://issues.apache.org/jira/browse/SPARK-34975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junqing Cai updated SPARK-34975:
Summary: skip.header.line.count dose not work in hive partitioned table
(was: skip.header.line.co
[
https://issues.apache.org/jira/browse/SPARK-34975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junqing Cai updated SPARK-34975:
Description:
when I use hive2.4.0 create external hive partitioned table with csv file, and
{code
[
https://issues.apache.org/jira/browse/SPARK-34762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316078#comment-17316078
]
Apache Spark commented on SPARK-34762:
--
User 'sarutak' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-34502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316056#comment-17316056
]
Kadir Selçuk commented on SPARK-34502:
--
Apache k with Jira problems
> Remove unuse
[
https://issues.apache.org/jira/browse/SPARK-34922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316050#comment-17316050
]
Apache Spark commented on SPARK-34922:
--
User 'tanelk' has created a pull request fo
[
https://issues.apache.org/jira/browse/SPARK-33357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316049#comment-17316049
]
Apache Spark commented on SPARK-33357:
--
User 'grarkydev' has created a pull request
[
https://issues.apache.org/jira/browse/SPARK-34922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316051#comment-17316051
]
Apache Spark commented on SPARK-34922:
--
User 'tanelk' has created a pull request fo
76 matches
Mail list logo
