[
https://issues.apache.org/jira/browse/SPARK-45002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759829#comment-17759829
]
Anish Shrigondekar commented on SPARK-45002:
PR here: [https://github.com/ap
Yang Jie created SPARK-45003:
Summary: Refine docstring of `asc/desc`
Key: SPARK-45003
URL: https://issues.apache.org/jira/browse/SPARK-45003
Project: Spark
Issue Type: Sub-task
Compone
Anish Shrigondekar created SPARK-45002:
--
Summary: Avoid uncaught exception from state store maintenance
task thread on error
Key: SPARK-45002
URL: https://issues.apache.org/jira/browse/SPARK-45002
[
https://issues.apache.org/jira/browse/SPARK-44996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-44996.
---
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42710
[https://
[
https://issues.apache.org/jira/browse/SPARK-44996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-44996:
-
Assignee: Dongjoon Hyun
> VolcanoFeatureStep should not create `DefaultVolcanoClient` i
[
https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-44999:
-
Summary: Refactor ExternalSorter to reduce checks on shouldPartition when
calling getPartition (was: Re
[
https://issues.apache.org/jira/browse/SPARK-44900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759821#comment-17759821
]
Yauheni Audzeichyk commented on SPARK-44900:
[~yxzhang] looks like it is jus
[
https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-44999:
-
Summary: Refactor ExternalSorter#getPartition to reduce checks on
shouldPartition (was: Refactor `Exter
[
https://issues.apache.org/jira/browse/SPARK-44900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759818#comment-17759818
]
Yuexin Zhang commented on SPARK-44900:
--
Hi [~varun2807] [~yaud] did you check the
Hyukjin Kwon created SPARK-45001:
Summary: Implement DataFrame.foreachPartition
Key: SPARK-45001
URL: https://issues.apache.org/jira/browse/SPARK-45001
Project: Spark
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/SPARK-41279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759803#comment-17759803
]
Hyukjin Kwon commented on SPARK-41279:
--
See also https://github.com/apache/spark/pu
[
https://issues.apache.org/jira/browse/SPARK-45000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759802#comment-17759802
]
Snoot.io commented on SPARK-45000:
--
User 'HyukjinKwon' has created a pull request for t
Hyukjin Kwon created SPARK-45000:
Summary: Implement DataFrame.foreach
Key: SPARK-45000
URL: https://issues.apache.org/jira/browse/SPARK-45000
Project: Spark
Issue Type: Sub-task
Co
[
https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-44999:
-
Description:
{code:java}
private def getPartition(key: K): Int = {
if (shouldPartition) partitione
[
https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759799#comment-17759799
]
Snoot.io commented on SPARK-44999:
--
User 'LuciferYang' has created a pull request for t
[
https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759798#comment-17759798
]
Snoot.io commented on SPARK-44999:
--
User 'LuciferYang' has created a pull request for t
[
https://issues.apache.org/jira/browse/SPARK-44999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-44999:
-
Description:
{code:java}
private def getPartition(key: K): Int = {
if (shouldPartition) partitione
Yang Jie created SPARK-44999:
Summary: Refactor `ExternalSorter#getPartition` to reduce the
number of i`f else` judgments
Key: SPARK-44999
URL: https://issues.apache.org/jira/browse/SPARK-44999
Project: S
[
https://issues.apache.org/jira/browse/SPARK-44997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759794#comment-17759794
]
Snoot.io commented on SPARK-44997:
--
User 'panbingkun' has created a pull request for th
[
https://issues.apache.org/jira/browse/SPARK-41279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759789#comment-17759789
]
Hyukjin Kwon commented on SPARK-41279:
--
You can run
{code}
def wrapped(itr):
f
[
https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie reassigned SPARK-43646:
Assignee: Yang Jie
> Make `connect` module daily test pass
>
[
https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie resolved SPARK-43646.
--
Fix Version/s: 3.5.0
4.0.0
Resolution: Fixed
Issue resolved by pull request
[
https://issues.apache.org/jira/browse/SPARK-44998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-44998:
--
Description:
I found a lot of retry parsing inprogress event log records in history server
log. The a
[
https://issues.apache.org/jira/browse/SPARK-44998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-44998:
--
Attachment: image-2023-08-29-10-47-08-027.png
> No need to retry parsing event log path again when Fil
[
https://issues.apache.org/jira/browse/SPARK-44998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhen Wang updated SPARK-44998:
--
Attachment: image-2023-08-29-10-47-43-567.png
> No need to retry parsing event log path again when Fil
Zhen Wang created SPARK-44998:
-
Summary: No need to retry parsing event log path again when
FileNotFoundException occurs
Key: SPARK-44998
URL: https://issues.apache.org/jira/browse/SPARK-44998
Project: Sp
[
https://issues.apache.org/jira/browse/SPARK-44860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kent Yao resolved SPARK-44860.
--
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42549
[https://github.com
[
https://issues.apache.org/jira/browse/SPARK-44965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruifeng Zheng reassigned SPARK-44965:
-
Assignee: Ruifeng Zheng
> Hide internal functions/variables from `pyspark.sql.functions
[
https://issues.apache.org/jira/browse/SPARK-44965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruifeng Zheng resolved SPARK-44965.
---
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42680
[https://
BingKun Pan created SPARK-44997:
---
Summary: Align example order (Python -> Scala/Java -> R) in all
Spark Doc Content
Key: SPARK-44997
URL: https://issues.apache.org/jira/browse/SPARK-44997
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-44995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-44995:
-
Assignee: Dongjoon Hyun
> Promote SparkKubernetesClientFactory to DeveloperApi
> --
[
https://issues.apache.org/jira/browse/SPARK-44995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-44995.
---
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42709
[https://
[
https://issues.apache.org/jira/browse/SPARK-44993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-44993:
-
Assignee: Dongjoon Hyun
> Add ShuffleChecksumUtils.compareChecksums by reusing
> Shuff
[
https://issues.apache.org/jira/browse/SPARK-44993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-44993.
---
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42707
[https://
[
https://issues.apache.org/jira/browse/SPARK-44996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-44996:
--
Description:
Since `VolcanoFeatureStep` creates `DefaultVolcanoClient` always, the unit test
[
https://issues.apache.org/jira/browse/SPARK-44996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-44996:
--
Description:
Since `VolcanoFeatureStep` creates `DefaultVolcanoClient` always, the unit test
Dongjoon Hyun created SPARK-44996:
-
Summary: VolcanoFeatureStep should not create
`DefaultVolcanoClient` if not needed
Key: SPARK-44996
URL: https://issues.apache.org/jira/browse/SPARK-44996
Project:
Dongjoon Hyun created SPARK-44995:
-
Summary: Promote SparkKubernetesClientFactory to DeveloperApi
Key: SPARK-44995
URL: https://issues.apache.org/jira/browse/SPARK-44995
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-44994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-44994:
-
Summary: Refine docstring of `DataFrame.filter` (was: Refine docstring for
`DataFrame.filter`)
[
https://issues.apache.org/jira/browse/SPARK-44993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-44993:
--
Summary: Add ShuffleChecksumUtils.compareChecksums by reusing
ShuffleChecksumTestHelp.compareC
Allison Wang created SPARK-44994:
Summary: Refine the docstring of `DataFrame.filter`
Key: SPARK-44994
URL: https://issues.apache.org/jira/browse/SPARK-44994
Project: Spark
Issue Type: Sub-ta
[
https://issues.apache.org/jira/browse/SPARK-44994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-44994:
-
Summary: Refine docstring for `DataFrame.filter` (was: Refine the
docstring of `DataFrame.filte
[
https://issues.apache.org/jira/browse/SPARK-44993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-44993:
--
Summary: Move compareChecksums from ShuffleChecksumTestHelpe to
ShuffleChecksumUtils (was: Mo
Dongjoon Hyun created SPARK-44993:
-
Summary: Move compareChecksums from ShuffleChecksumTestHelpe to
ShuffleChecksumUtils and move compareChecksums
Key: SPARK-44993
URL: https://issues.apache.org/jira/browse/SPARK-
[
https://issues.apache.org/jira/browse/SPARK-41279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759732#comment-17759732
]
Johannes Alberti commented on SPARK-41279:
--
[~gurwls223] thank you for your res
Holden Karau created SPARK-44992:
Summary: Add support for rack information from an environment
variable
Key: SPARK-44992
URL: https://issues.apache.org/jira/browse/SPARK-44992
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-44900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759703#comment-17759703
]
Varun Nalla commented on SPARK-44900:
-
[~yao] hope you got a chance to look into wha
[
https://issues.apache.org/jira/browse/SPARK-44991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nirav patel updated SPARK-44991:
Summary: Spark json schema inference and fromJson api having inconsistent
behavior (was: Spark js
[
https://issues.apache.org/jira/browse/SPARK-44991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nirav patel updated SPARK-44991:
Description:
Spark json reader can infer datatype of a fields. I am ingesting millions of
datapoi
nirav patel created SPARK-44991:
---
Summary: Spark json datasource reader and fromJson api having
inconsistent behavior
Key: SPARK-44991
URL: https://issues.apache.org/jira/browse/SPARK-44991
Project: Spa
[
https://issues.apache.org/jira/browse/SPARK-44989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-44989:
-
Assignee: Dongjoon Hyun
> Add a directional message to promote JIRA_ACCESS_TOKEN
>
[
https://issues.apache.org/jira/browse/SPARK-44989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-44989.
---
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42704
[https://
[
https://issues.apache.org/jira/browse/SPARK-44832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Herman van Hövell resolved SPARK-44832.
---
Fix Version/s: 3.5.0
Assignee: Herman van Hövell
Resolution: Fixed
>
Dongjoon Hyun created SPARK-44989:
-
Summary: Add a directional message to promote JIRA_ACCESS_TOKEN
Key: SPARK-44989
URL: https://issues.apache.org/jira/browse/SPARK-44989
Project: Spark
Issu
[
https://issues.apache.org/jira/browse/SPARK-44985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-44985.
---
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42699
[https://
[
https://issues.apache.org/jira/browse/SPARK-44985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-44985:
-
Assignee: Kent Yao
> Use toString instead of stacktrace for task reaper threadDump
> --
[
https://issues.apache.org/jira/browse/SPARK-44972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-44972:
-
Assignee: Kent Yao
> Eagerly check if the token is valid to align with the behavior of
[
https://issues.apache.org/jira/browse/SPARK-44972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-44972.
---
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42625
[https://
Flavio Odas created SPARK-44988:
---
Summary: Parquet INT64 (TIMESTAMP(NANOS,false)) throwing Illegal
Parquet type
Key: SPARK-44988
URL: https://issues.apache.org/jira/browse/SPARK-44988
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-44805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759615#comment-17759615
]
Jakub Wozniak commented on SPARK-44805:
---
Hello,
Is it possible to know any ETA on
[
https://issues.apache.org/jira/browse/SPARK-44983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk resolved SPARK-44983.
--
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42632
[https://github.com
[
https://issues.apache.org/jira/browse/SPARK-44974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Herman van Hövell resolved SPARK-44974.
---
Fix Version/s: 3.5.0
Resolution: Fixed
> Replace SparkSession/Dataset/KeyValu
[
https://issues.apache.org/jira/browse/SPARK-44976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YUBI LEE updated SPARK-44976:
-
Summary: Preserve full principal user name on executor side (was:
Utils.getCurrentUserName should retur
[
https://issues.apache.org/jira/browse/SPARK-44984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruifeng Zheng reassigned SPARK-44984:
-
Assignee: Ruifeng Zheng
> Remove _get_alias from DataFrame
> --
[
https://issues.apache.org/jira/browse/SPARK-44984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruifeng Zheng resolved SPARK-44984.
---
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42698
[https://
[
https://issues.apache.org/jira/browse/SPARK-42905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17757858#comment-17757858
]
zhangzhenhao edited comment on SPARK-42905 at 8/28/23 11:04 AM:
--
[
https://issues.apache.org/jira/browse/SPARK-44987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-44987:
-
Description: Assign a name and improve the error message format.
> Assign name to the error class _LEGAC
[
https://issues.apache.org/jira/browse/SPARK-44987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-44987:
-
Reporter: Max Gekk (was: BingKun Pan)
> Assign name to the error class _LEGACY_ERROR_TEMP_1100
> --
[
https://issues.apache.org/jira/browse/SPARK-44987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk reassigned SPARK-44987:
Assignee: Max Gekk
> Assign name to the error class _LEGACY_ERROR_TEMP_1100
> ---
Max Gekk created SPARK-44987:
Summary: Assign name to the error class
_LEGACY_ERROR_TEMP_1100[1017,1073,1074,1076,1125,1126]
Key: SPARK-44987
URL: https://issues.apache.org/jira/browse/SPARK-44987
Project
[
https://issues.apache.org/jira/browse/SPARK-44987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-44987:
-
Summary: Assign name to the error class _LEGACY_ERROR_TEMP_1100 (was:
Assign name to the error class
_
[
https://issues.apache.org/jira/browse/SPARK-44983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759530#comment-17759530
]
Hudson commented on SPARK-44983:
User 'MaxGekk' has created a pull request for this issu
[
https://issues.apache.org/jira/browse/SPARK-44986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759520#comment-17759520
]
ASF GitHub Bot commented on SPARK-44986:
User 'panbingkun' has created a pull re
[
https://issues.apache.org/jira/browse/SPARK-44986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BingKun Pan updated SPARK-44986:
Attachment: image-2023-08-28-16-47-11-582.png
> There should be a gap at the bottom of the HTML
>
[
https://issues.apache.org/jira/browse/SPARK-44986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BingKun Pan updated SPARK-44986:
Description:
Before:
!image-2023-08-28-16-47-11-582.png|width=794,height=392!
After:
!image-
[
https://issues.apache.org/jira/browse/SPARK-44986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BingKun Pan updated SPARK-44986:
Attachment: image-2023-08-28-16-46-04-705.png
> There should be a gap at the bottom of the HTML
>
BingKun Pan created SPARK-44986:
---
Summary: There should be a gap at the bottom of the HTML
Key: SPARK-44986
URL: https://issues.apache.org/jira/browse/SPARK-44986
Project: Spark
Issue Type: Sub
[
https://issues.apache.org/jira/browse/SPARK-44982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-44982.
--
Fix Version/s: 3.5.0
4.0.0
Resolution: Fixed
Issue resolved by pull
[
https://issues.apache.org/jira/browse/SPARK-44982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-44982:
Assignee: Hyukjin Kwon
> Mark Spark Connect configurations as static configuration
>
[
https://issues.apache.org/jira/browse/SPARK-44981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-44981.
--
Fix Version/s: 3.5.0
4.0.0
Resolution: Fixed
Issue resolved by pull
[
https://issues.apache.org/jira/browse/SPARK-44981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-44981:
Assignee: Hyukjin Kwon
> Filter out static configurations used in local mode
> --
Kent Yao created SPARK-44985:
Summary: Use toString instead of stacktrace for task reaper
threadDump
Key: SPARK-44985
URL: https://issues.apache.org/jira/browse/SPARK-44985
Project: Spark
Issue
Ruifeng Zheng created SPARK-44984:
-
Summary: Remove _get_alias from DataFrame
Key: SPARK-44984
URL: https://issues.apache.org/jira/browse/SPARK-44984
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-44819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759468#comment-17759468
]
BingKun Pan edited comment on SPARK-44819 at 8/28/23 7:36 AM:
Max Gekk created SPARK-44983:
Summary: Convert binary to string by to_char for the formats: hex,
base64, utf-8
Key: SPARK-44983
URL: https://issues.apache.org/jira/browse/SPARK-44983
Project: Spark
85 matches
Mail list logo