[jira] [Resolved] (SPARK-37137) Inline type hints for python/pyspark/conf.py

[ https://issues.apache.org/jira/browse/SPARK-37137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37137. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34411 [https://gi

[jira] [Assigned] (SPARK-37137) Inline type hints for python/pyspark/conf.py

[ https://issues.apache.org/jira/browse/SPARK-37137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37137: Assignee: Byron Hsu > Inline type hints for python/pyspark/conf.py >

[jira] [Assigned] (SPARK-37211) More descriptions and adding an image to the failure message about enabling GitHub Actions

[ https://issues.apache.org/jira/browse/SPARK-37211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37211: Assignee: (was: Apache Spark) > More descriptions and adding an image to the failure

[jira] [Assigned] (SPARK-37211) More descriptions and adding an image to the failure message about enabling GitHub Actions

[ https://issues.apache.org/jira/browse/SPARK-37211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37211: Assignee: Apache Spark > More descriptions and adding an image to the failure message abo

[jira] [Commented] (SPARK-37211) More descriptions and adding an image to the failure message about enabling GitHub Actions

[ https://issues.apache.org/jira/browse/SPARK-37211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17439042#comment-17439042 ] Apache Spark commented on SPARK-37211: -- User 'yutoacts' has created a pull request

[jira] [Resolved] (SPARK-37157) Inline type hints for python/pyspark/util.py

[ https://issues.apache.org/jira/browse/SPARK-37157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37157. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34438 [https://gi

[jira] [Assigned] (SPARK-37157) Inline type hints for python/pyspark/util.py

[ https://issues.apache.org/jira/browse/SPARK-37157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37157: Assignee: dch nguyen > Inline type hints for python/pyspark/util.py > ---

[jira] [Updated] (SPARK-37210) An error occurred while concurrently writing to different static partitions

[ https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhen Wang updated SPARK-37210: -- Description: An error occurred while concurrently writing to different static partitions. For writing

[jira] [Created] (SPARK-37211) More descriptions and adding an image to the failure message about enabling GitHub Actions

Yuto Akutsu created SPARK-37211: --- Summary: More descriptions and adding an image to the failure message about enabling GitHub Actions Key: SPARK-37211 URL: https://issues.apache.org/jira/browse/SPARK-37211

[jira] [Commented] (SPARK-37210) An error occurred while concurrently writing to different static partitions

[ https://issues.apache.org/jira/browse/SPARK-37210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17439025#comment-17439025 ] Zhen Wang commented on SPARK-37210: --- The test code can be executed normally in spark 2

[jira] [Created] (SPARK-37210) An error occurred while concurrently writing to different static partitions

Zhen Wang created SPARK-37210: - Summary: An error occurred while concurrently writing to different static partitions Key: SPARK-37210 URL: https://issues.apache.org/jira/browse/SPARK-37210 Project: Spark

[jira] [Commented] (SPARK-35496) Upgrade Scala 2.13 to 2.13.7

[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438986#comment-17438986 ] Kousuke Saruta commented on SPARK-35496: [~dongjoon] Thank you for letting me kn

[jira] [Commented] (SPARK-35496) Upgrade Scala 2.13 to 2.13.7

[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438984#comment-17438984 ] Dongjoon Hyun commented on SPARK-35496: --- Hi, [~sarutak]. [~LuciferYang] already up

[jira] [Commented] (SPARK-36895) Add Create Index syntax support

[ https://issues.apache.org/jira/browse/SPARK-36895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438920#comment-17438920 ] Apache Spark commented on SPARK-36895: -- User 'huaxingao' has created a pull request

[jira] [Commented] (SPARK-37208) Support mapping Spark gpu/fpga resource types to custom YARN resource type

[ https://issues.apache.org/jira/browse/SPARK-37208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438880#comment-17438880 ] Apache Spark commented on SPARK-37208: -- User 'tgravescs' has created a pull request

[jira] [Assigned] (SPARK-37208) Support mapping Spark gpu/fpga resource types to custom YARN resource type

[ https://issues.apache.org/jira/browse/SPARK-37208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37208: Assignee: Apache Spark > Support mapping Spark gpu/fpga resource types to custom YARN res

[jira] [Assigned] (SPARK-37208) Support mapping Spark gpu/fpga resource types to custom YARN resource type

[ https://issues.apache.org/jira/browse/SPARK-37208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37208: Assignee: (was: Apache Spark) > Support mapping Spark gpu/fpga resource types to cust

[jira] [Comment Edited] (SPARK-31726) Make spark.files available in driver with cluster deploy mode on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-31726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438862#comment-17438862 ] Martin Andersson edited comment on SPARK-31726 at 11/4/21, 6:10 PM: --

[jira] [Commented] (SPARK-31726) Make spark.files available in driver with cluster deploy mode on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-31726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438862#comment-17438862 ] Martin Andersson commented on SPARK-31726: -- I've also run into this issue. Tryi

[jira] [Updated] (SPARK-37209) YarnShuffleIntegrationSuite and other two similar cases in `resource-managers` test failed

[ https://issues.apache.org/jira/browse/SPARK-37209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-37209: - Attachment: success-unit-tests.log > YarnShuffleIntegrationSuite and other two similar cases in > `res

[jira] [Updated] (SPARK-37209) YarnShuffleIntegrationSuite and other two similar cases in `resource-managers` test failed

[ https://issues.apache.org/jira/browse/SPARK-37209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-37209: - Attachment: failed-unit-tests.log > YarnShuffleIntegrationSuite and other two similar cases in > `reso

[jira] [Updated] (SPARK-37209) YarnShuffleIntegrationSuite and other two similar cases in `resource-managers` test failed

[ https://issues.apache.org/jira/browse/SPARK-37209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-37209: - Description: Execute : # build/mvn clean package -DskipTests -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -P

[jira] [Created] (SPARK-37209) YarnShuffleIntegrationSuite and other two similar cases in `resource-managers` test failed

Yang Jie created SPARK-37209: Summary: YarnShuffleIntegrationSuite and other two similar cases in `resource-managers` test failed Key: SPARK-37209 URL: https://issues.apache.org/jira/browse/SPARK-37209 P

[jira] [Comment Edited] (SPARK-37198) pyspark.pandas read_csv() and to_csv() should handle local files

[ https://issues.apache.org/jira/browse/SPARK-37198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438789#comment-17438789 ] Chuck Connell edited comment on SPARK-37198 at 11/4/21, 3:42 PM: -

[jira] [Commented] (SPARK-37198) pyspark.pandas read_csv() and to_csv() should handle local files

[ https://issues.apache.org/jira/browse/SPARK-37198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438789#comment-17438789 ] Chuck Connell commented on SPARK-37198: --- There are many hints/techtips on the Inte

[jira] [Updated] (SPARK-37203) Fix NotSerializableException when observe with TypedImperativeAggregate

[ https://issues.apache.org/jira/browse/SPARK-37203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37203: --- Summary: Fix NotSerializableException when observe with TypedImperativeAggregate (was: Fix NotSeria

[jira] [Commented] (SPARK-37208) Support mapping Spark gpu/fpga resource types to custom YARN resource type

[ https://issues.apache.org/jira/browse/SPARK-37208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438784#comment-17438784 ] Thomas Graves commented on SPARK-37208: --- Note, I'm working on this. > Support map

[jira] [Created] (SPARK-37208) Support mapping Spark gpu/fpga resource types to custom YARN resource type

Thomas Graves created SPARK-37208: - Summary: Support mapping Spark gpu/fpga resource types to custom YARN resource type Key: SPARK-37208 URL: https://issues.apache.org/jira/browse/SPARK-37208 Project:

[jira] [Assigned] (SPARK-37038) Sample push down in DS v2

[ https://issues.apache.org/jira/browse/SPARK-37038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37038: --- Assignee: Huaxin Gao > Sample push down in DS v2 > - > >

[jira] [Resolved] (SPARK-37038) Sample push down in DS v2

[ https://issues.apache.org/jira/browse/SPARK-37038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37038. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34451 [https://gith

[jira] [Commented] (SPARK-37207) Python API does not have isEmpty

[ https://issues.apache.org/jira/browse/SPARK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438611#comment-17438611 ] Apache Spark commented on SPARK-37207: -- User 'dhirennavani' has created a pull requ

[jira] [Commented] (SPARK-37207) Python API does not have isEmpty

[ https://issues.apache.org/jira/browse/SPARK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438610#comment-17438610 ] Apache Spark commented on SPARK-37207: -- User 'dhirennavani' has created a pull requ

[jira] [Commented] (SPARK-37207) Python API does not have isEmpty

[ https://issues.apache.org/jira/browse/SPARK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438563#comment-17438563 ] Apache Spark commented on SPARK-37207: -- User 'dhirennavani' has created a pull requ

[jira] [Commented] (SPARK-37207) Python API does not have isEmpty

[ https://issues.apache.org/jira/browse/SPARK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438562#comment-17438562 ] Apache Spark commented on SPARK-37207: -- User 'dhirennavani' has created a pull requ

[jira] [Assigned] (SPARK-37207) Python API does not have isEmpty

[ https://issues.apache.org/jira/browse/SPARK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37207: Assignee: (was: Apache Spark) > Python API does not have isEmpty > --

[jira] [Assigned] (SPARK-37207) Python API does not have isEmpty

[ https://issues.apache.org/jira/browse/SPARK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37207: Assignee: Apache Spark > Python API does not have isEmpty > -

[jira] [Created] (SPARK-37207) Python API does not have isEmpty

Dhiren Navani created SPARK-37207: - Summary: Python API does not have isEmpty Key: SPARK-37207 URL: https://issues.apache.org/jira/browse/SPARK-37207 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-35149) I am facing this issue regularly, how to fix this issue.

[ https://issues.apache.org/jira/browse/SPARK-35149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438538#comment-17438538 ] freedom1993 feng commented on SPARK-35149: -- how to solve this problem > I am f

[jira] [Commented] (SPARK-35496) Upgrade Scala 2.13 to 2.13.7

[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438535#comment-17438535 ] Kousuke Saruta commented on SPARK-35496: [~LuciferYang] Scala 2.13.7 was release

[jira] [Resolved] (SPARK-37180) PySpark.pandas should support __version__

[ https://issues.apache.org/jira/browse/SPARK-37180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37180. -- Resolution: Won't Fix > PySpark.pandas should support __version__ > --

[jira] [Commented] (SPARK-37180) PySpark.pandas should support __version__

[ https://issues.apache.org/jira/browse/SPARK-37180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438526#comment-17438526 ] Hyukjin Kwon commented on SPARK-37180: -- Yeah I think we don't need this. > PySpark

[jira] [Commented] (SPARK-37180) PySpark.pandas should support __version__

[ https://issues.apache.org/jira/browse/SPARK-37180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438522#comment-17438522 ] dch nguyen commented on SPARK-37180: As Koalas was merged into Pyspark, so Should py

[jira] [Comment Edited] (SPARK-37180) PySpark.pandas should support __version__

[ https://issues.apache.org/jira/browse/SPARK-37180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438522#comment-17438522 ] dch nguyen edited comment on SPARK-37180 at 11/4/21, 7:31 AM: