[jira] [Commented] (SPARK-33782) Place spark.files, spark.jars and spark.files under the current working directory on the driver in K8S cluster mode

2022-10-31 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17626417#comment-17626417 ] pralabhkumar commented on SPARK-33782: -- [~hyukjin.kwon] [~dongjoon]  Please let me know, if this

[jira] [Commented] (SPARK-33782) Place spark.files, spark.jars and spark.files under the current working directory on the driver in K8S cluster mode

2022-10-11 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615997#comment-17615997 ] pralabhkumar commented on SPARK-33782: -- [~hyukjin.kwon]    Can u please help to review the PR .

[jira] [Commented] (SPARK-33782) Place spark.files, spark.jars and spark.files under the current working directory on the driver in K8S cluster mode

2022-09-13 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603426#comment-17603426 ] pralabhkumar commented on SPARK-33782: -- [~dongjoon] Please review the PR .  > Place spark.files,

[jira] [Commented] (SPARK-39965) Skip PVC cleanup when driver doesn't own PVCs

2022-08-08 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576793#comment-17576793 ] pralabhkumar commented on SPARK-39965: -- [~dongjoon] Thx for taking this .  This is really helpful

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

2022-08-07 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576516#comment-17576516 ] pralabhkumar commented on SPARK-39965: -- [~dongjoon]  Thx for replying. We don't see an issue

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

2022-08-04 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575554#comment-17575554 ] pralabhkumar commented on SPARK-39965: -- [~dongjoon] Please review. > Spark on K8s delete pvc even

[jira] [Updated] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

2022-08-02 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39965: - Description: >From Spark32 . as a part of [https://github.com/apache/spark/pull/32288] ,

[jira] [Updated] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

2022-08-02 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39965: - Description: >From Spark32 . as a part of [https://github.com/apache/spark/pull/32288] ,

[jira] [Commented] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

2022-08-02 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574494#comment-17574494 ] pralabhkumar commented on SPARK-39965: -- Gentle ping @[dongjoon-hyun

[jira] [Updated] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

2022-08-02 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39965: - Description: >From Spark32 . as a part of [https://github.com/apache/spark/pull/32288] ,

[jira] [Updated] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

2022-08-02 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39965: - Component/s: (was: Spark Core) > Spark on K8s delete pvc even though it's not being used. >

[jira] [Created] (SPARK-39965) Spark on K8s delete pvc even though it's not being used.

2022-08-02 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-39965: Summary: Spark on K8s delete pvc even though it's not being used. Key: SPARK-39965 URL: https://issues.apache.org/jira/browse/SPARK-39965 Project: Spark

[jira] [Commented] (SPARK-33782) Place spark.files, spark.jars and spark.files under the current working directory on the driver in K8S cluster mode

2022-08-02 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17574314#comment-17574314 ] pralabhkumar commented on SPARK-33782: -- [~hyukjin.kwon] I would like to work on this . Please let

[jira] [Comment Edited] (SPARK-39375) SPIP: Spark Connect - A client and server interface for Apache Spark.

2022-07-27 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571978#comment-17571978 ] pralabhkumar edited comment on SPARK-39375 at 7/27/22 2:51 PM: --- This is

[jira] [Commented] (SPARK-39375) SPIP: Spark Connect - A client and server interface for Apache Spark.

2022-07-27 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571978#comment-17571978 ] pralabhkumar commented on SPARK-39375: -- This is really good proposal and need of an hour

[jira] [Commented] (SPARK-39755) SPARK_LOCAL_DIRS locations are not randomized in K8s

2022-07-13 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566430#comment-17566430 ] pralabhkumar commented on SPARK-39755: -- [~hyukjin.kwon]  , please let me know if the above

[jira] [Commented] (SPARK-39755) SPARK_LOCAL_DIRS locations are not randomized in K8s

2022-07-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566164#comment-17566164 ] pralabhkumar commented on SPARK-39755: -- Problem seen on yarn side and the fix was

[jira] [Updated] (SPARK-39755) SPARK_LOCAL_DIRS locations are not randomized in K8s

2022-07-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39755: - Description: In org.apache.spark.util  getConfiguredLocalDirs     {code:java} if

[jira] [Updated] (SPARK-39755) SPARK_LOCAL_DIRS locations are not randomized in K8s

2022-07-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39755: - Description: In org.apache.spark.util  getConfiguredLocalDirs     {code:java} if

[jira] [Commented] (SPARK-39755) SPARK_LOCAL_DIRS locations are not randomized in K8s

2022-07-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566142#comment-17566142 ] pralabhkumar commented on SPARK-39755: -- [~dongjoon] Gentle ping .  > SPARK_LOCAL_DIRS locations

[jira] [Updated] (SPARK-39755) SPARK_LOCAL_DIRS locations are not randomized in K8s

2022-07-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39755: - Summary: SPARK_LOCAL_DIRS locations are not randomized in K8s (was: Spark-shuffle locations

[jira] [Updated] (SPARK-39755) Spark-shuffle locations are not randomized in K8s

2022-07-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39755: - Summary: Spark-shuffle locations are not randomized in K8s (was: Spark-shuffle locations are

[jira] [Commented] (SPARK-39755) Spark-shuffle locations are not randomized in K8s

2022-07-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565935#comment-17565935 ] pralabhkumar commented on SPARK-39755: -- [~hyukjin.kwon]  Please comment on the same.  >

[jira] [Updated] (SPARK-39755) Spark-shuffle locations are not randomized in K8s

2022-07-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39755: - Description: In org.apache.spark.util  getConfiguredLocalDirs     {code:java} if

[jira] [Created] (SPARK-39755) Spark-shuffle locations are not randomized in K8s

2022-07-12 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-39755: Summary: Spark-shuffle locations are not randomized in K8s Key: SPARK-39755 URL: https://issues.apache.org/jira/browse/SPARK-39755 Project: Spark Issue

[jira] [Commented] (SPARK-38292) Support `na_filter` for pyspark.pandas.read_csv

2022-06-23 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557987#comment-17557987 ] pralabhkumar commented on SPARK-38292: -- [~hyukjin.kwon] Please let me know if its ok . I'll do the

[jira] [Commented] (SPARK-38292) Support `na_filter` for pyspark.pandas.read_csv

2022-06-22 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557537#comment-17557537 ] pralabhkumar commented on SPARK-38292: -- [~hyukjin.kwon] Thx for the suggestion .    After going

[jira] [Commented] (SPARK-38292) Support `na_filter` for pyspark.pandas.read_csv

2022-06-21 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556851#comment-17556851 ] pralabhkumar commented on SPARK-38292: -- [~itholic] [~hyukjin.kwon]  Would like to discuss the

[jira] [Commented] (SPARK-39399) proxy-user support not working for Spark on k8s in cluster deploy mode

2022-06-20 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556302#comment-17556302 ] pralabhkumar commented on SPARK-39399: -- Gentle ping [~hyukjin.kwon]   [~dongjoon]  > proxy-user

[jira] [Commented] (SPARK-39399) proxy-user support not working for Spark on k8s in cluster deploy mode

2022-06-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17554904#comment-17554904 ] pralabhkumar commented on SPARK-39399: -- ping [~hyukjin.kwon]  , please help us on the same or

[jira] [Commented] (SPARK-38292) Support `na_filter` for pyspark.pandas.read_csv

2022-06-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17554551#comment-17554551 ] pralabhkumar commented on SPARK-38292: -- [~itholic] I would like to work on this .  > Support

[jira] [Created] (SPARK-39179) Improve the test coverage for pyspark/shuffle.py

2022-05-13 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-39179: Summary: Improve the test coverage for pyspark/shuffle.py Key: SPARK-39179 URL: https://issues.apache.org/jira/browse/SPARK-39179 Project: Spark Issue Type:

[jira] [Commented] (SPARK-39179) Improve the test coverage for pyspark/shuffle.py

2022-05-13 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17536555#comment-17536555 ] pralabhkumar commented on SPARK-39179: -- I am working on this .  > Improve the test coverage for

[jira] [Updated] (SPARK-39179) Improve the test coverage for pyspark/shuffle.py

2022-05-13 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39179: - Description: Improve the test coverage of shuffle.py (was: Improve the test coverage of

[jira] [Commented] (SPARK-39102) Replace the usage of guava's Files.createTempDir() with java.nio.file.Files.createTempDirectory()

2022-05-09 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533764#comment-17533764 ] pralabhkumar commented on SPARK-39102: -- Sure I'll work on this  > Replace the usage of guava's

[jira] [Commented] (SPARK-39097) Improve the test coverage for pyspark/taskcontext.py

2022-05-07 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533251#comment-17533251 ] pralabhkumar commented on SPARK-39097: -- [~hyukjin.kwon]  While analysis the unit test cases for

[jira] [Updated] (SPARK-39097) Improve the test coverage for pyspark/taskcontext.py

2022-05-07 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-39097: - Description: Improve the test coverage of taskcontext.py (was: Improve the test coverage of

[jira] [Commented] (SPARK-39102) Replace the usage of guava's Files.createTempDir() with java.nio.file.Files.createTempDirectory()

2022-05-05 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532443#comment-17532443 ] pralabhkumar commented on SPARK-39102: -- ping [~hyukjin.kwon]  > Replace the usage of guava's

[jira] [Created] (SPARK-39102) Replace the usage of guava's Files.createTempDir() with java.nio.file.Files.createTempDirectory()

2022-05-04 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-39102: Summary: Replace the usage of guava's Files.createTempDir() with java.nio.file.Files.createTempDirectory() Key: SPARK-39102 URL:

[jira] [Commented] (SPARK-38262) Upgrade Google guava to version 30.0-jre

2022-05-04 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531526#comment-17531526 ] pralabhkumar commented on SPARK-38262: -- [~bjornjorgensen]  So QQ , as part of this PR , it is not

[jira] [Created] (SPARK-39097) Improve the test coverage for pyspark/taskcontext.py

2022-05-03 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-39097: Summary: Improve the test coverage for pyspark/taskcontext.py Key: SPARK-39097 URL: https://issues.apache.org/jira/browse/SPARK-39097 Project: Spark Issue

[jira] [Commented] (SPARK-39097) Improve the test coverage for pyspark/taskcontext.py

2022-05-03 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531481#comment-17531481 ] pralabhkumar commented on SPARK-39097: -- I am working on this.  > Improve the test coverage for

[jira] [Commented] (SPARK-25355) Support --proxy-user for Spark on K8s

2022-04-29 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530083#comment-17530083 ] pralabhkumar commented on SPARK-25355: -- [~hyukjin.kwon]  Can u please us  or redirect us to someone 

[jira] [Commented] (SPARK-39029) Improve the test coverage for pyspark/broadcast.py

2022-04-26 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528269#comment-17528269 ] pralabhkumar commented on SPARK-39029: -- I am working on this .  > Improve the test coverage for

[jira] [Created] (SPARK-39029) Improve the test coverage for pyspark/broadcast.py

2022-04-26 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-39029: Summary: Improve the test coverage for pyspark/broadcast.py Key: SPARK-39029 URL: https://issues.apache.org/jira/browse/SPARK-39029 Project: Spark Issue

[jira] [Commented] (SPARK-38492) Improve the test coverage for PySpark

2022-04-17 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17523293#comment-17523293 ] pralabhkumar commented on SPARK-38492: -- on it . Thx > Improve the test coverage for PySpark >

[jira] [Created] (SPARK-38874) CLONE - Improve the test coverage for pyspark/ml module

2022-04-12 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-38874: Summary: CLONE - Improve the test coverage for pyspark/ml module Key: SPARK-38874 URL: https://issues.apache.org/jira/browse/SPARK-38874 Project: Spark

[jira] [Closed] (SPARK-38871) Improve the test coverage for PySpark/rddsampler.py

2022-04-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar closed SPARK-38871. This issue is wrongly created , hence closing it  > Improve the test coverage for

[jira] [Resolved] (SPARK-38871) Improve the test coverage for PySpark/rddsampler.py

2022-04-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar resolved SPARK-38871. -- Resolution: Invalid > Improve the test coverage for PySpark/rddsampler.py >

[jira] [Commented] (SPARK-38879) Improve the test coverage for pyspark/rddsampler.py

2022-04-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521135#comment-17521135 ] pralabhkumar commented on SPARK-38879: -- I will be working on this .  > Improve the test coverage

[jira] [Comment Edited] (SPARK-38879) Improve the test coverage for pyspark/rddsampler.py

2022-04-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521135#comment-17521135 ] pralabhkumar edited comment on SPARK-38879 at 4/12/22 1:07 PM: --- Please

[jira] [Updated] (SPARK-38879) Improve the test coverage for pyspark/rddsampler.py

2022-04-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-38879: - Description: Improve the test coverage of rddsampler.py (was: Improve the test coverage of

[jira] [Created] (SPARK-38879) Improve the test coverage for pyspark/rddsampler.py

2022-04-12 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-38879: Summary: Improve the test coverage for pyspark/rddsampler.py Key: SPARK-38879 URL: https://issues.apache.org/jira/browse/SPARK-38879 Project: Spark Issue

[jira] [Commented] (SPARK-38871) Improve the test coverage for PySpark/rddsampler.py

2022-04-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17521134#comment-17521134 ] pralabhkumar commented on SPARK-38871: -- Please close this one , wrongly cloned  > Improve the test

[jira] [Created] (SPARK-38876) CLONE - Improve the test coverage for pyspark/*.py

2022-04-12 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-38876: Summary: CLONE - Improve the test coverage for pyspark/*.py Key: SPARK-38876 URL: https://issues.apache.org/jira/browse/SPARK-38876 Project: Spark Issue

[jira] [Created] (SPARK-38872) CLONE - Improve the test coverage for pyspark/pandas module

2022-04-12 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-38872: Summary: CLONE - Improve the test coverage for pyspark/pandas module Key: SPARK-38872 URL: https://issues.apache.org/jira/browse/SPARK-38872 Project: Spark

[jira] [Created] (SPARK-38878) CLONE - Improve the test coverage for pyspark/statcounter.py

2022-04-12 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-38878: Summary: CLONE - Improve the test coverage for pyspark/statcounter.py Key: SPARK-38878 URL: https://issues.apache.org/jira/browse/SPARK-38878 Project: Spark

[jira] [Created] (SPARK-38875) CLONE - Improve the test coverage for pyspark/sql module

2022-04-12 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-38875: Summary: CLONE - Improve the test coverage for pyspark/sql module Key: SPARK-38875 URL: https://issues.apache.org/jira/browse/SPARK-38875 Project: Spark

[jira] [Created] (SPARK-38877) CLONE - Improve the test coverage for pyspark/find_spark_home.py

2022-04-12 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-38877: Summary: CLONE - Improve the test coverage for pyspark/find_spark_home.py Key: SPARK-38877 URL: https://issues.apache.org/jira/browse/SPARK-38877 Project: Spark

[jira] [Created] (SPARK-38871) Improve the test coverage for PySpark/rddsampler.py

2022-04-12 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-38871: Summary: Improve the test coverage for PySpark/rddsampler.py Key: SPARK-38871 URL: https://issues.apache.org/jira/browse/SPARK-38871 Project: Spark Issue

[jira] [Created] (SPARK-38873) CLONE - Improve the test coverage for pyspark/mllib module

2022-04-12 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-38873: Summary: CLONE - Improve the test coverage for pyspark/mllib module Key: SPARK-38873 URL: https://issues.apache.org/jira/browse/SPARK-38873 Project: Spark

[jira] [Commented] (SPARK-38854) Improve the test coverage for pyspark/statcounter.py

2022-04-11 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520374#comment-17520374 ] pralabhkumar commented on SPARK-38854: -- [~gurwls223] I would like to work on this , hence can u

[jira] [Updated] (SPARK-38854) Improve the test coverage for pyspark/statcounter.py

2022-04-11 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-38854: - Affects Version/s: 3.3.0 (was: 3.4.0) > Improve the test coverage

[jira] [Updated] (SPARK-38854) Improve the test coverage for pyspark/statcounter.py

2022-04-11 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-38854: - Description: Improve the test coverage of statcounter.py  (was: We should test when the

[jira] [Updated] (SPARK-38854) Improve the test coverage for pyspark/statcounter.py

2022-04-11 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-38854: - Fix Version/s: (was: 3.4.0) > Improve the test coverage for pyspark/statcounter.py >

[jira] [Created] (SPARK-38854) Improve the test coverage for pyspark/statcounter.py

2022-04-11 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-38854: Summary: Improve the test coverage for pyspark/statcounter.py Key: SPARK-38854 URL: https://issues.apache.org/jira/browse/SPARK-38854 Project: Spark Issue

[jira] [Commented] (SPARK-38637) pyspark.pandas.config.OptionError: "No such option: 'mode.chained_assignment'

2022-04-05 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17517393#comment-17517393 ] pralabhkumar commented on SPARK-38637: -- [~itholic] can I work on this  >

[jira] [Commented] (SPARK-24432) Add support for dynamic resource allocation

2022-01-19 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478618#comment-17478618 ] pralabhkumar commented on SPARK-24432: -- [~dongjoon]  one quick question .  - The K8s dynamic

[jira] [Commented] (SPARK-37491) Fix Series.asof when values of the series is not sorted

2022-01-10 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17472056#comment-17472056 ] pralabhkumar commented on SPARK-37491: -- Lets take example of  pser = pd.Series([2, 1, np.nan, 4],

[jira] [Commented] (SPARK-37491) Fix Series.asof when values of the series is not sorted

2022-01-09 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17471314#comment-17471314 ] pralabhkumar commented on SPARK-37491: -- I am working on it . > Fix Series.asof when values of the

[jira] [Commented] (SPARK-37491) Fix Series.asof when values of the series is not sorted

2022-01-05 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469471#comment-17469471 ] pralabhkumar commented on SPARK-37491: -- I would like to work on this . Basically the problem is in

[jira] [Comment Edited] (SPARK-37491) Fix Series.asof when values of the series is not sorted

2022-01-05 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469471#comment-17469471 ] pralabhkumar edited comment on SPARK-37491 at 1/5/22, 6:27 PM: --- I would

[jira] [Commented] (SPARK-37188) pyspark.pandas histogram accepts the title option but does not add a title to the plot

2021-11-18 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446305#comment-17446305 ] pralabhkumar commented on SPARK-37188: -- [~hyukjin.kwon]  Working on it . Thx > pyspark.pandas

[jira] [Comment Edited] (SPARK-37188) pyspark.pandas histogram accepts the title option but does not add a title to the plot

2021-11-18 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445978#comment-17445978 ] pralabhkumar edited comment on SPARK-37188 at 11/18/21, 2:56 PM: - IMHO ,

[jira] [Commented] (SPARK-37188) pyspark.pandas histogram accepts the title option but does not add a title to the plot

2021-11-18 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445978#comment-17445978 ] pralabhkumar commented on SPARK-37188: -- IMHO , the issue is in pyspark.pandas.plot plotly.py

[jira] [Comment Edited] (SPARK-37189) pyspark.pandas histogram accepts the range option but does not use it

2021-11-18 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445960#comment-17445960 ] pralabhkumar edited comment on SPARK-37189 at 11/18/21, 2:52 PM: -    

[jira] (SPARK-37189) pyspark.pandas histogram accepts the range option but does not use it

2021-11-18 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37189 ] pralabhkumar deleted comment on SPARK-37189: -- was (Author: pralabhkumar):     > pyspark.pandas histogram accepts the range option but does not use it >

[jira] [Commented] (SPARK-37189) pyspark.pandas histogram accepts the range option but does not use it

2021-11-18 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17445960#comment-17445960 ] pralabhkumar commented on SPARK-37189: -- IMHO , the issue is in pyspark.pandas.plot plotly.py

[jira] [Commented] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444256#comment-17444256 ] pralabhkumar commented on SPARK-37181: -- [~yikunkero] [~chconnell] . I'll work on this and will

[jira] [Comment Edited] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443764#comment-17443764 ] pralabhkumar edited comment on SPARK-37181 at 11/15/21, 2:10 PM: -

[jira] [Comment Edited] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443764#comment-17443764 ] pralabhkumar edited comment on SPARK-37181 at 11/15/21, 2:10 PM: -

[jira] [Commented] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443764#comment-17443764 ] pralabhkumar commented on SPARK-37181: -- However from users point of view , if user mention latin-1

[jira] [Comment Edited] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443720#comment-17443720 ] pralabhkumar edited comment on SPARK-37181 at 11/15/21, 10:32 AM: -- from

[jira] [Commented] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17443720#comment-17443720 ] pralabhkumar commented on SPARK-37181: -- from pyspark import pandas as ps latin-1 encoding is same

[jira] [Commented] (SPARK-30537) toPandas gets wrong dtypes when applied on empty DF when Arrow enabled

2021-10-21 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432812#comment-17432812 ] pralabhkumar commented on SPARK-30537: -- Thx [~hyukjin.kwon] , working on this  > toPandas gets

[jira] [Commented] (SPARK-30537) toPandas gets wrong dtypes when applied on empty DF when Arrow enabled

2021-10-21 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432805#comment-17432805 ] pralabhkumar commented on SPARK-30537: -- [~hyukjin.kwon]   I would like to work on this , please

[jira] [Commented] (SPARK-32285) Add PySpark support for nested timestamps with arrow

2021-10-20 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431140#comment-17431140 ] pralabhkumar commented on SPARK-32285: -- [~bryanc] . Please review the PR .  > Add PySpark support

[jira] [Commented] (SPARK-32161) Hide JVM traceback for SparkUpgradeException

2021-10-18 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430099#comment-17430099 ] pralabhkumar commented on SPARK-32161: -- [~hyukjin.kwon]    Since the PR is being merged , please

[jira] [Commented] (SPARK-32161) Hide JVM traceback for SparkUpgradeException

2021-10-13 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428185#comment-17428185 ] pralabhkumar commented on SPARK-32161: -- [~hyukjin.kwon]   Please let me if I can work on this .  

[jira] [Commented] (SPARK-32285) Add PySpark support for nested timestamps with arrow

2021-09-13 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414232#comment-17414232 ] pralabhkumar commented on SPARK-32285: -- [~hyukjin.kwon] just added the initial version for

[jira] [Commented] (SPARK-32285) Add PySpark support for nested timestamps with arrow

2021-09-12 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413679#comment-17413679 ] pralabhkumar commented on SPARK-32285: -- Thx , will share the PR in some time > Add PySpark support

[jira] [Commented] (SPARK-32285) Add PySpark support for nested timestamps with arrow

2021-09-11 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413482#comment-17413482 ] pralabhkumar commented on SPARK-32285: -- [~hyukjin.kwon] [~emkornfi...@gmail.com] I would like to

[jira] [Commented] (SPARK-36622) spark.history.kerberos.principal doesn't take value _HOST

2021-09-08 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17411859#comment-17411859 ] pralabhkumar commented on SPARK-36622: -- [~angerszhuuu] [~tgraves]   Please review the PR  >

[jira] [Commented] (SPARK-36622) spark.history.kerberos.principal doesn't take value _HOST

2021-09-06 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410578#comment-17410578 ] pralabhkumar commented on SPARK-36622: -- [~angerszhuuu] [~tgraves] [~hyukjin.kwon]   Have

[jira] [Updated] (SPARK-36622) spark.history.kerberos.principal doesn't take value _HOST

2021-09-06 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-36622: - Affects Version/s: 3.2.0 > spark.history.kerberos.principal doesn't take value _HOST >

[jira] [Commented] (SPARK-36622) spark.history.kerberos.principal doesn't take value _HOST

2021-09-03 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409617#comment-17409617 ] pralabhkumar commented on SPARK-36622: -- [~thejdeep]  Its better to have _HOST , its been common

[jira] [Created] (SPARK-36622) spark.history.kerberos.principal doesn't take value _HOST

2021-08-31 Thread pralabhkumar (Jira)
pralabhkumar created SPARK-36622: Summary: spark.history.kerberos.principal doesn't take value _HOST Key: SPARK-36622 URL: https://issues.apache.org/jira/browse/SPARK-36622 Project: Spark

[jira] [Commented] (SPARK-32924) Web UI sort on duration is wrong

2020-11-21 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236743#comment-17236743 ] pralabhkumar commented on SPARK-32924: -- [~rakson] [~hyukjin.kwon] Can I open PR for this ? > Web

[jira] [Commented] (SPARK-29103) CheckAnalysis for data source V2 ALTER TABLE ignores case sensitivity

2019-09-20 Thread pralabhkumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934368#comment-16934368 ] pralabhkumar commented on SPARK-29103: -- [~joseph.torres] In method  findNestedField of class

[jira] [Commented] (SPARK-25788) Elastic net penalties for GLMs

2019-07-18 Thread pralabhkumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887858#comment-16887858 ] pralabhkumar commented on SPARK-25788: -- [~shahid] I can work on this . Please let me know if its ok 

  1   2   >