[jira] [Updated] (SPARK-37758) [Scala] Enable PySpark scheduled job on ARM based self-hosted runner

2021-12-28 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37758: Summary: [Scala] Enable PySpark scheduled job on ARM based self-hosted runner (was: Enable

[jira] [Created] (SPARK-37772) [PYSPARK] Publish ApacheSparkGitHubActionImage arm64 docker image

2021-12-28 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37772: --- Summary: [PYSPARK] Publish ApacheSparkGitHubActionImage arm64 docker image Key: SPARK-37772 URL: https://issues.apache.org/jira/browse/SPARK-37772 Project: Spark

[jira] [Comment Edited] (SPARK-37758) Enable PySpark scheduled job on ARM based self-hosted runner

2021-12-28 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466294#comment-17466294 ] Yikun Jiang edited comment on SPARK-37758 at 12/29/21, 3:03 AM: *

[jira] [Comment Edited] (SPARK-37758) Enable PySpark scheduled job on ARM based self-hosted runner

2021-12-28 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466294#comment-17466294 ] Yikun Jiang edited comment on SPARK-37758 at 12/29/21, 3:02 AM: *

[jira] [Comment Edited] (SPARK-37758) Enable PySpark scheduled job on ARM based self-hosted runner

2021-12-28 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466294#comment-17466294 ] Yikun Jiang edited comment on SPARK-37758 at 12/29/21, 3:02 AM: *

[jira] [Commented] (SPARK-37758) Enable PySpark scheduled job on ARM based self-hosted runner

2021-12-28 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17466294#comment-17466294 ] Yikun Jiang commented on SPARK-37758: - * DistributedSuite.caching in memory, replicated (encryption

[jira] [Updated] (SPARK-35607) Migrate Spark Arm Job from Jenkins to GitHub Actions

2021-12-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-35607: Description: There were two Arm CI jobs in AMP lab:

[jira] [Created] (SPARK-37759) Add ARM based self-hosted runner to Apache Spark

2021-12-27 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37759: --- Summary: Add ARM based self-hosted runner to Apache Spark Key: SPARK-37759 URL: https://issues.apache.org/jira/browse/SPARK-37759 Project: Spark Issue Type:

[jira] [Updated] (SPARK-37757) Enable Spark test scheduled job on ARM based self-hosted runner

2021-12-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37757: Summary: Enable Spark test scheduled job on ARM based self-hosted runner (was: Enable spark test

[jira] [Updated] (SPARK-37757) Enable spark test scheduled job on ARM based self-hosted runner

2021-12-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37757: Summary: Enable spark test scheduled job on ARM based self-hosted runner (was: Enable scala test

[jira] [Created] (SPARK-37758) Enable PySpark scheduled job on ARM based self-hosted runner

2021-12-27 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37758: --- Summary: Enable PySpark scheduled job on ARM based self-hosted runner Key: SPARK-37758 URL: https://issues.apache.org/jira/browse/SPARK-37758 Project: Spark

[jira] [Created] (SPARK-37757) Enable scala test on ARM based self-hosted runner

2021-12-27 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37757: --- Summary: Enable scala test on ARM based self-hosted runner Key: SPARK-37757 URL: https://issues.apache.org/jira/browse/SPARK-37757 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35607) Migrate Spark Arm Job from Jenkins to GitHub Actions

2021-12-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-35607: Description: There were two Arm CI jobs in AMP lab:

[jira] [Updated] (SPARK-35607) Migrate Spark Arm Job from Jenkins to GitHub Actions

2021-12-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-35607: Issue Type: Umbrella (was: Test) > Migrate Spark Arm Job from Jenkins to GitHub Actions >

[jira] [Updated] (SPARK-37735) Add appId interface to KubernetesConf

2021-12-24 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37735: Summary: Add appId interface to KubernetesConf (was: Add appId interfrace to KubernetesConf) >

[jira] [Created] (SPARK-37735) Add appId interfrace to KubernetesConf

2021-12-24 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37735: --- Summary: Add appId interfrace to KubernetesConf Key: SPARK-37735 URL: https://issues.apache.org/jira/browse/SPARK-37735 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-37721) Failed to execute pyspark test in Win WSL

2021-12-22 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37721: --- Summary: Failed to execute pyspark test in Win WSL Key: SPARK-37721 URL: https://issues.apache.org/jira/browse/SPARK-37721 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-37145) Improvement for extending pod feature steps with KubernetesConf

2021-12-21 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37145: Parent: SPARK-36057 Issue Type: Sub-task (was: Improvement) > Improvement for extending

[jira] [Updated] (SPARK-36057) Support volcano/alternative schedulers

2021-12-21 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36057: Description: This is an umbrella issue for tracking the work for supporting Volcano & Yunikorn

[jira] [Updated] (SPARK-36057) Support volcano/alternative schedulers

2021-12-21 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36057: Description: This is an umbrella issue for tracking the work for supporting Volcano & Yunikorn

[jira] [Created] (SPARK-37687) Cleanup direct usage of OkHttpClient

2021-12-19 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37687: --- Summary: Cleanup direct usage of OkHttpClient Key: SPARK-37687 URL: https://issues.apache.org/jira/browse/SPARK-37687 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-37258) Upgrade kubernetes-client to 5.11.0 (add Volcano support)

2021-12-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37258: Summary: Upgrade kubernetes-client to 5.11.0 (add Volcano support) (was: Upgrade

[jira] [Updated] (SPARK-37258) Upgrade kubernetes-client to 5.11.0 to support Volcano

2021-12-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37258: Summary: Upgrade kubernetes-client to 5.11.0 to support Volcano (was: Upgrade kubernetes-client

[jira] [Updated] (SPARK-37258) Upgrade kubernetes-client to 5.11.0

2021-12-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37258: Summary: Upgrade kubernetes-client to 5.11.0 (was: Add Volcano support in kubernetes-client) >

[jira] [Commented] (SPARK-37145) Improvement for extending pod feature steps with KubernetesConf

2021-12-13 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458229#comment-17458229 ] Yikun Jiang commented on SPARK-37145: - cc [~holden]  > Improvement for extending pod feature steps

[jira] [Created] (SPARK-37498) test_reuse_worker_of_parallelize_range is flaky

2021-11-29 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37498: --- Summary: test_reuse_worker_of_parallelize_range is flaky Key: SPARK-37498 URL: https://issues.apache.org/jira/browse/SPARK-37498 Project: Spark Issue Type:

[jira] [Created] (SPARK-37489) Skip hasnans check in numops if eager_check disable

2021-11-29 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37489: --- Summary: Skip hasnans check in numops if eager_check disable Key: SPARK-37489 URL: https://issues.apache.org/jira/browse/SPARK-37489 Project: Spark Issue

[jira] [Created] (SPARK-37480) Configurations in docs/running-on-kubernetes.md are not uptodate

2021-11-28 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37480: --- Summary: Configurations in docs/running-on-kubernetes.md are not uptodate Key: SPARK-37480 URL: https://issues.apache.org/jira/browse/SPARK-37480 Project: Spark

[jira] [Commented] (SPARK-37258) Add Volcano support in kubernetes-client

2021-11-23 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447967#comment-17447967 ] Yikun Jiang commented on SPARK-37258: - Volcano extension have been merged in kubernetes client,

[jira] [Created] (SPARK-37372) Remove redundant Pod label editition

2021-11-18 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37372: --- Summary: Remove redundant Pod label editition Key: SPARK-37372 URL: https://issues.apache.org/jira/browse/SPARK-37372 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444235#comment-17444235 ] Yikun Jiang commented on SPARK-37181: - Actully, lating-1 is also do same covert in Python internal

[jira] [Comment Edited] (SPARK-37181) pyspark.pandas.read_csv() should support latin-1 encoding

2021-11-15 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17444235#comment-17444235 ] Yikun Jiang edited comment on SPARK-37181 at 11/16/21, 2:41 AM: Agree,

[jira] [Created] (SPARK-37331) Add the ability to creating resources before driver pod

2021-11-15 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37331: --- Summary: Add the ability to creating resources before driver pod Key: SPARK-37331 URL: https://issues.apache.org/jira/browse/SPARK-37331 Project: Spark Issue

[jira] [Commented] (SPARK-37258) Add Volcano support in kubernetes-client

2021-11-11 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442249#comment-17442249 ] Yikun Jiang commented on SPARK-37258: - https://github.com/fabric8io/kubernetes-client/pull/3580 >

[jira] [Commented] (SPARK-37197) Behaviour inconsistency between pandas and pandas API on Spark

2021-11-11 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17442189#comment-17442189 ] Yikun Jiang commented on SPARK-37197: - [~hyukjin.kwon] Thanks for cc, I will try to pick some issues

[jira] [Updated] (SPARK-37258) Add Volcano support in kubernetes-client

2021-11-09 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37258: Description: We need to add a volcano in [1]. There were some comments on k8s-client repo to

[jira] [Created] (SPARK-37258) Add Volcano support in kubernetes-client

2021-11-09 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37258: --- Summary: Add Volcano support in kubernetes-client Key: SPARK-37258 URL: https://issues.apache.org/jira/browse/SPARK-37258 Project: Spark Issue Type: Sub-task

[jira] [Comment Edited] (SPARK-36566) Add Spark appname as a label to the executor pods

2021-11-01 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17436885#comment-17436885 ] Yikun Jiang edited comment on SPARK-36566 at 11/1/21, 3:11 PM: --- Yep, it's

[jira] [Commented] (SPARK-36566) Add Spark appname as a label to the executor pods

2021-11-01 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17436885#comment-17436885 ] Yikun Jiang commented on SPARK-36566: - Yep, it's useful for me. Does it make sense if we also set

[jira] [Commented] (SPARK-36057) Support volcano/alternative schedulers

2021-10-25 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433576#comment-17433576 ] Yikun Jiang commented on SPARK-36057: - [~seedjeffwan] Hi, jiaxin, thanks for you attention, FYI

[jira] [Commented] (SPARK-37055) Apply 'compute.eager_check' across all the codebase

2021-10-20 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431077#comment-17431077 ] Yikun Jiang commented on SPARK-37055: - Looks like it's a method level results runtime validation

[jira] [Commented] (SPARK-36348) unexpected Index loaded: pd.Index([10, 20, None], name="x")

2021-10-19 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430392#comment-17430392 ] Yikun Jiang commented on SPARK-36348: - revisit this, it had been already fixed in current master

[jira] [Issue Comment Deleted] (SPARK-36231) Support arithmetic operations of Series containing Decimal(np.nan)

2021-10-18 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36231: Comment: was deleted (was: https://github.com/apache/spark/pull/34314) > Support arithmetic

[jira] [Commented] (SPARK-36231) Support arithmetic operations of Series containing Decimal(np.nan)

2021-10-18 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429994#comment-17429994 ] Yikun Jiang commented on SPARK-36231: - https://github.com/apache/spark/pull/34314 > Support

[jira] [Comment Edited] (SPARK-37039) np.nan series.astype(bool) should be True

2021-10-18 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429973#comment-17429973 ] Yikun Jiang edited comment on SPARK-37039 at 10/18/21, 12:27 PM: - Looks

[jira] [Commented] (SPARK-37039) np.nan series.astype(bool) should be True

2021-10-18 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429973#comment-17429973 ] Yikun Jiang commented on SPARK-37039: - working on this > np.nan series.astype(bool) should be True

[jira] [Updated] (SPARK-37039) np.nan series.astype(bool) should be True

2021-10-18 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37039: Parent: (was: SPARK-36000) Issue Type: Bug (was: Sub-task) > np.nan

[jira] [Updated] (SPARK-37039) np.nan series.astype(bool) should be True

2021-10-18 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-37039: Description: np.nan series.astype(bool) should be True, rather than Fasle:

[jira] [Created] (SPARK-37039) np.nan series.astype(bool) should be True

2021-10-18 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-37039: --- Summary: np.nan series.astype(bool) should be True Key: SPARK-37039 URL: https://issues.apache.org/jira/browse/SPARK-37039 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-36231) Support arithmetic operations of Series containing Decimal(np.nan)

2021-10-15 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429515#comment-17429515 ] Yikun Jiang commented on SPARK-36231: - working on this > Support arithmetic operations of Series

[jira] [Commented] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

2021-10-15 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429514#comment-17429514 ] Yikun Jiang commented on SPARK-36230: - working on this > hasnans for Series of Decimal(`NaN`) >

[jira] [Commented] (SPARK-36059) Add the ability to specify a scheduler & queue

2021-10-11 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426984#comment-17426984 ] Yikun Jiang commented on SPARK-36059: - We have the abilities of specify a scheduler on executor

[jira] [Comment Edited] (SPARK-36058) Support replicasets/job API

2021-09-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420568#comment-17420568 ] Yikun Jiang edited comment on SPARK-36058 at 9/28/21, 1:11 AM: --- After

[jira] [Comment Edited] (SPARK-36058) Support replicasets/job API

2021-09-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420568#comment-17420568 ] Yikun Jiang edited comment on SPARK-36058 at 9/27/21, 2:32 PM: --- After

[jira] [Comment Edited] (SPARK-36058) Support replicasets/job API

2021-09-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420568#comment-17420568 ] Yikun Jiang edited comment on SPARK-36058 at 9/27/21, 2:31 PM: --- After

[jira] [Comment Edited] (SPARK-36058) Support replicasets/job API

2021-09-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420568#comment-17420568 ] Yikun Jiang edited comment on SPARK-36058 at 9/27/21, 11:22 AM: After

[jira] [Commented] (SPARK-36058) Support replicasets/job API

2021-09-27 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420568#comment-17420568 ] Yikun Jiang commented on SPARK-36058: - After this, it makes the executor pod allocator pluggable, we

[jira] [Comment Edited] (SPARK-36337) decimal('Nan') is unsupported in net.razorvine.pickle

2021-09-05 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410300#comment-17410300 ] Yikun Jiang edited comment on SPARK-36337 at 9/6/21, 2:37 AM: -- Good news:

[jira] [Comment Edited] (SPARK-36337) decimal('Nan') is unsupported in net.razorvine.pickle

2021-09-05 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410300#comment-17410300 ] Yikun Jiang edited comment on SPARK-36337 at 9/6/21, 2:36 AM: -- Good news:

[jira] [Commented] (SPARK-36337) decimal('Nan') is unsupported in net.razorvine.pickle

2021-09-05 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410300#comment-17410300 ] Yikun Jiang commented on SPARK-36337: - Good news: the doubel Nan pickled issue solved in

[jira] [Updated] (SPARK-36348) unexpected Index loaded: pd.Index([10, 20, None], name="x")

2021-07-29 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36348: Description: {code:python} pidx = pd.Index([10, 20, 15, 30, 45, None], name="x") psidx =

[jira] [Updated] (SPARK-36348) unexpected Index loaded: pd.Index([10, 20, None], name="x")

2021-07-29 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36348: Description: {code:python} pidx = pd.Index([10, 20, 15, 30, 45, None], name="x") psidx =

[jira] [Created] (SPARK-36348) unexpected Index loaded: pd.Index([10, 20, None], name="x")

2021-07-29 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-36348: --- Summary: unexpected Index loaded: pd.Index([10, 20, None], name="x") Key: SPARK-36348 URL: https://issues.apache.org/jira/browse/SPARK-36348 Project: Spark

[jira] [Comment Edited] (SPARK-36000) Support creation and operations of ps.Series/Index with Decimal('NaN')

2021-07-28 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389243#comment-17389243 ] Yikun Jiang edited comment on SPARK-36000 at 7/29/21, 4:15 AM: ---

[jira] [Updated] (SPARK-36337) decimal('Nan') is unsupported in net.razorvine.pickle

2021-07-28 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36337: Description: Decimal('NaN') is not supported by net.razorvine.pickle now. In Python {code:java}

[jira] [Comment Edited] (SPARK-36000) Support creation and operations of ps.Series/Index with Decimal('NaN')

2021-07-28 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389243#comment-17389243 ] Yikun Jiang edited comment on SPARK-36000 at 7/29/21, 4:02 AM: ---

[jira] [Created] (SPARK-36337) decimal('Nan') is unsupported in net.razorvine.pickle

2021-07-28 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-36337: --- Summary: decimal('Nan') is unsupported in net.razorvine.pickle Key: SPARK-36337 URL: https://issues.apache.org/jira/browse/SPARK-36337 Project: Spark Issue

[jira] [Commented] (SPARK-36000) Support creation and operations of ps.Series/Index with Decimal('NaN')

2021-07-28 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17389243#comment-17389243 ] Yikun Jiang commented on SPARK-36000: - [~XinrongM] I did some investigation on this, I found the

[jira] [Commented] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-15 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381269#comment-17381269 ] Yikun Jiang commented on SPARK-36031: - [~XinrongM] OK, Thanks for your help. > Keep same behavior

[jira] [Created] (SPARK-36075) Support for specifiying executor/driver node selector

2021-07-09 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-36075: --- Summary: Support for specifiying executor/driver node selector Key: SPARK-36075 URL: https://issues.apache.org/jira/browse/SPARK-36075 Project: Spark Issue

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-07 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36031: External issue URL: (was: https://issues.apache.org/jira/browse/SPARK-34849) > Keep same

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-07 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36031: Issue Type: Improvement (was: Bug) > Keep same behavior with pandas for operations of series

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-07 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36031: External issue URL: https://issues.apache.org/jira/browse/SPARK-34849 > Keep same behavior with

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-07 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36031: Parent: (was: SPARK-35337) Issue Type: Bug (was: Technical task) > Keep same

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-07 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36031: Issue Type: Technical task (was: Sub-task) > Keep same behavior with pandas for operations of

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-07 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36031: Description: There are many operations for series doen't follow the pandas, such as:

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-07 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36031: Description: There are many operations for series doen't follow the pandas, such as:

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-07 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-36031: Description: There are many operations for series doen't follow the pandas, such as: >>> pser =

[jira] [Created] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-07 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-36031: --- Summary: Keep same behavior with pandas for operations of series with nan Key: SPARK-36031 URL: https://issues.apache.org/jira/browse/SPARK-36031 Project: Spark

[jira] [Commented] (SPARK-36019) Cannot run leveldb related UTs on Mac OS of M1 architecture

2021-07-05 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17375165#comment-17375165 ] Yikun Jiang commented on SPARK-36019: - FYI, related:

[jira] [Commented] (SPARK-36002) Consolidate tests for data-type-based operations of decimal Series

2021-07-04 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1737#comment-1737 ] Yikun Jiang commented on SPARK-36002: - [~XinrongM] I think we can merge test_decimal_ops into

[jira] [Created] (SPARK-35849) Make astype data-type-based for DecimalOps

2021-06-21 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-35849: --- Summary: Make astype data-type-based for DecimalOps Key: SPARK-35849 URL: https://issues.apache.org/jira/browse/SPARK-35849 Project: Spark Issue Type:

[jira] [Commented] (SPARK-35782) leveldbjni doesn't work in Apple Silicon on macOS

2021-06-17 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364790#comment-17364790 ] Yikun Jiang commented on SPARK-35782: - I'm not sure the common arm64 native can meet Apple Silicon

[jira] [Created] (SPARK-35721) Path level discover for python unittests

2021-06-10 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-35721: --- Summary: Path level discover for python unittests Key: SPARK-35721 URL: https://issues.apache.org/jira/browse/SPARK-35721 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-35708) Add BaseTest for DataTypeOps

2021-06-10 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-35708: --- Summary: Add BaseTest for DataTypeOps Key: SPARK-35708 URL: https://issues.apache.org/jira/browse/SPARK-35708 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-35342) Introduce DecimalOps

2021-06-08 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359314#comment-17359314 ] Yikun Jiang commented on SPARK-35342: - Looks like the Jira is open, can I help with it? :) >

[jira] [Created] (SPARK-35668) Use "concurrency" setting on Github Actions

2021-06-07 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-35668: --- Summary: Use "concurrency" setting on Github Actions Key: SPARK-35668 URL: https://issues.apache.org/jira/browse/SPARK-35668 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-35607) Migrate Spark Arm Job from Jenkins to GitHub Actions

2021-06-01 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-35607: Description: There were two Arm CI jobs in AMP lab:

[jira] [Updated] (SPARK-35607) Migrate Spark Arm Job from Jenkins to GitHub Actions

2021-06-01 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-35607: Description: There were two Arm CI jobs in AMP lab:

[jira] [Updated] (SPARK-35607) Migrate Spark Arm Job from Jenkins to GitHub Actions

2021-06-01 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-35607: Description: There were two Arm CI jobs in AMP lab:

[jira] [Created] (SPARK-35607) Migrate Spark Arm Job from Jenkins to GitHub Actions

2021-06-01 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-35607: --- Summary: Migrate Spark Arm Job from Jenkins to GitHub Actions Key: SPARK-35607 URL: https://issues.apache.org/jira/browse/SPARK-35607 Project: Spark Issue

[jira] [Updated] (SPARK-35584) Increase the timeout in FallbackStorageSuite

2021-05-31 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-35584: Description: The aarch64 case failed due to:   {{Discovery starting. Discovery completed in 2

[jira] [Commented] (SPARK-35584) Increase the timeout in FallbackStorageSuite

2021-05-31 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354753#comment-17354753 ] Yikun Jiang commented on SPARK-35584: - I also see some random timeout failed test on github action,

[jira] [Created] (SPARK-35584) Increase the timeout in FallbackStorageSuite

2021-05-31 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-35584: --- Summary: Increase the timeout in FallbackStorageSuite Key: SPARK-35584 URL: https://issues.apache.org/jira/browse/SPARK-35584 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-20 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348135#comment-17348135 ] Yikun Jiang commented on SPARK-35373: - related issue: 

[jira] [Commented] (SPARK-35458) ARM CI failed: failed to validate maven sha512

2021-05-20 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348129#comment-17348129 ] Yikun Jiang commented on SPARK-35458: - This is not a Arm only problem, but also a problem on the

[jira] [Updated] (SPARK-35458) ARM CI failed: failed to validate maven sha512

2021-05-20 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-35458: Description: Log:   Veryfing checksum from

[jira] [Issue Comment Deleted] (SPARK-35458) ARM CI failed: failed to validate maven sha512

2021-05-20 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang updated SPARK-35458: Comment: was deleted (was: cc [~shaneknapp]) > ARM CI failed: failed to validate maven sha512 >

[jira] [Created] (SPARK-35458) ARM CI failed: failed to validate maven sha512

2021-05-20 Thread Yikun Jiang (Jira)
Yikun Jiang created SPARK-35458: --- Summary: ARM CI failed: failed to validate maven sha512 Key: SPARK-35458 URL: https://issues.apache.org/jira/browse/SPARK-35458 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-35458) ARM CI failed: failed to validate maven sha512

2021-05-20 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348107#comment-17348107 ] Yikun Jiang commented on SPARK-35458: - cc [~shaneknapp] > ARM CI failed: failed to validate maven

<    1   2   3   4   5   6   7   >