[jira] [Commented] (SPARK-20624) SPIP: Add better handling for node shutdown

2024-02-02 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813592#comment-17813592 ] Yu-Jhe Li commented on SPARK-20624: --- Hi, we found the migrated shuffle files from the decommissioned

[jira] [Updated] (SPARK-46957) Migrated shuffle data files from the decommissioned node should be removed when job completed

2024-02-02 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-46957: -- Description: Hi, we have a long-lived Spark application run on a standalone cluster on GCP and we

[jira] [Updated] (SPARK-46957) Migrated shuffle data files from the decommissioned node should be removed when job completed

2024-02-02 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-46957: -- Description: Hi, we have a long-lived Spark application run on a standalone cluster on GCP and we

[jira] [Updated] (SPARK-46957) Migrated shuffle data files from the decommissioned node should be removed when job completed

2024-02-02 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-46957: -- Description: Hi, we have a long-lived Spark application run on a standalone cluster on GCP and we

[jira] [Created] (SPARK-46957) Migrated shuffle data files from the decommissioned node should be removed when job completed

2024-02-02 Thread Yu-Jhe Li (Jira)
Yu-Jhe Li created SPARK-46957: - Summary: Migrated shuffle data files from the decommissioned node should be removed when job completed Key: SPARK-46957 URL: https://issues.apache.org/jira/browse/SPARK-46957

[jira] [Resolved] (SPARK-30181) Throws runtime exception when filter metastore partition key that's not string type or integral types

2019-12-17 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li resolved SPARK-30181. --- Resolution: Duplicate > Throws runtime exception when filter metastore partition key that's not >

[jira] [Commented] (SPARK-30181) Throws runtime exception when filter metastore partition key that's not string type or integral types

2019-12-17 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998053#comment-16998053 ] Yu-Jhe Li commented on SPARK-30181: --- Yes, the bug is fixed in the master branch and latest branch-2.4.

[jira] [Updated] (SPARK-30181) Throws runtime exception when filter metastore partition key that's not string type or integral types

2019-12-16 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-30181: -- Description: SQL below will throw a runtime exception since spark-2.4.0. I think it's a bug brought

[jira] [Created] (SPARK-30181) Throws runtime exception when filter metastore partition key that's not string type or integral types

2019-12-09 Thread Yu-Jhe Li (Jira)
Yu-Jhe Li created SPARK-30181: - Summary: Throws runtime exception when filter metastore partition key that's not string type or integral types Key: SPARK-30181 URL: https://issues.apache.org/jira/browse/SPARK-30181

[jira] [Commented] (SPARK-27652) Caught Hive MetaException when query by partition (partition col start with underscore)

2019-11-04 Thread Yu-Jhe Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967213#comment-16967213 ] Yu-Jhe Li commented on SPARK-27652: --- Hi [~tongqqiu], I have issued this problem too. How did you solve

[jira] [Resolved] (SPARK-24492) Endless attempted task when TaskCommitDenied exception writing to S3A

2019-04-18 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li resolved SPARK-24492. --- Resolution: Won't Do Haven't happened again since upgrade to 2.3.2, i'm going to close this issue.

[jira] [Commented] (SPARK-24492) Endless attempted task when TaskCommitDenied exception writing to S3A

2018-06-11 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508065#comment-16508065 ] Yu-Jhe Li commented on SPARK-24492: --- [~ste...@apache.org] thanks for your replay, do you have any idea

[jira] [Commented] (SPARK-24492) Endless attempted task when TaskCommitDenied exception writing to S3A

2018-06-08 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506792#comment-16506792 ] Yu-Jhe Li commented on SPARK-24492: --- [~ste...@apache.org]: i thinks the TaskCommitDenied exception is

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Description: Hi, when we run Spark application under spark-2.2.0 on AWS spot instance and output

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Environment: (was: Spark version: spark-2.2.0) > Endless attempted task when TaskCommitDenied

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Shepherd: (was: Jiang Xingbo) > Endless attempted task when TaskCommitDenied exception >

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Description: Hi, when we run Spark application under spark-2.2.0 on AWS spot instance and output

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Description: Hi, when we run Spark application under spark-2.2.0 on AWS spot instance and output

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Description: Hi, when we run Spark application under spark-2.2.0 on AWS spot instance and output

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Description: Hi, when we run Spark application under spark-2.2.0 on AWS spot instance and output

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Attachment: retry_stage.png 螢幕快照 2018-05-16 上午11.10.57.png 螢幕快照

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Description: Hi, when we run Spark application under spark-2.2.0 on AWS spot instance and output

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Description: Hi, when we run Spark application under spark-2.2.0 on AWS spot instance and output

[jira] [Updated] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Jhe Li updated SPARK-24492: -- Description: Hi, when we run Spark application under spark-2.2.0 on AWS spot instance and output

[jira] [Created] (SPARK-24492) Endless attempted task when TaskCommitDenied exception

2018-06-07 Thread Yu-Jhe Li (JIRA)
Yu-Jhe Li created SPARK-24492: - Summary: Endless attempted task when TaskCommitDenied exception Key: SPARK-24492 URL: https://issues.apache.org/jira/browse/SPARK-24492 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23614) Union produces incorrect results when caching is used

2018-05-18 Thread Yu-Jhe Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480438#comment-16480438 ] Yu-Jhe Li commented on SPARK-23614: --- Is this bug happening only when 1) cached dataframe 2)