[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928674#comment-16928674 ] feiwang edited comment on SPARK-29037 at 9/12/19 3:48 PM: -- [~ad

[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928674#comment-16928674 ] feiwang edited comment on SPARK-29037 at 9/12/19 3:49 PM: -- [~ad

[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928674#comment-16928674 ] feiwang edited comment on SPARK-29037 at 9/12/19 3:58 PM: -- [~ad

[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928596#comment-16928596 ] feiwang edited comment on SPARK-29037 at 9/12/19 4:17 PM: -- [~ad

[jira] [Comment Edited] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-14 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928674#comment-16928674 ] feiwang edited comment on SPARK-29037 at 9/15/19 4:41 AM: -- [~ad

[jira] [Commented] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-09-14 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929916#comment-16929916 ] feiwang commented on SPARK-29037: - [~advancedxy] Hi, I found that even with dynamicParti

[jira] [Commented] (SPARK-29043) [History Server]Only one replay thread of FsHistoryProvider work because of straggler

2019-09-15 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929939#comment-16929939 ] feiwang commented on SPARK-29043: - Thanks, [~kabhwan]. > [History Server]Only one repl

[jira] [Commented] (SPARK-28945) Allow concurrent writes to different partitions with dynamic partition overwrite

2019-09-15 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930010#comment-16930010 ] feiwang commented on SPARK-28945: - [~cloud_fan] [~advancedxy] Hi, I think the exception

[jira] [Comment Edited] (SPARK-28945) Allow concurrent writes to different partitions with dynamic partition overwrite

2019-09-15 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930010#comment-16930010 ] feiwang edited comment on SPARK-28945 at 9/15/19 5:12 PM: -- [~cl

[jira] [Comment Edited] (SPARK-28945) Allow concurrent writes to different partitions with dynamic partition overwrite

2019-09-15 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930010#comment-16930010 ] feiwang edited comment on SPARK-28945 at 9/15/19 5:17 PM: -- [~cl

[jira] [Commented] (SPARK-28945) Allow concurrent writes to different partitions with dynamic partition overwrite

2019-09-15 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930221#comment-16930221 ] feiwang commented on SPARK-28945: - [~advancedxy] Thanks. Hope SPARK-28945 can be merged

[jira] [Comment Edited] (SPARK-28945) Allow concurrent writes to different partitions with dynamic partition overwrite

2019-09-15 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930221#comment-16930221 ] feiwang edited comment on SPARK-28945 at 9/16/19 4:22 AM: -- [~ad

[jira] [Created] (SPARK-29113) Some annotation errors in ApplicationCache.scala

2019-09-16 Thread feiwang (Jira)
feiwang created SPARK-29113: --- Summary: Some annotation errors in ApplicationCache.scala Key: SPARK-29113 URL: https://issues.apache.org/jira/browse/SPARK-29113 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-29113) Some annotation errors in ApplicationCache.scala

2019-09-16 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29113: Issue Type: Documentation (was: Bug) > Some annotation errors in ApplicationCache.scala > ---

[jira] [Created] (SPARK-29262) DataFrameWriter insertIntoPartition function

2019-09-26 Thread feiwang (Jira)
feiwang created SPARK-29262: --- Summary: DataFrameWriter insertIntoPartition function Key: SPARK-29262 URL: https://issues.apache.org/jira/browse/SPARK-29262 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-29262) DataFrameWriter insertIntoPartition function

2019-09-28 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29262: Description: InsertIntoPartition is a useful function. For SQL statement, relative syntax. {code:java} in

[jira] [Updated] (SPARK-29262) DataFrameWriter insertIntoPartition function

2019-09-28 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29262: Description: InsertIntoPartition is a useful function. For SQL statement, relative syntax. {code:java} in

[jira] [Created] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
feiwang created SPARK-29295: --- Summary: Duplicate result when dropping partition of an external table and then overwriting Key: SPARK-29295 URL: https://issues.apache.org/jira/browse/SPARK-29295 Project: Spa

[jira] [Updated] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29295: Description: When we drop a partition of a external table and then overwrite it, if we set CONVERT_METAST

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940598#comment-16940598 ] feiwang commented on SPARK-29295: - When we set CONVERT_METASTORE_PARQUET=true, it will u

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940599#comment-16940599 ] feiwang commented on SPARK-29295: - I have tested in 2.3.1 branch, without SPARK-25271, i

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940600#comment-16940600 ] feiwang commented on SPARK-29295: - cc [~cloud_fan] > Duplicate result when dropping par

[jira] [Comment Edited] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940600#comment-16940600 ] feiwang edited comment on SPARK-29295 at 9/30/19 1:52 AM: -- cc [

[jira] [Issue Comment Deleted] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-29 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29295: Comment: was deleted (was: I have tested in 2.3.1 branch, without SPARK-25271, it will always give duplic

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-30 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940754#comment-16940754 ] feiwang commented on SPARK-29295: - relative hive issue, https://issues.apache.org/jira/b

[jira] [Created] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-09-30 Thread feiwang (Jira)
feiwang created SPARK-29302: --- Summary: dynamic partition overwrite with speculation enabled Key: SPARK-29302 URL: https://issues.apache.org/jira/browse/SPARK-29302 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-09-30 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29302: Description: Now, for a dynamic partition overwrite operation, the filename of a task output is determin

[jira] [Updated] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-09-30 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29302: Description: Now, for a dynamic partition overwrite operation, the filename of a task output is determin

[jira] [Updated] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-09-30 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29302: Description: Now, for a dynamic partition overwrite operation, the filename of a task output is determin

[jira] [Commented] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-09-30 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941467#comment-16941467 ] feiwang commented on SPARK-29302: - You can add the code below into FileFormatWriterSuite

[jira] [Comment Edited] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-09-30 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941472#comment-16941472 ] feiwang edited comment on SPARK-29302 at 10/1/19 2:24 AM: -- For

[jira] [Commented] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-09-30 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941472#comment-16941472 ] feiwang commented on SPARK-29302: - For dynamic partition overwrite, when execute a task,

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

2019-09-30 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941475#comment-16941475 ] feiwang commented on SPARK-29295: - Yes, how can we resolve this issue? Only by upgrading

[jira] [Commented] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-09-30 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941485#comment-16941485 ] feiwang commented on SPARK-29302: - Yes, they are in a same stage, so they have a same jo

[jira] [Updated] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29302: Attachment: screenshot-1.png > dynamic partition overwrite with speculation enabled >

[jira] [Updated] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29302: Attachment: screenshot-2.png > dynamic partition overwrite with speculation enabled >

[jira] [Updated] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29302: Attachment: screenshot-1.png > dynamic partition overwrite with speculation enabled >

[jira] [Updated] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29302: Attachment: (was: screenshot-1.png) > dynamic partition overwrite with speculation enabled > -

[jira] [Commented] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16947446#comment-16947446 ] feiwang commented on SPARK-29302: - Sorry, I was on my National Day holiday for the past

[jira] [Comment Edited] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16947446#comment-16947446 ] feiwang edited comment on SPARK-29302 at 10/9/19 8:30 AM: -- Sorr

[jira] [Reopened] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang reopened SPARK-29302: - > dynamic partition overwrite with speculation enabled > ---

[jira] [Comment Edited] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16947446#comment-16947446 ] feiwang edited comment on SPARK-29302 at 10/9/19 8:31 AM: -- Sorr

[jira] [Commented] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16947451#comment-16947451 ] feiwang commented on SPARK-29302: - cc [~cloud_fan] > dynamic partition overwrite with s

[jira] [Comment Edited] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-11 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949229#comment-16949229 ] feiwang edited comment on SPARK-29302 at 10/11/19 7:46 AM: --- [~

[jira] [Commented] (SPARK-29302) dynamic partition overwrite with speculation enabled

2019-10-11 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949229#comment-16949229 ] feiwang commented on SPARK-29302: - [~dangdangdang] Hi, I have thought a simple solution

[jira] [Updated] (SPARK-29037) [Core] Spark gives duplicate result when an application was killed and rerun

2019-10-17 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29037: Description: For InsertIntoHadoopFsRelation operations. Case A: Application appA insert overwrite table t

[jira] [Commented] (SPARK-29262) DataFrameWriter insertIntoPartition function

2019-10-18 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954424#comment-16954424 ] feiwang commented on SPARK-29262: - I'll try to implement it. > DataFrameWriter insertIn

[jira] [Created] (SPARK-29542) [DOC] The description of `spark.sql.files.maxPartitionBytes` is confused.

2019-10-21 Thread feiwang (Jira)
feiwang created SPARK-29542: --- Summary: [DOC] The description of `spark.sql.files.maxPartitionBytes` is confused. Key: SPARK-29542 URL: https://issues.apache.org/jira/browse/SPARK-29542 Project: Spark

[jira] [Updated] (SPARK-29542) [DOC] The description of `spark.sql.files.maxPartitionBytes` is confused.

2019-10-21 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29542: Attachment: screenshot-1.png > [DOC] The description of `spark.sql.files.maxPartitionBytes` is confused. >

[jira] [Updated] (SPARK-29542) [DOC] The description of `spark.sql.files.maxPartitionBytes` is confused.

2019-10-21 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29542: Description: Hi,the description of `spark.sql.files.maxPartitionBytes` is shown as below. {code:java} The

[jira] [Updated] (SPARK-29542) [DOC] The description of `spark.sql.files.maxPartitionBytes` is confused.

2019-10-21 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29542: Description: Hi,the description of `spark.sql.files.maxPartitionBytes` is shown as below. {code:java} The

[jira] [Updated] (SPARK-29542) [DOC] The description of `spark.sql.files.maxPartitionBytes` is confused.

2019-10-21 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29542: Description: Hi,the description of `spark.sql.files.maxPartitionBytes` is shown as below. {code:java} The

[jira] [Updated] (SPARK-29542) [DOC] The description of `spark.sql.files.maxPartitionBytes` is confused.

2019-10-21 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29542: Description: Hi,the description of `spark.sql.files.maxPartitionBytes` is shown as below. {code:java} The

[jira] [Updated] (SPARK-29542) [DOC] The description of `spark.sql.files.maxPartitionBytes` is confused.

2019-10-21 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29542: Description: Hi,the description of `spark.sql.files.maxPartitionBytes` is shown as below. {code:java} The

[jira] [Updated] (SPARK-29542) [SQL][DOC] The descriptions of `spark.sql.files.*` are confused.

2019-10-22 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29542: Summary: [SQL][DOC] The descriptions of `spark.sql.files.*` are confused. (was: [DOC] The description of

[jira] [Updated] (SPARK-29542) [SQL][DOC] The descriptions of `spark.sql.files.*` are confused.

2019-10-22 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29542: Description: Hi,the description of `spark.sql.files.maxPartitionBytes` is shown as below. {code:java} The

[jira] [Commented] (SPARK-27736) Improve handling of FetchFailures caused by ExternalShuffleService losing track of executor registrations

2019-10-25 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960244#comment-16960244 ] feiwang commented on SPARK-27736: - Hi, we met this issue recently. [~joshrosen] [~tgrave

[jira] [Comment Edited] (SPARK-27736) Improve handling of FetchFailures caused by ExternalShuffleService losing track of executor registrations

2019-10-25 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960244#comment-16960244 ] feiwang edited comment on SPARK-27736 at 10/26/19 2:30 AM: --- Hi

[jira] [Created] (SPARK-29689) [UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size

2019-10-31 Thread feiwang (Jira)
feiwang created SPARK-29689: --- Summary: [UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size Key: SPARK-29689 URL: https://issues.apache.org/jira/browse/SPARK-29689

[jira] [Updated] (SPARK-29689) [WEB-UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size

2019-10-31 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29689: Summary: [WEB-UI] When task failed during reading shuffle data or other failure, enable show total shuffle

[jira] [Updated] (SPARK-29689) [WEB-UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size

2019-10-31 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29689: Attachment: screenshot-1.png > [WEB-UI] When task failed during reading shuffle data or other failure, >

[jira] [Updated] (SPARK-29689) [WEB-UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size

2019-10-31 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29689: Description: If task failed during reading shuffle data or because of executor loss, its shuffle read siz

[jira] [Updated] (SPARK-29689) [WEB-UI] When task failed during reading shuffle data or other failure, enable show total shuffle read size

2019-10-31 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29689: Description: As shown in the attachment, if task failed during reading shuffle data or because of executo

[jira] [Created] (SPARK-29857) [WEB UI] Support defer render the spark history summary page.

2019-11-12 Thread feiwang (Jira)
feiwang created SPARK-29857: --- Summary: [WEB UI] Support defer render the spark history summary page. Key: SPARK-29857 URL: https://issues.apache.org/jira/browse/SPARK-29857 Project: Spark Issue T

[jira] [Created] (SPARK-29860) [SQL] Fix data type mismatch issue for inSubQuery

2019-11-12 Thread feiwang (Jira)
feiwang created SPARK-29860: --- Summary: [SQL] Fix data type mismatch issue for inSubQuery Key: SPARK-29860 URL: https://issues.apache.org/jira/browse/SPARK-29860 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-29860) [SQL] Fix data type mismatch issue for inSubQuery

2019-11-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29860: Description: The follow statement would throw an exception. {code:java} sql("create table ta(id De

[jira] [Updated] (SPARK-29860) [SQL] Fix data type mismatch issue for inSubQuery

2019-11-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29860: Description: The follow statement would throw an exception. {code:java} sql("create table ta(id De

[jira] [Updated] (SPARK-29860) [SQL] Fix data type mismatch issue for inSubQuery

2019-11-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29860: Description: The follow statement would throw an exception. {code:java} sql("create table ta(id De

[jira] [Updated] (SPARK-29860) [SQL] Fix data type mismatch issue for inSubQuery

2019-11-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29860: Description: The follow statement would throw an exception. {code:java} sql("create table ta(id De

[jira] [Updated] (SPARK-29857) [WEB UI] Support defer render the spark history summary page.

2019-11-12 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-29857: Description: When there are many applications in spark history server, the renderer of history summary pa

[jira] [Created] (SPARK-34040) function runCliWithin of CliSuite can not cover some test cases

2021-01-06 Thread feiwang (Jira)
feiwang created SPARK-34040: --- Summary: function runCliWithin of CliSuite can not cover some test cases Key: SPARK-34040 URL: https://issues.apache.org/jira/browse/SPARK-34040 Project: Spark Issue

[jira] [Updated] (SPARK-34040) function runCliWithin of CliSuite can not cover some test cases

2021-01-06 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-34040: Description: Here is a test case(query contains '\n') which can not be covered by runCliWithin: {code:j

[jira] [Updated] (SPARK-34040) function runCliWithin of CliSuite can not cover some test cases

2021-01-06 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-34040: Description: Here is a test case(query contains two statements splitted by '\n') which can not be covere

[jira] [Updated] (SPARK-34040) function runCliWithin of CliSuite can not cover some test cases

2021-01-06 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-34040: Description: Here is a test case(query contains '\n') which can not be covered by runCliWithin: {code:j

[jira] [Created] (SPARK-30471) Fix issue when compare string and IntegerType

2020-01-09 Thread feiwang (Jira)
feiwang created SPARK-30471: --- Summary: Fix issue when compare string and IntegerType Key: SPARK-30471 URL: https://issues.apache.org/jira/browse/SPARK-30471 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-30471) Fix issue when compare string and IntegerType

2020-01-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-30471: Description: When we comparing a String Type and IntegerType: '2147483648'(StringType, which exceed Int.Ma

[jira] [Updated] (SPARK-30471) Fix issue when compare string and IntegerType

2020-01-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-30471: Description: When we comparing a String Type and IntegerType: '2147483648'(StringType, which exceed Int.Ma

[jira] [Created] (SPARK-30472) ANSI SQL: Cast String to Integer Type, throw exception on format invalid and overflow.

2020-01-09 Thread feiwang (Jira)
feiwang created SPARK-30472: --- Summary: ANSI SQL: Cast String to Integer Type, throw exception on format invalid and overflow. Key: SPARK-30472 URL: https://issues.apache.org/jira/browse/SPARK-30472 Project:

[jira] [Updated] (SPARK-30472) [SQL] ANSI SQL: Throw exception on format invalid and overflow when casting String to IntegerType.

2020-01-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-30472: Summary: [SQL] ANSI SQL: Throw exception on format invalid and overflow when casting String to IntegerType

[jira] [Created] (SPARK-31016) [DEPLOY] Pack the user jars when submitting Spark Application

2020-03-02 Thread feiwang (Jira)
feiwang created SPARK-31016: --- Summary: [DEPLOY] Pack the user jars when submitting Spark Application Key: SPARK-31016 URL: https://issues.apache.org/jira/browse/SPARK-31016 Project: Spark Issue Ty

[jira] [Updated] (SPARK-31016) [DEPLOY] Pack the user jars when submitting Spark Application

2020-03-02 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-31016: Description: Nowadays, Spark only pack the jars under $SPARK_HOME/jars. How about packing the user jars wh

[jira] [Created] (SPARK-31093) Fast fail while fetching shuffle data from a remote block unsuccessfully

2020-03-09 Thread feiwang (Jira)
feiwang created SPARK-31093: --- Summary: Fast fail while fetching shuffle data from a remote block unsuccessfully Key: SPARK-31093 URL: https://issues.apache.org/jira/browse/SPARK-31093 Project: Spark

[jira] [Updated] (SPARK-31093) Fast fail while fetching shuffle data unsuccessfully

2020-03-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-31093: Summary: Fast fail while fetching shuffle data unsuccessfully (was: Fast fail while fetching shuffle data

[jira] [Updated] (SPARK-31093) Fast fail while fetching shuffle data unsuccessfully

2020-03-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-31093: Description: When fetching shuffle data unsuccessfully, we put a FailureFetchResult into results(a linked

[jira] [Created] (SPARK-31179) Fast fail the connection while last shuffle connection failed in the last retry IO wait

2020-03-17 Thread feiwang (Jira)
feiwang created SPARK-31179: --- Summary: Fast fail the connection while last shuffle connection failed in the last retry IO wait Key: SPARK-31179 URL: https://issues.apache.org/jira/browse/SPARK-31179 Projec

[jira] [Updated] (SPARK-31179) Fast fail the connection while last shuffle connection failed in the last retry IO wait

2020-03-17 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-31179: Description: When reading shuffle data, maybe several fetch request sent to a same shuffle server. There

[jira] [Commented] (SPARK-27614) Executor shuffle fetch hang

2019-05-26 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16848388#comment-16848388 ] feiwang commented on SPARK-27614: - You can propose a PR to spark. It seems meaningful.

[jira] [Issue Comment Deleted] (SPARK-27614) Executor shuffle fetch hang

2019-05-26 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-27614: Comment: was deleted (was: You can propose a PR to spark. It seems meaningful.) > Executor shuffle fetch

[jira] [Created] (SPARK-27876) [CORE][SHUFFLE] Split large shuffle partition to multi-segments to enable transfer oversize shuffle partition block.

2019-05-29 Thread feiwang (JIRA)
feiwang created SPARK-27876: --- Summary: [CORE][SHUFFLE] Split large shuffle partition to multi-segments to enable transfer oversize shuffle partition block. Key: SPARK-27876 URL: https://issues.apache.org/jira/browse/SPA

[jira] [Updated] (SPARK-27876) [CORE][SHUFFLE] Split large shuffle partition to multi-segments to enable transfer oversize shuffle partition block.

2019-05-29 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-27876: Description: There is a limit for shuffle read. If a shuffle partition block's size is large than Intege

[jira] [Updated] (SPARK-27876) [CORE][SHUFFLE] Split large shuffle partition to multi-segments to enable transfer oversize shuffle partition block.

2019-05-29 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-27876: Description: There is a limit for shuffle read. If a shuffle partition block's size is large than Intege

[jira] [Updated] (SPARK-27876) Split large shuffle partition to multi-segments to enable transfer oversize shuffle partition block.

2019-05-30 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-27876: Affects Version/s: (was: 3.1.0) (was: 2.4.3) 2.3.2 >

[jira] [Created] (SPARK-28047) [UI] Little bug in executorspage.js

2019-06-13 Thread feiwang (JIRA)
feiwang created SPARK-28047: --- Summary: [UI] Little bug in executorspage.js Key: SPARK-28047 URL: https://issues.apache.org/jira/browse/SPARK-28047 Project: Spark Issue Type: Bug Componen

[jira] [Resolved] (SPARK-28047) [UI] Little bug in executorspage.js

2019-06-13 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang resolved SPARK-28047. - Resolution: Not A Problem > [UI] Little bug in executorspage.js >

[jira] [Updated] (SPARK-28194) [SQL] A NoSuchElementException maybe thrown when EnsureRequirement

2019-06-27 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-28194: Summary: [SQL] A NoSuchElementException maybe thrown when EnsureRequirement (was: a NoSuchElementExceptio

[jira] [Created] (SPARK-28194) a NoSuchElementException maybe thrown when EnsureRequirement

2019-06-27 Thread feiwang (JIRA)
feiwang created SPARK-28194: --- Summary: a NoSuchElementException maybe thrown when EnsureRequirement Key: SPARK-28194 URL: https://issues.apache.org/jira/browse/SPARK-28194 Project: Spark Issue Typ

[jira] [Updated] (SPARK-28194) [SQL] A NoSuchElementException maybe thrown when EnsureRequirement

2019-06-27 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-28194: Description: {code:java} java.util.NoSuchElementException: None.get at scala.None$.get(Option.sca

[jira] [Commented] (SPARK-28194) [SQL] A NoSuchElementException maybe thrown when EnsureRequirement

2019-07-04 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878664#comment-16878664 ] feiwang commented on SPARK-28194: - {code:java} create table if not exists test.nga(ac1

[jira] [Commented] (SPARK-27689) Error to execute hive views with spark

2019-07-30 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896247#comment-16896247 ] feiwang commented on SPARK-27689: - It seemed that this failure is caused by PR-SPARK-18

[jira] [Commented] (SPARK-27689) Error to execute hive views with spark

2019-07-30 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896249#comment-16896249 ] feiwang commented on SPARK-27689: - It seemed that this failure is caused by PR-SPARK-18

<    1   2   3   >