[jira] [Updated] (SPARK-34898) Send ExecutorMetricsUpdate EventLog appropriately

2021-04-06 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34898: --- Description: In current EventLoggingListener, we won't write SparkListenerExecutorMetricsUpdate message at

[jira] [Issue Comment Deleted] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-03-28 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Comment: was deleted (was: Hi [~Baohe Zhang] , This ticket proposes a new REST API:  http://:18080/api/v1/a

[jira] [Updated] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-03-26 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Description: When we want to get a specific stage's detail info with task information, it will return all t

[jira] [Commented] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-03-24 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17308305#comment-17308305 ] Ron Hu commented on SPARK-34488: Hi [~srowen], Thanks for merging this PR.  [~angerszhuu

[jira] [Updated] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-03-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Description: When we want to get a specific stage's detail info with task information, it will return all t

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-03-01 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-02-28 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-02-28 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Commented] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-02-25 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17291067#comment-17291067 ] Ron Hu commented on SPARK-34092: We clarified the need for query parameter taskStatus fo

[jira] [Updated] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-02-25 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Description: When we want to get a specific stage's detail info with task information, it will return all t

[jira] [Commented] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-02-23 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289646#comment-17289646 ] Ron Hu commented on SPARK-34488: It should be noted that this Jira  addresses query para

[jira] [Commented] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-02-23 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289644#comment-17289644 ] Ron Hu commented on SPARK-34092: It should be noted that this Jira  addresses query para

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: [~angerszhuuu] and [~ron8hu] discussed a generic and consistent way for overall stages, i.e. e

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: [~angerszhuuu] and [~ron8hu] discussed a generic and consistent way for overall stages, i.e. e

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: [~angerszhuuu] and [~ron8hu] discussed a generic and consistent way for endpoint /application/

[jira] [Commented] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288026#comment-17288026 ] Ron Hu commented on SPARK-26399: Add an example of task Metrics percentile Distributions

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: executorMetricsDistributions.json > Define query parameters to support various filtering conditi

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: taskMetricsDistributions.json > Define query parameters to support various filtering conditions

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-21 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: taskMetricsDistributions.json > Define query parameters to support various filtering conditions

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Summary: Define query parameters to support various filtering conditions in REST API for overall stages (wa

[jira] [Updated] (SPARK-26399) Define query parameters to support various filtering conditions in REST API for overall stages

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: [~angerszhuuu] and [~ron8hu] discussed a generic and consistent way for endpoint /application/

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Summary: Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a sp

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Attachment: executorMetricsDistributions.json > Support task Metrics Distributions and executor Metrics Dist

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Attachment: taskMetricsDistributions.json > Support task Metrics Distributions and executor Metrics Distribu

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution.  This

[jira] [Updated] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34488: --- Description: For a specific stage, it is useful to show the task metrics in percentile distribution. This

[jira] [Created] (SPARK-34488) Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage

2021-02-20 Thread Ron Hu (Jira)
Ron Hu created SPARK-34488: -- Summary: Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage Key: SPARK-34488 URL: https://issues.apache.org/jira/browse/SPARK-34

[jira] [Updated] (SPARK-34092) support filtering by task status in REST API call for a specific stage

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Summary: support filtering by task status in REST API call for a specific stage (was: Stage level RestFul A

[jira] [Updated] (SPARK-34092) Stage level RestFul API support filter by task status

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Description: Query parameter taskStatus can be used to filter the tasks meeting a specific status in the RE

[jira] [Updated] (SPARK-34092) Stage level RestFul API support filter by task status

2021-02-20 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-34092: --- Description: Query parameter taskStatus can be used to filter the tasks meeting a specific status in the RE

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-02-04 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-22 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/23/21, 3:39 AM: -- [~an

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-22 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/10/21, 6:22 PM: -- [~an

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/10/21, 6:22 PM: -- [~an

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/10/21, 6:20 PM: -- [~an

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/10/21, 6:18 PM: -- [~an

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-10 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261948#comment-17261948 ] Ron Hu edited comment on SPARK-26399 at 1/9/21, 9:25 PM: - [~ange

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-09 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261948#comment-17261948 ] Ron Hu commented on SPARK-26399: [~angerszhuuu] and [~ron8hu] discussed a generic and co

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260856#comment-17260856 ] Ron Hu edited comment on SPARK-26399 at 1/8/21, 5:18 AM: - The in

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260856#comment-17260856 ] Ron Hu commented on SPARK-26399: The initial description of this jira has this statement

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: lispark230_restapi_ex2_stages_failedTasks.json > Add new stage-level REST APIs and parameters >

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260854#comment-17260854 ] Ron Hu commented on SPARK-26399: [^lispark230_restapi_ex2_stages_failedTasks.json] > Ad

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260805#comment-17260805 ] Ron Hu edited comment on SPARK-26399 at 1/7/21, 10:23 PM: -- Dr.

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260805#comment-17260805 ] Ron Hu edited comment on SPARK-26399 at 1/7/21, 8:41 PM: - Dr. El

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260805#comment-17260805 ] Ron Hu edited comment on SPARK-26399 at 1/7/21, 8:40 PM: - Dr. El

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: lispark230_restapi_ex2_stages_withSummaries.json > Add new stage-level REST APIs and parameters

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260808#comment-17260808 ] Ron Hu commented on SPARK-26399: [^lispark230_restapi_ex2_stages_withSummaries.json] >

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-07 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260805#comment-17260805 ] Ron Hu commented on SPARK-26399: Dr. Elephant (https://github.com/linkedin/dr-elephant)

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 9:00 PM: - [~ange

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Description: Add the peak values for the metrics to the stages REST API. Also add a new executorSummary RES

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 7:46 PM: - [~ange

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 7:45 PM: - [~ange

[jira] [Comment Edited] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257844#comment-17257844 ] Ron Hu edited comment on SPARK-26399 at 1/3/21, 7:44 PM: - [~ange

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: executorMetricsSummary.json > Add new stage-level REST APIs and parameters > ---

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257846#comment-17257846 ] Ron Hu commented on SPARK-26399: [^executorMetricsSummary.json] > Add new stage-level R

[jira] [Updated] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26399: --- Attachment: stage_executorSummary_image1.png > Add new stage-level REST APIs and parameters > --

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257845#comment-17257845 ] Ron Hu commented on SPARK-26399: !stage_executorSummary_image1.png! > Add new stage-lev

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-03 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257844#comment-17257844 ] Ron Hu commented on SPARK-26399: [~angerszhuuu] found that the "executorSummary" field a

[jira] [Commented] (SPARK-26399) Add new stage-level REST APIs and parameters

2021-01-02 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257639#comment-17257639 ] Ron Hu commented on SPARK-26399: Hi [~Baohe Zhang] , This ticket proposes a new REST API

[jira] [Updated] (SPARK-26341) Expose executor memory metrics at the stage level, in the Stages tab

2020-12-02 Thread Ron Hu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-26341: --- Affects Version/s: (was: 3.1.0) 3.0.1 > Expose executor memory metrics at the sta

[jira] [Comment Edited] (SPARK-21322) support histogram in filter cardinality estimation

2017-11-20 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258601#comment-16258601 ] Ron Hu edited comment on SPARK-21322 at 11/21/17 1:43 AM: -- Pull

[jira] [Commented] (SPARK-21322) support histogram in filter cardinality estimation

2017-11-19 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258601#comment-16258601 ] Ron Hu commented on SPARK-21322: Pull request 19357 was created while there were several

[jira] [Created] (SPARK-22164) support histogram in estimating the cardinality of aggregate (or group-by) operator

2017-09-28 Thread Ron Hu (JIRA)
Ron Hu created SPARK-22164: -- Summary: support histogram in estimating the cardinality of aggregate (or group-by) operator Key: SPARK-22164 URL: https://issues.apache.org/jira/browse/SPARK-22164 Project: Spar

[jira] [Commented] (SPARK-17075) Cardinality Estimation of Predicate Expressions

2017-09-26 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181793#comment-16181793 ] Ron Hu commented on SPARK-17075: This jira has been resolved. Pull request 19357 was add

[jira] [Created] (SPARK-21322) support histogram in filter cardinality estimation

2017-07-05 Thread Ron Hu (JIRA)
Ron Hu created SPARK-21322: -- Summary: support histogram in filter cardinality estimation Key: SPARK-21322 URL: https://issues.apache.org/jira/browse/SPARK-21322 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-03-31 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-19408: --- Description: In SPARK-17075, we estimate cardinality of predicate expression "column (op) literal", where op

[jira] [Updated] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-03-24 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-19408: --- Target Version/s: 2.2.0 (was: 2.3.0) > cardinality estimation involving two columns of the same table >

[jira] [Updated] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-03-21 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-19408: --- Target Version/s: 2.3.0 (was: 2.2.0) > cardinality estimation involving two columns of the same table >

[jira] (SPARK-19408) cardinality estimation involving two columns of the same table

2017-01-30 Thread Ron Hu (JIRA)
Title: Message Title Ron Hu created an issue

[jira] [Issue Comment Deleted] (SPARK-17076) Cardinality estimation of join operator

2016-12-08 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-17076: --- Comment: was deleted (was: Hi, I am out of office 11 /15 through 11/18 with very limited Internet access. M

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-11-30 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710667#comment-15710667 ] Ron Hu commented on SPARK-16026: Hi Reynold, I previously worked on filter cardinality es

[jira] [Commented] (SPARK-17076) Cardinality estimation of join operator

2016-11-14 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665664#comment-15665664 ] Ron Hu commented on SPARK-17076: Hi, I am out of office 11 /15 through 11/18 with very l

[jira] [Issue Comment Deleted] (SPARK-16026) Cost-based Optimizer framework

2016-11-01 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Comment: was deleted (was: Design Specification of Spark Cost-Based Optimization is now posted.) > Cost-base

[jira] [Commented] (SPARK-17791) Join reordering using star schema detection

2016-10-28 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616590#comment-15616590 ] Ron Hu commented on SPARK-17791: This JIRA is indeed complementary to Cost-Based Optimize

[jira] [Commented] (SPARK-17626) TPC-DS performance improvements using star-schema heuristics

2016-10-08 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559044#comment-15559044 ] Ron Hu commented on SPARK-17626: In the CBO design spec we posted in https://issues.apac

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-09-07 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472000#comment-15472000 ] Ron Hu commented on SPARK-16026: Hi Srinath, Thank you for your comments. Let me answer t

[jira] [Issue Comment Deleted] (SPARK-16026) Cost-based Optimizer framework

2016-09-07 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Comment: was deleted (was: Hi Srinath, Thank you for your comments. Let me answer them one by one. First

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-09-07 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15471986#comment-15471986 ] Ron Hu commented on SPARK-16026: Hi Srinath, Thank you for your comments. Let me answer

[jira] [Updated] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Attachment: Spark_CBO_Design_Spec.pdf > Cost-based Optimizer framework > -- > >

[jira] [Updated] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Attachment: (was: Spark_CBO_Design_Spec_v_1_2.pdf) > Cost-based Optimizer framework > ---

[jira] [Updated] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Attachment: Spark_CBO_Design_Spec_v_1_2.pdf > Cost-based Optimizer framework > --

[jira] [Updated] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Attachment: (was: Spark_CBO_Design_Spec_v_1_2.pdf) > Cost-based Optimizer framework > ---

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422375#comment-15422375 ] Ron Hu commented on SPARK-16026: Design Specification of Spark Cost-Based Optimization is

[jira] [Issue Comment Deleted] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Comment: was deleted (was: I will post a pdf version of the design spec soon.) > Cost-based Optimizer framew

[jira] [Issue Comment Deleted] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Comment: was deleted (was: Design Specification of Spark Cost-Based Optimization) > Cost-based Optimizer fra

[jira] [Updated] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-16026: --- Attachment: Spark_CBO_Design_Spec_v_1_2.pdf Design Specification of Spark Cost-Based Optimization > Cost-bas

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422346#comment-15422346 ] Ron Hu commented on SPARK-16026: I will post a pdf version of the design spec soon. > Co

[jira] [Commented] (SPARK-16026) Cost-based Optimizer framework

2016-08-16 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422332#comment-15422332 ] Ron Hu commented on SPARK-16026: Sean, I wrote "Design Specification of Spark Cost-Based

[jira] [Closed] (SPARK-16990) Define the data structure to hold the statistics for CBO

2016-08-15 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu closed SPARK-16990. -- Resolution: Duplicate > Define the data structure to hold the statistics for CBO >

  1   2   >