[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user squito commented on the issue: https://github.com/apache/spark/pull/17700 thanks for the reminder, sorry it took me a couple days to circle back. merged to master and 2.2 And thanks a lot for this issue -- for taking another look at these after that last patch, realizing it was not ideal, then working quickly to get a fix in! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17700 @squito , do you have any further comment? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17700 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17700 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76040/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17700 **[Test build #76040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76040/testReport)** for PR 17700 at commit [`e1d618b`](https://github.com/apache/spark/commit/e1d618b3396b7b89b731d4f1b3d19781416fe85b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17700 **[Test build #76040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76040/testReport)** for PR 17700 at commit [`e1d618b`](https://github.com/apache/spark/commit/e1d618b3396b7b89b731d4f1b3d19781416fe85b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user squito commented on the issue: https://github.com/apache/spark/pull/17700 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17700 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76015/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17700 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17700 **[Test build #76015 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76015/testReport)** for PR 17700 at commit [`e1d618b`](https://github.com/apache/spark/commit/e1d618b3396b7b89b731d4f1b3d19781416fe85b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17700 Thanks @squito for clarification, sorry I misunderstood it. Regarding this new `memoryMetrics`, will all the memory related metrics be shown here, like what you mentioned in the JIRA? Also this PR #17625 exposes the Netty memory usage, should this memory usage also be shown here? I'm thinking of a extensible and reasonable place to track all the memory usages, so that our follow-up work will not break this API again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/17700 Yes that is what I was thinking from the conversation in the jira. We should do that now as to not cause more compatibility issues. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user squito commented on the issue:
https://github.com/apache/spark/pull/17700
Thanks for doing this quickly @jerryshao. I wasn't very clear about this on
the jira, but my original idea was to add another level of nesting in the json,
and put all the new memory related metrics under there (leaving `maxmemory` and
`memoryUsed` as they were for compatibility), eg.:
```
{
"id" : "1",
"hostPort" : "172.22.0.167:51490",
...
"memoryMetrics": {
"usedOnHeapStorageMemory": ...,
"usedOffHeapStorageMemory": ...,
...
}
}
```
I think that organization would be a little more clear.
@tgravescs wdyt?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17700 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17700 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75978/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17700 **[Test build #75978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75978/testReport)** for PR 17700 at commit [`135e233`](https://github.com/apache/spark/commit/135e233a1136db815a6a2e83916b8990f7317d4b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17700: [SPARK-20391][Core] Rename memory related fields in Exec...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17700 **[Test build #75978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75978/testReport)** for PR 17700 at commit [`135e233`](https://github.com/apache/spark/commit/135e233a1136db815a6a2e83916b8990f7317d4b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
