[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/15671 @jkbradley Updated --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #68101 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68101/consoleFull)** for PR 15671 at commit [`880ae3f`](https://github.com/apache/spark/commit/880ae3f9e754b9e74c300a3e3cbc791b4154d677). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68101/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #68101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68101/consoleFull)** for PR 15671 at commit [`880ae3f`](https://github.com/apache/spark/commit/880ae3f9e754b9e74c300a3e3cbc791b4154d677). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15671 @jkbradley Thanks for bringing that up. I'm ok with alternate solutions provided they don't require someone to remember to manually add or manually except a new param, and that we can ensure that we aren't logging params that could blow up the logs. We can discuss it on the JIRA, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/15671 I don't want to truncate Param strings because it would create invalid JSON in case people want to try to catch and parse the logs. I like the idea of allowing exceptions and possibly adding unit tests to ensure the logs do not blow up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15671 I created [SPARK-18253](https://issues.apache.org/jira/browse/SPARK-18253) to track it. We may have to get to it after 2.1 QA period. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68039/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #68039 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68039/consoleFull)** for PR 15671 at commit [`e77bdc4`](https://github.com/apache/spark/commit/e77bdc4761620f46f332239e1afb7ec46c1738cd). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/15671 @sethah I agree that manually listing traceable params is prone to mistake. I think we can log all params expect some params which are labeled `dont-log` in the individual algorithms. Or we can create a new methods `def logParams(params: Seq[Param], except: Seq[Param])`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #68039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68039/consoleFull)** for PR 15671 at commit [`e77bdc4`](https://github.com/apache/spark/commit/e77bdc4761620f46f332239e1afb7ec46c1738cd). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/15671 @sethah I have make changes according to the comments. Thanks for your reviewing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15671 IMO, the way we're doing this logging right now is unsustainable. It requires too much manual work. We can leave this discussion for a different JIRA, but what we could do is modify the `Instrumentation` class to just truncate the param value string after a certain number of characters. Then, we could even modify `Predictor.fit` to create Instrumentation and log all params. The only thing we'd have to do in the individual algorithms is log anything else - algorithm specific - that we want to add. I haven't tested this yet. @jkbradley @zhengruifeng what do you think? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67947/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #67947 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67947/consoleFull)** for PR 15671 at commit [`6d2d13f`](https://github.com/apache/spark/commit/6d2d13f79ae68e71d023e7c79d19586842d49c75). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15671 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15671: [SPARK-18206][ML]Add instrumentation logs to ML training...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15671 **[Test build #67947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67947/consoleFull)** for PR 15671 at commit [`6d2d13f`](https://github.com/apache/spark/commit/6d2d13f79ae68e71d023e7c79d19586842d49c75). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
