[jira] [Commented] (SPARK-23990) Instruments logging improvements - ML regression package
[ https://issues.apache.org/jira/browse/SPARK-23990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16442370#comment-16442370 ] Weichen Xu commented on SPARK-23990: [~josephkb] I agree that option 2b looks better. Updating PR... > Instruments logging improvements - ML regression package > > > Key: SPARK-23990 > URL: https://issues.apache.org/jira/browse/SPARK-23990 > Project: Spark > Issue Type: Sub-task > Components: ML >Affects Versions: 2.3.0 > Environment: Instruments logging improvements - ML regression package >Reporter: Weichen Xu >Priority: Major > Original Estimate: 120h > Remaining Estimate: 120h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-23990) Instruments logging improvements - ML regression package
[ https://issues.apache.org/jira/browse/SPARK-23990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441701#comment-16441701 ] Joseph K. Bradley commented on SPARK-23990: --- A complication was brought up by this PR: Some logging occurs in classes which are not Estimators (WeightedLeastSquares, IterativelyReweightedLeastSquares) and in static objects (RandomForest, GradientBoostedTrees). These may have an Instrumentation instance available (when used from an Estimator) or may not (when used in a unit test). Options include: 1. Make these require Instrumentation instances. This would require slightly awkward changes to unit tests. 2. Create something similar to Instrumentation or Logging which can store an Optional Instrumentation instance. If the Instrumentation is available, it can log via that; otherwise, it can call into regular Logging. 2a. This could be a trait like Logging. This is nice in that it requires fewer changes to existing logging code. 2b. This could be a class like Instrumentation. This is nice in that it standardizes all of MLlib around Instrumentation instead of Logging. I'd vote for 2b to standardize what we do in MLlib. Thoughts? > Instruments logging improvements - ML regression package > > > Key: SPARK-23990 > URL: https://issues.apache.org/jira/browse/SPARK-23990 > Project: Spark > Issue Type: Sub-task > Components: ML >Affects Versions: 2.3.0 > Environment: Instruments logging improvements - ML regression package >Reporter: Weichen Xu >Priority: Major > Original Estimate: 120h > Remaining Estimate: 120h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-23990) Instruments logging improvements - ML regression package
[ https://issues.apache.org/jira/browse/SPARK-23990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16439268#comment-16439268 ] Apache Spark commented on SPARK-23990: -- User 'WeichenXu123' has created a pull request for this issue: https://github.com/apache/spark/pull/21078 > Instruments logging improvements - ML regression package > > > Key: SPARK-23990 > URL: https://issues.apache.org/jira/browse/SPARK-23990 > Project: Spark > Issue Type: Sub-task > Components: ML >Affects Versions: 2.3.0 > Environment: Instruments logging improvements - ML regression package >Reporter: Weichen Xu >Priority: Major > Original Estimate: 120h > Remaining Estimate: 120h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
