Github user MrBago commented on a diff in the pull request:
https://github.com/apache/spark/pull/20837#discussion_r175564965
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala
---
@@ -517,6 +517,9 @@ class LogisticRegression @Since("1.2.0") (
(new MultivariateOnlineSummarizer, new MultiClassSummarizer)
)(seqOp, combOp, $(aggregationDepth))
}
+ instr.logNamedValue(Instrumentation.loggerTags.numExamples,
summarizer.count)
+ instr.logNamedValue("lowestLabelWeight",
labelSummarizer.histogram.min.toString)
+ instr.logNamedValue("highestLabelWeight",
labelSummarizer.histogram.min.toString)
--- End diff --
It might be useful to add a utility for logging arrays and vectors (we can
json encode them), but in the meantime I wanted to capture at least minimal
information about the data balance.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
