Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/15018#discussion_r95914244
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/regression/IsotonicRegression.scala
---
@@ -312,90 +313,120 @@ class IsotonicRegression private (private var
isotonic: Boolean) extends Seriali
}
/**
- * Performs a pool adjacent violators algorithm (PAV).
- * Uses approach with single processing of data where violators
- * in previously processed data created by pooling are fixed immediately.
- * Uses optimization of discovering monotonicity violating sequences
(blocks).
+ * Performs a pool adjacent violators algorithm (PAV). Implements the
algorithm originally
+ * described in [1], using the formulation from [2, 3]. Uses an array to
keep track of start
+ * and end indices of blocks.
*
- * @param input Input data of tuples (label, feature, weight).
+ * [1] Grotzinger, S. J., and C. Witzgall. "Projections onto order
simplexes." Applied
+ * mathematics and Optimization 12.1 (1984): 247-270.
+ *
+ * [2] Best, Michael J., and Nilotpal Chakravarti. "Active set
algorithms for isotonic
+ * regression; a unifying framework." Mathematical Programming 47.1-3
(1990): 425-439.
+ *
+ * [3] Best, Michael J., Nilotpal Chakravarti, and Vasant A. Ubhaya.
"Minimizing separable convex
+ * functions subject to simple chain constraints." SIAM Journal on
Optimization 10.3 (2000):
+ * 658-672.
+ *
+ * @param input Input data of tuples (label, feature, weight). Weights
must
+ be non-negative.
* @return Result tuples (label, feature, weight) where labels were
updated
* to form a monotone sequence as per isotonic regression
definition.
*/
private def poolAdjacentViolators(
input: Array[(Double, Double, Double)]): Array[(Double, Double,
Double)] = {
- if (input.isEmpty) {
- return Array.empty
+ val cleanInput = input.flatMap{ case (y, x, weight) =>
--- End diff --
filter would be simpler than flatMap
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
