Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14276
It's worth double-checking with @holdenk and @dbtsai. I think this is
working as intended since `WeightedLeastSquares` does show multiplying each
feature by sigma. To undo it you'd need to divide the partial gradient by its
square, and divide the squared coefficient value by its square too in the loss
term.
I suppose the logic is that features on a larger scale end up with small
coefficients and aren't penalized much in the loss function, so multiplying
them by their "scale" compensates. I think this only makes sense when fitting
an intercept too, but I haven't thought this through.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
