Github user MLnick commented on the issue:
https://github.com/apache/spark/pull/17090
Isn't deciding on the output schema for these methods essentially the same
as deciding on transform semantics in #12574 (apart from the issue of how, or
if, to have transform generate the "ground truth" items)?
So in a way I'm not certain this "circumvents" the discussion around
transform semantics / fitting into cross-validation & pipelines but rather
makes an implicit decision on it. If this is set here, it would be rather
awkward to have any future `transform` based recommend all functionality with a
different output schema.
My default choice in #12574 was to match the form of the existing `mllib`
methods. That is the same as here, and matches the format for the existing
`RankingMetrics` in `mllib`. So from that perspective it is the "easiest"
choice.
It's not necessarily the "best" choice - I go into the other option in
detail on #12574 and the related JIRA. Ideally it should match up with the
expected input schema for a new `RankingEvaluator` (not strictly necessary but
definitely preferable).
My concern here is that we make a quick decision implicitly due to time
constraints and are stuck with it down the line as it is exposed in the public
API.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
