Vladislav Glinskiy created SPARK-30851:
------------------------------------------
Summary: Add 'path' field to the 'LoadInstanceEnd' ML listener
event
Key: SPARK-30851
URL: https://issues.apache.org/jira/browse/SPARK-30851
Project: Spark
Issue Type: Improvement
Components: ML
Affects Versions: 3.0.0
Reporter: Vladislav Glinskiy
Fix For: 3.0.0
The
[LoadInstanceEnd|https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/events.scala#L92]
ML listener event that is added on the scope of SPARK-23674 has no 'path'
field which makes it impossible to determine from what path an ML instance was
loaded as well as there is no way to get instance's 'uid' via
[LoadInstanceStart|https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/events.scala#L84]
event.
The
[LoadInstanceEnd|https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/events.scala#L92]
must be changed to include 'path' field. Please, refer
[SaveInstanceStart|https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/events.scala#L101]
and
[SaveInstanceEnd|https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/events.scala#L109]
events. Both of them have `path` but
[LoadInstanceStart|https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/events.scala#L84]
and
[LoadInstanceEnd|https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/events.scala#L92]
are not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
