[ https://issues.apache.org/jira/browse/SPARK-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Akos Tomasits updated SPARK-22198: ---------------------------------- Summary: Java incompatibility when extending UnaryTransformer or Transformer (was: Java incompatibility when extending UnaryTransformer) > Java incompatibility when extending UnaryTransformer or Transformer > ------------------------------------------------------------------- > > Key: SPARK-22198 > URL: https://issues.apache.org/jira/browse/SPARK-22198 > Project: Spark > Issue Type: Bug > Components: Java API, ML > Affects Versions: 2.2.0 > Reporter: Akos Tomasits > Priority: Major > > It is not possible to create proper Java custom Transformer by extending > UnaryTransformer. > It seems that the method 'uid()' is called during object creation before the > provided 'uid' constructor parameter could be set. > This leads to the following error: > {quote} > java.lang.IllegalArgumentException: requirement failed: Param > <prefix>_1563950936fa__inputCol does not belong to <prefix>_d4105b75c4aa. > {quote} > If you extend UnaryTransformer and try to use it e.g. through CrossValidator, > you will need to explicitly include a constructor, which receives a String > parameter. As I saw in the source of built in transformers, this parameter is > a 'uid', which should be set in the object. However, it is not possible to do > it in time, because the uid() method is invoked (and its result might be > used) before this constructor finishes. > Sample class: > {quote} > public class TextCleaner extends UnaryTransformer<String, String, TextCleaner> > implements Serializable, DefaultParamsWritable, > DefaultParamsReadable<TextCleaner> \{ > private static final long serialVersionUID = 2658543236303100458L; > > private static final String sparkUidPrefix = "TextCleaner"; > > private final String sparkUid; > public TextCleaner() \{ > sparkUid = > org.apache.spark.ml.util.Identifiable$.MODULE$.randomUID(sparkUidPrefix); > } > public TextCleaner(String uid) \{ > sparkUid = uid; > } > > @Override > public String uid() \{ // This method is called by parent class, before > object creation finishes > return sparkUid; > } > ... > {quote} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org