[ https://issues.apache.org/jira/browse/SPARK-19311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gregor Moehler updated SPARK-19311: ----------------------------------- Description: When you define UDTs based on hierarchical traits UDFs disregard the type hierarchy: E.g. I have 2 UDTs based on 2 hierarchical traits. I then define 2 UDFs: The first one returns the derived type, the second takes the base type. This results in an error, although i believe it should be feasible: (...)cannot resolve 'UDF(UDF(22))' due to data type mismatch: argument 1 requires exampleBaseType type, however, 'UDF(22)' is of exampleFirstSubType type. The reason is that DataType defines override private[sql] def acceptsType(dataType: DataType) = this.getClass == dataType.getClass However I believe it should be: override private[sql] def acceptsType(dataType: DataType) = dataType match { case other: UserDefinedType[_] => this.getClass == other.getClass || this.userClass.isAssignableFrom(other.userClass) case _ => false } was: When you define UDTs based on hierarchical traits UDFs disregard the type hierarchy: E.g. I have 2 UDTs based on 2 hierarchical traits. I then define 2 UDFs: The first one returns the derived type, the second takes the base type. This results in an error, although i believe it should be feasible: (...)cannot resolve 'UDF(UDF(22))' due to data type mismatch: argument 1 requires exampleBaseType type, however, 'UDF(22)' is of exampleFirstSubType type. The reason is that DataType defines override private[sql] def acceptsType(dataType: DataType) = this.getClass == dataType.getClass However I believe it should be: ```override private[sql] def acceptsType(dataType: DataType) = dataType match { case other: UserDefinedType[_] => this.getClass == other.getClass || this.userClass.isAssignableFrom(other.userClass) case _ => false }``` > UDFs disregard UDT type hierarchy > --------------------------------- > > Key: SPARK-19311 > URL: https://issues.apache.org/jira/browse/SPARK-19311 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.1.0 > Reporter: Gregor Moehler > Original Estimate: 1h > Remaining Estimate: 1h > > When you define UDTs based on hierarchical traits UDFs disregard the type > hierarchy: > E.g. I have 2 UDTs based on 2 hierarchical traits. I then define 2 UDFs: The > first one returns the derived type, the second takes the base type. This > results in an error, although i believe it should be feasible: > (...)cannot resolve 'UDF(UDF(22))' due to data type mismatch: argument 1 > requires exampleBaseType type, however, 'UDF(22)' is of exampleFirstSubType > type. > The reason is that DataType defines > override private[sql] def acceptsType(dataType: DataType) = > this.getClass == dataType.getClass > However I believe it should be: > override private[sql] def acceptsType(dataType: DataType) = dataType match { > case other: UserDefinedType[_] => > this.getClass == other.getClass || > this.userClass.isAssignableFrom(other.userClass) > case _ => false > } -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org