Github user tarekauel commented on the pull request:
https://github.com/apache/spark/pull/7462#issuecomment-122764175
@EntilZha
1. `eval` and `nullSafeEval`
`eval` will be invoked to evaluate the expression. Most expressions should
return `null` if one of there arguments is `null`. In order to avoid that every
expression has to check if `left` or `right` is `null`, `nullSafeEval` has been
added. `eval` does the null check and calls `nullSafeEval`, see.
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala#L289-L313
You should override `eval` if you don't want to return `null`, if one the
arguments is `null`. Most of the times you will use `nullSafeEval`.
2.
`UnaryExpression`: Expression has one parameter (like `size(x)`)
`BinaryExpression`: Expression has two parameters (like `contains(a, b)`)
`ExpectsInputTypes`: Allows to automatically check if the argument type is
correct, see
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ExpectsInputTypes.scala#L42-L57.
You specify the allowed types by overriding `inputTypes`.
`ImplicitCastInputs`: The difference to `ExpectsInputTypes` is that this
tries to cast the value. Most string operations are implemented with a byte
array as input. A string can be "casted" to a byte array by calling
`.getBytes`. `ImplicitCastInputs` allows to call `contains(s: String, s2:
String)` and `contains(s: Array[Byte], s2: Array[Byte])`. Typically you use
this if a cast is reasonable. Cast from anything else to string is most of the
times reasonable, but casting a string (automatically == implicit) to an
integer value is most of the time not helpful. Users could still invoke the
`cast` function.
3. I don't know
4. Intellij allows to run most suites from the IDE. And have a look at
https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-RunningIndividualTests
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
