Github user lw-lin commented on a diff in the pull request:
https://github.com/apache/spark/pull/16719#discussion_r98346453
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionSetSuite.scala
---
@@ -32,6 +32,38 @@ class ExpressionSetSuite extends SparkFunSuite {
val aAndBSet = AttributeSet(aUpper :: bUpper :: Nil)
+ // An [AttributeReference] with almost the maximum hashcode, to make
testing canonicalize rules
+ // like `case GreaterThan(l, r) if l.hashcode > r.hashcode =>
GreaterThan(r, l)` easier
+ val maxHash =
+ Canonicalize.ignoreNamesTypes(
+ AttributeReference("maxHash", IntegerType)(exprId =
+ new ExprId(4, NamedExpression.jvmId) {
+ // maxHash's hashcode is calculated based on this exprId's
hashcode, so we set this
+ // exprId's hashCode to this specific value to make sure
maxHash's hashcode is almost
+ // `Int.MaxValue`
+ override def hashCode: Int = 826929706
--- End diff --
thanks.
the reason is in
[Canonicalize.scala#ignoreNamesTypes](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Canonicalize.scala#L41),
we're making copies of `e` (`maxHash` in this case):
```scala
private def ignoreNamesTypes(e: Expression): Expression = e match {
case a: AttributeReference =>
AttributeReference("none", a.dataType.asNullable)(exprId = a.exprId)
case _ => e
}
```
so, even if we `override def hashCode: Int = Int.MaxValue` on `maxHash`, it
has nothing to do with the copy's hashcode.
then i took a step back -- defined `exprId`'s hashcode to a specific value
(as provided in this patch), which would further affect the copied
attribute-reference's hashcode.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
