HeartSaVioR commented on a change in pull request #35680:
URL: https://github.com/apache/spark/pull/35680#discussion_r836230538
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameSessionWindowingSuite.scala
##########
@@ -495,4 +495,65 @@ class DataFrameSessionWindowingSuite extends QueryTest
with SharedSparkSession
validateWindowColumnInSchema(schema2, "session")
}
}
+
+ test("SPARK-38349: No need to filter events when gapDuration greater than
0") {
+ // negative value
Review comment:
Logic for verification can be deduplicated via adding inner-method in
this test like below:
```
def checkFilterCondition(
logicalPlan: LogicalPlan,
expectTimeRange: Boolean,
assertHintMsg: String): Unit = {
val filter = logicalPlan.find { plan =>
plan.isInstanceOf[Filter] && plan.children.head.isInstanceOf[Project]
}
assert(filter.isDefined)
val exist = filter.get.expressions.flatMap { expr =>
expr.collect { case gt: GreaterThan => gt }
}
if (expectTimeRange) {
assert(exist.nonEmpty, assertHintMsg)
} else {
assert(exist.isEmpty, assertHintMsg)
}
}
```
and call the method per case.
Let's use analyzed plan (`df.queryExecution.logical`) instead of
optimizedPlan which Spark can optimize the plan and may have changed the plan
out of the expectation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
