Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/22638#discussion_r222946564
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetCacheSuite.scala ---
@@ -127,16 +127,16 @@ class DatasetCacheSuite extends QueryTest with
SharedSQLContext with TimeLimits
}
test("cache UDF result correctly") {
- val expensiveUDF = udf({x: Int => Thread.sleep(5000); x})
- val df = spark.range(0, 10).toDF("a").withColumn("b",
expensiveUDF($"a"))
+ val expensiveUDF = udf({x: Int => Thread.sleep(2000); x})
--- End diff --
@mgaido91 OK, please correct me on this one. So we insert 2 rows .. i.e two
invocation of the UDF amounting to 2 * 2sec = 4 secs of execution. So wouldn't
a 2 sec fail time be ok ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
