Github user rxin commented on the issue:
https://github.com/apache/spark/pull/21822
Yea the extra check in test cases might've contributed to the longer test
time. Let me think about how to reduce it.
On Mon, Jul 23, 2018 at 11:28 PM Hyukjin Kwon <[email protected]>
wrote:
> @rxin <https://github.com/rxin>, I think this PR could possibly cause
> some performance effect given the latest test ran above
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93470/
> and from a rough scan of other builds:
>
> ArithmeticExpressionSuite:
> - SPARK-22499: Least and greatest should not generate codes beyond 64KB
(11 minutes, 51 seconds)
>
> CastSuite:
> - cast string to timestamp (8 minutes, 42 seconds)
>
> TPCDSQuerySuite:
> - q14a-v2.7 (2 minutes, 29 seconds)
>
> SQLQueryTestSuite:
> - subquery/in-subquery/in-joins.sql (2 minutes, 36 seconds)
>
> ContinuousStressSuite:
> - only one epoch (3 minutes, 21 seconds)
> - automatic epoch advancement (3 minutes, 21 seconds)
>
> vs
>
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7/4699/consoleFull
>
>
> ArithmeticExpressionSuite:
> - SPARK-22499: Least and greatest should not generate codes beyond 64KB
(7 minutes, 49 seconds)
>
> CastSuite:
> - cast string to timestamp (1 minute)
>
> TPCDSQuerySuite:
> - q14a-v2.7 (3 seconds, 442 milliseconds)
>
> SQLQueryTestSuite:
> - subquery/in-subquery/in-joins.sql (2 minutes, 21 seconds)
>
> ContinuousStressSuite:
> - only one epoch (3 minutes, 21 seconds)
> - automatic epoch advancement (3 minutes, 21 seconds)
>
> There could of course other factors like machine's status as well but I
> thought it's good to note while I am taking a look for those.
>
> â
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/21822#issuecomment-407295739>, or
mute
> the thread
>
<https://github.com/notifications/unsubscribe-auth/AATvPMFWaX8huI8hkWBghiP740jtu592ks5uJr6fgaJpZM4VXRDF>
> .
>
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
