[
https://issues.apache.org/jira/browse/SPARK-33273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248815#comment-17248815
]
Kent Yao commented on SPARK-33273:
----------------------------------
{code:java}
## once
build/sbt clean -Phive-2.3 -Pkinesis-asl -Pspark-ganglia-lgpl -Pyarn
-Phive-thriftserver test:package streaming-kinesis-asl-assembly/assembly
-Dsbt.override.build.repos=true
{code}
{code:java}
build/sbt -Phive "sql/testOnly *SQLQueryTestSuite -- -z
scalar-subquery-select.sql"
{code}
the possibility goes up to 5-10% when I run the 2nd command repeatedly on my mac
the SQL that hits the failure is not certain, but it seems always failed for
the output `min(t3d) ` w/ `0` as its wrong result.
here is one of the debug msg
{code:java}
14:22:32.702 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[10,2017-05-04 01:01:00.0]
14:22:33.481 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1a,2]
14:22:33.885 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[10,null]
14:22:33.885 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[null,2017-05-04 01:01:00.0]
14:22:34.329 ERROR org.apache.spark.sql.execution.HiveResult: debugging ==>[12]
14:22:34.918 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[null,val2a,null,200.83333333333334]
14:22:34.918 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1c,val1c,0,200.83333333333334]
14:22:34.918 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1c,val1c,0,200.83333333333334]
14:22:34.918 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1e,null,0,null]
14:22:34.918 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1e,null,0,null]
14:22:34.918 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1e,null,0,null]
14:22:35.240 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[19,2017-05-04 01:01:00.0]
14:22:35.636 ERROR org.apache.spark.sql.execution.HiveResult: debugging ==>[19]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1a,6]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1a,16]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1a,16]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1c,8]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1d,null]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1a,6]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1d,null]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1e,10]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1e,10]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1d,10]
14:22:36.183 ERROR org.apache.spark.sql.execution.HiveResult: debugging
==>[val1e,10]
14:22:36.206 ERROR org.apache.spark.sql.SQLQueryTestSuite: Error using configs:
[info] - subquery/scalar-subquery/scalar-subquery-select.sql *** FAILED *** (4
seconds, 522 milliseconds)
[info] subquery/scalar-subquery/scalar-subquery-select.sql
[info] Expected "...3333334
[info] val1c val1c [10 200.83333333333334
[info] val1c val1c 10 200.83333333333334
[info] val1e NULL 10 NULL
[info] val1e NULL 10 NULL
[info] val1e NULL 1]0 NULL", but got "...3333334
[info] val1c val1c [0 200.83333333333334
[info] val1c val1c 0 200.83333333333334
[info] val1e NULL 0 NULL
[info] val1e NULL 0 NULL
[info] val1e NULL ]0 NULL" Result did not match for query #8
[info] SELECT q1.t1a, q2.t2a, q1.min_t3d, q2.avg_t3d
[info] FROM (SELECT t1a, (SELECT min(t3d) FROM t3) min_t3d
[info] FROM t1
[info] WHERE t1a IN ('val1e', 'val1c')) q1
[info] FULL OUTER JOIN
[info] (SELECT t2a, (SELECT avg(t3d) FROM t3) avg_t3d
[info] FROM t2
[info] WHERE t2a IN ('val1c', 'val2a')) q2
[info] ON q1.t1a = q2.t2a
[info] AND q1.min_t3d < q2.avg_t3d (SQLQueryTestSuite.scala:464)
[info] org.scalatest.exceptions.TestFailedException:
{code}
> Fix Flaky Test: ThriftServerQueryTestSuite.
> subquery_scalar_subquery_scalar_subquery_select_sql
> -----------------------------------------------------------------------------------------------
>
> Key: SPARK-33273
> URL: https://issues.apache.org/jira/browse/SPARK-33273
> Project: Spark
> Issue Type: Bug
> Components: SQL, Tests
> Affects Versions: 3.1.0
> Reporter: Dongjoon Hyun
> Priority: Blocker
> Labels: correctness
> Attachments: failures.png
>
>
> -
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130369/testReport/org.apache.spark.sql.hive.thriftserver/ThriftServerQueryTestSuite/subquery_scalar_subquery_scalar_subquery_select_sql/
> {code}
> [info] - subquery/scalar-subquery/scalar-subquery-select.sql *** FAILED ***
> (3 seconds, 877 milliseconds)
> [info] Expected "[1]0 2017-05-04 01:01:0...", but got "[]0
> 2017-05-04 01:01:0..." Result did not match for query #3
> [info] SELECT (SELECT min(t3d) FROM t3) min_t3d,
> [info] (SELECT max(t2h) FROM t2) max_t2h
> [info] FROM t1
> [info] WHERE t1a = 'val1c' (ThriftServerQueryTestSuite.scala:197)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
