Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19344#discussion_r141836921
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
---
@@ -66,24 +64,11 @@ object TPCDSQueryBenchmark extends Logging {
classLoader = Thread.currentThread().getContextClassLoader)
// This is an indirect hack to estimate the size of each query's
input by traversing the
- // logical plan and adding up the sizes of all tables that appear in
the plan. Note that this
- // currently doesn't take WITH subqueries into account which might
lead to fairly inaccurate
- // per-row processing time for those cases.
+ // logical plan and adding up the sizes of all tables that appear in
the plan.
val queryRelations = scala.collection.mutable.HashSet[String]()
- spark.sql(queryString).queryExecution.logical.map {
- case UnresolvedRelation(t: TableIdentifier) =>
- queryRelations.add(t.table)
- case lp: LogicalPlan =>
- lp.expressions.foreach { _ foreach {
- case subquery: SubqueryExpression =>
- subquery.plan.foreach {
- case UnresolvedRelation(t: TableIdentifier) =>
- queryRelations.add(t.table)
- case _ =>
- }
- case _ =>
- }
- }
+ spark.sql(queryString).queryExecution.analyzed.map {
+ case SubqueryAlias(name, _: LogicalRelation) =>
+ queryRelations.add(name)
--- End diff --
IIUC ditto; `HiveTableRelation` never happens here.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
