MaxGekk commented on a change in pull request #27366:
URL: https://github.com/apache/spark/pull/27366#discussion_r455205295
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonBenchmark.scala
##########
@@ -508,6 +548,9 @@ object JsonBenchmark extends SqlBasedBenchmark {
jsonInDS(50 * 1000 * 1000, numIters)
jsonInFile(50 * 1000 * 1000, numIters)
datetimeBenchmark(rowsNum = 10 * 1000 * 1000, numIters)
+ // Benchmark pushdown filters that refer to top-level columns.
+ // TODO: Add benchmarks for filters with nested column attributes.
Review comment:
@dbtsai @dongjoon-hyun There is an umbrella ticket for nested fields
https://issues.apache.org/jira/browse/SPARK-25556 but it mentions only Parquet
and it has been resolved already. Should I add a sub-task there and re-open it,
or open separate JIRA?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
