EdisonWang created SPARK-27789:
----------------------------------
Summary: Use stopEarly in codegen of ColumnarBatchScan
Key: SPARK-27789
URL: https://issues.apache.org/jira/browse/SPARK-27789
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.4.3, 2.4.2, 2.4.1, 2.4.0
Reporter: EdisonWang
Suppose that we have a hive table like this
```sql("create table parquet_test (id int) using parquet")```, and our query
sql is `select id from parquet_test limit 10`.
With `spark.sql.hive.convertMetastoreParquet` set to false, the sql execution
will go into `InputAdapter`, in its codegen, it can use `stopEarly` to
accelerate local limit.
But if we set `spark.sql.hive.convertMetastoreParquet` to true, the sql
exectuion will go into `ColumnarBatchScan`, which didn't optimize local limit.
In this patch, We use `stopEarly` in `ColumnarBatchScan` as well
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
