devanbenz commented on issue #12088: URL: https://github.com/apache/datafusion/issues/12088#issuecomment-2305139528
Also I'm wondering why on my hardware I'm not seeing a UnionExec for the
following plan:
`EXPLAIN SELECT "hits.parquet"."OS" AS os, COUNT(1) FROM 'hits.parquet'
WHERE to_timestamp("hits.parquet"."EventTime") >= to_timestamp(200) AND
to_timestamp("hits.parquet"."EventTime") < to_timestamp(700) AND
"hits.parquet"."RegionID" = 839 GROUP BY os ORDER BY os ASC;`
I assume its just dependent on the hardware since its part of the physical
planner. I'm seeing the following output for this:
```
+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type | plan
|
+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan | Sort: os ASC NULLS LAST
|
| | Projection: hits.parquet.OS AS os, count(Int64(1))
|
| | Aggregate: groupBy=[[hits.parquet.OS]],
aggr=[[count(Int64(1))]]
|
| | Projection: hits.parquet.OS
|
| | Filter: __common_expr_4 >=
TimestampNanosecond(200000000000, None) AND __common_expr_4 <
TimestampNanosecond(700000000000, None) AND hits.parquet.RegionID = Int32(839)
|
| | Projection: to_timestamp(hits.parquet.EventTime)
AS __common_expr_4, hits.parquet.RegionID, hits.parquet.OS
|
| | TableScan: hits.parquet projection=[EventTime,
RegionID, OS], partial_filters=[to_timestamp(hits.parquet.EventTime) >=
TimestampNanosecond(200000000000, None), to_timestamp(hits.parquet.EventTime) <
TimestampNanosecond(700000000000, None), hits.parquet.RegionID = Int32(839)]
|
| physical_plan | SortPreservingMergeExec: [os@0 ASC NULLS LAST]
|
| | SortExec: expr=[os@0 ASC NULLS LAST],
preserve_partitioning=[true]
|
| | ProjectionExec: expr=[OS@0 as os, count(Int64(1))@1 as
count(Int64(1))]
|
| | AggregateExec: mode=FinalPartitioned, gby=[OS@0 as
OS], aggr=[count(Int64(1))]
|
| | CoalesceBatchesExec: target_batch_size=8192
|
| | RepartitionExec: partitioning=Hash([OS@0], 10),
input_partitions=10
|
| | AggregateExec: mode=Partial, gby=[OS@0 as OS],
aggr=[count(Int64(1))]
|
| | ProjectionExec: expr=[OS@2 as OS]
|
| | CoalesceBatchesExec:
target_batch_size=8192
|
| | FilterExec: __common_expr_4@0 >=
200000000000 AND __common_expr_4@0 < 700000000000 AND RegionID@1 = 839
|
| | ProjectionExec:
expr=[to_timestamp(EventTime@0) as __common_expr_4, RegionID@1 as RegionID,
OS@2 as OS]
|
| | ParquetExec: file_groups={10 groups:
[[Users/devan/Documents/OSS/datafusion/benchmarks/data/hits.parquet:0..1477997645],
[Users/devan/Documents/OSS/datafusion/benchmarks/data/hits.parquet:1477997645..2955995290],
[Users/devan/Documents/OSS/datafusion/benchmarks/data/hits.parquet:2955995290..4433992935],
[Users/devan/Documents/OSS/datafusion/benchmarks/data/hits.parquet:4433992935..5911990580],
[Users/devan/Documents/OSS/datafusion/benchmarks/data/hits.parquet:5911990580..7389988225],
...]}, projection=[EventTime, RegionID, OS],
predicate=to_timestamp(EventTime@4) >= 200000000000 AND
to_timestamp(EventTime@4) < 700000000000 AND RegionID@8 = 839,
pruning_predicate=CASE WHEN RegionID_null_count@2 = RegionID_row_count@3 THEN
false ELSE RegionID_min@0 <= 839 AND 839 <= RegionID_max@1 END,
required_guarantees=[RegionID in (839)] |
| |
|
+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
