pepijnve commented on PR #16319:
URL: https://github.com/apache/datafusion/pull/16319#issuecomment-2956628323
Looking at the `clickbench_partitioned` outliers. Wrt the code changes in
this PR they seem pretty similar yet one has basically the opposite result of
the other. What's interesting is that total time is actually lower for the run.
Query 13 1.13x slower
```
SELECT "SearchPhrase", COUNT(DISTINCT "UserID") AS u FROM 'hits.parquet'
WHERE "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY u DESC LIMIT 10;
SortExec: TopK(fetch=10), expr=[u@1 DESC], preserve_partitioning=[false]
ProjectionExec: expr=[SearchPhrase@0 as SearchPhrase, count(alias1)@1 as u]
AggregateExec: mode=Single, gby=[SearchPhrase@0 as SearchPhrase],
aggr=[count(alias1)]
AggregateExec: mode=Single, gby=[SearchPhrase@1 as SearchPhrase,
UserID@0 as alias1], aggr=[]
CoalesceBatchesExec: target_batch_size=8192
FilterExec: SearchPhrase@1 !=
DataSourceExec: file_groups={1 group:
[[Users/pepijn/RustroverProjects/datafusion/benchmarks/data/hits.parquet:0..14779976446]]},
projection=[UserID, SearchPhrase], file_type=parquet,
predicate=SearchPhrase@39 != , pruning_predicate=SearchPhrase_null_count@2 !=
row_count@3 AND (SearchPhrase_min@0 != OR != SearchPhrase_max@1),
required_guarantees=[SearchPhrase not in ()]
```
Query 28: 1.10x faster
```
SELECT REGEXP_REPLACE("Referer", '^https?://(?:www\.)?([^/]+)/.*$', '\1') AS
k, AVG(length("Referer")) AS l, COUNT(*) AS c, MIN("Referer") FROM
'hits.parquet' WHERE "Referer" <> '' GROUP BY k HAVING COUNT(*) > 100000 ORDER
BY l DESC LIMIT 25;
SortExec: TopK(fetch=25), expr=[l@1 DESC], preserve_partitioning=[false]
ProjectionExec:
expr=[regexp_replace(hits.parquet.Referer,Utf8("^https?://(?:www\.)?([^/]+)/.*$"),Utf8("\1"))@0
as k, avg(character_length(hits.parquet.Referer))@1 as l, count(Int64(1))@2 as
c, min(hits.parquet.Referer)@3 as min(hits.parquet.Referer)]
CoalesceBatchesExec: target_batch_size=8192
FilterExec: count(Int64(1))@2 > 100000
AggregateExec: mode=Single, gby=[regexp_replace(Referer@0,
^https?://(?:www\.)?([^/]+)/.*$, \1) as
regexp_replace(hits.parquet.Referer,Utf8("^https?://(?:www\.)?([^/]+)/.*$"),Utf8("\1"))],
aggr=[avg(character_length(hits.parquet.Referer)), count(Int64(1)),
min(hits.parquet.Referer)]
CoalesceBatchesExec: target_batch_size=8192
FilterExec: Referer@0 !=
DataSourceExec: file_groups={1 group:
[[Users/pepijn/RustroverProjects/datafusion/benchmarks/data/hits.parquet:0..14779976446]]},
projection=[Referer], file_type=parquet, predicate=Referer@14 != ,
pruning_predicate=Referer_null_count@2 != row_count@3 AND (Referer_min@0 != OR
!= Referer_max@1), required_guarantees=[Referer not in ()]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
