alamb commented on code in PR #8315:
URL: https://github.com/apache/arrow-datafusion/pull/8315#discussion_r1407830562
##########
datafusion/sqllogictest/test_files/limit.slt:
##########
@@ -379,6 +379,110 @@ SELECT COUNT(*) FROM (SELECT a FROM t1 WHERE a > 3 LIMIT
3 OFFSET 6);
----
1
+# generate BIGINT data from 1 to 1000
+statement ok
+CREATE TABLE t1000 (i BIGINT) AS
+WITH t AS (VALUES (0), (0), (0), (0), (0), (0), (0), (0), (0), (0))
+SELECT ROW_NUMBER() OVER (PARTITION BY t1.column1) FROM t t1, t t2, t t3;
Review Comment:
I double checked that this actually generates partitioned input and it does
indeed.
You can see on my local setup that the MemoryExec has multiple partitions
```
❯ explain select distinct i from t1000;
+---------------+-----------------------------------------------------------------------------------------------------+
| plan_type | plan
|
+---------------+-----------------------------------------------------------------------------------------------------+
| logical_plan | Aggregate: groupBy=[[t1000.i]], aggr=[[]]
|
| | TableScan: t1000 projection=[i]
|
| physical_plan | AggregateExec: mode=FinalPartitioned, gby=[i@0 as i],
aggr=[] |
| | CoalesceBatchesExec: target_batch_size=8192
|
| | RepartitionExec: partitioning=Hash([i@0], 16),
input_partitions=16 |
| | AggregateExec: mode=Partial, gby=[i@0 as i], aggr=[]
|
| | MemoryExec: partitions=16, partition_sizes=[1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1] |
| |
|
+---------------+-----------------------------------------------------------------------------------------------------+
```
##########
datafusion/sqllogictest/test_files/limit.slt:
##########
@@ -379,6 +379,110 @@ SELECT COUNT(*) FROM (SELECT a FROM t1 WHERE a > 3 LIMIT
3 OFFSET 6);
----
1
+# generate BIGINT data from 1 to 1000
+statement ok
+CREATE TABLE t1000 (i BIGINT) AS
+WITH t AS (VALUES (0), (0), (0), (0), (0), (0), (0), (0), (0), (0))
+SELECT ROW_NUMBER() OVER (PARTITION BY t1.column1) FROM t t1, t t2, t t3;
Review Comment:
I double checked that this actually generates partitioned input and it does
indeed.
You can see on my local setup that the MemoryExec has multiple partitions
```
❯ explain select distinct i from t1000;
+---------------+-----------------------------------------------------------------------------------------------------+
| plan_type | plan
|
+---------------+-----------------------------------------------------------------------------------------------------+
| logical_plan | Aggregate: groupBy=[[t1000.i]], aggr=[[]]
|
| | TableScan: t1000 projection=[i]
|
| physical_plan | AggregateExec: mode=FinalPartitioned, gby=[i@0 as i],
aggr=[] |
| | CoalesceBatchesExec: target_batch_size=8192
|
| | RepartitionExec: partitioning=Hash([i@0], 16),
input_partitions=16 |
| | AggregateExec: mode=Partial, gby=[i@0 as i], aggr=[]
|
| | MemoryExec: partitions=16, partition_sizes=[1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1] |
| |
|
+---------------+-----------------------------------------------------------------------------------------------------+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
