jorisvandenbossche commented on PR #14867:
URL: https://github.com/apache/arrow/pull/14867#issuecomment-1363323864
To illustrate what I mean, using a small example (and using this branch):
```
In [7]: table = pa.table({'key': ['a', 'b', 'a', 'b', 'a', 'b'], 'col':
range(6)})
In [8]: table = pa.Table.from_batches(table.to_batches(max_chunksize=3))
In [9]: table.to_pandas()
Out[9]:
key col
0 a 0
1 b 1
2 a 2
3 b 3
4 a 4
5 b 5
In [10]: table.group_by('key').aggregate([('col', 'sum')]).to_pandas()
Out[10]:
col_sum key
0 2 a
1 1 b
2 8 b
3 4 a
```
I created a table consisting of multiple chunks, and then the result is
incorrect as it is the concatentation of individual results of each chunk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
