rspears74 commented on issue #8975:
URL:
https://github.com/apache/arrow-datafusion/issues/8975#issuecomment-1911036949
Reopening with new error after correcting the `iter_to_array` issue. This
seems to have something to do with some of the specifics of my `Aggregator`
implementations. I'm able to get a successful run using a simplified
`Aggregator` that calculates the sum of values and outputs a simple `Struct`
that looks like `{sum: 100.0}`.
However, with my T-Digest and other custom aggregator (output looks
something like: `[{"value": "AString", "count": 10},...]` I'm successful in
Datafusion 33, but get the following errors in 34:
```
thread 'tokio-runtime-worker' panicked at
.../.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow-data-49.0.0/src/transform/list.rs:36:69:
range end index 2 out of range for slice of length 1
```
I recently started [a
discussion](https://github.com/apache/arrow-datafusion/discussions/8974), which
includes the stack trace of this error.
One difference between my successful sum `Aggregator` and this one is that
the output of the sum `Aggregator` is always a `Struct` with something in it
(i.e. `{sum: 0.0}`). Conversely, when I successfully run this in Datafusion 33,
many of the results are empty lists (`[]`). I suspect this may have something
to do with the problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
