jorgecarleitao commented on pull request #8222: URL: https://github.com/apache/arrow/pull/8222#issuecomment-699274727
I agree with your reasoning. AFAIK, `scalar` in the context of Arrow is not a mathematical scalar like `f64`: `scalar` is a "1-element representation of an Arrow Array" (see [pyarrow](https://arrow.apache.org/docs/python/api/arrays.html#scalars)). In this context, `ScalarValue::List(Arc<ScalarValue>)` is not implemented in Rust Arrow, but IMO we should have one, because there is a corresponding `ListArray` in the spec. With this in mind, another idea is to map your second idea to: * wrap those 2 (N in general) fields in a `ScalarValue::List(ScalarValue::List)`. I think that this was the original intention of those structures in the Arrow specs: to compose complex structures. This was also the reasoning behind emitting `ScalarValue`: an accumulator's state is a single row in the group of accumulators, but each state can hold arbitrarily complex datum. This would keep us using Arrow as the medium of holding and transmitting (immutable) data, and would keep the execution's schema a bit more predictable. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
