ZhangHuiGui commented on issue #41923:
URL: https://github.com/apache/arrow/issues/41923#issuecomment-2143796671

   > However, some code implicit shows that it might contains chunked array, 
see:
   
   In the expression system, ChunkedArray can be processed normally as input, 
and for some kernel functions, chunked array will be returned as output.
   
   If the input type is chunked array, chunked array is returned in 
`ExecuteScalarExpression` as follows:
   1. `NullHandling::COMPUTED_NO_PREALLOCATE` is registered when `AddKernel` is 
added
   2. The output type of kernel function is `nested-type` or `dictionary-type`
   3. The first two cases are not satisfied, but `can_write_into_slices = 
false` is set.
   
   `ExecuteScalarExpression`'s processing logic for chunked array is as follows:
   1. ExecBatch mainly saves `vector<Datum>`, and Datum can be chunked-array 
type.
   2. Chunked array is processed in the kernel of ExecuteScalarExpression. The 
processing logic is mainly in [1]. There are three processing scenarios:
    1). If `preallocating_all_buffers_ = false`, that is, we do not enable 
preallocation (`NullHandling::COMPUTED_NO_PREALLOCATE` or nested-types or 
dictionary-type), the chunks in chunked-array will be processed one by one and 
returned as chunked-array. [2]
    2). If `preallocate_contiguous_ = true`, it means that the output type of 
this kernel function supports preallocation (non-nested types), the chunks in 
chunked-array will be processed one by one through span_iterator_, and an array 
will be concatenated and returned. [3]
   3). If `preallocate_contiguous_ = false`, the processing logic is the same 
as `1)`, and chunked-array will be returned. [4]
   
   [1] 
https://github.com/apache/arrow/blob/0d9d699250ad98cdcc5c99735e1379b1e79bc195/cpp/src/arrow/compute/exec.cc#L424C1-L439C6
   
   [2] 
https://github.com/apache/arrow/blob/0d9d699250ad98cdcc5c99735e1379b1e79bc195/cpp/src/arrow/compute/exec.cc#L907C1-L927C6
   
   [3] 
https://github.com/apache/arrow/blob/0d9d699250ad98cdcc5c99735e1379b1e79bc195/cpp/src/arrow/compute/exec.cc#L856C1-L861C8
   
   [4] 
https://github.com/apache/arrow/blob/0d9d699250ad98cdcc5c99735e1379b1e79bc195/cpp/src/arrow/compute/exec.cc#L869C1-L874C9


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to