jayzhan211 commented on issue #8512: URL: https://github.com/apache/arrow-datafusion/issues/8512#issuecomment-1853872555
I think it will still be an issue in datafusion 34. I found the array we got in `merge_batch` is `ListArray[ListArray[I64Array[1,2], I64Array[3,4], I64Array[5,6]]]`, so when we convert it to `scalar_vec` (`convert_array_to_scalar_vec`), we got the unexpected result. I'm not sure whether we need to fix scalar_vec conversion or the values we got before `merge_batch`. Currently, there is no test cover array cases where go through `merge_batch`. And, I think it is hard to support this test since we don't have a way to create array in CSV table which go through `merge_batch`. With normal sql table, it will be optimized that does not have the same workflow like CSV table (go through `update_batch`) I will take a look to fix this issue and find a way to add this kind of test if possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
