Re: [I] Lots of code duplication in array_functions (`array_replace_all`, `array_replace_n`, `array_replace`, etc) [arrow-datafusion]

Sun, 03 Dec 2023 04:20:35 -0800 


alamb commented on issue #7988:
URL: 
https://github.com/apache/arrow-datafusion/issues/7988#issuecomment-1837464291

   > <img alt="Screenshot 2023-12-03 at 12 00 55" width="1402" 
src="https://private-user-images.githubusercontent.com/15869383/287496331-79f6ae3a-6362-4aae-9fa2-90b26944237a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTEiLCJleHAiOjE3MDE2MDYwMDYsIm5iZiI6MTcwMTYwNTcwNiwicGF0aCI6Ii8xNTg2OTM4My8yODc0OTYzMzEtNzlmNmFlM2EtNjM2Mi00YWFlLTlmYTItOTBiMjY5NDQyMzdhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFJV05KWUFYNENTVkVINTNBJTJGMjAyMzEyMDMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjMxMjAzVDEyMTUwNlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTc4YTE3MjgwZWFmMGFhNWEzZmMyZDU2ZjU3MDczNGQ0Y2NjMTk3MGMwOGM0YWI1NjVhZTRmODdkZTA2MjNhNmImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.UlClTLa3zZz_zrWPZifyhRslB2h7AJOLWflPUYGHhQo";>
   > `datafusion_common::scalar::ScalarValue::iter_to_array` is probably not 
the target function to cleanup ? Since it need to check the data types in 
Scalar and deal with them specifically.
   
   I think macros that generate specialized for for each scalar type could 
potentially be replaced
   
   
https://github.com/apache/arrow-datafusion/blob/075ff3ddfc78680d5da424ed63ffea1e38a6c57d/datafusion/common/src/scalar.rs#L1371-L1468
   
   This then makes a massive about of duplication:
   
https://github.com/apache/arrow-datafusion/blob/075ff3ddfc78680d5da424ed63ffea1e38a6c57d/datafusion/common/src/scalar.rs#L1544-L1765
   
   Specifically, rather than having specialized code for each possible list 
element type, perhaps we could have one code that accumulated the values 
(perhaps recursively calling iter_to_array) and one function that computes the 
offsets
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to