SparkApplicationMaster commented on PR #17456: URL: https://github.com/apache/datafusion/pull/17456#issuecomment-3263235043
Some caveats: 1) Tried to implement type signature like this: ```rust Signature::arrays(2, Some(ListCoercion::FixedSizedListToList), Volatility::Immutable) ``` but it coerces list field to same type which is not acceptable for this function 2) This function requres to be confugurable, but this is not implemented yet in datafusion as I know Keys can be deduplicated or an error can be raised according to spark-conf So by default a more permissive deduplicating behaviour is implemented, as it is more useful and passes spark-tests TODO added where the future work needed when configuration will be possible -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
