cloud-fan commented on pull request #29810: URL: https://github.com/apache/spark/pull/29810#issuecomment-961268006
There is inevitable randomness in the input of aggregate functions, because the shuffle reader may produce data with random orders, and we are not able to completely eliminate this randomness. For example, even `sum` has randomness, as adding up floating values with different orders can lead to slightly different results. I don't think `first/last/collect` has a significant difference and +1 to mark them as deterministic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
