snleee commented on pull request #7739: URL: https://github.com/apache/pinot/pull/7739#issuecomment-966649366
As we discussed, I don't think that we should `evenly distribute null value across all reducers` because this will break the partitioning contract. Instead, I think that the data owner should correctly purge the data correctly not to have the null value for the column that they sort & partition on. As long as we do the key salting, it's probably the best thing we can do given the skewed data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
