LuciferYang commented on PR #56323: URL: https://github.com/apache/spark/pull/56323#issuecomment-4643163456
The reported case looks like a runtime fallback gap rather than a planner-selection problem. `BytesToBytesMap.Location.append` can append the current key, then fail while growing the table. The existing fallback only becomes visible on a later distinct key, when append returns false and `getAggregationBufferFromUnsafeRow` returns null. I would prefer fixing that runtime path directly. The key question is how to avoid or recover from the `growAndRehash` allocation that failed in the stack trace. If we expose a cannot-grow state from the unsafe map, the patch should also explain what behavior it adds beyond the existing `append=false -> buffer=null -> spill` path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
