rich7420 commented on PR #6107: URL: https://github.com/apache/datafusion-comet/pull/6107#issuecomment-6875545428
The outer slowdown comes from the gather, not the placeholders: one NULL or empty row stops a chunk from being sliced, so all its entries go through `take`. I tried copying contiguous runs instead, but the map-shaped struct only improved by ~10% (within noise), and this path is shared with array `explode_outer`, so I'd rather handle it in a follow-up PR. Under load, the outer variants still used less CPU than the baseline. The chain cost isn't map-specific: array→array chains, already native on main, show the same gap (99.6 vs 83.5 ms). The lower generator here also carries a string column (`region`), so a fixed-width shortcut wouldn't apply. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
