I feel like I've seen this too and it's just a bug. You're not running out of memory.
Are you also setting io.sort.factor? that can help too. You might try as high as 100. Also have you tried a Combiner? if you can apply it it should help too as it is designed to reduce the amount of stuff spilled.
