Jay Booth wrote:
Did you set io.file.buffer.size (or whatever the property is) to a large value?
Just re-ran the benchmark with that bumped to 65536 (as proposed in http://www.cloudera.com/blog/tag/configuration/). The benchmark is still slower with jumbo frames than without (but difference was reduced a little by bumping io.file.buffer.size.
Also, even if you do set that, I'm not 100% sure that it will lead to use of the jumboframes, I'd have to take a look through the code to be sure. Another issue is that you may lose any networking performance gains as a result of increased buffer allocation costs and garbage collection.
I guess there must be something going on anyways - as there is clearly a performance drop-off, which surprised me, as lots of apps benefit significantly from jumbo frames.
Thanks for the feedback, -stephen -- Stephen Mulcahy, DI2, Digital Enterprise Research Institute, NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland http://di2.deri.ie http://webstar.deri.ie http://sindice.com
