[jira] [Commented] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112313#comment-16112313 ] Robert Schmidtke commented on MAPREDUCE-6923: - Hi Ravi, my guess is that since {{trans}}

[jira] [Comment Edited] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112313#comment-16112313 ] Robert Schmidtke edited comment on MAPREDUCE-6923 at 8/3/17 7:14 AM:

[jira] [Comment Edited] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112313#comment-16112313 ] Robert Schmidtke edited comment on MAPREDUCE-6923 at 8/3/17 7:12 AM:

[jira] [Updated] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Schmidtke updated MAPREDUCE-6923: Attachment: MAPREDUCE-6923.00.patch Initial patch from {{trunk}} that uses the

[jira] [Updated] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Schmidtke updated MAPREDUCE-6923: Attachment: MAPREDUCE-6923.01.patch Fixed indentation for {{checkstyle}} >

[jira] [Commented] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114083#comment-16114083 ] Robert Schmidtke commented on MAPREDUCE-6923: - I am using bytecode instrumentation to log

[jira] [Comment Edited] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114099#comment-16114099 ] Robert Schmidtke edited comment on MAPREDUCE-6923 at 8/4/17 8:18 AM:

[jira] [Comment Edited] (MAPREDUCE-6923) Optimize MapReduce Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121109#comment-16121109 ] Robert Schmidtke edited comment on MAPREDUCE-6923 at 8/10/17 6:07 AM:

[jira] [Commented] (MAPREDUCE-6923) Optimize MapReduce Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121109#comment-16121109 ] Robert Schmidtke commented on MAPREDUCE-6923: - Hi Ravi, {quote} When {{shuffleBufferSize

[jira] [Comment Edited] (MAPREDUCE-6923) Optimize MapReduce Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121109#comment-16121109 ] Robert Schmidtke edited comment on MAPREDUCE-6923 at 8/10/17 6:06 AM:

[jira] [Comment Edited] (MAPREDUCE-6923) Optimize MapReduce Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121109#comment-16121109 ] Robert Schmidtke edited comment on MAPREDUCE-6923 at 8/10/17 6:06 AM:

[jira] [Commented] (MAPREDUCE-6923) Optimize MapReduce Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121117#comment-16121117 ] Robert Schmidtke commented on MAPREDUCE-6923: - Thanks for coming back to my comments.

[jira] [Created] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

Robert Schmidtke created MAPREDUCE-6923: --- Summary: YARN Shuffle I/O for small partitions Key: MAPREDUCE-6923 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6923 Project: Hadoop Map/Reduce

[jira] [Updated] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Schmidtke updated MAPREDUCE-6923: Description: When a job configuration results in small partitions read by each

[jira] [Comment Edited] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112643#comment-16112643 ] Robert Schmidtke edited comment on MAPREDUCE-6923 at 8/3/17 12:28 PM:

[jira] [Commented] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions

[ https://issues.apache.org/jira/browse/MAPREDUCE-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112643#comment-16112643 ] Robert Schmidtke commented on MAPREDUCE-6923: - Fyi I have benchmarked another version