[jira] [Commented] (SPARK-15074) Spark shuffle service bottlenecked while fetching large amount of intermediate data

2016-05-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275339#comment-15275339 ] Reynold Xin commented on SPARK-15074: - transferTo is actually the part where it's wai

[jira] [Commented] (SPARK-15074) Spark shuffle service bottlenecked while fetching large amount of intermediate data

2016-05-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273416#comment-15273416 ] Apache Spark commented on SPARK-15074: -- User 'sitalkedia' has created a pull request

[jira] [Commented] (SPARK-15074) Spark shuffle service bottlenecked while fetching large amount of intermediate data

2016-05-05 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273241#comment-15273241 ] Sital Kedia commented on SPARK-15074: - Okay, I made a change to cache the index file

[jira] [Commented] (SPARK-15074) Spark shuffle service bottlenecked while fetching large amount of intermediate data

2016-05-02 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15268063#comment-15268063 ] Sital Kedia commented on SPARK-15074: - Sure, I will do that. Thanks for the quick res

[jira] [Commented] (SPARK-15074) Spark shuffle service bottlenecked while fetching large amount of intermediate data

2016-05-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15268060#comment-15268060 ] Reynold Xin commented on SPARK-15074: - If you want to prototype that and do some test

[jira] [Commented] (SPARK-15074) Spark shuffle service bottlenecked while fetching large amount of intermediate data

2016-05-02 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15268039#comment-15268039 ] Sital Kedia commented on SPARK-15074: - Since the index files contain 8 bytes (Long) p

[jira] [Commented] (SPARK-15074) Spark shuffle service bottlenecked while fetching large amount of intermediate data

2016-05-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15267932#comment-15267932 ] Reynold Xin commented on SPARK-15074: - Probably a good idea to explore. How big is th

[jira] [Commented] (SPARK-15074) Spark shuffle service bottlenecked while fetching large amount of intermediate data

2016-05-02 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15267659#comment-15267659 ] Sital Kedia commented on SPARK-15074: - [~rxin], [~srowen] - Please let me know what y