[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2019-01-25 Thread Yuval Degani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752451#comment-16752451 ] Yuval Degani commented on SPARK-9: -- Great questions, [~tgraves] * SparkRDMA (starting version

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2019-01-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752356#comment-16752356 ] Thomas Graves commented on SPARK-9: --- This is interesting, a few questions * I'm assuming all

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2019-01-25 Thread Peter Rudenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752282#comment-16752282 ] Peter Rudenko commented on SPARK-9: --- [~S71955] - there's no yet PR to include SparkRDMA to

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2019-01-25 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752044#comment-16752044 ] Sujith commented on SPARK-9: @[~yuvaldeg]  May i know where i can find PR related to new 

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2018-01-11 Thread Yuval Degani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322747#comment-16322747 ] Yuval Degani commented on SPARK-9: -- [~byronyi], thank you for kicking off the discussion again.

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2018-01-11 Thread Bairen Yi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322129#comment-16322129 ] Bairen Yi commented on SPARK-9: --- [~srowen] It is very clear to me that no GPL code is introduced in

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2018-01-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322055#comment-16322055 ] Sean Owen commented on SPARK-9: --- [~byronyi] what PR? There appears to be an unresolved question

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2018-01-11 Thread Bairen Yi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321960#comment-16321960 ] Bairen Yi commented on SPARK-9: --- Why this PR is being blocked? Seems there is no issue with respect

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-20 Thread Rob Vesse (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212462#comment-16212462 ] Rob Vesse commented on SPARK-9: --- [~yuvaldeg], thanks for providing additional clarifications If a

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-19 Thread Yuval Degani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211038#comment-16211038 ] Yuval Degani commented on SPARK-9: -- [~rvesse], thanks for taking the time to review. Regarding

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-18 Thread Rob Vesse (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209396#comment-16209396 ] Rob Vesse commented on SPARK-9: --- I'd be interested to know more about what performance testing you

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-13 Thread Yuval Degani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204041#comment-16204041 ] Yuval Degani commented on SPARK-9: -- [~jerryshao], thanks for clearing your point. I agree that

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202900#comment-16202900 ] Saisai Shao commented on SPARK-9: - {quote} I don't think that limited familiarity with a new

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-12 Thread Yuval Degani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202368#comment-16202368 ] Yuval Degani commented on SPARK-9: -- Good point [~jerryshao]. Regarding testing on a machines

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-12 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201950#comment-16201950 ] Saisai Shao commented on SPARK-9: - My concern is about how to maintain this code in the

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-11 Thread Yuval Degani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200664#comment-16200664 ] Yuval Degani commented on SPARK-9: -- Yes, transitioning from a ShuffleManager to a

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-11 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199987#comment-16199987 ] Liang-Chi Hsieh commented on SPARK-9: - Is any common components among the RDMA shuffle engine

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-11 Thread Andy Huang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199942#comment-16199942 ] Andy Huang commented on SPARK-9: [~r...@databricks.com] RDMA could be a very important network

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-10 Thread Yuval Degani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199293#comment-16199293 ] Yuval Degani commented on SPARK-9: -- We already published an open-source package that implements

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199242#comment-16199242 ] Sean Owen commented on SPARK-9: --- It's still something that should start as a package. I think that

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-10 Thread Yuval Degani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199052#comment-16199052 ] Yuval Degani commented on SPARK-9: -- [~srowen], [~viirya], thanks for your response. Regarding

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-10 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198716#comment-16198716 ] Liang-Chi Hsieh commented on SPARK-9: - As we have the pluggable mechanism to set up external

[jira] [Commented] (SPARK-22229) SPIP: RDMA Accelerated Shuffle Engine

2017-10-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198284#comment-16198284 ] Sean Owen commented on SPARK-9: --- RDMA is specialized networking hardware right? I don't think this