[jira] [Updated] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL
[ https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Lyubimov updated MAHOUT-1570: - Sprint: Jan/Feb-2016 (was: Mar/Apr-2016) > Adding support for Apache Flink as a backend for the Mahout DSL > --- > > Key: MAHOUT-1570 > URL: https://issues.apache.org/jira/browse/MAHOUT-1570 > Project: Mahout > Issue Type: Improvement > Components: Flink >Affects Versions: 0.11.2 >Reporter: Till Rohrmann >Assignee: Suneel Marthi > Labels: DSL, flink, scala > Fix For: 0.12.0 > > > With the finalized abstraction of the Mahout DSL plans from the backend > operations (MAHOUT-1529), it should be possible to integrate further backends > for the Mahout DSL. Apache Flink would be a suitable candidate to act as a > good execution backend. > With respect to the implementation, the biggest difference between Spark and > Flink at the moment is probably the incremental rollout of plans, which is > triggered by Spark's actions and which is not supported by Flink yet. > However, the Flink community is working on this issue. For the moment, it > should be possible to circumvent this problem by writing intermediate results > required by an action to HDFS and reading from there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL
[ https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Lyubimov updated MAHOUT-1570: - Sprint: Mar/Apr-2016 (was: Jan/Feb-2016) > Adding support for Apache Flink as a backend for the Mahout DSL > --- > > Key: MAHOUT-1570 > URL: https://issues.apache.org/jira/browse/MAHOUT-1570 > Project: Mahout > Issue Type: Improvement > Components: Flink >Affects Versions: 0.11.2 >Reporter: Till Rohrmann >Assignee: Suneel Marthi > Labels: DSL, flink, scala > Fix For: 0.12.0 > > > With the finalized abstraction of the Mahout DSL plans from the backend > operations (MAHOUT-1529), it should be possible to integrate further backends > for the Mahout DSL. Apache Flink would be a suitable candidate to act as a > good execution backend. > With respect to the implementation, the biggest difference between Spark and > Flink at the moment is probably the incremental rollout of plans, which is > triggered by Spark's actions and which is not supported by Flink yet. > However, the Flink community is working on this issue. For the moment, it > should be possible to circumvent this problem by writing intermediate results > required by an action to HDFS and reading from there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL
[ https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Lyubimov updated MAHOUT-1570: - Sprint: Jan/Feb-2016 (was: Nov/Dec-2015) > Adding support for Apache Flink as a backend for the Mahout DSL > --- > > Key: MAHOUT-1570 > URL: https://issues.apache.org/jira/browse/MAHOUT-1570 > Project: Mahout > Issue Type: Improvement > Components: Flink >Affects Versions: 0.11.2 >Reporter: Till Rohrmann >Assignee: Suneel Marthi > Labels: DSL, flink, scala > Fix For: 0.12.0 > > > With the finalized abstraction of the Mahout DSL plans from the backend > operations (MAHOUT-1529), it should be possible to integrate further backends > for the Mahout DSL. Apache Flink would be a suitable candidate to act as a > good execution backend. > With respect to the implementation, the biggest difference between Spark and > Flink at the moment is probably the incremental rollout of plans, which is > triggered by Spark's actions and which is not supported by Flink yet. > However, the Flink community is working on this issue. For the moment, it > should be possible to circumvent this problem by writing intermediate results > required by an action to HDFS and reading from there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL
[ https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated MAHOUT-1570: -- Component/s: Flink > Adding support for Apache Flink as a backend for the Mahout DSL > --- > > Key: MAHOUT-1570 > URL: https://issues.apache.org/jira/browse/MAHOUT-1570 > Project: Mahout > Issue Type: Improvement > Components: Flink >Affects Versions: 0.11.2 >Reporter: Till Rohrmann >Assignee: Suneel Marthi > Labels: DSL, flink, scala > Fix For: 0.12.0 > > > With the finalized abstraction of the Mahout DSL plans from the backend > operations (MAHOUT-1529), it should be possible to integrate further backends > for the Mahout DSL. Apache Flink would be a suitable candidate to act as a > good execution backend. > With respect to the implementation, the biggest difference between Spark and > Flink at the moment is probably the incremental rollout of plans, which is > triggered by Spark's actions and which is not supported by Flink yet. > However, the Flink community is working on this issue. For the moment, it > should be possible to circumvent this problem by writing intermediate results > required by an action to HDFS and reading from there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL
[ https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated MAHOUT-1570: -- Affects Version/s: 0.11.2 > Adding support for Apache Flink as a backend for the Mahout DSL > --- > > Key: MAHOUT-1570 > URL: https://issues.apache.org/jira/browse/MAHOUT-1570 > Project: Mahout > Issue Type: Improvement > Components: Flink >Affects Versions: 0.11.2 >Reporter: Till Rohrmann >Assignee: Suneel Marthi > Labels: DSL, flink, scala > Fix For: 0.12.0 > > > With the finalized abstraction of the Mahout DSL plans from the backend > operations (MAHOUT-1529), it should be possible to integrate further backends > for the Mahout DSL. Apache Flink would be a suitable candidate to act as a > good execution backend. > With respect to the implementation, the biggest difference between Spark and > Flink at the moment is probably the incremental rollout of plans, which is > triggered by Spark's actions and which is not supported by Flink yet. > However, the Flink community is working on this issue. For the moment, it > should be possible to circumvent this problem by writing intermediate results > required by an action to HDFS and reading from there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL
[ https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Lyubimov updated MAHOUT-1570: - Sprint: Nov/Dec-2015 (was: Sep/Oct-2015) > Adding support for Apache Flink as a backend for the Mahout DSL > --- > > Key: MAHOUT-1570 > URL: https://issues.apache.org/jira/browse/MAHOUT-1570 > Project: Mahout > Issue Type: Improvement >Reporter: Till Rohrmann >Assignee: Suneel Marthi > Labels: DSL, flink, scala > Fix For: 0.12.0 > > > With the finalized abstraction of the Mahout DSL plans from the backend > operations (MAHOUT-1529), it should be possible to integrate further backends > for the Mahout DSL. Apache Flink would be a suitable candidate to act as a > good execution backend. > With respect to the implementation, the biggest difference between Spark and > Flink at the moment is probably the incremental rollout of plans, which is > triggered by Spark's actions and which is not supported by Flink yet. > However, the Flink community is working on this issue. For the moment, it > should be possible to circumvent this problem by writing intermediate results > required by an action to HDFS and reading from there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL
[ https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Lyubimov updated MAHOUT-1570: - Sprint: Sep/Oct-2015 > Adding support for Apache Flink as a backend for the Mahout DSL > --- > > Key: MAHOUT-1570 > URL: https://issues.apache.org/jira/browse/MAHOUT-1570 > Project: Mahout > Issue Type: Improvement >Reporter: Till Rohrmann >Assignee: Alexey Grigorev > Labels: DSL, flink, scala > Fix For: 0.11.1 > > > With the finalized abstraction of the Mahout DSL plans from the backend > operations (MAHOUT-1529), it should be possible to integrate further backends > for the Mahout DSL. Apache Flink would be a suitable candidate to act as a > good execution backend. > With respect to the implementation, the biggest difference between Spark and > Flink at the moment is probably the incremental rollout of plans, which is > triggered by Spark's actions and which is not supported by Flink yet. > However, the Flink community is working on this issue. For the moment, it > should be possible to circumvent this problem by writing intermediate results > required by an action to HDFS and reading from there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL
[ https://issues.apache.org/jira/browse/MAHOUT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated MAHOUT-1570: -- Assignee: Sebastian Schelter Summary: Adding support for Apache Flink as a backend for the Mahout DSL (was: Adding support for Stratosphere as a backend for the Mahout DSL) Sebastian, Till: Do we keep this open? It would be good to have Flink integration to Mahout IMO. Adding support for Apache Flink as a backend for the Mahout DSL --- Key: MAHOUT-1570 URL: https://issues.apache.org/jira/browse/MAHOUT-1570 Project: Mahout Issue Type: Improvement Reporter: Till Rohrmann Assignee: Sebastian Schelter Labels: DSL, flink, scala With the finalized abstraction of logical Mahout DSL plans from the backend operations (MAHOUT-1529), it should be possible to integrate further backends for the Mahout DSL. I like to evaluate to what extent this can already be done for Stratosphere and what can be done to solve possibly occuring problems. The biggest difference between Spark and Stratosphere at the moment is probably the incremental rollout of plans, which is triggered by Spark's actions and which is not supported by Stratosphere yet. However, the Stratosphere team is working on this issue. For the moment, it should be possible to circumvent this problem by writing intermediate results required by an action to HDFS and reading from there. Thus, this work shall rather be considered as a proof of concept than a strongly efficient implementation and has the purpose to evaluate where the logical plan abstraction might be refined in order to support different backends. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
