[jira] [Comment Edited] (SPARK-6417) Add Linear Programming algorithm
[ https://issues.apache.org/jira/browse/SPARK-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15731333#comment-15731333 ] Ehsan Mohyedin Kermani edited comment on SPARK-6417 at 12/15/16 6:12 PM: - Here's my implementation inspired by Spark-TFOCS design: https://github.com/ehsanmok/spark-lp Please see the README file. Also detailed results are provided in chapter 4 of my thesis here: https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0340337 was (Author: ehsan mohyedin kermani): Here's my implementation inspired by Spark-TFOCS design: https://github.com/ehsanmok/spark-lp Empirical results will be added very soon. > Add Linear Programming algorithm > - > > Key: SPARK-6417 > URL: https://issues.apache.org/jira/browse/SPARK-6417 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Fan Jiang > Labels: features > > Linear programming is the problem of finding a vector x that minimizes a > linear function fTx subject to linear constraints: > minxfTx > such that one or more of the following hold: A·x ≤ b, Aeq·x = beq, l ≤ x ≤ u. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-6417) Add Linear Programming algorithm
[ https://issues.apache.org/jira/browse/SPARK-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15731333#comment-15731333 ] Ehsan Mohyedin Kermani edited comment on SPARK-6417 at 12/8/16 6:54 AM: Here's my implementation inspired by Spark-TFOCS design: https://github.com/ehsanmok/spark-lp Empirical results will be added very soon. was (Author: ehsan mohyedin kermani): Here's my implementation inspired by Spark-TFOCS design: https://github.com/ehsanmok/spark-lp > Add Linear Programming algorithm > - > > Key: SPARK-6417 > URL: https://issues.apache.org/jira/browse/SPARK-6417 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Fan Jiang > Labels: features > > Linear programming is the problem of finding a vector x that minimizes a > linear function fTx subject to linear constraints: > minxfTx > such that one or more of the following hold: A·x ≤ b, Aeq·x = beq, l ≤ x ≤ u. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6417) Add Linear Programming algorithm
[ https://issues.apache.org/jira/browse/SPARK-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15731333#comment-15731333 ] Ehsan Mohyedin Kermani commented on SPARK-6417: --- Here's my implementation inspired by Spark-TFOCS design: https://github.com/ehsanmok/spark-lp > Add Linear Programming algorithm > - > > Key: SPARK-6417 > URL: https://issues.apache.org/jira/browse/SPARK-6417 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Fan Jiang > Labels: features > > Linear programming is the problem of finding a vector x that minimizes a > linear function fTx subject to linear constraints: > minxfTx > such that one or more of the following hold: A·x ≤ b, Aeq·x = beq, l ≤ x ≤ u. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-6417) Add Linear Programming algorithm
[ https://issues.apache.org/jira/browse/SPARK-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ehsan Mohyedin Kermani updated SPARK-6417: -- Comment: was deleted (was: As I am working on the implementation, I have some serious concerns/questions. 1) Since I am heavily using distributed matrix abstractions provided in linalg, I realized it's impossible for me to submit it as a separate package when the implementation is completed. I don't think it's possible to define new abstractions like Aaron Staple's TFOCS package, for this task. 2) Adding some functionalities like transforming a local matrix to a BlockMatrix is necessary, as the underlying matrix is a distributed BlockMatrix and that's the only distributed matrix data structure with minimal matrix operations suitable for this task (any suggestions?) 3) Because, there's no literature on LP in mapReduce, my only approach is primal-dual IPM but with underlying distributed matrix data structures, so can that be called a distributed LP solver? if so in what degree? Please let me know your opinion and it'd be great to have someone who is willing to do the code review and guide me before any submission as I'm relatively new to Scala. ) > Add Linear Programming algorithm > - > > Key: SPARK-6417 > URL: https://issues.apache.org/jira/browse/SPARK-6417 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Fan Jiang > Labels: features > > Linear programming is the problem of finding a vector x that minimizes a > linear function fTx subject to linear constraints: > minxfTx > such that one or more of the following hold: A·x ≤ b, Aeq·x = beq, l ≤ x ≤ u. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Closed] (SPARK-13788) CholeskyDecomposition has side effects!
[ https://issues.apache.org/jira/browse/SPARK-13788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ehsan Mohyedin Kermani closed SPARK-13788. -- Resolution: Not A Problem > CholeskyDecomposition has side effects! > --- > > Key: SPARK-13788 > URL: https://issues.apache.org/jira/browse/SPARK-13788 > Project: Spark > Issue Type: Bug > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani > > Current CholeskyDecomposition implementation calling from lapack has side > effects due to mutating the array A representing the matrix in packed storage > format (see http://www.netlib.org/lapack/lug/node123.html ) and bx the known > right hand side values which should be captured by cloning arrays first! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13788) CholeskyDecomposition has side effects!
[ https://issues.apache.org/jira/browse/SPARK-13788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189700#comment-15189700 ] Ehsan Mohyedin Kermani commented on SPARK-13788: Please see my comment in github. > CholeskyDecomposition has side effects! > --- > > Key: SPARK-13788 > URL: https://issues.apache.org/jira/browse/SPARK-13788 > Project: Spark > Issue Type: Bug > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani > > Current CholeskyDecomposition implementation calling from lapack has side > effects due to mutating the array A representing the matrix in packed storage > format (see http://www.netlib.org/lapack/lug/node123.html ) and bx the known > right hand side values which should be captured by cloning arrays first! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-13788) CholeskyDecomposition has side effects!
[ https://issues.apache.org/jira/browse/SPARK-13788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ehsan Mohyedin Kermani updated SPARK-13788: --- Description: Current CholeskyDecomposition implementation calling from lapack has side effects due to mutating the array A representing the matrix in packed storage format (see http://www.netlib.org/lapack/lug/node123.html ) and bx the known right hand side values which should be captured by cloning arrays first! (was: Current CholeskyDecomposition implementation calling from lapack has side effects due to mutating the array A representing the matrix in packed storage format (see http://www.netlib.org/lapack/lug/node123.html ) and bx the known right hand side values which should be captured by cloning A first!) > CholeskyDecomposition has side effects! > --- > > Key: SPARK-13788 > URL: https://issues.apache.org/jira/browse/SPARK-13788 > Project: Spark > Issue Type: Bug > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani > Fix For: 2.0.0 > > > Current CholeskyDecomposition implementation calling from lapack has side > effects due to mutating the array A representing the matrix in packed storage > format (see http://www.netlib.org/lapack/lug/node123.html ) and bx the known > right hand side values which should be captured by cloning arrays first! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-13788) CholeskyDecomposition has side effects!
[ https://issues.apache.org/jira/browse/SPARK-13788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ehsan Mohyedin Kermani updated SPARK-13788: --- Description: Current CholeskyDecomposition implementation calling from lapack has side effects due to mutating the array A representing the matrix in packed storage format (see http://www.netlib.org/lapack/lug/node123.html ) and bx the known right hand side values which should be captured by cloning A first! (was: Current CholeskyDecomposition implementation calling from lapack has side effects due to mutating the array A representing the matrix in packed storage format (see http://www.netlib.org/lapack/lug/node123.html ) which should be captured by cloning A first!) > CholeskyDecomposition has side effects! > --- > > Key: SPARK-13788 > URL: https://issues.apache.org/jira/browse/SPARK-13788 > Project: Spark > Issue Type: Bug > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani > Fix For: 2.0.0 > > > Current CholeskyDecomposition implementation calling from lapack has side > effects due to mutating the array A representing the matrix in packed storage > format (see http://www.netlib.org/lapack/lug/node123.html ) and bx the known > right hand side values which should be captured by cloning A first! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-13788) CholeskyDecomposition has side effects!
[ https://issues.apache.org/jira/browse/SPARK-13788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ehsan Mohyedin Kermani updated SPARK-13788: --- Comment: was deleted (was: I'm going to send a PR in couple of minutes.) > CholeskyDecomposition has side effects! > --- > > Key: SPARK-13788 > URL: https://issues.apache.org/jira/browse/SPARK-13788 > Project: Spark > Issue Type: Bug > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani > Fix For: 2.0.0 > > > Current CholeskyDecomposition implementation calling from lapack has side > effects due to mutating the array A representing the matrix in packed storage > format (see http://www.netlib.org/lapack/lug/node123.html ) which should be > captured by cloning A first! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13788) CholeskyDecomposition has side effects!
[ https://issues.apache.org/jira/browse/SPARK-13788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15188263#comment-15188263 ] Ehsan Mohyedin Kermani commented on SPARK-13788: I'm going to send a PR in couple of minutes. > CholeskyDecomposition has side effects! > --- > > Key: SPARK-13788 > URL: https://issues.apache.org/jira/browse/SPARK-13788 > Project: Spark > Issue Type: Bug > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani > Fix For: 2.0.0 > > > Current CholeskyDecomposition implementation calling from lapack has side > effects due to mutating the array A representing the matrix in packed storage > format (see http://www.netlib.org/lapack/lug/node123.html ) which should be > captured by cloning A first! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-13788) CholeskyDecomposition has side effects!
Ehsan Mohyedin Kermani created SPARK-13788: -- Summary: CholeskyDecomposition has side effects! Key: SPARK-13788 URL: https://issues.apache.org/jira/browse/SPARK-13788 Project: Spark Issue Type: Bug Components: MLlib Affects Versions: 1.6.0 Reporter: Ehsan Mohyedin Kermani Fix For: 2.0.0 Current CholeskyDecomposition implementation calling from lapack has side effects due to mutating the array A representing the matrix in packed storage format (see http://www.netlib.org/lapack/lug/node123.html ) which should be captured by cloning A first! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11826) Subtract BlockMatrix
[ https://issues.apache.org/jira/browse/SPARK-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026313#comment-15026313 ] Ehsan Mohyedin Kermani commented on SPARK-11826: May I ask your opinion about this? (in case you haven't seen it). > Subtract BlockMatrix > > > Key: SPARK-11826 > URL: https://issues.apache.org/jira/browse/SPARK-11826 > Project: Spark > Issue Type: Improvement > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani >Priority: Minor > > It'd be more convenient to have subtract method for BlockMatrices. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-11826) Subtract BlockMatrix
[ https://issues.apache.org/jira/browse/SPARK-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ehsan Mohyedin Kermani updated SPARK-11826: --- Comment: was deleted (was: Each time I want to run ./dev/run-tests, it fails by saying - Failed to download pep8.py from "https://raw.githubusercontent.com/jcrocholl/pep8/1.6.2/pep8.py"; and I cannot find a reason for that. Do you have any idea, btw?) > Subtract BlockMatrix > > > Key: SPARK-11826 > URL: https://issues.apache.org/jira/browse/SPARK-11826 > Project: Spark > Issue Type: Improvement > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani >Priority: Minor > > It'd be more convenient to have subtract method for BlockMatrices. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-11826) Subtract BlockMatrix
[ https://issues.apache.org/jira/browse/SPARK-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022954#comment-15022954 ] Ehsan Mohyedin Kermani edited comment on SPARK-11826 at 11/23/15 10:33 PM: --- Each time I want to run ./dev/run-tests, it fails by saying - Failed to download pep8.py from "https://raw.githubusercontent.com/jcrocholl/pep8/1.6.2/pep8.py"; and I cannot find a reason for that. Do you have any idea, btw? was (Author: ehsan mohyedin kermani): Could you please check this, when you have time? > Subtract BlockMatrix > > > Key: SPARK-11826 > URL: https://issues.apache.org/jira/browse/SPARK-11826 > Project: Spark > Issue Type: Improvement > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani >Priority: Minor > > It'd be more convenient to have subtract method for BlockMatrices. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11826) Subtract BlockMatrix
[ https://issues.apache.org/jira/browse/SPARK-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022954#comment-15022954 ] Ehsan Mohyedin Kermani commented on SPARK-11826: Could you please check this, when you have time? > Subtract BlockMatrix > > > Key: SPARK-11826 > URL: https://issues.apache.org/jira/browse/SPARK-11826 > Project: Spark > Issue Type: Improvement > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani >Priority: Minor > > It'd be more convenient to have subtract method for BlockMatrices. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-11826) Subtract BlockMatrix
[ https://issues.apache.org/jira/browse/SPARK-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15020719#comment-15020719 ] Ehsan Mohyedin Kermani edited comment on SPARK-11826 at 11/21/15 10:03 PM: --- My main (major) motivation is related to https://issues.apache.org/jira/browse/SPARK-6417 and heavily using basic operations and possibly, later distributed linalg use cases. The only solution I can think of using add() only, would be multiplying each block after transforming toBreeze(), by negative identity for example, and perform the addition which ultimately is the subtract() (obvious implementation from add()), so it'd be a great help if we can have this and without it I'm not sure how we can achieve it differently? It might be a case that I'm missing something trivial here though! was (Author: ehsan mohyedin kermani): My main (major) motivation is related to [6417](https://issues.apache.org/jira/browse/SPARK-6417) and heavily using basic operations and possibly, later distributed linalg use cases. The only solution I can think of using add() only, would be multiplying each block after transforming toBreeze(), by negative identity for example, and perform the addition which ultimately is the subtract() (obvious implementation from add()), so it'd be a great help if we can have this and without it I'm not sure how we can achieve it differently? It might be a case that I'm missing something trivial here though! > Subtract BlockMatrix > > > Key: SPARK-11826 > URL: https://issues.apache.org/jira/browse/SPARK-11826 > Project: Spark > Issue Type: Improvement > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani >Priority: Minor > > It'd be more convenient to have subtract method for BlockMatrices. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11826) Subtract BlockMatrix
[ https://issues.apache.org/jira/browse/SPARK-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15020719#comment-15020719 ] Ehsan Mohyedin Kermani commented on SPARK-11826: My main (major) motivation is related to [6417](https://issues.apache.org/jira/browse/SPARK-6417) and heavily using basic operations and possibly, later distributed linalg use cases. The only solution I can think of using add() only, would be multiplying each block after transforming toBreeze(), by negative identity for example, and perform the addition which ultimately is the subtract() (obvious implementation from add()), so it'd be a great help if we can have this and without it I'm not sure how we can achieve it differently? It might be a case that I'm missing something trivial here though! > Subtract BlockMatrix > > > Key: SPARK-11826 > URL: https://issues.apache.org/jira/browse/SPARK-11826 > Project: Spark > Issue Type: Improvement > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani >Priority: Minor > > It'd be more convenient to have subtract method for BlockMatrices. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-11826) Subtract BlockMatrix
[ https://issues.apache.org/jira/browse/SPARK-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ehsan Mohyedin Kermani updated SPARK-11826: --- Comment: was deleted (was: I am going to send a PR now.) > Subtract BlockMatrix > > > Key: SPARK-11826 > URL: https://issues.apache.org/jira/browse/SPARK-11826 > Project: Spark > Issue Type: Improvement > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani > > It'd be more convenient to have subtract method for BlockMatrices. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-11826) Subtract BlockMatrix
[ https://issues.apache.org/jira/browse/SPARK-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ehsan Mohyedin Kermani updated SPARK-11826: --- Priority: Major (was: Minor) > Subtract BlockMatrix > > > Key: SPARK-11826 > URL: https://issues.apache.org/jira/browse/SPARK-11826 > Project: Spark > Issue Type: Improvement > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani > > It'd be more convenient to have subtract method for BlockMatrices. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11826) Subtract BlockMatrix
[ https://issues.apache.org/jira/browse/SPARK-11826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011857#comment-15011857 ] Ehsan Mohyedin Kermani commented on SPARK-11826: I am going to send a PR now. > Subtract BlockMatrix > > > Key: SPARK-11826 > URL: https://issues.apache.org/jira/browse/SPARK-11826 > Project: Spark > Issue Type: Improvement > Components: MLlib >Affects Versions: 1.6.0 >Reporter: Ehsan Mohyedin Kermani >Priority: Minor > > It'd be more convenient to have subtract method for BlockMatrices. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-11826) Subtract BlockMatrix
Ehsan Mohyedin Kermani created SPARK-11826: -- Summary: Subtract BlockMatrix Key: SPARK-11826 URL: https://issues.apache.org/jira/browse/SPARK-11826 Project: Spark Issue Type: Improvement Components: MLlib Affects Versions: 1.6.0 Reporter: Ehsan Mohyedin Kermani Priority: Minor It'd be more convenient to have subtract method for BlockMatrices. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-6417) Add Linear Programming algorithm
[ https://issues.apache.org/jira/browse/SPARK-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ehsan Mohyedin Kermani updated SPARK-6417: -- Comment: was deleted (was: Thank you Joseph for the advice! I have started with the starter kit and fixed some annotations to get a sense of contributing to Spark. I am going to work on the LP implementations and perhaps submit it as a package. Regards) > Add Linear Programming algorithm > - > > Key: SPARK-6417 > URL: https://issues.apache.org/jira/browse/SPARK-6417 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Fan Jiang > Labels: features > > Linear programming is the problem of finding a vector x that minimizes a > linear function fTx subject to linear constraints: > minxfTx > such that one or more of the following hold: A·x ≤ b, Aeq·x = beq, l ≤ x ≤ u. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6417) Add Linear Programming algorithm
[ https://issues.apache.org/jira/browse/SPARK-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009849#comment-15009849 ] Ehsan Mohyedin Kermani commented on SPARK-6417: --- As I am working on the implementation, I have some serious concerns/questions. 1) Since I am heavily using distributed matrix abstractions provided in linalg, I realized it's impossible for me to submit it as a separate package when the implementation is completed. I don't think it's possible to define new abstractions like Aaron Staple's TFOCS package, for this task. 2) Adding some functionalities like transforming a local matrix to a BlockMatrix is necessary, as the underlying matrix is a distributed BlockMatrix and that's the only distributed matrix data structure with minimal matrix operations suitable for this task (any suggestions?) 3) Because, there's no literature on LP in mapReduce, my only approach is primal-dual IPM but with underlying distributed matrix data structures, so can that be called a distributed LP solver? if so in what degree? Please let me know your opinion and it'd be great to have someone who is willing to do the code review and guide me before any submission as I'm relatively new to Scala. > Add Linear Programming algorithm > - > > Key: SPARK-6417 > URL: https://issues.apache.org/jira/browse/SPARK-6417 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Fan Jiang > Labels: features > > Linear programming is the problem of finding a vector x that minimizes a > linear function fTx subject to linear constraints: > minxfTx > such that one or more of the following hold: A·x ≤ b, Aeq·x = beq, l ≤ x ≤ u. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-10267) Add @Since annotation to ml.util
[ https://issues.apache.org/jira/browse/SPARK-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14741823#comment-14741823 ] Ehsan Mohyedin Kermani edited comment on SPARK-10267 at 11/15/15 7:03 PM: -- I don't think, util can have annotations by the guideline https://issues.apache.org/jira/browse/SPARK-7751 was (Author: ehsan mohyedin kermani): I don't think, util can annotations by the guideline https://issues.apache.org/jira/browse/SPARK-7751 > Add @Since annotation to ml.util > > > Key: SPARK-10267 > URL: https://issues.apache.org/jira/browse/SPARK-10267 > Project: Spark > Issue Type: Sub-task > Components: Documentation, ML >Reporter: Xiangrui Meng >Assignee: Ehsan Mohyedin Kermani >Priority: Minor > Labels: starter > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6417) Add Linear Programming algorithm
[ https://issues.apache.org/jira/browse/SPARK-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743896#comment-14743896 ] Ehsan Mohyedin Kermani commented on SPARK-6417: --- Thank you Joseph for the advice! I have started with the starter kit and fixed some annotations to get a sense of contributing to Spark. I am going to work on the LP implementations and perhaps submit it as a package. Regards > Add Linear Programming algorithm > - > > Key: SPARK-6417 > URL: https://issues.apache.org/jira/browse/SPARK-6417 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Fan Jiang > Labels: features > > Linear programming is the problem of finding a vector x that minimizes a > linear function fTx subject to linear constraints: > minxfTx > such that one or more of the following hold: A·x ≤ b, Aeq·x = beq, l ≤ x ≤ u. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-10267) Add @Since annotation to ml.util
[ https://issues.apache.org/jira/browse/SPARK-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14741823#comment-14741823 ] Ehsan Mohyedin Kermani commented on SPARK-10267: I don't think, util can annotations by the guideline https://issues.apache.org/jira/browse/SPARK-7751 > Add @Since annotation to ml.util > > > Key: SPARK-10267 > URL: https://issues.apache.org/jira/browse/SPARK-10267 > Project: Spark > Issue Type: Sub-task > Components: Documentation, ML >Reporter: Xiangrui Meng >Priority: Minor > Labels: starter > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6417) Add Linear Programming algorithm
[ https://issues.apache.org/jira/browse/SPARK-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14741389#comment-14741389 ] Ehsan Mohyedin Kermani commented on SPARK-6417: --- I'd like to implement LP, (and possibly use to for my research project). Can it be assigned to me? and any suggestions for the API and algorithms to implement? > Add Linear Programming algorithm > - > > Key: SPARK-6417 > URL: https://issues.apache.org/jira/browse/SPARK-6417 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Fan Jiang > Labels: features > > Linear programming is the problem of finding a vector x that minimizes a > linear function fTx subject to linear constraints: > minxfTx > such that one or more of the following hold: A·x ≤ b, Aeq·x = beq, l ≤ x ≤ u. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8534) Gini for regression metrics and evaluator
[ https://issues.apache.org/jira/browse/SPARK-8534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14726364#comment-14726364 ] Ehsan Mohyedin Kermani commented on SPARK-8534: --- I'd like to give it a shot but first I think, we need distributed scan function for computing the cumulative sum of the sorted predictions. Would it be possible to add that to RegressionMetrics or perhaps mllib.util first? An implementation was suggested here https://groups.google.com/forum/#!topic/spark-users/ts-FdB50ltY. > Gini for regression metrics and evaluator > - > > Key: SPARK-8534 > URL: https://issues.apache.org/jira/browse/SPARK-8534 > Project: Spark > Issue Type: New Feature > Components: ML, MLlib >Reporter: Joseph K. Bradley >Priority: Minor > > One common metric we do not have in RegressionMetrics or RegressionEvaluator > is Gini: [https://www.kaggle.com/wiki/Gini] > Implementing (normalized) Gini would be nice. However, it might be > expensive; I believe it would require sorting the labels. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
