[jira] [Commented] (SPARK-17027) PolynomialExpansion.choose is prone to integer overflow

2016-08-12 Thread Maciej Szymkiewicz (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-17027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418071#comment-15418071
 ] 

Maciej Szymkiewicz commented on SPARK-17027:


Yes, this exactly the problem. 


{code}
choose(14, 10)
// res0: Int = -182
{code}

> PolynomialExpansion.choose is prone to integer overflow 
> 
>
> Key: SPARK-17027
> URL: https://issues.apache.org/jira/browse/SPARK-17027
> Project: Spark
>  Issue Type: Bug
>  Components: ML
>Affects Versions: 1.6.0, 2.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Minor
>
> Current implementation computes power of k directly and because of that it is 
> susceptible to integer overflow on relatively small input (4 features, degree 
> equal 10).  It would be better to use recursive formula instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-17027) PolynomialExpansion.choose is prone to integer overflow

2016-08-11 Thread Apache Spark (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-17027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418112#comment-15418112
 ] 

Apache Spark commented on SPARK-17027:
--

User 'zero323' has created a pull request for this issue:
https://github.com/apache/spark/pull/14614

> PolynomialExpansion.choose is prone to integer overflow 
> 
>
> Key: SPARK-17027
> URL: https://issues.apache.org/jira/browse/SPARK-17027
> Project: Spark
>  Issue Type: Bug
>  Components: ML
>Affects Versions: 1.6.0, 2.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Minor
>
> Current implementation computes power of k directly and because of that it is 
> susceptible to integer overflow on relatively small input (4 features, degree 
> equal 10).  It would be better to use recursive formula instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-17027) PolynomialExpansion.choose is prone to integer overflow

2016-08-11 Thread Sean Owen (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-17027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418065#comment-15418065
 ] 

Sean Owen commented on SPARK-17027:
---

Is the problem in the naive calculation of n choose k?

{code}
  private def choose(n: Int, k: Int): Int = {
Range(n, n - k, -1).product / Range(k, 1, -1).product
  }
{code}

Let's just call 
http://commons.apache.org/proper/commons-math/apidocs/org/apache/commons/math3/util/CombinatoricsUtils.html#binomialCoefficient(int,%20int)

> PolynomialExpansion.choose is prone to integer overflow 
> 
>
> Key: SPARK-17027
> URL: https://issues.apache.org/jira/browse/SPARK-17027
> Project: Spark
>  Issue Type: Bug
>  Components: ML
>Affects Versions: 1.6.0, 2.0.0
>Reporter: Maciej Szymkiewicz
>Priority: Minor
>
> Current implementation computes power of k directly and because of that it is 
> susceptible to integer overflow on relatively small input (4 features, degree 
> equal 10).  It would be better to use recursive formula instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org