Re: [EXTERNAL] - Re: Problem with the ML ALS algorithm
Generally I would say 10s is a bit low, while a few 100s+ starts to make
sense. Of course it depends a lot on the specific use case, item catalogue
etc, user experience / platform, etc.
On Wed, Jun 26, 2019 at 3:57 PM Steve Pruitt wrote:
> I should have mentioned this is a synthetic dataset I create using some
> likelihood distributions of the rating values. I am only experimenting /
> learning. In practice though, the list of items is likely to be at least
> in the 10’s if not 100’s. Are even this item numbers to low?
>
>
>
> Thanks.
>
>
>
> -S
>
>
>
> *From:* Nick Pentreath
> *Sent:* Wednesday, June 26, 2019 9:09 AM
> *To:* user@spark.apache.org
> *Subject:* Re: [EXTERNAL] - Re: Problem with the ML ALS algorithm
>
>
>
> If the number of items is indeed 4, then another issue is the rank of the
> factors defaults to 10. Setting the "rank" parameter < 4 will help.
>
>
>
> However, if you only have 4 items, then I would propose that using ALS (or
> any recommendation model in fact) is not really necessary. There is not
> really enough information as well as sparsity, to make collaborative
> filtering useful. And you could simply recommend all items a user has not
> rated and the result would be the same essentially.
>
>
>
>
>
> On Wed, Jun 26, 2019 at 3:03 PM Steve Pruitt wrote:
>
> Number of users is 1055
>
> Number of items is 4
>
> Ratings values are either 120, 20, 0
>
>
>
>
>
> *From:* Nick Pentreath
> *Sent:* Wednesday, June 26, 2019 6:03 AM
> *To:* user@spark.apache.org
> *Subject:* [EXTERNAL] - Re: Problem with the ML ALS algorithm
>
>
>
> This means that the matrix that ALS is trying to factor is not positive
> definite. Try increasing regParam (try 0.1, 1.0 for example).
>
>
>
> What does the data look like? e.g. number of users, number of items,
> number of ratings, etc?
>
>
>
> On Wed, Jun 26, 2019 at 12:06 AM Steve Pruitt
> wrote:
>
> I get an inexplicable exception when trying to build an ALSModel with the
> implicit set to true. I can’t find any help online.
>
>
>
> Thanks in advance.
>
>
>
> My code is:
>
>
>
> ALS als = new ALS()
>
> .setMaxIter(5)
>
> .setRegParam(0.01)
>
> .setUserCol("customer")
>
> .setItemCol("item")
>
> .setImplicitPrefs(true)
>
> .setRatingCol("rating");
>
> ALSModel model = als.fit(training);
>
>
>
> The exception is:
>
> org.apache.spark.ml.optim.SingularMatrixException: LAPACK.dppsv returned 6
> because A is not positive definite. Is A derived from a singular matrix
> (e.g. collinear column values)?
>
> at
> org.apache.spark.mllib.linalg.CholeskyDecomposition$.checkReturnValue(CholeskyDecomposition.scala:65)
> ~[spark-mllib_2.11-2.3.1.jar:2.3.1]
>
> at
> org.apache.spark.mllib.linalg.CholeskyDecomposition$.solve(CholeskyDecomposition.scala:41)
> ~[spark-mllib_2.11-2.3.1.jar:2.3.1]
>
> at
> org.apache.spark.ml.recommendation.ALS$CholeskySolver.solve(ALS.scala:747)
> ~[spark-mllib_2.11-2.3.1.jar:2.3.1]
>
>
RE: [EXTERNAL] - Re: Problem with the ML ALS algorithm
I should have mentioned this is a synthetic dataset I create using some
likelihood distributions of the rating values. I am only experimenting /
learning. In practice though, the list of items is likely to be at least in
the 10’s if not 100’s. Are even this item numbers to low?
Thanks.
-S
From: Nick Pentreath
Sent: Wednesday, June 26, 2019 9:09 AM
To: user@spark.apache.org
Subject: Re: [EXTERNAL] - Re: Problem with the ML ALS algorithm
If the number of items is indeed 4, then another issue is the rank of the
factors defaults to 10. Setting the "rank" parameter < 4 will help.
However, if you only have 4 items, then I would propose that using ALS (or any
recommendation model in fact) is not really necessary. There is not really
enough information as well as sparsity, to make collaborative filtering useful.
And you could simply recommend all items a user has not rated and the result
would be the same essentially.
On Wed, Jun 26, 2019 at 3:03 PM Steve Pruitt
mailto:bpru...@opentext.com>> wrote:
Number of users is 1055
Number of items is 4
Ratings values are either 120, 20, 0
From: Nick Pentreath mailto:nick.pentre...@gmail.com>>
Sent: Wednesday, June 26, 2019 6:03 AM
To: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: [EXTERNAL] - Re: Problem with the ML ALS algorithm
This means that the matrix that ALS is trying to factor is not positive
definite. Try increasing regParam (try 0.1, 1.0 for example).
What does the data look like? e.g. number of users, number of items, number of
ratings, etc?
On Wed, Jun 26, 2019 at 12:06 AM Steve Pruitt
mailto:bpru...@opentext.com>> wrote:
I get an inexplicable exception when trying to build an ALSModel with the
implicit set to true. I can’t find any help online.
Thanks in advance.
My code is:
ALS als = new ALS()
.setMaxIter(5)
.setRegParam(0.01)
.setUserCol("customer")
.setItemCol("item")
.setImplicitPrefs(true)
.setRatingCol("rating");
ALSModel model = als.fit(training);
The exception is:
org.apache.spark.ml.optim.SingularMatrixException: LAPACK.dppsv returned 6
because A is not positive definite. Is A derived from a singular matrix (e.g.
collinear column values)?
at
org.apache.spark.mllib.linalg.CholeskyDecomposition$.checkReturnValue(CholeskyDecomposition.scala:65)
~[spark-mllib_2.11-2.3.1.jar:2.3.1]
at
org.apache.spark.mllib.linalg.CholeskyDecomposition$.solve(CholeskyDecomposition.scala:41)
~[spark-mllib_2.11-2.3.1.jar:2.3.1]
at
org.apache.spark.ml.recommendation.ALS$CholeskySolver.solve(ALS.scala:747)
~[spark-mllib_2.11-2.3.1.jar:2.3.1]
Re: [EXTERNAL] - Re: Problem with the ML ALS algorithm
If the number of items is indeed 4, then another issue is the rank of the
factors defaults to 10. Setting the "rank" parameter < 4 will help.
However, if you only have 4 items, then I would propose that using ALS (or
any recommendation model in fact) is not really necessary. There is not
really enough information as well as sparsity, to make collaborative
filtering useful. And you could simply recommend all items a user has not
rated and the result would be the same essentially.
On Wed, Jun 26, 2019 at 3:03 PM Steve Pruitt wrote:
> Number of users is 1055
>
> Number of items is 4
>
> Ratings values are either 120, 20, 0
>
>
>
>
>
> *From:* Nick Pentreath
> *Sent:* Wednesday, June 26, 2019 6:03 AM
> *To:* user@spark.apache.org
> *Subject:* [EXTERNAL] - Re: Problem with the ML ALS algorithm
>
>
>
> This means that the matrix that ALS is trying to factor is not positive
> definite. Try increasing regParam (try 0.1, 1.0 for example).
>
>
>
> What does the data look like? e.g. number of users, number of items,
> number of ratings, etc?
>
>
>
> On Wed, Jun 26, 2019 at 12:06 AM Steve Pruitt
> wrote:
>
> I get an inexplicable exception when trying to build an ALSModel with the
> implicit set to true. I can’t find any help online.
>
>
>
> Thanks in advance.
>
>
>
> My code is:
>
>
>
> ALS als = new ALS()
>
> .setMaxIter(5)
>
> .setRegParam(0.01)
>
> .setUserCol("customer")
>
> .setItemCol("item")
>
> .setImplicitPrefs(true)
>
> .setRatingCol("rating");
>
> ALSModel model = als.fit(training);
>
>
>
> The exception is:
>
> org.apache.spark.ml.optim.SingularMatrixException: LAPACK.dppsv returned 6
> because A is not positive definite. Is A derived from a singular matrix
> (e.g. collinear column values)?
>
> at
> org.apache.spark.mllib.linalg.CholeskyDecomposition$.checkReturnValue(CholeskyDecomposition.scala:65)
> ~[spark-mllib_2.11-2.3.1.jar:2.3.1]
>
> at
> org.apache.spark.mllib.linalg.CholeskyDecomposition$.solve(CholeskyDecomposition.scala:41)
> ~[spark-mllib_2.11-2.3.1.jar:2.3.1]
>
> at
> org.apache.spark.ml.recommendation.ALS$CholeskySolver.solve(ALS.scala:747)
> ~[spark-mllib_2.11-2.3.1.jar:2.3.1]
>
>
RE: [EXTERNAL] - Re: Problem with the ML ALS algorithm
Number of users is 1055
Number of items is 4
Ratings values are either 120, 20, 0
From: Nick Pentreath
Sent: Wednesday, June 26, 2019 6:03 AM
To: user@spark.apache.org
Subject: [EXTERNAL] - Re: Problem with the ML ALS algorithm
This means that the matrix that ALS is trying to factor is not positive
definite. Try increasing regParam (try 0.1, 1.0 for example).
What does the data look like? e.g. number of users, number of items, number of
ratings, etc?
On Wed, Jun 26, 2019 at 12:06 AM Steve Pruitt
mailto:bpru...@opentext.com>> wrote:
I get an inexplicable exception when trying to build an ALSModel with the
implicit set to true. I can’t find any help online.
Thanks in advance.
My code is:
ALS als = new ALS()
.setMaxIter(5)
.setRegParam(0.01)
.setUserCol("customer")
.setItemCol("item")
.setImplicitPrefs(true)
.setRatingCol("rating");
ALSModel model = als.fit(training);
The exception is:
org.apache.spark.ml.optim.SingularMatrixException: LAPACK.dppsv returned 6
because A is not positive definite. Is A derived from a singular matrix (e.g.
collinear column values)?
at
org.apache.spark.mllib.linalg.CholeskyDecomposition$.checkReturnValue(CholeskyDecomposition.scala:65)
~[spark-mllib_2.11-2.3.1.jar:2.3.1]
at
org.apache.spark.mllib.linalg.CholeskyDecomposition$.solve(CholeskyDecomposition.scala:41)
~[spark-mllib_2.11-2.3.1.jar:2.3.1]
at
org.apache.spark.ml.recommendation.ALS$CholeskySolver.solve(ALS.scala:747)
~[spark-mllib_2.11-2.3.1.jar:2.3.1]
Problem with the ML ALS algorithm
I get an inexplicable exception when trying to build an ALSModel with the
implicit set to true. I can’t find any help online.
Thanks in advance.
My code is:
ALS als = new ALS()
.setMaxIter(5)
.setRegParam(0.01)
.setUserCol("customer")
.setItemCol("item")
.setImplicitPrefs(true)
.setRatingCol("rating");
ALSModel model = als.fit(training);
The exception is:
org.apache.spark.ml.optim.SingularMatrixException: LAPACK.dppsv returned 6
because A is not positive definite. Is A derived from a singular matrix (e.g.
collinear column values)?
at
org.apache.spark.mllib.linalg.CholeskyDecomposition$.checkReturnValue(CholeskyDecomposition.scala:65)
~[spark-mllib_2.11-2.3.1.jar:2.3.1]
at
org.apache.spark.mllib.linalg.CholeskyDecomposition$.solve(CholeskyDecomposition.scala:41)
~[spark-mllib_2.11-2.3.1.jar:2.3.1]
at
org.apache.spark.ml.recommendation.ALS$CholeskySolver.solve(ALS.scala:747)
~[spark-mllib_2.11-2.3.1.jar:2.3.1]
