[
https://issues.apache.org/jira/browse/SPARK-11302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14974589#comment-14974589
]
eyal sharon commented on SPARK-11302:
-------------------------------------
Cool , I will try although I hope I fully captured all questions
1- Logpdf is also returns non reasonable value . mg.logpdf(Vectors.dense(
629,640,1.7188,618.19)) = -3891330.078277891 ( the exp is zero )
2 - my MU vector values -
[1055.3910505836575,1070.489299610895,1.39020554474708,1040.5907503867697]
3 - the correct answer , as return from Matlab for this given data point
mg.pdf(Vectors.dense(629,640,1.7188,618.19)) is around e-05
4- when running the model with a non covariance matrix , model yields
pdf - 7.293362507983666E-11, logpdf -23.341471333876257 . Again , these
results match with the Matlab model
5- these are the printed values from my script
mu:
[1055.3910505836575,1070.489299610895,1.39020554474708,1040.5907503867697]
sigma:
166769.00466698944 0.0 0.0 0.0
0.0 172041.5670061245 0.0 0.0
0.0 0.0 0.872524191943962 0.0
0.0 0.0 0.0 161848.9196719207
sigmaCov:
166769.00466698944 169336.6705268059 12.820670788921873
164243.93314092053
169336.6705268059 172041.5670061245 21.62590020524533
166678.01075856484
12.820670788921873 21.62590020524533 0.872524191943962
4.283255814732373
164243.93314092053 166678.01075856484 4.283255814732373
161848.9196719207
I hope it helps.
Best, Eyal
--
*This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are
addressed. Please note that any disclosure, copying or distribution of the
content of this information is strictly forbidden. If you have received
this email message in error, please destroy it immediately and notify its
sender.*
> Multivariate Gaussian Model with Covariance matrix return zero always
> ------------------------------------------------------------------------
>
> Key: SPARK-11302
> URL: https://issues.apache.org/jira/browse/SPARK-11302
> Project: Spark
> Issue Type: Bug
> Components: MLlib
> Reporter: eyal sharon
> Priority: Minor
>
> I have been trying to apply an Anomaly Detection model using Spark MLib.
> As an input, I feed the model with a mean vector and a Covariance matrix.
> ,assuming my features contain Co-variance.
> Here are my input for the model ,and the model returns zero for each data
> point for this input.
> MU vector -
> 1054.8, 1069.8, 1.3 ,1040.1
> Cov' matrix -
> 165496.0 , 167996.0, 11.0 , 163037.0
> 167996.0, 170631.0, 19.0, 165405.0
> 11.0, 19.0 , 0.0, 2.0
> 163037.0, 165405.0 2.0 , 160707.0
> Conversely, for the non covariance case, represented by this matrix ,the
> model is working and returns results as expected
> 165496.0, 0.0 , 0.0, 0.0
> 0.0, 170631.0, 0.0, 0.0
> 0.0 , 0.0 , 0.8, 0.0
> 0.0 , 0.0, 0.0, 160594.2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
