Re: [R] non-conformable arrays

2024-09-12 Thread Thierry Onkelinx 
Dear Peter,

That was indeed this issue. Thanks for the feedback.

Best regards,

ir. Thierry Onkelinx
Statisticus / Statistician

Vlaamse Overheid / Government of Flanders
INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND
FOREST
Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
[email protected]
Havenlaan 88 bus 73, 1000 Brussel
*Postadres:* Koning Albert II-laan 15 bus 186, 1210 Brussel
*Poststukken die naar dit adres worden gestuurd, worden ingescand en
digitaal aan de geadresseerde bezorgd. Zo kan de Vlaamse overheid haar
dossiers volledig digitaal behandelen. Poststukken met de vermelding
‘vertrouwelijk’ worden niet ingescand, maar ongeopend aan de geadresseerde
bezorgd.*
www.inbo.be

///
To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to say
what the experiment died of. ~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data. ~ Roger Brinner
The combination of some data and an aching desire for an answer does not
ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey
///




Op wo 11 sep 2024 om 18:41 schreef peter dalgaard :

> Hum... Two points: You are using |> a lot and tcrossprod() is a primitive,
> so ignores argnames. This makes me suspicious that things like
>
> ... ) |>  tcrossprod(x = mm)
>
> might not do what you think it does.
>
> E.g.,
>
> > a <- matrix(1,10,2)
> > b <- matrix(1,3,2)
> > tcrossprod(y=b, x=a)
>  [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
> [1,]222222222 2
> [2,]222222222 2
> [3,]222222222 2
> > tcrossprod(x=a, y=b)
>   [,1] [,2] [,3]
>  [1,]222
>  [2,]222
>  [3,]222
>  [4,]222
>  [5,]222
>  [6,]222
>  [7,]222
>  [8,]222
>  [9,]222
> [10,]222
>
> -pd
>
> > On 11 Sep 2024, at 17:55 , Thierry Onkelinx 
> wrote:
> >
> > Dear all,
> >
> > I'm puzzled by this error. When running tcrossprod() within the function
> it
> > returns the error message. The code also outputs the object a and b.
> > Running the tcrossprod() outside of the function works as expected.
> >
> >cat("a <-")
> >dput(a)
> >cat("b <-")
> >dput(b)
> >cat("tcrossprod(a, b)")
> >tcrossprod(a, b)
> >
> > a <-structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0,
> > 0, 0, 1), dim = c(10L, 2L), dimnames = list(c("9", "13", "21",
> > "29", "30", "37", "52", "53", "56", "70"),
> > c("tmp2028c70ae152b4c63bb7ab902158b408366217581",
> > "tmp2028c70ae152b4c63bb7ab902158b408366217582")), assign = c(1L,
> > 1L), contrasts = list(tmp2028c70ae152b4c63bb7ab902158b40836621758 =
> > "contr.treatment"))
> > b <-structure(c(-0.916362039446752, -0.849801808879291,
> -0.744535398206787,
> > 0.875896407785924, 0.822587420283086, 0.894210774042389), dim = 3:2)
> > tcrossprod(a, b)
> >
> > For those how like a fully reproducible example:
> > the offending line in the code:
> >
> https://github.com/inbo/multimput/blob/e1cd0cdff7d2868e4101c411f7508301c7be7482/R/impute_glmermod.R#L65
> > a (failing) unit test for the code:
> >
> https://github.com/inbo/multimput/blob/e1cd0cdff7d2868e4101c411f7508301c7be7482/tests/testthat/test_ccc_hurdle_impute.R#L10
> >
> > Best regards,
> >
> > ir. Thierry Onkelinx
> > Statisticus / Statistician
> >
> > Vlaamse Overheid / Government of Flanders
> > INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE
> AND
> > FOREST
> > Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
> > [email protected]
> > Havenlaan 88 bus 73, 1000 Brussel
> > *Postadres:* Koning Albert II-laan 15 bus 186, 1210 Brussel
> > *Poststukken die naar dit adres worden gestuurd, worden ingescand en
> > digitaal aan de geadresseerde bezorgd. Zo kan de Vlaamse overheid haar
> > dossiers volledig digitaal behandelen. Poststukken met de vermelding
> > ‘vertrouwelijk’ worden niet ingescand, maar ongeopend aan de
> geadresseerde
> > bezorgd.*
> > www.inbo.be
> >
> >
> ///
> > To call in the statistician after the experiment is done may be no more
> > than asking him to perform a post-mortem examination: he may be able to
> say
> > what the experiment died of. ~ Sir Ronald Aylmer Fisher
> > The plural of anecdote is not data. ~ Roger Brinner
> > The combination of some data and an aching desire for an answer does not
> > ensure that a reasonable answer can be extracted from a given body of
> data.
> > ~ John Tukey
> >
> //

Re: [R] non-conformable arrays

2024-09-11 Thread Ivan Krylov via R-help 
В Wed, 11 Sep 2024 17:55:53 +0200
Thierry Onkelinx  пишет:

> For those how like a fully reproducible example:
> the offending line in the code:
> https://github.com/inbo/multimput/blob/e1cd0cdff7d2868e4101c411f7508301c7be7482/R/impute_glmermod.R#L65
> a (failing) unit test for the code:
> https://github.com/inbo/multimput/blob/e1cd0cdff7d2868e4101c411f7508301c7be7482/tests/testthat/test_ccc_hurdle_impute.R#L10

Setting options(error = recover) and evaluating your test expression
directly using
eval(parse('testthat/test_ccc_hurdle_impute.R')[[1]][[3]]), I see:

Browse[1]> names(random)
[1] "Year"
Browse[1]> paste("~0 + ", 'Year') |>
+ as.formula() |>
+ model.matrix(data = data[missing_obs,]) |>
+ str()
 num [1:68, 1] 7 6 9 4 3 5 10 1 6 8 ...
 - attr(*, "dimnames")=List of 2
  ..$ : chr [1:68] "37" "56" "59" "114" ...
  ..$ : chr "Year"
 - attr(*, "assign")= int 1
Browse[1]> t(random[['Year']]) |> str()
 num [1:19, 1:10] 0.175 0.181 0.102 0.119 0.158 ...

...and they are indeed non-conformable. Why is random$Year a matrix?

-- 
Best regards,
Ivan

__
[email protected] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] non-conformable arrays

2024-09-11 Thread peter dalgaard 
Hum... Two points: You are using |> a lot and tcrossprod() is a primitive, so 
ignores argnames. This makes me suspicious that things like 

... ) |>  tcrossprod(x = mm)

might not do what you think it does.

E.g., 

> a <- matrix(1,10,2)
> b <- matrix(1,3,2)
> tcrossprod(y=b, x=a)
 [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,]222222222 2
[2,]222222222 2
[3,]222222222 2
> tcrossprod(x=a, y=b)
  [,1] [,2] [,3]
 [1,]222
 [2,]222
 [3,]222
 [4,]222
 [5,]222
 [6,]222
 [7,]222
 [8,]222
 [9,]222
[10,]222

-pd

> On 11 Sep 2024, at 17:55 , Thierry Onkelinx  wrote:
> 
> Dear all,
> 
> I'm puzzled by this error. When running tcrossprod() within the function it
> returns the error message. The code also outputs the object a and b.
> Running the tcrossprod() outside of the function works as expected.
> 
>cat("a <-")
>dput(a)
>cat("b <-")
>dput(b)
>cat("tcrossprod(a, b)")
>tcrossprod(a, b)
> 
> a <-structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0,
> 0, 0, 1), dim = c(10L, 2L), dimnames = list(c("9", "13", "21",
> "29", "30", "37", "52", "53", "56", "70"),
> c("tmp2028c70ae152b4c63bb7ab902158b408366217581",
> "tmp2028c70ae152b4c63bb7ab902158b408366217582")), assign = c(1L,
> 1L), contrasts = list(tmp2028c70ae152b4c63bb7ab902158b40836621758 =
> "contr.treatment"))
> b <-structure(c(-0.916362039446752, -0.849801808879291, -0.744535398206787,
> 0.875896407785924, 0.822587420283086, 0.894210774042389), dim = 3:2)
> tcrossprod(a, b)
> 
> For those how like a fully reproducible example:
> the offending line in the code:
> https://github.com/inbo/multimput/blob/e1cd0cdff7d2868e4101c411f7508301c7be7482/R/impute_glmermod.R#L65
> a (failing) unit test for the code:
> https://github.com/inbo/multimput/blob/e1cd0cdff7d2868e4101c411f7508301c7be7482/tests/testthat/test_ccc_hurdle_impute.R#L10
> 
> Best regards,
> 
> ir. Thierry Onkelinx
> Statisticus / Statistician
> 
> Vlaamse Overheid / Government of Flanders
> INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND
> FOREST
> Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
> [email protected]
> Havenlaan 88 bus 73, 1000 Brussel
> *Postadres:* Koning Albert II-laan 15 bus 186, 1210 Brussel
> *Poststukken die naar dit adres worden gestuurd, worden ingescand en
> digitaal aan de geadresseerde bezorgd. Zo kan de Vlaamse overheid haar
> dossiers volledig digitaal behandelen. Poststukken met de vermelding
> ‘vertrouwelijk’ worden niet ingescand, maar ongeopend aan de geadresseerde
> bezorgd.*
> www.inbo.be
> 
> ///
> To call in the statistician after the experiment is done may be no more
> than asking him to perform a post-mortem examination: he may be able to say
> what the experiment died of. ~ Sir Ronald Aylmer Fisher
> The plural of anecdote is not data. ~ Roger Brinner
> The combination of some data and an aching desire for an answer does not
> ensure that a reasonable answer can be extracted from a given body of data.
> ~ John Tukey
> ///
> 
> 
> 
>   [[alternative HTML version deleted]]
> 
> __
> [email protected] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: [email protected]  Priv: [email protected]

__
[email protected] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] non-conformable arrays

2013-07-24 Thread Thomas Stewart 
You need to be careful how you index.  In the example code, y is a 'zoo
series' and y[i] is gives the ith row, as in y[i, ].  This means t(y[i])
%*% y[i]  is actually a 2X2 matrix.  Also, the code c(S)[c(1,4,2)] picks
off the diagonal and lower triangular elements.  The example involves two
stocks, so there are 3 elements.  When you expand the example to 5 stocks,
there will be 15 elements (5 variances and 10 covariances).  Rather than
manually selecting the elements you can use something like this:

EWMA[i,]
 <- S
[lower.tri(S,diag=TRUE)]

Note, however, that you'll have to change the indexing in the final step if
you use this code.

Lastly, if you are expanding the code for five stocks, the returns matrix
should have 5 columns --- one for each stock.  Then,

 t(returns[i, ]) %*% returns[i, ]

will be 5X5 just as lambda * S is.


On Tue, Jul 23, 2013 at 5:29 AM, G Girija  wrote:

> Hi,
> I have 5 stock values and i am calculating EWMA
> followed the logic as given ind following link.[
> http://www.orecastingfinancialrisk.com/3.html<
> http://www.forecastingfinancialrisk.com/3.html>
> ]
>
> [EWMA = matrix(nrow=T,ncol=3) # create a matrix to hold the covariance
> matrix for each t
>
>
> lambda = 0.94
> S = cov(y)  # initial (t=1) covariance matrix
> EWMA[1,] = c(S)[c(1,4,2)]# extract the variances and covariancefor (i
> in 2:T){# loop though the sample
> S = lambda * S  + (1-lambda) * t(y[i]) %*% y[i]
> EWMA[i,] = c(S)[c(1,4,2)]   # convert matrix to vector  }
> EWMArho = EWMA[,3]/sqrt(EWMA[,1]*EWMA[,2])  # calculate correlations
>  ]
>
>
>
> EWMA[1,]<-c(S)[c(1,7,13,19,25,5)] --I am taking all diagonal values and one
> more value. ---Is it correct
>
> my returns matrix dim is  1x2496 and its transpose is 2496x1.
> so the resultant matrix will be 1x1.
> but we are adding it to lambda*S which is 5x5 matrix.may be I am
> getting error because of this. How to overcome this error? Pl help
>
> Error in lambda * S + (1 - lambda) * t(as.matrix(returns[i])) %*%
> (as.matrix(returns[i])) :
>   non-conformable arrays
>
> [[alternative HTML version deleted]]
>
> __
> [email protected] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Non-conformable arrays

2011-03-02 Thread Gregory Ryslik 
Perfect! Thank you!
On Mar 2, 2011, at 10:55 PM,   
wrote:

> Here is one way.
> 
> 1. make sure y.test is a factor
> 
> 2. Use
> 
> table(y.test, 
>  factor(PredictedTestCurrent, levels = levels(y.test)) 
> 
> 3. If PredictedTestCurrent is already a factor with the wrong levels, turn it 
> back into a character string vector first.
> 
> -Original Message-
> From: [email protected] [mailto:[email protected]] On 
> Behalf Of Gregory Ryslik
> Sent: Thursday, 3 March 2011 1:46 PM
> To: r-help Help
> Subject: [R] Non-conformable arrays
> 
> Hi Everyone,
> 
> I'm running some simulations where eventually I need to table the results. 
> The problem is, that while most simulations I have at least one predicted 
> outcome for each of the six possible categories, sometimes the algorithm 
> assigns all the outcomes and one category is left out. Thus when I try to add 
> the misclassification matrices I get an error. Is there a simple way I can 
> make sure that all my tables have the same size (with a row or column of 
> zeros) if appropriate without a messy "if" structure checking each condition?
> 
> To be more specific,
> 
> here's my line of code for the table command and the two matrices that I 
> sometimes have. Notice that in the second matrix, the "fad" column is 
> missing. Basically, I want all the columns and rows to be predetermined so 
> that no columns/rows go missing. Thanks for your help!
> 
> Kind regards,
> Greg
> 
> table(y.test,PredictedTestCurrent):
> 
> 
>  PredictedTestCurrent
> y.test adi car con fad gla mas
>   adi   9   0   0   0   0   0
>   car   0   6   1   0   0   3
>   con   1   0   3   0   0   0
>   fad   0   0   0   2   5   4
>   gla   0   1   0   0   6   3
>   mas   0   0   0   1   4   4
> 
> 
>  PredictedTestCurrent
> y.test adi car con gla mas
>   adi   8   0   0   0   0
>   car   0   8   0   0   1
>   con   2   0   3   0   0
>   fad   0   1   0   4   7
>   gla   0   0   0   3   5
>   mas   0   2   0   6   3
> 
> 
> 
> 
>   [[alternative HTML version deleted]]
> 
> __
> [email protected] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Non-conformable arrays

2011-03-02 Thread Bill.Venables 
Here is one way.

1. make sure y.test is a factor

2. Use

table(y.test, 
  factor(PredictedTestCurrent, levels = levels(y.test)) 

3. If PredictedTestCurrent is already a factor with the wrong levels, turn it 
back into a character string vector first.

-Original Message-
From: [email protected] [mailto:[email protected]] On 
Behalf Of Gregory Ryslik
Sent: Thursday, 3 March 2011 1:46 PM
To: r-help Help
Subject: [R] Non-conformable arrays

Hi Everyone,

I'm running some simulations where eventually I need to table the results. The 
problem is, that while most simulations I have at least one predicted outcome 
for each of the six possible categories, sometimes the algorithm assigns all 
the outcomes and one category is left out. Thus when I try to add the 
misclassification matrices I get an error. Is there a simple way I can make 
sure that all my tables have the same size (with a row or column of zeros) if 
appropriate without a messy "if" structure checking each condition?

To be more specific,

here's my line of code for the table command and the two matrices that I 
sometimes have. Notice that in the second matrix, the "fad" column is missing. 
Basically, I want all the columns and rows to be predetermined so that no 
columns/rows go missing. Thanks for your help!

Kind regards,
Greg

table(y.test,PredictedTestCurrent):


  PredictedTestCurrent
y.test adi car con fad gla mas
   adi   9   0   0   0   0   0
   car   0   6   1   0   0   3
   con   1   0   3   0   0   0
   fad   0   0   0   2   5   4
   gla   0   1   0   0   6   3
   mas   0   0   0   1   4   4


  PredictedTestCurrent
y.test adi car con gla mas
   adi   8   0   0   0   0
   car   0   8   0   0   1
   con   2   0   3   0   0
   fad   0   1   0   4   7
   gla   0   0   0   3   5
   mas   0   2   0   6   3




[[alternative HTML version deleted]]

__
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.