Thanks for all the responses. After re-examining my data I came to realize that second order interactions would be enough in my particular case. With second order instructions I managed to fit a model with less then 512MB RAM.
Thanks to everybody. --- John Fox <[EMAIL PROTECTED]> wrote: > Dear Alexander, > > > At 01:29 AM 10/17/2003 -0700, Alexander Sirotkin > \[at Yahoo\] wrote: > >I agree completely. > > > >In fact, I have about 5000 observations, which > should > >be enough. > >I was using 200 samples because of RAM limitations > and > > I'm afraid to think about what amount of RAM I'll > >need to fit an aov() for such data. > > > > > OK -- I didn't realize that you have 5000 > observations. Perhaps I didn't > read some of the earlier messages carefully enough. > > At the risk of getting you to repeat information > that you've already > provided, how many degrees of freedom are there in > the model that you're > trying to fit? I can create a 5000 by 5000 model > matrix on my relatively > anemic Windows machine, and surely (unless there's > some specification > error) your model should have many fewer df than > that if it includes just > the main effects and two-way interactions (or by all > interactions, do you > mean higher-order interactions as well?). > > Perhaps providing the following information would > help: What is the model > formula? Which variables are factors? How many > levels does each factor have? > > Regards, > John > > >--- John Fox <[EMAIL PROTECTED]> wrote: > > > Dear Alexander, > > > > > > If I understand you correctly, you have a sample > of > > > 200 observations. Even > > > if you had only two factors with 40 levels each, > the > > > main effects and > > > interactions of these factors would require > about > > > 1600 degrees of freedom > > > -- that is, more than the number of > observations. > > > This doesn't make a whole > > > lot of sense. > > > > > > I hope that this helps, > > > John > > > > > > At 05:03 PM 10/16/2003 -0700, Alexander Sirotkin > > > \[at Yahoo\] wrote: > > > > > > >--- Deepayan Sarkar <[EMAIL PROTECTED]> > wrote: > > > > > On Thursday 16 October 2003 17:59, Alexander > > > > > Sirotkin \[at Yahoo\] wrote: > > > > > > Thanks for all the help on my previous > > > questions. > > > > > > > > > > > > One more (hopefully last one) : I've been > very > > > > > > surprised when I tried to fit a model > (using > > > > > aov()) > > > > > > for a sample of size 200 and 10 variables > and > > > > > their > > > > > > interactions. > > > > > > > > > > That doesn't really say much. How many of > these > > > > > variables are factors ? How > > > > > many levels do they have ? And what is the > order > > > of > > > > > the interaction ? (Note > > > > > that for 10 numeric variables, if you allow > all > > > > > interactions, then there will > > > > > be a 100 terms in your model. This increases > for > > > > > factors.) > > > > > > > > > > In other words, how big is your model matrix > ? > > > (See > > > > > ?model.matrix) > > > > > > > > > > Deepayan > > > > > > > > > > > > > > > > >I see... > > > > > > > >Unfortunately, model.matrix() ran out of memory > :) > > > >I have 10 variables, 6 of which are factor, 2 > of > > > which > > > > > > > >have quite a lot of levels (about 40). And I > would > > > >like > > > >to allow all interactions. > > > > > > > >I understand your point about categorical > > > variables, > > > >but > > > >still - this does not seem like too much data > to > > > me. > > > > > > > > > > > >I remmeber fitting all kinds of models (mostly > > > >decision > > > >trees) for much, much larger data sets. > > > > > > > >______________________________________________ > > > >[EMAIL PROTECTED] mailing list > > > > > > >https://www.stat.math.ethz.ch/mailman/listinfo/r-help > > > > > > > >----------------------------------------------------- > > > John Fox > > > Department of Sociology > > > McMaster University > > > Hamilton, Ontario, Canada L8S 4M4 > > > email: [EMAIL PROTECTED] > > > phone: 905-525-9140x23604 > > > web: www.socsci.mcmaster.ca/jfox > > > > >----------------------------------------------------- > > > > > > > > >__________________________________ > >Do you Yahoo!? > search > >http://shopping.yahoo.com > > ----------------------------------------------------- > John Fox > Department of Sociology > McMaster University > Hamilton, Ontario, Canada L8S 4M4 > email: [EMAIL PROTECTED] > phone: 905-525-9140x23604 > web: www.socsci.mcmaster.ca/jfox > ----------------------------------------------------- > ______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help
