Re: [R] FIML in lme
On Aug 27, Douglas Bates wrote: F Z wrote: I was asked if lme can use FIML (Full Information Maximum Likelihood) instead of REML or ML but I don't know the answer. Does anybody know if this is implemented in R? To the best of my knowledge, FIML is ML so the answer is yes. For example, the phrase Full Information Maximum Likelihood is used in Singer and Willett (2004) Applied Longitudinal Data Analysis (Oxford University Press) as a synonym for maximum likelihood. I have seen FIML used to refer to a type of ML estimation where a missing data treatment is included in the estimation procedure (parameter estimates are derived from incomplete cases for only the variables present in the case, rather than simply discarding the cases), at least in the latent-variable SEM context, specifically in AMOS. This may be what Francisco is getting at. To my knowledge, no R packages implement this sort of FIML, for any class of models, although there are other available missing data treatments (EM, MCMC estimation). Chris -- Christopher N. Lawrence, Ph.D. Visiting Assistant Professor of Political Science Millsaps College 1701 N. State St Jackson, MS 39210 (601) 974-1438 / [EMAIL PROTECTED] __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
RE: [R] Reading SAS data into R
--- Begin quoted text From: Gilpin, Scott [EMAIL PROTECTED] -Original Message- From: [EMAIL PROTECTED] [mailto:r-help- [EMAIL PROTECTED] On Behalf Of Søren Højsgaard Sent: Friday, August 27, 2004 11:46 AM To: [EMAIL PROTECTED] Cc: Søren Højsgaard Subject: [R] Reading SAS data into R Dear all, One of my students (whom I am trying to convince to use R) wants to get a fairly large SAS dataset into R (about 150mB). An obvious and simple thing she tried was to write the dataset as a .csv-file and then read that into R, but that takes forever (or something close to that..). The dataset is so large, that exporting it as an Excel file from SAS is not feasible (more than 65000 lines). I am reluctant to ask her to go through all the data base steps (then she'll just stick to SAS...). Can anyone help me out on that one? What platform are you on, and how much memory do you have? 150mb isn't *that* large - but it will depend on your system. See the FAQs regarding memory issues, as well as ?mem.limits and ?gc It also depends on the type of data you're working with (integers take half the space of numerics) and what type of analysis you want to do. The CSV approach should work fine - but you'll want to use scan instead of read.table. You can use scan to read data in chunks (using skip and nlines), do something useful with this chunk of data, run gc(), and then read in another chunk. In general, users I've seen who try to go from SAS to R don't seem to realize that R is not a data manipulation language, and hence just try to shove their entire dataset into R and manipulate it (which is what they would do in SAS). Perl, awk, cut, etc. (not to mention DBMSes) are all very useful for processing data before putting it into R. ---End quoted text R is also a data manipulation language and once you get used to it it is better than SAS at data manipulation, for non-huge datasets. We have many examples of data manipulation with R on our web site http://biostat.mc.vanderbilt.edu (see especially the Alzola and Harrell text). What we do for importing SAS datasets is described at http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/SASexportHowto . This will preserve labels and value labels, handle dates, times, date/times, and get around problems we've faced in importing SAS V5 transport files using the foreign package, by having SAS run PROC EXPORT to create csv files. -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] Handling of special characters by xtable
It seems that xtable does not escape special characters such as % (which indicates a
comment line in LaTeX).
Try these few lines for example:
library(xtable)
q-data.frame(quantile(rnorm(100)))
xtable(q)
This produces:
% latex table generated in R 1.9.1 by xtable 1.2-3 package
% Sat Aug 28 16:11:05 2004
\begin{table}[ht]
\begin{center}
\begin{tabular}{rr}
\hline
quantile.rnorm.100.. \\
\hline
0% $-$2.02 \\
25% $-$0.70 \\
50% $-$0.16 \\
75% 0.54 \\
100% 2.16 \\
\hline
\end{tabular}
\end{center}
\end{table}
-
[[alternative HTML version deleted]]
__
[EMAIL PROTECTED] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] removing invariant columns from a matrix
I'm looking for an efficient way of removing zero-variance columns from a large matrix. Any suggestions? Thanks, - Moises [[alternative HTML version deleted]] __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
RE: [R] removing invariant columns from a matrix
Something like: keep - apply(myData, 2, function(x) diff(range(x)) 0) newData - myData[, keep] Andy From: Moises Hassan I'm looking for an efficient way of removing zero-variance columns from a large matrix. Any suggestions? Thanks, - Moises __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] removing invariant columns from a matrix
Moises Hassan [EMAIL PROTECTED] writes: I'm looking for an efficient way of removing zero-variance columns from a large matrix. Any suggestions? A[,apply(A,2,var)0] -- O__ Peter Dalgaard Blegdamsvej 3 c/ /'_ --- Dept. of Biostatistics 2200 Cph. N (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
RE: [R] ANCOVA
Dear Matt, The sequential sums of squares produced by anova() test for g ignoring x (and the interaction), x after g (and ignoring the interaction), and the x:g interaction after g and x. The second and third test are generally sensible, but the first doesn't adjust for x, which is probably not what you want in general (although you've constructed x to be independent of g, which is typically not the case). Note that you would not usually want to test for equal intercepts in the model that doesn't constrain the slopes to be equal (though you haven't done this): Among other things, 0 is outside the range of x. The Anova() function in the car package will produce so-called type-II tests. I hope that this helps. John -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Oliver Sent: Friday, August 27, 2004 4:10 PM To: [EMAIL PROTECTED] Subject: [R] ANCOVA Dear R-help list, I am attempting to understand the proper formulation of ANCOVA's in R. I would like to test both parallelism and intercept equality for some data sets, so I have generated an artificial data set to ease my understanding. This is what I have done #Limits of random error added to vectors min - -0.1 max - 0.1 x - c(c(1:10), c(1:10))+runif(20, min, max) x1 - c(c(1:10), c(1:10))+runif(20, min, max) y - c(c(1:10), c(10:1))+runif(20, min, max) z - c(c(1:10), c(11:20))+runif(20, min, max) g - as.factor(c(rep(1, 10), rep(2, 10))) #x and x1 have similar slopes and have the similar intercepts, #x and y have different slopes and different intercepts #x and z have similar slopes with different intercepts #These are my full effects models fitx1x - lm(x1~g+x+x:g) fityx - lm(y~g+x+x:g) fitzx - lm(z~g+x+x:g) anova(fitx1x) Analysis of Variance Table Response: x1 Df Sum Sq Mean Sq F value Pr(F) g 1 0.002 0.002 0.3348 0.5709 x 1 163.927 163.927 23456.8319 2e-16 *** g:x1 0.002 0.002 0.2671 0.6123 Residuals 16 0.112 0.007 --- Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 These results confirm that x and x1 do not have significantly different means with respect to g; There is a significant linear relationship between x and x1 independent of g There is no evidence that the slopes of x and x1 in the two g groups is different anova(fityx) Analysis of Variance Table Response: y Df Sum Sq Mean Sq F value Pr(F) g 1 0.012 0.012 1.7344 0.2064 x 1 0.003 0.003 0.4399 0.5166 g:x 1 164.947 164.947 24274.4246 2e-16 *** Residuals 160.1090.007 --- Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 These results confirm that x and y do not have significantly different means with respect to g; There is a not a significant linear relationship between x and y independent of g There is evidence that the slopes of x and y in the two g groups is significantly different. anova(fitzx) Analysis of Variance Table Response: z DfSum Sq Mean Sq F value Pr(F) g 1 501.07501.07 52709.9073 2e-16 *** x 1 165.39165.39 17398.4057 2e-16 *** g:x10.02 0.02 1.7472 0.2048 Residuals 160.15 0.01 --- Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 These results confirm that x and z have significantly different means with respect to g; There is a a significant linear relationship between x and z independent of g There is no evidence that the slopes of x and z in the two g groups is significantly different. What I don't understand is how to formulate the model so that I can tell if the intercepts between the g groups are different. Also, how would I formulate an ANCOVA if I am dealing with Model II regressions? Any help would be greatly appreciated. Matt == When you reach an equilibrium in biology, you're dead. - A. Mandell == Matthew J. Oliver Institute of Marine and Coastal Sciences 71 Dudley Road, New Brunswick New Jersey, 08901 http://marine.rutgers.edu/cool/ __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do
Re: [R] removing invariant columns from a matrix
Both the previous solutions seem to assume a numeric matrix. How about the following: A - array(letters[c(rep(1, 13), rep(2, 13), 1:26)], dim=c(13, 5)) A[, apply(A, 2, function(x)any(x[-1] != x[-length(x)]))] A[, apply(A, 2, function(x)any(x[-1] != x[-length(x)])] enjoy. spencer graves Peter Dalgaard wrote: Moises Hassan [EMAIL PROTECTED] writes: I'm looking for an efficient way of removing zero-variance columns from a large matrix. Any suggestions? A[,apply(A,2,var)0] __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
