On Sep 15, 2009, at 10:48 AM, ivo welch wrote:
dear R wizards: here is the strange question for the day. It seems
to me
that nrow() is very slow. Let me explain what I mean:
ds= data.frame( NA, x=rnorm(10000) ) ## a sample data set
system.time( { for (i in 1:10000) NA } ) ## doing nothing takes
virtually no time
user system elapsed
0.000 0.000 0.001
## this is something that should take time; we need to add 10,000
values
10,000 times
system.time( { for (i in 1:10000) mean(ds$x) } )
user system elapsed
0.416 0.001 0.416
## alas, this should be very fast. it is just reading off an
attribute of
ds. it takes almost a quarter of the time of mean()!
system.time( { for (i in 1:10000) nrow(ds) } )
user system elapsed
0.124 0.001 0.125
I am guessing that you are coming from a statistical paradigm where
there is an
implicit looping construct in a data step. In R you find the number of
rows not
with a loop, but with the nrow function used just once.
> ds= data.frame( NA, x=rnorm(10000) )
> system.time(nrow(ds))
user system elapsed
0 0 0
## here is an alternative way to implement nrows, which is already
much
faster:
system.time( { for (i in 1:10000) length(ds$x) } )
user system elapsed
0.041 0.000 0.041
is there a faster way to learn how big a data frame is?
> length(ds)
[1] 2
> nrow(ds)
[1] 10000
# Or:
> dim(ds)
[1] 10000 2
I know this sounds
silly, but this is inside a "by" statement, where I figure out how
many
observations are in each subset. strangely, this takes a whole lot of
time. I don't believe it is possible to ask "by" to attach an
attribute to
the data frame that stores the number of observations that it is
actually
passing.
pointers appreciated.
regards,
/iaw
--
Ivo Welch ([email protected], [email protected])
[[alternative HTML version deleted]]
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
David Winsemius, MD
Heritage Laboratories
West Hartford, CT
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
