Thanks Bill,
For reasons that aren't important here, I must start from a list.
Computing
the sum while generating the tables may be a solution but it means
doing
something in one piece of code that is unrelated to the surrounding
code.
Bad practice where I'm from. If it's needed it's needed but if I
can avoid
doing so, I will.
I haven't done any timing but because of the extra operations of
get and
assign, the non-loop implementation will likely suffer. It seems
you have
shown this to be true.
DAV
-----Original Message-----
From: William Dunlap [mailto:wdun...@tibco.com]
Sent: Monday, April 16, 2012 3:26 PM
To: David A Vavra; 'Bert Gunter'
Cc: r-help@r-project.org
Subject: RE: [R] Effeciently sum 3d table
Example in partial code:
Env <- CreatEnv() # my own function
Assign('final',T1-T1,envir=env)
L<-listOfTables
lapply(L,function(t) {
final <- get('final',envir=env) + t
assign('final',final,envir=env)
NULL
})
First, finish writing that code so it runs and you can make sure its
output is ok:
L <- lapply(1:50000, function(i) array(i:(i+3), c(2,2))) # list of
50,000
2x2 matrices
env <- new.env()
assign('final', L[[1]] - L[[1]], envir=env)
junk <- lapply(L, function(t) {
final <- get('final', envir=env) + t
assign('final', final, envir=env)
NULL
})
get('final', envir=env)
# [,1] [,2]
# [1,] 1250025000 1250125000
# [2,] 1250075000 1250175000
sum( (2:50001) ) # should be final[2,1]
# [1] 1250075000
You asked for something less "clunky".
You are fighting the system by using get() and assign(), just use
ordinary expression syntax to get and set variables:
final <- L[[1]]
for(i in seq_along(L)[-1]) final <- final + L[[i]]
final
# [,1] [,2]
# [1,] 1250025000 1250125000
# [2,] 1250075000 1250175000
The former took 0.22 seconds on my machine, the latter 0.06.
You don't have to compute the whole list of matrices before
doing the sum, just add to the current sum when you have
computed one matrix and then forget about it.
Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com
-----Original Message-----
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org
]
On Behalf
Of David A Vavra
Sent: Monday, April 16, 2012 11:35 AM
To: 'Bert Gunter'
Cc: r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
Thanks Gunter,
I mean what I think is the normal definition of 'sum' as in:
T1 + T2 + T3 + ...
It never occurred to me that there would be a question.
I have gotten the impression that a for loop is very inefficient.
Whenever
I
change them to lapply calls there is a noticeable improvement in
run time
for whatever reason. The problem with lapply here is that I
effectively
need
a global table to hold the final sum. lapply also wants to return a
value.
You may be correct that in the long run, the loop is the best.
There's a
lot
of extraneous memory wastage holding all of the tables in a list
as well
as
the return 'values'.
As an alternate and given a pre-existing list of tables, I was
thinking of
creating a temporary environment to hold the final result so it
could be
passed globally to each lapply execution level but that seems
clunky and
wasteful as well.
Example in partial code:
Env <- CreatEnv() # my own function
Assign('final',T1-T1,envir=env)
L<-listOfTables
lapply(L,function(t) {
final <- get('final',envir=env) + t
assign('final',final,envir=env)
NULL
})
But I was hoping for a more elegant and hopefully more efficient
solution.
Greg's suggestion for using reduce seems in order but as yet I'm
unfamiliar
with the function.
DAV
-----Original Message-----
From: Bert Gunter [mailto:gunter.ber...@gene.com]
Sent: Monday, April 16, 2012 12:42 PM
To: Greg Snow
Cc: David A Vavra; r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
Define "sum" . Do you mean you want to get a single sum for each
array? -- get marginal sums for each array? -- get a single array in
which each value is the sum of all the individual values at the
position?
Due thought and consideration for those trying to help by
formulating
your query carefully and concisely vastly increases the chance of
getting a useful answer. See the posting guide -- this is a skill
that
needs to be learned and the guide is quite helpful. And I must
acknowledge that it is a skill that I also have not yet mastered.
Concerning your query, I would only note that the two responses from
Greg and Petr that you received are unlikely to be significantly
faster than just using loops, since both are still essentially
looping
at the interpreted level. Whether either give you what you want, I
do
not know.
-- Bert
On Mon, Apr 16, 2012 at 8:53 AM, Greg Snow <538...@gmail.com> wrote:
Look at the Reduce function.
On Mon, Apr 16, 2012 at 8:28 AM, David A Vavra
<dava...@verizon.net>
wrote:
I have a large number of 3d tables that I wish to sum
Is there an efficient way to do this? Or perhaps a function I
can call?
I tried using do.call("sum",listoftables) but that returns a
single
value.
So far, it seems only a loop will do the job.
TIA,
DAV
--
Bert Gunter
Genentech Nonclinical Biostatistics
Internal Contact Info:
Phone: 467-7374
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biost
atistics/pdb-ncb-home.htm
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
