David: Here is a comparison of the gains to be made by vectorization (again, assuming I have interpreted your query correctly)
## create a list of arrays > z <- lapply(seq_len(10000),function(i)array(runif(24),dim=2:4)) ## Using an apply type approach > system.time(ans1 <- array(do.call(mapply,c(sum,z)),dim=2:4)) user system elapsed 0.62 0.00 0.62 ## vectorizing via rowSums and cbind > system.time(ans2 <-array(rowSums(do.call(cbind,z)),dim=2:4)) user system elapsed 0.02 0.00 0.02 > identical(ans1,ans2) [1] TRUE Cheers, Bert On Mon, Apr 16, 2012 at 1:19 PM, David A Vavra <dava...@verizon.net> wrote: > Thanks Bill, > > > > For reasons that aren't important here, I must start from a list. Computing > the sum while generating the tables may be a solution but it means doing > something in one piece of code that is unrelated to the surrounding code. > Bad practice where I'm from. If it's needed it's needed but if I can avoid > doing so, I will. > > > > I haven't done any timing but because of the extra operations of get and > assign, the non-loop implementation will likely suffer. It seems you have > shown this to be true. > > > > DAV > > > > > > -----Original Message----- > From: William Dunlap [mailto:wdun...@tibco.com] > Sent: Monday, April 16, 2012 3:26 PM > To: David A Vavra; 'Bert Gunter' > Cc: r-help@r-project.org > Subject: RE: [R] Effeciently sum 3d table > > > >> Example in partial code: > >> > >> Env <- CreatEnv() # my own function > >> Assign('final',T1-T1,envir=env) > >> L<-listOfTables > >> > >> lapply(L,function(t) { > >> final <- get('final',envir=env) + t > >> assign('final',final,envir=env) > >> NULL > >> }) > > > > First, finish writing that code so it runs and you can make sure its > > output is ok: > > > > L <- lapply(1:50000, function(i) array(i:(i+3), c(2,2))) # list of 50,000 > 2x2 matrices > > env <- new.env() > > assign('final', L[[1]] - L[[1]], envir=env) > > junk <- lapply(L, function(t) { > > final <- get('final', envir=env) + t > > assign('final', final, envir=env) > > NULL > > }) > > get('final', envir=env) > > # [,1] [,2] > > # [1,] 1250025000 1250125000 > > # [2,] 1250075000 1250175000 > >> sum( (2:50001) ) # should be final[2,1] > > # [1] 1250075000 > > > > You asked for something less "clunky". > > You are fighting the system by using get() and assign(), just use > > ordinary expression syntax to get and set variables: > > final <- L[[1]] > > for(i in seq_along(L)[-1]) final <- final + L[[i]] > > final > > # [,1] [,2] > > # [1,] 1250025000 1250125000 > > # [2,] 1250075000 1250175000 > > > > The former took 0.22 seconds on my machine, the latter 0.06. > > > > You don't have to compute the whole list of matrices before > > doing the sum, just add to the current sum when you have > > computed one matrix and then forget about it. > > > > Bill Dunlap > > Spotfire, TIBCO Software > > wdunlap tibco.com > > > > > >> -----Original Message----- > >> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] > On Behalf > >> Of David A Vavra > >> Sent: Monday, April 16, 2012 11:35 AM > >> To: 'Bert Gunter' > >> Cc: r-help@r-project.org > >> Subject: Re: [R] Effeciently sum 3d table > >> > >> Thanks Gunter, > >> > >> I mean what I think is the normal definition of 'sum' as in: > >> T1 + T2 + T3 + ... > >> It never occurred to me that there would be a question. > >> > >> I have gotten the impression that a for loop is very inefficient. Whenever > I > >> change them to lapply calls there is a noticeable improvement in run time > >> for whatever reason. The problem with lapply here is that I effectively > need > >> a global table to hold the final sum. lapply also wants to return a > value. > >> > >> You may be correct that in the long run, the loop is the best. There's a > lot > >> of extraneous memory wastage holding all of the tables in a list as well > as > >> the return 'values'. > >> > >> As an alternate and given a pre-existing list of tables, I was thinking of > >> creating a temporary environment to hold the final result so it could be > >> passed globally to each lapply execution level but that seems clunky and > >> wasteful as well. > >> > >> Example in partial code: > >> > >> Env <- CreatEnv() # my own function > >> Assign('final',T1-T1,envir=env) > >> L<-listOfTables > >> > >> lapply(L,function(t) { > >> final <- get('final',envir=env) + t > >> assign('final',final,envir=env) > >> NULL > >> }) > >> > >> But I was hoping for a more elegant and hopefully more efficient solution. > >> Greg's suggestion for using reduce seems in order but as yet I'm > unfamiliar > >> with the function. > >> > >> DAV > >> > >> > >> > >> -----Original Message----- > >> From: Bert Gunter [mailto:gunter.ber...@gene.com] > >> Sent: Monday, April 16, 2012 12:42 PM > >> To: Greg Snow > >> Cc: David A Vavra; r-help@r-project.org > >> Subject: Re: [R] Effeciently sum 3d table > >> > >> Define "sum" . Do you mean you want to get a single sum for each > >> array? -- get marginal sums for each array? -- get a single array in > >> which each value is the sum of all the individual values at the > >> position? > >> > >> Due thought and consideration for those trying to help by formulating > >> your query carefully and concisely vastly increases the chance of > >> getting a useful answer. See the posting guide -- this is a skill that > >> needs to be learned and the guide is quite helpful. And I must > >> acknowledge that it is a skill that I also have not yet mastered. > >> > >> Concerning your query, I would only note that the two responses from > >> Greg and Petr that you received are unlikely to be significantly > >> faster than just using loops, since both are still essentially looping > >> at the interpreted level. Whether either give you what you want, I do > >> not know. > >> > >> -- Bert > >> > >> On Mon, Apr 16, 2012 at 8:53 AM, Greg Snow <538...@gmail.com> wrote: > >> > Look at the Reduce function. > >> > > >> > On Mon, Apr 16, 2012 at 8:28 AM, David A Vavra <dava...@verizon.net> > >> wrote: > >> >> I have a large number of 3d tables that I wish to sum > >> >> Is there an efficient way to do this? Or perhaps a function I can call? > >> >> > >> >> I tried using do.call("sum",listoftables) but that returns a single > >> value. > >> >> > >> >> So far, it seems only a loop will do the job. > >> >> > >> >> > >> >> TIA, > >> >> DAV > >> > >> > >> -- > >> > >> Bert Gunter > >> Genentech Nonclinical Biostatistics > >> > >> Internal Contact Info: > >> Phone: 467-7374 > >> Website: > >> > http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biost > >> atistics/pdb-ncb-home.htm > >> > >> ______________________________________________ > >> R-help@r-project.org mailing list > >> https://stat.ethz.ch/mailman/listinfo/r-help > >> PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > >> and provide commented, minimal, self-contained, reproducible code. > > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. -- Bert Gunter Genentech Nonclinical Biostatistics Internal Contact Info: Phone: 467-7374 Website: http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.