Re: [R] Trapping option settings
Jim,
Thanks.
It wasn't sure if merely overriding the options function by placing one in
the global environment would guarantee it would be the one actually called.
In any case, I didn't know how to identify the caller. This is quite helpful
and looks promising. I'll give it a try.
DAV
-Original Message-
From: jim holtman [mailto:jholt...@gmail.com]
Sent: Monday, July 16, 2012 9:59 PM
To: David A Vavra
Cc: r-help@r-project.org
Subject: Re: [R] Trapping option settings
Here is one way by redefining 'options' so you can check for 'width'
and then call the 'options' in 'base':
options - # define 'options' in Global
+ function(...)
+ {
+ args - list(...) # get arguments
+ if ('width' %in% names(args)){ # see if 'width' is in them
+ .caller - sys.calls() # get where called from
+ if (length(.caller) == 1) # called from command line
+ .caller - Rgui
+ else .caller - as.character(.caller[[length(.caller) - 1]])[1]
+ cat(width being changed:, args[['width']], Called from,
.caller, '\n')
+ }
+ base::options(...) # call the real options
+ }
options(width = 123) # test at command line
width being changed: 123 Called from Rgui
f.myfunc - function() options(width = 456) # within a function
f.myfunc()
width being changed: 456 Called from f.myfunc
On Mon, Jul 16, 2012 at 9:01 PM, David A Vavra dava...@verizon.net wrote:
Something has been changing the setting the width option to 1 and not
resetting it. It does this intermittently. What I would like to do is trap
changing the setting so I can determine where the change is occurring and
hopefully fix it.
Is there any easy way to do this?
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Jim Holtman
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Trapping option settings
Thanks. Also helpful.
DAV
-Original Message-
From: William Dunlap [mailto:wdun...@tibco.com]
Sent: Wednesday, July 18, 2012 4:20 PM
To: David A Vavra; 'jim holtman'
Cc: r-help@r-project.org
Subject: RE: [R] Trapping option settings
Try using trace(), as in
trace(options, quote(print(as.list(sys.calls()
Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com
-Original Message-
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
On
Behalf Of David A Vavra
Sent: Wednesday, July 18, 2012 1:00 PM
To: 'jim holtman'
Cc: r-help@r-project.org
Subject: Re: [R] Trapping option settings
Jim,
Thanks.
It wasn't sure if merely overriding the options function by placing one in
the global environment would guarantee it would be the one actually
called.
In any case, I didn't know how to identify the caller. This is quite
helpful
and looks promising. I'll give it a try.
DAV
-Original Message-
From: jim holtman [mailto:jholt...@gmail.com]
Sent: Monday, July 16, 2012 9:59 PM
To: David A Vavra
Cc: r-help@r-project.org
Subject: Re: [R] Trapping option settings
Here is one way by redefining 'options' so you can check for 'width'
and then call the 'options' in 'base':
options - # define 'options' in Global
+ function(...)
+ {
+ args - list(...) # get arguments
+ if ('width' %in% names(args)){ # see if 'width' is in them
+ .caller - sys.calls() # get where called from
+ if (length(.caller) == 1) # called from command line
+ .caller - Rgui
+ else .caller - as.character(.caller[[length(.caller) - 1]])[1]
+ cat(width being changed:, args[['width']], Called from,
.caller, '\n')
+ }
+ base::options(...) # call the real options
+ }
options(width = 123) # test at command line
width being changed: 123 Called from Rgui
f.myfunc - function() options(width = 456) # within a function
f.myfunc()
width being changed: 456 Called from f.myfunc
On Mon, Jul 16, 2012 at 9:01 PM, David A Vavra dava...@verizon.net
wrote:
Something has been changing the setting the width option to 1 and
not
resetting it. It does this intermittently. What I would like to do is
trap
changing the setting so I can determine where the change is occurring
and
hopefully fix it.
Is there any easy way to do this?
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Jim Holtman
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] Trapping option settings
Something has been changing the setting the width option to 1 and not resetting it. It does this intermittently. What I would like to do is trap changing the setting so I can determine where the change is occurring and hopefully fix it. Is there any easy way to do this? __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] step function stops with subscript out of bounds
I've been having a problem using the step function to evaluate models. I've
simplified the code and get the same problem using the dataset Titanic. The
relevant code and output is below. The problem disappears (i.e., 'step' runs
correctly) if I rerun the code but change the 'loglm' call to explicitly
reference Titanic instead of X (as in: loglm(as.formula(Y),data=Titanic)).
What is causing this?
TIA,
DAV
--
catn-function(...) cat(...,\n)
local({ X-Titanic; print(class(X));
Y-paste('~',paste(names(dimnames(X)),collapse=*));
print(Y);
sm-loglm(as.formula(Y),data=X);
catn(SM); print(sm); catn('running');
step(sm,direction='backward') })
Output:
[1] table
[1] ~ Class*Sex*Age*Survived
SM
Call:
loglm(formula = as.formula(Y), data = X)
Statistics:
X^2 df P( X^2)
Likelihood Ratio 0 01
Pearson NaN 01
running
Start: AIC=64
~Class * Sex * Age * Survived
Error in loglin(data, margins, start = start, fit = fitted, param = param,
:
subscript out of bounds
Enter a frame number, or 0 to exit
1: local({
X - Titanic
print(class(X))
Y - paste(~, paste(names(dimnames(X)), collapse = *))
print(Y)
sm - loglm(as.formula(Y), data = X
2: eval.parent(substitute(eval(quote(expr), envir)))
3: eval(expr, p)
4: eval(expr, envir, enclos)
5: eval(quote({
X - Titanic
print(class(X))
Y - paste(~, paste(names(dimnames(X)), collapse = *))
print(Y)
sm - loglm(as.formula(Y), dat
6: eval(expr, envir, enclos)
7: #1: step(sm, direction = backward)
8: #1: drop1(fit, scope$drop, scale = scale, trace = trace, k = k, ...)
9: #1: drop1.default(fit, scope$drop, scale = scale, trace = trace, k = k,
...)
10: #1: update(object, as.formula(paste(~ . -, tt)), evaluate = FALSE)
11: #1: update.loglm(object, as.formula(paste(~ . -, tt)), evaluate =
FALSE)
12: #1: eval.parent(call)
13: #1: eval(expr, p)
14: #1: eval(expr, envir, enclos)
15: #1: loglm(formula = ~Class + Sex + Age + Survived + Class:Sex +
Class:Age + Sex:Age + Class:Survived + Sex:Survived + Age:Survived +
Class:Sex:Age + Class:
16: #1: loglm1(formula, data, ..., .call = .call, .formula = .formula)
17: #1: loglm1.default(formula, data, ..., .call = .call, .formula =
.formula)
18: #1: loglin(data, margins, start = start, fit = fitted, param = param,
eps = eps, iter = iter, print = print)
Selection: 0
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] step function stops with subscript out of bounds
Thanks.
I guess I still don't understand what's going on. It's not at all intuitive
that the table used should be in the search path. Why is it searching for
the table? Isn't the table already stored in the model? If the documentation
says this, I haven't found it. Needing to know what names I can't use is a
bit disconcerting. I can think of names with high probability of uniqueness
but how could I ever be sure?
Table X is built on the fly in a script. What I supplied here was a
portion of the remaining script. I suppose the solution is to store it in an
environment in the search path.
DAV
-Original Message-
From: Prof Brian Ripley [mailto:rip...@stats.ox.ac.uk]
Sent: Thursday, May 17, 2012 6:46 AM
To: David A Vavra
Cc: r-help@r-project.org
Subject: Re: [R] step function stops with subscript out of bounds
On 17/05/2012 09:25, David A Vavra wrote:
I've been having a problem using the step function to evaluate models.
I've
simplified the code and get the same problem using the dataset Titanic.
The
relevant code and output is below. The problem disappears (i.e., 'step'
runs
correctly) if I rerun the code but change the 'loglm' call to explicitly
reference Titanic instead of X (as in: loglm(as.formula(Y),data=Titanic)).
What is causing this?
A lack of understanding of 'non-standard evaluation'. X (or at least,
the X you want) is not visible from the standard search path.
TIA,
DAV
--
catn-function(...) cat(...,\n)
local({ X-Titanic; print(class(X));
Y-paste('~',paste(names(dimnames(X)),collapse=*));
print(Y);
sm-loglm(as.formula(Y),data=X);
catn(SM); print(sm); catn('running');
step(sm,direction='backward') })
Which will tell you
Error in eval(expr, envir, enclos) : could not find function loglm
If you correct that and use a vanilla session you will get
Error in loglm(formula = ~Class + Sex + Age + Survived + Class:Sex +
Class:Age + :
object 'X' not found
which is more informative.
So the solution is to
- use less easily masked names than 'X'.
- ensure the data object is visible on the search path.
Output:
[1] table
[1] ~ Class*Sex*Age*Survived
SM
Call:
loglm(formula = as.formula(Y), data = X)
Statistics:
X^2 df P( X^2)
Likelihood Ratio 0 01
Pearson NaN 01
running
Start: AIC=64
~Class * Sex * Age * Survived
Error in loglin(data, margins, start = start, fit = fitted, param = param,
:
subscript out of bounds
Enter a frame number, or 0 to exit
1: local({
X- Titanic
print(class(X))
Y- paste(~, paste(names(dimnames(X)), collapse = *))
print(Y)
sm- loglm(as.formula(Y), data = X
2: eval.parent(substitute(eval(quote(expr), envir)))
3: eval(expr, p)
4: eval(expr, envir, enclos)
5: eval(quote({
X- Titanic
print(class(X))
Y- paste(~, paste(names(dimnames(X)), collapse = *))
print(Y)
sm- loglm(as.formula(Y), dat
6: eval(expr, envir, enclos)
7: #1: step(sm, direction = backward)
8: #1: drop1(fit, scope$drop, scale = scale, trace = trace, k = k, ...)
9: #1: drop1.default(fit, scope$drop, scale = scale, trace = trace, k =
k,
...)
10: #1: update(object, as.formula(paste(~ . -, tt)), evaluate = FALSE)
11: #1: update.loglm(object, as.formula(paste(~ . -, tt)), evaluate =
FALSE)
12: #1: eval.parent(call)
13: #1: eval(expr, p)
14: #1: eval(expr, envir, enclos)
15: #1: loglm(formula = ~Class + Sex + Age + Survived + Class:Sex +
Class:Age + Sex:Age + Class:Survived + Sex:Survived + Age:Survived +
Class:Sex:Age + Class:
16: #1: loglm1(formula, data, ..., .call = .call, .formula = .formula)
17: #1: loglm1.default(formula, data, ..., .call = .call, .formula =
.formula)
18: #1: loglin(data, margins, start = start, fit = fitted, param = param,
eps = eps, iter = iter, print = print)
Selection: 0
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Brian D. Ripley, rip...@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] Effeciently sum 3d table
I have a large number of 3d tables that I wish to sum Is there an efficient way to do this? Or perhaps a function I can call? I tried using do.call(sum,listoftables) but that returns a single value. So far, it seems only a loop will do the job. TIA, DAV __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Effeciently sum 3d table
Thanks Gunter,
I mean what I think is the normal definition of 'sum' as in:
T1 + T2 + T3 + ...
It never occurred to me that there would be a question.
I have gotten the impression that a for loop is very inefficient. Whenever I
change them to lapply calls there is a noticeable improvement in run time
for whatever reason. The problem with lapply here is that I effectively need
a global table to hold the final sum. lapply also wants to return a value.
You may be correct that in the long run, the loop is the best. There's a lot
of extraneous memory wastage holding all of the tables in a list as well as
the return 'values'.
As an alternate and given a pre-existing list of tables, I was thinking of
creating a temporary environment to hold the final result so it could be
passed globally to each lapply execution level but that seems clunky and
wasteful as well.
Example in partial code:
Env - CreatEnv() # my own function
Assign('final',T1-T1,envir=env)
L-listOfTables
lapply(L,function(t) {
final - get('final',envir=env) + t
assign('final',final,envir=env)
NULL
})
But I was hoping for a more elegant and hopefully more efficient solution.
Greg's suggestion for using reduce seems in order but as yet I'm unfamiliar
with the function.
DAV
-Original Message-
From: Bert Gunter [mailto:gunter.ber...@gene.com]
Sent: Monday, April 16, 2012 12:42 PM
To: Greg Snow
Cc: David A Vavra; r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
Define sum . Do you mean you want to get a single sum for each
array? -- get marginal sums for each array? -- get a single array in
which each value is the sum of all the individual values at the
position?
Due thought and consideration for those trying to help by formulating
your query carefully and concisely vastly increases the chance of
getting a useful answer. See the posting guide -- this is a skill that
needs to be learned and the guide is quite helpful. And I must
acknowledge that it is a skill that I also have not yet mastered.
Concerning your query, I would only note that the two responses from
Greg and Petr that you received are unlikely to be significantly
faster than just using loops, since both are still essentially looping
at the interpreted level. Whether either give you what you want, I do
not know.
-- Bert
On Mon, Apr 16, 2012 at 8:53 AM, Greg Snow 538...@gmail.com wrote:
Look at the Reduce function.
On Mon, Apr 16, 2012 at 8:28 AM, David A Vavra dava...@verizon.net
wrote:
I have a large number of 3d tables that I wish to sum
Is there an efficient way to do this? Or perhaps a function I can call?
I tried using do.call(sum,listoftables) but that returns a single
value.
So far, it seems only a loop will do the job.
TIA,
DAV
--
Bert Gunter
Genentech Nonclinical Biostatistics
Internal Contact Info:
Phone: 467-7374
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biost
atistics/pdb-ncb-home.htm
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Effeciently sum 3d table
Thanks Petr, I'm after T1 + T2 + T3 + ... and your solution is giving a list of n items each containing sum(T[i]). I guess I should have been clearer in stating what I need. Cheers, DAV -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Petr Savicky Sent: Monday, April 16, 2012 11:07 AM To: r-help@r-project.org Subject: Re: [R] Effeciently sum 3d table On Mon, Apr 16, 2012 at 10:28:43AM -0400, David A Vavra wrote: I have a large number of 3d tables that I wish to sum Is there an efficient way to do this? Or perhaps a function I can call? I tried using do.call(sum,listoftables) but that returns a single value. So far, it seems only a loop will do the job. Hi. Use lapply(), for example listoftables - list(array(1:8, dim=c(2, 2, 2)), array(2:9, dim=c(2, 2, 2))) lapply(listoftables, sum) [[1]] [1] 36 [[2]] [1] 44 Hope this helps. Petr Savicky. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Effeciently sum 3d table
Thanks Greg, I think this may be what I'm after but the documentation for it isn't particularly clear. I hate it when someone documents a piece of code saying it works kinda like some other code (running elsewhere, of course) making the tacit assumption that everybody will immediately know what that means and implies. I'm sure I'll understand it once I know what it is trying to say. :) There's an item in the examples which may be exactly what I'm after. DAV -Original Message- From: Greg Snow [mailto:538...@gmail.com] Sent: Monday, April 16, 2012 11:54 AM To: David A Vavra Cc: r-help@r-project.org Subject: Re: [R] Effeciently sum 3d table Look at the Reduce function. On Mon, Apr 16, 2012 at 8:28 AM, David A Vavra dava...@verizon.net wrote: I have a large number of 3d tables that I wish to sum Is there an efficient way to do this? Or perhaps a function I can call? I tried using do.call(sum,listoftables) but that returns a single value. So far, it seems only a loop will do the job. TIA, DAV -- Gregory (Greg) L. Snow Ph.D. 538...@gmail.com __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Effeciently sum 3d table
Bert,
My apologies on the name.
I haven't kept any data on loop times. I don't know why lapply seems faster
but the difference is quite noticeable. It has struck me as odd. I would
have thought lapply would be slower. It has taken an effort to change my
thinking to force fit solutions to it but I've gotten used to it. As of now
I reserve loops to times when there are only a few iterations (as in 10) and
to solutions that require passing large amounts of information among
iterations. lapply is particularly handy when constructing lists.
As for vectorizing, see the code below. Note that it uses mapply but that
simply may have made implementation easier. However, if vectorizing gives an
improvement over looping, the mapply may be the reason.
f-function(x,y,z) catn(do something)
Vectorize(f,c('x','y'))
function (x, y, z)
{
args - lapply(as.list(match.call())[-1L], eval, parent.frame())
names - if (is.null(names(args)))
character(length(args))
else names(args)
dovec - names %in% vectorize.args
do.call(mapply, c(FUN = FUN, args[dovec], MoreArgs =
list(args[!dovec]),
SIMPLIFY = SIMPLIFY, USE.NAMES = USE.NAMES))
}
environment: 0x7fb3442553c8
DAV
-Original Message-
From: Bert Gunter [mailto:gunter.ber...@gene.com]
Sent: Monday, April 16, 2012 3:07 PM
To: David A Vavra
Cc: r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
David:
1. My first name is Bert.
2. It never occurred to me that there would be a question.
Indeed. But in fact you got solutions for two different
interpretations (Greg's is what you wanted). That is what I meant when
I said that clarity in asking the question is important.
3. I have gotten the impression that a for loop is very inefficient.
Whenever I
change them to lapply calls there is a noticeable improvement in run time
for whatever reason.
I'd like to see your data on this. My experience is that they are
typically comparable. Chambers in his Software for Data Analysis
book says (pp 213): (with apply type functions rather than explicit
loops), The computation should run faster... However, none of the
apply mechanisms changes the number of times the supplied functions is
called, so serious improvements will be limited to iterating simple
calculations many times.
4. You can get serious improvements by vectorizing; and you can do
that here, if I understand correctly, because all your arrays have
identical dim = d. Here's how:
## assume your list of arrays is in listoftables
alldat - do.call(cbind,listoftables) ## this might be the slow part
ans - array(.rowSums (allDat), dim = d)
See ?rowSums for explanations and caveats, especially with NA's .
Cheers,
Bert
On Mon, Apr 16, 2012 at 11:35 AM, David A Vavra dava...@verizon.net wrote:
Thanks Gunter,
I mean what I think is the normal definition of 'sum' as in:
T1 + T2 + T3 + ...
It never occurred to me that there would be a question.
I have gotten the impression that a for loop is very inefficient. Whenever
I
change them to lapply calls there is a noticeable improvement in run time
for whatever reason. The problem with lapply here is that I effectively
need
a global table to hold the final sum. lapply also wants to return a
value.
You may be correct that in the long run, the loop is the best. There's a
lot
of extraneous memory wastage holding all of the tables in a list as well
as
the return 'values'.
As an alternate and given a pre-existing list of tables, I was thinking of
creating a temporary environment to hold the final result so it could be
passed globally to each lapply execution level but that seems clunky and
wasteful as well.
Example in partial code:
Env - CreatEnv() # my own function
Assign('final',T1-T1,envir=env)
L-listOfTables
lapply(L,function(t) {
final - get('final',envir=env) + t
assign('final',final,envir=env)
NULL
})
But I was hoping for a more elegant and hopefully more efficient solution.
Greg's suggestion for using reduce seems in order but as yet I'm
unfamiliar
with the function.
DAV
-Original Message-
From: Bert Gunter [mailto:gunter.ber...@gene.com]
Sent: Monday, April 16, 2012 12:42 PM
To: Greg Snow
Cc: David A Vavra; r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
Define sum . Do you mean you want to get a single sum for each
array? -- get marginal sums for each array? -- get a single array in
which each value is the sum of all the individual values at the
position?
Due thought and consideration for those trying to help by formulating
your query carefully and concisely vastly increases the chance of
getting a useful answer. See the posting guide -- this is a skill that
needs to be learned and the guide is quite helpful. And I must
acknowledge that it is a skill that I also have not yet mastered.
Concerning your query, I would only note that the two responses from
Greg and Petr that you received are unlikely to be significantly
Re: [R] Effeciently sum 3d table
even now you _could_ be clearer I fail to see why it's unclear. I'm after T1 + T2 + T3 + ... Which would be one number ... i.e. the result you originally said you did not want. I think it's precisely what I want. If I have two 3d tables, T1 and T2, then say either 1) T1 + T2 2) T1 - T2 (1) yields a third table equal to the sum of the individual cells and (2) yields a table full of zeroes. At least it does for matrices. Are you saying the T1+T2+T3+... above is equivalent to: sum(T1)+sum(T2)+sum(T3)+ when the table has more than 2d? I tried it out by hand I get the result I'm after. What I want is a general solution. Reduce may be the answer but I find the documentation for it a bit daunting. Not to mention that it is far from obvious that I should have originally thought of using it. DAV -Original Message- From: David Winsemius [mailto:dwinsem...@comcast.net] Sent: Monday, April 16, 2012 3:26 PM To: David A Vavra Cc: 'Petr Savicky'; r-help@r-project.org Subject: Re: [R] Effeciently sum 3d table On Apr 16, 2012, at 2:43 PM, David A Vavra wrote: Thanks Petr, I'm after T1 + T2 + T3 + ... Which would be one number ... i.e. the result you originally said you did not want. and your solution is giving a list of n items each containing sum(T[i]). I guess I should have been clearer in stating what I need. Or even now you _could_ be clearer. Do you want successive partial sums? That would yield to: Reduce(+, listoftables, accumaulate=TRUE) Cheers, DAV -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org ] On Behalf Of Petr Savicky Sent: Monday, April 16, 2012 11:07 AM To: r-help@r-project.org Subject: Re: [R] Effeciently sum 3d table On Mon, Apr 16, 2012 at 10:28:43AM -0400, David A Vavra wrote: I have a large number of 3d tables that I wish to sum Is there an efficient way to do this? Or perhaps a function I can call? I tried using do.call(sum,listoftables) but that returns a single value. So far, it seems only a loop will do the job. Hi. Use lapply(), for example listoftables - list(array(1:8, dim=c(2, 2, 2)), array(2:9, dim=c(2, 2, 2))) lapply(listoftables, sum) [[1]] [1] 36 [[2]] [1] 44 Hope this helps. Petr Savicky. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. David Winsemius, MD West Hartford, CT __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Effeciently sum 3d table
Thanks Bill,
For reasons that aren't important here, I must start from a list. Computing
the sum while generating the tables may be a solution but it means doing
something in one piece of code that is unrelated to the surrounding code.
Bad practice where I'm from. If it's needed it's needed but if I can avoid
doing so, I will.
I haven't done any timing but because of the extra operations of get and
assign, the non-loop implementation will likely suffer. It seems you have
shown this to be true.
DAV
-Original Message-
From: William Dunlap [mailto:wdun...@tibco.com]
Sent: Monday, April 16, 2012 3:26 PM
To: David A Vavra; 'Bert Gunter'
Cc: r-help@r-project.org
Subject: RE: [R] Effeciently sum 3d table
Example in partial code:
Env - CreatEnv() # my own function
Assign('final',T1-T1,envir=env)
L-listOfTables
lapply(L,function(t) {
final - get('final',envir=env) + t
assign('final',final,envir=env)
NULL
})
First, finish writing that code so it runs and you can make sure its
output is ok:
L - lapply(1:5, function(i) array(i:(i+3), c(2,2))) # list of 50,000
2x2 matrices
env - new.env()
assign('final', L[[1]] - L[[1]], envir=env)
junk - lapply(L, function(t) {
final - get('final', envir=env) + t
assign('final', final, envir=env)
NULL
})
get('final', envir=env)
#[,1] [,2]
# [1,] 1250025000 1250125000
# [2,] 1250075000 1250175000
sum( (2:50001) ) # should be final[2,1]
# [1] 1250075000
You asked for something less clunky.
You are fighting the system by using get() and assign(), just use
ordinary expression syntax to get and set variables:
final - L[[1]]
for(i in seq_along(L)[-1]) final - final + L[[i]]
final
# [,1] [,2]
# [1,] 1250025000 1250125000
# [2,] 1250075000 1250175000
The former took 0.22 seconds on my machine, the latter 0.06.
You don't have to compute the whole list of matrices before
doing the sum, just add to the current sum when you have
computed one matrix and then forget about it.
Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com
-Original Message-
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
On Behalf
Of David A Vavra
Sent: Monday, April 16, 2012 11:35 AM
To: 'Bert Gunter'
Cc: r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
Thanks Gunter,
I mean what I think is the normal definition of 'sum' as in:
T1 + T2 + T3 + ...
It never occurred to me that there would be a question.
I have gotten the impression that a for loop is very inefficient. Whenever
I
change them to lapply calls there is a noticeable improvement in run time
for whatever reason. The problem with lapply here is that I effectively
need
a global table to hold the final sum. lapply also wants to return a
value.
You may be correct that in the long run, the loop is the best. There's a
lot
of extraneous memory wastage holding all of the tables in a list as well
as
the return 'values'.
As an alternate and given a pre-existing list of tables, I was thinking of
creating a temporary environment to hold the final result so it could be
passed globally to each lapply execution level but that seems clunky and
wasteful as well.
Example in partial code:
Env - CreatEnv() # my own function
Assign('final',T1-T1,envir=env)
L-listOfTables
lapply(L,function(t) {
final - get('final',envir=env) + t
assign('final',final,envir=env)
NULL
})
But I was hoping for a more elegant and hopefully more efficient solution.
Greg's suggestion for using reduce seems in order but as yet I'm
unfamiliar
with the function.
DAV
-Original Message-
From: Bert Gunter [mailto:gunter.ber...@gene.com]
Sent: Monday, April 16, 2012 12:42 PM
To: Greg Snow
Cc: David A Vavra; r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
Define sum . Do you mean you want to get a single sum for each
array? -- get marginal sums for each array? -- get a single array in
which each value is the sum of all the individual values at the
position?
Due thought and consideration for those trying to help by formulating
your query carefully and concisely vastly increases the chance of
getting a useful answer. See the posting guide -- this is a skill that
needs to be learned and the guide is quite helpful. And I must
acknowledge that it is a skill that I also have not yet mastered.
Concerning your query, I would only note that the two responses from
Greg and Petr that you received are unlikely to be significantly
faster than just using loops, since both are still essentially looping
at the interpreted level. Whether either give you what you want, I do
not know.
-- Bert
On Mon, Apr 16, 2012 at 8:53 AM, Greg Snow 538...@gmail.com wrote:
Look at the Reduce function.
On Mon
Re: [R] Effeciently sum 3d table
OK. I'll take your word for it. The mapply function calls do_mapply so I
would have thought it is passing the operation down to the C code. I haven't
tracked it any further than below.
mapply
function (FUN, ..., MoreArgs = NULL, SIMPLIFY = TRUE, USE.NAMES = TRUE)
{
FUN - match.fun(FUN)
dots - list(...)
answer - .Call(do_mapply, FUN, dots, MoreArgs, environment(),
PACKAGE = base)
... etc.
-Original Message-
From: Bert Gunter [mailto:gunter.ber...@gene.com]
Sent: Monday, April 16, 2012 4:13 PM
To: David A Vavra
Cc: r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
For purposes of clarity only...
On Mon, Apr 16, 2012 at 12:40 PM, David A Vavra dava...@verizon.net wrote:
Bert,
My apologies on the name.
I haven't kept any data on loop times. I don't know why lapply seems
faster
but the difference is quite noticeable. It has struck me as odd. I would
have thought lapply would be slower. It has taken an effort to change my
thinking to force fit solutions to it but I've gotten used to it. As of
now
I reserve loops to times when there are only a few iterations (as in 10)
and
to solutions that require passing large amounts of information among
iterations. lapply is particularly handy when constructing lists.
As for vectorizing, see the code below.
No. Despite the name, this is **not** what I mean by vectorization.
What I mean is pushing the loops down to the C level rather than doing
them at the interpreted level, which is where your code below still
leaves you.
-- Bert
Note that it uses mapply but that
simply may have made implementation easier. However, if vectorizing gives
an
improvement over looping, the mapply may be the reason.
f-function(x,y,z) catn(do something)
Vectorize(f,c('x','y'))
function (x, y, z)
{
args - lapply(as.list(match.call())[-1L], eval, parent.frame())
names - if (is.null(names(args)))
character(length(args))
else names(args)
dovec - names %in% vectorize.args
do.call(mapply, c(FUN = FUN, args[dovec], MoreArgs =
list(args[!dovec]),
SIMPLIFY = SIMPLIFY, USE.NAMES = USE.NAMES))
}
environment: 0x7fb3442553c8
DAV
-Original Message-
From: Bert Gunter [mailto:gunter.ber...@gene.com]
Sent: Monday, April 16, 2012 3:07 PM
To: David A Vavra
Cc: r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
David:
1. My first name is Bert.
2. It never occurred to me that there would be a question.
Indeed. But in fact you got solutions for two different
interpretations (Greg's is what you wanted). That is what I meant when
I said that clarity in asking the question is important.
3. I have gotten the impression that a for loop is very inefficient.
Whenever I
change them to lapply calls there is a noticeable improvement in run time
for whatever reason.
I'd like to see your data on this. My experience is that they are
typically comparable. Chambers in his Software for Data Analysis
book says (pp 213): (with apply type functions rather than explicit
loops), The computation should run faster... However, none of the
apply mechanisms changes the number of times the supplied functions is
called, so serious improvements will be limited to iterating simple
calculations many times.
4. You can get serious improvements by vectorizing; and you can do
that here, if I understand correctly, because all your arrays have
identical dim = d. Here's how:
## assume your list of arrays is in listoftables
alldat - do.call(cbind,listoftables) ## this might be the slow part
ans - array(.rowSums (allDat), dim = d)
See ?rowSums for explanations and caveats, especially with NA's .
Cheers,
Bert
On Mon, Apr 16, 2012 at 11:35 AM, David A Vavra dava...@verizon.net
wrote:
Thanks Gunter,
I mean what I think is the normal definition of 'sum' as in:
T1 + T2 + T3 + ...
It never occurred to me that there would be a question.
I have gotten the impression that a for loop is very inefficient.
Whenever
I
change them to lapply calls there is a noticeable improvement in run time
for whatever reason. The problem with lapply here is that I effectively
need
a global table to hold the final sum. lapply also wants to return a
value.
You may be correct that in the long run, the loop is the best. There's a
lot
of extraneous memory wastage holding all of the tables in a list as well
as
the return 'values'.
As an alternate and given a pre-existing list of tables, I was thinking
of
creating a temporary environment to hold the final result so it could be
passed globally to each lapply execution level but that seems clunky and
wasteful as well.
Example in partial code:
Env - CreatEnv() # my own function
Assign('final',T1-T1,envir=env)
L-listOfTables
lapply(L,function(t) {
final - get('final',envir=env) + t
assign('final',final,envir=env)
NULL
})
But I was hoping for a more elegant and hopefully more efficient
solution.
Greg's
Re: [R] Effeciently sum 3d table
OK, then. Thanks. I've read the docs more carefully and Reduce does indeed look like the ticket. For whatever reason, the first time I looked at the documentation my initial reaction was: huh? DAV -Original Message- From: David Winsemius [mailto:dwinsem...@comcast.net] Sent: Monday, April 16, 2012 4:55 PM To: David A Vavra Cc: r-help@r-project.org Subject: Re: [R] Effeciently sum 3d table On Apr 16, 2012, at 4:04 PM, David A Vavra wrote: even now you _could_ be clearer I fail to see why it's unclear. I'm after T1 + T2 + T3 + ... Which would be one number ... i.e. the result you originally said you did not want. I think it's precisely what I want. If I have two 3d tables, T1 and T2, then say either 1) T1 + T2 2) T1 - T2 (1) yields a third table equal to the sum of the individual cells and (2) yields a table full of zeroes. At least it does for matrices. Are you saying the T1+T2+T3+... above is equivalent to: sum(T1)+sum(T2)+sum(T3)+ when the table has more than 2d? I tried it out by hand I get the result I'm after. For me (with my slightly constricted mindset) it would have been clearer to have started out talking about matrices and arrays. An example would have save a bunch of time. What I want is a general solution. Reduce may be the answer but I find the documentation for it a bit daunting. Not to mention that it is far from obvious that I should have originally thought of using it. It is a function designed to do exactly what you requested: Reduce uses a binary function to successively combine the elements of a given vector. As it turns out the term 'vector' in this case includes lists of classed and/or dimensioned objects rather than being restricted to atomic vectors. -- David. DAV -Original Message- From: David Winsemius [mailto:dwinsem...@comcast.net] Sent: Monday, April 16, 2012 3:26 PM To: David A Vavra Cc: 'Petr Savicky'; r-help@r-project.org Subject: Re: [R] Effeciently sum 3d table On Apr 16, 2012, at 2:43 PM, David A Vavra wrote: Thanks Petr, I'm after T1 + T2 + T3 + ... Which would be one number ... i.e. the result you originally said you did not want. and your solution is giving a list of n items each containing sum(T[i]). I guess I should have been clearer in stating what I need. Or even now you _could_ be clearer. Do you want successive partial sums? That would yield to: Reduce(+, listoftables, accumaulate=TRUE) Cheers, DAV -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org ] On Behalf Of Petr Savicky Sent: Monday, April 16, 2012 11:07 AM To: r-help@r-project.org Subject: Re: [R] Effeciently sum 3d table On Mon, Apr 16, 2012 at 10:28:43AM -0400, David A Vavra wrote: I have a large number of 3d tables that I wish to sum Is there an efficient way to do this? Or perhaps a function I can call? I tried using do.call(sum,listoftables) but that returns a single value. So far, it seems only a loop will do the job. Hi. Use lapply(), for example listoftables - list(array(1:8, dim=c(2, 2, 2)), array(2:9, dim=c(2, 2, 2))) lapply(listoftables, sum) [[1]] [1] 36 [[2]] [1] 44 Hope this helps. Petr Savicky. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. David Winsemius, MD West Hartford, CT David Winsemius, MD West Hartford, CT __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Effeciently sum 3d table
Thanks again, Greg. I must have gotten up on the wrong side of the keyboard
this morning and been having a spate of dim insight. What you've said here
makes things clearer.
DAV
-Original Message-
From: Greg Snow [mailto:538...@gmail.com]
Sent: Monday, April 16, 2012 5:42 PM
To: David A Vavra
Cc: r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
Here is a simple example:
mylist - replicate(4, matrix(rnorm(12), ncol=3), simplify=FALSE)
A - Reduce( `+`, mylist )
B - mylist[[1]] + mylist[[2]] + mylist[[3]] + mylist[[4]]
all.equal(A,B)
[1] TRUE
Basically what Reduce does is it first applies the function (`+` in
this case) to the 1st 2 elements of mylist, then applies it to that
result and the 3rd element, then that result and the 4th element (and
would continue on if mylist had more than 4 elements). It is
basically a way to create functions like sum from functions like `+`
which only work on 2 objects at a time.
Another way to see what it is doing is to run something like:
Reduce( function(a,b){ cat(I am adding,a,and,b,\n); a+b }, 1:10 )
The Reduce function will probably not be any faster than a really well
written loop, but will probably be faster (both to write the command
and to run) than a poorly designed naive loop application.
On Mon, Apr 16, 2012 at 12:52 PM, David A Vavra dava...@verizon.net wrote:
Thanks Greg,
I think this may be what I'm after but the documentation for it isn't
particularly clear. I hate it when someone documents a piece of code
saying
it works kinda like some other code (running elsewhere, of course) making
the tacit assumption that everybody will immediately know what that means
and implies.
I'm sure I'll understand it once I know what it is trying to say. :)
There's
an item in the examples which may be exactly what I'm after.
DAV
-Original Message-
From: Greg Snow [mailto:538...@gmail.com]
Sent: Monday, April 16, 2012 11:54 AM
To: David A Vavra
Cc: r-help@r-project.org
Subject: Re: [R] Effeciently sum 3d table
Look at the Reduce function.
On Mon, Apr 16, 2012 at 8:28 AM, David A Vavra dava...@verizon.net
wrote:
I have a large number of 3d tables that I wish to sum
Is there an efficient way to do this? Or perhaps a function I can call?
I tried using do.call(sum,listoftables) but that returns a single
value.
So far, it seems only a loop will do the job.
TIA,
DAV
--
Gregory (Greg) L. Snow Ph.D.
538...@gmail.com
--
Gregory (Greg) L. Snow Ph.D.
538...@gmail.com
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] Add grid lines to levelplot
I'm using the levelplot function in the lattice package. I am plotting a grid of cells and I want grid lines drawn between cells. I've spent a lot of time trying different options. I've also looked at panel.levelplot. The border parameter seems to be ignored or it doesn't mean cell border color. The panel.levelplot calls grid.rect but forces lwd=1e-5 instead the passed lwd. On the surface, this looks mighty small. Could this be the problem? I'm hoping there's a simple solution. Is there some way to get them without writing my own panel function? DAV [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Indexing multi-dimensional table
I want to take slices of a multi-dimensional table (or array) without knowing the number of dimensions in advance. As a test I tried using (in this example a 3d table): do.call(`[`, list(tbl, x,NULL,NULL)] where I built the list on the fly. It works great as long as I only want the first dimension however when I try a different dimension, say with list(tbl,NULL,x,NULL), I get 0 x 0 matrix as the result. I thought this was because I wasn't calling the right function but there is no `[.table` or `[.matrix` or even `[.array`. Am I going about this the wrong way? DAV [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Indexing multi-dimensional table
(This does imply you knew the number of dimensions was 3.) Yes, at run time. It looks as though the Nulls became 0's. So if you wanted to use do.call(`[` then this succeeds: do.call(`[`, list(tbl, x, 1:dim(tbl)[2], 1:dim(tbl)[3]) ) ... As does this using the empty comma approach: eval(parse(text= paste(tbl[ ,x, , , ])) ) I tried the eval course but that struck me as slower. Perhaps not? It's not that I'm set on using '['. I was under the impression that was how the eval expression is eventually parsed. I also thought the empty commas were eventually passed as NULLs. Wasn't aware of the 'str' function. Could come in handy down the road. Thanks. DAV __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Indexing multi-dimensional table
From: William Dunlap [mailto:wdun...@tibco.com] You can build the 2nd argument to do.call with alist() instead. alist() produces a list that you can use c() and subscripting on to add or modify arguments. It is usually better to encapsulate this sort of thing in a function like extract() that has a convenient interface. Thanks. alist seems to produce empty list entries for missing arguments. It wasn't clear to me how to generate one at run time. For example, how does one append an empty list element? c(x,) produces argument 2 is empty. However I've discovered c(alist(x,y),alist(),alist(z)) produces an empty entry between y and z. This will likely fit better in the code I currently have. Continuing the annoying tradition of partial quotes A matter of taste, I guess. Speaking strictly for myself, I am annoyed by needless repetition (particularly of mostly irrelevant matter such as output, signature tags, subject name, to/from lines and quotes of quotes of quotes) not to mention the tedium of such in a long reply stream making it hard sometimes to locate the relevant replies. Omitting the things I've listed perforce means a partial quote. DAV __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Unintended loading of package:datasets
The dataset package is being loaded apparently by one of the packages that I am using. The loading of the datasets takes a long time and I would like to eliminate it. I thought the datasets were effectively examples so don't understand why they would be required at all. 1) How can I determine what is causing the datasets to be loaded? 2) How can I stop them from doing so? I am using the following: Rpart, grDevices, graphics, stats, utils, methods, base There is also an environment named 'Autoloads' TIA __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Unintended loading of package:datasets
Thanks. Perhaps something else is going on. There is a large time period
(about 20 sec.) after the message about loading the package. More
investigation, I suppose.
Thanks again,
DAV
-Original Message-
From: Rolf Turner [mailto:r.tur...@auckland.ac.nz]
Sent: Sunday, May 10, 2009 7:34 PM
To: David A Vavra
Cc: r-help@r-project.org
Subject: Re: [R] Unintended loading of package:datasets
On 11/05/2009, at 9:17 AM, David A Vavra wrote:
The dataset package is being loaded apparently by one of the
packages that I
am using. The loading of the datasets takes a long time and I would
like to
eliminate it. I thought the datasets were effectively examples so
don't
understand why they would be required at all.
1) How can I determine what is causing the datasets to be loaded?
2) How can I stop them from doing so?
I am using the following:
Rpart, grDevices, graphics, stats, utils, methods, base
There is also an environment named 'Autoloads'
TIA
The datasets (note the ``s'') is a required R package which is
*always* loaded
automatically --- and in my experience instantaneously.
I don't know about a dataset (singular) package. There does not
appear to be
one on CRAN.
There is some confusion in what you are doing.
cheers,
Rolf Turner
##
Attention:\ This e-mail message is privileged and confid...{{dropped:13}}
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
