[R] Testing.

[Rolf Turner]

I have just changed my email address with the R-help mailing lists, and
I want to check that things are working.  Sorry for the noise.

cheers,

Rolf Turner

-- 
Honorary Research Fellow
Department of Statistics
University of Auckland
Stats. Dep't. (secretaries) phone:
 +64-9-373-7599 ext. 89622
Home phone: +64-9-480-4619

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing for R CMD INSTALL

[Duncan Murdoch]

On 24/07/2021 11:22 a.m., Andrew Simmons wrote:

Hello,


I was wondering if anyone has a way to test if a package is currently being
installed. My solution was to check if environment variable "R_INSTALL_PKG"
was unset, something like:

"R CMD INSTALL-ing" <- function ()
!is.na(Sys.getenv("R_INSTALL_PKG", NA))

Unfortunately, I couldn't find what I was looking for with ?"environment
variables". So if anyone has any better methods, I'd be happy to hear them,
thank you!


Normally if you want to execute special code during installation, you'd 
add a Makevars or Makefile to your package and do it there, but 
R_INSTALL_PKG should be defined during a source install.


I believe that no code from the package is executed during a binary 
install:  it just copies files into the appropriate places.


Duncan

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing for R CMD INSTALL

[Bert Gunter]

Does ?installed.packages help?

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Sat, Jul 24, 2021 at 8:30 AM Andrew Simmons  wrote:

> Hello,
>
>
> I was wondering if anyone has a way to test if a package is currently being
> installed. My solution was to check if environment variable "R_INSTALL_PKG"
> was unset, something like:
>
> "R CMD INSTALL-ing" <- function ()
> !is.na(Sys.getenv("R_INSTALL_PKG", NA))
>
> Unfortunately, I couldn't find what I was looking for with ?"environment
> variables". So if anyone has any better methods, I'd be happy to hear them,
> thank you!
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing for R CMD INSTALL

[Andrew Simmons]

Hello,


I was wondering if anyone has a way to test if a package is currently being
installed. My solution was to check if environment variable "R_INSTALL_PKG"
was unset, something like:

"R CMD INSTALL-ing" <- function ()
!is.na(Sys.getenv("R_INSTALL_PKG", NA))

Unfortunately, I couldn't find what I was looking for with ?"environment
variables". So if anyone has any better methods, I'd be happy to hear them,
thank you!

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Gabor Grothendieck]

In case it is of interest this problem can be solved with an
unconstrained optimizer,
here optim, like this:

proj <- function(x) x / sqrt(sum(x * x))
opt <- optim(c(0, 0, 1), function(x) f(proj(x)))
proj(opt$par)
## [1] 5.388907e-09 7.071068e-01 7.071068e-01

On Fri, May 21, 2021 at 11:01 AM Hans W  wrote:
>
> Just by chance I came across the following example of minimizing
> a simple function
>
> (x,y,z) --> 2 (x^2 - y z)
>
> on the unit sphere, the only constraint present.
> I tried it with two starting points, x1 = (1,0,0) and x2 = (0,0,1).
>
> #-- Problem definition in R
> f = function(x)  2 * (x[1]^2 - x[2]*x[3])   # (x,y,z) |-> 2(x^2 -yz)
> g = function(x)  c(4*x[1], 2*x[3], 2*x[2])  # its gradient
>
> x0 = c(1, 0, 0); x1 = c(0, 0, 1)# starting points
> xmin = c(0, 1/sqrt(2), 1/sqrt(2))   # true minimum -1
>
> heq = function(x)  1-x[1]^2-x[2]^2-x[3]^2   # staying on the sphere
> conf = function(x) {# constraint function
> fun = x[1]^2 + x[2]^2 + x[3]^2 - 1
> return(list(ceq = fun, c = NULL))
> }
>
> I tried all the nonlinear optimization solvers in R packages that
> allow for equality constraints: 'auglag()' in alabama, 'solnl()' in
> NlcOptim, 'auglag()' in nloptr, 'solnp()' in Rsolnp, or even 'donlp2()'
> from the Rdonlp2 package (on R-Forge).
>
> None of them worked from both starting points:
>
> # alabama
> alabama::auglag(x0, fn = f, gr = g, heq = heq)  # right (inaccurate)
> alabama::auglag(x1, fn = f, gr = g, heq = heq)  # wrong
>
> # NlcOptim
> NlcOptim::solnl(x0, objfun = f, confun = conf)  # wrong
> NlcOptim::solnl(x1, objfun = f, confun = conf)  # right
>
> # nloptr
> nloptr::auglag(x0, fn = f, heq = heq)   # wrong
> # nloptr::auglag(x1, fn = f, heq = heq) # not returning
>
> # Rsolnp
> Rsolnp::solnp(x0, fun = f, eqfun = heq) # wrong
> Rsolnp::solnp(x1, fun = f, eqfun = heq) # wrong
>
> # Rdonlp2
> Rdonlp2::donlp2(x0, fn = f, nlin = list(heq),   # wrong
>nlin.lower = 0, nlin.upper = 0)
> Rdonlp2::donlp2(x1, fn = f, nlin = list(heq),   # right
>nlin.lower = 0, nlin.upper = 0)  # (fast and exact)
>
> The problem with starting point x0 appears to be that the gradient at
> that point, projected onto the unit sphere, is zero. Only alabama is
> able to handle this somehow.
>
> I do not know what problem most solvers have with starting point x1.
> The fact that Rdonlp2 is the fastest and most accurate is no surprise.
>
> If anyone with more experience with one or more of these packages can
> give a hint of what I made wrong, or how to change calling the solver
> to make it run correctly, please let me know.
>
> Thanks  -- HW
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



-- 
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Abby Spurdle]

I meant:
x0 = c (1, 1e-3, 0)

Not:
x0 = c (1, 1e6, 0)

So, large intentional error may work too.
Possibly, better...?

On Thu, May 27, 2021 at 6:00 PM Abby Spurdle  wrote:
>
> If I can re-answer the original post:
> There's a relatively simple solution.
> (For these problems, at least).
>
> #wrong
> x0 = c (1, 0, 0)
> NlcOptim::solnl(x0, objfun = f, confun = conf)$par
> Rdonlp2::donlp2(x0, fn = f, nlin = list(heq), nlin.lower = 0,
> nlin.upper = 0)$par
>
> #right
> x0 = c (1, 1e6, 0)
> NlcOptim::solnl(x0, objfun = f, confun = conf)$par
> Rdonlp2::donlp2(x0, fn = f, nlin = list(heq), nlin.lower = 0,
> nlin.upper = 0)$par
>
> So, problems with the starting point, appear to be very *specific*.
> Hence, a small amount of intentional error resolves the problem.
>
> Presumably, there are more efficient solutions, that the package
> maintainers may (or may not) want to address.
>
>
> On Thu, May 27, 2021 at 3:27 PM Abby Spurdle  wrote:
> >
> > I need to retract my previous post.
> > (Except the part that the R has extremely good numerical capabilities).
> >
> > I ran some of the examples, and Hans W was correct.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Abby Spurdle]

If I can re-answer the original post:
There's a relatively simple solution.
(For these problems, at least).

#wrong
x0 = c (1, 0, 0)
NlcOptim::solnl(x0, objfun = f, confun = conf)$par
Rdonlp2::donlp2(x0, fn = f, nlin = list(heq), nlin.lower = 0,
nlin.upper = 0)$par

#right
x0 = c (1, 1e6, 0)
NlcOptim::solnl(x0, objfun = f, confun = conf)$par
Rdonlp2::donlp2(x0, fn = f, nlin = list(heq), nlin.lower = 0,
nlin.upper = 0)$par

So, problems with the starting point, appear to be very *specific*.
Hence, a small amount of intentional error resolves the problem.

Presumably, there are more efficient solutions, that the package
maintainers may (or may not) want to address.


On Thu, May 27, 2021 at 3:27 PM Abby Spurdle  wrote:
>
> I need to retract my previous post.
> (Except the part that the R has extremely good numerical capabilities).
>
> I ran some of the examples, and Hans W was correct.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Abby Spurdle]

I need to retract my previous post.
(Except the part that the R has extremely good numerical capabilities).

I ran some of the examples, and Hans W was correct.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[J C Nash]

As someone who works on trying to improve the optimization codes in R,
though mainly in the unconstrained and bounds-constrained area, I think my
experience is more akin to that of HWB. That is, for some problems -- and
the example in question does have a reparametrization that removes the
constraint, so it could be considered artificial -- there seem to be
many more "bad" starting points than "good" ones. Abby's experience may
be very different, which is more or less the way things are in this type
of work. Or things will work for several years with a class of problems,
then the alligator of an unanticipated "bad" result will take a chunk out
of your posterior.

R users often have advanced expertise in different areas, so I suspect
a lot of cases involve using a poor parameterization. To my mind, the
real issue is less "the program should just work" than "the program
should let the user know there may be issues with the proposed solution".
Such diagnostics are very tricky to devise, and even more awkward to
implement in a way that doesn't become intrusive and time-wasting.
For example, in the optimx package, we can compute the KKT conditions
(first and second derivative checks on proposed solutions), but that
can often take many times the computing effort of finding the solution.
And when there are constraints, KKT conditions are less useful, since
they are for optima "not on the constraint".

This is going to be an ongoing challenge, and it affects a lot of areas
of work now that very complicated calculations are being carried out in
the name of AI and ML (either machine learning or maximum likelihood --
your choice). I'll welcome off-list ideas and efforts to devise better
diagnostics for proposed optimization solutions. We'll need them.

Best, John Nash


On 2021-05-24 10:47 p.m., Abby Spurdle wrote:
> I received an off-list email, questioning the relevance of my post.
> So, I thought I should clarify.
> 
> If an optimization algorithm is dependent on the starting point (or
> other user-selected parameters), and then fails to find the "correct"
> solution because the starting point (or other user-selected
> parameters) are unsuitable, then that, in itself, does not indicate a
> problem with the algorithm.
> 
> In other words, the R's packages listed in this thread appear to be
> working fine.
> (Or at least, there's no clear counter-evidence against).
> 
> One solution is to project the surface (here, equality constraints) on
> to lower dimensions, as already suggested.
> Another much simpler solution, is to use two algorithms, where one
> selects one or more starting points.
> (These could be the solution to an initial optimization, or chosen at
> random, or a combination of both).
> 
> Both of these approaches generalize to a broader set of problems.
> And I assume that there are other (possibly much better) approaches.
> However, that's an off-list discussion...
> 
> All and all, I would say R has extremely good numerical capabilities.
> Which are even more useful still, with the use of well chosen
> mathematical and statistical graphics.
> 
> 
> On Sun, May 23, 2021 at 5:25 PM Abby Spurdle  wrote:
>>
>> For a start, there's two local minima.
>>
>> Add to that floating point errors.
>> And possible assumptions by the package authors.
>>
>> begin code
>> f <- function (x, y, sign)
>> {   unsign.z <- sqrt (1 - x^2 - y^2)
>> 2 * (x^2 - sign * y * unsign.z)
>> }
>>
>> north.f <- function (x, y) f (x, y, +1)
>> south.f <- function (x, y) f (x, y, -1)
>>
>> N <- 100
>> p0 <- par (mfrow = c (1, 2) )
>> plotf_cfield (north.f, c (-1.1, 1.1),
>> main="north",
>> ncontours=10, n=N, raster=TRUE, hcv=TRUE)
>> plotf_cfield (south.f, c (-1.1, 1.1),
>> main="south",
>> ncontours=10, n=N, raster=TRUE, hcv=TRUE)
>> par (p0)
>> end code 
>>
>> Please ignore R warnings.
>> I'm planning to reinvent this package soon.
>> And also, it wasn't designed for circular heatmaps.
>>
>>
>> On Sat, May 22, 2021 at 8:02 PM Hans W  wrote:
>>>
>>> Yes. "*on* the unit sphere" means on the surface, as you can guess
>>> from the equality constraint. And 'auglag()' does find the minimum, so
>>> no need for a special approach.
>>>
>>> I was/am interested in why all these other good solvers get stuck,
>>> i.e., do not move away from the starting point. And how to avoid this
>>> in general, not only for this specific example.
>>>
>>>
>>> On Sat, 22 May 2021 at 09:44, Abby Spurdle  wrote:

 Sorry, this might sound like a poor question:
 But by "on the unit sphere", do you mean on the ***surface*** of the 
 sphere?
 In which case, can't the surface of a sphere be projected onto a pair
 of circles?
 Where the cost function is reformulated as a function of two (rather
 than three) variables.

> 
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read 

Re: [R] Testing optimization solvers with equality constraints

[Abby Spurdle]

I received an off-list email, questioning the relevance of my post.
So, I thought I should clarify.

If an optimization algorithm is dependent on the starting point (or
other user-selected parameters), and then fails to find the "correct"
solution because the starting point (or other user-selected
parameters) are unsuitable, then that, in itself, does not indicate a
problem with the algorithm.

In other words, the R's packages listed in this thread appear to be
working fine.
(Or at least, there's no clear counter-evidence against).

One solution is to project the surface (here, equality constraints) on
to lower dimensions, as already suggested.
Another much simpler solution, is to use two algorithms, where one
selects one or more starting points.
(These could be the solution to an initial optimization, or chosen at
random, or a combination of both).

Both of these approaches generalize to a broader set of problems.
And I assume that there are other (possibly much better) approaches.
However, that's an off-list discussion...

All and all, I would say R has extremely good numerical capabilities.
Which are even more useful still, with the use of well chosen
mathematical and statistical graphics.


On Sun, May 23, 2021 at 5:25 PM Abby Spurdle  wrote:
>
> For a start, there's two local minima.
>
> Add to that floating point errors.
> And possible assumptions by the package authors.
>
> begin code
> f <- function (x, y, sign)
> {   unsign.z <- sqrt (1 - x^2 - y^2)
> 2 * (x^2 - sign * y * unsign.z)
> }
>
> north.f <- function (x, y) f (x, y, +1)
> south.f <- function (x, y) f (x, y, -1)
>
> N <- 100
> p0 <- par (mfrow = c (1, 2) )
> plotf_cfield (north.f, c (-1.1, 1.1),
> main="north",
> ncontours=10, n=N, raster=TRUE, hcv=TRUE)
> plotf_cfield (south.f, c (-1.1, 1.1),
> main="south",
> ncontours=10, n=N, raster=TRUE, hcv=TRUE)
> par (p0)
> end code 
>
> Please ignore R warnings.
> I'm planning to reinvent this package soon.
> And also, it wasn't designed for circular heatmaps.
>
>
> On Sat, May 22, 2021 at 8:02 PM Hans W  wrote:
> >
> > Yes. "*on* the unit sphere" means on the surface, as you can guess
> > from the equality constraint. And 'auglag()' does find the minimum, so
> > no need for a special approach.
> >
> > I was/am interested in why all these other good solvers get stuck,
> > i.e., do not move away from the starting point. And how to avoid this
> > in general, not only for this specific example.
> >
> >
> > On Sat, 22 May 2021 at 09:44, Abby Spurdle  wrote:
> > >
> > > Sorry, this might sound like a poor question:
> > > But by "on the unit sphere", do you mean on the ***surface*** of the 
> > > sphere?
> > > In which case, can't the surface of a sphere be projected onto a pair
> > > of circles?
> > > Where the cost function is reformulated as a function of two (rather
> > > than three) variables.
> > >

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Abby Spurdle]

Sorry, missed the top line of code.
library (barsurf)

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Abby Spurdle]

For a start, there's two local minima.

Add to that floating point errors.
And possible assumptions by the package authors.

begin code
f <- function (x, y, sign)
{   unsign.z <- sqrt (1 - x^2 - y^2)
2 * (x^2 - sign * y * unsign.z)
}

north.f <- function (x, y) f (x, y, +1)
south.f <- function (x, y) f (x, y, -1)

N <- 100
p0 <- par (mfrow = c (1, 2) )
plotf_cfield (north.f, c (-1.1, 1.1),
main="north",
ncontours=10, n=N, raster=TRUE, hcv=TRUE)
plotf_cfield (south.f, c (-1.1, 1.1),
main="south",
ncontours=10, n=N, raster=TRUE, hcv=TRUE)
par (p0)
end code 

Please ignore R warnings.
I'm planning to reinvent this package soon.
And also, it wasn't designed for circular heatmaps.


On Sat, May 22, 2021 at 8:02 PM Hans W  wrote:
>
> Yes. "*on* the unit sphere" means on the surface, as you can guess
> from the equality constraint. And 'auglag()' does find the minimum, so
> no need for a special approach.
>
> I was/am interested in why all these other good solvers get stuck,
> i.e., do not move away from the starting point. And how to avoid this
> in general, not only for this specific example.
>
>
> On Sat, 22 May 2021 at 09:44, Abby Spurdle  wrote:
> >
> > Sorry, this might sound like a poor question:
> > But by "on the unit sphere", do you mean on the ***surface*** of the sphere?
> > In which case, can't the surface of a sphere be projected onto a pair
> > of circles?
> > Where the cost function is reformulated as a function of two (rather
> > than three) variables.
> >
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Hans W]

Yes. "*on* the unit sphere" means on the surface, as you can guess
from the equality constraint. And 'auglag()' does find the minimum, so
no need for a special approach.

I was/am interested in why all these other good solvers get stuck,
i.e., do not move away from the starting point. And how to avoid this
in general, not only for this specific example.


On Sat, 22 May 2021 at 09:44, Abby Spurdle  wrote:
>
> Sorry, this might sound like a poor question:
> But by "on the unit sphere", do you mean on the ***surface*** of the sphere?
> In which case, can't the surface of a sphere be projected onto a pair
> of circles?
> Where the cost function is reformulated as a function of two (rather
> than three) variables.
>

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Abby Spurdle]

Sorry, this might sound like a poor question:
But by "on the unit sphere", do you mean on the ***surface*** of the sphere?

In which case, can't the surface of a sphere be projected onto a pair
of circles?
Where the cost function is reformulated as a function of two (rather
than three) variables.


On Sat, May 22, 2021 at 3:01 AM Hans W  wrote:
>
> Just by chance I came across the following example of minimizing
> a simple function
>
> (x,y,z) --> 2 (x^2 - y z)
>
> on the unit sphere, the only constraint present.
> I tried it with two starting points, x1 = (1,0,0) and x2 = (0,0,1).
>
> #-- Problem definition in R
> f = function(x)  2 * (x[1]^2 - x[2]*x[3])   # (x,y,z) |-> 2(x^2 -yz)
> g = function(x)  c(4*x[1], 2*x[3], 2*x[2])  # its gradient
>
> x0 = c(1, 0, 0); x1 = c(0, 0, 1)# starting points
> xmin = c(0, 1/sqrt(2), 1/sqrt(2))   # true minimum -1
>
> heq = function(x)  1-x[1]^2-x[2]^2-x[3]^2   # staying on the sphere
> conf = function(x) {# constraint function
> fun = x[1]^2 + x[2]^2 + x[3]^2 - 1
> return(list(ceq = fun, c = NULL))
> }
>
> I tried all the nonlinear optimization solvers in R packages that
> allow for equality constraints: 'auglag()' in alabama, 'solnl()' in
> NlcOptim, 'auglag()' in nloptr, 'solnp()' in Rsolnp, or even 'donlp2()'
> from the Rdonlp2 package (on R-Forge).
>
> None of them worked from both starting points:
>
> # alabama
> alabama::auglag(x0, fn = f, gr = g, heq = heq)  # right (inaccurate)
> alabama::auglag(x1, fn = f, gr = g, heq = heq)  # wrong
>
> # NlcOptim
> NlcOptim::solnl(x0, objfun = f, confun = conf)  # wrong
> NlcOptim::solnl(x1, objfun = f, confun = conf)  # right
>
> # nloptr
> nloptr::auglag(x0, fn = f, heq = heq)   # wrong
> # nloptr::auglag(x1, fn = f, heq = heq) # not returning
>
> # Rsolnp
> Rsolnp::solnp(x0, fun = f, eqfun = heq) # wrong
> Rsolnp::solnp(x1, fun = f, eqfun = heq) # wrong
>
> # Rdonlp2
> Rdonlp2::donlp2(x0, fn = f, nlin = list(heq),   # wrong
>nlin.lower = 0, nlin.upper = 0)
> Rdonlp2::donlp2(x1, fn = f, nlin = list(heq),   # right
>nlin.lower = 0, nlin.upper = 0)  # (fast and exact)
>
> The problem with starting point x0 appears to be that the gradient at
> that point, projected onto the unit sphere, is zero. Only alabama is
> able to handle this somehow.
>
> I do not know what problem most solvers have with starting point x1.
> The fact that Rdonlp2 is the fastest and most accurate is no surprise.
>
> If anyone with more experience with one or more of these packages can
> give a hint of what I made wrong, or how to change calling the solver
> to make it run correctly, please let me know.
>
> Thanks  -- HW
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[J C Nash]

I might (and that could be a stretch) be expert in unconstrained problems,
but I've nowhere near HWB's experience in constrained ones.

My main reason for wanting gradients is to know when I'm at a solution.
In practice for getting to the solution, I've often found secant methods
work faster, though that is not universal nor even "mostly", but more
frequently than my intuition suggests.

Best, JN

On 2021-05-21 3:31 p.m., Mark Leeds wrote:
> Hi Hans: I can't help as far as the projection of the gradient onto the
> constraint but it may give insight just to see what the value of
> the gradient itself is when the optimization stops.
> 
> John Nash ( definitely one of THE expeRts when it comes to optimization in
> R )
> often strongly recommends  to supply gradients so I'm   not sure what those
> functions are
> doing that don't allow it as an argument. I guess some numerical
> approximation.
> 
> Hopefully John or Ravi will chime in with their expertise when they see
> this posting.
> 
> 
> Mark
> 
> P.S: You may want to try Rvminb. John wrote that one and it allows for
> constraints ( I remember it
> working nicely  for me when I had problems with some other ones but I don't
> remember which ones ) but
> I'm not certain  whether it can handle equalities.
> 
> 
> 
> 
> 
> 
> On Fri, May 21, 2021 at 2:06 PM Hans W  wrote:
> 
>> Mark, you're right, and it's a bit embarrassing as I thought I had
>> looked at it closely enough.
>>
>> This solves the problem for 'alabama::auglag()' in both cases, but NOT for
>>
>>   * NlcOptim::solnl -- with x0
>>   * nloptr::auglag  -- both x0, x1
>>   * Rsolnp::solnp   -- with x0
>>   * Rdonlp::donlp2  -- with x0
>>
>> as for these solver calls the gradient function g was *not* used.
>>
>> Actually, 'solnl()' and 'solnp()' do not allow a gradient argument,
>> 'nloptr::auglag()' says it does not use a supplied gradient, and
>> 'donlp2' again does not provide it.
>> Gradients, if needed, are computed internally which in most cases is
>> sufficient, anyway.
>>
>> So the question remains:
>> Is the fact that the projection of the gradient onto the constraint is
>> zero, is this the reason for the solvers not finding the minimum?
>>
>> And how to avoid this? Except, maybe, checking the gradient for all
>> the given constraints
>>
>> Thanks  --HW
>>
>>
>>
>> On Fri, 21 May 2021 at 17:58, Mark Leeds  wrote:
>>>
>>> Hi Hans: I think  that you are missing minus signs in the 2nd and 3rd
>> elements of your gradient.
>>> Also, I don't know how all of the optimixation functions work as far as
>> their arguments but it's best to supply
>>> the gradient when possible. I hope it helps.
>>>
>>
> 
>   [[alternative HTML version deleted]]
> 
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Mark Leeds]

Hi Hans: I can't help as far as the projection of the gradient onto the
constraint but it may give insight just to see what the value of
the gradient itself is when the optimization stops.

John Nash ( definitely one of THE expeRts when it comes to optimization in
R )
often strongly recommends  to supply gradients so I'm   not sure what those
functions are
doing that don't allow it as an argument. I guess some numerical
approximation.

Hopefully John or Ravi will chime in with their expertise when they see
this posting.


Mark

P.S: You may want to try Rvminb. John wrote that one and it allows for
constraints ( I remember it
working nicely  for me when I had problems with some other ones but I don't
remember which ones ) but
I'm not certain  whether it can handle equalities.






On Fri, May 21, 2021 at 2:06 PM Hans W  wrote:

> Mark, you're right, and it's a bit embarrassing as I thought I had
> looked at it closely enough.
>
> This solves the problem for 'alabama::auglag()' in both cases, but NOT for
>
>   * NlcOptim::solnl -- with x0
>   * nloptr::auglag  -- both x0, x1
>   * Rsolnp::solnp   -- with x0
>   * Rdonlp::donlp2  -- with x0
>
> as for these solver calls the gradient function g was *not* used.
>
> Actually, 'solnl()' and 'solnp()' do not allow a gradient argument,
> 'nloptr::auglag()' says it does not use a supplied gradient, and
> 'donlp2' again does not provide it.
> Gradients, if needed, are computed internally which in most cases is
> sufficient, anyway.
>
> So the question remains:
> Is the fact that the projection of the gradient onto the constraint is
> zero, is this the reason for the solvers not finding the minimum?
>
> And how to avoid this? Except, maybe, checking the gradient for all
> the given constraints
>
> Thanks  --HW
>
>
>
> On Fri, 21 May 2021 at 17:58, Mark Leeds  wrote:
> >
> > Hi Hans: I think  that you are missing minus signs in the 2nd and 3rd
> elements of your gradient.
> > Also, I don't know how all of the optimixation functions work as far as
> their arguments but it's best to supply
> > the gradient when possible. I hope it helps.
> >
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Hans W]

Mark, you're right, and it's a bit embarrassing as I thought I had
looked at it closely enough.

This solves the problem for 'alabama::auglag()' in both cases, but NOT for

  * NlcOptim::solnl -- with x0
  * nloptr::auglag  -- both x0, x1
  * Rsolnp::solnp   -- with x0
  * Rdonlp::donlp2  -- with x0

as for these solver calls the gradient function g was *not* used.

Actually, 'solnl()' and 'solnp()' do not allow a gradient argument,
'nloptr::auglag()' says it does not use a supplied gradient, and
'donlp2' again does not provide it.
Gradients, if needed, are computed internally which in most cases is
sufficient, anyway.

So the question remains:
Is the fact that the projection of the gradient onto the constraint is
zero, is this the reason for the solvers not finding the minimum?

And how to avoid this? Except, maybe, checking the gradient for all
the given constraints

Thanks  --HW



On Fri, 21 May 2021 at 17:58, Mark Leeds  wrote:
>
> Hi Hans: I think  that you are missing minus signs in the 2nd and 3rd 
> elements of your gradient.
> Also, I don't know how all of the optimixation functions work as far as their 
> arguments but it's best to supply
> the gradient when possible. I hope it helps.
>

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing optimization solvers with equality constraints

[Mark Leeds]

Hi Hans: I think  that you are missing minus signs in the 2nd and 3rd
elements of your gradient.
Also, I don't know how all of the optimixation functions work as far as
their arguments but it's best to supply
the gradient when possible. I hope it helps.






On Fri, May 21, 2021 at 11:01 AM Hans W  wrote:

> Just by chance I came across the following example of minimizing
> a simple function
>
> (x,y,z) --> 2 (x^2 - y z)
>
> on the unit sphere, the only constraint present.
> I tried it with two starting points, x1 = (1,0,0) and x2 = (0,0,1).
>
> #-- Problem definition in R
> f = function(x)  2 * (x[1]^2 - x[2]*x[3])   # (x,y,z) |-> 2(x^2 -yz)
> g = function(x)  c(4*x[1], 2*x[3], 2*x[2])  # its gradient
>
> x0 = c(1, 0, 0); x1 = c(0, 0, 1)# starting points
> xmin = c(0, 1/sqrt(2), 1/sqrt(2))   # true minimum -1
>
> heq = function(x)  1-x[1]^2-x[2]^2-x[3]^2   # staying on the sphere
> conf = function(x) {# constraint function
> fun = x[1]^2 + x[2]^2 + x[3]^2 - 1
> return(list(ceq = fun, c = NULL))
> }
>
> I tried all the nonlinear optimization solvers in R packages that
> allow for equality constraints: 'auglag()' in alabama, 'solnl()' in
> NlcOptim, 'auglag()' in nloptr, 'solnp()' in Rsolnp, or even 'donlp2()'
> from the Rdonlp2 package (on R-Forge).
>
> None of them worked from both starting points:
>
> # alabama
> alabama::auglag(x0, fn = f, gr = g, heq = heq)  # right (inaccurate)
> alabama::auglag(x1, fn = f, gr = g, heq = heq)  # wrong
>
> # NlcOptim
> NlcOptim::solnl(x0, objfun = f, confun = conf)  # wrong
> NlcOptim::solnl(x1, objfun = f, confun = conf)  # right
>
> # nloptr
> nloptr::auglag(x0, fn = f, heq = heq)   # wrong
> # nloptr::auglag(x1, fn = f, heq = heq) # not returning
>
> # Rsolnp
> Rsolnp::solnp(x0, fun = f, eqfun = heq) # wrong
> Rsolnp::solnp(x1, fun = f, eqfun = heq) # wrong
>
> # Rdonlp2
> Rdonlp2::donlp2(x0, fn = f, nlin = list(heq),   # wrong
>nlin.lower = 0, nlin.upper = 0)
> Rdonlp2::donlp2(x1, fn = f, nlin = list(heq),   # right
>nlin.lower = 0, nlin.upper = 0)  # (fast and exact)
>
> The problem with starting point x0 appears to be that the gradient at
> that point, projected onto the unit sphere, is zero. Only alabama is
> able to handle this somehow.
>
> I do not know what problem most solvers have with starting point x1.
> The fact that Rdonlp2 is the fastest and most accurate is no surprise.
>
> If anyone with more experience with one or more of these packages can
> give a hint of what I made wrong, or how to change calling the solver
> to make it run correctly, please let me know.
>
> Thanks  -- HW
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing optimization solvers with equality constraints

[Hans W]

Just by chance I came across the following example of minimizing
a simple function

(x,y,z) --> 2 (x^2 - y z)

on the unit sphere, the only constraint present.
I tried it with two starting points, x1 = (1,0,0) and x2 = (0,0,1).

#-- Problem definition in R
f = function(x)  2 * (x[1]^2 - x[2]*x[3])   # (x,y,z) |-> 2(x^2 -yz)
g = function(x)  c(4*x[1], 2*x[3], 2*x[2])  # its gradient

x0 = c(1, 0, 0); x1 = c(0, 0, 1)# starting points
xmin = c(0, 1/sqrt(2), 1/sqrt(2))   # true minimum -1

heq = function(x)  1-x[1]^2-x[2]^2-x[3]^2   # staying on the sphere
conf = function(x) {# constraint function
fun = x[1]^2 + x[2]^2 + x[3]^2 - 1
return(list(ceq = fun, c = NULL))
}

I tried all the nonlinear optimization solvers in R packages that
allow for equality constraints: 'auglag()' in alabama, 'solnl()' in
NlcOptim, 'auglag()' in nloptr, 'solnp()' in Rsolnp, or even 'donlp2()'
from the Rdonlp2 package (on R-Forge).

None of them worked from both starting points:

# alabama
alabama::auglag(x0, fn = f, gr = g, heq = heq)  # right (inaccurate)
alabama::auglag(x1, fn = f, gr = g, heq = heq)  # wrong

# NlcOptim
NlcOptim::solnl(x0, objfun = f, confun = conf)  # wrong
NlcOptim::solnl(x1, objfun = f, confun = conf)  # right

# nloptr
nloptr::auglag(x0, fn = f, heq = heq)   # wrong
# nloptr::auglag(x1, fn = f, heq = heq) # not returning

# Rsolnp
Rsolnp::solnp(x0, fun = f, eqfun = heq) # wrong
Rsolnp::solnp(x1, fun = f, eqfun = heq) # wrong

# Rdonlp2
Rdonlp2::donlp2(x0, fn = f, nlin = list(heq),   # wrong
   nlin.lower = 0, nlin.upper = 0)
Rdonlp2::donlp2(x1, fn = f, nlin = list(heq),   # right
   nlin.lower = 0, nlin.upper = 0)  # (fast and exact)

The problem with starting point x0 appears to be that the gradient at
that point, projected onto the unit sphere, is zero. Only alabama is
able to handle this somehow.

I do not know what problem most solvers have with starting point x1.
The fact that Rdonlp2 is the fastest and most accurate is no surprise.

If anyone with more experience with one or more of these packages can
give a hint of what I made wrong, or how to change calling the solver
to make it run correctly, please let me know.

Thanks  -- HW

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing.

[Rolf Turner]

My apologies for the noise.  Please ignore this message.
I am just trying to test out message filters in a new mail client that
I am learning to use.

Again, sorry for the noise.

cheers,

Rolf Turner

-- 
Honorary Research Fellow
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing wether my dataset follows a poisson distribution with R

[Bert Gunter]

That is logically impossible.

You can only show that there is insufficient evidence (according to
whatever evidentiary criterion you have chosen) to show that the data were
*not* a (iid or other) sample from a Poisson. This may seem esoteric, but
it is not. (The simplest incantation is that you can reject, but not accept
a hypothesis in the classical N-P frequentist framework ). Given sufficient
data, you will almost always have enough evidence to be formally
inconsistent with any prechosen model/distribution. This does not mean that
the model is not useful, however.

Further discussion would meander even farther offtopic. These are
statistical (and philosophy of science) issues that don't properly belong
here. Mea culpa.

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Jul 21, 2020 at 7:27 PM Paul Bernal  wrote:

> Dear friends,
>
> I have a sample dataset, which is basically the number of transits through
> a particular waterway, and is on a daily basis.
>
> MyDat <- dataset$DailyTransits
>
> What I´d like to do is to test whether MyDat follows a poisson distribution
> or not. What R function could accomplish this?
>
> Any help and/or guidance will be greatly appreciated,
>
> Best regards,
>
> Paul
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing wether my dataset follows a poisson distribution with R

[David Winsemius]

Your first check might be to see in the mean and sd are "reasonably" 
close. Next approach would be to see if the `qqplot` of that vector has 
an arguably straight-line relationship with a random draw from a Poisson 
random generator function with the same mean.


?rpois

?qqplot

And do remember that Rhelp is a plain-text mailing list.

--

David

On 7/21/20 7:26 PM, Paul Bernal wrote:

Dear friends,

I have a sample dataset, which is basically the number of transits through
a particular waterway, and is on a daily basis.

MyDat <- dataset$DailyTransits

What I´d like to do is to test whether MyDat follows a poisson distribution
or not. What R function could accomplish this?

Any help and/or guidance will be greatly appreciated,

Best regards,

Paul

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing wether my dataset follows a poisson distribution with R

[Paul Bernal]

Dear friends,

I have a sample dataset, which is basically the number of transits through
a particular waterway, and is on a daily basis.

MyDat <- dataset$DailyTransits

What I´d like to do is to test whether MyDat follows a poisson distribution
or not. What R function could accomplish this?

Any help and/or guidance will be greatly appreciated,

Best regards,

Paul

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing my package : unstated dependency to self in package tests

[Servet Ahmet Çizmeli]

That worked. Thanks.


From: Michael Dewey 
Sent: Sunday, February 16, 2020 5:54 PM
To: Servet Ahmet Çizmeli ; 
r-help@r-project.org 
Subject: Re: [R] testing my package : unstated dependency to self in package 
tests

When something similar happened to me I found it went away when I added
Suggests: 
to the DESCRIPTION file. Whether this will work for you I have no idea.

Michael

On 16/02/2020 11:03, Servet Ahmet Çizmeli wrote:
> I am updating my CRAN package geoSpectral. I get the following Warning during 
> R CMD check :
>
> ...
> * checking for unstated dependencies in �tests� ... WARNING
> 'library' or 'require' call not declared from: �geoSpectral�
> 
>
>
> All the .R files I have under the testhat directory begin by :
> library(geoSpectral)
> library(testthat)
>
> and there I call package functions directly (without the prefix  geoSpectal:: 
>  )
> See 
> https://github.com/cran/geoSpectral/blob/master/tests/testthat/Spectra_tests.R
>
> Searching the web, I found examples where the same Warning has been issued 
> for some other packages. But in my case the package in question is my own 
> package I am testing
>
> Confused and at loss.  Anyone with ideas?
> regards
> Servet
>
>[[alternative HTML version deleted]]
>
>
>
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

--
Michael
http://www.dewey.myzen.co.uk/home.html

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing my package : unstated dependency to self in package tests

[Jeff Newmiller]

I think the Posting Guide would call this the wrong mailing list for this 
question: should be on R-package-devel.

On February 16, 2020 3:03:55 AM PST, "Servet Ahmet Çizmeli" 
 wrote:
>I am updating my CRAN package geoSpectral. I get the following Warning
>during R CMD check :
>
>...
>* checking for unstated dependencies in �tests� ... WARNING
>'library' or 'require' call not declared from: �geoSpectral�
>
>
>
>All the .R files I have under the testhat directory begin by :
>library(geoSpectral)
>library(testthat)
>
>and there I call package functions directly (without the prefix 
>geoSpectal::  )
>See
>https://github.com/cran/geoSpectral/blob/master/tests/testthat/Spectra_tests.R
>
>Searching the web, I found examples where the same Warning has been
>issued for some other packages. But in my case the package in question
>is my own package I am testing
>
>Confused and at loss.  Anyone with ideas?
>regards
>Servet
>
>   [[alternative HTML version deleted]]

-- 
Sent from my phone. Please excuse my brevity.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing my package : unstated dependency to self in package tests

[Michael Dewey]

When something similar happened to me I found it went away when I added 
Suggests: 

to the DESCRIPTION file. Whether this will work for you I have no idea.

Michael

On 16/02/2020 11:03, Servet Ahmet Çizmeli wrote:

I am updating my CRAN package geoSpectral. I get the following Warning during R 
CMD check :

...
* checking for unstated dependencies in �tests� ... WARNING
'library' or 'require' call not declared from: �geoSpectral�



All the .R files I have under the testhat directory begin by :
library(geoSpectral)
library(testthat)

and there I call package functions directly (without the prefix  geoSpectal::  )
See 
https://github.com/cran/geoSpectral/blob/master/tests/testthat/Spectra_tests.R

Searching the web, I found examples where the same Warning has been issued for 
some other packages. But in my case the package in question is my own package I 
am testing

Confused and at loss.  Anyone with ideas?
regards
Servet

[[alternative HTML version deleted]]




__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



--
Michael
http://www.dewey.myzen.co.uk/home.html

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] testing my package : unstated dependency to self in package tests

[Servet Ahmet Çizmeli]

I am updating my CRAN package geoSpectral. I get the following Warning during R 
CMD check :

...
* checking for unstated dependencies in �tests� ... WARNING
'library' or 'require' call not declared from: �geoSpectral�



All the .R files I have under the testhat directory begin by :
library(geoSpectral)
library(testthat)

and there I call package functions directly (without the prefix  geoSpectal::  )
See 
https://github.com/cran/geoSpectral/blob/master/tests/testthat/Spectra_tests.R

Searching the web, I found examples where the same Warning has been issued for 
some other packages. But in my case the package in question is my own package I 
am testing

Confused and at loss.  Anyone with ideas?
regards
Servet

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing for normality in categorical data

[Jim Lemon]

Hi Nancy,
The chickwts dataset contains one sort-of continuous variable (weight)
and a categorical variable (feed). Two things that will help you to
understand what you are trying to do is to "eyeball" the "weight"
data:

# this shows you the rough distribution of chick weights
hist(chickwts$weight)
# this shows you how well the distribution of weights fits a normal distribution
qqnorm(chickwts$weight)

For the Shapiro-Wilks statistic on the distribution of all of the weights:

shapiro.test(chickwts$weight)

and if you really want to test the normality within the feed groups:

by(chickwts$weight,chickwts$feed,shapiro.test)

Now because the p-values returned are all fairly large, you can accept
the null hypothesis of normality.
As Bert has noted, it looks like you are just throwing the data into
the functions without really knowing what you are doing. Hopefully,
the above will get you started.

Jim

On Sat, Oct 5, 2019 at 11:19 PM Nancy Felix  wrote:
>
> Hello
> I have data that are categorical both independent variable and dependent as
> well having levels more than 3. How can i check the normality of my data?
>
> I have tried the example given of Shapiro-Wilk for levels of factors
>
> data
> summary(chickwts)
>
> ## linear model and ANOVA
> fm <- lm(weight ~ feed, data = chickwts)
> anova(fm)
>
> ## QQ plot for residuals + Shapiro-Wilk test
> shapiro.test(residuals(fm))
>
> ## separate tests for all groups of observations
> ## (with some formatting)
> do.call("rbind", with(chickwts, tapply(weight, feed,
>function(x) unlist(shapiro.test(x)[c("statistic", "p.value")]
>
> But ended up with Error message that x should be numeric and more comments
> see below.
> Hope to get some help on this
>
> Thanks,
> Nancy
>
> ## linear model and ANOVA
> > fm <- lm(retaliation ~ occupation, data = kazi)
> Warning messages:
> 1: In model.response(mf, "numeric") :
>   using type = "numeric" with a factor response will be ignored
> 2: In Ops.factor(y, z$residuals) : ‘-’ not meaningful for factors
> > anova(fm)
> Error in if (ssr < 1e-10 * mss) warning("ANOVA F-tests on an essentially
> perfect fit are unreliable") :
>   missing value where TRUE/FALSE needed
> In addition: Warning message:
> In Ops.factor(object$residuals, 2) : ‘^’ not meaningful for factors
> > ## QQ plot for residuals + Shapiro-Wilk test
> > shapiro.test(residuals(fm))
> Error in class(y) <- oldClass(x) :
>   adding class "factor" to an invalid object
> > ## separate tests for all groups of observations
> > ## (with some formatting)
> > do.call("rbind", with(kazi, tapply(retaliation, occupation,
> +function(x)
> unlist(shapiro.test(x)[c("statistic", "p.value")]
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing for normality in categorical data

[Bert Gunter]

Categorical data cannot be normal. What you are doing is statistical
nonsense, as your error messages suggest.  You need to consult a local
statistician for help.

Furthermore, statistical questions are generally OT on this list, which is
about R programming.

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Sat, Oct 5, 2019 at 6:19 AM Nancy Felix  wrote:

> Hello
> I have data that are categorical both independent variable and dependent as
> well having levels more than 3. How can i check the normality of my data?
>
> I have tried the example given of Shapiro-Wilk for levels of factors
>
> data
> summary(chickwts)
>
> ## linear model and ANOVA
> fm <- lm(weight ~ feed, data = chickwts)
> anova(fm)
>
> ## QQ plot for residuals + Shapiro-Wilk test
> shapiro.test(residuals(fm))
>
> ## separate tests for all groups of observations
> ## (with some formatting)
> do.call("rbind", with(chickwts, tapply(weight, feed,
>function(x) unlist(shapiro.test(x)[c("statistic", "p.value")]
>
> But ended up with Error message that x should be numeric and more comments
> see below.
> Hope to get some help on this
>
> Thanks,
> Nancy
>
> ## linear model and ANOVA
> > fm <- lm(retaliation ~ occupation, data = kazi)
> Warning messages:
> 1: In model.response(mf, "numeric") :
>   using type = "numeric" with a factor response will be ignored
> 2: In Ops.factor(y, z$residuals) : ‘-’ not meaningful for factors
> > anova(fm)
> Error in if (ssr < 1e-10 * mss) warning("ANOVA F-tests on an essentially
> perfect fit are unreliable") :
>   missing value where TRUE/FALSE needed
> In addition: Warning message:
> In Ops.factor(object$residuals, 2) : ‘^’ not meaningful for factors
> > ## QQ plot for residuals + Shapiro-Wilk test
> > shapiro.test(residuals(fm))
> Error in class(y) <- oldClass(x) :
>   adding class "factor" to an invalid object
> > ## separate tests for all groups of observations
> > ## (with some formatting)
> > do.call("rbind", with(kazi, tapply(retaliation, occupation,
> +function(x)
> unlist(shapiro.test(x)[c("statistic", "p.value")]
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing for normality in categorical data

[Nancy Felix]

Hello
I have data that are categorical both independent variable and dependent as
well having levels more than 3. How can i check the normality of my data?

I have tried the example given of Shapiro-Wilk for levels of factors

data
summary(chickwts)

## linear model and ANOVA
fm <- lm(weight ~ feed, data = chickwts)
anova(fm)

## QQ plot for residuals + Shapiro-Wilk test
shapiro.test(residuals(fm))

## separate tests for all groups of observations
## (with some formatting)
do.call("rbind", with(chickwts, tapply(weight, feed,
   function(x) unlist(shapiro.test(x)[c("statistic", "p.value")]

But ended up with Error message that x should be numeric and more comments
see below.
Hope to get some help on this

Thanks,
Nancy

## linear model and ANOVA
> fm <- lm(retaliation ~ occupation, data = kazi)
Warning messages:
1: In model.response(mf, "numeric") :
  using type = "numeric" with a factor response will be ignored
2: In Ops.factor(y, z$residuals) : ‘-’ not meaningful for factors
> anova(fm)
Error in if (ssr < 1e-10 * mss) warning("ANOVA F-tests on an essentially
perfect fit are unreliable") :
  missing value where TRUE/FALSE needed
In addition: Warning message:
In Ops.factor(object$residuals, 2) : ‘^’ not meaningful for factors
> ## QQ plot for residuals + Shapiro-Wilk test
> shapiro.test(residuals(fm))
Error in class(y) <- oldClass(x) :
  adding class "factor" to an invalid object
> ## separate tests for all groups of observations
> ## (with some formatting)
> do.call("rbind", with(kazi, tapply(retaliation, occupation,
+function(x)
unlist(shapiro.test(x)[c("statistic", "p.value")]

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing in spatial sure models

[MacQueen, Don]

You could start by going to the CRAN website, clicking on the "Task Views" item 
on the left, then clicking on "Spatial". This will bring you to a page with 
extensive information about doing things with spatial data in R. It includes 
some brief descriptions of the purposes/capabilities of many spatial-related 
packages in R. Beyond that, when you have the name of a test method, the folks 
at R-sig-geo should be able to help identify which packages might have it.

-- 
Don MacQueen

Lawrence Livermore National Laboratory
7000 East Ave., L-627
Livermore, CA 94550
925-423-1062


On 1/26/17, 3:58 AM, "R-help on behalf of Mª Pilar Gonzalez Casimiro" 
 wrote:


Good afternoon,

I would like to know how to test for homogeneity in spatial sure models.

Thank you,

Pilar


  Pilar González Casimiro
  Facultad de Ciencias Económicas y Empresariales
  Avda. Lehendakari Aguirre, 83 48015 Bilbao
  tfno: 94 601 3730

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing in spatial sure models

[Bert Gunter]

Wrong list.

For statistical questions (which this is), post to
stats.stackexchange.com. I suspect you will have to frame your query
more coherently (context, model, etc.) to get a response even there.

Cheers,
Bert
Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Thu, Jan 26, 2017 at 3:58 AM, Mª Pilar Gonzalez Casimiro
 wrote:
>
> Good afternoon,
>
> I would like to know how to test for homogeneity in spatial sure models.
>
> Thank you,
>
> Pilar
>
>
>  Pilar González Casimiro
>  Facultad de Ciencias Económicas y Empresariales
>  Avda. Lehendakari Aguirre, 83 48015 Bilbao
>  tfno: 94 601 3730
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] testing in spatial sure models

[Mª Pilar Gonzalez Casimiro]

Good afternoon,

I would like to know how to test for homogeneity in spatial sure models.

Thank you,

Pilar


 Pilar González Casimiro
 Facultad de Ciencias Económicas y Empresariales
 Avda. Lehendakari Aguirre, 83 48015 Bilbao
 tfno: 94 601 3730

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether clusters in a PCA plot are significantly different from one another

[Marchesi, Julian]

Dear Micheal

So I would be much better off just reporting the PCA as is and conclude what i 
can from plot

cheers

Julian

Julian R. Marchesi

Deputy Director and Professor of Clinical Microbiome Research at the  Centre 
for Digestive and Gut Health, Imperial College London, London W2 1NY Tel: +44 
(0)20 331 26197

and

Professor of Human Microbiome Research at the School of Biosciences, Museum 
Avenue, Cardiff University, Cardiff, CF10 3AT, Tel: +44 (0)29 208 74188, Fax: 
+44 (0)29 20874305, Mobile 07885 569144





From: Michael Friendly 
Sent: 07 January 2017 17:15
To: Marchesi, Julian; 'r-help@r-project.org'
Subject: Re: testing whether clusters in a PCA plot are significantly different 
from one another

Significance tests for group differences in a MANOVA of
lm(cbind(pc1, pc2) ~ group)

will get you what you want, but you are advised DON'T DO THIS, at least
without a huge grain of salt and a slew of mea culpas.
Otherwise, you are committing p-value abuse and contributing to the
notion that significance tests must be used to justify all conclusions.

The p-values will not be correct under standard normal theory of the
multivariate GLM because the pc1 and pc2 were chosen to optimize
the variance accounted for by their linear combinations and there
is no theory that can correct for this, AFAIK.  The cluster "group"
assignment was also chosen to optimize some (other) criterion.



--
Michael Friendly Email: friendly AT yorku DOT ca
Professor, Psychology Dept. & Chair, Quantitative Methods
York University  Voice: 416 736-2100 x66249 Fax: 416 736-5814
4700 Keele StreetWeb:   http://www.datavis.ca
Toronto, ONT  M3J 1P3 CANADA

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether clusters in a PCA plot are significantly different from one another

[Michael Friendly]

Significance tests for group differences in a MANOVA of
lm(cbind(pc1, pc2) ~ group)

will get you what you want, but you are advised DON'T DO THIS, at least 
without a huge grain of salt and a slew of mea culpas.
Otherwise, you are committing p-value abuse and contributing to the 
notion that significance tests must be used to justify all conclusions.


The p-values will not be correct under standard normal theory of the
multivariate GLM because the pc1 and pc2 were chosen to optimize
the variance accounted for by their linear combinations and there
is no theory that can correct for this, AFAIK.  The cluster "group"
assignment was also chosen to optimize some (other) criterion.



--
Michael Friendly Email: friendly AT yorku DOT ca
Professor, Psychology Dept. & Chair, Quantitative Methods
York University  Voice: 416 736-2100 x66249 Fax: 416 736-5814
4700 Keele StreetWeb:   http://www.datavis.ca
Toronto, ONT  M3J 1P3 CANADA

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether clusters in a PCA plot are significantly different from one another

[Marchesi, Julian]

many thanks david for such a swift response, really appreciate your help

cheers

Julian

Julian R. Marchesi

Deputy Director and Professor of Clinical Microbiome Research at the  Centre 
for Digestive and Gut Health, Imperial College London, London W2 1NY Tel: +44 
(0)20 331 26197

and

Professor of Human Microbiome Research at the School of Biosciences, Museum 
Avenue, Cardiff University, Cardiff, CF10 3AT, Tel: +44 (0)29 208 74188, Fax: 
+44 (0)29 20874305, Mobile 07885 569144





From: David L Carlson <dcarl...@tamu.edu>
Sent: 06 January 2017 15:29
To: Marchesi, Julian; r-help@r-project.org
Subject: RE: [R] testing whether clusters in a PCA plot are significantly 
different from one another

In that case you should be able to use manova where pc1 and pc2 are the 
independent (response) variables and group (Baseline, HFD+P, HFD) is the 
dependent (explanatory) variable. Something like lm(cbind(pc1, pc2)~group). 
That will give you slopes for HFD+P and HFD (difference in mean relative to 
Baseline), t-values, and p-values for each component. You can get further 
diagnostics using package candisc. But your sample size is very small so there 
may be better approaches that a statistician specializing in medical research 
could suggest.

David C

-Original Message-
From: Marchesi, Julian [mailto:j.march...@imperial.ac.uk]
Sent: Friday, January 6, 2017 9:02 AM
To: David L Carlson
Subject: Re: [R] testing whether clusters in a PCA plot are significantly 
different from one another

Dear David

The clusters are defined by the metadata which tells R where to draw the lines 
- no more no less

How would I put a P value to those clusters?

cheers

Julian

Julian R. Marchesi

Deputy Director and Professor of Clinical Microbiome Research at the  Centre 
for Digestive and Gut Health, Imperial College London, London W2 1NY Tel: +44 
(0)20 331 26197

and

Professor of Human Microbiome Research at the School of Biosciences, Museum 
Avenue, Cardiff University, Cardiff, CF10 3AT, Tel: +44 (0)29 208 74188, Fax: 
+44 (0)29 20874305, Mobile 07885 569144





From: David L Carlson <dcarl...@tamu.edu>
Sent: 06 January 2017 14:26
To: Marchesi, Julian
Subject: RE: [R] testing whether clusters in a PCA plot are significantly 
different from one another

You do not say how you defined the clusters in the plot that you attached. If 
you used the variables summarized by the principal components, the answer is 
yes, they are "significantly different".

Cluster analysis creates homogeneous clusters that will almost always be 
"significantly different" using standard tests such as analysis of variance. 
BUT these tests are only meaningful when the clusters are defined independently 
of the data.


David L. Carlson
Department of Anthropology
Texas A University



-Original Message-
From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of Marchesi, Julian
Sent: Friday, January 6, 2017 1:43 AM
To: 'r-help@r-project.org' <r-help@r-project.org>
Subject: [R] testing whether clusters in a PCA plot are significantly different 
from one another

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether clusters in a PCA plot are significantly different from one another

[David L Carlson]

In that case you should be able to use manova where pc1 and pc2 are the 
independent (response) variables and group (Baseline, HFD+P, HFD) is the 
dependent (explanatory) variable. Something like lm(cbind(pc1, pc2)~group). 
That will give you slopes for HFD+P and HFD (difference in mean relative to 
Baseline), t-values, and p-values for each component. You can get further 
diagnostics using package candisc. But your sample size is very small so there 
may be better approaches that a statistician specializing in medical research 
could suggest.

David C

-Original Message-
From: Marchesi, Julian [mailto:j.march...@imperial.ac.uk] 
Sent: Friday, January 6, 2017 9:02 AM
To: David L Carlson
Subject: Re: [R] testing whether clusters in a PCA plot are significantly 
different from one another

Dear David

The clusters are defined by the metadata which tells R where to draw the lines 
- no more no less

How would I put a P value to those clusters?

cheers

Julian

Julian R. Marchesi

Deputy Director and Professor of Clinical Microbiome Research at the  Centre 
for Digestive and Gut Health, Imperial College London, London W2 1NY Tel: +44 
(0)20 331 26197

and

Professor of Human Microbiome Research at the School of Biosciences, Museum 
Avenue, Cardiff University, Cardiff, CF10 3AT, Tel: +44 (0)29 208 74188, Fax: 
+44 (0)29 20874305, Mobile 07885 569144





From: David L Carlson <dcarl...@tamu.edu>
Sent: 06 January 2017 14:26
To: Marchesi, Julian
Subject: RE: [R] testing whether clusters in a PCA plot are significantly 
different from one another

You do not say how you defined the clusters in the plot that you attached. If 
you used the variables summarized by the principal components, the answer is 
yes, they are "significantly different".

Cluster analysis creates homogeneous clusters that will almost always be 
"significantly different" using standard tests such as analysis of variance. 
BUT these tests are only meaningful when the clusters are defined independently 
of the data.


David L. Carlson
Department of Anthropology
Texas A University



-Original Message-
From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of Marchesi, Julian
Sent: Friday, January 6, 2017 1:43 AM
To: 'r-help@r-project.org' <r-help@r-project.org>
Subject: [R] testing whether clusters in a PCA plot are significantly different 
from one another

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] testing whether clusters in a PCA plot are significantly different from one another

[Marchesi, Julian]

Rplot_PCA.pdf
Description: Rplot_PCA.pdf
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing significance of individual regression slopes

[Bert Gunter]

You can't if I understand correctly: there is no individual subject
regression coefficient, only a variance component for a random subject
intercept. Do you mean that you want to "test" whether that component
is nonzero ?(It is, of course).  If so, IIRC, lmer eschews such tests
for technical reasons -- they are based on approximations that lmer's
authors (esp. Doug Bates) contend are unreliable.

However, a web search on "test significance of lmer variance
components" brought up this:

https://www.r-bloggers.com/random-regression-coefficients-using-lme4/

and the "lmerTest" package, both of which seem relevant, again
assuming I have correctly divined your intent. You will have to
consult the literature or seek advice elsewhere (purely statistical
matters are generally OT here) to determine whether they are
appropriate for your situation.

Cheers,
Bert






Bert Gunter

"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Tue, Sep 27, 2016 at 8:05 AM, Patzelt, Edward  wrote:
> Hi R-help,
>
> I have an lmer logistic regression with a within subjects IV and subject as
> a random factor:
>
> model <- lmer(optimal_choice ~ level_one_value_difference + (1|subid), data
> = dat)
>
> What I want is to test if the individual subject regression coefficient is
> significantly different from 0.
>
>
> --
> Edward H Patzelt | Clinical Science PhD Student
> Psychology | Harvard University
> *Computational Cognitive Neuroscience Laboratory
> *
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing significance of individual regression slopes

[Patzelt, Edward]

Hi R-help,

I have an lmer logistic regression with a within subjects IV and subject as
a random factor:

model <- lmer(optimal_choice ~ level_one_value_difference + (1|subid), data
= dat)

What I want is to test if the individual subject regression coefficient is
significantly different from 0.


-- 
Edward H Patzelt | Clinical Science PhD Student
Psychology | Harvard University
*Computational Cognitive Neuroscience Laboratory
*

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing installed package of rJava in Linux

[Stephen Connolly]

The variable DISPLAY is what is causing problems. Run the command 'unset
DISPLAY' before running R .

Stephen


On 09/03/16 01:06 PM, Uwe Ligges wrote:
> I do not get this: If it works it is OK to use it. If it does not work,
> you can't
> 
> Best,
> Uwe Kigges
> 
> 
> 
> 
> 
> On 09.03.2016 17:44, Santosh wrote:
>> Thanks for your response. Since the test failed due to X11 connectivity
>> reasons, is it okay to use it in applications where X11 server
>> connectivity
>> is not required?
>> Thanks and much appreciated,
>> Santosh
>>
>> On Tue, Mar 8, 2016 at 10:41 PM, Uwe Ligges
>> >> wrote:
>>
>>>
>>>
>>> On 09.03.2016 02:19, Santosh wrote:
>>>
 Dear Rxperts..
 I installed rJava on 64-bit Linux system and apparently it installed
 without errors.However, I got the following error message when I
 tried to
 test the installed package.

 
 Error in .jcall("RJavaTools", "Ljava/lang/Object;", "newInstance",
 .jfindClass(class),  :
 java.lang.InternalError: Can't connect to X11 window server
 using ':0'
 as
 the value of the DISPLAY variable.
 Calls: new -> new -> .J -> .jcall -> .jcheck -> .Call
 Execution halted

>>>
>>>
>>> Apparently you do not have an X server running or no X forwarding
>>> enabled?
>>>
>>> Best,
>>> Uwe Ligges
>>>
>>>
>>> 

 Would highly appreciate your tips/suggestions..

 Thanks and much appreciated,
 Santosh

  [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
> 
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing installed package of rJava in Linux

[Uwe Ligges]

I do not get this: If it works it is OK to use it. If it does not work, 
you can't


Best,
Uwe Kigges





On 09.03.2016 17:44, Santosh wrote:

Thanks for your response. Since the test failed due to X11 connectivity
reasons, is it okay to use it in applications where X11 server connectivity
is not required?
Thanks and much appreciated,
Santosh

On Tue, Mar 8, 2016 at 10:41 PM, Uwe Ligges  new -> .J -> .jcall -> .jcheck -> .Call
Execution halted




Apparently you do not have an X server running or no X forwarding enabled?

Best,
Uwe Ligges





Would highly appreciate your tips/suggestions..

Thanks and much appreciated,
Santosh

 [[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.




[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing installed package of rJava in Linux

[Santosh]

Thanks for your response. Since the test failed due to X11 connectivity
reasons, is it okay to use it in applications where X11 server connectivity
is not required?
Thanks and much appreciated,
Santosh

On Tue, Mar 8, 2016 at 10:41 PM, Uwe Ligges  wrote:

>
>
> On 09.03.2016 02:19, Santosh wrote:
>
>> Dear Rxperts..
>> I installed rJava on 64-bit Linux system and apparently it installed
>> without errors.However, I got the following error message when I tried to
>> test the installed package.
>>
>> 
>> Error in .jcall("RJavaTools", "Ljava/lang/Object;", "newInstance",
>> .jfindClass(class),  :
>>java.lang.InternalError: Can't connect to X11 window server using ':0'
>> as
>> the value of the DISPLAY variable.
>> Calls: new -> new -> .J -> .jcall -> .jcheck -> .Call
>> Execution halted
>>
>
>
> Apparently you do not have an X server running or no X forwarding enabled?
>
> Best,
> Uwe Ligges
>
>
> 
>>
>> Would highly appreciate your tips/suggestions..
>>
>> Thanks and much appreciated,
>> Santosh
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing installed package of rJava in Linux

[Uwe Ligges]

On 09.03.2016 02:19, Santosh wrote:

Dear Rxperts..
I installed rJava on 64-bit Linux system and apparently it installed
without errors.However, I got the following error message when I tried to
test the installed package.


Error in .jcall("RJavaTools", "Ljava/lang/Object;", "newInstance",
.jfindClass(class),  :
   java.lang.InternalError: Can't connect to X11 window server using ':0' as
the value of the DISPLAY variable.
Calls: new -> new -> .J -> .jcall -> .jcheck -> .Call
Execution halted



Apparently you do not have an X server running or no X forwarding enabled?

Best,
Uwe Ligges





Would highly appreciate your tips/suggestions..

Thanks and much appreciated,
Santosh

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing installed package of rJava in Linux

[Santosh]

Dear Rxperts..
I installed rJava on 64-bit Linux system and apparently it installed
without errors.However, I got the following error message when I tried to
test the installed package.


Error in .jcall("RJavaTools", "Ljava/lang/Object;", "newInstance",
.jfindClass(class),  :
  java.lang.InternalError: Can't connect to X11 window server using ':0' as
the value of the DISPLAY variable.
Calls: new -> new -> .J -> .jcall -> .jcheck -> .Call
Execution halted


Would highly appreciate your tips/suggestions..

Thanks and much appreciated,
Santosh

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing a non-binary categorical variable with coxph and robust variance

[Brant Inman]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing Restrictions on Beta (long-run coefficients), reproducible example

[mrrox]

The code given below estimates a VEC model with 4 cointegrating vectors. It
is a reproducible code, so just copy and paste into your R console (or
script editor).

nobs = 200
e = rmvnorm(n=nobs,sigma=diag(c(.5,.5,.5,.5,.5)))
e1.ar1 = arima.sim(model=list(ar=.75),nobs,innov=e[,1])
e2.ar1 = arima.sim(model=list(ar=.75),nobs,innov=e[,2])
e3.ar1 = arima.sim(model=list(ar=.75),nobs,innov=e[,3])
e4.ar1 = arima.sim(model=list(ar=.75),nobs,innov=e[,4])
y5 = cumsum(e[,5])
y1 = y5 + e1.ar1
y2 = y5 + e2.ar1
y3 = y5 + e3.ar1
y4 = y5 + e4.ar1
data = cbind(y1,y2,y3,y4,y5)

jcointt = ca.jo(data,ecdet="const",type="trace",K=2,spec="transitory")
summary(jcointt)
# estimate VECM with 4 cointegrating vectors
vecm <- cajorls(jcointt,r=4)
summary(vecm$rlm)
print(vecm)


I want to re-estimate the model with the following restrictions put on the
coinegrating vectors:

   ect1   ect2  ect3   ect4
y1.l1   1  0  0 0
y2.l1b1.1 1  0 0
y3.l1b2.1 0  1 0
y4.l1b3.1 0  0 1
y5.l1b4.1b4.2   b4.3   b4.4
constantc1  c2 c3c4  


here, b1.1 through to b4.1 are the coefficients (β1,β2,β3,β4) of the first
cointegrating vector. Similarly, b4.4 and c4 are coefficients of the fourth
cointegrating equation. Then, in order to test the restrictions on
Coinegrating Vectors, I run the following code:

test <- blrtest(jcointt,H=H1,r=4)


However, I do not know how I should specify the H1 matrix in this instance.
I was wondering if someone could demonstrate how I should go ahead with
testing the restrictions on long run equations and then re-estimate the
model using the above restrictions:

vecm2 <- cajorls(test,r=4)
summary(vecm2$rlm)
print(vecm2)


How should I specify the H1 matrix above in order to re-estimate the
re-parameterised cointegrating equations? I want to use the coefficients of
the first cointegrating equation (ect1) for inference.



--
View this message in context: 
http://r.789695.n4.nabble.com/Testing-Restrictions-on-Beta-long-run-coefficients-reproducible-example-tp4713512.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether two character vectors contain (the same) items in the same order

[Robert Baer]

On 8/6/2015 5:25 AM, Federico Calboli wrote:

Hi All,

let’s assume I have a vector of letters drawn only once from the alphabet:

x = sample(letters, 15, replace = F)
x
  [1] z t g l u d w x a q k j f n “v

y = x[c(1:7,9:8, 10:12, 14, 15, 13)]

I would now like to test how good a match y is for x.  Obviously I can 
transform the letters in numbers and use a rank test, but I was left wondering 
whether this is the only solution and whether there are more appropriate 
solutions that are already implemented in R (I am not going to reinvent the 
wheel if I can avoid it).

BW

F

Perhaps
install.packages(stringdist)
help(package = 'stringdist')







--
Federico Calboli
Ecological Genetics Research Unit
Department of Biosciences
PO Box 65 (Biocenter 3, Viikinkaari 1)
FIN-00014 University of Helsinki
Finland

federico.calb...@helsinki.fi

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether two character vectors contain (the same) items in the same order

[Robert Baer]

And I probably should have included this link:
http://journal.r-project.org/archive/2014-1/loo.pdf

On 8/8/2015 12:50 PM, Robert Baer wrote:



On 8/6/2015 5:25 AM, Federico Calboli wrote:

Hi All,

let’s assume I have a vector of letters drawn only once from the 
alphabet:


x = sample(letters, 15, replace = F)
x
  [1] z t g l u d w x a q k j f n “v

y = x[c(1:7,9:8, 10:12, 14, 15, 13)]

I would now like to test how good a match y is for x.  Obviously I 
can transform the letters in numbers and use a rank test, but I was 
left wondering whether this is the only solution and whether there 
are more appropriate solutions that are already implemented in R (I 
am not going to reinvent the wheel if I can avoid it).


BW

F

Perhaps
install.packages(stringdist)
help(package = 'stringdist')







--
Federico Calboli
Ecological Genetics Research Unit
Department of Biosciences
PO Box 65 (Biocenter 3, Viikinkaari 1)
FIN-00014 University of Helsinki
Finland

federico.calb...@helsinki.fi

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html

and provide commented, minimal, self-contained, reproducible code.




__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether two character vectors contain (the same) items in the same order

[Federico Calboli]

 On 7 Aug 2015, at 01:59, Bert Gunter bgunter.4...@gmail.com wrote:
 
 Boris:
 
 You may be right, but it seems like esp to me based on the op's 
 non-description of likelihood of coming from the same noisy process. My 
 response would be: seek local statistical help, as your replies indicate a 
 good deal of statistical confusion.
 
 Cheers,
 Bert

Bert,

as this is R-help and not cross-validated I am looking for a precanned function 
that would test whether the order of characters in two character vectors comes 
from the same (noisy) process.  I would thus expect you to say something on the 
lines of:

function X uses method Y to do something like that
function W uses method Z to do something like that
…

look into those, figure out exactly what you are testing and use the most 
appropiate function.  

The whys and wherefores are for me to deal with, I just want to know whether 
someone has built a function that does, or seems to do, what I asked for.  As I 
said, this is R-help, and I seek help for R use.

I do concede that my original question might have left many wondering, but I 
guess my reply to Boris would have cleared any doubts.  I am therefore puzzled 
by the great deal of confusion on your part in understanding the purpose of my 
question and, in general, of this list.

Best wishes

F


 
 
 
 On Thursday, August 6, 2015, Boris Steipe boris.ste...@utoronto.ca wrote:
 You are looking for what is known as the Cayley distance between vectors - 
 an edit distance that allows only transpositions. RSeek mentions PerMallows 
 (https://cran.r-project.org/web/packages/PerMallows/PerMallows.pdf) and 
 Rankluster 
 (https://cran.r-project.org/web/packages/Rankcluster/Rankcluster.pdf) as 
 packages that support work with Cayley distances. It seems to me that 
 distCayley() in Rankcluster does what you want. From the examples:
 
 x=1:5
 y=c(2,3,1,4,5)
 distCayley(x,y)
 8
 
 
 Cheers,
 Boris
 
 
 
 
 
 On Aug 6, 2015, at 9:51 AM, Federico Calboli federico.calb...@helsinki.fi 
 wrote:
 
 
  On 6 Aug 2015, at 15:40, Bert Gunter bgunter.4...@gmail.com wrote:
 
  Define goodness of match .  For exact matches, see ?== , all.equal, 
  etc.
 
  Fair point.  I would define it as a number that tells me how likely it is 
  that the same (noisy) process produced both lists.
 
  BW
 
  F
 
 
 
 
 
  Bert
 
  On Thursday, August 6, 2015, Federico Calboli 
  federico.calb...@helsinki.fi wrote:
  Hi All,
 
  let’s assume I have a vector of letters drawn only once from the alphabet:
 
  x = sample(letters, 15, replace = F)
  x
  [1] z t g l u d w x a q k j f n “v
 
  y = x[c(1:7,9:8, 10:12, 14, 15, 13)]
 
  I would now like to test how good a match y is for x.  Obviously I can 
  transform the letters in numbers and use a rank test, but I was left 
  wondering whether this is the only solution and whether there are more 
  appropriate solutions that are already implemented in R (I am not going to 
  reinvent the wheel if I can avoid it).
 
  BW
 
  F
 
 
  --
  Federico Calboli
  Ecological Genetics Research Unit
  Department of Biosciences
  PO Box 65 (Biocenter 3, Viikinkaari 1)
  FIN-00014 University of Helsinki
  Finland
 
  federico.calb...@helsinki.fi
 
  __
  R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
  https://stat.ethz.ch/mailman/listinfo/r-help
  PLEASE do read the posting guide 
  http://www.R-project.org/posting-guide.html
  and provide commented, minimal, self-contained, reproducible code.
 
 
  --
  Bert Gunter
 
  Data is not information. Information is not knowledge. And knowledge is 
  certainly not wisdom.
-- Clifford Stoll
 
 
 
  --
  Federico Calboli
  Ecological Genetics Research Unit
  Department of Biosciences
  PO Box 65 (Biocenter 3, Viikinkaari 1)
  FIN-00014 University of Helsinki
  Finland
 
  federico.calb...@helsinki.fi
 
  __
  R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
  https://stat.ethz.ch/mailman/listinfo/r-help
  PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
  and provide commented, minimal, self-contained, reproducible code.
 
 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 
 
 -- 
 Bert Gunter
 
 Data is not information. Information is not knowledge. And knowledge is 
 certainly not wisdom.
-- Clifford Stoll
 


--
Federico Calboli
Ecological Genetics Research Unit
Department of Biosciences
PO Box 65 (Biocenter 3, Viikinkaari 1)
FIN-00014 University of Helsinki
Finland

federico.calb...@helsinki.fi

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help

Re: [R] testing whether two character vectors contain (the same) items in the same order

[David Winsemius]

On Aug 7, 2015, at 12:22 AM, Federico Calboli wrote:

 
 On 7 Aug 2015, at 01:59, Bert Gunter bgunter.4...@gmail.com wrote:
 
 Boris:
 
 You may be right, but it seems like esp to me based on the op's 
 non-description of likelihood of coming from the same noisy process. My 
 response would be: seek local statistical help, as your replies indicate a 
 good deal of statistical confusion.
 
 Cheers,
 Bert
 
 Bert,
 
 as this is R-help and not cross-validated I am looking for a precanned 
 function that would test whether the order of characters in two character 
 vectors comes from the same (noisy) process.  I would thus expect you to say 
 something on the lines of:
 
 function X uses method Y to do something like that
 function W uses method Z to do something like that
 …
 
 look into those, figure out exactly what you are testing and use the most 
 appropiate function.  
 
 The whys and wherefores are for me to deal with, I just want to know whether 
 someone has built a function that does, or seems to do, what I asked for.  As 
 I said, this is R-help, and I seek help for R use.

 findFn(levenshtein)
found 57 matches;  retrieving 3 pages
2 3 
Downloaded 44 links in 17 packages.


 stringdist::stringdist( paste0(x, collapse=), paste0(letters[y], 
collapse=) )
[1] 30

-- 
HTH;
David.

 
 I do concede that my original question might have left many wondering, but I 
 guess my reply to Boris would have cleared any doubts.  I am therefore 
 puzzled by the great deal of confusion on your part in understanding the 
 purpose of my question and, in general, of this list.
 
 Best wishes
 
 F
 
 
 
 
 
 On Thursday, August 6, 2015, Boris Steipe boris.ste...@utoronto.ca wrote:
 You are looking for what is known as the Cayley distance between vectors - 
 an edit distance that allows only transpositions. RSeek mentions PerMallows 
 (https://cran.r-project.org/web/packages/PerMallows/PerMallows.pdf) and 
 Rankluster 
 (https://cran.r-project.org/web/packages/Rankcluster/Rankcluster.pdf) as 
 packages that support work with Cayley distances. It seems to me that 
 distCayley() in Rankcluster does what you want. From the examples:
 
 x=1:5
 y=c(2,3,1,4,5)
 distCayley(x,y)
 8
 
 
 Cheers,
 Boris
 
 
 
 
 
 On Aug 6, 2015, at 9:51 AM, Federico Calboli federico.calb...@helsinki.fi 
 wrote:
 
 
 On 6 Aug 2015, at 15:40, Bert Gunter bgunter.4...@gmail.com wrote:
 
 Define goodness of match .  For exact matches, see ?== , all.equal, 
 etc.
 
 Fair point.  I would define it as a number that tells me how likely it is 
 that the same (noisy) process produced both lists.
 
 BW
 
 F
 
 
 
 
 
 Bert
 
 On Thursday, August 6, 2015, Federico Calboli 
 federico.calb...@helsinki.fi wrote:
 Hi All,
 
 let’s assume I have a vector of letters drawn only once from the alphabet:
 
 x = sample(letters, 15, replace = F)
 x
 [1] z t g l u d w x a q k j f n “v
 
 y = x[c(1:7,9:8, 10:12, 14, 15, 13)]
 
 I would now like to test how good a match y is for x.  Obviously I can 
 transform the letters in numbers and use a rank test, but I was left 
 wondering whether this is the only solution and whether there are more 
 appropriate solutions that are already implemented in R (I am not going to 
 reinvent the wheel if I can avoid it).
 
 BW
 
 F
 
 
 --
 Federico Calboli
 Ecological Genetics Research Unit
 Department of Biosciences
 PO Box 65 (Biocenter 3, Viikinkaari 1)
 FIN-00014 University of Helsinki
 Finland
 
 federico.calb...@helsinki.fi
 
 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide 
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 
 
 --
 Bert Gunter
 
 Data is not information. Information is not knowledge. And knowledge is 
 certainly not wisdom.
  -- Clifford Stoll
 
 
 
 --
 Federico Calboli
 Ecological Genetics Research Unit
 Department of Biosciences
 PO Box 65 (Biocenter 3, Viikinkaari 1)
 FIN-00014 University of Helsinki
 Finland
 
 federico.calb...@helsinki.fi
 
 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 
 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 
 
 -- 
 Bert Gunter
 
 Data is not information. Information is not knowledge. And knowledge is 
 certainly not wisdom.
   -- Clifford Stoll
 
 
 
 --
 Federico Calboli
 Ecological Genetics Research Unit
 Department of Biosciences
 PO Box 65 (Biocenter 3, 

[R] testing whether two character vectors contain (the same) items in the same order

[Federico Calboli]

Hi All,

let’s assume I have a vector of letters drawn only once from the alphabet:

x = sample(letters, 15, replace = F)
x
 [1] z t g l u d w x a q k j f n “v

y = x[c(1:7,9:8, 10:12, 14, 15, 13)]

I would now like to test how good a match y is for x.  Obviously I can 
transform the letters in numbers and use a rank test, but I was left wondering 
whether this is the only solution and whether there are more appropriate 
solutions that are already implemented in R (I am not going to reinvent the 
wheel if I can avoid it).

BW

F


--
Federico Calboli
Ecological Genetics Research Unit
Department of Biosciences
PO Box 65 (Biocenter 3, Viikinkaari 1)
FIN-00014 University of Helsinki
Finland

federico.calb...@helsinki.fi

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether two character vectors contain (the same) items in the same order

[Bert Gunter]

Boris:

You may be right, but it seems like esp to me based on the op's
non-description of likelihood of coming from the same noisy process. My
response would be: seek local statistical help, as your replies indicate a
good deal of statistical confusion.

Cheers,
Bert



On Thursday, August 6, 2015, Boris Steipe boris.ste...@utoronto.ca wrote:

 You are looking for what is known as the Cayley distance between vectors
 - an edit distance that allows only transpositions. RSeek mentions
 PerMallows (
 https://cran.r-project.org/web/packages/PerMallows/PerMallows.pdf) and
 Rankluster (
 https://cran.r-project.org/web/packages/Rankcluster/Rankcluster.pdf) as
 packages that support work with Cayley distances. It seems to me that
 distCayley() in Rankcluster does what you want. From the examples:

 x=1:5
 y=c(2,3,1,4,5)
 distCayley(x,y)
 8


 Cheers,
 Boris





 On Aug 6, 2015, at 9:51 AM, Federico Calboli federico.calb...@helsinki.fi
 javascript:; wrote:

 
  On 6 Aug 2015, at 15:40, Bert Gunter bgunter.4...@gmail.com
 javascript:; wrote:
 
  Define goodness of match .  For exact matches, see ?== , all.equal,
 etc.
 
  Fair point.  I would define it as a number that tells me how likely it
 is that the same (noisy) process produced both lists.
 
  BW
 
  F
 
 
 
 
 
  Bert
 
  On Thursday, August 6, 2015, Federico Calboli 
 federico.calb...@helsinki.fi javascript:; wrote:
  Hi All,
 
  let’s assume I have a vector of letters drawn only once from the
 alphabet:
 
  x = sample(letters, 15, replace = F)
  x
  [1] z t g l u d w x a q k j f n “v
 
  y = x[c(1:7,9:8, 10:12, 14, 15, 13)]
 
  I would now like to test how good a match y is for x.  Obviously I can
 transform the letters in numbers and use a rank test, but I was left
 wondering whether this is the only solution and whether there are more
 appropriate solutions that are already implemented in R (I am not going to
 reinvent the wheel if I can avoid it).
 
  BW
 
  F
 
 
  --
  Federico Calboli
  Ecological Genetics Research Unit
  Department of Biosciences
  PO Box 65 (Biocenter 3, Viikinkaari 1)
  FIN-00014 University of Helsinki
  Finland
 
  federico.calb...@helsinki.fi javascript:;
 
  __
  R-help@r-project.org javascript:; mailing list -- To UNSUBSCRIBE and
 more, see
  https://stat.ethz.ch/mailman/listinfo/r-help
  PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
  and provide commented, minimal, self-contained, reproducible code.
 
 
  --
  Bert Gunter
 
  Data is not information. Information is not knowledge. And knowledge
 is certainly not wisdom.
-- Clifford Stoll
 
 
 
  --
  Federico Calboli
  Ecological Genetics Research Unit
  Department of Biosciences
  PO Box 65 (Biocenter 3, Viikinkaari 1)
  FIN-00014 University of Helsinki
  Finland
 
  federico.calb...@helsinki.fi javascript:;
 
  __
  R-help@r-project.org javascript:; mailing list -- To UNSUBSCRIBE and
 more, see
  https://stat.ethz.ch/mailman/listinfo/r-help
  PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
  and provide commented, minimal, self-contained, reproducible code.

 __
 R-help@r-project.org javascript:; mailing list -- To UNSUBSCRIBE and
 more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



-- 
Bert Gunter

Data is not information. Information is not knowledge. And knowledge is
certainly not wisdom.
   -- Clifford Stoll

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether two character vectors contain (the same) items in the same order

[Bert Gunter]

Define goodness of match .  For exact matches, see ?== , all.equal, etc.

Bert

On Thursday, August 6, 2015, Federico Calboli federico.calb...@helsinki.fi
wrote:

 Hi All,

 let’s assume I have a vector of letters drawn only once from the alphabet:

 x = sample(letters, 15, replace = F)
 x
  [1] z t g l u d w x a q k j f n “v

 y = x[c(1:7,9:8, 10:12, 14, 15, 13)]

 I would now like to test how good a match y is for x.  Obviously I can
 transform the letters in numbers and use a rank test, but I was left
 wondering whether this is the only solution and whether there are more
 appropriate solutions that are already implemented in R (I am not going to
 reinvent the wheel if I can avoid it).

 BW

 F


 --
 Federico Calboli
 Ecological Genetics Research Unit
 Department of Biosciences
 PO Box 65 (Biocenter 3, Viikinkaari 1)
 FIN-00014 University of Helsinki
 Finland

 federico.calb...@helsinki.fi javascript:;

 __
 R-help@r-project.org javascript:; mailing list -- To UNSUBSCRIBE and
 more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



-- 
Bert Gunter

Data is not information. Information is not knowledge. And knowledge is
certainly not wisdom.
   -- Clifford Stoll

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether two character vectors contain (the same) items in the same order

[Boris Steipe]

You are looking for what is known as the Cayley distance between vectors - an 
edit distance that allows only transpositions. RSeek mentions PerMallows 
(https://cran.r-project.org/web/packages/PerMallows/PerMallows.pdf) and 
Rankluster 
(https://cran.r-project.org/web/packages/Rankcluster/Rankcluster.pdf) as 
packages that support work with Cayley distances. It seems to me that 
distCayley() in Rankcluster does what you want. From the examples:

x=1:5
y=c(2,3,1,4,5)
distCayley(x,y)
8


Cheers,
Boris





On Aug 6, 2015, at 9:51 AM, Federico Calboli federico.calb...@helsinki.fi 
wrote:

 
 On 6 Aug 2015, at 15:40, Bert Gunter bgunter.4...@gmail.com wrote:
 
 Define goodness of match .  For exact matches, see ?== , all.equal, etc.
 
 Fair point.  I would define it as a number that tells me how likely it is 
 that the same (noisy) process produced both lists.
 
 BW
 
 F
 
 
 
 
 
 Bert
 
 On Thursday, August 6, 2015, Federico Calboli federico.calb...@helsinki.fi 
 wrote:
 Hi All,
 
 let’s assume I have a vector of letters drawn only once from the alphabet:
 
 x = sample(letters, 15, replace = F)
 x
 [1] z t g l u d w x a q k j f n “v
 
 y = x[c(1:7,9:8, 10:12, 14, 15, 13)]
 
 I would now like to test how good a match y is for x.  Obviously I can 
 transform the letters in numbers and use a rank test, but I was left 
 wondering whether this is the only solution and whether there are more 
 appropriate solutions that are already implemented in R (I am not going to 
 reinvent the wheel if I can avoid it).
 
 BW
 
 F
 
 
 --
 Federico Calboli
 Ecological Genetics Research Unit
 Department of Biosciences
 PO Box 65 (Biocenter 3, Viikinkaari 1)
 FIN-00014 University of Helsinki
 Finland
 
 federico.calb...@helsinki.fi
 
 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 
 
 -- 
 Bert Gunter
 
 Data is not information. Information is not knowledge. And knowledge is 
 certainly not wisdom.
   -- Clifford Stoll
 
 
 
 --
 Federico Calboli
 Ecological Genetics Research Unit
 Department of Biosciences
 PO Box 65 (Biocenter 3, Viikinkaari 1)
 FIN-00014 University of Helsinki
 Finland
 
 federico.calb...@helsinki.fi
 
 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing whether two character vectors contain (the same) items in the same order

[Federico Calboli]

 On 6 Aug 2015, at 15:40, Bert Gunter bgunter.4...@gmail.com wrote:
 
 Define goodness of match .  For exact matches, see ?== , all.equal, etc.

Fair point.  I would define it as a number that tells me how likely it is that 
the same (noisy) process produced both lists.

BW

F




 
 Bert
 
 On Thursday, August 6, 2015, Federico Calboli federico.calb...@helsinki.fi 
 wrote:
 Hi All,
 
 let’s assume I have a vector of letters drawn only once from the alphabet:
 
 x = sample(letters, 15, replace = F)
 x
  [1] z t g l u d w x a q k j f n “v
 
 y = x[c(1:7,9:8, 10:12, 14, 15, 13)]
 
 I would now like to test how good a match y is for x.  Obviously I can 
 transform the letters in numbers and use a rank test, but I was left 
 wondering whether this is the only solution and whether there are more 
 appropriate solutions that are already implemented in R (I am not going to 
 reinvent the wheel if I can avoid it).
 
 BW
 
 F
 
 
 --
 Federico Calboli
 Ecological Genetics Research Unit
 Department of Biosciences
 PO Box 65 (Biocenter 3, Viikinkaari 1)
 FIN-00014 University of Helsinki
 Finland
 
 federico.calb...@helsinki.fi
 
 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 
 
 -- 
 Bert Gunter
 
 Data is not information. Information is not knowledge. And knowledge is 
 certainly not wisdom.
-- Clifford Stoll
 


--
Federico Calboli
Ecological Genetics Research Unit
Department of Biosciences
PO Box 65 (Biocenter 3, Viikinkaari 1)
FIN-00014 University of Helsinki
Finland

federico.calb...@helsinki.fi

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing for significant differences between groups in multiple linear regression

[Bert Gunter]

Look no further!  The answer is yes.

However,  if you are interested in why your query is probably nonsense
and why overall tests of significance are a **really bad idea** in
most scientific contexts (imho, anyway), then I suggest you post to a
statistical list like stats.stackexchange.com .

... oh, and while you're at it, please read the posting guide for this
list (see link below) and, in particular, DO NOT POST IN HTML, which,
as you can see here, often becomes a mess on this **plain text**
mailing list.

Cheers,
Bert


Bert Gunter
Genentech Nonclinical Biostatistics
(650) 467-7374

Data is not information. Information is not knowledge. And knowledge
is certainly not wisdom.
Clifford Stoll




On Fri, Jan 23, 2015 at 1:46 AM, Janka Vanschoenwinkel
janka.vanschoenwin...@uhasselt.be wrote:
 Dear R-colleagues,

 I am looking for a way to test whether one regression has significant
 different coefficients and overall results for 10 groups (grouping variable
 is irr).



 *What I have*

 The regression is:

 Depend = temp + temp² + perc + perc² + conti è split up for multiple groups
 of irr


   *Dataset = Alldata (real dataset has over 5 IDs)*

 *ID*

 *irr *

 *(= grouping variable)*

 *temp*

 *perc*

 *conti*

 *Depend*

 *w*

 1

 1

 10

 34

 26

 8

 23

 2

 1

 11

 36

 27

 6

 58

 3

 1

 26

 57

 45

 3

 76

 4

 2

 23

 68

 24

 2

 4

 5

 2

 6

 26

 8

 1

 323

 6

 2

 3

 17

 56

 6

 45

 7

 3

 17

 39

 17

 5

 57



 I can obtain the different regression coefficients for the different groups
 with the following code (other codes are possible as wel).


 datairrigation - split(Alldata, Alldata$irr)

 model.per.irrigation - lapply(datairrigation, function (x) {

   lm(Depend~ temp + temp² + perc + perc² + conti,

  weights=w, data = x)

 })


 OR I can do it manually by splitting all the data in subsets (and then I
 also receive the R²…)



 *What I don’t have*

 However, now I don’t know how to compare those regressions to test whether
 they differ significantly over all the groups.

 (Preferably, I would like to test the coefficients individually (temp(group
 1) = temp(group2)) and the regression as a whole between the groups.)



 *Note*

 I know that one way to test differences in significance between groups, is
 to use dummy variables of that group, in the regression. Yet, this is no
 option for my model because it only allows exogenous variables in the
 regression (and irrigation is an endogenous variable because the farmer can
 decide himself if he irrigates or not).



 Thank you very much in advance! I really appreciate your help!


 Janka


 P Please consider the environment before printing this e-mail

 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing for significant differences between groups in multiple linear regression

[Janka Vanschoenwinkel]

Dear R-colleagues,

I am looking for a way to test whether one regression has significant
different coefficients and overall results for 10 groups (grouping variable
is irr).



*What I have*

The regression is:

Depend = temp + temp² + perc + perc² + conti è split up for multiple groups
of irr


  *Dataset = Alldata (real dataset has over 5 IDs)*

*ID*

*irr *

*(= grouping variable)*

*temp*

*perc*

*conti*

*Depend*

*w*

1

1

10

34

26

8

23

2

1

11

36

27

6

58

3

1

26

57

45

3

76

4

2

23

68

24

2

4

5

2

6

26

8

1

323

6

2

3

17

56

6

45

7

3

17

39

17

5

57



I can obtain the different regression coefficients for the different groups
with the following code (other codes are possible as wel).


datairrigation - split(Alldata, Alldata$irr)

model.per.irrigation - lapply(datairrigation, function (x) {

  lm(Depend~ temp + temp² + perc + perc² + conti,

 weights=w, data = x)

})


OR I can do it manually by splitting all the data in subsets (and then I
also receive the R²…)



*What I don’t have*

However, now I don’t know how to compare those regressions to test whether
they differ significantly over all the groups.

(Preferably, I would like to test the coefficients individually (temp(group
1) = temp(group2)) and the regression as a whole between the groups.)



*Note*

I know that one way to test differences in significance between groups, is
to use dummy variables of that group, in the regression. Yet, this is no
option for my model because it only allows exogenous variables in the
regression (and irrigation is an endogenous variable because the farmer can
decide himself if he irrigates or not).



Thank you very much in advance! I really appreciate your help!


Janka


P Please consider the environment before printing this e-mail

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing general hypotheses on regression coefficients

[bonsxanco]

On Sunday, September 7, 2014 5:47 PM, peter dalgaard pda...@gmail.com wrote: 
 On 06 Sep 2014, at 12:24 , bonsxanco bonsxa...@yahoo.com wrote: 
 
  
  1) 8th grade algebra tells me B2/B1 == 0 == B2 =0; 
  
  EViews (econometrics program) doesn't have the same opinion: 
  
  Wald test on my real model (edited): 
  
  * H0: B3/B2 = 0 - F-stat = 37.82497 
  * H0: B3 = 0- F-stat = 16.31689 
 
 
 And when the econometrics program contradicts what you learned in 8th grade, 
 surely the latter is wrong and the former is right, because it is done by a 
 computer and computers cannot be wrong? ;-) 
I simply thought that there was a standard way to do this: EViews and Stata 
both give the exact same F statistic for my original problem. Given that these 
programs were not developed by the same author (AFAIK), there is some specific 
way to reformulate the restriction which make EViews and Stata give the same 
answer.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing general hypotheses on regression coefficients

[bonsxanco]

On Monday, September 8, 2014 6:46 PM, Greg Snow 538...@gmail.com wrote:

 [very good suggestions]



Thank you Greg for dedicating some time to my problem and giving
advice on how I can tackle the issue. It is very appreciated.
Unfortunately I think I will use another program for my original
problem. Anyway, I'll go through all your suggestions, time
permitting.

Best,

Chris

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing general hypotheses on regression coefficients

[Greg Snow]

Others have discussed some of the theoretical approaches (delta
method), but as has also been pointed out, this is a mailing list
about R, not theory, so here are some approaches to your question from
the approach of those of us who like programming R more than
remembering theory.

I assume that one reason you may be interested in B2/B1 is that you
want the confidence interval on the quantity, not just the test of
whether it is 0 (that test being equivalent to B2=0 unless B1 is
exactly equal to 0).  So I will focus more on confidence intervals
(which you can use as tests by seeing if the null value is in the
interval/region or not).

Approach 1, simulation:

If all the assumptions hold for the linear regression, then the
parameter estimates are considered to by multivariate normal.  You can
get the covariance matrix for this normal using the vcov function on
the summary of your fitted object.  Now you can use the mvrnorm
function with the estimated means and covariance to generate a bunch
of observations from this multivariate normal and compute B2/B1 or
some combination of B2/B1 and B4/B3 for each observation.  These
values represent the distribution of interest and you can calculate a
confidence interval by finding the quantiles of the values (0.025 and
0.975 for 95%) or finding the HPD interval (minimum width interval),
the emp.hpd function in the TeachingDemos package is one way to do
this.  For your second hypothesis you could look at B2/B1 - B4/B3 = 0
or (B2/B1) / (B4/B3) = 1, or create a joint confidence region on the 2
ratios and see if the x=y line intersects that region.

Approach 2, bootstrap:

Bootstrap the whole process, fit the regression model then find the
ratio of the estimates.  Find the bootstrap confidence interval of the
ratio(s), follow above advice.

Approach 3, Bayes:

Fit a Bayesian regression model and look at the posterior distribution
of the ratio(s) of interest, calculate the credible interval/region
(the steps will be similar to the previous approaches).

Approach 4, simulate from the null:

Fit your regression model under then null hypothesis of interest being
true (for a more complicated null, your second, you may need to use
optimization or quadratic programming to allow some values to vary,
but have others dependent on those, then find the least squares
solution).  Now simulate data based on that model, fit the full
regression to the simulated data sets and compare the parameter
estimates (or ratios thereof) to the parameter estimates from the
original data.


You could try any of these approaches for hypotheses where traditional
linear hypotheses work and compare the results from the traditional
approach to the above approaches to see how they compare (and how many
iterations/samples you will need).

On Fri, Sep 5, 2014 at 8:17 PM, Chris bonsxa...@yahoo.com wrote:
 Hi.

 Say I have a model like

 y = a + B1*x1 + B2*x2 + B3*x3 + B4*x4 + e

 and I want to test

 H0: B2/B1 = 0

 or

 H0: B2/B1=B4/B3

 (whatever H1). How can I proceed?

 I now about car::linearHypothesis, but I can't figure out a way to do the
 tests above.

 Any hint?

 Thanks.

 C

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



-- 
Gregory (Greg) L. Snow Ph.D.
538...@gmail.com

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing general hypotheses on regression coefficients

[peter dalgaard]

On 06 Sep 2014, at 12:24 , bonsxanco bonsxa...@yahoo.com wrote:

 
 1) 8th grade algebra tells me B2/B1 == 0 == B2 =0;
 
 EViews (econometrics program) doesn't have the same opinion:
 
 Wald test on my real model (edited):
 
 * H0: B3/B2 = 0 - F-stat = 37.82497 
 * H0: B3 = 0- F-stat = 16.31689 

And when the econometrics program contradicts what you learned in 8th grade, 
surely the latter is wrong and the former is right, because it is done by a 
computer and computers cannot be wrong? ;-)

Probably what this shows most of all is a weakness of the Wald test approach: 
The s.e. of (b3hat/b2hat) will likely differ from s.e.(b3hat)/b2hat and hence 
the test statistics will differ even though they really test the same 
hypothesis. Actually, there are two generic weaknesses: (a) the somewhat 
arbitrary choice of test statistic and (b) the fact that the s.e. is not 
calculated at the null value of the parameter, but at the estimate.

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd@cbs.dk  Priv: pda...@gmail.com

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing general hypotheses on regression coefficients

[Scott Kostyshak]

Hi Chris,

 On Fri, Sep 5, 2014 at 7:17 PM, Chris bonsxa...@yahoo.com wrote:
 Hi.

 Say I have a model like

 y = a + B1*x1 + B2*x2 + B3*x3 + B4*x4 + e

 and I want to test

 H0: B2/B1 = 0

As noted by Bert, think about this.

 or

 H0: B2/B1=B4/B3

 (whatever H1). How can I proceed?

 I now about car::linearHypothesis, but I can't figure out a way to do the
 tests above.

 Any hint?

Take a look at car::deltaMethod. I suggest you study the theory of the
delta method. If you happen to have taken a graduate
statistics/econometrics class it should not be difficult and can
provide some insights. If not, at least consider that the delta method
can lead to misleading estimates (biased standard errors) in many
cases for finite samples. You might want to run some simulations to
get a feel for it.

Best,

Scott


--
Scott Kostyshak
Economics PhD Candidate
Princeton University

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing general hypotheses on regression coefficients

[bonsxanco]

Hi.

First of all, thanks to all who have replied.

 1) 8th grade algebra tells me B2/B1 == 0 == B2 =0;

EViews (econometrics program) doesn't have the same opinion:

Wald test on my real model (edited):

* H0: B3/B2 = 0 - F-stat = 37.82497 
* H0: B3 = 0- F-stat = 16.31689 

 2) I suspect you would need to provide more context for the other


The context is this: I'm estimating a model which is:

d(y) = a + B1*y(-1) + B2*X_p(-1) + B3*X_n(-1) + other + error

where X_p and X_n are partial sum decompositions of positive and negative 
shocks:

X_p(t) = X_p(t-1) + (d(X_p(t))0)*d(X_p(t)) ; X_p(0)=0
X_n(t) = X_n(t-1) + (d(X_n(t))0)*d(X_n(t)) ; X_p(0)=0

I think this is enough, but I can provide the full references.

Now, back to the problem: testing B2/B1=0 tells me about that the long term 
effect, while testing for B2/B1=B3/B1 tells me that about the equality of long 
term effects to negative and positive shocks.

 car::deltaMethod

I just gave a quick look and searched about delta method, but I can't see how 
it would help in testing the restrictions above. I'll read more about it, 
though, as it seems interesting, thanks for the pointer.

(Sorry if this e-mail goes out of context, but the first time I sent it through 
gmane, as I wasn't subscribed.)

Chris

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing general hypotheses on regression coefficients

[bonsxanco]

Scott said:


 car::deltaMethod

I said:

 I just gave a quick look and searched about delta method, but I can't
 see how it would help in testing the restrictions above. 

Actually it seems that it should be the way to go: I just noticed under the 
EViews Wald test window the message Delta method computed using analytic 
derivatives.


Anyway, I wonder if there's some easier way to do it.

Best,

Chris

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing general hypotheses on regression coefficients

[Chris]

Hi.

Say I have a model like

y = a + B1*x1 + B2*x2 + B3*x3 + B4*x4 + e

and I want to test

H0: B2/B1 = 0

or

H0: B2/B1=B4/B3

(whatever H1). How can I proceed?

I now about car::linearHypothesis, but I can't figure out a way to do the 
tests above.

Any hint?

Thanks.

C

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing general hypotheses on regression coefficients

[Søren Højsgaard]

AFAICS you are not testing a linear hypothesis (which is of the form Lb=b0 
where L is a matrix and b=(a,B1,B2,B3,B3) is the parameter vector).

If, for simplicity, your model is E(y) = a + bx then -a/b is the x-value for 
which y is zero.

When you turn to estimates then u = -a/b is the ratio of two (typically 
correlated) normal variables and such a ratio is *not* normal. (Just think of 
the Cauchy distribution.)

One approach is to calculate the approximate variance of u and then construct a 
Wald test or similar while hoping for the best. Alternatively one could perhaps 
try with a parametric bootstrap test. 

Just ideas. Good luck.
Søren




-Original Message-
From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On 
Behalf Of Chris
Sent: 6. september 2014 04:17
To: r-h...@stat.math.ethz.ch
Subject: [R] Testing general hypotheses on regression coefficients

Hi.

Say I have a model like

y = a + B1*x1 + B2*x2 + B3*x3 + B4*x4 + e

and I want to test

H0: B2/B1 = 0

or

H0: B2/B1=B4/B3

(whatever H1). How can I proceed?

I now about car::linearHypothesis, but I can't figure out a way to do the tests 
above.

Any hint?

Thanks.

C

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing general hypotheses on regression coefficients

[Bert Gunter]

Well:

1) 8th grade algebra tells me B2/B1 == 0 == B2 =0;

2) I suspect you would need to provide more context for the other, as
you may be going about this entirely incorrectly (have you consulted a
local statistician?):  your nonlinear hypothesis probably can be made
linear under the right parametrization, but context might suggest
something entirely different than the approach that motivated your
query.

3) But forget all that! -- this is a list about the R language, not
statistics -- which seems to be the essence of your query --  although
I grant that the intersection is nonempty. But for statistics help,
you should try a statistics list like stats.stackexchange.com instead.

Cheers,
Bert

Bert Gunter
Genentech Nonclinical Biostatistics
(650) 467-7374

Data is not information. Information is not knowledge. And knowledge
is certainly not wisdom.
Clifford Stoll




On Fri, Sep 5, 2014 at 7:17 PM, Chris bonsxa...@yahoo.com wrote:
 Hi.

 Say I have a model like

 y = a + B1*x1 + B2*x2 + B3*x3 + B4*x4 + e

 and I want to test

 H0: B2/B1 = 0

 or

 H0: B2/B1=B4/B3

 (whatever H1). How can I proceed?

 I now about car::linearHypothesis, but I can't figure out a way to do the
 tests above.

 Any hint?

 Thanks.

 C

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] testing and comparing transformations to get a gaussian distribution

[Diederick Stoffers]

Hi guys,

I distinctly remember having used an R toolbox that compared different 
transformation with regard to normality stats in the past, can’t find anything 
on google. Does anybody have a clue?

Thanks,

Diederick
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing and comparing transformations to get a gaussian distribution

[Greg Snow]

There is the boxcox function in the MASS package that will look at the
Box Cox family of transformations.

On Mon, Jun 2, 2014 at 9:15 AM, Diederick Stoffers d.stoff...@gmail.com wrote:
 Hi guys,

 I distinctly remember having used an R toolbox that compared different 
 transformation with regard to normality stats in the past, can’t find 
 anything on google. Does anybody have a clue?

 Thanks,

 Diederick
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



-- 
Gregory (Greg) L. Snow Ph.D.
538...@gmail.com

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing parallel regression assumption

[Rui Barradas]

Hello,

Take a look at

http://stats.stackexchange.com/questions/58772/brant-test-in-r

Hope this helps,

Rui Barradas

Em 19-05-2014 01:40, caoweina escreveu:

Dear Rose :
 I saw your questions about the R function that performs brant test. Have 
you worked out ?  Please give some advice. I need to do  the brant test in R . 
Thank you very much!
sincerely!

Anna
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing parallel regression assumption

[caoweina]

Dear Rose :
I saw your questions about the R function that performs brant test. Have 
you worked out ?  Please give some advice. I need to do  the brant test in R . 
Thank you very much!
   sincerely!

   Anna
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing correlation of equation in a SUR model fitted by systemfit

[Arne Henningsen]

Dear Paul

On 15 April 2014 19:23, Paul Smith phh...@gmail.com wrote:
 How to test whether the correlation in the matrix of correlation of a
 two-equations SUR model fitted by package systemfit are significant?

You can use a likelihood-ratio test to compare the SUR model with the
corresponding OLS model. The only difference is that the OLS model
assumes that all off-diagonal elements of the covariance matrix of the
residuals of the different equations are zero. An example:

library( systemfit )

# load data set
data( Kmenta )

# specify system of equations
eqDemand - consump ~ price + income
eqSupply - consump ~ price + farmPrice + trend
system - list( demand = eqDemand, supply = eqSupply )

# estimate OLS model
fitols - systemfit( system, data=Kmenta )

# estimate SUR model
fitsur - systemfit( system, SUR, data = Kmenta )

# LR test: OLS vs. SUR
lrtest( fitols, fitsur )

# estimate iterated SUR model
fititsur - systemfit( system, SUR, data = Kmenta, maxit = 100 )

# LR test: OLS vs. SUR
lrtest( fitols, fititsur )


If you have further questions regarding the systemfit package, you can
also use the help forum at systemfit's R-Forge site:

https://r-forge.r-project.org/projects/systemfit/

... and please do not forget to cite systemfit in your publications
(see output of the R command 'citation(systemfit)').

Best regards,
Arne

-- 
Arne Henningsen
http://www.arne-henningsen.name

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing correlation of equation in a SUR model fitted by systemfit

[Paul Smith]

Dear All,

How to test whether the correlation in the matrix of correlation of a
two-equations SUR model fitted by package systemfit are significant?

Thanks in advance,

Paul

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing simple slopes for cross-level interactions

[Nastassia J. Hajal]

Hello,

I would like to probe a significant 2-way, cross-level interaction effect
from a linear mixed effects model that I ran using nlme.

My model is as follows:

mlmmodel - lme(fixed = RegDiseng ~ Happy + TraitHAPPYmean +
Happy*TraitHAPPYmean,
random = ~ Happy | ID, data = data, na.action=na.omit)

DV=RegDiseng
Level 1 variable = Happy
Level 2 variable = TraitHAPPYMean

I'd like to plot and test the significance of simple slopes a la Preacher
(i.e. http://www.quantpsy.org/interact/hlm2.htm) . I was wondering if
anyone knows a simple way to do this directly in R, so that I do not have
to extract the estimates from my R output, enter them into the website, and
then re-enter that code into R.

Many thanks!

Nastassia

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] testing xts values in if command?

[ce]

Dear all ,

xts objects give error in if command :
Error in if  :
  missing value where TRUE/FALSE needed

 library(quantmod)
 getSymbols(SPY)

   SPY[2007-01-03]$SPY.Adjusted  SPY[2007-01-04]$SPY.Adjusted
 [,1]

If I use as.numeric function it works :

 SPY[2007-01-03]$SPY.Adjusted  as.numeric(SPY[2007-01-04]$SPY.Adjusted)
   SPY.Adjusted
2007-01-03FALSE

 as.numeric(SPY[2007-01-03]$SPY.Adjusted)  
 as.numeric(SPY[2007-01-04]$SPY.Adjusted)
[1] FALSE


Is this the expected behavior ? 
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing xts values in if command?

[Joshua Ulrich]

On Wed, Jan 29, 2014 at 1:25 PM, ce zadi...@excite.com wrote:

 Dear all ,

 xts objects give error in if command :
 Error in if  :
   missing value where TRUE/FALSE needed

 library(quantmod)
 getSymbols(SPY)

   SPY[2007-01-03]$SPY.Adjusted  SPY[2007-01-04]$SPY.Adjusted
  [,1]

 If I use as.numeric function it works :

 SPY[2007-01-03]$SPY.Adjusted  as.numeric(SPY[2007-01-04]$SPY.Adjusted)
SPY.Adjusted
 2007-01-03FALSE

 as.numeric(SPY[2007-01-03]$SPY.Adjusted)  
 as.numeric(SPY[2007-01-04]$SPY.Adjusted)
 [1] FALSE


 Is this the expected behavior ?

Yes, see: http://stackoverflow.com/q/7097437/271616
--
Joshua Ulrich  |  about.me/joshuaulrich
FOSS Trading  |  www.fosstrading.com

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing if xts date exists ?

[Joshua Ulrich]

You can use the which.i argument to [.xts:

 is.null(SPY[2009-01-18,which.i=TRUE])
[1] TRUE

Best,
--
Joshua Ulrich  |  about.me/joshuaulrich
FOSS Trading  |  www.fosstrading.com


On Sat, Jan 25, 2014 at 9:27 AM, ce zadi...@excite.com wrote:

 Dear all


 How to test if xts date exists ? is.null doesn't work.  SPY[2009-01-18]
 doesn't exist but I can't catch it in my script.

 library(quantmod)

 getSymbols(SPY)

  SPY[2009-01-16]
SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted
 2009-01-1685.8685.99   83.05 85.06  39923720076.58



  SPY[2009-01-18]
  SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted


  is.null(SPY[2009-01-18])
 [1] FALSE

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] testing if xts date exists ?

[ce]

Dear all


How to test if xts date exists ? is.null doesn't work.  SPY[2009-01-18] 
doesn't exist but I can't catch it in my script.

library(quantmod)

getSymbols(SPY)

 SPY[2009-01-16]
   SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted
2009-01-1685.8685.99   83.05 85.06  39923720076.58



 SPY[2009-01-18]
 SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted


 is.null(SPY[2009-01-18])
[1] FALSE

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing if xts date exists ?

[arun]

!length(SPY[2009-01-18])
#[1] TRUE

  !length(SPY[2009-01-16])
#[1] FALSE

#or
 !nrow(SPY[2009-01-16])

A.K.


On Saturday, January 25, 2014 10:27 AM, ce zadi...@excite.com wrote:
Dear all


How to test if xts date exists ? is.null doesn't work.  SPY[2009-01-18] 
doesn't exist but I can't catch it in my script.

library(quantmod)

getSymbols(SPY)

 SPY[2009-01-16]
           SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted
2009-01-16    85.86    85.99   83.05     85.06  399237200        76.58



 SPY[2009-01-18]
     SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted


 is.null(SPY[2009-01-18])
[1] FALSE

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] testing for bimodal and for dip in between modes in R

[Simon Zehnder]

Testing for bimodality is rather testing for unimodality. Hartigan and Hartigan 
(1985) presented the Dip-Test which is implemented in the R package DipTest 
with a much better approximation of the test distribution. If the test 
statistic is too high unimodality is rejected. To estimate the dip point you 
could choose among several possibilities: (1) A very easy method is to use the 
kmeans function for a kmeans cluster and use the point in the middle of the 
connecting line between the kmeans cluster centers. (2) You could estimate a 
finite mixture distribution and take the middle of the connecting line of the 
modes.

Best

Simon
 
On 24 Nov 2013, at 20:41, Felix Breden bre...@sfu.ca wrote:

 Hi 
 I have distributions that are typically bimodal (see attached .pdf), and I 
 would like to test for bimodality, and then estimate the point between the 
 two modes, the dip in the distributions. any help would be greatly 
 appreciated.
 thanks
 felix 
 m66.junction.aln.pairwise.histogram.pdf__
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] testing for bimodal and for dip in between modes in R

[Felix Breden]

Hi 
I have distributions that are typically bimodal (see attached .pdf), and I 
would like to test for bimodality, and then estimate the point between the two 
modes, the dip in the distributions. any help would be greatly appreciated.
thanks
felix 

m66.junction.aln.pairwise.histogram.pdf
Description: Adobe PDF document
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing.

[Rolf Turner]

Please ignore.  My apologies for the noise.

cheers,

Rolf Turner

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing custom linear contrasts with Welch correction (anova function)

[Michael Cantinotti]

Dear R users,

I am doing custom contrasts with R (comparison of group means).
Everything works fine, but I would like to test the 3 contrasts with and 
without a Welch correction for unequal variances.

I can replicate SPSS results when equal variances are assumed, but I do 
not manage to test the contrasts when equal variances are not assumed. I 
pasted below the results with R and SPSS (at the bottom of my email). R 
provides F, while SPSS provides t (F = t squared).

Is there a way to apply the Welch correction to custom contrasts with R 
anova function (or another correction for unequal variances, like 
Brown-Forsythe), or do I need to use another package when equal 
variances cannot be assumed ?

Regards,

Michael


==
# R Syntax
==

# c1 : group 1 vs. group 2.
# c2 : group 3 vs. group 4.
# c3 : group 1+2 vs. group 3+4.

c1 - c(1, -1,  0,  0)
c2 - c(0,  0,  1, -1)
c3 - c(1,  1, -1, -1)

modele.global - lm(var ~ cond,
  data=Plan_CR_4_Modifie)

modele.contrastes - lm(var ~ C(cond, c1, 1)
 + C(cond, c2, 1)
 + C(cond, c3, 1),
  data=Plan_CR_4_Modifie)

anova(modele.global)
anova(modele.contrastes)

--
R Results:

  anova(modele.global)
Analysis of Variance Table

Response: var
   Df Sum Sq Mean Sq F valuePr(F)
cond   3 49 16.  7.4973 0.0007824 ***
Residuals 28 61  2.1786
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

  anova(modele.contrastes)
Analysis of Variance Table

Response: var
Df Sum Sq Mean Sq F valuePr(F)
C(cond, c1, 1)  1  1   1.000  0.4590 0.5036442
C(cond, c2, 1)  1 16  16.000  7.3443 0.0113548 *
C(cond, c3, 1)  1 32  32.000 14.6885 0.0006571 ***
Residuals  28 61   2.179
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1




==
* SPSS Syntax - CONTRAST TESTS.
==

ONEWAY
   var BY cond
   /CONTRAST= 1 -1 0 0  /CONTRAST= 0 0 1 -1  /CONTRAST= 1 1 -1 -1
   /STATISTICS DESCRIPTIVES HOMOGENEITY
   /MISSING ANALYSIS
   /POSTHOC = SCHEFFE LSD BONFERRONI ALPHA(.05).


--
SPSS Results - Contrasts:

Assume equal variances

||-|--|--|--|---|
|Contrast|Value of Contrast|Std. Error|t |df|Sig. (2-tailed)|
||-|--|--|--|---|
|1   |-.50 |.738  |-.678 |28 |.504   |
||-|--|--|--|---|
|2   |-2.00|.738  |-2.710|28 |.011   |
||-|--|--|--|---|
|3   |-4.00|1.044 |-3.833|28 |.001   |
|

Does not assume equal variances

||-|--|--|--|---|
|Contrast|Value of Contrast|Std. Error|t |df|Sig. (2-tailed)|
||-|--|--|--|---|
|1   |-.50 |.627  |-.798 |11.603|.441   |
||-|--|--|--|---|
|2   |-2.00|.835 |-2.397|10.155|.037   |
||-|--|--|--|---|
|3   |-4.00|1.044 |-3.833|19.431|.001   |
|


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing for weak exogeneity in a SUR ECM

[Arne Henningsen]

Dear Kathrinchen

It seems to me that your question is about statistics rather than
about R and systemfit. If you find out how the statistical test should
be conducted theoretically, I can probably advise you how to implement
the test in R (systemfit).

Best wishes,
Arne


On 11 July 2013 13:21, Kathrinchen katha.m...@web.de wrote:
 Dear all,

 I have set up a Labour Demand Error Correction Model for some German federal
 states.

 As I expect the labour markets to be correlated I used a Seemingly Unrelated
 Regression using systemfit in R.

 My Model  is:

 d(emp)_it = c + alpha*ln(emp)_i,t-1 + beta_1*ln(gdp)_i,t-1 + +
 beta_2*ln(wage)_i,t-1 + + beta_1*ln(i)_i,t-1 + gamma_1*d(gdp)_it +
 gamma_2*d(wage)_it

 with emp_it being the employment in state i at time t, i stands for the real
 interest rate, ln() is the logarithmed data, while d() stands for the
 difference operator.

 I would like to test now for weak exogeneity and I am not quite sure what
 kind of regression to run.  If I run:
 d(gdp)_it = c + alpha*ln(emp)_i,t-1 + beta_1*ln(gdp)_i,t-1 + +
 beta_2*ln(wage)_i,t-1 + + beta_1*ln(i)_i,t-1 + gamma_1*d(emp)_it +
 gamma_2*d(wage)_it

 with Systemfit, alpha is statistically significant, so I have to reject the
 hypothesis of weak exogeneity...Literature is in my opinion not so clear on
 what to test!

 I use data from an application, they conclude that endogeneity is not a
 problem: they regress the possible endogenous variables on the presumed
 equilibrium relation, a constant and one autoregressive lag - here I am not
 sure, what they mean.

 I would very much appreciate your help!

 Thanks a lot!





 --
 View this message in context: 
 http://r.789695.n4.nabble.com/Testing-for-weak-exogeneity-in-a-SUR-ECM-tp4671321.html
 Sent from the R help mailing list archive at Nabble.com.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



-- 
Arne Henningsen
http://www.arne-henningsen.name

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing of Diagnostic residuals in R

[ntamjo achille]

Hi there,

 I want to ask a question about any function in r that helps test residuals of  
the vector error correction model. I find it on Pfaff(2008) but he tests only 
residual for VAR(vector autoregressive model).

I need to workout Portmanteau test, Normality test and Heteroskedasticty for 
the VECM. Anyway, Pfaff proposes a diagnostic test for the function vec2var. 
But it ends up to test VAR residuals. Is anybody can tell me if it is possible 
to use that function to analyse the diagnostic of vecm residuals?

Thank you

Vec2var codes:
vecm.level - vec2var(vecm1, r = 2)

vecm.norm - normality(vecm.level)

vecm.arch - arch(vecm.level)
vecm.serial - serial(vecm.level)

Diagnostic for VAR
var2c.serial - serial(VAR)
 var2c.arch - arch(VAR)
 var2c.norm - normality(VAR)


I want something like
vecmfemales-ca.jo(b,type=trace,spec=transitory)

vecm.r2-serial.test(vecmfemales)
vecm.norm - normality(vecm.r2) but i receive this message


Error in serial.test(vecmfemale) : 
Please provide an object of class 'varest', generated by 'var()', or an object 
of class 'vec2var' generated by 'vec2var()'.
By reading this message, i can conclude that there is only 2 ways to make 
residuals in R: VAR and vec2var?




Thank you for your answer
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing for weak exogeneity in a SUR ECM

[Kathrinchen]

Dear all,

I have set up a Labour Demand Error Correction Model for some German federal
states.

As I expect the labour markets to be correlated I used a Seemingly Unrelated
Regression using systemfit in R.

My Model  is:

d(emp)_it = c + alpha*ln(emp)_i,t-1 + beta_1*ln(gdp)_i,t-1 + +
beta_2*ln(wage)_i,t-1 + + beta_1*ln(i)_i,t-1 + gamma_1*d(gdp)_it +
gamma_2*d(wage)_it

with emp_it being the employment in state i at time t, i stands for the real
interest rate, ln() is the logarithmed data, while d() stands for the
difference operator.

I would like to test now for weak exogeneity and I am not quite sure what
kind of regression to run.  If I run: 
d(gdp)_it = c + alpha*ln(emp)_i,t-1 + beta_1*ln(gdp)_i,t-1 + +
beta_2*ln(wage)_i,t-1 + + beta_1*ln(i)_i,t-1 + gamma_1*d(emp)_it +
gamma_2*d(wage)_it

with Systemfit, alpha is statistically significant, so I have to reject the
hypothesis of weak exogeneity...Literature is in my opinion not so clear on
what to test!

I use data from an application, they conclude that endogeneity is not a
problem: they regress the possible endogenous variables on the presumed
equilibrium relation, a constant and one autoregressive lag - here I am not
sure, what they mean.

I would very much appreciate your help!

Thanks a lot!





--
View this message in context: 
http://r.789695.n4.nabble.com/Testing-for-weak-exogeneity-in-a-SUR-ECM-tp4671321.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing for significance of overlap in three sets - mantelhaen test?

[Brian Smith]

Hi,

My apologies for the naive question!

I have three overlapping sets and I want to find the probability of finding
a larger/greater intersection for 'A intersect B intersect C' (in the
example below, I want to find the probability of finding more than 135
elements that are common in sets A, B  C). For a two set problem, I guess
I would do a Fisher or chi-square test. Here is what I have attempted so
far:

#

### Prepare a 3 way contingency table:
mytable - array(c(135,116,385,6256,
48,97,274,9555),
  dim = c(2,2,2),
  dimnames = list(
Is_C = c('Yes','No'),
Is_B = c('Yes','No'),
Is_A = c('Yes','No')))

## test
mantelhaen.test(myrabbit, exact = TRUE, alternative = greater)

## end code

Is this the right test (alongwith the current parameters) to determine what
I want or is there a more appropriate test for this?



many thanks!!

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing for significance of overlap in three sets - mantelhaen test?

[Bert Gunter]

As this seems to be a statistics, not an R, question, it is off topic
here. Post on a statistics list like stats.stackexchange.com instead.

-- Bert

On Tue, Mar 12, 2013 at 6:22 AM, Brian Smith bsmith030...@gmail.com wrote:
 Hi,

 My apologies for the naive question!

 I have three overlapping sets and I want to find the probability of finding
 a larger/greater intersection for 'A intersect B intersect C' (in the
 example below, I want to find the probability of finding more than 135
 elements that are common in sets A, B  C). For a two set problem, I guess
 I would do a Fisher or chi-square test. Here is what I have attempted so
 far:

 #

 ### Prepare a 3 way contingency table:
 mytable - array(c(135,116,385,6256,
 48,97,274,9555),
   dim = c(2,2,2),
   dimnames = list(
 Is_C = c('Yes','No'),
 Is_B = c('Yes','No'),
 Is_A = c('Yes','No')))

 ## test
 mantelhaen.test(myrabbit, exact = TRUE, alternative = greater)

 ## end code

 Is this the right test (alongwith the current parameters) to determine what
 I want or is there a more appropriate test for this?



 many thanks!!

 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



-- 

Bert Gunter
Genentech Nonclinical Biostatistics

Internal Contact Info:
Phone: 467-7374
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] testing the multiple regression model

[dada]

Hi
I have 25 samples in my dataset. I have written a multiple regression model
and I would like to test it. 
I would like to train my model on 20 samples and then test it on 5
remaining. However I would like to test the model several times, each time
using different 5 samples out of 25 and check how well the model performs
each time. 
As I am new to R, I am not sure how the script should look like. Could you
please help me with this ? Thank you

 




--
View this message in context: 
http://r.789695.n4.nabble.com/testing-the-multiple-regression-model-tp4657119.html
Sent from the R help mailing list archive at Nabble.com.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing continuous zero-inflated response

[Achim Zeileis]

On Sun, 27 Jan 2013, Kay Cichini wrote:


That said,


wilcox_test(x ~ factor(y), distribution = exact)


or the same with oneway_test, i.e would be ok?


Yep, exactly.

And you could also look at chisq_test(factor(x  0) ~ factor(y), 
distribtuion = approximate()) or something like that. Or of course 
Fisher's exact test.


Best,
Z



2013/1/27 Achim Zeileis achim.zeil...@uibk.ac.at


On Sun, 27 Jan 2013, Kay Cichini wrote:

 Thanks for the reply!


Still, aren't there issues with 2-sample test vs y and excess zeroes
(-many ties), like for Mann-Whitney-U tests?



If you use the (approximate) exact distribution, that is no problem.

The problem with the Wilcoxon/Mann-Whitney test and ties is only that the
simple recursion formula for computing the exact distribution only works
without ties. Thus, it's not the exact distribution that is wrong but only
the standard algorithm for evaluating it.

Best,
Z

 Kind regards,

Kay


2013/1/26 Achim Zeileis achim.zeil...@uibk.ac.at

 On Fri, 25 Jan 2013, Kay Cichini wrote:


 Hello,



I'm searching for a test that applies to a dataset (N=36) with a
continuous zero-inflated dependent variable



In a regression setup, one can use a regression model with a response
censored at zero. survreg() in survival fits such models, tobit() in AER
is
a convenience interface for this special case.

If the effects of a regressor can be different for the probability of a
zero and the mean of the non-zero observations, then a two-part model can
be used. E.g. a probit fit (via glm) plus a truncated regression (via
truncreg in the package of the same name).

However:


 and only one nominal grouping variable with 2 levels (balanced).





In that case I would probably use no regression model but two-sample
permutation tests, e.g. via the coin package.


 In fact there are 4 response variables of this kind which I plan to test


seperately - the amount of zeroes ranges from 75 to 97%..



That means you have between one (!) and nine non-zero observations. In
the
former case, it will be hard to model anything. And even in the latter
case
it will be hard to investigate the probability of zero and the mean of
the
non-zero observations separately.

I would start out with a simple two-way table of (y  0) vs group and
conduct Fisher's exact test.

And then you might try also your favorite two sample test of y vs group,
preferably using the approximate exact distribution.

Hope that helps,
Z

 I searched the web and found several modelling approaches but have the


feeling that they are overly complex for my very simple dataset.

Thanks in advance for any help!
Kay

--

Kay Cichini, MSc Biol

Grubenweg 22, 6071 Aldrans

Tel.: 0650 9359101

E-Mail: kay.cich...@gmail.com

Web: www.theBioBucket.blogspot.co.athttp://www.theBioBucket.**
blogspot.co.at http://www.theBioBucket.blogspot.co.at
http://www.thebiobucket.**blo**gspot.co.at/ http://blogspot.co.at/
http://www.**thebiobucket.blogspot.co.at/http://www.thebiobucket.blogspot.co.at/





http://www.**theBioBucket.**blogspot.co.athttp://theBioBucket.blogspot.co.at
http://www.**theBioBucket.blogspot.co.athttp://www.theBioBucket.blogspot.co.at




 --


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-helphttps://stat.ethz.ch/mailman/**listinfo/r-help
https://stat.**ethz.ch/mailman/listinfo/r-**helphttps://stat.ethz.ch/mailman/listinfo/r-help



PLEASE do read the posting guide http://www.R-project.org/**
posting-guide.html 
http://www.R-project.org/**posting-guide.htmlhttp://www.R-project.org/posting-guide.html




and provide commented, minimal, self-contained, reproducible code.





--

Kay Cichini, MSc Biol

Grubenweg 22, 6071 Aldrans

Tel.: 0650 9359101

E-Mail: kay.cich...@gmail.com

Web: www.theBioBucket.blogspot.co.**athttp://www.theBioBucket.blogspot.co.at
http://www.thebiobucket.**blogspot.co.at/http://www.thebiobucket.blogspot.co.at/

http://www.**theBioBucket.blogspot.co.athttp://www.theBioBucket.blogspot.co.at


--

[[alternative HTML version deleted]]

__**
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/**
posting-guide.html http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.





--

Kay Cichini, MSc Biol

Grubenweg 22, 6071 Aldrans

Tel.: 0650 9359101

E-Mail: kay.cich...@gmail.com

Web: 
www.theBioBucket.blogspot.co.athttp://www.thebiobucket.blogspot.co.at/http://www.theBioBucket.blogspot.co.at
--

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide 

Re: [R] Testing continuous zero-inflated response

[Kay Cichini]

Many thanks - this was very helpful!

Regards, Kay
Am 28.01.2013 13:19 schrieb Achim Zeileis achim.zeil...@uibk.ac.at:

 On Sun, 27 Jan 2013, Kay Cichini wrote:

  That said,

  wilcox_test(x ~ factor(y), distribution = exact)


 or the same with oneway_test, i.e would be ok?


 Yep, exactly.

 And you could also look at chisq_test(factor(x  0) ~ factor(y),
 distribtuion = approximate()) or something like that. Or of course
 Fisher's exact test.

 Best,
 Z


 2013/1/27 Achim Zeileis achim.zeil...@uibk.ac.at

  On Sun, 27 Jan 2013, Kay Cichini wrote:

  Thanks for the reply!


 Still, aren't there issues with 2-sample test vs y and excess zeroes
 (-many ties), like for Mann-Whitney-U tests?


 If you use the (approximate) exact distribution, that is no problem.

 The problem with the Wilcoxon/Mann-Whitney test and ties is only that the
 simple recursion formula for computing the exact distribution only works
 without ties. Thus, it's not the exact distribution that is wrong but
 only
 the standard algorithm for evaluating it.

 Best,
 Z

  Kind regards,

 Kay


 2013/1/26 Achim Zeileis achim.zeil...@uibk.ac.at

  On Fri, 25 Jan 2013, Kay Cichini wrote:


  Hello,


 I'm searching for a test that applies to a dataset (N=36) with a
 continuous zero-inflated dependent variable


  In a regression setup, one can use a regression model with a response
 censored at zero. survreg() in survival fits such models, tobit() in
 AER
 is
 a convenience interface for this special case.

 If the effects of a regressor can be different for the probability of a
 zero and the mean of the non-zero observations, then a two-part model
 can
 be used. E.g. a probit fit (via glm) plus a truncated regression (via
 truncreg in the package of the same name).

 However:


  and only one nominal grouping variable with 2 levels (balanced).



  In that case I would probably use no regression model but two-sample
 permutation tests, e.g. via the coin package.


  In fact there are 4 response variables of this kind which I plan to
 test

  seperately - the amount of zeroes ranges from 75 to 97%..


  That means you have between one (!) and nine non-zero observations.
 In
 the
 former case, it will be hard to model anything. And even in the latter
 case
 it will be hard to investigate the probability of zero and the mean of
 the
 non-zero observations separately.

 I would start out with a simple two-way table of (y  0) vs group and
 conduct Fisher's exact test.

 And then you might try also your favorite two sample test of y vs
 group,
 preferably using the approximate exact distribution.

 Hope that helps,
 Z

  I searched the web and found several modelling approaches but have the

  feeling that they are overly complex for my very simple dataset.

 Thanks in advance for any help!
 Kay

 --

 Kay Cichini, MSc Biol

 Grubenweg 22, 6071 Aldrans

 Tel.: 0650 9359101

 E-Mail: kay.cich...@gmail.com

 Web: www.theBioBucket.blogspot.co.**athttp://www.theBioBucket.**
 **
 blogspot.co.at 
 http://www.theBioBucket.**blogspot.co.athttp://www.theBioBucket.blogspot.co.at
 
 http://www.thebiobucket.blo**gspot.co.at/ 
 http://blogspot.co.at/
 http://www.**thebiobucket.**blogspot.co.at/http://thebiobucket.blogspot.co.at/
 http://www.**thebiobucket.blogspot.co.at/http://www.thebiobucket.blogspot.co.at/
 



  http://www.**theBioBucket.**b**logspot.co.athttp://blogspot.co.at
 http://**theBioBucket.blogspot.co.athttp://theBioBucket.blogspot.co.at
 
 http://www.**theBioBucket.**blogspot.co.athttp://theBioBucket.blogspot.co.at
 http://www.**theBioBucket.blogspot.co.athttp://www.theBioBucket.blogspot.co.at
 



  --


 [[alternative HTML version deleted]]

 __**
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help
 https://**stat.ethz.ch/mailman/listinfo/r-helphttps://stat.ethz.ch/mailman/**listinfo/r-help
 
 https://stat.**ethz.ch/**mailman/listinfo/r-**helphttp://ethz.ch/mailman/listinfo/r-**help
 http**s://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help
 


  PLEASE do read the posting guide http://www.R-project.org/**
 posting-guide.html 
 http://www.R-project.org/posting-guide.htmlhttp://www.R-project.org/**posting-guide.html
 http://www.**R-project.org/posting-guide.**htmlhttp://www.R-project.org/posting-guide.html
 



 and provide commented, minimal, self-contained, reproducible code.




 --

 Kay Cichini, MSc Biol

 Grubenweg 22, 6071 Aldrans

 Tel.: 0650 9359101

 E-Mail: kay.cich...@gmail.com

 Web: www.theBioBucket.blogspot.co.athttp://www.theBioBucket.**
 blogspot.co.at http://www.theBioBucket.blogspot.co.at
 http://www.thebiobucket.**blo**gspot.co.at/ http://blogspot.co.at/
 http://www.**thebiobucket.blogspot.co.at/http://www.thebiobucket.blogspot.co.at/
 

 http://www.**theBioBucket.**blogspot.co.athttp://theBioBucket.blogspot.co.at
 

Re: [R] Testing continuous zero-inflated response

[Kay Cichini]

Thanks for the reply!

Still, aren't there issues with 2-sample test vs y and excess zeroes
(-many ties), like for Mann-Whitney-U tests?

Kind regards,
Kay


2013/1/26 Achim Zeileis achim.zeil...@uibk.ac.at

 On Fri, 25 Jan 2013, Kay Cichini wrote:

  Hello,

 I'm searching for a test that applies to a dataset (N=36) with a
 continuous zero-inflated dependent variable


 In a regression setup, one can use a regression model with a response
 censored at zero. survreg() in survival fits such models, tobit() in AER is
 a convenience interface for this special case.

 If the effects of a regressor can be different for the probability of a
 zero and the mean of the non-zero observations, then a two-part model can
 be used. E.g. a probit fit (via glm) plus a truncated regression (via
 truncreg in the package of the same name).

 However:


  and only one nominal grouping variable with 2 levels (balanced).


 In that case I would probably use no regression model but two-sample
 permutation tests, e.g. via the coin package.


  In fact there are 4 response variables of this kind which I plan to test
 seperately - the amount of zeroes ranges from 75 to 97%..


 That means you have between one (!) and nine non-zero observations. In the
 former case, it will be hard to model anything. And even in the latter case
 it will be hard to investigate the probability of zero and the mean of the
 non-zero observations separately.

 I would start out with a simple two-way table of (y  0) vs group and
 conduct Fisher's exact test.

 And then you might try also your favorite two sample test of y vs group,
 preferably using the approximate exact distribution.

 Hope that helps,
 Z

  I searched the web and found several modelling approaches but have the
 feeling that they are overly complex for my very simple dataset.

 Thanks in advance for any help!
 Kay

 --

 Kay Cichini, MSc Biol

 Grubenweg 22, 6071 Aldrans

 Tel.: 0650 9359101

 E-Mail: kay.cich...@gmail.com

 Web: 
 www.theBioBucket.blogspot.co.**athttp://www.theBioBucket.blogspot.co.at
 http://www.thebiobucket.**blogspot.co.at/http://www.thebiobucket.blogspot.co.at/
 http://www.**theBioBucket.blogspot.co.athttp://www.theBioBucket.blogspot.co.at
 
 --

 [[alternative HTML version deleted]]

 __**
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/**
 posting-guide.html http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 

Kay Cichini, MSc Biol

Grubenweg 22, 6071 Aldrans

Tel.: 0650 9359101

E-Mail: kay.cich...@gmail.com

Web: 
www.theBioBucket.blogspot.co.athttp://www.thebiobucket.blogspot.co.at/http://www.theBioBucket.blogspot.co.at
--

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing continuous zero-inflated response

[Achim Zeileis]

On Sun, 27 Jan 2013, Kay Cichini wrote:


Thanks for the reply!

Still, aren't there issues with 2-sample test vs y and excess zeroes
(-many ties), like for Mann-Whitney-U tests?


If you use the (approximate) exact distribution, that is no problem.

The problem with the Wilcoxon/Mann-Whitney test and ties is only that the 
simple recursion formula for computing the exact distribution only works 
without ties. Thus, it's not the exact distribution that is wrong but only 
the standard algorithm for evaluating it.


Best,
Z


Kind regards,
Kay


2013/1/26 Achim Zeileis achim.zeil...@uibk.ac.at


On Fri, 25 Jan 2013, Kay Cichini wrote:

 Hello,


I'm searching for a test that applies to a dataset (N=36) with a
continuous zero-inflated dependent variable



In a regression setup, one can use a regression model with a response
censored at zero. survreg() in survival fits such models, tobit() in AER is
a convenience interface for this special case.

If the effects of a regressor can be different for the probability of a
zero and the mean of the non-zero observations, then a two-part model can
be used. E.g. a probit fit (via glm) plus a truncated regression (via
truncreg in the package of the same name).

However:


 and only one nominal grouping variable with 2 levels (balanced).




In that case I would probably use no regression model but two-sample
permutation tests, e.g. via the coin package.


 In fact there are 4 response variables of this kind which I plan to test

seperately - the amount of zeroes ranges from 75 to 97%..



That means you have between one (!) and nine non-zero observations. In the
former case, it will be hard to model anything. And even in the latter case
it will be hard to investigate the probability of zero and the mean of the
non-zero observations separately.

I would start out with a simple two-way table of (y  0) vs group and
conduct Fisher's exact test.

And then you might try also your favorite two sample test of y vs group,
preferably using the approximate exact distribution.

Hope that helps,
Z

 I searched the web and found several modelling approaches but have the

feeling that they are overly complex for my very simple dataset.

Thanks in advance for any help!
Kay

--

Kay Cichini, MSc Biol

Grubenweg 22, 6071 Aldrans

Tel.: 0650 9359101

E-Mail: kay.cich...@gmail.com

Web: www.theBioBucket.blogspot.co.**athttp://www.theBioBucket.blogspot.co.at
http://www.thebiobucket.**blogspot.co.at/http://www.thebiobucket.blogspot.co.at/

http://www.**theBioBucket.blogspot.co.athttp://www.theBioBucket.blogspot.co.at


--

[[alternative HTML version deleted]]

__**
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/**
posting-guide.html http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.





--

Kay Cichini, MSc Biol

Grubenweg 22, 6071 Aldrans

Tel.: 0650 9359101

E-Mail: kay.cich...@gmail.com

Web: 
www.theBioBucket.blogspot.co.athttp://www.thebiobucket.blogspot.co.at/http://www.theBioBucket.blogspot.co.at
--

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing continuous zero-inflated response

[Kay Cichini]

That said,

 wilcox_test(x ~ factor(y), distribution = exact)

or the same with oneway_test, i.e would be ok?


2013/1/27 Achim Zeileis achim.zeil...@uibk.ac.at

 On Sun, 27 Jan 2013, Kay Cichini wrote:

  Thanks for the reply!

 Still, aren't there issues with 2-sample test vs y and excess zeroes
 (-many ties), like for Mann-Whitney-U tests?


 If you use the (approximate) exact distribution, that is no problem.

 The problem with the Wilcoxon/Mann-Whitney test and ties is only that the
 simple recursion formula for computing the exact distribution only works
 without ties. Thus, it's not the exact distribution that is wrong but only
 the standard algorithm for evaluating it.

 Best,
 Z

  Kind regards,
 Kay


 2013/1/26 Achim Zeileis achim.zeil...@uibk.ac.at

  On Fri, 25 Jan 2013, Kay Cichini wrote:

  Hello,


 I'm searching for a test that applies to a dataset (N=36) with a
 continuous zero-inflated dependent variable


 In a regression setup, one can use a regression model with a response
 censored at zero. survreg() in survival fits such models, tobit() in AER
 is
 a convenience interface for this special case.

 If the effects of a regressor can be different for the probability of a
 zero and the mean of the non-zero observations, then a two-part model can
 be used. E.g. a probit fit (via glm) plus a truncated regression (via
 truncreg in the package of the same name).

 However:


  and only one nominal grouping variable with 2 levels (balanced).



 In that case I would probably use no regression model but two-sample
 permutation tests, e.g. via the coin package.


  In fact there are 4 response variables of this kind which I plan to test

 seperately - the amount of zeroes ranges from 75 to 97%..


 That means you have between one (!) and nine non-zero observations. In
 the
 former case, it will be hard to model anything. And even in the latter
 case
 it will be hard to investigate the probability of zero and the mean of
 the
 non-zero observations separately.

 I would start out with a simple two-way table of (y  0) vs group and
 conduct Fisher's exact test.

 And then you might try also your favorite two sample test of y vs group,
 preferably using the approximate exact distribution.

 Hope that helps,
 Z

  I searched the web and found several modelling approaches but have the

 feeling that they are overly complex for my very simple dataset.

 Thanks in advance for any help!
 Kay

 --

 Kay Cichini, MSc Biol

 Grubenweg 22, 6071 Aldrans

 Tel.: 0650 9359101

 E-Mail: kay.cich...@gmail.com

 Web: www.theBioBucket.blogspot.co.athttp://www.theBioBucket.**
 blogspot.co.at http://www.theBioBucket.blogspot.co.at
 http://www.thebiobucket.**blo**gspot.co.at/ http://blogspot.co.at/
 http://www.**thebiobucket.blogspot.co.at/http://www.thebiobucket.blogspot.co.at/
 

 http://www.**theBioBucket.**blogspot.co.athttp://theBioBucket.blogspot.co.at
 http://www.**theBioBucket.blogspot.co.athttp://www.theBioBucket.blogspot.co.at
 

  --

 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-helphttps://stat.ethz.ch/mailman/**listinfo/r-help
 https://stat.**ethz.ch/mailman/listinfo/r-**helphttps://stat.ethz.ch/mailman/listinfo/r-help
 
 PLEASE do read the posting guide http://www.R-project.org/**
 posting-guide.html 
 http://www.R-project.org/**posting-guide.htmlhttp://www.R-project.org/posting-guide.html
 

 and provide commented, minimal, self-contained, reproducible code.




 --

 Kay Cichini, MSc Biol

 Grubenweg 22, 6071 Aldrans

 Tel.: 0650 9359101

 E-Mail: kay.cich...@gmail.com

 Web: 
 www.theBioBucket.blogspot.co.**athttp://www.theBioBucket.blogspot.co.at
 http://www.thebiobucket.**blogspot.co.at/http://www.thebiobucket.blogspot.co.at/
 http://www.**theBioBucket.blogspot.co.athttp://www.theBioBucket.blogspot.co.at
 
 --

 [[alternative HTML version deleted]]

 __**
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/**
 posting-guide.html http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 

Kay Cichini, MSc Biol

Grubenweg 22, 6071 Aldrans

Tel.: 0650 9359101

E-Mail: kay.cich...@gmail.com

Web: 
www.theBioBucket.blogspot.co.athttp://www.thebiobucket.blogspot.co.at/http://www.theBioBucket.blogspot.co.at
--

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing continuous zero-inflated response

[Kay Cichini]

Hello,

I'm searching for a test that applies to a dataset (N=36) with a continuous
zero-inflated dependent variable and only one nominal grouping variable
with 2 levels (balanced).

In fact there are 4 response variables of this kind which I plan to test
seperately - the amount of zeroes ranges from 75 to 97%..

I searched the web and found several modelling approaches but have the
feeling that they are overly complex for my very simple dataset.

Thanks in advance for any help!
Kay

-- 

Kay Cichini, MSc Biol

Grubenweg 22, 6071 Aldrans

Tel.: 0650 9359101

E-Mail: kay.cich...@gmail.com

Web: 
www.theBioBucket.blogspot.co.athttp://www.thebiobucket.blogspot.co.at/http://www.theBioBucket.blogspot.co.at
--

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Testing continuous zero-inflated response

[Achim Zeileis]

On Fri, 25 Jan 2013, Kay Cichini wrote:


Hello,

I'm searching for a test that applies to a dataset (N=36) with a 
continuous zero-inflated dependent variable


In a regression setup, one can use a regression model with a response 
censored at zero. survreg() in survival fits such models, tobit() in AER 
is a convenience interface for this special case.


If the effects of a regressor can be different for the probability of a 
zero and the mean of the non-zero observations, then a two-part model can 
be used. E.g. a probit fit (via glm) plus a truncated regression (via 
truncreg in the package of the same name).


However:


and only one nominal grouping variable with 2 levels (balanced).


In that case I would probably use no regression model but two-sample 
permutation tests, e.g. via the coin package.



In fact there are 4 response variables of this kind which I plan to test
seperately - the amount of zeroes ranges from 75 to 97%..


That means you have between one (!) and nine non-zero observations. In the 
former case, it will be hard to model anything. And even in the latter 
case it will be hard to investigate the probability of zero and the 
mean of the non-zero observations separately.


I would start out with a simple two-way table of (y  0) vs group and 
conduct Fisher's exact test.


And then you might try also your favorite two sample test of y vs group, 
preferably using the approximate exact distribution.


Hope that helps,
Z


I searched the web and found several modelling approaches but have the
feeling that they are overly complex for my very simple dataset.

Thanks in advance for any help!
Kay

--

Kay Cichini, MSc Biol

Grubenweg 22, 6071 Aldrans

Tel.: 0650 9359101

E-Mail: kay.cich...@gmail.com

Web: 
www.theBioBucket.blogspot.co.athttp://www.thebiobucket.blogspot.co.at/http://www.theBioBucket.blogspot.co.at
--

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing proportional odds assumption in R

[Thomas Yee]

Hi,

M1 and M2 are extreme in that all or none of the variables have
parallel lines on the logit scale.  One can try fitting a partial
POM, which remains fraught (but not as much as M2) because if
the lines intersect for a particular variable where the data lie
then there will be numerical problems.

For example, try something like

M3 - vglm(y ~ x1 + x2 + x3, data = data,
  family = cumulative(parallel = FALSE ~ x3 - 1))

which will only allow nonparallelism wrt the x3 variable.

Another idea is to try transforming each x variable if
it crashes... that might help.

BTW, can use lrtest(), e.g., lrtest(M3, M2)

cheers

Thomas

I want to test whether the proportional odds assumption for an ordered
regression is met.   
 
The UCLA website points out that there is no mathematical way to test the
proportional odds assumption 
(http://www.ats.ucla.edu/stat//R/dae/ologit.htm),
and use graphical inspection (We were unable to locate a facility in 
R to   
perform any of the tests commonly used to test the parallel 
slopes   
assumption.).

  

However, I found a pdf by Agresti suggesting a method using the 
vglm 
function, the pdf is called Examples of Using R for Modeling 
Ordinal
Data.


 M1 - vglm(y ~ x1 + x2 + x3, data=data, 
family=cumulative(parallel=TRUE))

 M2 - vglm(y ~ x1 + x2 + x3, data=data, family=cumulative, maxit=100)

pchisq(deviance(Mo.1)-deviance(Mo.2), 
df=df.residual(Mo.1)-df.residual(Mo.2),

lower.tail=FALSE)

If the test is significant, the proportional odds assumption is (might be)
violated.

However, running this procedure in my dataset with 5 predictors (3
dichotomous, 2 z-standardized metric) and an ordinal dependent variable
(0,1,2,3) in a sample of N=2500 leads to various problems, maybe you can
help me out how to solve these.

Warning messages:
(1) Error in dotC(name = tapplymat1, mat = as.double(mat),
as.integer(nr),  :
  NA/NaN/Inf in foreign function call (arg 1
(2) In matrix.power(wz, M = M, power = 0.5, fast = TRUE) :
  Some weight matrices have negative eigenvalues. They
will be assigned NAs
(3)
In Deviance.categorical.data.vgam(mu = mu, y = y, w = w, residuals =
residuals,  :
  fitted values close to 0 or 1
(4) In log(prob) : NaNs produced

The last two don't seem to be as critical and the first two, seeing that
models with that errors at least provide a p-value (the first to errors
lead to a p--value of 1).

Thank you

   [[alternative HTML version deleted]]


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing proportional odds assumption in R

[Eiko Fried]

I want to test whether the proportional odds assumption for an ordered
regression is met.

The UCLA website points out that there is no mathematical way to test the
proportional odds assumption (http://www.ats.ucla.edu/stat//R/dae/ologit.htm),
and use graphical inspection (We were unable to locate a facility in R to
perform any of the tests commonly used to test the parallel slopes
assumption.).

However, I found a pdf by Agresti suggesting a method using the vglm
function, the pdf is called Examples of Using R for Modeling Ordinal
Data.

 M1 - vglm(y ~ x1 + x2 + x3, data=data, family=cumulative(parallel=TRUE))
 M2 - vglm(y ~ x1 + x2 + x3, data=data, family=cumulative, maxit=100)

pchisq(deviance(Mo.1)-deviance(Mo.2), df=df.residual(Mo.1)-df.residual(Mo.2),
lower.tail=FALSE)

If the test is significant, the proportional odds assumption is (might be)
violated.

However, running this procedure in my dataset with 5 predictors (3
dichotomous, 2 z-standardized metric) and an ordinal dependent variable
(0,1,2,3) in a sample of N=2500 leads to various problems, maybe you can
help me out how to solve these.

Warning messages:
(1) Error in dotC(name = tapplymat1, mat = as.double(mat),
as.integer(nr),  :
  NA/NaN/Inf in foreign function call (arg 1
(2) In matrix.power(wz, M = M, power = 0.5, fast = TRUE) :
  Some weight matrices have negative eigenvalues. They
will be assigned NAs
(3)
In Deviance.categorical.data.vgam(mu = mu, y = y, w = w, residuals =
residuals,  :
  fitted values close to 0 or 1
(4) In log(prob) : NaNs produced

The last two don't seem to be as critical and the first two, seeing that
models with that errors at least provide a p-value (the first to errors
lead to a p--value of 1).

Thank you

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Testing the equality of two variances

[Tammy Ma]

Dear R-User,

I met the problem to test equality of variance.

Two sample units:
conjps-c(9.41,10.45,10.78,10.73,11.11,11.12,11.59,11.04,11.63)

ms-c(4.11,5.10,5.70,6.46,6.04,6.16, 6.24,6.32,7.33)

Then I use the F test to test:



•Test Equality
of
Two Variances

 F
test to compare two variances



data: 
conjps and ms 

F = 0.5419, num df = 8,
denom df = 8,
p-value = 0.4045

alternative hypothesis: true ratio of
variances is not equal to 1 

95 percent confidence interval:

 0.1222368 2.4024170 

sample estimates:

ratio of variances 


0.5419076 


but If conjps is replaced by conjps1: shifting the first value and the last 
value position
conjps1-c(11.36,10.45,10.78,10.73,11.11,11.12,11.59,11.04,9.41)

I do believe we get the same result when testing the equity of variance. but 
how do I distringuish this two cases by testing?
Kind regards,Tammy



  
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.