Re: [R] unexpected behavior in apply
If an oven expects fried potatoes and I put a cake in, I would hope it
complains or does nothing rather than surreptitiously poisoning my cake.
Jiefei's finding that "6" becomes " 6" during matrix coercion (apparently
for aesthetic reasons only) feels more like the latter. But I appreciate the
explanation and the solutions.
-Original Message-
From: PIKAL Petr
Sent: Monday, October 11, 2021 5:15 AM
To: Jiefei Wang ; Derickson, Ryan, VHA NCOD
Cc: r-help@r-project.org
Subject: [EXTERNAL] RE: [R] unexpected behavior in apply
Hi
it is not surprising at all.
from apply documentation
Arguments
X
an array, including a matrix.
data.frame is not matrix or array (even if it rather resembles one)
So if you put a cake into oven you cannot expect getting fried potatoes from
it.
For data frames sapply or lapply is preferable as it is designed for lists
and data frame is (again from documentation)
A data frame is a list of variables of the same number of rows with unique
row names, given class "data.frame".
> sapply(d,function(x) all(x[!is.na(x)]<=3))
d1d2d3
FALSE TRUE FALSE
Cheers
Petr
> -Original Message-
> From: R-help On Behalf Of Jiefei Wang
> Sent: Friday, October 8, 2021 8:22 PM
> To: Derickson, Ryan, VHA NCOD
> Cc: r-help@r-project.org
> Subject: Re: [R] unexpected behavior in apply
>
> Ok, it turns out that this is documented, even though it looks surprising.
>
> First of all, the apply function will try to convert any object with the
dim
> attribute to a matrix(my intuition agrees with you that there should be no
> conversion), so the first step of the apply function is
>
> > as.matrix.data.frame(d)
> d1 d2 d3
> [1,] "a" "1" NA
> [2,] "b" "2" NA
> [3,] "c" "3" " 6"
>
> Since the data frame `d` is a mixture of character and non-character
values,
> the non-character value will be converted to the character using the
function
> `format`. However, the problem is that the NA value will also be formatted
to
> the character
>
> > format(c(NA, 6))
> [1] "NA" " 6"
>
> That's where the space comes from. It is purely for making the result
pretty...
> The character NA will be removed later, but the space is not stripped. I
would
> say this is not a good design, and it might be worth not including the NA
value
> in the format function. At the current stage, I will suggest using the
function
> `lapply` to do what you want.
>
> > lapply(d, FUN=function(x)all(x[!is.na(x)] <= 3))
> $d1
> [1] FALSE
> $d2
> [1] TRUE
> $d3
> [1] FALSE
>
> Everything should work as you expect.
>
> Best,
> Jiefei
>
> On Sat, Oct 9, 2021 at 2:03 AM Jiefei Wang wrote:
> >
> > Hi,
> >
> > I guess this can tell you what happens behind the scene
> >
> >
> > > d<-data.frame(d1 = letters[1:3],
> > + d2 = c(1,2,3),
> > + d3 = c(NA,NA,6))
> > > apply(d, 2, FUN=function(x)x)
> > d1 d2 d3
> > [1,] "a" "1" NA
> > [2,] "b" "2" NA
> > [3,] "c" "3" " 6"
> > > "a"<=3
> > [1] FALSE
> > > "2"<=3
> > [1] TRUE
> > > "6"<=3
> > [1] FALSE
> >
> > Note that there is an additional space in the character value " 6",
> > that's why your comparison fails. I do not understand why but this
> > might be a bug in R
> >
> > Best,
> > Jiefei
> >
> > On Sat, Oct 9, 2021 at 1:49 AM Derickson, Ryan, VHA NCOD via R-help
> > wrote:
> > >
> > > Hello,
> > >
> > > I'm seeing unexpected behavior when using apply() compared to a for
> loop when a character vector is part of the data subjected to the apply
> statement. Below, I check whether all non-missing values are <= 3. If I
> include a character column, apply incorrectly returns TRUE for d3. If I
only
> pass the numeric columns to apply, it is correct for d3. If I use a for
loop, it is
> correct.
> > >
> > > > d<-data.frame(d1 = letters[1:3],
> > > + d2 = c(1,2,3),
> > > + d3 = c(NA,NA,6))
> > > >
> > > > d
> > > d1 d2 d3
> > > 1 a 1 NA
> > > 2 b 2 NA
> > > 3 c 3 6
> > > >
> > > > # results are incorrect
> > > > apply(d, 2, FUN=function(x)all(x[!is.na(x)] <= 3))
> > >d1d2d3
> > > FALSE TRUE TRUE
> > > >
> > > > # results are correct
> > > > apply(d[,2:3], 2, FUN=function(x)all(x[!is.na(x)] <= 3))
> > >d2d3
> > > TRUE FALSE
> > > >
> > > > # results are correct
> > > > for(i in names(d)){
> > > + print(all(d[!is.na(d[,i]),i] <= 3)) }
> > > [1] FALSE
> > > [1] TRUE
> > > [1] FALSE
> > >
> > >
> > > Finally, if I remove the NA values from d3 and include the character
> column in apply, it is correct.
> > >
> > > > d<-data.frame(d1 = letters[1:3],
> > > + d2 = c(1,2,3),
> > > + d3 = c(4,5,6))
> > > >
> > > > d
> > > d1 d2 d3
> > > 1 a 1 4
> > > 2 b 2 5
> > > 3 c 3 6
> > > >
> > > > # results are correct
> > > > apply(d, 2, FUN=function(x)all(x[!is.na(x)] <= 3))
> > >d1d2d3
> > > FALSE TRUE FALSE
> > >
> > >
> > > Can someone help me understand what's happening?
> > >
> > >
Re: [R] How to rotate only one panel by 90 degrees in R plot?
Thank you!
On Wed, Oct 13, 2021 at 12:00 PM Deepayan Sarkar
wrote:
>
> On Wed, Oct 13, 2021 at 12:13 PM Luigi Marongiu
> wrote:
> >
> > I have seen that the only package that easily rotate the plot is
> > ggplot, so I ran:
> > ```
> > library(ggplot2)
> > df = data.frame(MR = c(negative_mr, uncertain_mr, positive_mr),
> > FCN = c(negative_fcn, uncertain_fcn, positive_fcn))
> > p <- ggplot(df, aes(x=MR)) +
> > geom_density()
> > p + coord_flip()
> > ```
> > Even in this case, the plot is correctly rotated, but I can't place it
> > in the allocated panel. ggplot2 simply overwrites the whole plot. This
> > means I need to do the whole thing in ggplot2 (would lattice have an
> > equivalent?) and split the plot into uneven panels with ggplot2.
> > Changing x into y is a clever approach, but it is not the same as
> > rotating a plot. But YES, that is exactly what I wanted to plot. Thank
> > you!
>
> Well, traditional R graphics (as well as lattice) requires a more DIY
> approach. As Paul already indicated, you need to control what you are
> plotting instead of relying on plot(density(...)) doing the right
> thing.
>
> Modifying your original code:
>
> ```
> layout(matrix(c(1,2),nrow=1), widths=c(3,1)) # split panels unevenly
>
> plot(negative_x, negative_y, ylim=c(0,0.5), xlim=c(0,41), cex=1.5,
> xlab=expression(bold("X")),
> ylab=expression(bold("Y")))
> points(positive_x, positive_y, pch=16, cex=1.5)
> points(uncertain_x, uncertain_y, pch=16, cex=1.5, col="grey")
> legend("topleft",
>legend = c("Positives", "Negatives", "Uncertains"),
>pch = c(16, 1, 16), col=c("black", "black", "grey"), cex=0.8)
>
> ## calculate density (but don't plot yet)
> d <- density(c(negative_y, uncertain_y, positive_y))
>
> ## Your original code is equivalent to this
> ## plot(d,
> ## yaxt="n", xaxt="n", main=NA, ylab=NA, xlab=NA)
>
> plot(d$y, d$x, type = "l",
> ##yaxt="n",
> xaxt="n", main=NA, ylab=NA, xlab=NA)
> ```
>
> Note that in the last plot, I have plot(d$y, d$x, ...) instead of
> plot(d$x, d$y, ...).
>
> I have commented out your yaxt="n" to highlight something that may not
> be initially obvious, which is that the axis limits of your two plots
> do not match. To ensure that, you would additionally need to match
> ylim:
>
> ```
> plot(d$y, d$x, type = "l", ylim=c(0,0.5),
> yaxt="n", xaxt="n", main=NA, ylab=NA, xlab=NA)
> ```
>
> Best,
> -Deepayan
>
> > On Wed, Oct 13, 2021 at 1:17 AM Bert Gunter wrote:
> > >
> > > I don't know the gridGraphics package, and I haven't looked closely at
> > > what you are trying to do. But note that lattice functions construct grid
> > > "grobs" that can be saved and plotted in arbitrary, including rotated,
> > > viewports, using the print.trellis function. I frankly am pretty ignorant
> > > about such things, but this simple little example might give you some
> > > notion of how to proceed. You may also be able to do what you want with
> > > grid.layout() and pushing a suitably rotated viewport onto a layout.
> > > Others would have to advise on such details, if so.
> > >
> > > If I'm wrong and this is useless, just ignore without comment.
> > >
> > >
> > >
> > > dp <- densityplot(~y, main = "",
> > > xlab = "", ylab = "")
> > > grid.newpage()
> > > pushViewport(
> > >viewport(width = unit(.5,"npc"),
> > > height = unit(.3,"npc"),
> > > angle = 270))
> > > print(dp, newp = FALSE, ## this is the print.trellis method
> > > panel.width = list(1,"npc"),
> > > panel.height = list(1, "npc")
> > > )
> > >
> > > Bert Gunter
> > >
> > > "The trouble with having an open mind is that people keep coming along
> > > and sticking things into it."
> > > -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
> > >
> > >
> > > On Tue, Oct 12, 2021 at 1:43 PM Luigi Marongiu
> > > wrote:
> > >>
> > >> Hello,
> > >> I would like to show a density plot of the Y axis. To do that, I would
> > >> like to split the plot into a panel 2/3 long and a density plot 1/3
> > >> long. The problem is that, since the density is on the Y axis, the
> > >> density plot should be rotated byb90 degrees. I tried with the package
> > >> gridGraphics but it rotates both panels.
> > >> ```
> > >> negative_y <- runif(50, 0, 0.099)
> > >> negative_x <- runif(50, 1, 40)
> > >> positive_y <- c(runif(30, 0.2, 0.5), runif(20, 0.4, 0.5))
> > >> positive_x <- c(runif(30, 25, 40), runif(20, 10, 25))
> > >> uncertain_y <- runif(10, 0.099, 0.2)
> > >> uncertain_x <- runif(10, 2, 40)
> > >> # plot on MR/FCN space
> > >> layout(matrix(c(1,2),nrow=1), widths=c(3,1)) # split panels unevenly
> > >> plot(negative_x, negative_y, ylim=c(0,0.5), xlim=c(0,41), cex=1.5,
> > >> xlab=expression(bold("X")),
> > >> ylab=expression(bold("Y")))
> > >> points(positive_x, positive_y, pch=16, cex=1.5)
> > >> points(uncertain_x, uncertain_y, pch=16, cex=1.5, col="grey")
> > >> legend("topleft",
> > >> legend = c("Positives", "Negatives",
Re: [R] How to rotate only one panel by 90 degrees in R plot?
On Wed, Oct 13, 2021 at 12:13 PM Luigi Marongiu
wrote:
>
> I have seen that the only package that easily rotate the plot is
> ggplot, so I ran:
> ```
> library(ggplot2)
> df = data.frame(MR = c(negative_mr, uncertain_mr, positive_mr),
> FCN = c(negative_fcn, uncertain_fcn, positive_fcn))
> p <- ggplot(df, aes(x=MR)) +
> geom_density()
> p + coord_flip()
> ```
> Even in this case, the plot is correctly rotated, but I can't place it
> in the allocated panel. ggplot2 simply overwrites the whole plot. This
> means I need to do the whole thing in ggplot2 (would lattice have an
> equivalent?) and split the plot into uneven panels with ggplot2.
> Changing x into y is a clever approach, but it is not the same as
> rotating a plot. But YES, that is exactly what I wanted to plot. Thank
> you!
Well, traditional R graphics (as well as lattice) requires a more DIY
approach. As Paul already indicated, you need to control what you are
plotting instead of relying on plot(density(...)) doing the right
thing.
Modifying your original code:
```
layout(matrix(c(1,2),nrow=1), widths=c(3,1)) # split panels unevenly
plot(negative_x, negative_y, ylim=c(0,0.5), xlim=c(0,41), cex=1.5,
xlab=expression(bold("X")),
ylab=expression(bold("Y")))
points(positive_x, positive_y, pch=16, cex=1.5)
points(uncertain_x, uncertain_y, pch=16, cex=1.5, col="grey")
legend("topleft",
legend = c("Positives", "Negatives", "Uncertains"),
pch = c(16, 1, 16), col=c("black", "black", "grey"), cex=0.8)
## calculate density (but don't plot yet)
d <- density(c(negative_y, uncertain_y, positive_y))
## Your original code is equivalent to this
## plot(d,
## yaxt="n", xaxt="n", main=NA, ylab=NA, xlab=NA)
plot(d$y, d$x, type = "l",
##yaxt="n",
xaxt="n", main=NA, ylab=NA, xlab=NA)
```
Note that in the last plot, I have plot(d$y, d$x, ...) instead of
plot(d$x, d$y, ...).
I have commented out your yaxt="n" to highlight something that may not
be initially obvious, which is that the axis limits of your two plots
do not match. To ensure that, you would additionally need to match
ylim:
```
plot(d$y, d$x, type = "l", ylim=c(0,0.5),
yaxt="n", xaxt="n", main=NA, ylab=NA, xlab=NA)
```
Best,
-Deepayan
> On Wed, Oct 13, 2021 at 1:17 AM Bert Gunter wrote:
> >
> > I don't know the gridGraphics package, and I haven't looked closely at what
> > you are trying to do. But note that lattice functions construct grid
> > "grobs" that can be saved and plotted in arbitrary, including rotated,
> > viewports, using the print.trellis function. I frankly am pretty ignorant
> > about such things, but this simple little example might give you some
> > notion of how to proceed. You may also be able to do what you want with
> > grid.layout() and pushing a suitably rotated viewport onto a layout. Others
> > would have to advise on such details, if so.
> >
> > If I'm wrong and this is useless, just ignore without comment.
> >
> >
> >
> > dp <- densityplot(~y, main = "",
> > xlab = "", ylab = "")
> > grid.newpage()
> > pushViewport(
> >viewport(width = unit(.5,"npc"),
> > height = unit(.3,"npc"),
> > angle = 270))
> > print(dp, newp = FALSE, ## this is the print.trellis method
> > panel.width = list(1,"npc"),
> > panel.height = list(1, "npc")
> > )
> >
> > Bert Gunter
> >
> > "The trouble with having an open mind is that people keep coming along and
> > sticking things into it."
> > -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
> >
> >
> > On Tue, Oct 12, 2021 at 1:43 PM Luigi Marongiu
> > wrote:
> >>
> >> Hello,
> >> I would like to show a density plot of the Y axis. To do that, I would
> >> like to split the plot into a panel 2/3 long and a density plot 1/3
> >> long. The problem is that, since the density is on the Y axis, the
> >> density plot should be rotated byb90 degrees. I tried with the package
> >> gridGraphics but it rotates both panels.
> >> ```
> >> negative_y <- runif(50, 0, 0.099)
> >> negative_x <- runif(50, 1, 40)
> >> positive_y <- c(runif(30, 0.2, 0.5), runif(20, 0.4, 0.5))
> >> positive_x <- c(runif(30, 25, 40), runif(20, 10, 25))
> >> uncertain_y <- runif(10, 0.099, 0.2)
> >> uncertain_x <- runif(10, 2, 40)
> >> # plot on MR/FCN space
> >> layout(matrix(c(1,2),nrow=1), widths=c(3,1)) # split panels unevenly
> >> plot(negative_x, negative_y, ylim=c(0,0.5), xlim=c(0,41), cex=1.5,
> >> xlab=expression(bold("X")),
> >> ylab=expression(bold("Y")))
> >> points(positive_x, positive_y, pch=16, cex=1.5)
> >> points(uncertain_x, uncertain_y, pch=16, cex=1.5, col="grey")
> >> legend("topleft",
> >> legend = c("Positives", "Negatives", "Uncertains"),
> >> pch = c(16, 1, 16), col=c("black", "black", "grey"), cex=0.8)
> >> # plot density
> >> plot(density(c(negative_y, uncertain_y, positive_y)),
> >> yaxt="n", xaxt="n", main=NA, ylab=NA, xlab=NA)
> >> library(gridGraphics)
> >> grab_grob <- function(){
> >> grid.echo()
> >> grid.grab()
> >> }
>
[R] How to find local minimum between distributions using mixtools?
Hello,
I have two peaks distributions (and some noise values in between), and
I used mixtools to characterize the two overlapping data. How can I
find the minimum between the peak distributions?
The sigma value of the second population is close to that value, but I
am not sure if this is really the cut-off point between the two
distributions (by eye, I would say the value is 0.125 against 0.1 of
the sigma value). Is there a better approach?
Thanks
```
set.seed(10)
negative_mr <- runif(50, 0, 0.099)
negative_fcn <- runif(50, 1, 40)
positive_mr <- c(runif(30, 0.2, 0.5), runif(20, 0.4, 0.5))
positive_fcn <- c(runif(30, 25, 40), runif(20, 10, 25))
uncertain_mr <- runif(10, 0.099, 0.2)
uncertain_fcn <- runif(10, 2, 40)
df <- data.frame(Y=c(negative_mr, uncertain_mr, positive_mr),
X=c(negative_fcn, uncertain_fcn, positive_fcn),
GROUP=c(rep("negative", length(negative_mr)),
rep("uncertain", length(uncertain_mr)),
rep("positive", length(positive_mr
library(mixtools)
model = normalmixEM((x = df$Y))
summary(model)
# plot
plot(model, which=2)
Cut_off <- model$sigma[2]
Cut_offE <- 0.125
abline(v=Cut_off, col="blue", lwd=2)
abline(v=Cut_offE, col="blue", lwd=2, lty=2)
plot(df$Y[df$GROUP=="negative"]~df$X[df$GROUP=="negative"],
pch=1, ylim=c(0,0.5), xlim=c(0,41), cex=1.5,
xlab=expression(bold("X")),
ylab=expression(bold("Y")))
points(df$Y[df$GROUP=="positive"]~df$X[df$GROUP=="positive"],
pch=16, cex=1.5)
points(df$Y[df$GROUP=="uncertain"]~df$X[df$GROUP=="uncertain"],
pch=16, cex=1.5, col="grey")
abline(h=Cut_off, col="blue", lwd=2)
abline(h=Cut_offE, col="blue", lwd=2, lty=2)
```
--
Best regards,
Luigi
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] How to rotate only one panel by 90 degrees in R plot?
I have seen that the only package that easily rotate the plot is
ggplot, so I ran:
```
library(ggplot2)
df = data.frame(MR = c(negative_mr, uncertain_mr, positive_mr),
FCN = c(negative_fcn, uncertain_fcn, positive_fcn))
p <- ggplot(df, aes(x=MR)) +
geom_density()
p + coord_flip()
```
Even in this case, the plot is correctly rotated, but I can't place it
in the allocated panel. ggplot2 simply overwrites the whole plot. This
means I need to do the whole thing in ggplot2 (would lattice have an
equivalent?) and split the plot into uneven panels with ggplot2.
Changing x into y is a clever approach, but it is not the same as
rotating a plot. But YES, that is exactly what I wanted to plot. Thank
you!
On Wed, Oct 13, 2021 at 1:17 AM Bert Gunter wrote:
>
> I don't know the gridGraphics package, and I haven't looked closely at what
> you are trying to do. But note that lattice functions construct grid "grobs"
> that can be saved and plotted in arbitrary, including rotated, viewports,
> using the print.trellis function. I frankly am pretty ignorant about such
> things, but this simple little example might give you some notion of how to
> proceed. You may also be able to do what you want with grid.layout() and
> pushing a suitably rotated viewport onto a layout. Others would have to
> advise on such details, if so.
>
> If I'm wrong and this is useless, just ignore without comment.
>
>
>
> dp <- densityplot(~y, main = "",
> xlab = "", ylab = "")
> grid.newpage()
> pushViewport(
>viewport(width = unit(.5,"npc"),
> height = unit(.3,"npc"),
> angle = 270))
> print(dp, newp = FALSE, ## this is the print.trellis method
> panel.width = list(1,"npc"),
> panel.height = list(1, "npc")
> )
>
> Bert Gunter
>
> "The trouble with having an open mind is that people keep coming along and
> sticking things into it."
> -- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
>
>
> On Tue, Oct 12, 2021 at 1:43 PM Luigi Marongiu
> wrote:
>>
>> Hello,
>> I would like to show a density plot of the Y axis. To do that, I would
>> like to split the plot into a panel 2/3 long and a density plot 1/3
>> long. The problem is that, since the density is on the Y axis, the
>> density plot should be rotated byb90 degrees. I tried with the package
>> gridGraphics but it rotates both panels.
>> ```
>> negative_y <- runif(50, 0, 0.099)
>> negative_x <- runif(50, 1, 40)
>> positive_y <- c(runif(30, 0.2, 0.5), runif(20, 0.4, 0.5))
>> positive_x <- c(runif(30, 25, 40), runif(20, 10, 25))
>> uncertain_y <- runif(10, 0.099, 0.2)
>> uncertain_x <- runif(10, 2, 40)
>> # plot on MR/FCN space
>> layout(matrix(c(1,2),nrow=1), widths=c(3,1)) # split panels unevenly
>> plot(negative_x, negative_y, ylim=c(0,0.5), xlim=c(0,41), cex=1.5,
>> xlab=expression(bold("X")),
>> ylab=expression(bold("Y")))
>> points(positive_x, positive_y, pch=16, cex=1.5)
>> points(uncertain_x, uncertain_y, pch=16, cex=1.5, col="grey")
>> legend("topleft",
>> legend = c("Positives", "Negatives", "Uncertains"),
>> pch = c(16, 1, 16), col=c("black", "black", "grey"), cex=0.8)
>> # plot density
>> plot(density(c(negative_y, uncertain_y, positive_y)),
>> yaxt="n", xaxt="n", main=NA, ylab=NA, xlab=NA)
>> library(gridGraphics)
>> grab_grob <- function(){
>> grid.echo()
>> grid.grab()
>> }
>> g <- grab_grob()
>> grid.newpage()
>> pushViewport(viewport(width=0.7,angle=270))
>> grid.draw(g)
>> ```
>> How can I rotate only the second panel? I tried to assign the second
>> plot to an object p and then call grid.draw(p), or to assign g to the
>> second plot, but they did not work...
>> Thanks
>>
>>
>> --
>> Best regards,
>> Luigi
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
--
Best regards,
Luigi
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
