Re: [R] R does not start from (Debian) linux command line - error with doWithOneRestart() - segmentation fault

[Ashim Kapoor]

Dear Henrik,

We upgraded R and rebooted the computer and the error went away. Hence
I do not think we will get any strange results anymore.

Still, I could be mistaken and I am still curious as to what happened
to my R installation. Please find the answers inline below.

On Thu, Apr 8, 2021 at 9:59 PM Henrik Bengtsson
 wrote:
>
> Ashim, as Martin says, there's something really weird going on with
> your core R installation.  This is definitely not expected, and I
> don't know think if I've every seen this reported before.  Here are
> some questions/comments that might help you move forward and for
> others to pitch in:
>
> 1. How did you install R?
>

IIRC I installed R by doing:

apt-get install r-base r-base-dev

( following the instructions on this page :
https://cloud.r-project.org/ ) I went to this page and clicked on
"Download R for Linux" and then clicked on  "debian". I think, since
the page uses frames, the url does not change when I do those 2 steps.

> 2. What happens if you call:
>
> $ Rscript --vanilla -e "1+2"
>

$ Rscript --vanilla -e "1+2"
[1] 3


> 3. What happens if you call:
>
> $ Rscript --vanilla --default-packages=base -e "1+2"
>

$ Rscript --vanilla --default-packages=base -e "1+2"
[1] 3

> 4. What happens if you call:
>
> $ Rscript --vanilla --default-packages=base,methods -e "1+2"
>
$ Rscript --vanilla --default-packages=base,methods -e "1+2"
[1] 3

> If any of 2-4 gives a different result, that's a first clue.
>
> /Henrik
>

Many thanks for your help,
Best Regards,
Ashim

> On Wed, Apr 7, 2021 at 1:17 AM Ashim Kapoor  wrote:
> >
> > On Wed, Apr 7, 2021 at 12:51 PM Martin Maechler
> >  wrote:
> > >
> > > > Ashim Kapoor
> > > > on Wed, 7 Apr 2021 10:35:14 +0530 writes:
> > >
> > > > Dear R experts,
> > >
> > > > Here is my problem :
> > >
> > > > R startup FAILS with an error message. The error message
> > > > is more meaningful when I do invoke R via sudo OR as
> > > > root. I attach the startup messages when I invoke R as :
> > >
> > > > 1. as non root user 2. with sudo 3. as Root user.
> > >
> > > > The error messages ( mentioned in snippets below ) are
> > > > more meaningful to me in the above mentioned order.
> > >
> > > Thank you, Ashim.
> > >
> > > Yes, the messages point to something really bad.
> > >
> > > OTOH ("On the other hand"), what we *can* see is that you try to
> > > start R version 3.6.3.
> > >
> >
> > My apologies for not being clear.
> >
> > I would like to clarify :
> >
> > 1. The error message from the incantation `R` is not very informative.
> >
> > 2. The error message from the incantation `sudo R` says (among other
> > things which don't seem significant to me):
> > Error in doWithOneRestart(return(` \\x82\\x0ccPV`), restart) :
> >   not a proper file name
> >
> > 3. The error message from invoking R as a root user says:
> >
> > Error: not a proper file name
> > Fatal error: unable to initialize the JIT
> >
> > I found this on the internet :
> > https://stackoverflow.com/questions/19512165/error-in-dowithonerestart
> >
> > The error messages and the above link point to issues with JAVA, but
> > it does not say how to fix them.
> >
> > That is why I thought : there is an .xlsx file which has non english
> > characters which is messing with
> > Java.
> >
> > Query : is there a way to do JAVA garbage collection when R is not starting 
> > ?
> >
> > > While that is not extremely old, it may well be older than
> > > several other pieces of software (or even hardware) that you are
> > > running with.
> > >
> > > I very very  *strongly* recommend to use an R version 4.0.x ... and why 
> > > not
> > > use the latest  4.0.5 ?
> > >
> >
> > We upgraded to 4.0.5 but it did not make a difference. We restarted
> > the computer and that fixed the error.
> > I think the JAVA garbage collection kicked in when we reset the
> > computer and that is why it was fixed on rebooting. Not sure though.
> >
> > > Then, it may also be caused by a mismatch of system libraries
> > > and your oldish version of R. ... but there I'd strongly
> > > recommend consulting with other Debian users, notably as there
> > > is a dedicated  mailing list  R-SIG-Debian --> do subscribe
> > > there, and ask -- with more details on how you got your R: Is it
> > > the default R on your Debian, which version of Debian,  etc.
> > >
> > > Last but not least, Dirk Eddelbuettel, the maintainer of the
> > > official R Debian package maintains a nice web page -- part of
> > > the official CRAN web pages, but unfortunately a bit hidden
> > > nowadays, (not the least because CRAN still uses frames (würg!!)):
> > >
> > >   https://cloud.r-project.org/bin/linux/debian/
> > >
> > > A very nice and useful page,  much underrated and underused,
> > > probably.
> > >
> >
> > Thank you for this.
> >
> > > Best regards,
> > >
> > > Martin Maechler
> > > ETH Zurich  and  R Core Team
> > >
> > >
> > > > When I google around for the error message, it looks like
> > > > 

Re: [R] Unable to extract gene list from chromosome

[David Winsemius]

On 4/8/21 3:42 PM, pooja sinha wrote:
> Hi David,
>
> Sorry I forgot to attach the file. Now it's attached.


Now when I go back and check the values of the setup variables after 
seeing an error on the last call,

Error in .processResults(postRes, mart = mart, sep = sep, fullXmlQuery = 
fullXmlQuery,  :
   Query ERROR: caught BioMart::Exception::Database: Error during query 
execution: You have an error in your SQL syntax; check the manual that 
corresponds to your MySQL server version for the right syntax to use 
near 'AND (main.seq_region_end_1020 >= '15108600' OR 
main.seq_region_end_1020 >= '9115' at line 1

I now notice:


AT_AC_Gene$chr

#NULL

Changing that to AT_AC_Gene$Chromosome_number gets at least a startup 
message from the server:

Batch submitting query 
[==>---] 
5% eta:  1m

Error in .processResults(postRes, mart = mart, sep = sep, fullXmlQuery = 
fullXmlQuery,  :
   Query ERROR: caught BioMart::Exception::Database: Error during query 
execution: You have an error in your SQL syntax; check the manual that 
corresponds to your MySQL server version for the right syntax to use 
near 'AND (main.seq_region_end_1020 >= '15108600' OR 
main.seq_region_end_1020 >= '9115' at line 1

But then I get the same error before about SQL syntax error.


Then I ran it with only complete cases and now get no error but again 
see no hits:

str(final_1)
'data.frame':    0 obs. of  6 variables:
  $ external_gene_name: logi
  $ ensembl_gene_id   : logi
  $ start_position    : logi
  $ end_position  : logi
  $ rgd_symbol    : logi
  $ chromosome_name   : logi


I also see a lot of NA's in that dataset and when I just send the first 
10 rows of the request, I get no error (but also no matches.)


So you clearly are not giving us all the data or all the code, but I'm 
finally wondering if you just don't have an data that matches teh 
external datasets in your chosen "biomart". Can you offer a smaller 
dataset that you know with certainty should produce a match?


Alternatively, you might want to post this instead at the BioConductor 
mailing list. They are the people who have a better chance of spotting 
obvious errors. I've found two likely code-related errors but I'm not a 
computational biostatistician.

David

>
>
> Thanks,
> Puja
>
> On Thu, Apr 8, 2021 at 6:01 PM David Winsemius  > wrote:
>
>
> On 4/8/21 2:30 PM, pooja sinha wrote:
> > Hi All,
> >
> > I am trying to extract gene list from chromosome number and
> position, for
> > that I am using biomaRt in R but I am getting error messages as
> shown
> > below. Also below is the code I am using for extraction.
> >
> > library("biomaRt")
> > listMarts()
> > ensembl <- useMart("ensembl")
> > datasets <- listDatasets(ensembl)
> > ensembl = useDataset("rnorvegicus_gene_ensembl",mart=ensembl)
> > AT_AC_Gene <-
> read.csv("AT-AC-methylkit_biomart-4-7-21.csv",header=T)
>
>
> #--- a this point I get
>
> Error in file(file, "rt") : cannot open the connection
> In addition: Warning message:
> In file(file, "rt") :
>    cannot open file 'AT-AC-methylkit_biomart-4-7-21.csv': No such
> file
> or directory
>
> > attributes <-
> >
> 
> c("external_gene_name","ensembl_gene_id","start_position","end_position","rgd_symbol","chromosome_name")
> > filters <- c("chromosome_name","start","end")
> > values <- list(AT_AC_Gene$chr,AT_AC_Gene$start,AT_AC_Gene$end)
> > final_1 <- getBM(attributes=attributes, filters=filters,
> values=values,
> > mart=ensembl)
> >
> > The code runs well without any error but the final1 output has 0
> > observations of 6 variables. Why?
> >
> > Can anyone help me with this?
>
>
> You are more likely to get a useful response on the BioC mailing
> list.
> It appears you have a dependenciy of a csv file that you have not
> told
> us about.
>
>
> -- 
>
> David
>
> >
> >
> > Thanks,
> >
> > Puja
> >
> >       [[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org  mailing list
> -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Unable to extract gene list from chromosome

[David Winsemius]

On 4/8/21 2:30 PM, pooja sinha wrote:

Hi All,

I am trying to extract gene list from chromosome number and position, for
that I am using biomaRt in R but I am getting error messages as shown
below. Also below is the code I am using for extraction.

library("biomaRt")
listMarts()
ensembl <- useMart("ensembl")
datasets <- listDatasets(ensembl)
ensembl = useDataset("rnorvegicus_gene_ensembl",mart=ensembl)
AT_AC_Gene <- read.csv("AT-AC-methylkit_biomart-4-7-21.csv",header=T)



#--- a this point I get

Error in file(file, "rt") : cannot open the connection
In addition: Warning message:
In file(file, "rt") :
  cannot open file 'AT-AC-methylkit_biomart-4-7-21.csv': No such file 
or directory



attributes <-
c("external_gene_name","ensembl_gene_id","start_position","end_position","rgd_symbol","chromosome_name")
filters <- c("chromosome_name","start","end")
values <- list(AT_AC_Gene$chr,AT_AC_Gene$start,AT_AC_Gene$end)
final_1 <- getBM(attributes=attributes, filters=filters, values=values,
mart=ensembl)

The code runs well without any error but the final1 output has 0
observations of 6 variables. Why?

Can anyone help me with this?



You are more likely to get a useful response on the BioC mailing list. 
It appears you have a dependenciy of a csv file that you have not told 
us about.



--

David




Thanks,

Puja

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Unable to extract gene list from chromosome

[pooja sinha]

Hi All,

I am trying to extract gene list from chromosome number and position, for
that I am using biomaRt in R but I am getting error messages as shown
below. Also below is the code I am using for extraction.

library("biomaRt")
listMarts()
ensembl <- useMart("ensembl")
datasets <- listDatasets(ensembl)
ensembl = useDataset("rnorvegicus_gene_ensembl",mart=ensembl)
AT_AC_Gene <- read.csv("AT-AC-methylkit_biomart-4-7-21.csv",header=T)
attributes <-
c("external_gene_name","ensembl_gene_id","start_position","end_position","rgd_symbol","chromosome_name")
filters <- c("chromosome_name","start","end")
values <- list(AT_AC_Gene$chr,AT_AC_Gene$start,AT_AC_Gene$end)
final_1 <- getBM(attributes=attributes, filters=filters, values=values,
mart=ensembl)

The code runs well without any error but the final1 output has 0
observations of 6 variables. Why?

Can anyone help me with this?


Thanks,

Puja

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Variance of the sample mean by bootstrapping

[thebudget72]

Hello,

I'm studying bootstrapping.

I devised a code that draw a sample of heights, and then compute the 
sample mean of those heights.

I want to compute the variance of the sample mean by bootstrapping.

I am comparing this with the "real" variance of the sample meanand with 
an "estimated" variance of the sample mean.

Is my code correct? The results looks very similar in all three cases.

I just want to be sure that I made everything right.

Thanks in advance!

#!/usr/bin/env R

its <- 1
mean <- 180
sd <- 50
N <- 1

heights <- rnorm(N, mean, sd)

sample.mean <- mean(heights)

print(paste0("Real mean: ", mean))
print(paste0("Sample mean: ", sample.mean))

# unbiased sample variance
# the var() function already uses the unbiased estimator
#sample.var <- (N/(N-1))*var(heights)
sample.var <- var(heights)

mean.var.real <- (sd^2)/N
print(paste0("Actual variance of the sample mean: ", mean.var.real))

mean.var.sample <- sample.var/N
print(paste0("Estimated variance of the sample mean: ", mean.var.sample))

# Compute variance by bootstrap
boot.means <- rep(NA, its)
for(it in 1:its) {
   boot.h <- sample(heights, N, replace=T)
   boot.means[it] <- mean(boot.h)
}

mean.var.boot <- var(boot.means)
print(paste0("Bootstrapped variance of the sample mean: ", mean.var.boot))


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] R does not start from (Debian) linux command line - error with doWithOneRestart() - segmentation fault

[Henrik Bengtsson]

Ashim, as Martin says, there's something really weird going on with
your core R installation.  This is definitely not expected, and I
don't know think if I've every seen this reported before.  Here are
some questions/comments that might help you move forward and for
others to pitch in:

1. How did you install R?

2. What happens if you call:

$ Rscript --vanilla -e "1+2"

3. What happens if you call:

$ Rscript --vanilla --default-packages=base -e "1+2"

4. What happens if you call:

$ Rscript --vanilla --default-packages=base,methods -e "1+2"

If any of 2-4 gives a different result, that's a first clue.

/Henrik

On Wed, Apr 7, 2021 at 1:17 AM Ashim Kapoor  wrote:
>
> On Wed, Apr 7, 2021 at 12:51 PM Martin Maechler
>  wrote:
> >
> > > Ashim Kapoor
> > > on Wed, 7 Apr 2021 10:35:14 +0530 writes:
> >
> > > Dear R experts,
> >
> > > Here is my problem :
> >
> > > R startup FAILS with an error message. The error message
> > > is more meaningful when I do invoke R via sudo OR as
> > > root. I attach the startup messages when I invoke R as :
> >
> > > 1. as non root user 2. with sudo 3. as Root user.
> >
> > > The error messages ( mentioned in snippets below ) are
> > > more meaningful to me in the above mentioned order.
> >
> > Thank you, Ashim.
> >
> > Yes, the messages point to something really bad.
> >
> > OTOH ("On the other hand"), what we *can* see is that you try to
> > start R version 3.6.3.
> >
>
> My apologies for not being clear.
>
> I would like to clarify :
>
> 1. The error message from the incantation `R` is not very informative.
>
> 2. The error message from the incantation `sudo R` says (among other
> things which don't seem significant to me):
> Error in doWithOneRestart(return(` \\x82\\x0ccPV`), restart) :
>   not a proper file name
>
> 3. The error message from invoking R as a root user says:
>
> Error: not a proper file name
> Fatal error: unable to initialize the JIT
>
> I found this on the internet :
> https://stackoverflow.com/questions/19512165/error-in-dowithonerestart
>
> The error messages and the above link point to issues with JAVA, but
> it does not say how to fix them.
>
> That is why I thought : there is an .xlsx file which has non english
> characters which is messing with
> Java.
>
> Query : is there a way to do JAVA garbage collection when R is not starting ?
>
> > While that is not extremely old, it may well be older than
> > several other pieces of software (or even hardware) that you are
> > running with.
> >
> > I very very  *strongly* recommend to use an R version 4.0.x ... and why not
> > use the latest  4.0.5 ?
> >
>
> We upgraded to 4.0.5 but it did not make a difference. We restarted
> the computer and that fixed the error.
> I think the JAVA garbage collection kicked in when we reset the
> computer and that is why it was fixed on rebooting. Not sure though.
>
> > Then, it may also be caused by a mismatch of system libraries
> > and your oldish version of R. ... but there I'd strongly
> > recommend consulting with other Debian users, notably as there
> > is a dedicated  mailing list  R-SIG-Debian --> do subscribe
> > there, and ask -- with more details on how you got your R: Is it
> > the default R on your Debian, which version of Debian,  etc.
> >
> > Last but not least, Dirk Eddelbuettel, the maintainer of the
> > official R Debian package maintains a nice web page -- part of
> > the official CRAN web pages, but unfortunately a bit hidden
> > nowadays, (not the least because CRAN still uses frames (würg!!)):
> >
> >   https://cloud.r-project.org/bin/linux/debian/
> >
> > A very nice and useful page,  much underrated and underused,
> > probably.
> >
>
> Thank you for this.
>
> > Best regards,
> >
> > Martin Maechler
> > ETH Zurich  and  R Core Team
> >
> >
> > > When I google around for the error message, it looks like
> > > there is an .xlsx file which has non english characters
> > > which is messing with Java.
> >
> > > I do not know how to fix this. I tried :-
> >
> > > R --vanilla
> >
> > > so that it would not use any startup scripts but that also
> > > does not work.
> >
> > > - snip
> > > 
> > 
> >
> > > When I try to start R from the command line :
> >
> > > ~$ R
> >
> > >  *** caught segfault *** address (nil), cause 'unknown'
> >
> > > Traceback: 1: NextMethod(.Generic) 2:
> > > Ops.numeric_version(R_version_built_under, "3.0.0") 3:
> > > testRversion(pkgInfo, package, pkgpath) 4:
> > > library(package, lib.loc = lib.loc, character.only = TRUE,
> > > logical.return = TRUE, warn.conflicts = warn.conflicts,
> > > quietly = quietly, mask.ok = mask.ok, exclude = exclude,
> > > include.only = include.only, attach.required =
> > > attach.required) 5: doTryCatch(return(expr), name,
> > > parentenv, 

Re: [R] /usr/local/lib/R/site-library is not writable

[Jan van der Laan]

I would actually go a step in the other direction: per project 
libraries. For example by adding a .Rprofile file to your project 
directory. This ensures that everybody working on a project uses the 
same version of the packages (even on different machines e.g. on shared 
folders).


This can give issues when a new version of R arrives, but that is 
usually easy to solve. Either hard code the path to the old R-version or 
decide to update all packages in a project to the new R-version (and 
test that everything is still working ok).


We have the most often used packages installed centrally on the 
server/network, so I actually usually end up with a mixture of central, 
personal and project libraries. Theory vs practice.


HTH,
Jan



On 08-04-2021 02:58, Dirk Eddelbuettel wrote:

Hi Gene,

"It's complicated". (Not really, but listen for a sec...)

We need to ship a default policy that makes sense for all / most
situations.  So

- users cannot write into /usr/local/lib/R/site-library -- unless they are
   set up to, but adding them to the 'group' that owns that directory

- root can (but ideally one should not run as root as one generally does not
   now what code you might get slipped in a tar.gz); but root can enable users

- so we recommend letting (some or all) users write there by explicitly
   adding them to an appropriate group.

Personally, I do not think personal libraries are a good idea on shared
machines because you can end up with a different set of package (versions)
than your colleague on the same machine.  And or you running shiny from $HOME
have different packages than shiny running as server. And on and on. Other
people differ, and that is fine. If one wants personal libraries one can.

I must have explained the reasoning and fixes a dozen times each on
r-sig-debian (where you could have asked this too) and StackOverflow. At
least the latter can be searched so look at this set:
https://stackoverflow.com/search?q=user%3Ame+is%3Aanser+%2Fusr%2Flocal%2Flib%2FR%2Fsite-library

Happy to take it offline too, and who knows, we even get to meet for a coffee
one of these days.

Hope this helps, Dirk



__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.