Re: [Rd] [New Patch] Fix disk corruption when writing
I have now committed changes to R-devel (rev 72898) that seem to catch
large and small errors. They only give a warning if the error happens
when the connection is closed, because that can happen asynchronously:
I didn't want to mess up some later unrelated computation that triggered
garbage collection.
I will wait a while before porting these to R-patched, because there may
still be some problems to clean up.
Duncan Murdoch
On 07/07/2017 11:42 AM, Duncan Murdoch wrote:
On 07/07/2017 11:13 AM, Serguei Sokol wrote:
Le 07/07/2017 à 16:52, Duncan Murdoch a écrit :
On 07/07/2017 9:54 AM, Serguei Sokol wrote:
Le 07/07/2017 à 01:09, Duncan Murdoch a écrit :
On 06/07/2017 6:44 PM, Sokol Serguei wrote:
Duncan Murdoch has written at Thu, 6 Jul 2017 13:58:10 -0400
On 06/07/2017 5:21 AM, Serguei Sokol wrote:
I propose the following patch against the current
R-devel/src/main/connection.c (cf. attached file).
It gives (on my linux box):
> fc=file("/dev/full", "w")
> write.csv("a", file=fc)
Error in writeLines(paste(col.names, collapse = sep), file, sep = eol) :
system call failure on writing
> close(fc)
Serguei.
I suspect that flushing on every write will slow things down too much.
That's quite plausible.
I think a better approach is to catch the error in the Rconn_printf
calls (as R-devel currently does), and also catch errors on
con->close(con). This one requires more changes to the source, so it
may be a day or two before I commit.
I have testes it on file_close() and it works (cf. attached patch):
fc=file("/dev/full", "w")
write.csv("a", file=fc)
close(fc)
Error in close.connection(fc) : closing file failed
One thing I have to look into: is anyone making use of the fact that
the R-level close.connection() function can return -1 to signal an
error? Base R ignores that, which is why we need to do something, but
if packages are using it, things need to be changed carefully. I
can't just change it to raise an error instead.
As you can see in the patch, no need to change close.connection() function
but we have to add a test of con->status to all *_close() functions
(gzfile_close() and co.)
You missed my point. Currently the R close() function may return -1 to signal
that there was an error closing. We can't change that to an error if packages
are using it.
May be I missed it but finally, me too, I was saying that we don't have to do
so.
Anyhow, the situation of writing to full disk cannot be passed in silence.
IMHO, trigger an error would be the most appropriate in this situation but if
for legacy
or any other reason we cannot do so, let whistle a warning, at least.
Here few tests with a new small patch:
> fc=file("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
[1] -1
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=gzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=xzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=bzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
Note that if we test only status < 0 (without errno) then there are too many
warnings
on seemingly "innocent" file closings.
Could you give an example of how to get status < 0 on a valid closing?
If you remove "&& errno" and leave only "if (status < 0)" in the previous patch
then during make I have many warnings, e.g. :
Warning messages:
1: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/compiler/Meta/nsInfo.rds' failed:
Success
2: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/compiler/Meta/package.rds'
failed: Success
3: In close(con) :
closing '/home/sokol/dev/R/R-devel/library/compiler/R/compiler.rdx' failed:
Success
4: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/Meta/nsInfo.rds' failed:
Success
5: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/Meta/package.rds' failed:
Success
6: In close(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/R/tools.rdx' failed: Success
7: In close(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/R/sysdata.rdx' failed:
Success
8: In close.connection(con) :
closing '../../library/parallel/Meta/Rd.rds' failed: Success
9: In close.connection(con) :
closing '../../library/parallel/help/aliases.rds' failed: Success
10: In close.connection(file) :
closing '../../library/parallel/DESCRIPTION' failed: Success
You are probably seeing cases where status is never set: then status is
NA_INTEGER, which (in C) is negative.
Duncan Murdoch
Note "Succes" as the reason of "failure".
And if I us
[Rd] bug: deparse sometimes omits parentheses for unary operators
When turning a language object into a string (e.g. using dput, deparse, or just printing it), R usually inserts parentheses so that the printed string is an accurate representation of the underlying AST. Example (good): (expr = substitute(a * 10, list(a = quote(if (TRUE) 1 else 0 (if (TRUE) 1 else 0) * 10 eval(expr) [1] 10 The representation is incorrect, however, when unary operators around language constructs like "if", "for" etc. are involved. Example (bad): (expr = substitute(-a * 10, list(a = quote(if (TRUE) 1 else 0 -if (TRUE) 1 else 0 * 10 The parentheses around the "if"-construct are missing. The expected output (the true representation of the expression) is, in fact: -(if (TRUE) 1 else 0) * 10 as can be tested when evaluating it (which shows that this is not a bug in "substitute"): eval(expr) [1] -10 The deparsed string on the other hand evaluates to: -if (TRUE) 1 else 0 * 10 [1] -1 Even when using deparse with control="all", which comes, per its help file, "closest to making 'deparse()' an inverse of 'parse()'", this bug persists: deparse(expr, control="all") [1] "quote(-if (TRUE) 1 else 0 * 10)" Only some language constructs appear to be affected. Unary operators applied to function calls, other unary operators, or binary operators render as expected: (expr = substitute(-a * 10, list(a = quote(!0 -(!0) * 10 I am using the R version 3.4.0 (2017-04-21) platform x86_64-redhat-linux-gnu package from Fedora 25. Regards, Martin Binder __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [Bug Fix] Default values not applied to ... arguments
I'm glad I could help, and I look forward to reading your patch so that I can
learn more about the R internals. It'd be nice to close this 4 year old bug.
Sahil
Original Message
From: Tomas Kalibera
Sent: July 7, 2017 11:10:05 AM PDT
To: Sahil Kang
Subject: Re: [Rd] [Bug Fix] Default values not applied to ... arguments
As I said the old behavior had been for several years both in AST and BC
interpreter. I fixed it in AST and I will fix it in BC. I know exactly
where it is and how to fix it, there is no more help I could use on this
issue and I don't think there is any need to discuss this further, and
particularly so on R-devel. Please do not send speculative emails to
R-devel, in the interest of saving time of people following the list.
Thanks
Tomas
On 07/07/2017 11:42 AM, Sahil Kang wrote:
> Yes, I see what you mean. My patch only disables JIT compilation for
> closures. If a user manually compiles a closure, however, the bug pops
> up again.
>
> I think the bug may either lie in how the byte-compiler compiles
> closures, or how closures with compiled body's are executed by the AST
> interpreter (without my patch applied):
>
> compiler::enableJIT(0) # turn off JIT compilation
> f <- function(...) { g(...) }
> g <- function(a=4, b=5, c=6) {
> print(c(missing(a), missing(b), missing(c)))
> b
> }
>
> f(1,,3) # works fine
> # [1] FALSE TRUE FALSE
> # [1] 5
>
> f(1,,3) # works fine
> # [1] FALSE TRUE FALSE
> # [1] 5
>
> ff_1 <- compiler::compile(f) # compile f
> ff_2 <- compiler::cmpfun(f) # compile body of closure
>
> eval(ff_1)(1,,3) # works fine
> # [1] FALSE TRUE FALSE
> # [1] 5
>
> eval(ff_2)(1,,3) # bug shows up again
> # [1] FALSE TRUE FALSE
> # Error in g(...) : argument is missing, with no default
>
> Thanks,
> Sahil
>
> On 07/06/2017 09:29 AM, Tomas Kalibera wrote:
>>
>> Thanks for the report, I've fixed 15199 in the AST interpreter in
>> 72664, I will fix it in the byte-code interpreter as well.
>>
>> If you ever needed to disable the JIT, there is API for that, see
>> ?compiler. Note though that by disabling the JIT you won't disable
>> the byte-code interpreter, because code also gets compiled when
>> packages are installed or when the compiler is invoked explicitly.
>>
>> Best,
>> Tomas
>>
>> On 07/06/2017 04:40 PM, Sahil Kang wrote:
>>> Hi Duncan, Martin
>>>
>>> Here's a small patch that fixes bug 15199 reported at:
>>> https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=15199
>>>
>>> I was able to reproduce the bug as Duncan had outlined just fine,
>>> but I did notice that when we debug(f), the problem went away.
>>> I later realized that f(1,,3) behaved correctly the first time it
>>> was executed, but misbehaved as documented on subsequent calls.
>>> This narrowed the problem down to the byte-compilation that occurs
>>> on subsequent function calls.
>>>
>>> This patch prevents byte-compilation on closure objects.
>>> Although it's a less than ideal solution, this patch fixes the bug
>>> at least until the underlying byte-compilation issue can be found
>>> (I'm currently scrutinizing the promiseArgs function at eval.c:2771).
>>>
>>> Thanks,
>>> Sahil
>>>
>>>
>>> __
>>> R-devel@r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] make check-recommended hanging on up-to-date Rdevel from SVN
> On 6 Jul 2017, at 23:26 , Gabriel Becker wrote:
>
> Hi all,
>
> I'm getting an issue with Rdevel where make check-recommended hangs
> consistently for me on Mac El Capitan when checking the Matrix package. I
> did svn update and tools/rsync_recommended earlier today and it didn't fix
> the issue.
>
> Specifically, it is hanging on the
>
> * checking dependencies in R code ...
Not happening for me on Sierra, it seems. However, I have a minimal source
install of R-devel, so that
* checking package dependencies ... NOTE
Package suggested but not available for checking: ‘expm’
Packages which this enhances but not available for checking:
‘MatrixModels’ ‘graph’ ‘SparseM’ ‘sfsmisc’
... and it is thinkable that the issue is with one of those(?)
-pd
>
>
> stage (while checking Matrix, it passes fine for MASS and lattice).
> Currently I'm getting the same behavior when I do
>
> tools:::.check_packages("/tests/RecPackages/Matrix")
>
>
> Is this a known issue and if not, do other people see this behavior?
>
> Best,
> ~G
>
>
> Various info:
>
> Build script:
>
> #!/bin/bash
>
> export CC="gcc -std=gnu99 -fsanitize=address"
> # make -O0 if we want full debugging symbols
> export CFLAGS="-fno-omit-frame-pointer -g -O2 -Wall -pedantic -mtune=native"
> #export CFLAGS="-fno-omit-frame-pointer -g -O0 -Wall -pedantic
> -mtune=native"
>
> export CXX="g++ -fsanitize=address -fno-omit-frame-pointer"
> export F77="gfortran -arch x86_64"
> export FC="gfortran -arch x86_64"
> export MAIN_LDFLAGS=" -fsanitize=address"
>
> ../checkedout/Rsource/rawtrunk/configure
> --prefix=/Users/beckerg4/local/Rdevel --enable-R-framework
> --enable-memory-profiling
> make -j3
> make install
>
> Svn Info:
>
> beckerg4-T4G3QN:rawtrunk beckerg4$ *svn info*
>
> Path: .
>
> Working Copy Root Path: /Users/beckerg4/gabe/checkedout/Rsource/rawtrunk
>
> URL: https://svn.r-project.org/R/trunk
>
> Relative URL: ^/trunk
>
> Repository Root: https://svn.r-project.org/R
>
> Repository UUID: 00db46b3-68df-0310-9c12-caf00c1e9a41
>
> Revision: 72894
>
> Node Kind: directory
>
> Schedule: normal
>
> Last Changed Author: lawrence
>
> Last Changed Rev: 72894
>
> Last Changed Date: 2017-07-06 00:12:06 -0700 (Thu, 06 Jul 2017)
>
>
>
> Svn status (no local changes)
>
> beckerg4-T4G3QN:rawtrunk beckerg4$* svn status*
>
> beckerg4-T4G3QN:rawtrunk beckerg4$
>
>
> Session info after Matrix is attached:
>
> *> library(Matrix)*
>
> *> sessionInfo()*
>
> R Under development (unstable) (2017-07-06 r72894)
>
> Platform: x86_64-apple-darwin15.6.0 (64-bit)
>
> Running under: OS X El Capitan 10.11.6
>
>
> Matrix products: default
>
> BLAS: /Users/beckerg4/gabe/Rdevelbuild/lib/libRblas.dylib
>
> LAPACK: /Users/beckerg4/gabe/Rdevelbuild/lib/libRlapack.dylib
>
>
> locale:
>
> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>
>
> attached base packages:
>
> [1] stats graphics grDevices utils datasets methods base
>
>
> other attached packages:
>
> [1] Matrix_1.2-10
>
>
> loaded via a namespace (and not attached):
>
> [1] compiler_3.5.0 grid_3.5.0 lattice_0.20-35
>
>
>
> --
> Gabriel Becker, PhD
> Associate Scientist (Bioinformatics)
> Genentech Research
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd@cbs.dk Priv: pda...@gmail.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] italic font on cairo devices in R 3.4
On Fri, Jul 07, 2017 at 07:08:52PM +0200, Ilia Kats wrote:
> Interesting. I did not have the package installed, but I did at some point
> extract Helvetica from some MacOSX font files and R was using that just fine
> until 3.3. This is how the plot looks in 3.4 (still using Helvetica):
> https://ptpb.pw/HikX.pdf . After removing Helvetica, installing the
> ttf-mscorefonts-installer package, and running fc-cache --force the plot
> looks like this: https://ptpb.pw/CM8A.pdf
The second plot looks worse, in other words, and the Microsoft fonts
didn't help.
Maybe the Cairo device should be giving better warning messages.
Anyway it sounds like you are describing a regression so maybe someone
can track down the commit that created this problem.
Thanks,
Frederick
> Also note that the standard pdf device works fine: https://ptpb.pw/3Ml1.pdf
> , it's just the cairo devices (both pdf and svg) that have the issue.
> Unfortunately I need to use cairo_pdf due to unicode characters in axis
> labels.
>
> Cheers, Ilia
>
>
> Original Message
> Subject: Re: [Rd] italic font on cairo devices in R 3.4
> Date: 2017-07-07 18:17:34 +0200
> From: frederik
> To: Ilia Kats
> CC: r-devel, r-help
> > Hi Ilia,
> >
> > I'm running Arch Linux, R 3.4.0.
> >
> > Here's my test.pdf from your minimal example: https://ptpb.pw/HxsA.pdf
> >
> > It doesn't look pixelated to me...
> >
> > Here's a post that I wrote when I solved my last font problem in R,
> > almost 2 years ago:
> >
> > https://stackoverflow.com/a/40940331/5087283
> >
> > I had to install some Microsoft font packages, which is sad, because
> > there are some perfectly good free fonts that R could be using
> > instead. It could be considered a bug that R requires Microsoft fonts,
> > at least by default. However, does this even fix your problem? I.e. if
> > you install the corresponding Debian Microsoft font packages, does the
> > text appear anti-aliased?
> >
> > Frederick
> >
> > On Fri, Jul 07, 2017 at 10:30:46AM +0200, Ilia Kats wrote:
> > > [cross-post from R-help]
> > >
> > > Hi all,
> > >
> > > I have the following problem: Since R 3.4.0, italic fonts rendered on
> > > Cairo
> > > devices appear pixelated. Here's a minimal example:
> > > cairo_pdf('test.pdf')
> > > plot(1:10, ylab=expression(italic(test)))
> > > dev.off()
> > >
> > > The same problem occurs with bolditalic, but not bold. I am using Debian
> > > Stretch. Several friends tried the same on their machines, another Debian
> > > machine has the same problem. On MacOSX the output was not pixelated, but
> > > it
> > > wasn't italic either. Ubuntu 16.04.2 xenial works fine. My impression is
> > > that R somehow can't find the proper font to use and falls back to
> > > something
> > > weird. Ideas?
> > >
> > > Note that I'm not subscribed to the list, so please CC me in replies.
> > >
> > > Cheers, Ilia
> > >
> > > __
> > > R-devel@r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-devel
> > >
>
> --
> The first is to ensure your partner understands that nature has root
> privileges - nature doesn't have to make sense.
> -- Telsa Gwynne
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] italic font on cairo devices in R 3.4
Interesting. I did not have the package installed, but I did at some
point extract Helvetica from some MacOSX font files and R was using that
just fine until 3.3. This is how the plot looks in 3.4 (still using
Helvetica): https://ptpb.pw/HikX.pdf . After removing Helvetica,
installing the ttf-mscorefonts-installer package, and running fc-cache
--force the plot looks like this: https://ptpb.pw/CM8A.pdf
Also note that the standard pdf device works fine:
https://ptpb.pw/3Ml1.pdf , it's just the cairo devices (both pdf and
svg) that have the issue. Unfortunately I need to use cairo_pdf due to
unicode characters in axis labels.
Cheers, Ilia
Original Message
Subject: Re: [Rd] italic font on cairo devices in R 3.4
Date: 2017-07-07 18:17:34 +0200
From: frederik
To: Ilia Kats
CC: r-devel, r-help
Hi Ilia,
I'm running Arch Linux, R 3.4.0.
Here's my test.pdf from your minimal example: https://ptpb.pw/HxsA.pdf
It doesn't look pixelated to me...
Here's a post that I wrote when I solved my last font problem in R,
almost 2 years ago:
https://stackoverflow.com/a/40940331/5087283
I had to install some Microsoft font packages, which is sad, because
there are some perfectly good free fonts that R could be using
instead. It could be considered a bug that R requires Microsoft fonts,
at least by default. However, does this even fix your problem? I.e. if
you install the corresponding Debian Microsoft font packages, does the
text appear anti-aliased?
Frederick
On Fri, Jul 07, 2017 at 10:30:46AM +0200, Ilia Kats wrote:
[cross-post from R-help]
Hi all,
I have the following problem: Since R 3.4.0, italic fonts rendered on Cairo
devices appear pixelated. Here's a minimal example:
cairo_pdf('test.pdf')
plot(1:10, ylab=expression(italic(test)))
dev.off()
The same problem occurs with bolditalic, but not bold. I am using Debian
Stretch. Several friends tried the same on their machines, another Debian
machine has the same problem. On MacOSX the output was not pixelated, but it
wasn't italic either. Ubuntu 16.04.2 xenial works fine. My impression is
that R somehow can't find the proper font to use and falls back to something
weird. Ideas?
Note that I'm not subscribed to the list, so please CC me in replies.
Cheers, Ilia
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
The first is to ensure your partner understands that nature has root
privileges - nature doesn't have to make sense.
-- Telsa Gwynne
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] italic font on cairo devices in R 3.4
Hi Ilia,
I'm running Arch Linux, R 3.4.0.
Here's my test.pdf from your minimal example: https://ptpb.pw/HxsA.pdf
It doesn't look pixelated to me...
Here's a post that I wrote when I solved my last font problem in R,
almost 2 years ago:
https://stackoverflow.com/a/40940331/5087283
I had to install some Microsoft font packages, which is sad, because
there are some perfectly good free fonts that R could be using
instead. It could be considered a bug that R requires Microsoft fonts,
at least by default. However, does this even fix your problem? I.e. if
you install the corresponding Debian Microsoft font packages, does the
text appear anti-aliased?
Frederick
On Fri, Jul 07, 2017 at 10:30:46AM +0200, Ilia Kats wrote:
> [cross-post from R-help]
>
> Hi all,
>
> I have the following problem: Since R 3.4.0, italic fonts rendered on Cairo
> devices appear pixelated. Here's a minimal example:
> cairo_pdf('test.pdf')
> plot(1:10, ylab=expression(italic(test)))
> dev.off()
>
> The same problem occurs with bolditalic, but not bold. I am using Debian
> Stretch. Several friends tried the same on their machines, another Debian
> machine has the same problem. On MacOSX the output was not pixelated, but it
> wasn't italic either. Ubuntu 16.04.2 xenial works fine. My impression is
> that R somehow can't find the proper font to use and falls back to something
> weird. Ideas?
>
> Note that I'm not subscribed to the list, so please CC me in replies.
>
> Cheers, Ilia
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [New Patch] Fix disk corruption when writing
On 07/07/2017 11:13 AM, Serguei Sokol wrote:
Le 07/07/2017 à 16:52, Duncan Murdoch a écrit :
On 07/07/2017 9:54 AM, Serguei Sokol wrote:
Le 07/07/2017 à 01:09, Duncan Murdoch a écrit :
On 06/07/2017 6:44 PM, Sokol Serguei wrote:
Duncan Murdoch has written at Thu, 6 Jul 2017 13:58:10 -0400
On 06/07/2017 5:21 AM, Serguei Sokol wrote:
I propose the following patch against the current
R-devel/src/main/connection.c (cf. attached file).
It gives (on my linux box):
> fc=file("/dev/full", "w")
> write.csv("a", file=fc)
Error in writeLines(paste(col.names, collapse = sep), file, sep = eol) :
system call failure on writing
> close(fc)
Serguei.
I suspect that flushing on every write will slow things down too much.
That's quite plausible.
I think a better approach is to catch the error in the Rconn_printf
calls (as R-devel currently does), and also catch errors on
con->close(con). This one requires more changes to the source, so it
may be a day or two before I commit.
I have testes it on file_close() and it works (cf. attached patch):
fc=file("/dev/full", "w")
write.csv("a", file=fc)
close(fc)
Error in close.connection(fc) : closing file failed
One thing I have to look into: is anyone making use of the fact that
the R-level close.connection() function can return -1 to signal an
error? Base R ignores that, which is why we need to do something, but
if packages are using it, things need to be changed carefully. I
can't just change it to raise an error instead.
As you can see in the patch, no need to change close.connection() function
but we have to add a test of con->status to all *_close() functions
(gzfile_close() and co.)
You missed my point. Currently the R close() function may return -1 to signal
that there was an error closing. We can't change that to an error if packages
are using it.
May be I missed it but finally, me too, I was saying that we don't have to do
so.
Anyhow, the situation of writing to full disk cannot be passed in silence.
IMHO, trigger an error would be the most appropriate in this situation but if
for legacy
or any other reason we cannot do so, let whistle a warning, at least.
Here few tests with a new small patch:
> fc=file("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
[1] -1
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=gzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=xzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=bzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
Note that if we test only status < 0 (without errno) then there are too many
warnings
on seemingly "innocent" file closings.
Could you give an example of how to get status < 0 on a valid closing?
If you remove "&& errno" and leave only "if (status < 0)" in the previous patch
then during make I have many warnings, e.g. :
Warning messages:
1: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/compiler/Meta/nsInfo.rds' failed:
Success
2: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/compiler/Meta/package.rds'
failed: Success
3: In close(con) :
closing '/home/sokol/dev/R/R-devel/library/compiler/R/compiler.rdx' failed:
Success
4: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/Meta/nsInfo.rds' failed:
Success
5: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/Meta/package.rds' failed:
Success
6: In close(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/R/tools.rdx' failed: Success
7: In close(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/R/sysdata.rdx' failed:
Success
8: In close.connection(con) :
closing '../../library/parallel/Meta/Rd.rds' failed: Success
9: In close.connection(con) :
closing '../../library/parallel/help/aliases.rds' failed: Success
10: In close.connection(file) :
closing '../../library/parallel/DESCRIPTION' failed: Success
You are probably seeing cases where status is never set: then status is
NA_INTEGER, which (in C) is negative.
Duncan Murdoch
Note "Succes" as the reason of "failure".
And if I use thus compiled R, at startup I get:
Warning message:
In close(con) :
closing '/home/sokol/dev/R/R-devel/library/base/R/base.rdx' failed: Success
R Under development (unstable) (2017-06-01 r72753) -- "Unsuffered Consequences"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' fo
Re: [Rd] [New Patch] Fix disk corruption when writing
Le 07/07/2017 à 16:52, Duncan Murdoch a écrit :
On 07/07/2017 9:54 AM, Serguei Sokol wrote:
Le 07/07/2017 à 01:09, Duncan Murdoch a écrit :
On 06/07/2017 6:44 PM, Sokol Serguei wrote:
Duncan Murdoch has written at Thu, 6 Jul 2017 13:58:10 -0400
On 06/07/2017 5:21 AM, Serguei Sokol wrote:
I propose the following patch against the current
R-devel/src/main/connection.c (cf. attached file).
It gives (on my linux box):
> fc=file("/dev/full", "w")
> write.csv("a", file=fc)
Error in writeLines(paste(col.names, collapse = sep), file, sep = eol) :
system call failure on writing
> close(fc)
Serguei.
I suspect that flushing on every write will slow things down too much.
That's quite plausible.
I think a better approach is to catch the error in the Rconn_printf
calls (as R-devel currently does), and also catch errors on
con->close(con). This one requires more changes to the source, so it
may be a day or two before I commit.
I have testes it on file_close() and it works (cf. attached patch):
fc=file("/dev/full", "w")
write.csv("a", file=fc)
close(fc)
Error in close.connection(fc) : closing file failed
One thing I have to look into: is anyone making use of the fact that
the R-level close.connection() function can return -1 to signal an
error? Base R ignores that, which is why we need to do something, but
if packages are using it, things need to be changed carefully. I
can't just change it to raise an error instead.
As you can see in the patch, no need to change close.connection() function
but we have to add a test of con->status to all *_close() functions
(gzfile_close() and co.)
You missed my point. Currently the R close() function may return -1 to signal
that there was an error closing. We can't change that to an error if packages
are using it.
May be I missed it but finally, me too, I was saying that we don't have to do
so.
Anyhow, the situation of writing to full disk cannot be passed in silence.
IMHO, trigger an error would be the most appropriate in this situation but if
for legacy
or any other reason we cannot do so, let whistle a warning, at least.
Here few tests with a new small patch:
> fc=file("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
[1] -1
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=gzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=xzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=bzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
Note that if we test only status < 0 (without errno) then there are too many
warnings
on seemingly "innocent" file closings.
Could you give an example of how to get status < 0 on a valid closing?
If you remove "&& errno" and leave only "if (status < 0)" in the previous patch
then during make I have many warnings, e.g. :
Warning messages:
1: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/compiler/Meta/nsInfo.rds' failed:
Success
2: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/compiler/Meta/package.rds' failed:
Success
3: In close(con) :
closing '/home/sokol/dev/R/R-devel/library/compiler/R/compiler.rdx' failed:
Success
4: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/Meta/nsInfo.rds' failed:
Success
5: In close.connection(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/Meta/package.rds' failed:
Success
6: In close(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/R/tools.rdx' failed: Success
7: In close(con) :
closing '/home/sokol/dev/R/R-devel/library/tools/R/sysdata.rdx' failed:
Success
8: In close.connection(con) :
closing '../../library/parallel/Meta/Rd.rds' failed: Success
9: In close.connection(con) :
closing '../../library/parallel/help/aliases.rds' failed: Success
10: In close.connection(file) :
closing '../../library/parallel/DESCRIPTION' failed: Success
Note "Succes" as the reason of "failure".
And if I use thus compiled R, at startup I get:
Warning message:
In close(con) :
closing '/home/sokol/dev/R/R-devel/library/base/R/base.rdx' failed: Success
R Under development (unstable) (2017-06-01 r72753) -- "Unsuffered Consequences"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Typ
Re: [Rd] [New Patch] Fix disk corruption when writing
On 07/07/2017 9:54 AM, Serguei Sokol wrote:
Le 07/07/2017 à 01:09, Duncan Murdoch a écrit :
On 06/07/2017 6:44 PM, Sokol Serguei wrote:
Duncan Murdoch has written at Thu, 6 Jul 2017 13:58:10 -0400
On 06/07/2017 5:21 AM, Serguei Sokol wrote:
I propose the following patch against the current
R-devel/src/main/connection.c (cf. attached file).
It gives (on my linux box):
> fc=file("/dev/full", "w")
> write.csv("a", file=fc)
Error in writeLines(paste(col.names, collapse = sep), file, sep = eol) :
system call failure on writing
> close(fc)
Serguei.
I suspect that flushing on every write will slow things down too much.
That's quite plausible.
I think a better approach is to catch the error in the Rconn_printf
calls (as R-devel currently does), and also catch errors on
con->close(con). This one requires more changes to the source, so it
may be a day or two before I commit.
I have testes it on file_close() and it works (cf. attached patch):
fc=file("/dev/full", "w")
write.csv("a", file=fc)
close(fc)
Error in close.connection(fc) : closing file failed
One thing I have to look into: is anyone making use of the fact that
the R-level close.connection() function can return -1 to signal an
error? Base R ignores that, which is why we need to do something, but
if packages are using it, things need to be changed carefully. I
can't just change it to raise an error instead.
As you can see in the patch, no need to change close.connection() function
but we have to add a test of con->status to all *_close() functions
(gzfile_close() and co.)
You missed my point. Currently the R close() function may return -1 to signal
that there was an error closing. We can't change that to an error if packages
are using it.
May be I missed it but finally, me too, I was saying that we don't have to do
so.
Anyhow, the situation of writing to full disk cannot be passed in silence.
IMHO, trigger an error would be the most appropriate in this situation but if
for legacy
or any other reason we cannot do so, let whistle a warning, at least.
Here few tests with a new small patch:
> fc=file("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
[1] -1
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=gzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=xzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=bzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
Note that if we test only status < 0 (without errno) then there are too many
warnings
on seemingly "innocent" file closings.
Could you give an example of how to get status < 0 on a valid closing?
Duncan Murdoch
Serguei.
Le 05/07/2017 à 15:33, Serguei Sokol a écrit :
Le 05/07/2017 à 14:46, Serguei Sokol a écrit :
Le 05/07/2017 à 13:09, Duncan Murdoch a écrit :
On 05/07/2017 5:26 AM, January W. wrote:
I tried the newest patch, but it does not seem to work for me (on
Linux). Despite the check in Rconn_printf, the write.csv happily
writes
to /dev/full and does not report an error. When I added a
printf("%d\n",
res); to the Rconn_printf() definition, I see only positive values
returned by the vfprintf call.
That's likely because you aren't writing enough to actually
trigger a write to disk during the write. Writes are buffered,
and the error doesn't happen
until the buffer is written.
I can confirm this behavior with fvprintf(). Small and medium sized
writings
on /dev/full don't trigger error and 1MB does.
But if fprintf() is used, it returns a negative value from the very
first byte written.
I correct myself. In my test, fprintf() returned -1 for another
reason (connection was already closed
at this moment)
However, if fvprintf(...) is followed by res=fflush(con) then res is -1
if we try to write on /dev/full. May be we have to use this to trigger
an error message in R.
Serguei.
The regression test I put in had this problem; I'm working on
MacOS and Windows, so I never got to actually try it before
committing.
Unfortunately, it doesn't look possible to catch the final flush
of the buffer when the connection is closed, so small writes won't
trigger any error.
It's also possible that whatever system you're on doesn't signal
an error when the write fails.
Duncan Murdoch
Cheers,
j.
On 4 July 2017 at 21:37, Duncan Murdoch mailto:murdoch.dun...@gmail.com>> wrote:
On 04/07/2017 11:50 AM, Jean-Sébastien Bevilacqua wrote:
Hello,
You can find here a patch to fix disk corruption.
When your disk is full, the write function exit without
error
but the file
is truncated.
https://bugs.r-
Re: [Rd] [New Patch] Fix disk corruption when writing
Le 07/07/2017 à 01:09, Duncan Murdoch a écrit :
On 06/07/2017 6:44 PM, Sokol Serguei wrote:
Duncan Murdoch has written at Thu, 6 Jul 2017 13:58:10 -0400
On 06/07/2017 5:21 AM, Serguei Sokol wrote:
I propose the following patch against the current
R-devel/src/main/connection.c (cf. attached file).
It gives (on my linux box):
> fc=file("/dev/full", "w")
> write.csv("a", file=fc)
Error in writeLines(paste(col.names, collapse = sep), file, sep = eol) :
system call failure on writing
> close(fc)
Serguei.
I suspect that flushing on every write will slow things down too much.
That's quite plausible.
I think a better approach is to catch the error in the Rconn_printf
calls (as R-devel currently does), and also catch errors on
con->close(con). This one requires more changes to the source, so it
may be a day or two before I commit.
I have testes it on file_close() and it works (cf. attached patch):
fc=file("/dev/full", "w")
write.csv("a", file=fc)
close(fc)
Error in close.connection(fc) : closing file failed
One thing I have to look into: is anyone making use of the fact that
the R-level close.connection() function can return -1 to signal an
error? Base R ignores that, which is why we need to do something, but
if packages are using it, things need to be changed carefully. I
can't just change it to raise an error instead.
As you can see in the patch, no need to change close.connection() function
but we have to add a test of con->status to all *_close() functions
(gzfile_close() and co.)
You missed my point. Currently the R close() function may return -1 to signal that there was an error closing. We can't change that to an error if packages
are using it.
May be I missed it but finally, me too, I was saying that we don't have to do
so.
Anyhow, the situation of writing to full disk cannot be passed in silence.
IMHO, trigger an error would be the most appropriate in this situation but if
for legacy
or any other reason we cannot do so, let whistle a warning, at least.
Here few tests with a new small patch:
> fc=file("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
[1] -1
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=gzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=xzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
> fc=bzfile("/dev/full", "w"); write.csv("a", file=fc); (res=close(fc))
NULL
Warning message:
In close.connection(fc) :
closing '/dev/full' failed: No space left on device
Note that if we test only status < 0 (without errno) then there are too many
warnings
on seemingly "innocent" file closings.
Serguei.
Le 05/07/2017 à 15:33, Serguei Sokol a écrit :
Le 05/07/2017 à 14:46, Serguei Sokol a écrit :
Le 05/07/2017 à 13:09, Duncan Murdoch a écrit :
On 05/07/2017 5:26 AM, January W. wrote:
I tried the newest patch, but it does not seem to work for me (on
Linux). Despite the check in Rconn_printf, the write.csv happily
writes
to /dev/full and does not report an error. When I added a
printf("%d\n",
res); to the Rconn_printf() definition, I see only positive values
returned by the vfprintf call.
That's likely because you aren't writing enough to actually
trigger a write to disk during the write. Writes are buffered,
and the error doesn't happen
until the buffer is written.
I can confirm this behavior with fvprintf(). Small and medium sized
writings
on /dev/full don't trigger error and 1MB does.
But if fprintf() is used, it returns a negative value from the very
first byte written.
I correct myself. In my test, fprintf() returned -1 for another
reason (connection was already closed
at this moment)
However, if fvprintf(...) is followed by res=fflush(con) then res is -1
if we try to write on /dev/full. May be we have to use this to trigger
an error message in R.
Serguei.
The regression test I put in had this problem; I'm working on
MacOS and Windows, so I never got to actually try it before
committing.
Unfortunately, it doesn't look possible to catch the final flush
of the buffer when the connection is closed, so small writes won't
trigger any error.
It's also possible that whatever system you're on doesn't signal
an error when the write fails.
Duncan Murdoch
Cheers,
j.
On 4 July 2017 at 21:37, Duncan Murdoch mailto:murdoch.dun...@gmail.com>> wrote:
On 04/07/2017 11:50 AM, Jean-Sébastien Bevilacqua wrote:
Hello,
You can find here a patch to fix disk corruption.
When your disk is full, the write function exit without
error
but the file
is truncated.
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17243
Thanks. I didn't see that when it
Re: [Rd] attributes on symbols
Here is a simpler example:
ex <- as.name("a")
attr(ex, "test") <- 1
quote(a)
a
attr(,"test")
[1] 1
Torsten
On Thu, 6 Jul 2017, William Dunlap wrote:
The multcomp package has code in multcomp:::expression2coef that attaches the
'coef' attribute to
symbols. Since there is only one symbol object in a session with a given name,
this means that
this attaching has a global effect. Should this be quietly allowed or should
there be a warning or
an error?
E.g.,
str(quote(Education))
# symbol Education
lmod <- stats::lm(Fertility ~ ., data = datasets::swiss)
glmod <- multcomp::glht(lmod, c("Agriculture=0", "Education=0"))
str(quote(Education))
# symbol Education
# - attr(*, "coef")= num 1
Bill Dunlap
TIBCO Software
wdunlap tibco.com
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] italic font on cairo devices in R 3.4
[cross-post from R-help]
Hi all,
I have the following problem: Since R 3.4.0, italic fonts rendered on
Cairo devices appear pixelated. Here's a minimal example:
cairo_pdf('test.pdf')
plot(1:10, ylab=expression(italic(test)))
dev.off()
The same problem occurs with bolditalic, but not bold. I am using Debian
Stretch. Several friends tried the same on their machines, another
Debian machine has the same problem. On MacOSX the output was not
pixelated, but it wasn't italic either. Ubuntu 16.04.2 xenial works
fine. My impression is that R somehow can't find the proper font to use
and falls back to something weird. Ideas?
Note that I'm not subscribed to the list, so please CC me in replies.
Cheers, Ilia
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [Bug Fix] Default values not applied to ... arguments
Yes, I see what you mean. My patch only disables JIT compilation for
closures. If a user manually compiles a closure, however, the bug pops
up again.
I think the bug may either lie in how the byte-compiler compiles
closures, or how closures with compiled body's are executed by the AST
interpreter (without my patch applied):
compiler::enableJIT(0) # turn off JIT compilation
f <- function(...) { g(...) }
g <- function(a=4, b=5, c=6) {
print(c(missing(a), missing(b), missing(c)))
b
}
f(1,,3) # works fine
# [1] FALSE TRUE FALSE
# [1] 5
f(1,,3) # works fine
# [1] FALSE TRUE FALSE
# [1] 5
ff_1 <- compiler::compile(f) # compile f
ff_2 <- compiler::cmpfun(f) # compile body of closure
eval(ff_1)(1,,3) # works fine
# [1] FALSE TRUE FALSE
# [1] 5
eval(ff_2)(1,,3) # bug shows up again
# [1] FALSE TRUE FALSE
# Error in g(...) : argument is missing, with no default
Thanks,
Sahil
On 07/06/2017 09:29 AM, Tomas Kalibera wrote:
>
> Thanks for the report, I've fixed 15199 in the AST interpreter in
> 72664, I will fix it in the byte-code interpreter as well.
>
> If you ever needed to disable the JIT, there is API for that, see
> ?compiler. Note though that by disabling the JIT you won't disable the
> byte-code interpreter, because code also gets compiled when packages
> are installed or when the compiler is invoked explicitly.
>
> Best,
> Tomas
>
> On 07/06/2017 04:40 PM, Sahil Kang wrote:
>> Hi Duncan, Martin
>>
>> Here's a small patch that fixes bug 15199 reported at:
>> https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=15199
>>
>> I was able to reproduce the bug as Duncan had outlined just fine, but
>> I did notice that when we debug(f), the problem went away.
>> I later realized that f(1,,3) behaved correctly the first time it was
>> executed, but misbehaved as documented on subsequent calls.
>> This narrowed the problem down to the byte-compilation that occurs on
>> subsequent function calls.
>>
>> This patch prevents byte-compilation on closure objects.
>> Although it's a less than ideal solution, this patch fixes the bug at
>> least until the underlying byte-compilation issue can be found (I'm
>> currently scrutinizing the promiseArgs function at eval.c:2771).
>>
>> Thanks,
>> Sahil
>>
>>
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
