[Rd] capture.output(): Using a rawConnection() [linear] instead of textConnection() [exponential]?
I've noticed that the processing time for the default capture.output()
grows exponentially in the number of characters outputted/captured.
The default settings sinks to a temporary textConnection(). When
instead sinking to a rawConnection(), the processing time becomes
linear. See below example and attached PNG figure [also at
http://xfer.aroma-project.org/tmp/20140203/capture.output_text-vs-raw.png].
I know little about text encoding, but I wouldn't be surprised I'm
overseeing encoding issues when sinking to raw followed by
rawToChar(). According to the below example, both approaches seem to
work with my current setup (sessionInfo() below), but on the other
hand it's only writing an ASCII "a" character multiple times.
Q. Is it possible to utilize rawConnection() instead of
textConnection() and still get the same output regardless of
encodings?
Q. What causes textConnection() to be exponential when rawConnection()
is linear?
REPRODUCIBLE EXAMPLE:
captureViaText <- function(..., local=TRUE, envir=parent.frame()) {
eval({
# Basically what capture.output() does by default
file <- textConnection("rval", open="w", local=local)
capture.output(..., file=file)
close(file)
rval
}, envir=envir)
} # captureViaText()
captureViaRaw <- function(..., local=TRUE, envir=parent.frame()) {
eval({
# Basically what capture.output() does by default
file <- rawConnection(raw(0L), open="w")
capture.output(..., file=file)
res <- rawConnectionValue(file)
close(file)
res <- rawToChar(res)
res <- unlist(strsplit(res, split="\n", fixed=TRUE))
res
}, envir=envir)
} # captureViaRaw()
data <- data.frame(size=10^seq(from=3, to=7.9, by=0.1),
captureViaText=NA, captureViaRaw=NA)
names <- colnames(data)[-1]
for (kk in which(is.na(data[[ncol(data)]]))) {
# String to output
x <- rep("a", times=data$size[kk])
x[seq(from=1L, to=length(x), by=1e3)] <- "\n"
x <- paste(x, collapse="")
cat(sprintf("Output size: %d characters\n", nchar(x)))
# Benchmark
values <- list()
for (name in names) {
print(name)
fcn <- get(name, mode="function")
t <- system.time({
res <- fcn(cat(x))
}, gcFirst=TRUE)
n <- sum(nchar(res))
values[[name]] <- res
rm(list="res")
cat(sprintf("%s: %.2g seconds to capture %d characters\n", name, t[3], n))
data[kk,name] <- t[3]
} # for (name ...)
# Assert same captures
for (cc in seq_along(values)) stopifnot(identical(values[[cc]], values[[1]]))
rm(list="values")
} # for (size ...)
local({
png("capture.output_text-vs-raw.png", width=840, height=840)
on.exit(dev.off())
plot(data[,c(1,2)], type="n", xlab="Number of characters", ylab="Time (s)")
for (cc in 2:ncol(data)) lines(data[,c(1,cc)], lwd=2, col=cc)
legend("topleft", lwd=2, col=2:ncol(data), names, bty="n")
})
> sessionInfo()
R Under development (unstable) (2014-01-30 r64901)
Platform: x86_64-w64-mingw32/x64 (64-bit)
locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] tools_3.1.0
Thanks,
Henrik
<>__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] String Vector Encoding
On Feb 3, 2014, at 10:06 AM, Saptarshi Guha wrote: > True but I was hoping someone could point me where in the source code this > happens. > src/main/serialize.c @1655 and @1663 > Cheers > Saptarshi > On Feb 3, 2014 3:37 AM, "Duncan Murdoch" wrote: > >> On 14-02-03 3:41 AM, Saptarshi Guha wrote: >> >>> Hello, >>> >>> I was reading through serialize.c and i couldn't answer something. >>> >>> In readItem, case CHARSXP, rules exists to adjust the read string for >>> string encoding. >>> >> >> This is described in the R Internals manual. >> >> Duncan Murdoch >> >> >>> Q1. I couldn't find where the encoding of the elements of the string >>> vector >>> are written? Is it when writeItem writes out the attributes of the item? >>> But i couldn't find encoding in the attributes field >>> >>> x <- "fa\xE7ile" >>> Encoding(x) <- "latin1" >>> xx <- iconv(x, "latin1", "UTF-8") >>> y=c(x,xx) >>> attributes(y) >>> >>> NULL >>> >>> Apologies if this has been asked before >>> Regards >>> Saptarshi >>> >>>[[alternative HTML version deleted]] >>> >>> __ >>> R-devel@r-project.org mailing list >>> https://stat.ethz.ch/mailman/listinfo/r-devel >>> >>> >> > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] seq range argument
This is slightly more verbose, but perhaps
do.call("seq", as.list(c(extendrange(D_orig, f=0.1), len=100)))
Best,
Ista
On Mon, Feb 3, 2014 at 9:00 AM, Lorenz, David wrote:
> Berry,
> It sounds like you just need a little helper function like this:
>
> ser <- function(x, len=100, f=0.1) {
>dr <- extendrange(x, f=f)
>return(seq(dr[1L], dr[2L], length.out=len))
> }
>
> I called it ser, short for sequence extended range. Use it thusly:
>
> Ijustneed <- ser(D_orig)
>
> Hope this helps. I use and create little help functions like this all the
> time. Even extendrange could be considered a helper function as it is only
> a couple of lines long.
> Dave
>
>
>
> On Mon, Feb 3, 2014 at 6:43 AM, Berry Boessenkool <
> berryboessenk...@hotmail.com> wrote:
>
>> Hello dear developers,
>>
>> I find myself often having the result of "range" oder "extendrange", which
>> I want to create a sequence with.
>> But "seq" needs two seperate arguments "from" and "two".
>> Could an argument "range" be added?
>>
>> Otherwise I will have to create an object with the range (may come from a
>> longer calculation), index twice from it and remove it again - kind of an
>> unnecessary bunch of code, I think.
>>
>> Here's an example:
>>
>> # What I currently do:
>> D_orig <- rnorm(20, sd=30)
>> D_range <- extendrange(D_orig, f=0.1)
>> Ijustneed <- seq(D_range[1], D_range[2], len=100)
>> rm(D_range)
>>
>> # what I'd like to have instead:
>> D_orig <- rnorm(20, sd=30)
>> Ijustneed <- seq(range=extendrange(D_orig, f=0.1), len=100)
>>
>> regards,
>> Berry
>>
>> -
>> Berry Boessenkool
>> Potsdam, Germany
>> -
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
> [[alternative HTML version deleted]]
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] seq range argument
Thanks Dave!
I settled on something similar, but more general, for my function collection
package.
I just thought this could be wanted by more people and thus interesting for
base::seq.
In case anyone is interested in my solution, keep reading.
Berry
# Add a range argument to seq
seqr <- function(from=1, to=1, range, ...)
{
# Input checking:
if(!is.vector(range)) stop("'range' must be a vector.")
if(!is.numeric(range)) stop("'range' must be numeric.")
# only set from and to if range is given as input:
if(!missing(range))
{
from <- range[1] # first
to <- tail(range,1) # and last value
if(length(range)>2L)
{
from <- min(range, finite=TRUE) # min
to <- max(range, finite=TRUE) # and max
}
}
# now call seq with from and to (obtained from range)
seq(from=from, to=to, ...)
}
# Examples
m <- c(41, 12, 38, 29, 50, 39, 22)
seqr(range=extendrange(m, f=0.1), len=5)
# Takes min and max of range if thae vector has more than two elements.
seqr(range=m, by=3)
---
From: lor...@usgs.gov
Date: Mon, 3 Feb 2014 08:00:41 -0600
Subject: Re: [Rd] seq range argument
To: berryboessenk...@hotmail.com
CC: r-devel@r-project.org
Berry, It sounds like you just need a little helper function like this:
ser <- function(x, len=100, f=0.1) { dr <- extendrange(x, f=f)
return(seq(dr[1L], dr[2L], length.out=len))
}
I called it ser, short for sequence extended range. Use it thusly:
Ijustneed <- ser(D_orig)
Hope this helps. I use and create little help functions like this all the
time. Even extendrange could be considered a helper function as it is only a
couple of lines long.
Dave
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] String Vector Encoding
True but I was hoping someone could point me where in the source code this happens. Cheers Saptarshi On Feb 3, 2014 3:37 AM, "Duncan Murdoch" wrote: > On 14-02-03 3:41 AM, Saptarshi Guha wrote: > >> Hello, >> >> I was reading through serialize.c and i couldn't answer something. >> >> In readItem, case CHARSXP, rules exists to adjust the read string for >> string encoding. >> > > This is described in the R Internals manual. > > Duncan Murdoch > > >> Q1. I couldn't find where the encoding of the elements of the string >> vector >> are written? Is it when writeItem writes out the attributes of the item? >> But i couldn't find encoding in the attributes field >> >> x <- "fa\xE7ile" >> Encoding(x) <- "latin1" >> xx <- iconv(x, "latin1", "UTF-8") >> y=c(x,xx) >> attributes(y) >> >> NULL >> >> Apologies if this has been asked before >> Regards >> Saptarshi >> >> [[alternative HTML version deleted]] >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> >> > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] seq range argument
Berry,
It sounds like you just need a little helper function like this:
ser <- function(x, len=100, f=0.1) {
dr <- extendrange(x, f=f)
return(seq(dr[1L], dr[2L], length.out=len))
}
I called it ser, short for sequence extended range. Use it thusly:
Ijustneed <- ser(D_orig)
Hope this helps. I use and create little help functions like this all the
time. Even extendrange could be considered a helper function as it is only
a couple of lines long.
Dave
On Mon, Feb 3, 2014 at 6:43 AM, Berry Boessenkool <
berryboessenk...@hotmail.com> wrote:
> Hello dear developers,
>
> I find myself often having the result of "range" oder "extendrange", which
> I want to create a sequence with.
> But "seq" needs two seperate arguments "from" and "two".
> Could an argument "range" be added?
>
> Otherwise I will have to create an object with the range (may come from a
> longer calculation), index twice from it and remove it again - kind of an
> unnecessary bunch of code, I think.
>
> Here's an example:
>
> # What I currently do:
> D_orig <- rnorm(20, sd=30)
> D_range <- extendrange(D_orig, f=0.1)
> Ijustneed <- seq(D_range[1], D_range[2], len=100)
> rm(D_range)
>
> # what I'd like to have instead:
> D_orig <- rnorm(20, sd=30)
> Ijustneed <- seq(range=extendrange(D_orig, f=0.1), len=100)
>
> regards,
> Berry
>
> -
> Berry Boessenkool
> Potsdam, Germany
> -
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Release plans for R versions 3.0.3 and 3.1.0
The upcoming wrap-up release 3.0.3 is scheduled for March 6. The start of the next series, 3.1.0 is scheduled for April 10. Detailed schedules will be published on developer.r-project.org. Package maintainers, especially of recommended packages, are asked to time potential modifications to avoid collisions with the release process. For the Core Team Peter D. -- Peter Dalgaard, Professor Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] seq range argument
Hello dear developers, I find myself often having the result of "range" oder "extendrange", which I want to create a sequence with. But "seq" needs two seperate arguments "from" and "two". Could an argument "range" be added? Otherwise I will have to create an object with the range (may come from a longer calculation), index twice from it and remove it again - kind of an unnecessary bunch of code, I think. Here's an example: # What I currently do: D_orig <- rnorm(20, sd=30) D_range <- extendrange(D_orig, f=0.1) Ijustneed <- seq(D_range[1], D_range[2], len=100) rm(D_range) # what I'd like to have instead: D_orig <- rnorm(20, sd=30) Ijustneed <- seq(range=extendrange(D_orig, f=0.1), len=100) regards, Berry - Berry Boessenkool Potsdam, Germany - __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Fwd: linker issue
On 14-02-03 7:00 AM, Samuel Kemp wrote: Hi, I am trying to compile C++ with R and I am coming up against - what I believe - is a linker configuration issue. For example, using has_develop() gives the following message... has_devel() I assume you're using the function from devtools. It tests whether you have your development environment set up correctly for use with devtools, and this result shows you don't. You might try using the base R tools (e.g. R CMD SHLIB from the command line) to see if this is a devtools problem or more basic. If it's a devtools problem, you should contact the maintainer of that package. If it's not just a devtools problem and you still need help, you need to describe what you did in more detail. What is your PATH? What about other environment variables? What version of R are you using? What version of Rtools? Duncan Murdoch "C:/PROGRA~1/R/R-30~1.2/bin/i386/R" --vanilla CMD SHLIB foo.c gcc -m32 -I"C:/PROGRA~1/R/R-30~1.2/include" -DNDEBUG -I"d:/RCompile/CRANpkg/extralibs64/local/include" -O3 -Wall -std=gnu99 -mtune=core2 -c foo.c -o foo.o gcc -m32 -shared -s -static-libgcc -o foo.dll tmp.def foo.o -Ld:/RCompile/CRANpkg/extralibs64/local/lib/i386 -Ld:/RCompile/CRANpkg/extralibs64/local/lib -LC:/PROGRA~1/R/R-30~1.2/bin/i386 -lR collect2: ld returned 9 exit status Error in inDL(x, as.logical(local), as.logical(now), ...) : unable to load shared object 'C:/TEMP/Rtmpes5hJL/foo.dll': LoadLibrary failure: The specified module could not be found. I am running Windows 7 (32-bit), Rtools has been installed and is in my path (i.e. I can run gcc/g++). The object file (.o) gets created but then the linking goes awry. Another thing here is that I do not know where the directory RCompile is as it appears to be trying to link some 64-bit stuff (i.e. extralibs64). Any help/thoughts would be fantastic, Sam [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Fwd: linker issue
Hi, I am trying to compile C++ with R and I am coming up against - what I believe - is a linker configuration issue. For example, using has_develop() gives the following message... > has_devel() "C:/PROGRA~1/R/R-30~1.2/bin/i386/R" --vanilla CMD SHLIB foo.c gcc -m32 -I"C:/PROGRA~1/R/R-30~1.2/include" -DNDEBUG -I"d:/RCompile/CRANpkg/extralibs64/local/include" -O3 -Wall -std=gnu99 -mtune=core2 -c foo.c -o foo.o gcc -m32 -shared -s -static-libgcc -o foo.dll tmp.def foo.o -Ld:/RCompile/CRANpkg/extralibs64/local/lib/i386 -Ld:/RCompile/CRANpkg/extralibs64/local/lib -LC:/PROGRA~1/R/R-30~1.2/bin/i386 -lR collect2: ld returned 9 exit status Error in inDL(x, as.logical(local), as.logical(now), ...) : unable to load shared object 'C:/TEMP/Rtmpes5hJL/foo.dll': LoadLibrary failure: The specified module could not be found. I am running Windows 7 (32-bit), Rtools has been installed and is in my path (i.e. I can run gcc/g++). The object file (.o) gets created but then the linking goes awry. Another thing here is that I do not know where the directory RCompile is as it appears to be trying to link some 64-bit stuff (i.e. extralibs64). Any help/thoughts would be fantastic, Sam [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] String Vector Encoding
On 14-02-03 3:41 AM, Saptarshi Guha wrote: Hello, I was reading through serialize.c and i couldn't answer something. In readItem, case CHARSXP, rules exists to adjust the read string for string encoding. This is described in the R Internals manual. Duncan Murdoch Q1. I couldn't find where the encoding of the elements of the string vector are written? Is it when writeItem writes out the attributes of the item? But i couldn't find encoding in the attributes field x <- "fa\xE7ile" Encoding(x) <- "latin1" xx <- iconv(x, "latin1", "UTF-8") y=c(x,xx) attributes(y) NULL Apologies if this has been asked before Regards Saptarshi [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] LinkingTo self
Hello, Shipping header files for a package in inst/include and let other packages use it with LinkingTo is popular. Unfortunately for the package itself, we still need to use something like : PKG_CPPFLAGS+=-I../inst/include/ in the Makevars and Makevars.win files. Could this become automatic ? So if a package has src/ and inst/include/ can we have it so that a -I flag is set accordingly. Would anyone be interested in reviewing a patch against R-devel for such feature ? Romain __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] String Vector Encoding
Hello, I was reading through serialize.c and i couldn't answer something. In readItem, case CHARSXP, rules exists to adjust the read string for string encoding. Q1. I couldn't find where the encoding of the elements of the string vector are written? Is it when writeItem writes out the attributes of the item? But i couldn't find encoding in the attributes field x <- "fa\xE7ile" Encoding(x) <- "latin1" xx <- iconv(x, "latin1", "UTF-8") y=c(x,xx) attributes(y) NULL Apologies if this has been asked before Regards Saptarshi [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
