[R] filehash package: error in unserialize(con)
This question is related to the filehash package only. I would like to use the filehash package because of it's ability to deal with large amounts of data. I've set up a filehash database on a network drive and populated it with several GB of data. Somehow, I must have corrupted the files because I cannot access the objects anymore. Here is what I do: require(filehash) filehashOption(defaultType = RDS) # need this, dbInit will fail db - dbInit(//nas-msw-07/app/Prices/Historical) hPP - db2env(db)# put all the data in an enviroment (commod xyz ex %in% ls(hPP)) [1] TRUE dbExists(db, commod xyz ex) [1] TRUE dbFetch(db, commod xyz ex) Error in unserialize(con) : error reading from connection hPP[[commod xyz ex]] Error in unserialize(con) : error reading from connection How do I get to touch the data again? It's really slow to recreate the filehash again, and it seems to be very fragile. Any suggestions much appreciated. Thank you, Adrian Dragulescu __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Problem Connecting to Oracle with R from Windows XP
You also need the ROracle package. On 8/15/07, Song, Alex [EMAIL PROTECTED] wrote: Hello, I installed RGui 2.5.1 and package DBI on Windows XP and tried to connect to Oracle database which is on a Linux server. When I tried to use dbDriver(Oracle), I got an error as follows: drv - dbDriver(Oracle) Error in do.call(as.character(drvName), list(...)) : could not find function Oracle Could anyone tell me how to connect to Oracle with R from Windows XP? Do I need to configure any environment variables? Do I need to configure ODBC? Do I need to install any other packages? I have Oracle client 10g installed on my computer and I can connect to the Oracle database using other client software like Toad, SQLPlus, or SQL Developer. Thank you very much. Alex Alex Song Data Management Specialist / Spécialiste en gestion des données National Forest Inventory / Inventaire forestier national Pacific Forestry Centre / Centre de foresterie du Pacifique Canadian Forest Service / Service canadien des forêts Natural Resources Canada / Ressources naturelles Canada Government of Canada / Gouvernement du Canada 506 West Burnside Road / 506 chemin Burnside ouest Victoria, BC V8Z 1M5 Phone (250) 363-3342 Facs: (250) 363-0775 Email: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] the large dataset problem
Take a look at the package filehash. It allows you to work with large
objects in R (bigger than your RAM) by storing them on the disk. The
objects are represented as pointers in R and have a small footprint in
memory. You can load all of them in an environment and access them with the
$ operator. I think filehash is more general than R.huge. R.huge works very
well with numerical 2D data only.
Adrian Dragulescu
On 7/31/07, Eric Doviak [EMAIL PROTECTED] wrote:
Just a note of thanks for all the help I have received. I haven't gotten a
chance to implement any of your suggestions because I'm still trying to
catalog all of them! Thank you so much!
Just to recap (for my own benefit and to create a summary for others):
Bruce Bernzweig suggested using the R.huge package.
Ben Bolker pointed out that my original message wasn't clear and asked
what I want to do with the data. At this point, just getting a dataset
loaded would be wonderful, so I'm trying to trim variables (and if possible,
I would also like to trim observations). He also provided an example of
vectorizing.
Ted Harding suggested that I use AWK to process the data and provided the
necessary code. He also tested his code on older hardware running GNU-Linux
(or Unix?) and showed that AWK can process the data even when the computer
has very little memory and processing power. Jim Holtman had similar success
when he used Cygwin's UNIX utilities on a machine running MS Windows. They
both used the following code:
gawk 'BEGIN{FS=,}{print $(1) , $(1000) , $(1275) , $(5678)}'
tempxx.txt newdata.csv
Fortunately, there is a version of GAWK for MS Windows. ... Not that I
like MS Windows. It's just that I'm forced to use that 19th century
operating system on the job. (After using Debian at home and happily running
RKWard for my dissertation, returning to Windows World is downright
depressing).
Roland Rau suggested that I use a database with RSQLite and pointed out
that RODBC can work with MS Access. He also pointed me to a sub-chapter in
Venables and Ripley's _S Programming_ and The Whole-Object View pages in
John Chamber's _Programming with Data_.
Greg Snow recommended biglm for regression analysis with data that is
too large to fit into memory.
Last, but not least, Peter Dalgaard pointed out that there are options
within R. He suggests using the colClasses= argument for when reading data
and the what= argument for scanning data, so that you don't load more
columns than necessary. He also provided the following script:
dict - readLines(
ftp://www.sipp.census.gov/pub/sipp/2004/l04puw1d.txt;)
D.lines - grep(^D , dict)
vdict - read.table(con - textConnection(dict[D.lines])); close(con)
head(vdict)
I'll try these solutions and report back on my success.
Thanks again!
- Eric
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] learning lattice graphics
Check the documentation link from http://cm.bell-labs.com/cm/ms/departments/sia/project/trellis/software.writing.html Adrian On 5/26/07, Tyler Smith [EMAIL PROTECTED] wrote: Hi, I've just produced my first lattice plot - the graphic is very impressive, but I only partly understand how it works. I crimped from examples in MASS and the help pages to get most of what I want done. Howver, I find both MASS and the help pages are a little too terse for my needs as a beginner. Can anyone suggest web or print resources for lattice graphics that provide a more thorough introduction? I've been through the FAQ and standard manuals, but didn't see what I needed there. Thanks, Tyler __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] how to avoid infinite loops when sourcing R files
Hello,
I have a bunch of R files in a directory and I want to source all of them
with something like lapply(files, source).
I have a main.R file
source(C:/Temp/set.parms.R)
parms - set.parms()
do.calcs(parms)
cat(I'm done with main.R\n)
Then I have set.parms.R function
set.parms - function(){
cat(I'm in set.parms.\n); flush.console()
directory - C:/Temp/
files - dir(directory, \\.[rR]$, full.name=T)
files - files[-grep(set.parms.R, files)] # remove infinite loop
lapply(files, source) # source them all
cat(Exiting set.parms.\n); flush.console()
}
And other functions f1, f2, f3, etc. in the same directory that also
source set.parms.R. For example:
f1 - function(){
source(H:/user/R/RMG/Energy/VaR/Overnight/Test/set.parms.R)
cat(I add two numbers.\n); flush.console()
}
Because of the source command in f1, I get into an infinite loop. This
must be a common situation but I don't know how to avoid it.
I need the source(set.parms) in f1, f2, f3, etc. because I want use a
different combination of them in other projects.
Thanks,
Adrian
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] R in a production environment
Recently there have been a lot of discussions on the list on the use of R in industry and comparisons with SAS SPSS. These are very important questions and it might be worth having them all grouped together for reference. I would like to add to these topics a related one. Can we share our experience of using R in a production environment? I work in a financial company that is cautiously interested in using R in a production environment. I'm trying to convince them that R is serious software, modern and powerful. Their first question is who else out there is using R. I can guess from some contributed packages, or from the list postings, but it would be great if there would be a centralized place. We could have a standard set of questions and a contact person (maybe on the wiki site?) What do you think? I believe it could create a snowball effect to increase the visibility and use of R beyond academia and personal use. Thanks, Adrian __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] autofill for a tkentry field
Hello all,
I'm experimenting with tlk/tk and I find it very useful. I would like to
have the text of a tkentry be automatically filled when the user starts
typing, according to a predetermined list. I've found a solution on the
tcl wiki pages, but I don't quite know how to translate it in R... Here is
the tcl code: http://wiki.tcl.tk/13267
And here is my code that does not work.
autofill - function(){
cat(I was here!)
entry - tclvalue(f1)
ind - grep(entry, fruitList)
if (length(ind)0){f1 - tclVar(fruitList[ind[1]])}
}
fruitList - c(Apple, Banana, Peach, Pear, Plum, Mango)
tkdestroy(tt)
tt - tktoplevel()
pw - tkframe(tt)
ppw - tk2labelframe(pw, text=Fruits:)
f1 - tclVar()
entry1.f - tkentry(ppw, width=30, textvariable=f1, validate=key,
validatecommand=autofill)
tkgrid(entry1.f)
tkgrid(ppw)
tkgrid(pw)
I don't know how to pass arguments to the autofill function, so I can
manipulate the textvariable. Any help is much appreciated. If we get a
working solution, maybe we should add this example to
http://www.sciviews.org/_rgui/tcltk/index.html.
Thank you,
Adrian
__
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] Installing package rpvm under Windows
Hello, I'm trying to install the rpvm package under Windows, but I am having problems. I have pvm3.4 installed properly. I've defined the system variables PVM_ROOT = C:\PROGRA~1\pvm3.4\ PVM_ARCH = win32 When I try to install, I get this: C:\R\PackagesRcmd INSTALL rpvm_1.0.1.tar.gz -- Making package rpvm ** WARNING: this package has a configure script It probably needs manual configuration ** adding build stamp to DESCRIPTION making DLL ... gcc -shared -s -o rpvm.dll rpvm.def rpvm_core.o rpvm_ser.o utils.orpvm_res.o -Lc:/PROGRA~1/R/R-24~1.0/bin -lR rpvm_core.o:rpvm_core.c:(.text+0x44): undefined reference to `pvm_perror' rpvm_core.o:rpvm_core.c:(.text+0x49): undefined reference to `pvm_exit' rpvm_core.o:rpvm_core.c:(.text+0x197): undefined reference to `pvm_mytid' rpvm_core.o:rpvm_core.c:(.text+0x1c7): undefined reference to `pvm_parent' rpvm_core.o:rpvm_core.c:(.text+0x207): undefined reference to `pvm_exit' rpvm_core.o:rpvm_core.c:(.text+0x2a8): undefined reference to `pvm_pstat' rpvm_core.o:rpvm_core.c:(.text+0x352): undefined reference to `pvm_kill' rpvm_core.o:rpvm_core.c:(.text+0x3ba): undefined reference to `pvm_tasks' rpvm_core.o:rpvm_core.c:(.text+0x6bf): undefined reference to `pvm_spawn' rpvm_core.o:rpvm_core.c:(.text+0x7b7): undefined reference to `pvm_initsend' rpvm_core.o:rpvm_core.c:(.text+0x7f7): undefined reference to `pvm_mkbuf' rpvm_core.o:rpvm_core.c:(.text+0x837): undefined reference to `pvm_freebuf' rpvm_core.o:rpvm_core.c:(.text+0x867): undefined reference to `pvm_getsbuf' rpvm_core.o:rpvm_core.c:(.text+0x897): undefined reference to `pvm_getrbuf' rpvm_core.o:rpvm_core.c:(.text+0x8d7): undefined reference to `pvm_setsbuf' rpvm_core.o:rpvm_core.c:(.text+0x917): undefined reference to `pvm_setrbuf' rpvm_core.o:rpvm_core.c:(.text+0x97d): undefined reference to `pvm_pkint' rpvm_core.o:rpvm_core.c:(.text+0x9f2): undefined reference to `pvm_pkdouble' rpvm_core.o:rpvm_core.c:(.text+0xa43): undefined reference to `pvm_pkstr' rpvm_core.o:rpvm_core.c:(.text+0xa9e): undefined reference to `pvm_pkint' rpvm_core.o:rpvm_core.c:(.text+0xad4): undefined reference to `pvm_pkint' rpvm_core.o:rpvm_core.c:(.text+0xb2e): undefined reference to `pvm_pkint' rpvm_core.o:rpvm_core.c:(.text+0xb5f): undefined reference to `pvm_pkdouble' rpvm_core.o:rpvm_core.c:(.text+0xbc1): undefined reference to `pvm_pkint' rpvm_core.o:rpvm_core.c:(.text+0xbf8): undefined reference to `pvm_pkstr' rpvm_core.o:rpvm_core.c:(.text+0xcc5): undefined reference to `pvm_pkint' rpvm_core.o:rpvm_core.c:(.text+0xcf5): undefined reference to `pvm_pkint' rpvm_core.o:rpvm_core.c:(.text+0xdc5): undefined reference to `pvm_pkint' rpvm_core.o:rpvm_core.c:(.text+0xdf5): undefined reference to `pvm_pkdouble' rpvm_core.o:rpvm_core.c:(.text+0xe6e): undefined reference to `pvm_send' rpvm_core.o:rpvm_core.c:(.text+0xee4): undefined reference to `pvm_mcast' rpvm_core.o:rpvm_core.c:(.text+0xf43): undefined reference to `pvm_recv' rpvm_core.o:rpvm_core.c:(.text+0xf9e): undefined reference to `pvm_nrecv' rpvm_core.o:rpvm_core.c:(.text+0xffe): undefined reference to `pvm_probe' rpvm_core.o:rpvm_core.c:(.text+0x107c): undefined reference to `pvm_trecv' rpvm_core.o:rpvm_core.c:(.text+0x113d): undefined reference to `pvm_bufinfo' rpvm_core.o:rpvm_core.c:(.text+0x1217): undefined reference to `pvm_upkint' rpvm_core.o:rpvm_core.c:(.text+0x12ac): undefined reference to `pvm_upkdouble' rpvm_core.o:rpvm_core.c:(.text+0x130b): undefined reference to `pvm_upkstr' rpvm_core.o:rpvm_core.c:(.text+0x1374): undefined reference to `pvm_upkint' rpvm_core.o:rpvm_core.c:(.text+0x13b4): undefined reference to `pvm_upkstr' rpvm_core.o:rpvm_core.c:(.text+0x1421): undefined reference to `pvm_upkint' rpvm_core.o:rpvm_core.c:(.text+0x146f): undefined reference to `pvm_upkint' rpvm_core.o:rpvm_core.c:(.text+0x14c1): undefined reference to `pvm_upkint' rpvm_core.o:rpvm_core.c:(.text+0x150f): undefined reference to `pvm_upkdouble' rpvm_core.o:rpvm_core.c:(.text+0x1561): undefined reference to `pvm_upkint' rpvm_core.o:rpvm_core.c:(.text+0x15be): undefined reference to `pvm_upkint' rpvm_core.o:rpvm_core.c:(.text+0x1611): undefined reference to `pvm_upkint' rpvm_core.o:rpvm_core.c:(.text+0x166e): undefined reference to `pvm_upkdouble' rpvm_core.o:rpvm_core.c:(.text+0x16c0): undefined reference to `pvm_config' rpvm_core.o:rpvm_core.c:(.text+0x1894): undefined reference to `pvm_start_pvmd' rpvm_core.o:rpvm_core.c:(.text+0x191c): undefined reference to `pvm_addhosts' rpvm_core.o:rpvm_core.c:(.text+0x19c7): undefined reference to `pvm_delhosts' rpvm_core.o:rpvm_core.c:(.text+0x1a27): undefined reference to `pvm_halt' rpvm_core.o:rpvm_core.c:(.text+0x1acc): undefined reference to `pvm_mstat' rpvm_core.o:rpvm_core.c:(.text+0x1b53): undefined reference to `pvm_joingroup' rpvm_core.o:rpvm_core.c:(.text+0x1ba3): undefined reference to
Re: [R] Suppress blanks/spaces in character
c - abcdef ghi gsub( +, , c) [1] abc def ghi On Fri, 27 Oct 2006 [EMAIL PROTECTED] wrote: Hi all I'm have a character vector and would like to suppress the blanks if there are more than one after the other. Example: Character value is: abc def ghi The result should be: abc def ghi I know that it's possible to delete the leading blanks with the command trim. But how can I delete blanks within a character? Thank you very much in advance for an answer Thomas __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] ROracle error in Windows. Memory could not be read.
I've seen from earlier posts that other people had problems installing ROracle under Windows. I run R-2.3.1. I got the Windows binaries for ROracle from http://stat.bell-labs.com/RS-DBI/download/index.html Here is my session: require(ROracle) Loading required package: ROracle Loading required package: DBI [1] TRUE drv - dbDriver(Oracle) drv OraDriver:(1648) con - dbConnect(drv, user=USER, password=PASS, dbname=TEST) A window pops up, with the message: The instruction at 0x6260c621 referenced memory at 0x4a280ae2. The memory could not be read. And R freezes. Something is working, because I get to establish an Oracle driver. Any ideas would be much appreciated. Thank you, Adrian __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] greek letters, text, and values in labels
Hello, I want to have a title that will look something like: Results for \theta=2.1, given that I have a variable theta=2.1, and \theta should show on the screen like the greek letter. I've tried a lot of things: theta - 2.1 plot(1:10, main=expression(paste(Results for, theta, =, eval(theta or using bquote plot(1:10, main=paste(Results for , bquote(theta == .(theta or using substitute, etc. I could not make it work. This should be easy. I would appreciate your help. Thanks, Adrian __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] weird behavior of nsmall in format
From the help page of format, nsmall should control the number of digits. format(0.123456789, nsmall = 10) [1] 0.1234567890 format(0.123456789, nsmall = 1) [1] 0.1234568 format(0.123456789, nsmall = 2) [1] 0.1234568 format(0.123456789, nsmall = 8) [1] 0.12345679 It adds zeros fine but for format(0.123456789, nsmall = 1) I want the result to be 0.1. I want to format numbers with a fixed number of digits. A combination of round and format + nsmall does the job. sprintf will do it too, but I wondered if the current implementation of nsmall in format is correct. Thanks, Adrian __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] how to clear the last error?
Hello, I would like to clear the last error that I get with geterrmessage(). Not even rm(list=ls()) clears it. Can I set it to NULL somehow? Thank you, Adrian __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] problem with DCOM server in R 2.1.0
Hello list, I just installed R 2.1.0. When I try to run the Server 01 - Basic test I get the error Fatal error: unable to open the base package. Any solution? Thank you, Adrian Dragulescu __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] dumping the summary of lm to a text file
Hello list, I have a linear regression ctl- c(4.17,5.58,5.18,6.11,4.50,4.61,5.17,4.53,5.33,5.14) trt- c(4.81,4.17,4.41,3.59,5.87,3.83,6.03,4.89,4.32,4.69) group - gl(2,10,20, labels=c(Ctl,Trt)) weight - c(ctl, trt) reg- lm(weight ~ group) sreg - summary(reg) and I would like to dump exactly what I see on the console with print(sreg) to a text file. I've tried using cat but it did not work. I've read the list questions but I did not see a solution. Can you help with this? Thank you, Adrian Dragulescu __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] memory problems with lm
Hello list, I've seen the recent discussions documenting problems with lm. I have encountered the following problem. I use WinXP Pro with service pack 1, and R 1.9.0, on a XEON 2GHz, with 1GB of RAM. eff.fro std.dev mean NSTRDSP 7.403749e-01 1.215686e-01 CPFGEP 9.056763e+00 1.815686e+00 WSWOLF 4.703588e+05 1.112832e+05 NPILGRIM 1.017640e+06 2.134335e+05 WSNMILE 1.367312e+07 1.892021e+06 WSHIDESL 1.830811e+07 1.892021e+06 reg - lm(log(mean) ~ log(std.dev), data=eff.fro) Error in model.matrix.default(mt, mf, contrasts) : cannot allocate vector of length 1074790452 log(eff.fro$mean) [1] -2.1072763 0.5964635 11.6198339 12.2710808 14.4531561 [6] 14.4531561 reg - lm(log(eff.fro$mean) ~ log(eff.fro$std.dev)) Error: cannot allocate vector of size 3360077 Kb lef - log(eff.fro) lef std.dev mean NSTRDSP -0.3005986 -2.1072763 CPFGEP2.2035117 0.5964635 WSWOLF 13.0612512 11.6198339 NPILGRIM 13.8329973 12.2710808 WSNMILE 16.4309427 14.4531561 WSHIDESL 16.7228546 14.4531561 lef - log(eff.fro) reg - lm(lef$mean ~ lef$std.dev) Here the my computer completely crashed. A window poped-up and said memory problem at address ..., and if I want to debug. I ran the same code one more time, and it worked but it did not work how I wanted (where is the slope?): reg - lm(lef$mean ~ lef$std.dev) reg Call: lm(formula = lef$mean ~ lef$std.dev) Coefficients: (Intercept) 8.548 summary(reg) Call: lm(formula = lef$mean ~ lef$std.dev) Residuals: 1 2 3 4 5 6 -10.655 -7.951 3.072 3.723 5.905 5.905 Coefficients: Estimate Std. Error t value Pr(|t|) (Intercept)8.548 2.9992.85 0.0358 * --- Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 Residual standard error: 7.346 on 5 degrees of freedom I ran again: reg - lm(log(mean) ~ log(std.dev), data=eff.fro) and I get the pop-up: The instruction at 0x6b4c45a5 referenced memory at 0x0032374a. The memory could not be read. Click OK to terminate the program. Any ideas? Thank you, Adrian __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
RE: [R] memory problems with lm
If I enforce the variables to be numeric it works fine. str(eff.fro) `data.frame': 6 obs. of 2 variables: $ std.dev: num [, 1:6] 7.40e-01 9.06e+00 4.70e+05 1.02e+06 1.37e+07 ... ..- attr(*, dimnames)=List of 1 .. ..$ : chr NSTRDSP CPFGEP WSWOLF NPILGRIM ... $ mean : num 1.22e-01 1.82e+00 1.11e+05 2.13e+05 1.89e+06 ... gc() used (Mb) gc trigger (Mb) Ncells 578941 15.51166886 31.2 Vcells 589444 4.52377385 18.2 eff.fro std.dev mean NSTRDSP 7.403749e-01 1.215686e-01 CPFGEP 9.056763e+00 1.815686e+00 WSWOLF 4.703588e+05 1.112832e+05 NPILGRIM 1.017640e+06 2.134335e+05 WSNMILE 1.367312e+07 1.892021e+06 WSHIDESL 1.830811e+07 1.892021e+06 reg - lm(log(as.numeric(mean)) ~ log(as.numeric(std.dev)), data=eff.fro) reg Call: lm(formula = log(as.numeric(mean)) ~ log(as.numeric(std.dev)), data = eff.fro) Coefficients: (Intercept) log(as.numeric(std.dev)) -1.63680.9864 Adrian On Thu, 29 Apr 2004, Liaw, Andy wrote: Can you show us the output of str(eff.fro)? Do you have other things in the global environment or the search path that's taking up memory? What does gc() say? Andy From: Adrian Dragulescu Hello list, I've seen the recent discussions documenting problems with lm. I have encountered the following problem. I use WinXP Pro with service pack 1, and R 1.9.0, on a XEON 2GHz, with 1GB of RAM. eff.fro std.dev mean NSTRDSP 7.403749e-01 1.215686e-01 CPFGEP 9.056763e+00 1.815686e+00 WSWOLF 4.703588e+05 1.112832e+05 NPILGRIM 1.017640e+06 2.134335e+05 WSNMILE 1.367312e+07 1.892021e+06 WSHIDESL 1.830811e+07 1.892021e+06 reg - lm(log(mean) ~ log(std.dev), data=eff.fro) Error in model.matrix.default(mt, mf, contrasts) : cannot allocate vector of length 1074790452 log(eff.fro$mean) [1] -2.1072763 0.5964635 11.6198339 12.2710808 14.4531561 [6] 14.4531561 reg - lm(log(eff.fro$mean) ~ log(eff.fro$std.dev)) Error: cannot allocate vector of size 3360077 Kb lef - log(eff.fro) lef std.dev mean NSTRDSP -0.3005986 -2.1072763 CPFGEP2.2035117 0.5964635 WSWOLF 13.0612512 11.6198339 NPILGRIM 13.8329973 12.2710808 WSNMILE 16.4309427 14.4531561 WSHIDESL 16.7228546 14.4531561 lef - log(eff.fro) reg - lm(lef$mean ~ lef$std.dev) Here the my computer completely crashed. A window poped-up and said memory problem at address ..., and if I want to debug. I ran the same code one more time, and it worked but it did not work how I wanted (where is the slope?): reg - lm(lef$mean ~ lef$std.dev) reg Call: lm(formula = lef$mean ~ lef$std.dev) Coefficients: (Intercept) 8.548 summary(reg) Call: lm(formula = lef$mean ~ lef$std.dev) Residuals: 1 2 3 4 5 6 -10.655 -7.951 3.072 3.723 5.905 5.905 Coefficients: Estimate Std. Error t value Pr(|t|) (Intercept)8.548 2.9992.85 0.0358 * --- Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 Residual standard error: 7.346 on 5 degrees of freedom I ran again: reg - lm(log(mean) ~ log(std.dev), data=eff.fro) and I get the pop-up: The instruction at 0x6b4c45a5 referenced memory at 0x0032374a. The memory could not be read. Click OK to terminate the program. Any ideas? Thank you, Adrian __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html -- Notice: This e-mail message, together with any attachments, contains information of Merck Co., Inc. (One Merck Drive, Whitehouse Station, New Jersey, USA 08889), and/or its affiliates (which may be known outside the United States as Merck Frosst, Merck Sharp Dohme or MSD and in Japan as Banyu) that may be confidential, proprietary copyrighted and/or legally privileged. It is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please notify us immediately by reply e-mail and then delete it from your system. -- __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] cumulative density function
Because the result is a list, you can get your number like this: integrate(dnorm,0,2)[[1]] You don't have an expression for the probability function? You can do a nonlinear regression to a distribution you belive in, and then integrate that. Or, you can calculate the experimental cdf by simple summation. Best, Adrian __ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] profile error on an nls object
Hello all, This is the error message that I get. hyp.res - nls(log(y)~log(pdf.hyperb(theta,X)), data=dataModel, + start=list(theta=thetaE0), + trace=TRUE) 45.54325 : 0.100 1.3862944 -4.5577142 0.0005503 3.728302 : 0.0583857346 0.4757772859 -4.9156128701 0.0005563154 1.584317 : 0.0194149477 0.3444648833 -4.9365149150 0.0004105426 1.569333 : 0.0139310639 0.3824648048 -4.9024001228 0.0004089738 1.569311 : 0.0137155342 0.3888648619 -4.8979817546 0.0004137501 1.569311 : 0.0136895846 0.3893564152 -4.8976182201 0.0004141057 1.569311 : 0.0136876315 0.3894059947 -4.8975821760 0.0004141343 hyp.res.S - summary(hyp.res) hyp.res.S Formula: log(y) ~ log(pdf.hyperb(theta, X)) Parameters: Estimate Std. Error t value Pr(|t|) theta1 0.0136876 0.0359964 0.3800.705 theta2 0.3894060 0.3079860 1.2640.211 theta3 -4.8975822 0.2219928 -22.062 2e-16 *** theta4 0.0004141 0.0005457 0.7590.451 --- Signif. codes: 0 `***' 0.001 `**' 0.01 `*' 0.05 `.' 0.1 ` ' 1 Residual standard error: 0.1542 on 66 degrees of freedom Correlation of Parameter Estimates: theta1theta2theta3 theta2 -0.02168 theta3 -0.02029 0.997736 theta4 -0.97182 -0.008054 -0.008952 pr1 - profile(hyp.res) 1.825584 : 0.3894059947 -4.8975821760 0.0004141343 1.58426 : 0.373691474 -4.909091289 0.000824045 1.583673 : 0.4176596873 -4.8774106487 0.0008176545 1.583670 : 0.4196944963 -4.8760375504 0.0008187918 1.583670 : 0.4199010211 -4.8758854269 0.0008188162 1.624899 : 0.449756713 -4.854643555 0.001215014 1.624743 : 0.46804752 -4.84185838 0.00122343 1.624741 : 0.470384534 -4.840195293 0.001224199 1.624741 : 0.470638282 -4.840013199 0.001224298 1.624741 : 0.470670500 -4.839990112 0.001224309 1.692158 : 0.522188258 -4.803565745 0.001635778 1.691853 : 0.540794581 -4.791027785 0.001650730 1.691847 : 0.544973564 -4.788090229 0.001652321 1.691847 : 0.545500818 -4.787718964 0.001652616 1.691847 : 0.545592388 -4.787654441 0.001652658 1.784749 : 0.622277872 -4.734086833 0.002091090 1.784039 : 0.642139831 -4.721442413 0.00211 1.784022 : 0.649929188 -4.716068239 0.002119126 1.784021 : 0.651094995 -4.715267692 0.002119963 1.784021 : 0.65136956 -4.71507850 0.00212012 1.784021 : 0.651420684 -4.715043256 0.002120153 1.901667 : 0.760981513 -4.639871097 0.002604136 1.899765 : 0.782870773 -4.627113544 0.002644289 1.899703 : 0.798044378 -4.616913842 0.002653139 1.899699 : 0.800930030 -4.614996517 0.002655646 1.899699 : 0.801815115 -4.614404602 0.002656286 1.899699 : 0.802033012 -4.614258879 0.002656455 1.899699 : 0.802090888 -4.614220164 0.002656499 2.042311 : 0.960722487 -4.508069592 0.003221186 2.036028 : 0.985442669 -4.495703574 0.003291724 2.035767 : 1.017849320 -4.474692317 0.003317203 2.035736 : 1.026482531 -4.469203277 0.003326355 2.035733 : 1.029937702 -4.466987738 0.003329627 2.035732 : 1.031114541 -4.466233213 0.003330776 2.035732 : 1.03153247 -4.46596514 0.00333118 2.035732 : 1.031679019 -4.465871141 0.003331322 2.035732 : 1.031730150 -4.465838341 0.003331371 1.583425 : 3.595503e-01 -4.918824e+00 1.793668e-05 1.583270 : 3.783622e-01 -4.905413e+00 2.175254e-05 1.58327 : 3.793739e-01 -4.904683e+00 2.134353e-05 1.58327 : 3.794562e-01 -4.904622e+00 2.133204e-05 1.624847 : 0.3695765499 -4.9116128267 -0.0003687005 1.624748 : 0.3877816645 -4.8985581022 -0.0003699797 1.624747 : 0.3890277914 -4.8976645481 -0.0003706884 1.624747 : 0.3891837724 -4.8975508910 -0.0003707385 1.693266 : 0.3989123147 -4.8904787597 -0.0007628478 1.693170 : 0.4172349434 -4.8774484377 -0.0007692982 1.693168 : 0.4193992605 -4.8759045536 -0.0007705839 1.693168 : 0.4197576966 -4.8756463144 -0.0007707548 1.693168 : 0.4198094076 -4.8756090629 -0.0007707816 1.788041 : 0.450653198 -4.853510937 -0.001173674 1.787884 : 0.469850125 -4.840178495 -0.001186483 1.787878 : 0.473941725 -4.837285088 -0.001188994 1.787878 : 0.474783800 -4.836687703 -0.001189518 1.787878 : 0.474960093 -4.836562526 -0.001189627 1.787878 : 0.474996400 -4.836536741 -0.001189650 1.908362 : 0.531011661 -4.796878005 -0.001614805 1.907987 : 0.552282889 -4.782714884 -0.001637055 1.907965 : 0.560264832 -4.777154923 -0.001642437 1.907964 : 0.562346209 -4.775709348 -0.001644026 1.907963 : 0.56295255 -4.77528733 -0.00164447 1.907963 : 0.563123513 -4.775168321 -0.001644596 1.907963 : 0.563172014 -4.775134556 -0.001644632 2.053048 : 0.653571857 -4.712183501 -0.002111091 2.051874 : 0.678913471 -4.696423681 -0.002150202 2.051785 : 0.69544095 -4.68518302 -0.00216328 2.051772 : 0.701166962 -4.681323343 -0.002168564 2.051770 : 0.703471045 -4.679765756 -0.002170603 2.051769 : 0.704360670 -4.679164327 -0.002171397 2.051769 : 0.704707691 -4.678929683 -0.002171706 2.051769 : 0.704842651 -4.678838422 -0.002171826 2.051769 : 0.704895042 -4.678802995 -0.002171872
