Re: [R] how to change the ff properties of a ff-related R object after the original ff output folder has been moved

Tao, I do assume that the ff-files are still at some location and not deleted by a finalizer. The following explains how to manipulate file locations with ff and ffdf objects. Kind regards jens library(ff) path1 - c:/tmp path2 - c:/tmp2 # create ffdf, # using non-standard path sets

[R] [R-pkgs] package bit64 with new functionality

please check the ANNOUNCEMENT-0.9-Details.txt file and the package help files. Kind regards Jens Oehlschlägel Munich, 8.11.2012 ___ R-packages mailing list r-packa...@r-project.org https://stat.ethz.ch/mailman/listinfo/r-packages

Re: [R] [R-sig-hpc] Quickest way to make a large empty file on disk?

the example below and see how big file creation is immediate. Jens Oehlschlägel library(ff) library(snowfall) ncpus - 2 n - 1e8 system.time( + x - ff(vmode=double, length=n, filename=c:/Temp/x.ff) + ) User System verstrichen 0.010.00

Re: [R] ff objects saving problem

as a separate ff file. Jens Oehlschlägel __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained

[R] [R-pkgs] ff version 2.2.0

Dear R community, The next release of package ff is available on CRAN. With kind help of Brian Ripley it now supports the Win64 and Sun versions of R. It has three major functional enhancements: a) new fast in-memory sorting and ordering functions (single-threaded) b) ff now supports on-disk

Re: [R] Pass By Value Questions

will share the data and SOME features (like the 'length') while OTHER features (like 'dim') are copied on modify (see 'vt' for an powerful application of this concept). You might want to have a look at 'ffapply' and friends and at 'chunk'. HTH Jens Oehlschlägel

[R] ff objects and ordinary analytical functions.

responsibility to extract those chunks from ff, a database or whatever other source. HTH Jens Oehlschlägel __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org

Re: [R] Can saved R object .RData files be loaded by more than one R sessions for read only purpose?

multiple processes writing simultaneously to the same ff datafile. (it is your responsibility to avoid conflicts and to make sure you do not suffer problems with delayed cache refreshs as can happen on network drives) HTH Jens Oehlschlägel __ R-help

Re: [R] Can saved R object .RData files be loaded by more than one R sessions for read only purpose?

multiple processes writing simultaneously to the same ff datafile. (it is your responsibility to avoid conflicts and to make sure you do not suffer problems with delayed cache refreshs as can happen on network drives) HTH Jens Oehlschlägel __ R-help

Re: [R] How to deal with more than 6GB dataset using R?

Matthew, You might want to look at function read.table.ffdf in the ff package, which can read large csv files in chunks and store the result in a binary format on disk that can be quickly accessed from R. ff allows you to access complete columns (returned as a vector or array) or subsets of

Re: [R] Assign Formulas to Arrays or Matrices?

for assigning formulas to arrays use an array of list nr form.arr[[31,5]]y ~ 1 + 2 Jens Oehlschlägel -Ursprüngliche Nachricht- Von: McLovin Gesendet: Jul 6, 2010 9:13:49 AM An: r-help@r-project.org Betreff: [R] Assign Formulas to Arrays or Matrices? Hi, I am very new to R. I am

Re: [R] memory management in R

, short integers etc.). Jens Oehlschlägel -Ursprüngliche Nachricht- Von: john mull...@fastmail.fm Gesendet: Jun 16, 2010 12:20:17 PM An: r-help@r-project.org Betreff: [R] memory management in R I have volunteered to give a short talk on memory management in R to my local R user group

Re: [R] how to read CSV file in R?

,] # returns a subset of data Do read and understand the help concerning filename location and implications for finalizers and permanency. Cheers Jens Oehlschlägel -Ursprüngliche Nachricht- Von: Joris Meys jorism...@gmail.com Gesendet: Jun 8, 2010 1:11:20 PM An: dhanush dhana...@gmail.com

Re: [R] appending objects to file created with save()

If you work with large data you might want to look at the ff package - useful if your data is close or above your RAM. The package has ffsave where with option add=TRUE you can add data to an existing ff archive. With ff data is stored outside of R in files, only meta-data is stored within R.

Re: [R] ff for 64-bit windows and 64-bit R

Lawrence, My understanding is that only a minor change is needed in ff's C++ layer in order to remove the 64bit compiler warnings/errors. The C++ layer is maintained by Daniel Adler, who can give you an outlook if/when he plans to attack this. Until a 64bit version of ff is available, you

Re: [R] how to work with big matrices and the ff-package?

matrix because this saves disk space and improves caching.   HTH Jens Oehlschlägel         __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html

Re: [R] as.ffdf.data.frame now breaks if using pattern

Ramon, for me this works setwd(d:/tmp) ffd - as.ffdf(d, col_args=list(pattern = paste(getwd(), /fftmp, sep = ))) filename(ffd) $x [1] d:/tmp/fftmp35c34861.ff $y [1] d:/tmp/fftmp5be946bb.ff $z [1] d:/tmp/fftmp26c49ce.ff Jens -Ursprüngliche Nachricht- Von: Ramon Diaz-Uriarte

Re: [R] ff package: ff objects don't reload compl etely on NFS drives from a different machine

Try to close the file on the first nfs client before reopening it on the second nfs client. NFS has something called close-to-open cache consistency. This means that two clients which have the same nfs file open, cannot rely on seeing the updates from the respective other client. If one clients

[R] [R-pkgs] New version of package ff

Dear R community, Package bit version 1.1-3 and ff version 2.1.2 is available on CRAN and should be useful to handle large datasets. It adds convenient utilities for managing ff objects and files (see ?ffsave) and removes some performance bottlenecks. In case you experience unexpected

Re: [R] A question about the ff package

logical in a ff logical or ff boolean and then do chunked looping over both - the ff with the subscripts and the ffdf - and in each chunk convert the logical selection to integers, see 2) HTH Jens Oehlschlägel P.S. you might want to try the newer version on r-forge. It has several

Re: [R] Ops method does not dispatch on either of two classes

it is not possible to write Ops methods for classes a and b such that |.a is called in a | b ? I don't see how I can get any hook into the dispatch mechanism, my methods are always bypassed if the classes of e1 and e2 differ (simple example below). Best wishes for 2010 Jens Oehlschlägel ca

[R] Ops method does not dispatch on either of two classes

dispatched? Jens Oehlschlägel __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.

Re: [R] Save workspace with ff objects

*and* all ff-files into a ffarchive, i.e. two files ffarchive.RData and ffarchive.ffData from which you can restore all or a selection of ff objects / files using the ffload() command. Regards Jens Oehlschlägel __ R-help@r-project.org mailing list https

Re: [R] questions on the ff package

compiling your OS. Please test and let us know your experience. Regards Jens Oehlschlägel __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html

[R] questions on the ff package

there are new empty rows in the file matFF # 2) Instead of a matrix you create a ffdf data.frame #which you can also give more rows using nrow- #An example of this is in read.table.ffdf #which reads a csv file in chunks and extends the #number of rows in the ffdf Jens Oehlschlägel

Re: [R] Error: cannot allocate vector of size...

. The code below should give an idea of how to do pairwise stats on columns where each pair fits easily into RAM. In the real world, you would not create the data but import it using read.csv.ffdf (expect that reading your file takes longer than reading/writing the ffdf). Regards Jens

[R] [R-pkgs] New version of package ff

Dear R community, ff Version 2.1.1 is available on CRAN. It now supports large data.frames, csv import/export, packed atomic datatypes and bit filtering from package 'bit' on which it depends from now. Some performance results in seconds from test data with 78 mio rows and 7 columns on a 3 GB

Re: [R] Hand-crafting an .RData file

be written row by row from your program) and link the file into R as a single ff_matrix. Since ffdf in ff is new, I give a mini-tutorial below. Let me know how that works for you. Kind regards Jens Oehlschlägel library(ff) # Create example csv fnam - /tmp/example.csv write.csv(data.frame

Re: [R] Incremental ReadLines

Gene, You might want to look at function read.csv.ffdf from package ff which can read large csv-files into a ffdf object. That's kind of data.frame which is stored on disk resp. in the file-system-cache. Once you subscript part of it, you get a regular data.frame. Jens Oehlschlägel

[R] (no subject)

Hi, Does anyone know where the following package is available: Holleczek B, Gondos A, Brenner H. PeriodR - an R package to calculate long term survival estimates using period analysis. Methods of Information in Medicine 2009; 48: 123-128. Thanks Jens Oehlschlägel -- GRATIS für alle GMX

[R] questions on csv reading

-dispatch over the MORE specific class POSIXct. Why? Jens Oehlschlägel -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http

Re: [R] Is there any difference between - and =

not assign in function arguments unless you have good reasons like in system.time(x - something) HTH Jens Oehlschlägel P.S. Disclaimer: you can consider me biased towards -, never trust experts, whether experienced or not. P.P.S. a puzzle, following an old tradition: What is going on here

[R] [R-pkgs] New package: bit 1.0

' are made S3 generic, 'xor.default' is implemented much faster than in base R (this should go into base R). The package has automated regression-tests and is hopefully useful for better handling large datasets, together with packages 'rindex' and 'ff'. Best regards Jens Oehlschlägel Munich

Re: [R] R 2.7.0 is released

Many thanks to the core team for an impressive list of new improvements ... o strwidth() and strheight() gain 'font' and 'vfont' arguments and accept in-line pars such as 'family' in the same way as text() does. (Longstanding wish of PR#776) ... and for not having forgotten an 8 year old wish!

Re: [R] cbind function

,]33 [4,]41 Warning message: In cbind(1:4, 1:3) : number of rows of result is not a multiple of vector length (arg 2) Best regards Jens Oehlschlägel -- __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE