Re: [R] efficient equivalent to read.csv / write.csv

On Tue, Sep 28, 2010 at 5:02 PM, statquant2 wrote: > > Hello all, > the test I provided was just to pinpoint that for loading once a big csv A file that can be read in under 2 seconds is not big. > file with read.csv was quicker than read.csv.sql... I have already > "optimized" my calls to read.

Re: [R] efficient equivalent to read.csv / write.csv

Hello all, the test I provided was just to pinpoint that for loading once a big csv file with read.csv was quicker than read.csv.sql... I have already "optimized" my calls to read.csv for my particular problem, but is a simple call to read.csv was quicker than read.csv.sql I doubt that specifying

Re: [R] efficient equivalent to read.csv / write.csv

On Tue, Sep 28, 2010 at 1:24 PM, statquant2 wrote: > > Hi, after testing > R) system.time(read.csv("myfile.csv")) >   user  system elapsed >  1.126   0.038   1.177 > > R) system.time(read.csv.sql("myfile.csv")) >   user  system elapsed >  1.405   0.025   1.439 > Warning messages: > 1: closing unus

Re: [R] efficient equivalent to read.csv / write.csv

On 29/09/2010 6:24 a.m., statquant2 wrote: Hi, after testing R) system.time(read.csv("myfile.csv")) user system elapsed 1.126 0.038 1.177 R) system.time(read.csv.sql("myfile.csv")) user system elapsed 1.405 0.025 1.439 Warning messages: 1: closing unused connection 4 ()

Re: [R] efficient equivalent to read.csv / write.csv

To speed things up, you certainly want to give R more clues about your data files by being more explicit by many of the arguments (cf. help(read.table), especially specifying argument 'colClasses' makes a big difference. /Henrik On Tue, Sep 28, 2010 at 10:24 AM, statquant2 wrote: > > Hi, after t

Re: [R] efficient equivalent to read.csv / write.csv

Hi, after testing R) system.time(read.csv("myfile.csv")) user system elapsed 1.126 0.038 1.177 R) system.time(read.csv.sql("myfile.csv")) user system elapsed 1.405 0.025 1.439 Warning messages: 1: closing unused connection 4 () 2: closing unused connection 3 () It seems that

Re: [R] efficient equivalent to read.csv / write.csv

On Mon, Sep 27, 2010 at 7:49 AM, statquant2 wrote: > > thank you very much for this sql package, the thing is that thoses table I > read are loaded into memory once and for all, and then we work with the > data.frames... > Do you think then that this is going to be quicker (as I would have thougth

Re: [R] efficient equivalent to read.csv / write.csv

thank you very much for this sql package, the thing is that thoses table I read are loaded into memory once and for all, and then we work with the data.frames... Do you think then that this is going to be quicker (as I would have thougth that building the SQL DB from the flat file would already be

Re: [R] efficient equivalent to read.csv / write.csv

On 26.09.2010 14:38, statquant2 wrote: Hello everyone, I currently run R code that have to read 100 or more large csv files (>= 100 Mo), and usually write csv too. My collegues and I like R very much but are a little bit ashtonished by how slow those functions are. We have looked on every argu

Re: [R] efficient equivalent to read.csv / write.csv

On Sun, Sep 26, 2010 at 8:38 AM, statquant2 wrote: > > Hello everyone, > I currently run R code that have to read 100 or more large csv files (>= 100 > Mo), and usually write csv too. > My collegues and I like R very much but are a little bit ashtonished by how > slow those functions are. We have

[R] efficient equivalent to read.csv / write.csv

Hello everyone, I currently run R code that have to read 100 or more large csv files (>= 100 Mo), and usually write csv too. My collegues and I like R very much but are a little bit ashtonished by how slow those functions are. We have looked on every argument of those functions and if specifying s