Re: [R] How to set a filter during reading tables

Thanks, Juliet. It works for filtering columns. I am also wondering if there is a way to filter rows. Thanks again. -james One can use colClasses to set which columns get read in. For the columns you don't want you can set those to NULL. For example, cc - c(NULL,rep(numeric,9)) myData -

Re: [R] How to set a filter during reading tables

I think you can use readLines(n=1) in loop to skip unwanted rows. On Mon, Jun 1, 2009 at 12:56 AM, g...@ucalgary.ca wrote: Thanks, Juliet. It works for filtering columns. I am also wondering if there is a way to filter rows. Thanks again. -james One can use colClasses to set which columns

Re: [R] How to set a filter during reading tables

The sqldf package can read a subset of rows and columns into R without reading the entire file into R. There are a few caveats: - It does not support ftp so you will need to download the file to your computer first as shown in the example below - since value is an SQL keyword it turns value

Re: [R] How to set a filter during reading tables

Since there are many rows, using read.table we spent too much on reading in rows that we do not want. We are wondering if there is a way to read only rows that we are interested in. Thanks, -james I think you can use readLines(n=1) in loop to skip unwanted rows. On Mon, Jun 1, 2009 at 12:56

Re: [R] How to set a filter during reading tables

There are several things you can tell read.table to make it faster. First, as mentioned, setting colClasses helps. I think telling read.table how many rows and columns there are also helps. When this was not sufficient, I've had to do the data processing using Python, Perl, or awk. If that had

Re: [R] How to set a filter during reading tables

One can use colClasses to set which columns get read in. For the columns you don't want you can set those to NULL. For example, cc - c(NULL,rep(numeric,9)) myData - read.table(myFile.txt,header=TRUE,colClasses=cc,nrow=numRows). On Wed, May 27, 2009 at 12:27 PM, g...@ucalgary.ca wrote: We are

[R] How to set a filter during reading tables

We are reading big tables, such as, Chemicals - read.table('ftp://ftp.bls.gov/pub/time.series/wp/wp.data.7.Chemicals',header = TRUE, sep = '\t', as.is =T) I was wondering if it is possible to set a filter during loading so that we just load what we want not the whole table each time. Thanks,