[R] Reshape large Data Frame to new format

2014-03-24 Thread Dark
respectively 40 and 60 codes. I'm a big fan of the data.table package so maybe someone has an solution using that package? But of course any solution is welcome:-) Thanks for any help in advance, Regards Dark -- View this message in context: http://r.789695.n4.nabble.com/Reshape-large-Data-Frame

Re: [R] Create rows for columns in dataframe

2013-08-14 Thread Dark
Hi A.K, Thanks for your great help. I'm now running your first suggestion on a 600.000 row sample after verifying it works on a smaller sample. It's now been running for 40 minutes. Which method do you think will be faster? Regards Derk -- View this message in context:

Re: [R] Create rows for columns in dataframe

2013-08-14 Thread Dark
Hi Arun, The second method is indeed working much faster. It worked fast for my 600.000 row record. Still I have 2 bigger files where processing becomes an issue even though I have lots of memory (32 gig) for the second statement: res2-reshape(dat2,idvar=newCol,varying=list(2:26),direction=long)

[R] Create rows for columns in dataframe

2013-08-13 Thread Dark
Hi experts, I have a dataframe with 100k+ records. it has a key/id column and 25 code columns. I would like to restructure it having a row for each code column. I have a structure like this (used dput): structure(list(DSYSRTKY = structure(c(1L, 2L, 3L, 3L, 4L, 4L), .Names = c(1, 2, 3, 4, 5, 6),

Re: [R] Create rows for columns in dataframe

2013-08-13 Thread Dark
Hi, My desired output for my sample!! using dput(): structure(list(ID = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48), DSYSRTKY = c(10005, 10005,

[R] Add column to dataframe based on code in other column

2013-08-08 Thread Dark
Hi all, I have a dataframe of users which contain US-state codes. Now I want to add a column named REGION based on the state code. I have already done a mapping: NorthEast - c(07, 20, 22, 30, 31, 33, 39, 41, 47) MidWest - c(14, 15, 16, 17, 23, 24, 26, 28, 35, 36, 43, 52) South - c(01, 04, 08,

[R] Retreiving correct data from combining two datasets

2013-08-05 Thread Dark
Hi all,I have two datasets:*Dataset 1 - List of Users:*ID Name C1 C2 C3 C23 C24 C25*Dataset 2 - List of Codes*Code Description CategoryThe code fields in the user-dataset do not have to contain a value and if they have a value they dont have to correspond with the Codes-dataset.Now I

[R] Retreiving correct data from combining two datasets

2013-08-05 Thread Dark
Hi all, I have two datasets: Dataset 1 - List of Users, the layout looks like this: ID Name C1 C2 C3 C23 C24 C25 Dataset 2 - List of Codes, the layout looks like this: Code Description Category The code fields in the user-dataset do not have to contain a value and if they

[R] Add a column to a data frame with value based on the percentile of the row

2013-07-31 Thread Dark
Hi all, I think this should be an easy question for the guru's out here. I have this large data frame (2.500.000 rows, 15 columns) and I want to add a column named SEGMENT to it. The first 5% rows (first 125.000 rows) should have the value Top 5% in the SEGMENT column Then the rows from 5% to

Re: [R] Add a column to a data frame with value based on the percentile of the row

2013-07-31 Thread Dark
Works like a charm, thanks a lot! -- View this message in context: http://r.789695.n4.nabble.com/Add-a-column-to-a-data-frame-with-value-based-on-the-percentile-of-the-row-tp4672711p4672728.html Sent from the R help mailing list archive at Nabble.com.

Re: [R] Add a column to a data frame with value based on the percentile of the row

2013-07-31 Thread Dark
Hi Arun Kirshna, I have tested your method and it will work for me. I only run into one problem. Before I want to do this operation I have sorted my data frame so my rownumbers ar not subsequent. You can see if you first order your example data frame like: dat1 - dat1[order(-dat1$value),]

Re: [R] Saving multiple rda-files as one rda-file

2013-07-25 Thread Dark
Really no one has any suggestions on this issue? -- View this message in context: http://r.789695.n4.nabble.com/Saving-multiple-rda-files-as-one-rda-file-tp4672041p4672278.html Sent from the R help mailing list archive at Nabble.com. __

Re: [R] Saving multiple rda-files as one rda-file

2013-07-25 Thread Dark
Hi, Yes maybe I should have been more clear on my problem. I want to append the different data-frames back into one variable ( rbind ) and save it as one R Data file. Regards Derk -- View this message in context:

[R] Saving multiple rda-files as one rda-file

2013-07-22 Thread Dark
Hi all, For a project we have to process some very large CSV files (up to 40 gig) To reduce them in size and increase operating performance I wanted to store them as RData files. Since it was to big I decided to split the csv and saving those parts as separate .RDA files. So far so good. Now I