To update on this. I ran the same command on a grid of computers with 32gb
ram, and it completed in 15 seconds, compared to the ~20 minutes on my
desktop.
Simply a ram issue as suspected by a few on the list here.
Thanks
--
View this message in context:
Provide and 'str' and 'object.size' of the object
so that we can see what you are working with. My rule of thumb is
that no single object should take more than 25-30% of memory since
copies may be made. So the reasons things are taking 20 minutes is
you might be paging. It is always good
I think you are probably paging on your system. Turn on your
performance metrics and look at it. If the object you are processing
is all numeric, it would seem to require about 3.5GB of space (50% of
available memory). Provide and 'str' and 'object.size' of the object
so that we can see what
Two questions:
1) Are there any good R guides/sites with information/techniques for dealing
with large datasets in R? (Large being ~2 mil rows and ~200 columns)
2) My specific problem with this dataset.
I am essentially trying to convert a date and add it to a data frame. I
imagine any 'data
4 matches
Mail list logo