Hi, today I went into a problem, when I had to iterate over a big result set. ActiveRecord produces a huge array of model instances, which consumed 1GB of memory on my machine. A comparable perl DBI script only used some KB for iterating over the result set.
Then I was suggested to use batch (#find_each) for the problem. But basically batch does the same thing by splitting the query into several queries, which only return 1000 model instances in an array, which still consumes more memory than necessary, but not that much. The problem: it was much slower (it took 25 minutes, while the version without batch took 90 seconds and the perl script took only 40 seconds) on my legacy database. So I was searching for a method, which yields a model instance at a time from the result set of the query, but I could not find anything. At the end, I tried to find a solution myself and so I came up to this simple solution https://github.com/tvw/ar-each_model I want to put into discussion here, since I wonder, why similar things do not exist in ActiveRecord. Regards Thomas -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
