I work with Sean on this project, hence my 2 bob's worth below:
On 13/05/2010, at 2:50 PM, Anthony Richardson wrote: > 80,000 records is pretty small. Sure, but there are multiple passes over the data being done, with quite lengthy calculations at each stage. Iterating over selections of data, calculating interim results, then writing the results out again... there's only so many ways to do that. > Assuming your calcuations/processing can be expressed in the native function > language of your Database you should be able to get the database to do this > fast. This is the bread and butter of databases. They could, true. We are going to have to switch from MySQL to Postgres in the not-too-distant future, unfortunately, so I wouldn't look forward to this solution. > Almost certainly using ruby to manually process 80,000 objects is a bad idea > within a request. Yep, discovered that pretty quickly. We're using delayed_job and ajax progress indicators to manage the tasks. It's the sort of thing where you spend quite a bit of time configuring a lot of info, then execute the calculations. -- You received this message because you are subscribed to the Google Groups "Ruby or Rails Oceania" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/rails-oceania?hl=en.
