On 13/05/2010, at 2:43 PM, Sean Seefried wrote: > I've got a question that I hope generates healthy debate and perhaps > even a solution for me. Without going into too much detail I'm > working on a project in which we perform calculations on a large > hierarchical data set. We haven't used the acts_as_tree or > acts_as_nested plugins because each level in the hierarchy has a well > defined role and various attributes that only fit at that level in the > hierarchy. To give you a brief taste the hierarchy roughly goes: local > government area, precinct, building type, consumption.
I would look at some database solutions to speed up access and aggregation. But the biggest question is, do you have total control over the database? Or are you plugging in to someone else's system and have to take what you get? I had problems with queries that were accessing multi million row tables, the queries were taking up to 5-10 minutes... the final solution was actually making ANOTHER "cooked data" table that contained all the cooked data of the major datums I needed to query off. This cached table was updated constantly. First by doing a complete run through the system, inserting the rows on every table, and then kept up to date with after save hooks on the records in question. The resulting table size was only about 10mb, even though it was about 400,000 rows long. Then I would query against this table (in the 10th of a second range on queries) and find the id's of the main table I needed, which then I selected out... all using ActiveRecord abstractions nicely and cleanly. This worked because instead of the database having to deal with 4 different 300-800mb tables, it could just fly through the tiny table that just consisted of about 5 integer columns and about 20 boolean columns. All data was always current because whenever a row in the main table changed, it updated it's corresponding "cooked data" table row by doing a query on itself across the 4 different tables and updating the data. The solution was neat and simple and still working today and getting whacked by users as hard as ever. Of course, YMMV but you can contact me off list if you want more data :) Mikel http://rubyx.com/ http://lindsaar.net/ -- You received this message because you are subscribed to the Google Groups "Ruby or Rails Oceania" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/rails-oceania?hl=en.
