On 12/30/09 9:58 AM, "Saravanan" <sarankannan2...@gmail.com> wrote:
> Waqqas Jabbar, thank you for your answer. This is a simple solution. > By the way, I am inserting few million rows a day. I am accessing this > table from many other application. It is not a Django dedicated table. > I have an external database reader with my customized open and close. > When I add few million rows a day, and I am retaining data for three > months, this adds up to atleast 90 million rows. > It is better to split, a table for everyday's data. > > On Dec 29, 11:36 pm, Waqqas Jabbar <waqqas.jab...@gmail.com> wrote: >> You can make one table and have date-time field in it corresponding to the >> time when the >> row is inserted. Then you can give a query to get for a day, days, month, or >> a year. >> >> On Wed, Dec 30, 2009 at 2:08 AM, Saravanan <sarankannan2...@gmail.com>wrote: >> >>> Here I have an issue, I don't know how to proceed. >>> Here what I have in models.py >>> I have a base class: for example say class base_class(models.Model) >>> In this base class I have definition for a db table, class Meta: >>> abstract = True >>> I have an manager class : class manager_class(models.Manager) >>> In this derived class I coded return reverse functionality >> >>> In view, >>> I am deriving a new class view_class(base_class) >>> Here I am setting Class Meta:db_table = my_table. >> >>> In this scenario, my_table is dynamic. Its physical structure is same >>> however, a new table created for every day. >>> Hence, there will be more than one table with same skeleton for every >>> day. >>> When I search for based on date, it should search from corresponding >>> date db table. >>> It is doing it only for the first request. Successive requests it is >>> searching from the same table. (Table name is part build using date >>> which user dynamically enters). >>> I manipulated the table name with user entered date however it is not >>> recognized!!! in the consecutive search. >>> When I restart my apache, it fetchs from the request table. >> >>> How can I tell this code to search from different table for every >>> request. (once again physical structures are >>> the same just one table for every day). In one sentence, When I print >>> the table name it prints correct table name however actual data is >>> fetched from first requested table. >> >>> -- >> >>> You received this message because you are subscribed to the Google Groups >>> "Django users" group. >>> To post to this group, send email to django-us...@googlegroups.com. >>> To unsubscribe from this group, send email to >>> django-users+unsubscr...@googlegroups.com<django-users%2bunsubscr...@googleg >>> roups.com> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/django-users?hl=en. > > -- > > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-us...@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > > Hello, Here are a couple of thoughts: - Splitting the data up in some other way For example, have 31 tables, for each day of the month, or some other ranges. Record which month a record was inserted for, and clear out records past the 3 month horizon. That would at least cut down the number of records in each table. Generating 31 models will be a pain, but you could probably do something with metaclasses, which leads to... - Use metaclasses I'm not sure how to dynamically register a new model with the ORM, so that would be an obstacle, but I'm pretty sure you could use metaclasses to generate models with a dynamic table name. Every time I've seen that technique used, though, an app restart was required. You could also probably do something with the new multi-db functionality. I'd make sure that your backend has trouble with 90 million records. Depending on the complexity of the table, even these optimizations might not be worth it. An index on the date field might be sufficient. Hope that helps, Peter -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
