Re: How to avoid listing all objects initially until searching

[Jean Stebens]

Tom Evans wrote:
> On Fri, Dec 11, 2009 at 4:04 PM, Jean Stebens <neme...@twilightzone.lu> wrote:
>> Daniel Roseman wrote:
>>> On Dec 11, 2:51 pm, Jean Stebens <neme...@twilightzone.lu> wrote:
>>>> Hi there,
>>>>
>>>> I have a rather large set of objects in my database, around 200k, using
>>>> the django admin interface and listing them is always terrible slow as
>>>> it always fetches all entries (paginated). I usually just run a few
>>>> searches or add some filters on the list. Is there a way to have the
>>>> list not shown initially until I do add some filters/searches?
>>>>
>>>> Cheers,
>>>> Jean
>>> This seems strange. Are you talking about the changelist view? This
>>> never fetches all objects - it only fetches the objects for one page
>>> at a time.
>>> --
>>> DR.
>>>
>> Yes, I'm talking about the changelist in the admin interface. With my
>> current set of 200k entries I have 1990 pages each containing 100 entries.
>>
>> applying a filter on that initial list makes it load relative fast. I've
>> tried commenting out any calculated fields when displaying the objects,
>> but that doesnt improve the performance, my best guess is the initial
>> "show all" queryset set off by the framework.
>>
>> Cheers,
>> Jean
>>
> 
> Do you have custom sorting on that model? Eg, setting 'ordering' on
> the models meta class?
> 
> You need to look at the SQL query being generated for your initial
> query and figure out why it is slow. Ordering by a field that is not a
> dbindex would require a full table scan in most db engines, and that
> would be quite slow.
> 
> Once you restrict the elements by a filter that is indexed, that would
> speed it up so that you would not notice ordering by an unindexed
> field.
> 
> Cheers
> 
> Tom

The default ordering for the object is set on it's id, which is defined
as primary key for the object. The complete database is optimized in
terms of indexes as this is a live monitoring system with more than 40k
servers.

Any idea how to get the generated SQL from django? (except capping it in
innotop)

Cheers,
Jean

--

You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: How to avoid listing all objects initially until searching

[Jean Stebens]

Daniel Roseman wrote:
> On Dec 11, 2:51 pm, Jean Stebens <neme...@twilightzone.lu> wrote:
>> Hi there,
>>
>> I have a rather large set of objects in my database, around 200k, using
>> the django admin interface and listing them is always terrible slow as
>> it always fetches all entries (paginated). I usually just run a few
>> searches or add some filters on the list. Is there a way to have the
>> list not shown initially until I do add some filters/searches?
>>
>> Cheers,
>> Jean
> 
> This seems strange. Are you talking about the changelist view? This
> never fetches all objects - it only fetches the objects for one page
> at a time.
> --
> DR.
> 
> --
> 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To post to this group, send email to django-us...@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.
> 
> 
Yes, I'm talking about the changelist in the admin interface. With my
current set of 200k entries I have 1990 pages each containing 100 entries.

applying a filter on that initial list makes it load relative fast. I've
tried commenting out any calculated fields when displaying the objects,
but that doesnt improve the performance, my best guess is the initial
"show all" queryset set off by the framework.

Cheers,
Jean

--

You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

How to avoid listing all objects initially until searching

[Jean Stebens]

Hi there,

I have a rather large set of objects in my database, around 200k, using
the django admin interface and listing them is always terrible slow as
it always fetches all entries (paginated). I usually just run a few
searches or add some filters on the list. Is there a way to have the
list not shown initially until I do add some filters/searches?

Cheers,
Jean

--

You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Admin inlines onetoone related object not saved

[Jean Stebens]

Hi there,

I have the following models:

class A(models.Model):
id = models.AutoField(primary_key=True)

class B(models.Model):
mya = models.OneToOneField(A, primary_key=True)
text = models.TextField(default='Init')

In admin interface, I have used an inline to include B when
changing/adding an A. Adding/Changing an A without touching the B inline
results in A beeing saved, but not B. Changing the textfield from B into
sth other than default and it works, but I want to keep the default as
is - Is there any solution for this?

Thanks!
Jean


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Intermediary model and filter_horizontal problem

[Jean Stebens]

Hi All,

My Problem: I'm adapting the generated models from inspectdb in order to 
use a legacy application with django. The application database schema 
has intermediary tables in it - I'd want to use the nice 
"filter_horizontal" instead of inlines. Inlines do work, so the 
intermediary table lookup works fine - but with filter_horizontal the 
admin panel just doesnt show up the relations - no warnings nor errors.

Thanks for having a look at it,
Greetings from Luxemburg,
Jean

#models.py

class Person(models.Model):
   id = models.AutoField(primary_key=True, db_column='PersonId')
   name = models.CharField(max_length=300, db_column='PersonName')
   groups = models.ManyToManyField('Group', through='Membership')

class Group(models.Model):
   id = models.AutoField(primary_key=True, db_column='GroupId')
   name = models.CharField(max_length=300, db_column='GroupName')
   members = models.ManyToManyField('Person', through='Membership')

class Membership(models.Model):
   id = models.AutoField(primary_key=True, db_column='Id')
   person = models.ForeignKey(Person, db_column='PersonId')
   group = models.ForeignKey(Group, db_column='GroupId')

#admin.py

class Membership_inline(admin.TabularInline):
   model = Membership
   extra = 1

class PersonAdmin(admin.ModelAdmin):
   search_fields = ['name']
   #inlines = [ Membership_inline ]
   filter_horizontal = ('groups',)

class GroupAdmin(admin.ModelAdmin):
   search_fields = ['name']
   #inlines = [ Membership_inline ]
   filter_horizontal = ('persons',)

admin.site.register(Person, PersonAdmin)
admin.site.register(Group, GroupAdmin)




--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: Custom model field and Django-admin -- error

[Jean Stebens]

Alex Gaynor wrote:
> On Mon, Aug 10, 2009 at 8:45 AM, BenW wrote:
>   
>> Hello,
>>
>> I'm working with a legacy database that stores datetimes as unsigned
>> ints.  Rather than do the conversion with properties on the model I've
>> written a custom Field 'UnixDateTimeField':
>>
>> class UnixDateTimeField(models.DateTimeField):
>>
>>__metaclass__ = models.SubfieldBase
>>
>>def get_internal_type(self):
>>return 'PositiveIntegerField'
>>
>>def to_python(self, value):
>>if value is None or isinstance(value, datetime):
>>return value
>>if isinstance(value, date):
>>return datetime(value.year, value.month, value.day)
>>return datetime.fromtimestamp( float(value) )
>>
>>def get_db_prep_value(self, value):
>>return int( time.mktime( value.timetuple() ) )
>>
>>def value_to_string(self, obj):
>>value = self._get_val_from_obj(obj)
>>return self.to_python(value).strftime('%Y-%m-%d %H:%M:%S')
>>
>> class MyModel(models.Model):
>>time = UnixDateTimeField()
>>
>> # This Works:
>> 
 MyModel.objects.filter(time__lte=datetime.now())
 
>> [, , etc..]
>> 
>> # This works:
>> 
 MyModel.objects.all()[0].time
 
>> datetime.datetime(2009, 8, 10, 6, 40, 7)
>> 
>> # Doesn't work:
>> 
 MyModel.objects.all().values('time')
 
>> [{'time': 1249911607L}, {'time': 1249911607L}, {'time':
>> 1249911607L}, ...]
>> 
>> The same thing happens in the Admin when I specify date_hierarchy in
>> my ModelAdmin a one of these fields.  Why are the standard accessor
>> methods (namely 'to_python()') not being called here?  How can I make
>> the custom Field more robust?
>>
>> Thank you,
>>
>> Ben
>> 
>
> This looks like ticket #9619: http://code.djangoproject.com/ticket/9619
>
> Alex
>
>   
Beside that - I would like to see the UnixDateTimeField in django by 
default - it's not uncommon to use unix timestamps for applications 
which do support quite alot of databases, especially when mssql is one 
of them.

Cheers,
Jean

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Intermediary model and filter_horizontal problem

[Jean Stebens]

Hi All,

My Problem: I'm adapting the generated models from inspectdb in order to 
use a legacy application with django. The application database schema 
has intermediary tables in it - I'd want to use the nice 
"filter_horizontal" instead of inlines. Inlines do work, so the 
intermediary table lookup works fine - but with filter_horizontal the 
admin panel just doesnt show up the relations - no warnings nor errors.

Thanks for having a look at it,
Greetings from Luxemburg,
Jean

#models.py

class Person(models.Model):
id = models.AutoField(primary_key=True, db_column='PersonId')
name = models.CharField(max_length=300, db_column='PersonName')
groups = models.ManyToManyField('Group', through='Membership')

class Group(models.Model):
id = models.AutoField(primary_key=True, db_column='GroupId')
name = models.CharField(max_length=300, db_column='GroupName')
members = models.ManyToManyField('Person', through='Membership')

class Membership(models.Model):
id = models.AutoField(primary_key=True, db_column='Id')
person = models.ForeignKey(Person, db_column='PersonId')
group = models.ForeignKey(Group, db_column='GroupId')

#admin.py

class Membership_inline(admin.TabularInline):
model = Membership
extra = 1

class PersonAdmin(admin.ModelAdmin):
search_fields = ['name']
#inlines = [ Membership_inline ]
filter_horizontal = ('groups',)

class GroupAdmin(admin.ModelAdmin):
search_fields = ['name']
#inlines = [ Membership_inline ]
filter_horizontal = ('persons',)

admin.site.register(Person, PersonAdmin)
admin.site.register(Group, GroupAdmin)



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---