subject:"Django performance issue that's troubling"

Re: Django performance issue that's troubling

2018-08-24 Thread Jim Illback

Thanks much, Tim. I’ll check it out - probably will need even more tuning down 
the road...

On Aug 24, 2018, at 1:00 PM, Tim Vogt (Tim Vogt) 
mailto:timtv...@gmail.com>> wrote:

Also found this perhaps helpfull
https://blog.labdigital.nl/working-with-huge-data-sets-in-django-169453bca049

Op 24 aug. 2018, om 21:28 heeft Jim Illback 
mailto:subaru...@hotmail.com>> het volgende geschreven:

Vijay, your suggestion worked. I had tried this previously but it was using a 
FBV and it didn’t work. But with this CBV app, it works perfectly. Thanks.

Matthew, I’ll definitely keep your suggestion in mind - maybe back on my FBV 
application if I refactor!

Thanks much to both of you!
Jim

On Aug 24, 2018, at 11:19 AM, Matthew Pava 
mailto:matthew.p...@iss.com>> wrote:

It’s important that you set the widget to a HiddenInput.  It only has to render 
it with the current value.  If you are keeping the widget to be a Select widget 
and then setting a CSS class to make it hidden, the system will still have to 
load all 37K values.  I guess I’m missing something in your explanation.

From: django-users@googlegroups.com<mailto:django-users@googlegroups.com> 
[mailto:django-users@googlegroups.com] On Behalf Of Jim Illback
Sent: Friday, August 24, 2018 1:16 PM
To: Django users
Subject: Re: Django performance issue that's troubling

Matthew, thanks - that was my original code, but even hidden, it has to fill in 
all possible values - 37K - so it takes multiple seconds.

Vijay, I’ll try your suggestion.

Thanks much,
Jim


On Aug 24, 2018, at 11:01 AM, Matthew Pava 
mailto:matthew.p...@iss.com>> wrote:

You can keep the field, but change its widget to a HiddenInput.
https://stackoverflow.com/questions/6862250/change-a-django-form-field-to-a-hidden-field



From: django-users@googlegroups.com<mailto:django-users@googlegroups.com> 
[mailto:django-users@googlegroups.com] On Behalf Of Vijay Khemlani
Sent: Friday, August 24, 2018 12:59 PM
To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>
Subject: Re: Django performance issue that's troubling

Please reply to the user group

"But, in order to have the update work, it needs the ForeignKey to the parent 
in the form"

As far as I know, it doesn't need to (if the entry already exists in the 
database and you are just editing it), you just need to set up the fields that 
you want to be editable by the user


On Fri, Aug 24, 2018 at 2:06 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
Vijay, think of this is a parent - child relationship. I have 37K parents. Now 
I want to update a child of one of the parents. But, in order to have the 
update work, it needs the ForeignKey to the parent in the form. However, to add 
that key, Django must then allow it to update that field and thus has to fill 
in all 37K entries. Is there no way to tell Django that the FK relationship can 
not be changed? If I leave it off of the form, the update fails because it is a 
required field. If I add it to the form, Django has to allow all potential 
update values to be shown. What’s the way out of this mess?

Thanks,
Jim




Begin forwarded message:

From: Vijay Khemlani mailto:vkhem...@gmail.com>>
Subject: Re: Django performance issue that's troubling
Date: August 24, 2018 at 5:07:07 AM PDT
To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>
Reply-To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>

To tell you the truth I couldn't understand some parts of your message, but if 
your AddictionForm is a ModelForm then you should be able to limit its fields 
to just the ones you want your users to be able to update.

https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/

Regards!



On Thu, Aug 23, 2018 at 8:19 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
Vijay, you are exactly correct. Thank you much!

Now, knowing the problem, I still don’t have a great solution. Here’s how it is 
supposed to work.

Someone chooses a client from the list or a search. All the possible support 
records are shown, whether they exist or not, on a screen to hyperlink to one 
of them. If the record exists, the URL includes the PK and transfers to the 
UpdateView. This is similar for an AddView if the record does not exist. 
However, in the Update or Add, I don’t want the primary key changed. So, I hid 
the field (and thus all the records were read). However, now I can’t limit that 
field to only one value. If I format the entire SELECT tag with only one 
OPTION, Django gives an invalid expression for the ID or Name because it is 
based on my query set I passed in that fills out the values (ID and name). If I 
use the form’s field value, all the fields are read. Especially on an Add, I 
don’t know a way to fill the client ID field in such a way that Django will 
take it as valid.

My thoughts on possible solutions:
1. Override the get query set method to only

Re: Django performance issue that's troubling

2018-08-24 Thread Tim Vogt (Tim Vogt)

Also found this perhaps helpfull
https://blog.labdigital.nl/working-with-huge-data-sets-in-django-169453bca049

> Op 24 aug. 2018, om 21:28 heeft Jim Illback  het 
> volgende geschreven:
> 
> Vijay, your suggestion worked. I had tried this previously but it was using a 
> FBV and it didn’t work. But with this CBV app, it works perfectly. Thanks.
> 
> Matthew, I’ll definitely keep your suggestion in mind - maybe back on my FBV 
> application if I refactor!
> 
> Thanks much to both of you!
> Jim
> 
>> On Aug 24, 2018, at 11:19 AM, Matthew Pava > <mailto:matthew.p...@iss.com>> wrote:
>> 
>> It’s important that you set the widget to a HiddenInput.  It only has to 
>> render it with the current value.  If you are keeping the widget to be a 
>> Select widget and then setting a CSS class to make it hidden, the system 
>> will still have to load all 37K values.  I guess I’m missing something in 
>> your explanation.
>>  
>> From: django-users@googlegroups.com <mailto:django-users@googlegroups.com> 
>> [mailto:django-users@googlegroups.com 
>> <mailto:django-users@googlegroups.com>] On Behalf Of Jim Illback
>> Sent: Friday, August 24, 2018 1:16 PM
>> To: Django users
>> Subject: Re: Django performance issue that's troubling
>>  
>> Matthew, thanks - that was my original code, but even hidden, it has to fill 
>> in all possible values - 37K - so it takes multiple seconds.
>>  
>> Vijay, I’ll try your suggestion.
>>  
>> Thanks much,
>> Jim
>> 
>> 
>> On Aug 24, 2018, at 11:01 AM, Matthew Pava > <mailto:matthew.p...@iss.com>> wrote:
>>  
>> You can keep the field, but change its widget to a HiddenInput.
>> https://stackoverflow.com/questions/6862250/change-a-django-form-field-to-a-hidden-field
>>  
>> <https://stackoverflow.com/questions/6862250/change-a-django-form-field-to-a-hidden-field>
>>  
>>  
>>  
>> From: django-users@googlegroups.com <mailto:django-users@googlegroups.com> 
>> [mailto:django-users@googlegroups.com 
>> <mailto:django-users@googlegroups.com>] On Behalf Of Vijay Khemlani
>> Sent: Friday, August 24, 2018 12:59 PM
>> To: django-users@googlegroups.com <mailto:django-users@googlegroups.com>
>> Subject: Re: Django performance issue that's troubling
>>  
>> Please reply to the user group
>>  
>> "But, in order to have the update work, it needs the ForeignKey to the 
>> parent in the form"
>>  
>> As far as I know, it doesn't need to (if the entry already exists in the 
>> database and you are just editing it), you just need to set up the fields 
>> that you want to be editable by the user
>>  
>>  
>> On Fri, Aug 24, 2018 at 2:06 PM Jim Illback > <mailto:subaru...@hotmail.com>> wrote:
>> Vijay, think of this is a parent - child relationship. I have 37K parents. 
>> Now I want to update a child of one of the parents. But, in order to have 
>> the update work, it needs the ForeignKey to the parent in the form. However, 
>> to add that key, Django must then allow it to update that field and thus has 
>> to fill in all 37K entries. Is there no way to tell Django that the FK 
>> relationship can not be changed? If I leave it off of the form, the update 
>> fails because it is a required field. If I add it to the form, Django has to 
>> allow all potential update values to be shown. What’s the way out of this 
>> mess?
>>  
>> Thanks,
>> Jim
>>  
>> 
>> 
>> 
>> Begin forwarded message:
>>  
>> From: Vijay Khemlani mailto:vkhem...@gmail.com>>
>> Subject: Re: Django performance issue that's troubling
>> Date: August 24, 2018 at 5:07:07 AM PDT
>> To: django-users@googlegroups.com <mailto:django-users@googlegroups.com>
>> Reply-To: django-users@googlegroups.com 
>> <mailto:django-users@googlegroups.com>
>>  
>> To tell you the truth I couldn't understand some parts of your message, but 
>> if your AddictionForm is a ModelForm then you should be able to limit its 
>> fields to just the ones you want your users to be able to update.
>>  
>> https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/ 
>> <https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/>
>>  
>> Regards!
>>  
>>  
>>  
>> On Thu, Aug 23, 2018 at 8:19 PM Jim Illback > <mailto:subaru...@hotmail.com>> wrote:
>> Vijay, you are exactly correct. Thank you much! 
>>  
>> Now, knowing the problem, I still don’t have a great solution. Here’s how it

Re: Django performance issue that's troubling

2018-08-24 Thread Jim Illback

Vijay, your suggestion worked. I had tried this previously but it was using a 
FBV and it didn’t work. But with this CBV app, it works perfectly. Thanks.

Matthew, I’ll definitely keep your suggestion in mind - maybe back on my FBV 
application if I refactor!

Thanks much to both of you!
Jim

On Aug 24, 2018, at 11:19 AM, Matthew Pava 
mailto:matthew.p...@iss.com>> wrote:

It’s important that you set the widget to a HiddenInput.  It only has to render 
it with the current value.  If you are keeping the widget to be a Select widget 
and then setting a CSS class to make it hidden, the system will still have to 
load all 37K values.  I guess I’m missing something in your explanation.

From: django-users@googlegroups.com<mailto:django-users@googlegroups.com> 
[mailto:django-users@googlegroups.com] On Behalf Of Jim Illback
Sent: Friday, August 24, 2018 1:16 PM
To: Django users
Subject: Re: Django performance issue that's troubling

Matthew, thanks - that was my original code, but even hidden, it has to fill in 
all possible values - 37K - so it takes multiple seconds.

Vijay, I’ll try your suggestion.

Thanks much,
Jim


On Aug 24, 2018, at 11:01 AM, Matthew Pava 
mailto:matthew.p...@iss.com>> wrote:

You can keep the field, but change its widget to a HiddenInput.
https://stackoverflow.com/questions/6862250/change-a-django-form-field-to-a-hidden-field



From: django-users@googlegroups.com<mailto:django-users@googlegroups.com> 
[mailto:django-users@googlegroups.com] On Behalf Of Vijay Khemlani
Sent: Friday, August 24, 2018 12:59 PM
To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>
Subject: Re: Django performance issue that's troubling

Please reply to the user group

"But, in order to have the update work, it needs the ForeignKey to the parent 
in the form"

As far as I know, it doesn't need to (if the entry already exists in the 
database and you are just editing it), you just need to set up the fields that 
you want to be editable by the user


On Fri, Aug 24, 2018 at 2:06 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
Vijay, think of this is a parent - child relationship. I have 37K parents. Now 
I want to update a child of one of the parents. But, in order to have the 
update work, it needs the ForeignKey to the parent in the form. However, to add 
that key, Django must then allow it to update that field and thus has to fill 
in all 37K entries. Is there no way to tell Django that the FK relationship can 
not be changed? If I leave it off of the form, the update fails because it is a 
required field. If I add it to the form, Django has to allow all potential 
update values to be shown. What’s the way out of this mess?

Thanks,
Jim




Begin forwarded message:

From: Vijay Khemlani mailto:vkhem...@gmail.com>>
Subject: Re: Django performance issue that's troubling
Date: August 24, 2018 at 5:07:07 AM PDT
To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>
Reply-To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>

To tell you the truth I couldn't understand some parts of your message, but if 
your AddictionForm is a ModelForm then you should be able to limit its fields 
to just the ones you want your users to be able to update.

https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/

Regards!



On Thu, Aug 23, 2018 at 8:19 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
Vijay, you are exactly correct. Thank you much!

Now, knowing the problem, I still don’t have a great solution. Here’s how it is 
supposed to work.

Someone chooses a client from the list or a search. All the possible support 
records are shown, whether they exist or not, on a screen to hyperlink to one 
of them. If the record exists, the URL includes the PK and transfers to the 
UpdateView. This is similar for an AddView if the record does not exist. 
However, in the Update or Add, I don’t want the primary key changed. So, I hid 
the field (and thus all the records were read). However, now I can’t limit that 
field to only one value. If I format the entire SELECT tag with only one 
OPTION, Django gives an invalid expression for the ID or Name because it is 
based on my query set I passed in that fills out the values (ID and name). If I 
use the form’s field value, all the fields are read. Especially on an Add, I 
don’t know a way to fill the client ID field in such a way that Django will 
take it as valid.

My thoughts on possible solutions:
1. Override the get query set method to only pass in the single record - will 
that work?
2. I’ve been trying to show the field in a  tag and override the View’s post 
method by aftering is_valid(), but it always returns invalid expression errors.

What am I missing?

Thanks much again -
Jim




On Aug 23, 2018, at 3:48 PM, Vijay Khemlani 
mailto:vkhem...@gmail.com>> wrote:

Maybe your update form has a field related to a model with thousands of 
r

RE: Django performance issue that's troubling

2018-08-24 Thread Matthew Pava

It’s important that you set the widget to a HiddenInput.  It only has to render 
it with the current value.  If you are keeping the widget to be a Select widget 
and then setting a CSS class to make it hidden, the system will still have to 
load all 37K values.  I guess I’m missing something in your explanation.

From: django-users@googlegroups.com [mailto:django-users@googlegroups.com] On 
Behalf Of Jim Illback
Sent: Friday, August 24, 2018 1:16 PM
To: Django users
Subject: Re: Django performance issue that's troubling

Matthew, thanks - that was my original code, but even hidden, it has to fill in 
all possible values - 37K - so it takes multiple seconds.

Vijay, I’ll try your suggestion.

Thanks much,
Jim

On Aug 24, 2018, at 11:01 AM, Matthew Pava 
mailto:matthew.p...@iss.com>> wrote:

You can keep the field, but change its widget to a HiddenInput.
https://stackoverflow.com/questions/6862250/change-a-django-form-field-to-a-hidden-field

From: django-users@googlegroups.com<mailto:django-users@googlegroups.com> 
[mailto:django-users@googlegroups.com] On Behalf Of Vijay Khemlani
Sent: Friday, August 24, 2018 12:59 PM
To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>
Subject: Re: Django performance issue that's troubling

Please reply to the user group

"But, in order to have the update work, it needs the ForeignKey to the parent 
in the form"

As far as I know, it doesn't need to (if the entry already exists in the 
database and you are just editing it), you just need to set up the fields that 
you want to be editable by the user

On Fri, Aug 24, 2018 at 2:06 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
Vijay, think of this is a parent - child relationship. I have 37K parents. Now 
I want to update a child of one of the parents. But, in order to have the 
update work, it needs the ForeignKey to the parent in the form. However, to add 
that key, Django must then allow it to update that field and thus has to fill 
in all 37K entries. Is there no way to tell Django that the FK relationship can 
not be changed? If I leave it off of the form, the update fails because it is a 
required field. If I add it to the form, Django has to allow all potential 
update values to be shown. What’s the way out of this mess?

Thanks,
Jim

Begin forwarded message:

From: Vijay Khemlani mailto:vkhem...@gmail.com>>
Subject: Re: Django performance issue that's troubling
Date: August 24, 2018 at 5:07:07 AM PDT
To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>
Reply-To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>

To tell you the truth I couldn't understand some parts of your message, but if 
your AddictionForm is a ModelForm then you should be able to limit its fields 
to just the ones you want your users to be able to update.

https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/

Regards!

On Thu, Aug 23, 2018 at 8:19 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
Vijay, you are exactly correct. Thank you much!

Now, knowing the problem, I still don’t have a great solution. Here’s how it is 
supposed to work.

Someone chooses a client from the list or a search. All the possible support 
records are shown, whether they exist or not, on a screen to hyperlink to one 
of them. If the record exists, the URL includes the PK and transfers to the 
UpdateView. This is similar for an AddView if the record does not exist. 
However, in the Update or Add, I don’t want the primary key changed. So, I hid 
the field (and thus all the records were read). However, now I can’t limit that 
field to only one value. If I format the entire SELECT tag with only one 
OPTION, Django gives an invalid expression for the ID or Name because it is 
based on my query set I passed in that fills out the values (ID and name). If I 
use the form’s field value, all the fields are read. Especially on an Add, I 
don’t know a way to fill the client ID field in such a way that Django will 
take it as valid.

My thoughts on possible solutions:
1. Override the get query set method to only pass in the single record - will 
that work?
2. I’ve been trying to show the field in a  tag and override the View’s post 
method by aftering is_valid(), but it always returns invalid expression errors.

What am I missing?

Thanks much again -
Jim

On Aug 23, 2018, at 3:48 PM, Vijay Khemlani 
mailto:vkhem...@gmail.com>> wrote:

Maybe your update form has a field related to a model with thousands of 
records? like rendering a select combo box with thousands of choices?

On Thu, Aug 23, 2018 at 5:30 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
I have a Django app in development that performed perfectly with about 20 
client records. Each client could have one or more support records and there 
are 40 potential support records, connected to the client with a ForeignKey 
(client_id). Then, I loading ab

Re: Django performance issue that's troubling

2018-08-24 Thread Jim Illback

Matthew, thanks - that was my original code, but even hidden, it has to fill in 
all possible values - 37K - so it takes multiple seconds.

Vijay, I’ll try your suggestion.

Thanks much,
Jim

On Aug 24, 2018, at 11:01 AM, Matthew Pava 
mailto:matthew.p...@iss.com>> wrote:

You can keep the field, but change its widget to a HiddenInput.
https://stackoverflow.com/questions/6862250/change-a-django-form-field-to-a-hidden-field



From: django-users@googlegroups.com<mailto:django-users@googlegroups.com> 
[mailto:django-users@googlegroups.com] On Behalf Of Vijay Khemlani
Sent: Friday, August 24, 2018 12:59 PM
To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>
Subject: Re: Django performance issue that's troubling

Please reply to the user group

"But, in order to have the update work, it needs the ForeignKey to the parent 
in the form"

As far as I know, it doesn't need to (if the entry already exists in the 
database and you are just editing it), you just need to set up the fields that 
you want to be editable by the user


On Fri, Aug 24, 2018 at 2:06 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
Vijay, think of this is a parent - child relationship. I have 37K parents. Now 
I want to update a child of one of the parents. But, in order to have the 
update work, it needs the ForeignKey to the parent in the form. However, to add 
that key, Django must then allow it to update that field and thus has to fill 
in all 37K entries. Is there no way to tell Django that the FK relationship can 
not be changed? If I leave it off of the form, the update fails because it is a 
required field. If I add it to the form, Django has to allow all potential 
update values to be shown. What’s the way out of this mess?

Thanks,
Jim



Begin forwarded message:

From: Vijay Khemlani mailto:vkhem...@gmail.com>>
Subject: Re: Django performance issue that's troubling
Date: August 24, 2018 at 5:07:07 AM PDT
To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>
Reply-To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>

To tell you the truth I couldn't understand some parts of your message, but if 
your AddictionForm is a ModelForm then you should be able to limit its fields 
to just the ones you want your users to be able to update.

https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/

Regards!



On Thu, Aug 23, 2018 at 8:19 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
Vijay, you are exactly correct. Thank you much!

Now, knowing the problem, I still don’t have a great solution. Here’s how it is 
supposed to work.

Someone chooses a client from the list or a search. All the possible support 
records are shown, whether they exist or not, on a screen to hyperlink to one 
of them. If the record exists, the URL includes the PK and transfers to the 
UpdateView. This is similar for an AddView if the record does not exist. 
However, in the Update or Add, I don’t want the primary key changed. So, I hid 
the field (and thus all the records were read). However, now I can’t limit that 
field to only one value. If I format the entire SELECT tag with only one 
OPTION, Django gives an invalid expression for the ID or Name because it is 
based on my query set I passed in that fills out the values (ID and name). If I 
use the form’s field value, all the fields are read. Especially on an Add, I 
don’t know a way to fill the client ID field in such a way that Django will 
take it as valid.

My thoughts on possible solutions:
1. Override the get query set method to only pass in the single record - will 
that work?
2. I’ve been trying to show the field in a  tag and override the View’s post 
method by aftering is_valid(), but it always returns invalid expression errors.

What am I missing?

Thanks much again -
Jim



On Aug 23, 2018, at 3:48 PM, Vijay Khemlani 
mailto:vkhem...@gmail.com>> wrote:

Maybe your update form has a field related to a model with thousands of 
records? like rendering a select combo box with thousands of choices?

On Thu, Aug 23, 2018 at 5:30 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
I have a Django app in development that performed perfectly with about 20 
client records. Each client could have one or more support records and there 
are 40 potential support records, connected to the client with a ForeignKey 
(client_id). Then, I loading about 37K client records. Search queries perform 
almost instantaneously. A paginated list of support records with 40 pages and 
10 clients per page comes back in half a second. Great performance.

However, when I click on any record in the list to go to update a specific 
support record for that specific client (it uses the PK in the URL so is a 
direct call to the UpdateView routine), I get about 10 second response times 
and these kind of statistics from Django-debug-toolbar:
SQL: 5 queries, 20ms (so clearly not a DB related pe

RE: Django performance issue that's troubling

2018-08-24 Thread Matthew Pava

You can keep the field, but change its widget to a HiddenInput.
https://stackoverflow.com/questions/6862250/change-a-django-form-field-to-a-hidden-field

From: django-users@googlegroups.com [mailto:django-users@googlegroups.com] On 
Behalf Of Vijay Khemlani
Sent: Friday, August 24, 2018 12:59 PM
To: django-users@googlegroups.com
Subject: Re: Django performance issue that's troubling

Please reply to the user group

"But, in order to have the update work, it needs the ForeignKey to the parent 
in the form"

As far as I know, it doesn't need to (if the entry already exists in the 
database and you are just editing it), you just need to set up the fields that 
you want to be editable by the user

On Fri, Aug 24, 2018 at 2:06 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
Vijay, think of this is a parent - child relationship. I have 37K parents. Now 
I want to update a child of one of the parents. But, in order to have the 
update work, it needs the ForeignKey to the parent in the form. However, to add 
that key, Django must then allow it to update that field and thus has to fill 
in all 37K entries. Is there no way to tell Django that the FK relationship can 
not be changed? If I leave it off of the form, the update fails because it is a 
required field. If I add it to the form, Django has to allow all potential 
update values to be shown. What’s the way out of this mess?

Thanks,
Jim

Begin forwarded message:

From: Vijay Khemlani mailto:vkhem...@gmail.com>>
Subject: Re: Django performance issue that's troubling
Date: August 24, 2018 at 5:07:07 AM PDT
To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>
Reply-To: django-users@googlegroups.com<mailto:django-users@googlegroups.com>

To tell you the truth I couldn't understand some parts of your message, but if 
your AddictionForm is a ModelForm then you should be able to limit its fields 
to just the ones you want your users to be able to update.

https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/

Regards!

On Thu, Aug 23, 2018 at 8:19 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
Vijay, you are exactly correct. Thank you much!

Now, knowing the problem, I still don’t have a great solution. Here’s how it is 
supposed to work.

Someone chooses a client from the list or a search. All the possible support 
records are shown, whether they exist or not, on a screen to hyperlink to one 
of them. If the record exists, the URL includes the PK and transfers to the 
UpdateView. This is similar for an AddView if the record does not exist. 
However, in the Update or Add, I don’t want the primary key changed. So, I hid 
the field (and thus all the records were read). However, now I can’t limit that 
field to only one value. If I format the entire SELECT tag with only one 
OPTION, Django gives an invalid expression for the ID or Name because it is 
based on my query set I passed in that fills out the values (ID and name). If I 
use the form’s field value, all the fields are read. Especially on an Add, I 
don’t know a way to fill the client ID field in such a way that Django will 
take it as valid.

My thoughts on possible solutions:
1. Override the get query set method to only pass in the single record - will 
that work?
2. I’ve been trying to show the field in a  tag and override the View’s post 
method by aftering is_valid(), but it always returns invalid expression errors.

What am I missing?

Thanks much again -
Jim

On Aug 23, 2018, at 3:48 PM, Vijay Khemlani 
mailto:vkhem...@gmail.com>> wrote:

Maybe your update form has a field related to a model with thousands of 
records? like rendering a select combo box with thousands of choices?

On Thu, Aug 23, 2018 at 5:30 PM Jim Illback 
mailto:subaru...@hotmail.com>> wrote:
I have a Django app in development that performed perfectly with about 20 
client records. Each client could have one or more support records and there 
are 40 potential support records, connected to the client with a ForeignKey 
(client_id). Then, I loading about 37K client records. Search queries perform 
almost instantaneously. A paginated list of support records with 40 pages and 
10 clients per page comes back in half a second. Great performance.

However, when I click on any record in the list to go to update a specific 
support record for that specific client (it uses the PK in the URL so is a 
direct call to the UpdateView routine), I get about 10 second response times 
and these kind of statistics from Django-debug-toolbar:
SQL: 5 queries, 20ms (so clearly not a DB related performance issue)
TIME:  Resource usage:
User CPU time 8827 msec
System CPU time 137 msec
Elapsed time 9003 msec
Context switches 1 voluntary, 9639 involuntary
Browser timing:
domLoading 9026 (+52130) msec

CACHE: 0 calls in 0.00ms (so clearly not a cache issue either)

Here are some details: Django 2.1, python 3.6.3
Hardware: iMac with 32GB, i7 quad core
Installed a

Re: Django performance issue that's troubling

2018-08-24 Thread Vijay Khemlani

Please reply to the user group

"But, in order to have the update work, it needs the ForeignKey to the
parent in the form"

As far as I know, it doesn't need to (if the entry already exists in the
database and you are just editing it), you just need to set up the fields
that you want to be editable by the user


On Fri, Aug 24, 2018 at 2:06 PM Jim Illback  wrote:

> Vijay, think of this is a parent - child relationship. I have 37K parents.
> Now I want to update a child of one of the parents. But, in order to have
> the update work, it needs the ForeignKey to the parent in the form.
> However, to add that key, Django must then allow it to update that field
> and thus has to fill in all 37K entries. Is there no way to tell Django
> that the FK relationship can not be changed? If I leave it off of the form,
> the update fails because it is a required field. If I add it to the form,
> Django has to allow all potential update values to be shown. What’s the way
> out of this mess?
>
> Thanks,
> Jim
>
>
> Begin forwarded message:
>
> *From: *Vijay Khemlani 
> *Subject: **Re: Django performance issue that's troubling*
> *Date: *August 24, 2018 at 5:07:07 AM PDT
> *To: *django-users@googlegroups.com
> *Reply-To: *django-users@googlegroups.com
>
> To tell you the truth I couldn't understand some parts of your message,
> but if your AddictionForm is a ModelForm then you should be able to limit
> its fields to just the ones you want your users to be able to update.
>
> https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/
>
> Regards!
>
>
>
> On Thu, Aug 23, 2018 at 8:19 PM Jim Illback  wrote:
>
>> Vijay, you are exactly correct. Thank you much!
>>
>> Now, knowing the problem, I still don’t have a great solution. Here’s how
>> it is supposed to work.
>>
>> Someone chooses a client from the list or a search. All the possible
>> support records are shown, whether they exist or not, on a screen to
>> hyperlink to one of them. If the record exists, the URL includes the PK and
>> transfers to the UpdateView. This is similar for an AddView if the record
>> does not exist. However, in the Update or Add, I don’t want the primary key
>> changed. So, I hid the field (and thus all the records were read). However,
>> now I can’t limit that field to only one value. If I format the entire
>> SELECT tag with only one OPTION, Django gives an invalid expression for the
>> ID or Name because it is based on my query set I passed in that fills out
>> the values (ID and name). If I use the form’s field value, all the fields
>> are read. Especially on an Add, I don’t know a way to fill the client ID
>> field in such a way that Django will take it as valid.
>>
>> My thoughts on possible solutions:
>> 1. Override the get query set method to only pass in the single record -
>> will that work?
>> 2. I’ve been trying to show the field in a  tag and override the
>> View’s post method by aftering is_valid(), but it always returns invalid
>> expression errors.
>>
>> What am I missing?
>>
>> Thanks much again -
>> Jim
>>
>>
>> On Aug 23, 2018, at 3:48 PM, Vijay Khemlani  wrote:
>>
>> Maybe your update form has a field related to a model with thousands of
>> records? like rendering a select combo box with thousands of choices?
>>
>> On Thu, Aug 23, 2018 at 5:30 PM Jim Illback 
>> wrote:
>>
>>> I have a Django app in development that performed perfectly with about
>>> 20 client records. Each client could have one or more support records and
>>> there are 40 potential support records, connected to the client with a
>>> ForeignKey (client_id). Then, I loading about 37K client records. Search
>>> queries perform almost instantaneously. A paginated list of support records
>>> with 40 pages and 10 clients per page comes back in half a second. Great
>>> performance.
>>>
>>> However, when I click on any record in the list to go to update a
>>> specific support record for that specific client (it uses the PK in the URL
>>> so is a direct call to the UpdateView routine), I get about 10 second
>>> response times and these kind of statistics from Django-debug-toolbar:
>>> *SQL*: 5 queries, 20ms (so clearly not a DB related performance issue)
>>> *TIME*:  Resource usage:
>>> User CPU time 8827 msec
>>> System CPU time 137 msec
>>> Elapsed time 9003 msec
>>> Context switches 1 voluntary, 9639 involuntary
>>> Browser timing:
>>> domLoading 9026 (+52130) msec
>>>
>>> *CACHE*: 0 calls in 0.00ms (so clearly not

Re: Django performance issue that's troubling

2018-08-24 Thread Vijay Khemlani

To tell you the truth I couldn't understand some parts of your message, but
if your AddictionForm is a ModelForm then you should be able to limit its
fields to just the ones you want your users to be able to update.

https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/

Regards!



On Thu, Aug 23, 2018 at 8:19 PM Jim Illback  wrote:

> Vijay, you are exactly correct. Thank you much!
>
> Now, knowing the problem, I still don’t have a great solution. Here’s how
> it is supposed to work.
>
> Someone chooses a client from the list or a search. All the possible
> support records are shown, whether they exist or not, on a screen to
> hyperlink to one of them. If the record exists, the URL includes the PK and
> transfers to the UpdateView. This is similar for an AddView if the record
> does not exist. However, in the Update or Add, I don’t want the primary key
> changed. So, I hid the field (and thus all the records were read). However,
> now I can’t limit that field to only one value. If I format the entire
> SELECT tag with only one OPTION, Django gives an invalid expression for the
> ID or Name because it is based on my query set I passed in that fills out
> the values (ID and name). If I use the form’s field value, all the fields
> are read. Especially on an Add, I don’t know a way to fill the client ID
> field in such a way that Django will take it as valid.
>
> My thoughts on possible solutions:
> 1. Override the get query set method to only pass in the single record -
> will that work?
> 2. I’ve been trying to show the field in a  tag and override the View’s
> post method by aftering is_valid(), but it always returns invalid
> expression errors.
>
> What am I missing?
>
> Thanks much again -
> Jim
>
>
> On Aug 23, 2018, at 3:48 PM, Vijay Khemlani  wrote:
>
> Maybe your update form has a field related to a model with thousands of
> records? like rendering a select combo box with thousands of choices?
>
> On Thu, Aug 23, 2018 at 5:30 PM Jim Illback  wrote:
>
>> I have a Django app in development that performed perfectly with about 20
>> client records. Each client could have one or more support records and
>> there are 40 potential support records, connected to the client with a
>> ForeignKey (client_id). Then, I loading about 37K client records. Search
>> queries perform almost instantaneously. A paginated list of support records
>> with 40 pages and 10 clients per page comes back in half a second. Great
>> performance.
>>
>> However, when I click on any record in the list to go to update a
>> specific support record for that specific client (it uses the PK in the URL
>> so is a direct call to the UpdateView routine), I get about 10 second
>> response times and these kind of statistics from Django-debug-toolbar:
>> *SQL*: 5 queries, 20ms (so clearly not a DB related performance issue)
>> *TIME*:  Resource usage:
>> User CPU time 8827 msec
>> System CPU time 137 msec
>> Elapsed time 9003 msec
>> Context switches 1 voluntary, 9639 involuntary
>> Browser timing:
>> domLoading 9026 (+52130) msec
>>
>> *CACHE*: 0 calls in 0.00ms (so clearly not a cache issue either)
>>
>> Here are some details: Django 2.1, python 3.6.3
>> Hardware: iMac with 32GB, i7 quad core
>> Installed apps: Bootstrap3 v9.1.0, crispy forms v1.7.0, easyPDF v0.1.1,
>> registration-redux v2.2, Pillow v4.3.0
>>
>> Here’s one of the specific UpdateView routines being executed - there are
>> only 5 fields in the AddictionForm plus the prime key (no image or file
>> attachments).
>>
>> class AddictionUpdate(PermissionRequiredMixin, UpdateView):
>> model = Addiction
>> form_class = AddictionForm
>> template_name = 'addiction_update.html'
>> permission_required = 'clientapp.change_addiction'
>>
>> The slow response appears to be on UpdateView pages only, even though all
>> updates are called with the prime key (ID) in the URL. Can anyone give me
>> guidance where to look for improving the performance? Why all the
>> involuntary context switches? Why the large domLoading time?
>>
>> Thanks in advance -
>> Jim
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/76F251D2-A429-4700-9F69-053673BDB91F%40hotmail.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to

Re: Django performance issue that's troubling

2018-08-23 Thread Jim Illback

Vijay, you are exactly correct. Thank you much!

Now, knowing the problem, I still don’t have a great solution. Here’s how it is
supposed to work.

Someone chooses a client from the list or a search. All the possible support
records are shown, whether they exist or not, on a screen to hyperlink to one
of them. If the record exists, the URL includes the PK and transfers to the
UpdateView. This is similar for an AddView if the record does not exist.
However, in the Update or Add, I don’t want the primary key changed. So, I hid
the field (and thus all the records were read). However, now I can’t limit that
field to only one value. If I format the entire SELECT tag with only one
OPTION, Django gives an invalid expression for the ID or Name because it is
based on my query set I passed in that fills out the values (ID and name). If I
use the form’s field value, all the fields are read. Especially on an Add, I
don’t know a way to fill the client ID field in such a way that Django will
take it as valid.

My thoughts on possible solutions:
1. Override the get query set method to only pass in the single record - will
that work?
2. I’ve been trying to show the field in a tag and override the View’s post
method by aftering is_valid(), but it always returns invalid expression errors.

What am I missing?

Thanks much again -
Jim

On Aug 23, 2018, at 3:48 PM, Vijay Khemlani
mailto:vkhem...@gmail.com>> wrote:

Maybe your update form has a field related to a model with thousands of
records? like rendering a select combo box with thousands of choices?

On Thu, Aug 23, 2018 at 5:30 PM Jim Illback
mailto:subaru...@hotmail.com>> wrote:
I have a Django app in development that performed perfectly with about 20
client records. Each client could have one or more support records and there
are 40 potential support records, connected to the client with a ForeignKey
(client_id). Then, I loading about 37K client records. Search queries perform
almost instantaneously. A paginated list of support records with 40 pages and
10 clients per page comes back in half a second. Great performance.

However, when I click on any record in the list to go to update a specific
support record for that specific client (it uses the PK in the URL so is a
direct call to the UpdateView routine), I get about 10 second response times
and these kind of statistics from Django-debug-toolbar:
SQL: 5 queries, 20ms (so clearly not a DB related performance issue)
TIME: Resource usage:
User CPU time 8827 msec
System CPU time 137 msec
Elapsed time 9003 msec
Context switches 1 voluntary, 9639 involuntary
Browser timing:
domLoading 9026 (+52130) msec

CACHE: 0 calls in 0.00ms (so clearly not a cache issue either)

Here are some details: Django 2.1, python 3.6.3
Hardware: iMac with 32GB, i7 quad core
Installed apps: Bootstrap3 v9.1.0, crispy forms v1.7.0, easyPDF v0.1.1,
registration-redux v2.2, Pillow v4.3.0

Here’s one of the specific UpdateView routines being executed - there are only
5 fields in the AddictionForm plus the prime key (no image or file attachments).

class AddictionUpdate(PermissionRequiredMixin, UpdateView):
model = Addiction
form_class = AddictionForm
template_name = 'addiction_update.html'
permission_required = 'clientapp.change_addiction'

The slow response appears to be on UpdateView pages only, even though all
updates are called with the prime key (ID) in the URL. Can anyone give me
guidance where to look for improving the performance? Why all the involuntary
context switches? Why the large domLoading time?

Thanks in advance -
Jim

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
django-users+unsubscr...@googlegroups.com.
To post to this group, send email to
django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/76F251D2-A429-4700-9F69-053673BDB91F%40hotmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django performance issue that's troubling

2018-08-23 Thread Vijay Khemlani

Maybe your update form has a field related to a model with thousands of
records? like rendering a select combo box with thousands of choices?

On Thu, Aug 23, 2018 at 5:30 PM Jim Illback  wrote:

> I have a Django app in development that performed perfectly with about 20
> client records. Each client could have one or more support records and
> there are 40 potential support records, connected to the client with a
> ForeignKey (client_id). Then, I loading about 37K client records. Search
> queries perform almost instantaneously. A paginated list of support records
> with 40 pages and 10 clients per page comes back in half a second. Great
> performance.
>
> However, when I click on any record in the list to go to update a specific
> support record for that specific client (it uses the PK in the URL so is a
> direct call to the UpdateView routine), I get about 10 second response
> times and these kind of statistics from Django-debug-toolbar:
> *SQL*: 5 queries, 20ms (so clearly not a DB related performance issue)
> *TIME*:  Resource usage:
> User CPU time 8827 msec
> System CPU time 137 msec
> Elapsed time 9003 msec
> Context switches 1 voluntary, 9639 involuntary
> Browser timing:
> domLoading 9026 (+52130) msec
>
> *CACHE*: 0 calls in 0.00ms (so clearly not a cache issue either)
>
> Here are some details: Django 2.1, python 3.6.3
> Hardware: iMac with 32GB, i7 quad core
> Installed apps: Bootstrap3 v9.1.0, crispy forms v1.7.0, easyPDF v0.1.1,
> registration-redux v2.2, Pillow v4.3.0
>
> Here’s one of the specific UpdateView routines being executed - there are
> only 5 fields in the AddictionForm plus the prime key (no image or file
> attachments).
>
> class AddictionUpdate(PermissionRequiredMixin, UpdateView):
> model = Addiction
> form_class = AddictionForm
> template_name = 'addiction_update.html'
> permission_required = 'clientapp.change_addiction'
>
> The slow response appears to be on UpdateView pages only, even though all
> updates are called with the prime key (ID) in the URL. Can anyone give me
> guidance where to look for improving the performance? Why all the
> involuntary context switches? Why the large domLoading time?
>
> Thanks in advance -
> Jim
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/76F251D2-A429-4700-9F69-053673BDB91F%40hotmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CALn3ei2korG5V3dDcCUQuKiAhG6tzAz%3DJZssNpGJpkAOyKAXpQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Django performance issue that's troubling

2018-08-23 Thread Jim Illback

I have a Django app in development that performed perfectly with about 20
client records. Each client could have one or more support records and there
are 40 potential support records, connected to the client with a ForeignKey
(client_id). Then, I loading about 37K client records. Search queries perform
almost instantaneously. A paginated list of support records with 40 pages and
10 clients per page comes back in half a second. Great performance.

CACHE: 0 calls in 0.00ms (so clearly not a cache issue either)

Here are some details: Django 2.1, python 3.6.3
Hardware: iMac with 32GB, i7 quad core
Installed apps: Bootstrap3 v9.1.0, crispy forms v1.7.0, easyPDF v0.1.1,
registration-redux v2.2, Pillow v4.3.0

Here’s one of the specific UpdateView routines being executed - there are only
5 fields in the AddictionForm plus the prime key (no image or file attachments).

class AddictionUpdate(PermissionRequiredMixin, UpdateView):
model = Addiction
form_class = AddictionForm
template_name = 'addiction_update.html'
permission_required = 'clientapp.change_addiction'

Thanks in advance -
Jim

--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/76F251D2-A429-4700-9F69-053673BDB91F%40hotmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django performance issue that's troubling

Re: Django performance issue that's troubling

Re: Django performance issue that's troubling

RE: Django performance issue that's troubling

Re: Django performance issue that's troubling

RE: Django performance issue that's troubling

Re: Django performance issue that's troubling

Re: Django performance issue that's troubling

Re: Django performance issue that's troubling

Re: Django performance issue that's troubling

Django performance issue that's troubling

11 matches

Site Navigation

Mail list logo

Footer information