Re: Threads and db connection handling question

[Cristiano Coelho]

Yes, mainly some things that needs to be done async and you don't want to 
keep up the request for those such as email sending or logging. So this 
work is just offloaded to a thread pool and the request returns instantly. 
Also when you are using auto scaling services from amazon and since each 
thread pool is started per server process, it means your pools can also 
automatically scale, you don't really want to get into all the issues of 
configuring celery and messages queues and such just for this.

El miércoles, 1 de junio de 2016, 23:19:29 (UTC-3), Stephen Butler escribió:
>
> Is there a good reason to do this with your own custom thread pool 
> management inside Django and (I'm assuming) WSGI? Celery is a well 
> understood solution to the problem of background tasks and has a really 
> nice API.
>
> On Wed, Jun 1, 2016 at 5:34 PM, Cristiano Coelho  > wrote:
>
>> Let me start saying sorry if this actually belongs to django-users rather 
>> than developers.
>>
>> I'm curious about (and if someone can point me to the code) how exactly 
>> django handles database connections (in particular when persistent 
>> connections are used). As I can tell from the docs, they are kept by thread 
>> and opened and closed (or returned to what ever pool it uses if persistent 
>> connections are used) per request.
>>
>> Now what happens when you use either a new thread, or something like 
>> python's thread pool (either through the new python 3 api or old python 2 
>> multiprocessing.pool.ThreadPool class)? It seems like connections are 
>> correctly opened, and commited if any data modification query is executed, 
>> but it also seems like they are never returned/closed, which is not bad in 
>> the case of a thread pool, as you know that thread will want to have that 
>> connection up for as long as it lives.
>> What happens exactly if the thread / thread pool dies? On postgres at 
>> least (with django 1.9.5) it seems like the connection is returned/closed 
>> in cases the whole app server is restarted, but might be left open if the 
>> thread unexpectly dies.
>>
>> With postgres I have been experiencing some issues with connections 
>> leaking, my app uses some thread pools that are basically started with 
>> django. Now I can't really find the source of the leak, as the connections 
>> are correctly closed if I restart the machine (I'm using amazon cloud 
>> services), and it seems that they are also correctly closed on app updates 
>> which basically means restarting Apache, but in some very specific cases, 
>> those thread pools ends up leaking the connection.
>>
>> Does django have any code to listen to thread exit and gracefully close 
>> the connection held by it? Also, is there any chance that a connection may 
>> leak if the server is restarted before a request is finished? As it seems 
>> like django returns the connection only after a request is over on those 
>> cases.
>>
>> Also, if the connection gets corrupted/closed by the server, does django 
>> re try to open it or is that thread's connection dead for ever and 
>> basically the thread unusable?
>>
>> There's really not a lot of documentation on what happens when you use 
>> django's ORM on threads that are not part of the current request, hopefully 
>> I can get pointed to some code or docs about this.
>>
>> There's a good response here 
>> http://stackoverflow.com/questions/1303654/threaded-django-task-doesnt-automatically-handle-transactions-or-db-connections
>>  
>> about some issues with threads and django connections but it seems old.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/9c330b4b-ebd7-4abb-b03d-dffa21d245af%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3a77c0c3-4a57-470e-8ee2-8805ec4d7e35%40googlegroups.com.
For more options, visit 

Re: Threads and db connection handling question

[Cristiano Coelho]

That's pretty close but on a much more difficult level, since it is about 
multi processing? Things starts to get very odd with multi processing and 
django, compared to threads that you can easily launch a new thread or pool 
without any special work, it is just the database connections handling that 
has some kind possible connection leak that I'm trying to figure out why. 
Also the stack overflow post I linked has some good explanation on the 
django db connection handling but is quite outdated and things probably 
have changed a bit and it isn't clear if there's any kind of connection 
handling on thread exit or you need to manually close any connection.

El miércoles, 1 de junio de 2016, 22:41:42 (UTC-3), Tim Graham escribió:
>
> Here's a ticket requesting the documentation you seek (as far as I 
> understand):
> https://code.djangoproject.com/ticket/20562
>
> On Wednesday, June 1, 2016 at 6:34:05 PM UTC-4, Cristiano Coelho wrote:
>>
>> Let me start saying sorry if this actually belongs to django-users rather 
>> than developers.
>>
>> I'm curious about (and if someone can point me to the code) how exactly 
>> django handles database connections (in particular when persistent 
>> connections are used). As I can tell from the docs, they are kept by thread 
>> and opened and closed (or returned to what ever pool it uses if persistent 
>> connections are used) per request.
>>
>> Now what happens when you use either a new thread, or something like 
>> python's thread pool (either through the new python 3 api or old python 2 
>> multiprocessing.pool.ThreadPool class)? It seems like connections are 
>> correctly opened, and commited if any data modification query is executed, 
>> but it also seems like they are never returned/closed, which is not bad in 
>> the case of a thread pool, as you know that thread will want to have that 
>> connection up for as long as it lives.
>> What happens exactly if the thread / thread pool dies? On postgres at 
>> least (with django 1.9.5) it seems like the connection is returned/closed 
>> in cases the whole app server is restarted, but might be left open if the 
>> thread unexpectly dies.
>>
>> With postgres I have been experiencing some issues with connections 
>> leaking, my app uses some thread pools that are basically started with 
>> django. Now I can't really find the source of the leak, as the connections 
>> are correctly closed if I restart the machine (I'm using amazon cloud 
>> services), and it seems that they are also correctly closed on app updates 
>> which basically means restarting Apache, but in some very specific cases, 
>> those thread pools ends up leaking the connection.
>>
>> Does django have any code to listen to thread exit and gracefully close 
>> the connection held by it? Also, is there any chance that a connection may 
>> leak if the server is restarted before a request is finished? As it seems 
>> like django returns the connection only after a request is over on those 
>> cases.
>>
>> Also, if the connection gets corrupted/closed by the server, does django 
>> re try to open it or is that thread's connection dead for ever and 
>> basically the thread unusable?
>>
>> There's really not a lot of documentation on what happens when you use 
>> django's ORM on threads that are not part of the current request, hopefully 
>> I can get pointed to some code or docs about this.
>>
>> There's a good response here 
>> http://stackoverflow.com/questions/1303654/threaded-django-task-doesnt-automatically-handle-transactions-or-db-connections
>>  
>> about some issues with threads and django connections but it seems old.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/97c0b2ae-d208-4a4e-9c5e-16ec62d328e4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Threads and db connection handling question

[Tim Graham]

Here's a ticket requesting the documentation you seek (as far as I 
understand):
https://code.djangoproject.com/ticket/20562

On Wednesday, June 1, 2016 at 6:34:05 PM UTC-4, Cristiano Coelho wrote:
>
> Let me start saying sorry if this actually belongs to django-users rather 
> than developers.
>
> I'm curious about (and if someone can point me to the code) how exactly 
> django handles database connections (in particular when persistent 
> connections are used). As I can tell from the docs, they are kept by thread 
> and opened and closed (or returned to what ever pool it uses if persistent 
> connections are used) per request.
>
> Now what happens when you use either a new thread, or something like 
> python's thread pool (either through the new python 3 api or old python 2 
> multiprocessing.pool.ThreadPool class)? It seems like connections are 
> correctly opened, and commited if any data modification query is executed, 
> but it also seems like they are never returned/closed, which is not bad in 
> the case of a thread pool, as you know that thread will want to have that 
> connection up for as long as it lives.
> What happens exactly if the thread / thread pool dies? On postgres at 
> least (with django 1.9.5) it seems like the connection is returned/closed 
> in cases the whole app server is restarted, but might be left open if the 
> thread unexpectly dies.
>
> With postgres I have been experiencing some issues with connections 
> leaking, my app uses some thread pools that are basically started with 
> django. Now I can't really find the source of the leak, as the connections 
> are correctly closed if I restart the machine (I'm using amazon cloud 
> services), and it seems that they are also correctly closed on app updates 
> which basically means restarting Apache, but in some very specific cases, 
> those thread pools ends up leaking the connection.
>
> Does django have any code to listen to thread exit and gracefully close 
> the connection held by it? Also, is there any chance that a connection may 
> leak if the server is restarted before a request is finished? As it seems 
> like django returns the connection only after a request is over on those 
> cases.
>
> Also, if the connection gets corrupted/closed by the server, does django 
> re try to open it or is that thread's connection dead for ever and 
> basically the thread unusable?
>
> There's really not a lot of documentation on what happens when you use 
> django's ORM on threads that are not part of the current request, hopefully 
> I can get pointed to some code or docs about this.
>
> There's a good response here 
> http://stackoverflow.com/questions/1303654/threaded-django-task-doesnt-automatically-handle-transactions-or-db-connections
>  
> about some issues with threads and django connections but it seems old.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/08b5e953-717d-4f6a-83ef-4df3e53de847%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Threads and db connection handling question

[Cristiano Coelho]

Let me start saying sorry if this actually belongs to django-users rather 
than developers.

I'm curious about (and if someone can point me to the code) how exactly 
django handles database connections (in particular when persistent 
connections are used). As I can tell from the docs, they are kept by thread 
and opened and closed (or returned to what ever pool it uses if persistent 
connections are used) per request.

Now what happens when you use either a new thread, or something like 
python's thread pool (either through the new python 3 api or old python 2 
multiprocessing.pool.ThreadPool class)? It seems like connections are 
correctly opened, and commited if any data modification query is executed, 
but it also seems like they are never returned/closed, which is not bad in 
the case of a thread pool, as you know that thread will want to have that 
connection up for as long as it lives.
What happens exactly if the thread / thread pool dies? On postgres at least 
(with django 1.9.5) it seems like the connection is returned/closed in 
cases the whole app server is restarted, but might be left open if the 
thread unexpectly dies.

With postgres I have been experiencing some issues with connections 
leaking, my app uses some thread pools that are basically started with 
django. Now I can't really find the source of the leak, as the connections 
are correctly closed if I restart the machine (I'm using amazon cloud 
services), and it seems that they are also correctly closed on app updates 
which basically means restarting Apache, but in some very specific cases, 
those thread pools ends up leaking the connection.

Does django have any code to listen to thread exit and gracefully close the 
connection held by it? Also, is there any chance that a connection may leak 
if the server is restarted before a request is finished? As it seems like 
django returns the connection only after a request is over on those cases.

Also, if the connection gets corrupted/closed by the server, does django re 
try to open it or is that thread's connection dead for ever and basically 
the thread unusable?

There's really not a lot of documentation on what happens when you use 
django's ORM on threads that are not part of the current request, hopefully 
I can get pointed to some code or docs about this.

There's a good response here 
http://stackoverflow.com/questions/1303654/threaded-django-task-doesnt-automatically-handle-transactions-or-db-connections
 
about some issues with threads and django connections but it seems old.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9c330b4b-ebd7-4abb-b03d-dffa21d245af%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Trac is broken

[Tim Graham]

If this problem has been reported before, I don't remember it. Nothing 
about our Trac installation has changed recently that I'm aware of. In case 
it helps, I just updated to Trac 1.0.11.

It's unclear if the problem would be our problem and or an issue with Trac 
but for what it's worth, our bug tracker for code.djangoproject.com is:
https://github.com/django/code.djangoproject.com/issues

I guess we need steps to reproduce the empty cookie.

On Wednesday, June 1, 2016 at 7:06:14 AM UTC-4, Krzysztof Jurewicz wrote:
>
> Trying to browse any ticket on code.djangoproject.com yields 403 
> Forbidden, with a message “TICKET_VIEW privileges are required to perform 
> this operation. You don't have the required permissions.”. An attempt to 
> login via https://code.djangoproject.com/login results in 502. It has 
> been broken that way for some days (at least). Is there a plan to fix this? 
> Browsing tickets via web.archive.org is not very handy… 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0118f7b6-3980-422f-8970-fdbf7fef63cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Trac is broken

[Krzysztof Jurewicz]

It seems that the issue is somehow related to empty trac_session cookie which 
somehow has got set. Here is the curl request which can be used to obtain the 
403 response:

curl -I -H 'Cookie: trac_session=' https://code.djangoproject.com/ticket/2659

Clearing the cookies solved the problem, so the issue is probably low-priority.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/87pos1fd5y.fsf%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Trac is broken

[Krzysztof Jurewicz]

Trying to browse any ticket on code.djangoproject.com yields 403 Forbidden, 
with a message “TICKET_VIEW privileges are required to perform this operation. 
You don't have the required permissions.”. An attempt to login via 
https://code.djangoproject.com/login results in 502. It has been broken that 
way for some days (at least). Is there a plan to fix this? Browsing tickets via 
web.archive.org is not very handy…

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/87r3chff24.fsf%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Possibility to define default placeholder configuration

[Michał Lechman]

Exactly. Sorry but I haven't found it before.

--
Michal

W dniu środa, 1 czerwca 2016 11:31:02 UTC+2 użytkownik Jonas Obrist napisał:
>
> Hi Michal,
>
> Do you mean something like what is being discussed at 
> https://github.com/divio/django-cms/pull/5006?
>
> Jonas
>
> On Wednesday, June 1, 2016 at 6:25:48 PM UTC+9, Michał Lechman wrote:
>>
>> Hi all,
>>
>> I would like to know if is there any reason why currently there is no 
>> option to define default placeholder configuration (i.e. such as "default" 
>> item in CMS_LANGUAGES setting). In my opinion it would be really helpful 
>> for example to change the default language_fallback behaviour which is not 
>> alwways appropiate. Is it a design decision?
>>
>> --
>> Michal
>>
>

-- 
Message URL: 
https://groups.google.com/d/msg/django-cms-developers/topic-id/message-id
Unsubscribe: send a message to 
django-cms-developers+unsubscr...@googlegroups.com
--- 
You received this message because you are subscribed to the Google Groups 
"django CMS developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-cms-developers+unsubscr...@googlegroups.com.
To view this discussion on the web, visit 
https://groups.google.com/d/msgid/django-cms-developers/fdc045ca-4914-435b-a88f-45f0236707cf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Possibility to define default placeholder configuration

[Jonas Obrist]

Hi Michal,

Do you mean something like what is being discussed 
at https://github.com/divio/django-cms/pull/5006?

Jonas

On Wednesday, June 1, 2016 at 6:25:48 PM UTC+9, Michał Lechman wrote:
>
> Hi all,
>
> I would like to know if is there any reason why currently there is no 
> option to define default placeholder configuration (i.e. such as "default" 
> item in CMS_LANGUAGES setting). In my opinion it would be really helpful 
> for example to change the default language_fallback behaviour which is not 
> alwways appropiate. Is it a design decision?
>
> --
> Michal
>

-- 
Message URL: 
https://groups.google.com/d/msg/django-cms-developers/topic-id/message-id
Unsubscribe: send a message to 
django-cms-developers+unsubscr...@googlegroups.com
--- 
You received this message because you are subscribed to the Google Groups 
"django CMS developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-cms-developers+unsubscr...@googlegroups.com.
To view this discussion on the web, visit 
https://groups.google.com/d/msgid/django-cms-developers/c5563221-4959-4ed5-ae7d-201c2a7cbd6d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Possibility to define default placeholder configuration

[Michał Lechman]

Hi all,

I would like to know if is there any reason why currently there is no 
option to define default placeholder configuration (i.e. such as "default" 
item in CMS_LANGUAGES setting). In my opinion it would be really helpful 
for example to change the default language_fallback behaviour which is not 
alwways appropiate. Is it a design decision?

--
Michal

-- 
Message URL: 
https://groups.google.com/d/msg/django-cms-developers/topic-id/message-id
Unsubscribe: send a message to 
django-cms-developers+unsubscr...@googlegroups.com
--- 
You received this message because you are subscribed to the Google Groups 
"django CMS developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-cms-developers+unsubscr...@googlegroups.com.
To view this discussion on the web, visit 
https://groups.google.com/d/msgid/django-cms-developers/bb7128f9-b2bb-412b-bf40-fa90c5a58e3e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.