Re: Database connection retry

2017-03-08 Thread Aymeric Augustin 
Hello,

> On 8 Mar 2017, at 21:23, Chris Foresman  wrote:
> 
> I'll chime in to say I've had a similar problem related to the shell and I 
> couldn't sort out how to address it.

In such situations, AFAIK, the following works:

from django.db import connection
connection.close()

Then Django will reopen the connection for the next database query.

HTH,

-- 
Aymeric.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/E5F0989A-97C8-4E7E-8C69-FA08E60F76B6%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Database connection retry

2017-03-08 Thread Adam Johnson 
Chris, whilst I'm sure you could work something out, it probably wouldn't
generally work as database connections contain a lot of state, such whether
or not we're in a transaction and variables.

On 8 March 2017 at 20:23, Chris Foresman  wrote:

> I'll chime in to say I've had a similar problem related to the shell and I
> couldn't sort out how to address it.
>
> Our database servers will drop connections that last longer than 10
> minutes. So basically can never do a task I might otherwise use the shell
> for that would take longer than 10 minutes of typing things in the shell.
> The workaround I eventually arrived at was copying all the data I pulled
> from previous runs into a doc in my text editor, and restart the shell
> every time the database connection dropped. Eventually I was able to just
> copy and paste enough stuff from that doc to get everything done in the 10
> minute limit. Is there a way (e.g. the DatabaseWrapper mentioned above) to
> get the shell to reconnect without stopping the shell and restarting from
> scratch every time?
>
> On Tuesday, March 7, 2017 at 7:30:27 AM UTC-6, James Pic wrote:
>>
>> It seems like we have 2 kind of issues:
>>
>> - code broke runserver,
>> - network broke runserver.
>>
>> In the first case, runserver waits for a code reload event which is
>> perfect ;)
>> In the second case, runserver also waits for a code reload event, which
>> is not very intuitive after fixing a network error.
>>
>> So if we want to handle both case, we indeed need to detect if an error
>> is caused by code or networking, which is defined by CACHES, DATABASES and
>> CHANNEL_LAYERS.
>>
>> Perhaps we could add a special attribute to the exception, so
>> DatabaseWrapper.get_new_connection()'s call of:
>>
>> connection = Database.connect(**conn_params)
>>
>> Would become something like:
>>
>> try:
>> connection = Database.connect(**conn_params)
>> except Exception as e:
>> e.network_error = True
>> raise
>>
>> Another way would be to inspect exc info or have a pre-defined list of
>> exceptions that are to be considered as network error, which involves
>> referencing to exceptions potentially defined by other packages such as
>> redis.
>>
>> While that may seem a lot for runserver, I've restrained myself from
>> talking about what this could look like in production so far in the
>> discussion, but I feel like even production deployment could somehow
>> benefit from this at some point, so that might be worth the effort after
>> all.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/cc47bf21-9bbd-4728-9f2f-
> 968405d128ad%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM0kFw%3Dr5ZvS-h7548dGSX1XgGEYweZOqS2XrRn%3DpdrD6Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Database connection retry

2017-03-08 Thread Chris Foresman 
I'll chime in to say I've had a similar problem related to the shell and I 
couldn't sort out how to address it.

Our database servers will drop connections that last longer than 10 
minutes. So basically can never do a task I might otherwise use the shell 
for that would take longer than 10 minutes of typing things in the shell. 
The workaround I eventually arrived at was copying all the data I pulled 
from previous runs into a doc in my text editor, and restart the shell 
every time the database connection dropped. Eventually I was able to just 
copy and paste enough stuff from that doc to get everything done in the 10 
minute limit. Is there a way (e.g. the DatabaseWrapper mentioned above) to 
get the shell to reconnect without stopping the shell and restarting from 
scratch every time?

On Tuesday, March 7, 2017 at 7:30:27 AM UTC-6, James Pic wrote:
>
> It seems like we have 2 kind of issues:
>
> - code broke runserver,
> - network broke runserver.
>
> In the first case, runserver waits for a code reload event which is 
> perfect ;)
> In the second case, runserver also waits for a code reload event, which is 
> not very intuitive after fixing a network error.
>
> So if we want to handle both case, we indeed need to detect if an error is 
> caused by code or networking, which is defined by CACHES, DATABASES and 
> CHANNEL_LAYERS.
>
> Perhaps we could add a special attribute to the exception, so 
> DatabaseWrapper.get_new_connection()'s call of:
>
> connection = Database.connect(**conn_params) 
>
> Would become something like:
>
> try:
> connection = Database.connect(**conn_params)
> except Exception as e:
> e.network_error = True
> raise
>
> Another way would be to inspect exc info or have a pre-defined list of 
> exceptions that are to be considered as network error, which involves 
> referencing to exceptions potentially defined by other packages such as 
> redis.
>
> While that may seem a lot for runserver, I've restrained myself from 
> talking about what this could look like in production so far in the 
> discussion, but I feel like even production deployment could somehow 
> benefit from this at some point, so that might be worth the effort after 
> all.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/cc47bf21-9bbd-4728-9f2f-968405d128ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Feature idea: forms signals

2017-03-08 Thread Melvyn Sopacua 
On Wednesday 08 March 2017 18:18:26 Melvyn Sopacua wrote:
> On Monday 06 March 2017 10:10:41 David Seddon wrote:
> > Hi all,
> > 
> > One thing I've always found challenging with Django is to adjust the
> > functionality of forms from other apps.  An example might be to add
> > an extra field to a login form provided by a third party module.
> > 
> > What I end up doing is often overriding the urlconf, view and form
> > class. Or, worse, monkey patching the form class.
> > 
> > A much nicer way to do this would be to add a few well chosen
> > signals
> > to forms.
> > 
> > Potential ones could be:
> > 
> > - post_init
> 
> Putting in a +1 and use case for pre_init: Inject dynamically
> generated form.prefix.
> 
> Right now I have to inject PrefixedFormView in every view using it and
> carefully watch my mro of already complex views.
> 
> Use case is having several modals containing forms on a single page
> ("Edit this", "Add related", "New this"). They need to be prefixed to
> deal with naming conflicts since id_{fieldname} will not be page
> unique anymore.
> 
> With the signal, I can do away with the view mixin and the WrappedForm
> is solely responsible for it.
> I have no preference for the technology used (signal, hook, injection,
> new buzzword), as long as the net result is that the object creating
> the prefix can handle the prefix for all views.
> 
> Is there a ticket yet?
> 

Apologies for the formatting.
https://gist.github.com/melvyn-sopacua/dbf3c8f71023d6fc261131cb0a851f58


-- 
Melvyn Sopacua

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1667937.4n8HccigN7%40devstation.
For more options, visit https://groups.google.com/d/optout.

Re: Feature idea: forms signals

2017-03-08 Thread Melvyn Sopacua 
On Monday 06 March 2017 10:10:41 David Seddon wrote:
> Hi all,
> 
> One thing I've always found challenging with Django is to adjust the
> functionality of forms from other apps.  An example might be to add an
> extra field to a login form provided by a third party module.
> 
> What I end up doing is often overriding the urlconf, view and form
> class. Or, worse, monkey patching the form class.
> 
> A much nicer way to do this would be to add a few well chosen signals
> to forms.
> 
> Potential ones could be:
> 
> - post_init

Putting in a +1 and use case for pre_init: Inject dynamically generated 
form.prefix.

Right now I have to inject PrefixedFormView in every view using it and 
carefully watch 
my mro of already complex views.

Use case is having several modals containing forms on a single page ("Edit 
this", "Add 
related", "New this"). They need to be prefixed to deal with naming conflicts 
since 
id_{fieldname} will not be page unique anymore.

With the signal, I can do away with the view mixin and the WrappedForm is 
solely 
responsible for it.
I have no preference for the technology used (signal, hook, injection, new 
buzzword), 
as long as the net result is that the object creating the prefix can handle the 
prefix for 
all views.

Is there a ticket yet?

Use case code:

*class PrefixedFormView(*generic.FormView*):def post(*self, request, 
***args, 
kwargs*):*self.prefix *= *request.GET.get*('prefix'*, *None)
return 
/super/()*.post*(*request, ***args, kwargs*)*

*class WrappedForm(/object/):def __init__(*self, form_class*: *BaseForm, 
action_url*: *str, is_multipart*: *bool *= False*, 
model_instance*: *Model 
*= None*, initial*: *dict *= None*, prefix*: *str *= 'auto'*, 
kwargs*):*
*

**if *prefix *== 'auto':*self.prefix *= *uuid4*()*.hex  
  *else:
*self.prefix *= *prefixself.form *= None
*self._set_query_prefix*()

def _set_query_prefix(*self*):*final_url *= 
*urlparse*(*self.action_url*)
*query *= *QueryDict*(*query_string*=*final_url.query, mutable*=True)
*query[*'prefix'*] *= *self.prefixparts *= (*final_url.scheme, 
final_url.netloc, 
final_url.path, final_url.params, query.urlencode*()*, 
final_url.fragment*)
*self.action_url *= *urlunparse*(*parts*)


*

-- 
Melvyn Sopacua

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1561133.ak0JmRi9Co%40devstation.
For more options, visit https://groups.google.com/d/optout.

Re: In-memory queryset

2017-03-08 Thread Neal Todd 
Hi Paul,

Depending on your use cases django-modelcluster might be of some use:

https://github.com/wagtail/django-modelcluster

It's primarly for dealing with a bunch of models that have foreign key 
relationships, but before they're written to a database (i.e before they 
have primary keys that allow the foreign keys to work). At some point they 
are typically committed to a database but they also could be ephemeral or 
serialised and handed off to something else.

It only supports a subset of the QuerySet API (pragmatically that which 
allows it to serve the purpose it was created for).

Cheers, Neal   

On Tuesday, March 7, 2017 at 2:57:57 PM UTC, pa...@dabapps.com wrote:
>
>
> It would be really convenient for me if there was an implementation of the 
> QuerySet API which instead of using a database as its data source, used 
> in-memory model instance that had not been persisted to the database at all.
>
> I looked around and found nothing like this.
>
> Is this because nothing like this exists? Is it because it's a terrible 
> idea for a reason that's not obvious to me? Would it not be awesome for the 
> API for interacting with collections of in-memory instances and with a 
> database-backed collection to be the same?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e5749e59-167f-48fd-9563-7cb69b5e9f6b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.