Re: Improving aggregate support (#14030)

[Josh Smeaton]

Hi Michael,

Thanks for taking a look, much appreciated. I haven't created a PR yet 
because I've rebased on top of Ansii's Lookups refactor which hasn't hit 
Master yet (and is changing slightly based on feedback of his PR). To 
answer your questions:

Why were the aggregate specific fields, properties, etc. added to 
> ExpressionNode, instead of being on django.db.models.aggregates?
>

Originally they were, but then Sum('field') + Sum('anotherfield') didn't 
work as that expression produces an ExpressionNode with AggregateNode 
children. That ExpressionNode (+ (Sum, Sum)) has to behave like an 
Aggregate. The previous SQLEvaluator had specific methods that related to 
F() expressions and DateModifierNodes for exactly this reason, which have 
now been pushed down to the ExpressionNode.

- Why does an ExpressionNode containing any Aggregates need to pollute the 
> entire tree with that knowledge?
>

As mentioned above. The ExpressionNode becomes an Aggregate to the outside 
world and should be treated like one. The leaves of the Expression tree 
form the actual aggregate functions, but the root of the tree forms the 
entire Aggregate expression. Using the example above:

.annotate(label=Sum('field') + Sum('anotherfield')) ->

ExpressionNode(+):
: Sum(field)
: Sum(anotherfield)

The framework needs to know several things about an Aggregate, like its 
output type and the type of columns that are wrapped. When there are 
multiple aggregate functions in a single expression, the root needs to 
check its children and make a decision on what to present.

Instead of using is_ordinal and is_computed (confusing name, why not 
> is_float), you could eliminate those and define output_type for each 
> Aggregate that does something special.
>

These are remnants of existing aggregates, and can probably be replaced by 
setting the field_type property directly. I will look into doing that.

What does is_summary mean?
>

It means that .aggregate() has been invoked which evaluates immediately. 
Internally the aggregate is prepared differently though as it has to check 
if it refers to an aggregate annotation. I don't like how the query sets 
the is_summary property directly though, and might grow .prepare() to also 
accept an is_summary parameter.

Cheers,

- Josh

On Sunday, 12 January 2014 10:35:58 UTC+11, Michael Manfre wrote:
>
> With minor tweaks to django-mssql's SQLCompiler, I am able to pass the 
> aggregation and aggregation_regress tests. If you create the pull request, 
> comments can be attached to specific lines of code.
>
> I haven't had time to do a full review of your changes, but my initial 
> questions and comments are:
>
> - Why were the aggregate specific fields, properties, etc. added to 
> ExpressionNode, instead of being on django.db.models.aggregates?
> - Why does an ExpressionNode containing any Aggregates need to pollute the 
> entire tree with that knowledge?
> - Instead of using is_ordinal and is_computed (confusing name, why not 
> is_float), you could eliminate those and define output_type for each 
> Aggregate that does something special.
> - What does is_summary mean?
>
> Regards,
> Michael Manfre
>
>
> On Sat, Jan 11, 2014 at 3:06 AM, Josh Smeaton 
>  > wrote:
>
>> A milestone has been reached. The diff is here: 
>> https://github.com/jarshwah/django/compare/akaariai:lookups_3...aggregates-14030?expand=1
>>
>>- Existing annotations and aggregations working as they were
>>- Complex annotations and complex aggregations now work
>>- Complex annotations can now be referenced by an aggregation
>>- Cleaned up the logic relating to aggregating over an annotation
>>
>> See the tests for the new functionality here: 
>> https://github.com/jarshwah/django/blob/aggregates-14030/tests/aggregation/tests.py#L662
>>
>> Things I still need to tackle:
>>
>>- GIS
>>- Custom backend implementation of aggregates and annotations 
>>- Annotations that aren't aggregates. I think this should be 
>>extremely easy, but I might not be aware of some underlying difficulties.
>>
>> Things that should be done in the future, as I don't think there will be 
>> time for this just now:
>>
>>- Change order_by (and other functions) to accept Expressions.
>>
>> I would think that, as it stands, this code is ready for review with the 
>> above caveats. If there are any backend authors reading (mssql, 
>> django-nonrel), I would encourage you to look at the implementation here, 
>> and let me know if there's anything glaringly obvious (to you) that is 
>> missing or broken.
>>
>> Regards,
>>
>> - Josh
>>
>> On Thursday, 9 January 2014 00:33:08 UTC+11, Josh Smeaton wrote:
>>>
>>> I'll preface by saying it's late for me, and I'm about to sign off, so 
>>> any mistakes below I will correct tomorrow after some sleep.
>>>  
>>>
 To write a custom aggregate one needed two classes, one for user facing 
 API, one for implementing that API for SQL queries.
>>>

Re: Website integrated mobile app for disaster

[Russell Keith-Magee]

Hi Akanksha,

We haven't started the GSoC process internally yet, so we haven't put
together a collection of sample projects for this year; but if you want to
see what we recommended last year, it's on our Wiki:

https://code.djangoproject.com/wiki/SummerOfCode2013

Out of those projects, the Validation project was completed last year, so
it's obviously off the candidate list.

I can think of one additional project that has been mentioned in passing on
the developers list: formalising the Meta class.

This project would involve refactoring and documenting the interface to the
Model._meta object. This interface is central to the operation of Django
itself, but *technically* isn't backwards compatible. There are some
aspects of this interface that aren't especially well designed from a
public facing perspective, and there are some features that involve subtle
duplication. The refactoring project would take a look at all these
interfaces, and come up with a clean API that we can formally publish. As a
proof of concept that the API definition is complete, the project might
involve developing an alternate implementation for some other data source
-- for example, a simple wrapper for a NoSQL store -- that can then be used
as a duck-type compatible replacement for a "normal" Django model.

The key part about any project proposal for GSoC - you need to convince us
that you know what you're planning to do, and convince us that your able to
complete it. A key part of this is a project plan with fine grained
estimates. A proposal that says "12 weeks - refactor meta" won't convince
us; a proposal that gives plausible blocks of work filling 3 days-1 week
chunks indicates that probably understand the problem, and will be able to
deliver. However, in order to develop this sort of project plan, you're
going to need to do some research about the problem - and that research
*isn't* something we're going to help with. The fact that you are able to
do the research at all is part of our application process to establish that
you're capable of completing the project.

Yours,
Russ Magee %-)


On Sat, Jan 11, 2014 at 9:43 PM, Akanksha Shrivastava <
akanksha.shrivasta...@gmail.com> wrote:

> Thankyou so much for your guidance sir! i m looking forward to present my
> ideas for improvement of django. However it would be great if you all could
> please suggest me with some ideas regarding SoC. Waiting for your reply
> eagerly and to work for the very same too.
>
>
> On Friday, January 10, 2014 11:37:44 PM UTC+5:30, Akanksha Shrivastava
> wrote:
>>
>>   Hello everyone!
>>
>> I am currently working on this project of mine and was looking forward
>> for your feedback. I am looking forward to present this idea as my idea for
>> Google Summer Of Code, and I am really looking forward to join this
>> community and contribute whatever I can for this community.
>>
>> They say change is inevitable. The nature has its own path, it hears no
>> one and follows no one. It is unbiased to all and has been the most furious
>> of all the changes. The very sudden changes in nature lead to what we call
>> calamity or disaster. Our priority at such a time is to pray for safety of
>> the lives and work as hard as we can for the very same. However, we always
>> wonder what if we had even a little idea about what was about to happen.
>> And if not the magical time machine then at least time to reach up to this
>> information should have been early. For the most uncertain and
>> unpredictable events of life like these, I would like to present to you my
>> idea.
>>
>>
>>  Project-A website integrated mobile app which traces a person all the
>> time continuously and as soon as any calamity occurs, alerts the website
>> within minutes, thus, allowing us to take action in a quick and swift
>> manner. This may be highly beneficial in saving lives by expanding our time
>> interval to respond and react. The location can be integrated by google
>> maps or google earth, while the mobile app can be integrated with the
>> website containing the database of people, whose current status of nature
>> of environment will be in constant scanning under this project.
>>
>>
>>  Area of Application-This idea can be implemented at places where there
>> is a very high risk of calamity like Japan (people residing there) and at
>> places where travelers may be affected, like a train (people travelling)
>> accidents or any such unfortunate event.
>>
>> This app is highly useful at places where calamity strikes and we search
>> out for either newspapers or media for information regarding the very same.
>> However, we still remain uncertain regarding our family or friend as we
>> only get the idea of the situation of the place and of the incident that
>> has taken place. Not only we will be in touch with our near and dear once,
>> but we will also be informed about the calamity and may reach out to as
>> many people and as soon as possible. This app is highly beneficial not only
>> 

Re: Improving aggregate support (#14030)

[Michael Manfre]

With minor tweaks to django-mssql's SQLCompiler, I am able to pass the
aggregation and aggregation_regress tests. If you create the pull request,
comments can be attached to specific lines of code.

I haven't had time to do a full review of your changes, but my initial
questions and comments are:

- Why were the aggregate specific fields, properties, etc. added to
ExpressionNode, instead of being on django.db.models.aggregates?
- Why does an ExpressionNode containing any Aggregates need to pollute the
entire tree with that knowledge?
- Instead of using is_ordinal and is_computed (confusing name, why not
is_float), you could eliminate those and define output_type for each
Aggregate that does something special.
- What does is_summary mean?

Regards,
Michael Manfre


On Sat, Jan 11, 2014 at 3:06 AM, Josh Smeaton wrote:

> A milestone has been reached. The diff is here:
> https://github.com/jarshwah/django/compare/akaariai:lookups_3...aggregates-14030?expand=1
>
>- Existing annotations and aggregations working as they were
>- Complex annotations and complex aggregations now work
>- Complex annotations can now be referenced by an aggregation
>- Cleaned up the logic relating to aggregating over an annotation
>
> See the tests for the new functionality here:
> https://github.com/jarshwah/django/blob/aggregates-14030/tests/aggregation/tests.py#L662
>
> Things I still need to tackle:
>
>- GIS
>- Custom backend implementation of aggregates and annotations
>- Annotations that aren't aggregates. I think this should be extremely
>easy, but I might not be aware of some underlying difficulties.
>
> Things that should be done in the future, as I don't think there will be
> time for this just now:
>
>- Change order_by (and other functions) to accept Expressions.
>
> I would think that, as it stands, this code is ready for review with the
> above caveats. If there are any backend authors reading (mssql,
> django-nonrel), I would encourage you to look at the implementation here,
> and let me know if there's anything glaringly obvious (to you) that is
> missing or broken.
>
> Regards,
>
> - Josh
>
> On Thursday, 9 January 2014 00:33:08 UTC+11, Josh Smeaton wrote:
>>
>> I'll preface by saying it's late for me, and I'm about to sign off, so
>> any mistakes below I will correct tomorrow after some sleep.
>>
>>
>>> To write a custom aggregate one needed two classes, one for user facing
>>> API, one for implementing that API for SQL queries.
>>
>>
>> I've taken note of this already, and that is the reason I've kept the
>> sql.Aggregate class where it is, though it's not used internally anymore.
>> There are two options that I can see right now:
>>
>> - The non-compatible way involves moving the sql_function attribute to
>> the Aggregate from the SQLAggregate, and everything should work fine
>> - The backward-compatible way might involve re-adding the "add_to_query"
>> function, and using the SQLAggregate to monkeypatch the Aggregate by
>> copying over the sql_function attribute. I would have to experiment and
>> test this though. I assume this would be the more appropriate option. We
>> can include a DeprecationWarning to phase it out.
>>
>>
>>> Another case you should check is how GIS works *without* rewriting
>>> anything in django.contrib.gis
>>
>>
>> Good idea, that's how I'll approach the aggregate stuff. Not sure that I
>> can leave it completely alone due to Expressions though, but we'll see (and
>> hope).
>>
>> I'll try to check your work in detail as soon as possible. Unfortunately
>>> I am very busy at the moment, so I can't guarantee anything.
>>
>>
>> Totally understandable, I appreciate your comments regardless. It's just
>> unfortunate there aren't more people deeply familiar with the ORM - it's
>> bloody complicated :P
>>
>> I'm now to the point where all tests in aggregate, aggregate_regress,
>> expressions, expressions_regress, and lookup pass. I've got a small problem
>> with .count() regressions failing, but am working through that now.
>> Tomorrow I should be able to start playing with new functionality and
>> writing new tests. I'll also take a look into running tests on GIS and
>> fixing any errors that rear their heads there.
>>
>> Cheers,
>>
>> - Josh
>>
>>
>> On Thursday, 9 January 2014 00:05:23 UTC+11, Anssi Kääriäinen wrote:
>>>
>>> On Friday, January 3, 2014 6:34:13 PM UTC+2, Josh Smeaton wrote:

 I now have all tests passing on Postgres and SQLite (See 
 here).
 I still haven't touched GIS, but I believe it should be as simple as
 changing the query to directly call prepare on the expression, and removing
 references to SQLEvaluator.

 The big problem is that if we drop the SQLEvaluator and similarly the
> models/aggregates.py <-> models/sql/aggregates.py ways of writing
> Expressions and Aggregates, then there 

Re: Migrations and swappable models/AUTH_USER_MODEL

[Andrew Godwin]

Good points. Regarding 1), I can't specially control reconstruction of
fields from migrations and raise errors only in that case - it's just
calling the normal __init__. Since I'm going to have to write out a call to
settings.AUTH_USER_MODEL anyway - the whole point of swappable=True is so
the migration serialiser knows to do that - having swappable be True or
False makes no difference (the only behaviour it affects is the migration
serialiser, for now, and that's never run on re-serialised models).
Nevertheless, I'll make sure it's forced to True just in case.

On 2), that's an issue I hadn't thought of. There's special syntax for
depending on the first migration of an app which I could inject, but it'd
have to be chosen at migration creation time, so not terribly useful. The
only concrete way to make sure this works is to have a dependency that's
defined based on the setting, e.g.

class Migration(migrations.Migration):
dependencies = [
("comments", "0001_initial"),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]

That function would just be lambda value: (value.split(".", 1)[0],
"__first__") - we already have the __first__ special migration name which
resolves to the start of any app (it's used for depending on apps with no
migrations at the moment, in case they gain them in future)

Andrew


On Sat, Jan 11, 2014 at 9:43 PM, Shai Berger  wrote:

> On Saturday 11 January 2014 11:45:55 Andrew Godwin wrote:
> > OK, I like that approach Russ - an inverted 1b. It'll be a bit harder to
> > explain in the docs, but it won't catch anyone out unawares, which is the
> > key thing.
> >
> I like it too.
>
> And foot-in-mouth re referencing swappables notwithstanding, there's two
> points I want to raise:
>
> 1) This approach is good for the models files; I think a more explicit
> approach
> should be taken for the migration files, where we don't currently have to
> worry
> about backwards compatibility. I'd like to see swappable=None in the model
> transformed to swappable=True in the migration, and swappable=None in the
> migration treated as an error.
>
> In particular, I don't like Marc's idea about migrations continuing to work
> when a target model, which used to be non-swappable, is being swapped out.
> The
> app's "main" code won't continue to work in this scenario, for reasons Russ
> has noted; I don't see why migrations should not be treated as part of the
> app's code.
>
> 2) Swappability of foreign-key targets in migrations may induce dynamic
> migration dependencies (if my migration creates a foreign key to the user
> model, and that model happens to be in a migrated app, my migration now
> depends on some -- not even necessarily the initial -- migration in that
> app;
> we'll only know at migrate time, not at makemigration time). I'd like to
> make
> this more explicit too -- e.g. add some placeholder, or a function call to
> calculate the dependency, in the generated list of dependencies. Although
> in
> principle it can be induced from the operations, I think it would make
> possible migration dependency issues much easier to troubleshoot.
>
> An alternative would to decree that swappable-replacement models cannot
> live
> in migrated apps; I think that is a very bad option.
>
> Shai.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/138953391.ZAfFGT0eZV%40deblack
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1upCnbp4%2B0fN0qsENAunKLWaUXZE-hSdW49icmfxBpeL%3DA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: python querry on firebug extention .

[Tim Graham]

Please don't crosspost to both django-developers and django-users. The 
topic of this list is the development of Django itself, django-users is for 
usage questions, although it's not clear to me how this question is related 
to Django.

On Saturday, January 11, 2014 1:18:00 PM UTC-5, JAI PRAKASH SINGH wrote:
>
> hello 
>
>i am working on selenium module of python, i know how to make 
> extension  of firebug with selenium, but i  want to know how to use 
> firebug  extension with request module / mechanize . i search a lot but 
> unable to find it , please help . 
>
> technique similar like :-
>
> from selenium import webdriver
> fp = webdriver.FirefoxProfile()
> fp.add_extension(extension='firebug-1.8.4.xpi')fp.set_preference("extensions.firebug.currentVersion",
>  "1.8.4") #Avoid startup screenbrowser = webdriver.Firefox(firefox_profile=fp)
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/47581c62-5871-4a05-83cf-5919f2f7a949%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Migrations and swappable models/AUTH_USER_MODEL

[Shai Berger]

On Thursday 09 January 2014 01:10:09 Raphaël Barrois wrote:
> On Wed, 8 Jan 2014 20:27:54 +0200
> 
> Shai Berger  wrote:
> > as people already use auth.get_user_model() -- so, you can
> > monkeypatch *that* when generating migrations, and let them just keep
> > doing what they do today.
> 
> Actually, people don't use auth.get_user_model() in their models
> declaration, they use ``settings.AUTH_USER_MODEL``, as described in the
> documentation:
> https://docs.djangoproject.com/en/1.6/topics/auth/customizing/#referencing-> 
> the-user-model
> 

Yes, I stand corrected.
> 
> Actually, you have two problems with a third-party app:
> 
> - It needs to have a way to specify "I want a FK to whatever the user has
> chosen for a AUTH_USER_MODEL", no matter the *type* of its PK field. =>
> This mustn't depend on what the third-party-app developer's actual
> AUTH_USER_MODEL setting was while generating the migration 

As I've said in response to another mail in this thread, a variation on 
Russell's suggestion, where the migration adds a ForeignKey with 
swappable=True, should allow this to be done.

> if the user changes the model, this should be helped by migrations, for
> instance by put a warning e.g "Looks like you're changing a swappable
> model, where should I put automatic related migrations ? (these migrations
> shouldn't go into the app with the FKey to AUTH_USER_MODEL, which may be
> out of the user's control — e.g third party apps).
> 
I don't think related migrations should be generated automatically. There are 
too many issues -- besides third-party apps, there are non-migrated apps. 
Editing an app's models under its feet is very bad practice.

> Another problem is apps that provide a swappable model whose default is
> AUTH_USER_MODEL but that may be customized through their own setting, for
> instance ``return getattr(settings, 'MY_TARGET_MODEL',
> settings.AUTH_USER_MODEL)``.

No, such apps should be using the same getattr call  in their migrations -- 
that is, this is one of the cases where editing migration files manually should 
be required; and it doesn't matter if the default  target model happens to be 
swappable. It is enough that a referenced model is decided by settings.

On a related note, this strengthens the case for explicit references to 
dynamic dependencies; a migration for such an app should, IMO, look something 
like:

from django.db import migrations, models
from django.conf import settings

my_target_model = getattr(settings, 'MY_TARGET_MODEL',
  settings.AUTH_USER_MODEL)

class Migration(migrations.Migration):

dependencies = [("migrations", "0001_initial"),
migrations.model_created(my_target_model)]

operations = [
migrations.DeleteModel("Tribble"),
migrations.AddField("MyModel", "target",
models.ForeignKey(my_target_model)),
]

On swappable models, the dynamic dependency could be inferred automatically 
from the operations; in this case, where the target model is not (or at least 
not necessarily) swappable, the placeholder reference is required.

Shai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3541296.khKWS2QS0i%40deblack.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Migrations and swappable models/AUTH_USER_MODEL

[Shai Berger]

On Saturday 11 January 2014 11:45:55 Andrew Godwin wrote:
> OK, I like that approach Russ - an inverted 1b. It'll be a bit harder to
> explain in the docs, but it won't catch anyone out unawares, which is the
> key thing.
> 
I like it too.

And foot-in-mouth re referencing swappables notwithstanding, there's two 
points I want to raise:

1) This approach is good for the models files; I think a more explicit approach 
should be taken for the migration files, where we don't currently have to worry 
about backwards compatibility. I'd like to see swappable=None in the model 
transformed to swappable=True in the migration, and swappable=None in the 
migration treated as an error.

In particular, I don't like Marc's idea about migrations continuing to work 
when a target model, which used to be non-swappable, is being swapped out. The 
app's "main" code won't continue to work in this scenario, for reasons Russ 
has noted; I don't see why migrations should not be treated as part of the 
app's code.

2) Swappability of foreign-key targets in migrations may induce dynamic 
migration dependencies (if my migration creates a foreign key to the user 
model, and that model happens to be in a migrated app, my migration now 
depends on some -- not even necessarily the initial -- migration in that app; 
we'll only know at migrate time, not at makemigration time). I'd like to make 
this more explicit too -- e.g. add some placeholder, or a function call to 
calculate the dependency, in the generated list of dependencies. Although in 
principle it can be induced from the operations, I think it would make 
possible migration dependency issues much easier to troubleshoot.

An alternative would to decree that swappable-replacement models cannot live 
in migrated apps; I think that is a very bad option.

Shai.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/138953391.ZAfFGT0eZV%40deblack.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Feature request: post/pre commit/rollback signals

[Aymeric Augustin]

On 11 janv. 2014, at 11:05, Andrey Antukh  wrote:

> In my opinion, "the default case are solved with atomic block" seems to be a 
> workaround instead of a solution.

As a short reminder, we’re talking about inconsistencies that arise when:

1) you do something in the database;
2) you do something related in another system — send an email, enqueue a task, 
cache a value;
3) the database transaction is rolled back, canceling the effects of 1) but not 
those of 2).

I may have been unclear when I closed the ticket. I’m not saying that atomic 
blocks solve the problem. I’m saying that the problem doesn’t happen in 
autocommit mode, which is the default in Django >= 1.6.

If you’re using transactions (i.e. ATOMIC_REQUESTS or atomic blocks), you may 
encounter the problem. But then I’m positive that transaction signals aren’t 
the solution. 

> I understand that signals in general are evil and register global callbacks 
> is not a very good solution, but I think, that a orm should give some generic 
> facility to attach code to execute when a transaction is successfully 
> committed or rolled back.

I’m not having this argument once more, so, there you go:
https://github.com/aaugustin/django-transaction-signals
https://pypi.python.org/pypi/django-transaction-signals-do-not-use

You may also be interested in:
https://github.com/davehughes/django-transaction-signals
https://pypi.python.org/pypi/django_transaction_signals

(The name conflict is unfortunate; I didn’t check before writing my package; at 
least that forced me to include a warning in the name.)

Jokes aside, transaction signals are just a (wrong) means to an (legitimate) 
end. I’ve documented the solutions I’m aware of:
https://github.com/aaugustin/django-transaction-signals#alternatives

Additional ideas welcome!

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/989FB6EA-12A2-4D54-9821-FB8473F3AF29%40polytechnique.org.
For more options, visit https://groups.google.com/groups/opt_out.

python querry on firebug extention .

[JAI PRAKASH SINGH]

hello 

   i am working on selenium module of python, i know how to make 
extension  of firebug with selenium, but i  want to know how to use 
firebug  extension with request module / mechanize . i search a lot but 
unable to find it , please help . 

technique similar like :-

from selenium import webdriver
fp = webdriver.FirefoxProfile()
fp.add_extension(extension='firebug-1.8.4.xpi')fp.set_preference("extensions.firebug.currentVersion",
 "1.8.4") #Avoid startup screenbrowser = webdriver.Firefox(firefox_profile=fp)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/623fb8c1-a2ad-4fcd-8b4b-09ee7bc5c01f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Website integrated mobile app for disaster

[Akanksha Shrivastava]

Thankyou so much for your guidance sir! i m looking forward to present my 
ideas for improvement of django. However it would be great if you all could 
please suggest me with some ideas regarding SoC. Waiting for your reply 
eagerly and to work for the very same too.

On Friday, January 10, 2014 11:37:44 PM UTC+5:30, Akanksha Shrivastava 
wrote:
>
>   Hello everyone!
>
> I am currently working on this project of mine and was looking forward for 
> your feedback. I am looking forward to present this idea as my idea for 
> Google Summer Of Code, and I am really looking forward to join this 
> community and contribute whatever I can for this community.
>
> They say change is inevitable. The nature has its own path, it hears no 
> one and follows no one. It is unbiased to all and has been the most furious 
> of all the changes. The very sudden changes in nature lead to what we call 
> calamity or disaster. Our priority at such a time is to pray for safety of 
> the lives and work as hard as we can for the very same. However, we always 
> wonder what if we had even a little idea about what was about to happen. 
> And if not the magical time machine then at least time to reach up to this 
> information should have been early. For the most uncertain and 
> unpredictable events of life like these, I would like to present to you my 
> idea.
>
>
>  Project-A website integrated mobile app which traces a person all the 
> time continuously and as soon as any calamity occurs, alerts the website 
> within minutes, thus, allowing us to take action in a quick and swift 
> manner. This may be highly beneficial in saving lives by expanding our time 
> interval to respond and react. The location can be integrated by google 
> maps or google earth, while the mobile app can be integrated with the 
> website containing the database of people, whose current status of nature 
> of environment will be in constant scanning under this project.
>
>
>  Area of Application-This idea can be implemented at places where there 
> is a very high risk of calamity like Japan (people residing there) and at 
> places where travelers may be affected, like a train (people travelling) 
> accidents or any such unfortunate event.
>
> This app is highly useful at places where calamity strikes and we search 
> out for either newspapers or media for information regarding the very same. 
> However, we still remain uncertain regarding our family or friend as we 
> only get the idea of the situation of the place and of the incident that 
> has taken place. Not only we will be in touch with our near and dear once, 
> but we will also be informed about the calamity and may reach out to as 
> many people and as soon as possible. This app is highly beneficial not only 
> for safety of our near and dear ones but also gives us a chance to prove 
> humanity. This is a simple app which does not require millions of buttons 
> selections, neither it involves to confuse people by weird and annoying 
> passwords, especially at the time of calamity who could afford to type 
> passwords and choose from list of buttons instead of saving his or her 
> life. This is a disaster management technique which will ensure people that 
> their families or any close person has the idea that he or she is in danger 
> or trouble, so one can respond to the very same as soon as one can think 
> of. One knows the value of human life today, and this will help us to save 
> plenty of such lives.
>
>
>  Past cases-This project could have been implemented in the most 
> unfortunate cases of tsunami and earthquakes which created a massive life 
> loss, whereas it could also have been used for train accidents and airplane 
> crashes. There are plenty of cases in the world which signify the power of 
> mother earth, and these are the only things which are not controlled by 
> human beings yet. This app gives us a chance to prove that we can at least 
> try our level best to keep track of such mishaps.
>
>  
> Success criteria-This project will ensure faster disaster recovery and 
> disaster management in every country.
>
>- 
>
>One looks up to a source of information regarding their close ones, we 
>provide them very same and in shorter span of time.
>- 
>
>We will be alerted of any mishap much before the usual timing.
>
>   Technology involved-In this project, we have made an app which gets and 
> returns the data from the database which is placed in the website. This app 
> is always active with the people who are about to be affected. We get 
> constant alerts regarding their current state and as soon as any disaster 
> occurs, the person just clicks a button in the phone, it is instantly 
> updated on our website and in the case of emergency, there is alert which 
> is sent to the people close to them (whose database is already collected 
> and stored in the website) also, we alert as many people a little far 
> around that area regarding the incident and 

Re: Migrations and swappable models/AUTH_USER_MODEL

[Andrew Godwin]

OK, I like that approach Russ - an inverted 1b. It'll be a bit harder to
explain in the docs, but it won't catch anyone out unawares, which is the
key thing.

Andrew


On Thu, Jan 9, 2014 at 1:28 AM, Russell Keith-Magee  wrote:

>
>
>
> On Thu, Jan 9, 2014 at 12:04 AM, Andrew Godwin wrote:
>
>> So, the last major bug left in migrations that I'm aware of (bar the
>> completion of the GIS backends) is dealing with swappable models - in
>> particular, of course, AUTH_USER_MODEL.
>>
>> As long as you're not using any third-party apps, then everything works
>> fine, but as soon as third-party apps with migrations come into the picture
>> there's an issue. These apps have to ship with migrations that reference
>> any project's current user model - and the way swapping is currently done
>> means that ForeignKeys end up with concrete references to the user model
>> _at the time the migration was made_.
>>
>> There are a couple of potential approaches here:
>>
>> 1) Introduce a new value that can be used as a "to" parameter on
>> ForeignKeys which resolves to a swapped model in its internal
>> string-to-model converter. I'm thinking something like
>> "__swapped__.AUTH_USER_MODEL" - I would use settings here, but I can
>> imagine people having an app called settings.
>>
>> The problem here is I'd then have to make ForeignKey deconstruct to this
>> special value if the model it was pointing to was the replacement model -
>> whether or not it's got settings.AUTH_USER_MODEL in the models.py file
>> (there's no way to tell if it has an actual string or a setting reference
>> in its models.py declaration). This means that some FKs might be too
>> eagerly converted into swapped references.
>>
>> 1b) Like 1), but rather than having the value automatically inserted,
>> require apps to actually use this new string rather than a direct reference
>> to settings.AUTH_USER_MODEL, meaning we can tell if it's swapped or not and
>> deconstruct it appropriately.
>>
>> 2) Change ForeignKey's constructor to automatically point itself to the
>> new swapped-in model if it's pointing to the model that was swapped out (so
>> ForeignKey("auth.User") automatically becomes
>> ForeignKey("myapp.CustomUser") internally.
>>
>>
>> Now, I prefer the cleanliness of 2), but I suspect there was a reason
>> this wasn't done when swappable models were first introduced; that said, I
>> haven't seen any coding patterns in the wild this would disrupt.
>>
>
> The implementation of swappable models is just exploiting the pre-existing
> late-binding feature of Django's model tools - a string reference to a
> model class is always valid, and will be resolved at runtime;
> settings.AUTH_USER_MODEL is just a string. Migrations notwithstanding,
>
> We didn't automagically convert auth.User into myapp.CustomUser because
> this acts as a validation aid. There's no guarantee that myapp.CustomUser
> adheres to the same user contract as auth.User. If a third party app
> references user.username, the app will break in production if you supply a
> user model that doesn't have a username field. A hard reference to a
> swapped model (either as auth.User *or* as "auth.User") can be identified
> as a validation issue on startup, flagging that the third party app isn't
> ready to support swappable user models. An app author can then signal that
> they're swappable-user ready by making the code change replacing the hard
> reference with a settings.AUTH_USER_MODEL reference.
>
> So - there's a reason not to do (2) IMHO.
>
> Functionally, the problem is that ForeignKey(myauth.CustomUser) and
> ForeignKey(settings.AUTH_USER_MODEL) are both legal foreign key references,
> and if AUTH_USER_MODEL=CustomUser, then they both point to the same model -
> but only the second is a "swappable" reference.
>
> In the internals, this resolves to the fact that while auth.User knows
> that it's been swapped out (and can identify this fact), but FKs to
> myauth.CustomUser don't know if this particular reference is a hard
> reference to a particular model, or a swapped in reference.
>
> As an approach, (1b) concerns me a little - we've talking about a feature
> in the wild that's only been out there for 2 releases, and required third
> party apps to adapt in response to that change - now we're asking them to
> adapt again to flag that the reference is swappable. It's a very small
> change, but it feels like code churn to me, with the potential to upset the
> wider community.
>
> My preference would be (1), but with a metadata clarification.
>
> We are in a position to make an educated guess that a FK reference was to
> a swappable model. If the FK definition uses a string, and that string
> matches the definition of one of the SWAPPABLE meta declarations, then we
> can say the FK is a reference to a swappable.
>
> This will make an incorrect guess in one specific case -- String-based
> explicit FK references to the replacement model. (i.e.,
> 

Re: Feature request: post/pre commit/rollback signals

[Andrey Antukh]

Hi!

In my opinion, "the default case are solved with atomic block" seems to be
a workaround instead of a solution. I understand that signals in general
are evil and register global callbacks is not a very good solution, but I
think, that a orm should give some generic facility to attach code to
execute when a transaction is successfully committed or rolled back.

For example, one possible use case can be this:

import threading

from django.dispatch import receiver
from django.core.signals import request_started

from .signals import post_commit

_local = threading.local()
_local.post_commit_tasks = []

@receiver(post_commit, dispatch_uid="_run_defered_tasks")
def _run_defered_tasks(sender, **kwargs):
while _local.post_commit_tasks:
task = _local.post_commit_tasks.pop(0)
task[0](*task[1], **task[2])

@receiver(request_started, dispatch_uid="_clean_tasks")
def _clean_tasks(sender, **kwargs):
_local.post_commit_tasks = []

def delay_until_transaction(func, *args, **kwargs):
_local.post_commit_tasks.append((func, args, kwargs))

This is one of much other examples that try to solve this (problem/use
case) out of django, and I think that django/django-orm should provide some
facility to defer execution some code after a transaction is commited or
rolled back.

In the last aaugustin comment, he suggest one approach without using global
signals, and it seems  very interesting approach. Can these  approach to be
generalized and included in django?

As Jesus said, it not seems a isolated problem, and would be awesome if
django provides some generic and good way to do this.

Greetings.
Andrey.




2014/1/10 Andreas Pelme 

>
> On 10 jan 2014, at 09:19, Jesús Espino  wrote:
>
> > Hi folks!
> >
> > The propose is add post and pre signals for commit and rollback actions
> (as sqlalchemy orm events). This allows attach some code when a transaction
> is committed or rolled back.
> >
> > I have some problem, such as send email only when a transaction is
> committed successfully, and it's can be done with simple monkey patching
> the db backend. But it not appear to be an isolated problem, and would be
> awesome if these signals are included in core.
> >
> > If a purpose is accepted, I can take care of making the issue +
> pull-request.
> >
> > Greetings.
> > Jesus.
>
>
> Hi,
>
> This is a tricky issue that has been discussed earlier, you will probably
> want to check out ticket #14051:
>
> https://code.djangoproject.com/ticket/14051
>
> —
> Andreas Pelme
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/948D5F1E-F86D-44A0-9EEA-F1E9A6A4A382%40pelme.se
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>



-- 
Andrey Antukh - Андрей Антух -  / 
http://www.niwi.be 
https://github.com/niwibe

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAKn%3DmOOq-dE8Nc9b604DnUkriRp2mhW90nx5nfc9PUuNyHn4dA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Improving aggregate support (#14030)

[Josh Smeaton]

A milestone has been reached. The diff is here: 
https://github.com/jarshwah/django/compare/akaariai:lookups_3...aggregates-14030?expand=1

   - Existing annotations and aggregations working as they were
   - Complex annotations and complex aggregations now work
   - Complex annotations can now be referenced by an aggregation
   - Cleaned up the logic relating to aggregating over an annotation
   
See the tests for the new functionality here: 
https://github.com/jarshwah/django/blob/aggregates-14030/tests/aggregation/tests.py#L662

Things I still need to tackle:

   - GIS
   - Custom backend implementation of aggregates and annotations
   - Annotations that aren't aggregates. I think this should be extremely 
   easy, but I might not be aware of some underlying difficulties.

Things that should be done in the future, as I don't think there will be 
time for this just now:

   - Change order_by (and other functions) to accept Expressions.

I would think that, as it stands, this code is ready for review with the 
above caveats. If there are any backend authors reading (mssql, 
django-nonrel), I would encourage you to look at the implementation here, 
and let me know if there's anything glaringly obvious (to you) that is 
missing or broken.

Regards,

- Josh

On Thursday, 9 January 2014 00:33:08 UTC+11, Josh Smeaton wrote:
>
> I'll preface by saying it's late for me, and I'm about to sign off, so any 
> mistakes below I will correct tomorrow after some sleep.
>  
>
>> To write a custom aggregate one needed two classes, one for user facing 
>> API, one for implementing that API for SQL queries.
>
>
> I've taken note of this already, and that is the reason I've kept the 
> sql.Aggregate class where it is, though it's not used internally anymore. 
> There are two options that I can see right now:
>
> - The non-compatible way involves moving the sql_function attribute to the 
> Aggregate from the SQLAggregate, and everything should work fine
> - The backward-compatible way might involve re-adding the "add_to_query" 
> function, and using the SQLAggregate to monkeypatch the Aggregate by 
> copying over the sql_function attribute. I would have to experiment and 
> test this though. I assume this would be the more appropriate option. We 
> can include a DeprecationWarning to phase it out.
>  
>
>> Another case you should check is how GIS works *without* rewriting 
>> anything in django.contrib.gis
>
>
> Good idea, that's how I'll approach the aggregate stuff. Not sure that I 
> can leave it completely alone due to Expressions though, but we'll see (and 
> hope). 
>
> I'll try to check your work in detail as soon as possible. Unfortunately I 
>> am very busy at the moment, so I can't guarantee anything.
>
>
> Totally understandable, I appreciate your comments regardless. It's just 
> unfortunate there aren't more people deeply familiar with the ORM - it's 
> bloody complicated :P
>
> I'm now to the point where all tests in aggregate, aggregate_regress, 
> expressions, expressions_regress, and lookup pass. I've got a small problem 
> with .count() regressions failing, but am working through that now. 
> Tomorrow I should be able to start playing with new functionality and 
> writing new tests. I'll also take a look into running tests on GIS and 
> fixing any errors that rear their heads there.
>
> Cheers,
>
> - Josh
>  
>
> On Thursday, 9 January 2014 00:05:23 UTC+11, Anssi Kääriäinen wrote:
>>
>> On Friday, January 3, 2014 6:34:13 PM UTC+2, Josh Smeaton wrote:
>>>
>>> I now have all tests passing on Postgres and SQLite (See 
>>> here).
>>>  
>>> I still haven't touched GIS, but I believe it should be as simple as 
>>> changing the query to directly call prepare on the expression, and removing 
>>> references to SQLEvaluator.
>>>
>>> The big problem is that if we drop the SQLEvaluator and similarly the 
 models/aggregates.py <-> models/sql/aggregates.py ways of writing 
 Expressions and Aggregates, then there will be a lot of broken 3rd party 
 aggregates and expressions. Even if the APIs are private I am not willing 
 to just drop the old way. So, some backwards compatibility is required.

>>>
>>> What level of backwards compatibility would be required? I like the idea 
>>> of using the @add_implementation decorator to extend Expressions and 
>>> Aggregates, but would that be enough? I've not used django-nonrel or any 
>>> other non-official backend, so I'm really not sure what would be required 
>>> or where a line should be drawn. The SQLEvaluator API (mostly) has moved 
>>> down to ExpressionNode, so the callers only change by calling .prepare on 
>>> the value rather than wrapping it in an Evaluator.
>>>
>>
>> The biggest problem are 3rd party Aggregates. To write a custom aggregate 
>> one needed two classes, one for user facing API, one for implementing that 
>> API for SQL queries.
>>
>> Taking first