Re: Should annotations allow model field constraints?

[Josh Smeaton]

I've marked the patch as ready for checkin. Instantiating fields without 
any arguments for decimal field is now supported by all backends. The 
format_number function has learnt to understand max_digits or 
decimal_places being None. The format_number function needs review though. 
When decimal_places is None, the decimal will be rounded to max_digits 
(using context.prec). I've decided to throw an error if the result is 
rounded though, so that the value must be within max_digits.

This patch does:

- fixes the regression
- ignores precision and decimal places for annotations in the base class
- documents that model fields for output_field take no args

This patch does not:

- allow using classes instead of instances
- take into consideration any arguments passed to model fields

I don't feel that passing classes and then having to instantiate them 
internally is a nice enough API for the complexity it brings, especially 
when fields can be created without arguments by the user. If we come across 
a field that does not allow construction without arguments, then we should 
fix that field. If, in the future, we do want to support some arguments 
then users will have a weird choice between passing an instance or a class.

As for support of model field arguments, I think that's a separate design 
decision and patch, and shouldn't hold up fixing the previous behaviour of 
aggregation over decimal fields.

Regards,

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c6cad148-27ea-4f41-8c50-1755d5a4e5dd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: Prevent data loss in the admin

[Steven Cummings]

FWIW, this was on my mind a few years ago and I created this [1] ticket. It
spidered out into one or two more, but the basic ideas were:

* optimistic locking could be implemented by enhancing saves with
conditions (i.e., save_if)
* a row-modified count could help that function also tell you if the save
condition passed or failed
* rows modified *may* be used now to update data via QuerySet.update, but
allegedly not all of the DB implementations provide the count in the same
way (e.g., matched vs. updated)
* the modified count discrepancy was worth spinning up another ticket to
discuss and investigate

I agree with Russell, I think the need for this is much broader than the
admin.

[1] https://code.djangoproject.com/ticket/16549

On Mon Dec 08 2014 at 7:10:28 PM Russell Keith-Magee <
russ...@keith-magee.com> wrote:

> Hi Rune,
>
> I agree with Tim. This is clearly a problem that exists, and I agree it
> would be good to fix it. However, I wouldn't want to see this as an "Admin
> only" fix.
>
> I'd rather see a generic hook that Admin then leverages as the "first
> customer". A similar approach was taken with object-level permissions - a
> generic framework for object-level permissions was added, and admin was
> modified to support that API as part of the proof that the API was
> sufficient.
>
> From an implementation perspective, I also have a mild negative reaction
> to the idea of using a separate table in the database for locks. Using a
> database row to marshal this behavior sounds like adding load to a database
> when all you need is a transient marker. A backend API that lets you use
> memcache, redis, or even a file-based backend might be called for. And
> that's assuming we take the "separate lock table" approach - I can think of
> other approaches that involve version indicators on the model itself. These
> are just initial impressions, however; I haven't given the problem a whole
> lot of thought, so these might not be viable ideas given a little more
> consideration.
>
> Yours,
> Russ Magee %-)
>
>
> On Mon, Dec 8, 2014 at 11:37 PM, Tim Graham  wrote:
>
>> Hi Rune,
>>
>> It's not clear to me that a fix that's coupled to the admin is the way to
>> go when this problem could exist for any model form, etc.
>>
>> One ticket you didn't mention that seems like it could be a good place to
>> start is https://code.djangoproject.com/ticket/16549
>>
>> Tim
>>
>> On Monday, December 8, 2014 8:04:30 AM UTC-5, Rune Kaagaard wrote:
>>>
>>> @Daniele
>>>
>>> I struggled a bit with how to display the error message, and your
>>> solution makes perfect sense.
>>>
>>> Best,
>>> Rune Kaagaard
>>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/f1a56260-89d7-4439-aed1-9407d799f62f%40googlegroups.com
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAJxq84_Kz_6ZCTzxEjaZm61hdfYcKnQNHw-xyPr%3DzX%3Dtc8GYvw%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAODHHFAJct5FCDD8gcrB0_L-yOwXuyMx6PzmeHzHacyHgVhC9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: Prevent data loss in the admin

[Russell Keith-Magee]

Hi Rune,

I agree with Tim. This is clearly a problem that exists, and I agree it
would be good to fix it. However, I wouldn't want to see this as an "Admin
only" fix.

I'd rather see a generic hook that Admin then leverages as the "first
customer". A similar approach was taken with object-level permissions - a
generic framework for object-level permissions was added, and admin was
modified to support that API as part of the proof that the API was
sufficient.

>From an implementation perspective, I also have a mild negative reaction to
the idea of using a separate table in the database for locks. Using a
database row to marshal this behavior sounds like adding load to a database
when all you need is a transient marker. A backend API that lets you use
memcache, redis, or even a file-based backend might be called for. And
that's assuming we take the "separate lock table" approach - I can think of
other approaches that involve version indicators on the model itself. These
are just initial impressions, however; I haven't given the problem a whole
lot of thought, so these might not be viable ideas given a little more
consideration.

Yours,
Russ Magee %-)


On Mon, Dec 8, 2014 at 11:37 PM, Tim Graham  wrote:

> Hi Rune,
>
> It's not clear to me that a fix that's coupled to the admin is the way to
> go when this problem could exist for any model form, etc.
>
> One ticket you didn't mention that seems like it could be a good place to
> start is https://code.djangoproject.com/ticket/16549
>
> Tim
>
> On Monday, December 8, 2014 8:04:30 AM UTC-5, Rune Kaagaard wrote:
>>
>> @Daniele
>>
>> I struggled a bit with how to display the error message, and your
>> solution makes perfect sense.
>>
>> Best,
>> Rune Kaagaard
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/f1a56260-89d7-4439-aed1-9407d799f62f%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAJxq84_Kz_6ZCTzxEjaZm61hdfYcKnQNHw-xyPr%3DzX%3Dtc8GYvw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Experimental APIs DEP

[Marc Tamlyn]

I'm going to agree with Tim, it's a good idea but it is hard to agree on
its impact without the specifics.

On 8 December 2014 at 15:04, Tim Graham  wrote:

> I am skeptical like Carl, but it seems to me the time to introduce and
> discuss this DEP is when there is an actual feature that we think could
> benefit from this policy so we don't have to talk in hypotheticals.
>
>
> On Monday, December 8, 2014 7:00:44 AM UTC-5, benjaoming wrote:
>>
>> Hi guys!
>>
>> As an external user of Django, relying on its stability, I'd like to
>> comment In general, I think this is possibly a good improvement.. but
>> possibly also a very dangerous one...
>>
>> On Monday, December 8, 2014 12:38:18 AM UTC+1, Russell Keith-Magee wrote:
>>>
>>>
>>> On Sun, Dec 7, 2014 at 6:35 PM, Shai Berger  wrote:
>>>
 I like the general idea of experimental API, although Carl and
 Aymeric's notes
 are important: If we do this, we need to very picky in its use, or else
 it
 just becomes an easy route to avoid committment. In particular, there
 should
 be a hard-and-fast rule that nothing should be made an "experimental
 API" if
 it can just be done outside of core.

 For the example given -- backend API -- I think that, again, Carl and
 Aymeric
 are right; the examples I have more in mind for this are user-facing ORM
 features like the expressions, or custom lookups. I think it might be
 better
 to let people play with them more, in a released version, before
 setting the
 APIs in concrete.

 With that in mind, I would also reconsider adding a silent warning when
 the
 features are used -- because, if they are for general use (as opposed
 to the
 use of, say, 3rd-party backend developers) then there's a significant
 use-case
 where the person who would use the API is not the person who configures
 the
 warnings on the CI.

>>>
>>> To my mind, the role of this new status is closer to "provisional",
>>> rather than "experimental". It's a recognition of the fact that no matter
>>> how many times we ask for pre-release testing, the first *real* test of any
>>> API is when it sees real-world use.
>>>
>>
>>
>> What I would fear is not so much that experimentals/provisionals end up
>> in other parts of Django itself: The Django team is well-disciplined and
>> will understand its own release protocol :)
>>
>> I'm more worried that the application ecology starts suffering from
>> symptoms of an experimental attitude in the Django project...
>>
>> This DEP would definitely have to be understood by the outside
>> app-developer: For the stability of third-party apps, app maintainers
>> should consider to put "don't use experimental APIs" in their contribution
>> guidelines and make sure to go through the painful process of rejecting
>> well-intended PRs relying on experimental components. Especially if it's
>> decided not to raise Warnings when experimental modules are imported.
>>
>> For the projects and applications that do use experimentals, it would be
>> nice to have a guarantee of a smooth API transition when the next Django
>> ships. DEP states that APIs can change, but that it *shouldn't* do so.
>> If stated that the API should not change, that causes for a much more
>> significant guarantee, and sets the bar reasonably high for new
>> experimental components.
>>
>> The amount of deprecations in Django in general is very high, and
>> currently, supporting both South and db.migrations is quite hard. So adding
>> more complexity to multi-version Django support does not sound very
>> attractive to me and should be avoided at all costs.
>>
>> Some suggestions for the "experimental" definition / ruleset:
>>
>>- API has been thoroughly agreed and is not subject to change.
>>- Only stability and performance is a questionable part that makes a
>>feature experimental.
>>- Internal parts of an experimental component are expected to change
>>more than usual
>>- Something does not get pulled out without a deprecation warning,
>>even though it's experimental
>>
>> If experimental components will be allowed to change their APIs, I would
>> definitely suggest a Warning to be raised so they're kept out of the app
>> ecology.
>> The relevant part of the DEP reads:
>>
>> Instead, an Experimental designation will allow APIs to be included in
>>> Django and released without being beholden to the full deprecation cycle.
>>>
>>
>> If you allow that to happen, then in essence, anything can be released
>> and removed again. But there should be a clear intention when something is
>> released, that it will be made permanent, right?
>>
>> DEP also states:
>>
>> Effort will be taken to communicate to users that the APIs in question
>>> are not stable to avoid inadvertent use.
>>>
>>
>> I don't think that's possible. If the feature is attractive, it will be
>> used.
>>
>> RKM wrote:

Re: Proposal: Prevent data loss in the admin

[Tim Graham]

Hi Rune,

It's not clear to me that a fix that's coupled to the admin is the way to 
go when this problem could exist for any model form, etc.

One ticket you didn't mention that seems like it could be a good place to 
start is https://code.djangoproject.com/ticket/16549

Tim

On Monday, December 8, 2014 8:04:30 AM UTC-5, Rune Kaagaard wrote:
>
> @Daniele
>
> I struggled a bit with how to display the error message, and your solution 
> makes perfect sense.
>
> Best,
> Rune Kaagaard
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f1a56260-89d7-4439-aed1-9407d799f62f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Experimental APIs DEP

[Tim Graham]

I am skeptical like Carl, but it seems to me the time to introduce and 
discuss this DEP is when there is an actual feature that we think could 
benefit from this policy so we don't have to talk in hypotheticals.

On Monday, December 8, 2014 7:00:44 AM UTC-5, benjaoming wrote:
>
> Hi guys!
>
> As an external user of Django, relying on its stability, I'd like to 
> comment In general, I think this is possibly a good improvement.. but 
> possibly also a very dangerous one...
>
> On Monday, December 8, 2014 12:38:18 AM UTC+1, Russell Keith-Magee wrote:
>>
>>
>> On Sun, Dec 7, 2014 at 6:35 PM, Shai Berger  wrote:
>>
>>> I like the general idea of experimental API, although Carl and Aymeric's 
>>> notes
>>> are important: If we do this, we need to very picky in its use, or else 
>>> it
>>> just becomes an easy route to avoid committment. In particular, there 
>>> should
>>> be a hard-and-fast rule that nothing should be made an "experimental 
>>> API" if
>>> it can just be done outside of core.
>>>
>>> For the example given -- backend API -- I think that, again, Carl and 
>>> Aymeric
>>> are right; the examples I have more in mind for this are user-facing ORM
>>> features like the expressions, or custom lookups. I think it might be 
>>> better
>>> to let people play with them more, in a released version, before setting 
>>> the
>>> APIs in concrete.
>>>
>>> With that in mind, I would also reconsider adding a silent warning when 
>>> the
>>> features are used -- because, if they are for general use (as opposed to 
>>> the
>>> use of, say, 3rd-party backend developers) then there's a significant 
>>> use-case
>>> where the person who would use the API is not the person who configures 
>>> the
>>> warnings on the CI.
>>>
>>
>> To my mind, the role of this new status is closer to "provisional", 
>> rather than "experimental". It's a recognition of the fact that no matter 
>> how many times we ask for pre-release testing, the first *real* test of any 
>> API is when it sees real-world use. 
>>
>
>
> What I would fear is not so much that experimentals/provisionals end up in 
> other parts of Django itself: The Django team is well-disciplined and will 
> understand its own release protocol :)
>
> I'm more worried that the application ecology starts suffering from 
> symptoms of an experimental attitude in the Django project...
>
> This DEP would definitely have to be understood by the outside 
> app-developer: For the stability of third-party apps, app maintainers 
> should consider to put "don't use experimental APIs" in their contribution 
> guidelines and make sure to go through the painful process of rejecting 
> well-intended PRs relying on experimental components. Especially if it's 
> decided not to raise Warnings when experimental modules are imported.
>
> For the projects and applications that do use experimentals, it would be 
> nice to have a guarantee of a smooth API transition when the next Django 
> ships. DEP states that APIs can change, but that it *shouldn't* do so. If 
> stated that the API should not change, that causes for a much more 
> significant guarantee, and sets the bar reasonably high for new 
> experimental components.
>
> The amount of deprecations in Django in general is very high, and 
> currently, supporting both South and db.migrations is quite hard. So adding 
> more complexity to multi-version Django support does not sound very 
> attractive to me and should be avoided at all costs.
>
> Some suggestions for the "experimental" definition / ruleset:
>
>- API has been thoroughly agreed and is not subject to change.
>- Only stability and performance is a questionable part that makes a 
>feature experimental.
>- Internal parts of an experimental component are expected to change 
>more than usual
>- Something does not get pulled out without a deprecation warning, 
>even though it's experimental
>
> If experimental components will be allowed to change their APIs, I would 
> definitely suggest a Warning to be raised so they're kept out of the app 
> ecology.
> The relevant part of the DEP reads:
>
> Instead, an Experimental designation will allow APIs to be included in 
>> Django and released without being beholden to the full deprecation cycle.
>>
>
> If you allow that to happen, then in essence, anything can be released and 
> removed again. But there should be a clear intention when something is 
> released, that it will be made permanent, right?
>
> DEP also states:
>
> Effort will be taken to communicate to users that the APIs in question are 
>> not stable to avoid inadvertent use.
>>
>
> I don't think that's possible. If the feature is attractive, it will be 
> used.
>
> RKM wrote:
>
> due to fears over whether the public-facing API is suitable for use. 
>>
>
> This could be rephrased: Because some components do not get the exposure 
> necessary for thorough testing through the availability and use of the beta 
> releases, we need 

Re: Proposal: Prevent data loss in the admin

[Rune Kaagaard]

@Daniele

I struggled a bit with how to display the error message, and your solution 
makes perfect sense.

Best,
Rune Kaagaard

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/022511a4-e5d2-4b7e-9656-befec87cd595%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: Prevent data loss in the admin

[Daniele Procida]

On Mon, Dec 8, 2014, Rune Kaagaard  wrote:

>I've made a branch that adds optimistic concurrency control to the admin: 
>https://github.com/runekaagaard/django-lock-the-admin/compare/adminlock. It 
>works for the main change object and inlines and has tests. All tests 
>passes with postgres and sqlite.

I like the concept.

I would prefer however to see something like:

3 items on this page have been updated by another user while you were 
editing them.

You can:

* see the other user's changes[link] in a new browser window, and apply 
your changes there, or
* refresh this pages[link] (Warning! your changes here will be lost!) 
and reapply your changes here

This way the user is invited to compare the differences and to copy their 
changes over, rather than simply lose them.

Daniele

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20141208125127.743977646%40mail.wservices.ch.
For more options, visit https://groups.google.com/d/optout.

Re: uuid field short websafe representation

[Andrey Antukh]

Hi.

2014-12-08 10:41 GMT+01:00 Radek Svarz :

> Michael, Florian, I understand your remarks.
>
> Allow me to explain more.
>
> I do not advocate to replace the code by the one posted by me. I rather
> advocate to improve it.
>
> ad 1) I just react to the current implementation, where in the case of
> other DBMS than PostgreSQL the hex value in 32 chars is stored. In such
> cases I propose to store it in a smaller amount of 21 characters. ( =
> storage optimization )
>
As far as I know, postgresql internally stores it in binary format.


>
> ad 2) web safe representation
> The goal is to translate URLs back and forth. E.g.
> server.com/apiv1/client/uuid/0b043d5842ca4cab9750b705018f4a1f should
> allow direct mapping to the ORM object. I am not sure whether the current
> implementation allows that. Or at least the documentation is not clear
> about it.
>

> ad >> A non-standard, compressed unique value is not a UUID.
>  Base64 is just different encoding, so value wise you still get the same
> UUID.
>

In my opinon, -1.

The presentation format of uuid should not affect in any way the storage of
it. Currently with backends that supports nativelly uuid types as
postgresql, stores it in the most efficient way.

My two cents!

Regards.
Andrey


> Radek
>
> On Sunday, December 7, 2014 10:16:00 AM UTC+1, Florian Apolloner wrote:
>>
>> +1 to everything you said, if someone wants a "websafe" representation,
>> they can always just manually call safe_uuid on the UUID instance.
>>
>> On Saturday, December 6, 2014 6:00:58 PM UTC+1, Michael Manfre wrote:
>>>
>>> A non-standard, compressed unique value is not a UUID. Also, this forces
>>> database backends to store the value in a string datatype and doesn't allow
>>> taking advantage of uuid specific datatypes. E.g. Postgresql couldn't use
>>> its uuid datatype. Any data can be made safe for any specific usage, e.g.
>>> URLs, but there is no reason to enforce this requirement in all uses of the
>>> data because not everyone will expose a UUID in a URL.
>>>
>>> -1 from me.
>>>
>>> Regards,
>>> Michael Manfre
>>>
>>> On Sat, Dec 6, 2014 at 11:31 AM, Radek Svarz  wrote:
>>>
 Hi, I am glad to see the UUID field coming to 1.8

 Bellow is how we do it now in 1.7.

 The advantages:

  - it only uses 21 chars (instead of 32)

  - chars are safe for URLs

  - uuid is created when default is called

 I advocate to have the short websafe representation. What do you think?

 code:

> def safe_uuid():
> return uuid.uuid4().bytes.encode('base64').rstrip('=\n').replace('/',
> '_')
>


> class UUIDField(CharField) :
> """
> UUIDField stored in 21 Chars
> Example: uuid = UUIDField(primary_key=True, editable=False)
> """
> description = "UUIDField stored in 21 Chars"
> def __init__(self, *args, **kwargs):
> kwargs['max_length'] = kwargs.get('max_length', 22 )
> kwargs['default'] = safe_uuid
> CharField.__init__(self, *args, **kwargs)
>
> def deconstruct(self):
> name, path, args, kwargs = super(UUIDField, self).deconstruct()
> return name, path, args, kwargs


 Radek

  --
 You received this message because you are subscribed to the Google
 Groups "Django developers (Contributions to Django itself)" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-develop...@googlegroups.com.
 To post to this group, send email to django-d...@googlegroups.com.
 Visit this group at http://groups.google.com/group/django-developers.
 To view this discussion on the web visit https://groups.google.com/d/
 msgid/django-developers/1c29dfae-6483-465c-939e-
 f4319120781f%40googlegroups.com
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/396e017f-3c57-486c-a7d3-6d9f0a2e9d7a%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Andrey Antukh - Андрей Антух - 

Proposal: Prevent data loss in the admin

[Rune Kaagaard]

Hi

I've made a branch that adds optimistic concurrency control to the admin: 
https://github.com/runekaagaard/django-lock-the-admin/compare/adminlock. It 
works for the main change object and inlines and has tests. All tests 
passes with postgres and sqlite.

This prevents users from accidentally overwriting each others changes. To 
confirm this is is the case today [1]:

1. Open the same change page in the admin in two tabs.
2. Make different modifications to them. 
3. Press save in both tabs. 
4. The changes in the last saved tab silently overwrites the first ones.

In my proposal an edit conflict looks like:












































It works by:

- Adding a "admin.Lock" model with version number, pk and content type of 
the objects being edited.
- Adding "lock_version" hidden fields to the main object form and inline 
forms.
- Wrapping the save part of "changeform_view" in a transaction that rolls 
back if a any object on the page has a stale version.
- Showing an error message if the user wasn't allowed to save.

What do you think, is this something I should open a ticket for and start 
working on?

*Scope*

This proposal doesn't cover editable inlines [2], but they are kind of 
unusable as is, unless you are a single admin, that really knows what you 
are doing. I would love to work on that for v1.8, but prefers getting a 
locking mechanism in place first without it!?

The same goes for locking of admin actions and the cryptic error you get 
when you are deleting an already deleted inline [3].

Rails supports both pessimistic and optimistic locking [4] [5]. It would be 
cool to have a "contrib.locking" module, but I think it's out of scope for 
this proposal. Plus, it will be easy to change the locking method when/if 
such a module happens.

*Implementation*

1. Should there be a settings.ADMIN_USE_LOCKING?
2. Should there be a ModelAdmin.use_locking property?
3. Instead of having an "admin.Lock" model you could have a "version" field 
on each table and make a LockingModel available. I didn't do that because i 
wanted to prevent data loss in the admin out of the box, also for existing 
projects. What's your take on that?

*Should it be a third party app?*

I think the "killer app" for Django should be safe by default. Third party 
solutions already exists, though [6] [7] [8].

*Test project*

To test the UI of this feature, use the adminlock-test branch:

git clone https://github.com/runekaagaard/django-lock-the-admin.git
cd django-lock-the-admin
git checkout adminlock-test
cd test_project
python manage.py migrate
python manage.py createsuperuser
python manage.py runserver

*References:*

- [1] https://code.djangoproject.com/ticket/11652
- [2] https://code.djangoproject.com/ticket/11313
- [3] https://code.djangoproject.com/ticket/15574
- 
[4] http://edgeapi.rubyonrails.org/classes/ActiveRecord/Locking/Optimistic.html
- 
[5] http://api.rubyonrails.org/classes/ActiveRecord/Locking/Pessimistic.html
- [6] https://github.com/saxix/django-concurrency
- [7] https://github.com/RobCombs/django-locking
- [8] https://github.com/runekaagaard/django-admin-locking

Best,
Rune Kaagaard

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ec0a7667-68c3-4ae7-8e46-71cc7c9bbc01%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: uuid field short websafe representation

[Florian Apolloner]

Hi Radekm

On Monday, December 8, 2014 10:41:41 AM UTC+1, Radek Svarz wrote:
>
> ad 1) I just react to the current implementation, where in the case of 
> other DBMS than PostgreSQL the hex value in 32 chars is stored. In such 
> cases I propose to store it in a smaller amount of 21 characters. ( = 
> storage optimization )
>
 
I guess storing the raw bytes there would make sense.

ad 2) web safe representation
> The goal is to translate URLs back and forth. E.g. 
> server.com/apiv1/client/uuid/0b043d5842ca4cab9750b705018f4a1f 
> 
>  
> should allow direct mapping to the ORM object. I am not sure whether the 
> current implementation allows that. Or at least the documentation is not 
> clear about it.
>

I'd expect an UUIDField to return and take UUID objects as values, or a 
RFC4122 compatible string representation. And regarding url safety, how is 
a random uuid like 729dfd34-5681-409a-8d6a-ee25b4cf3f58 not safe? The 4 
byte difference to your proposed variant (the one from your server.com url) 
is imo not worth any hazzle (if you want you can easily strip '-').

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f286e13c-bca3-4a0c-a223-708af322504d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Experimental APIs DEP

[benjaoming]

Hi guys!

As an external user of Django, relying on its stability, I'd like to 
comment In general, I think this is possibly a good improvement.. but 
possibly also a very dangerous one...

On Monday, December 8, 2014 12:38:18 AM UTC+1, Russell Keith-Magee wrote:
>
>
> On Sun, Dec 7, 2014 at 6:35 PM, Shai Berger  > wrote:
>
>> I like the general idea of experimental API, although Carl and Aymeric's 
>> notes
>> are important: If we do this, we need to very picky in its use, or else it
>> just becomes an easy route to avoid committment. In particular, there 
>> should
>> be a hard-and-fast rule that nothing should be made an "experimental API" 
>> if
>> it can just be done outside of core.
>>
>> For the example given -- backend API -- I think that, again, Carl and 
>> Aymeric
>> are right; the examples I have more in mind for this are user-facing ORM
>> features like the expressions, or custom lookups. I think it might be 
>> better
>> to let people play with them more, in a released version, before setting 
>> the
>> APIs in concrete.
>>
>> With that in mind, I would also reconsider adding a silent warning when 
>> the
>> features are used -- because, if they are for general use (as opposed to 
>> the
>> use of, say, 3rd-party backend developers) then there's a significant 
>> use-case
>> where the person who would use the API is not the person who configures 
>> the
>> warnings on the CI.
>>
>
> To my mind, the role of this new status is closer to "provisional", rather 
> than "experimental". It's a recognition of the fact that no matter how many 
> times we ask for pre-release testing, the first *real* test of any API is 
> when it sees real-world use. 
>


What I would fear is not so much that experimentals/provisionals end up in 
other parts of Django itself: The Django team is well-disciplined and will 
understand its own release protocol :)

I'm more worried that the application ecology starts suffering from 
symptoms of an experimental attitude in the Django project...

This DEP would definitely have to be understood by the outside 
app-developer: For the stability of third-party apps, app maintainers 
should consider to put "don't use experimental APIs" in their contribution 
guidelines and make sure to go through the painful process of rejecting 
well-intended PRs relying on experimental components. Especially if it's 
decided not to raise Warnings when experimental modules are imported.

For the projects and applications that do use experimentals, it would be 
nice to have a guarantee of a smooth API transition when the next Django 
ships. DEP states that APIs can change, but that it *shouldn't* do so. If 
stated that the API should not change, that causes for a much more 
significant guarantee, and sets the bar reasonably high for new 
experimental components.

The amount of deprecations in Django in general is very high, and 
currently, supporting both South and db.migrations is quite hard. So adding 
more complexity to multi-version Django support does not sound very 
attractive to me and should be avoided at all costs.

Some suggestions for the "experimental" definition / ruleset:

   - API has been thoroughly agreed and is not subject to change.
   - Only stability and performance is a questionable part that makes a 
   feature experimental.
   - Internal parts of an experimental component are expected to change 
   more than usual
   - Something does not get pulled out without a deprecation warning, even 
   though it's experimental

If experimental components will be allowed to change their APIs, I would 
definitely suggest a Warning to be raised so they're kept out of the app 
ecology.
The relevant part of the DEP reads:

Instead, an Experimental designation will allow APIs to be included in 
> Django and released without being beholden to the full deprecation cycle.
>

If you allow that to happen, then in essence, anything can be released and 
removed again. But there should be a clear intention when something is 
released, that it will be made permanent, right?

DEP also states:

Effort will be taken to communicate to users that the APIs in question are 
> not stable to avoid inadvertent use.
>

I don't think that's possible. If the feature is attractive, it will be 
used.

RKM wrote:

due to fears over whether the public-facing API is suitable for use. 
>

This could be rephrased: Because some components do not get the exposure 
necessary for thorough testing through the availability and use of the beta 
releases, we need experimental components in the final releases.

The goal is to get a broader test audience for *more rapid* releases and to 
have *faster* maturing of new features. Just be careful that this stuff 
doesn't end up where it's not supposed to, that's my main worry.

Other than that, it's really easy to release django applications, and 
features not agreed to enter the django mainline have always been referred 
to release as an external 

Re: uuid field short websafe representation

[Radek Svarz]

Michael, Florian, I understand your remarks.

Allow me to explain more.

I do not advocate to replace the code by the one posted by me. I rather 
advocate to improve it.

ad 1) I just react to the current implementation, where in the case of 
other DBMS than PostgreSQL the hex value in 32 chars is stored. In such 
cases I propose to store it in a smaller amount of 21 characters. ( = 
storage optimization ) 

ad 2) web safe representation
The goal is to translate URLs back and forth. E.g. 
server.com/apiv1/client/uuid/0b043d5842ca4cab9750b705018f4a1f should allow 
direct mapping to the ORM object. I am not sure whether the current 
implementation allows that. Or at least the documentation is not clear 
about it.

ad >> A non-standard, compressed unique value is not a UUID.
 Base64 is just different encoding, so value wise you still get the same 
UUID. 

Radek

On Sunday, December 7, 2014 10:16:00 AM UTC+1, Florian Apolloner wrote:
>
> +1 to everything you said, if someone wants a "websafe" representation, 
> they can always just manually call safe_uuid on the UUID instance.
>
> On Saturday, December 6, 2014 6:00:58 PM UTC+1, Michael Manfre wrote:
>>
>> A non-standard, compressed unique value is not a UUID. Also, this forces 
>> database backends to store the value in a string datatype and doesn't allow 
>> taking advantage of uuid specific datatypes. E.g. Postgresql couldn't use 
>> its uuid datatype. Any data can be made safe for any specific usage, e.g. 
>> URLs, but there is no reason to enforce this requirement in all uses of the 
>> data because not everyone will expose a UUID in a URL.
>>
>> -1 from me.
>>
>> Regards,
>> Michael Manfre
>>
>> On Sat, Dec 6, 2014 at 11:31 AM, Radek Svarz  wrote:
>>
>>> Hi, I am glad to see the UUID field coming to 1.8
>>>
>>> Bellow is how we do it now in 1.7.
>>>
>>> The advantages:
>>>
>>>  - it only uses 21 chars (instead of 32)
>>>
>>>  - chars are safe for URLs
>>>
>>>  - uuid is created when default is called
>>>
>>> I advocate to have the short websafe representation. What do you think?
>>>
>>> code:
>>>
 def safe_uuid():
 return 
 uuid.uuid4().bytes.encode('base64').rstrip('=\n').replace('/', '_')

>>>  
>>>
 class UUIDField(CharField) :
 """
 UUIDField stored in 21 Chars
 Example: uuid = UUIDField(primary_key=True, editable=False)
 """ 
 description = "UUIDField stored in 21 Chars"
 def __init__(self, *args, **kwargs):
 kwargs['max_length'] = kwargs.get('max_length', 22 )
 kwargs['default'] = safe_uuid 
 CharField.__init__(self, *args, **kwargs)
 
 def deconstruct(self):
 name, path, args, kwargs = super(UUIDField, self).deconstruct()
 return name, path, args, kwargs
>>>
>>>
>>> Radek
>>>
>>>  -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/1c29dfae-6483-465c-939e-f4319120781f%40googlegroups.com
>>>  
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/396e017f-3c57-486c-a7d3-6d9f0a2e9d7a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.