Re: New-style middleware

[Carl Meyer]

Hi Torsten,

I worked on the design and implementation of new-style middleware.

On 11/22/2016 01:30 PM, Torsten Bronger wrote:
> Hallöchen!
> 
> Considering the following old-style middleware class:
> 
> class ExceptionsMiddleware:
> def process_exception(self, request, exception):
> ...
> 
> I convert this to new-style middleware by inserting two methods:
> 
> class ExceptionsMiddleware:
> def __init__(self, get_response):
> self.get_response = get_response
> def __call__(self, request):
> return self.get_response(request)
> def process_exception(self, request, exception):
> ...
> 
> Is this really correct?  Actually, the old way looks more concise to
> me.  In particular, is there a reason why Django does not provide a
> non-deprecated base class for middleware like:
> 
> class Middleware:
> def __init__(self, get_response):
> self.get_response = get_response
> def __call__(self, request):
> return self.get_response(request)

Yes, I agree with you that for middleware which don't implement request
or response processing (only process_exception, process_view, or
process_template_response), the new style is less concise, because it
requires implementing a boilerplate __init__ and __call__. To be honest,
the primary consideration in the new middleware was request and response
processing, and your case did not receive as much attention as it
perhaps should have.

I've considered a few possible approaches to fix this, and I think your
suggestion of a built-in base class that implements the
simplest-possible version of __init__ and __call__ makes sense; it is
the least magical and most explicit option.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/1e256b81-9e79-88ec-46d9-ee8041e5d712%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Testing a reusable app which uses a database

[Carl Meyer]

On 10/03/2016 01:41 PM, Victor Porton wrote:
> I try to make a reusable app which uses transaction.atomic().
> 
> When I run my test.py (below) for my reusable app, I get the below errors.
> 
> Is it possible to test this reusable app outside of a Django project?

It is not possible to test it without setting up Django, no. But you
don't need a full-fledged "project" for this, the bare minimum you need
to do is

import django
from django.conf import settings

settings.configure()  # if you need any settings, pass as a dict
django.setup()

before running your tests.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/0c2f6add-3aab-cb11-981e-3147e203a12a%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: An almost reusable app, how to make it reusable?

[Carl Meyer]

Hi Victor,

On 10/03/2016 01:06 PM, Victor Porton wrote:
> I've created an app which is "almost" reusable: it depends on a variable
> in /setting.py of our project .
> 
> from .settings import TRANSACTION_ATTEMPT_COUNT
> 
> Can it be made into an reusable app?

Certainly. Change the import to `from django.conf import settings`, so
you import the settings from Django rather than from your project (this
is generally the best way to access settings anyway). Then establish
some reasonable default value for the setting, and access it like
`getattr(settings, 'TRANSACTION_ATTEMPT_COUNT', default_value)`. Then
your app can work in any project, even if that project doesn't set a
value for that setting.

If there is no reasonable default value, and you want to force any
project using your app to set some value explicitly, you can also just
raise an error if the setting doesn't exist and in the exception message
instruct the developer that they must set the setting.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/91f5fe1f-5436-13d1-dedf-6e48bf1dcab1%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Extracting the username:password from the host

[Carl Meyer]

Hi Christophe,

On 09/12/2016 03:35 PM, Christophe Pettus wrote:
> I've encountered a service that does postbacks as part of its API.
> The only authentication mechanism is putting the username and
> password in the POST URL in the form:
> 
> https://user:p...@example.com/api/endpoint
> 
> Is there a portable way of extracting that information from the
> request object?

Yes, it should be in `request.META['HTTP_AUTHORIZATION']`; see
https://www.djangosnippets.org/snippets/243/ for an example.

The user:pass@host URL syntax is just a way of expressing credentials
for HTTP Basic Authentication; the values should end up in the
Authorization header.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/ff41ac39-6fb9-43c1-b254-8751ba925dc7%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Is a good practice to put app inside another?

[Carl Meyer]

On 06/03/2016 01:37 PM, Neto wrote:
> My project has several apps, some of them depend on others, it is ok to
> add for example 3 apps within another app?

Yep, there's nothing wrong with this. An "app" is just a Python package;
nesting Python packages to achieve a logical grouping/hierarchy is a
perfectly reasonable thing to do.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/575209E6.6080709%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Possible bug with transaction.on_commit

[Carl Meyer]

Hi Bart,

On 05/16/2016 02:10 PM, barthel...@infobart.com wrote:
> I believe that I encountered a bug or at least a serious limitation with
> transaction.on_commit in Django 1.9.x. Essentially, the on_commit hooks
> seem to be tied to the database connection instead of the transaction.
> This means that unrelated transactions may trigger on_commit hooks,
> which results in undesired execution order.
[...]
> I wanted to know if someone encountered this issue or if I am
> misunderstanding on_commit before opening a ticket.

I am the author of on_commit, and I think the behavior you have
encountered is a bug that should be fixed. Doing additional database
work in `on_commit` callbacks wasn't really a use case I had in mind in
the design; I made sure it basically worked, but clearly didn't explore
it sufficiently in cases of multiple registered hooks.

Storing the on_commit state on the connection is not really optional,
since that's where _all_ transaction-related state is stored, but I
think it should still be possible to fix this. Perhaps by having
`run_and_clear_commit_hooks` copy the list of hooks into a local
variable and immediately clear the state on the connection before it
calls any of the hooks? Not entirely sure, need to add a failing test
and play with it a bit.

Thanks for catching and reporting this. If you'd be willing to file a
ticket in Trac, that'd be great.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/573AB193.5020807%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django formset hidden id field

[Carl Meyer]

Hi Rob,

On 05/03/2016 12:13 PM, Rob Ladd wrote:
> Carl Meyer said:
> 
> "Whatever queryset you pass to the model formset limits the 
> available rows for editing. The end user can edit the PK to refer to any 
> item in that queryset, but not any item in the table. "
> 
> That's not true, based on my observation. 
> As long as the PK refers to any object of that type, it can be edited by
> monkeying with the hidden id field.

I replied to this in the other thread you started:
https://groups.google.com/d/topic/django-users/tJpKWbij1B0/discussion

Short version, I can't reproduce the behavior you describe.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/572D2673.2010800%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django formset security and injecting PKs in formset hidden id fields

[Carl Meyer]

Hi Rob,

On 05/03/2016 12:23 PM, Rob Ladd wrote:
> I've noticed something troubling in Django formsets:
> 
> Each formset.form has a hidden field with the id of the model being edited. 
> 
> All one would need to do is change this id and submit, and the default
> formset |clean()| or |save()| methods don't bat an eye.

First of all, if you believe that you've found a security issue in
Django, please err on the side of caution and email
secur...@djangoproject.com, rather than reporting it in public. Thanks!

Second: can you provide a sample project demonstrating the behavior you
describe? I'm unable to reproduce that behavior, and by code inspection
I don't see how that behavior could occur, either. If you pass in the
`queryset` keyword argument when instantiating the formset, only objects
matching that queryset will be editable via the formset. Meddling with
IDs will at best allow you to create a new object, never to edit one
that's outside of `queryset`. But formsets always allow creation of new
objects.

It is true that the given queryset could match some new object on submit
that wasn't part of the queryset when the formset was initially
displayed, if that object was created or edited such that it now matches
the queryset filters in the meantime. There isn't any reasonable way for
Django to address this problem: there isn't necessarily any location
available to store state in between display and submit other than the
page, and that's under the control of the user. You should just be sure
that the filters applied to the `queryset` accurately describe the
limitations you want applied to the editing abilities of the user being
shown the formset. In a typical use case, the `queryset` would apply a
filter like `(owner=request.user)`, allowing a user to edit only the
objects they own.

If you still believe you're seeing a security issue here, please email
secur...@djangoproject.com with step by step reproduction instructions.

Thanks,

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/572D253C.4070301%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django project structure

[Carl Meyer]

Hi André,

On 02/04/2016 02:47 PM, André Meyer-Vitali wrote:
> How would I modify my contest app so that it knows about multiple
> contests? Database models (and queries), templates, forms, urls need to
> be adapted, I guess. Is it sufficient to add an ID to the models such
> that a specific month's data can be selected, for example?
> 
> Is there an example to check out somewhere?

These are hard questions to answer without knowing more about your
contest app: how it works currently, what it does, what models it has,
and how you want it to work with a new contest each month.

I would imagine something like having a Contest model (maybe with a
start_date field and an end_date field?), so you can create a new
Contest anytime you want, and then having any other models in the app
all link (with a ForeignKey) to the Contest model, so all their
instances are specific to one Contest. That would be the more flexible
approach.

But I suppose if you are sure that you want "one contest each month" and
that's it, no more flexibility is needed, you could just add date fields
to some of the other models and then modify your queries to select only
objects for the current month?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/56B3C8DB.8060304%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django project structure

[Carl Meyer]

Hi André,

On 02/04/2016 02:05 PM, André Meyer-Vitali wrote:
> There's quite some confusion here about the best way to structure a
> Django project.
> 
> I'm developing a photography related web site with a news section (based
> on the poll tutorial) and a contest app (among others). The contest app
> should be reused every month for a new contest. How does this work? What
> need to change? The  templates, app, database? I hope that the app does
> not need to change, but how can the monthly instances be accessed
> separately? So far, I couldn't find information and examples for more
> complicated web sites. Where can I find this?

I recommend simply modifying your contest app so that it knows about
multiple contests, not just one. That would be the usual way to approach
this.

(Django does have the "url namespace" feature, which lets you mount
multiple "instances" of an app's urlconf at different URLs, but that
feature doesn't do anything for you at the model or database layer, so
all the "instances" of the app would still share the same data. So I
doubt that this feature actually helps with your problem. Personally,
I've never actually found a good use case for url namespaces, outside of
contrib.admin.)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/56B3C318.3060406%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Unintuitive behaviour of management commands with multiple databases

[Carl Meyer]

Hi Vinay,

On 02/02/2016 12:14 PM, 'Vinay Sajip' via Django users wrote:
> I'm not arguing for any particular different routing scheme to be
> included - only for management commands to be able to be written to
> respect --data arguments passed to them, and which can easily treat the
> passed value as the default database to use just for that command
> invocation, when that value is different to whatever
> settings.DATABASES['default'] is. A quick look at the builtin management
> commands shows there's a lot of usage
> of connections[options.get('database')] going on, and yet externally
> written management commands aren't encouraged to use the same approach,
> and multiple settings files are the suggested alternative?

There's nothing wrong with connections[options.get('database')] if you
want to get a specific database connection object; that's public API and
you can use it. In the case of the built-in commands, that's useful
because they need to do low-level things with the connection; I don't
think it helps you for regular ORM use. It doesn't allow changing the
default connection for ORM usage. Where the built-in management commands
do use the higher-level ORM, they use the public `using` APIs.

In other words, the built-in management commands aren't doing anything
different from what I originally recommended you do (just use `using`).

If you want to write a management command that has a --database option
like the built-in ones and makes heavy use of the ORM, you can a)
explicitly provide `using` where needed , b) use
django-dynamic-db-router, or c) hold your nose and monkeypatch
DEFAULT_DB_ALIAS, since it's just a one-shot management command and you
don't need to be as concerned about cleaning up after yourself.

Personally I would go for (a) or (b), depending on just how much I was
using the ORM in the management command. I'm not convinced that a fourth
(or third supported public) option is necessary here. If it's just a few
queries, you use `using`, if it's more than that and you want a broader
policy applied, you use a database router.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/56B10877.10900%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Unintuitive behaviour of management commands with multiple databases

[Carl Meyer]

Hi Vinay,

On 02/02/2016 10:56 AM, 'Vinay Sajip' via Django users wrote:
> On Tuesday, February 2, 2016 at 5:08:40 PM UTC, Carl Meyer wrote: 
> 
> You can't (well, you might be able to by poking around in the internals
> of the django.db.connections object, but I'd strongly advise against
> that). The proper (and thread-safe) way to achieve the equivalent is to
> 
> Well, for any given run of the application (on which the example I
> posted is based), only one database is ever used. The idea of multiple
> databases was to allow us to select which one is used for any particular
> Python process run, and we expected to be able to run management
> commands which would (based on a settings configuration) determine both
> which database to use and e.g. how to populate parts of it ready for
> deployment to particular customers.

For this scenario, I would use a different settings file (which can all
"inherit" the same common settings via the `from common_settings import
*` trick) for each "customer", with a different default database defined
in each. In fact, is there any reason for one process to have access to
another process' database at all, in your scenario? I don't think you
even have a use-case for Django's multi-db support here at all, just a
use case for multiple settings file variants. Then you can use the
--settings option with management commands to choose which settings to
use (or the DJANGO_SETTINGS_MODULE env var where that's convenient); you
don't need to define your own custom option.

Django's multi-db support is designed for what you're calling "run-time
routing," where a single process needs to access multiple different
databases at different times. That's not your situation; I'm not sure
you even need or want multi-db.

> Oddly, setting
> django.db.utils.DEFAULT_DB_ALIAS = alias does the trick. Yes, I know
> that's a no-no, but it seems to me that for this kind of case where you
> don't need run-time routing, something like django-dynamic-db-router
> shouldn't really be needed, useful though it might be for the run-time
> routing case. Everything else works off the settings module, and the
> fact that Django caches the default database to use such that a
> management command can't change it seems like a design flaw.

The "design flaw" you are observing here is not specific to databases,
but to Django settings. Django settings in general are not designed to
be modified at runtime; they define a static configuration for a given
process. Some settings can in practice safely be modified on-the-fly
(but still probably shouldn't be), some cannot, because their value is
cached for efficiency or other reasons: in general modifying settings at
runtime is not a supported technique. And it's not a _necessary_
technique either, in your case, when you could just instead use the
right settings for each process from the beginning.

> I suppose I
> was expecting transaction.atomic(using=XXX) to not just open a
> transaction on that database, but also make it the default for the scope
> of the block (except if explicitly overridden by using() for particular
> models in code called from that block). The current behaviour seems to
> violate the principle of least surprise (this my first encounter with
> multiple databases).

I think Django's design in this case is superior to the one you propose,
because it keeps two concepts orthogonal that don't need to be linked
(transactions and multi-db-routing) and avoids implicit global or
thread-local state changes in operations that don't obviously imply such
changes, making the overall system more flexible and predictable.

I suppose we'd need a more scientific survey to establish which behavior
is more surprising to more people :-)

> Is there really no case for an in_database()
> context manager in Django itself?

Perhaps. In my experience the more typical uses for multi-db are
amenable to other types of routing (e.g. models X, Y are always routed
to database A, model Z is always routed to database B, or more complex
schemes), and dynamically-scoped routing (such as that provided by
`in_database`) isn't as commonly needed.

Django provides the database router abstraction, which is a very
flexible system for implementing whatever kind of multi-db routing you
need. The burden of proof is heavy on any particular routing scheme to
demonstrate that it is so frequently needed that it should be bundled
with Django itself, rather than being a separate reusable package for
those who need it.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://gro

Re: Unintuitive behaviour of management commands with multiple databases

[Carl Meyer]

Hi Vinay,

On 02/02/2016 09:52 AM, 'Vinay Sajip' via Django users wrote:
> 
> On Tuesday, February 2, 2016 at 3:27:10 PM UTC, Carl Meyer wrote:
> 
> Nothing in your code ever "overrides" settings.DATABASES['default'].
> 
> 
> Dear Carl,
> 
> Thanks for the quick response. I /had/ thought of that, and tried adding
> the statement settings.DATABASES['default'] = settings.DATABASES[alias]
> just before the with block, and it had no effect - the result was the
> same - so I took it out. How else are you supposed to override the
> default database?

You can't (well, you might be able to by poking around in the internals
of the django.db.connections object, but I'd strongly advise against
that). The proper (and thread-safe) way to achieve the equivalent is to
do what the third-party project I linked (django-dynamic-db-router)
does: write a db router that references some thread-local state to
determine which database to route queries to, and write a context
manager or whatever to modify that thread-local state. Or just use
django-dynamic-db-router, since it already does what you want :-)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/56B0E273.8070604%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Unintuitive behaviour of management commands with multiple databases

[Carl Meyer]

Hi Vinay,

On 02/02/2016 08:11 AM, 'Vinay Sajip' via Django users wrote:
> I've set up a simple project using with two databases, foo and bar in
> settings.py:
> 
> |
> DATABASES ={
> 'foo':{
> 'ENGINE':'django.db.backends.sqlite3',
> 'NAME':os.path.join(BASE_DIR,'foo.sqlite'),
> },
> 'bar':{
> 'ENGINE':'django.db.backends.sqlite3',
> 'NAME':os.path.join(BASE_DIR,'bar.sqlite'),
> }
> }
> DATABASES['default']=DATABASES['foo']
> 
> |
> 
> I have a simple model:
> 
> |
> classThing(models.Model):
> name =models.CharField(max_length=20,unique=True)
> |
> 
> I have a simple management command:
> 
> |
> classCommand(BaseCommand):
> help ='Add all the things.'
> 
> 
> defadd_arguments(self,parser):
> parser.add_argument('--database',nargs='?',metavar='DATABASE',
> default='default',
> help='Database alias to use')
> parser.add_argument('things',nargs='+',metavar='THING',
> help='Things to add')
> 
> 
> defhandle(self,*args,**options):
> alias=options['database']
> self.stdout.write('using alias %s'%alias)
> withtransaction.atomic(using=alias):
> forname inoptions['things']:
> try:
> Thing.objects.create(name=name)
> self.stdout.write('Added %s.\n'%name)
> exceptIntegrityErrorase:
> self.stderr.write('Failed to add thing %r: %s'%(name,e))
> break
> |
> 
> After running migrations to set up the two databases, using python
> manage.py migrate --data foo and python manage.py migrate --data bar, I
> then run python manage.py add_things fizz buzz which results in two
> records being added to foo.sqlite, as expected. 
> 
> If I then run python manage.py add_things fizz buzz --data bar, it seems
> reasonable to expect it to add the records to bar.sqlite. However, this
> is not what happens: it tries to add them to foo.sqlite, even though
> I've specified using=alias with the alias set to bar. So I get a
> constraint violation:
> 
> |
> usingaliasbar
> Failedto add thing 'fizz':UNIQUE constraint failed:hello_thing.name
> |
> 
> What have I overlooked? In a real case the atomic block might be
> manipulating lots of models in nested code, and I can't see that it's
> practical to call using() for every model. Somewhere, it looks like
> Django code is caching a connection based on what
> settings.DATABASES['default'] was when settings was imported, even
> though it is being overridden in the command line. If not actually a
> bug, this behaviour doesn't seem particularly intuitive, so any advice
> would be gratefully received.

Nothing in your code ever "overrides" settings.DATABASES['default'].
Defining an atomic block on a certain database has no effect on the
default database within that block, it just opens a transaction on the
requested database. "Which database is default" and "which database(s)
has/have transactions open on them" are completely orthogonal questions,
there is no implicit link between them. Nothing prevents you from having
simultaneous transactions open on multiple databases, and interspersing
queries to different databases while those transactions are open.

If calling .using() explicitly to use a non-default database is
burdensome (and the situation can't be handled automatically with a
custom database router class), you could look at a third-party solution
like django-dynamic-db-router [1], which provides an `in_database`
context manager that does what you were expecting `transactions.atomic`
to do for you.

Carl

[1]
https://github.com/ambitioninc/django-dynamic-db-router/blob/master/docs/quickstart.rst

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/56B0CA99.2090308%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django imports models for apps that aren't enabled?

[Carl Meyer]

Hi Stodge,

On 01/05/2016 08:57 AM, Stodge wrote:
> I'm using Django 1.6.9. When the dev server (and hence Apache) starts,
> is Django supposed to import models for apps that exist on disk, but
> aren't enabled in the settings? I'm seeing this behaviour and I wasn't
> expecting it. Thanks

Django shouldn't do that on its own. AFAIK the only reason a models file
should be imported is a) because it's listed in INSTALLED_APPS, or b)
because something else that is imported imports it (possibly also c) an
installed model has an FK linking to it with a string reference?).

My first guess would be that you're seeing case (b). If you just add an
"assert False" at module-level in the models file in question, the
resulting traceback should show you the import chain leading to its import.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/568BE8AE.4040507%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: List of available Jinja2 global symbols

[Carl Meyer]

On 11/22/2015 06:21 PM, Patrick Spencer wrote:
> I see what you mean about not being about to have a global list of
> symbols because everything is subject to change. But it would be nice to
> have a default list of symbols available depending on the default
> initial setup.

Yes, the default additions to the Jinja2 template context should be
documented, and (as far as I can tell) are not. I've filed
https://code.djangoproject.com/ticket/25804 for this.

> I think the csrf_token and request token are available in the
> django.templates.backend.jinja2 module, as knbk pointed out.
> 
> I still don't think the jinja template processor is able to load
> context_processors. Right now my messages aren't working and I think
> this is the reason.

That's right, the Jinja2 backend doesn't support context processors.
There's an accepted ticket for it
(https://code.djangoproject.com/ticket/24694) and I have an
implementation floating around, just need to polish it up, integrate it
with Django, and add tests and docs.

> Also, I have django debug bar but when I load a jinja2 template with it
> it says that no templates have been loaded and I can't see any context
> processors. I think this is a bug. 

Yes, I believe that the debug-toolbar's support for detecting rendered
templates is DTL-specific (it predates Django's Jinja2 support by quite
a few years). Really this is an enhancement request for
django-debug-toolbar, though it may require better debug hooks in the
Django template rendering system so DDT can hook in at the generic
template-rendering layer rather than the DTL-engine layer.

> Does anyone have an idea about how to tell if django is actually loading
> the context_processors when it loads a jinja template. Also, does anyone
> have an idea how to make messages work with jinja templates?

The simplest solution for now may be to add context-processor support to
the Jinja2 backend, which can be done without too much difficulty by
subclassing it.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/565394A5.4050104%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Reducing redundancy in tests, logged-in vs. not-logged-in

[Carl Meyer]

Hi Tim,

On 11/16/2015 01:35 PM, Tim Chase wrote:
> Are there best practices for writing tests where every case has a
> "not logged in" and a "logged in" pairing?  I'm finding myself writing
> too much redundant test-code where the only difference is whether the
> user is anonymous (and thus the test should fail) or logged in with
> appropriate rights (and thus the test should pass).
> 
> For context, this is being run within Django-WebTest as recommended
> to me by Carl Meyer in case it has features I haven't yet encountered
> that would help me.
> 
> Are there best practices to keep these sorts of tests DRY?

Now I'm going to recommend another testing tool: py.test :-)

I handle this type of situation by using py.test parametrization.
Usually I have a bunch of views that are all login-required, so I just
write a single test to check that a view returns redirect-to-login if
accessed anonymously, and then parametrize that test over all the urls
it should apply to. It looks something like this (parametrizing over
just two views):

@pytest.mark.parametrize(
'urlname,urlkwargs',
[('oneview', {}), ('otherview', {'id': 999})],
)
def test_login_required(client, urlname, urlkwargs):
url = reverse(urlname, kwargs=urlkwargs)
resp = client.get(url)

assert utils.redirects_to(resp) == reverse('accounts_login')

I use url-reversing in my tests; the parametrization would be slightly
simpler if you just used raw URLs. The `client` kwarg is a py.test
fixture that returns a WebTest client instance. `utils.redirects_to` is
just a helper that pulls out the `Location` header from the response,
and strips out everything but the path portion.

HTH,

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/564A414F.4040302%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: following 302s in WebTest form.submit() (was Best practices for writing functional tests to exercise forms?

[Carl Meyer]

Hi Tim,

On 11/13/2015 10:06 AM, Tim Chase wrote:
> On 2015-11-09 12:41, Tim Chase wrote:
>> On 2015-11-06 01:09, Carl Meyer wrote:
>>> [1] https://pypi.python.org/pypi/WebTest
>>> [2] https://pypi.python.org/pypi/django-webtest
>>
>> Just to follow up, django-webtest has been pretty much exactly what
>> I was looking for.  Thanks!
> 
> Monkeying around with my tests, I have the following snippet:
> 
> response = self.app.get(self.login_url)
> login_form = response.forms["login"]
> login_form["username"] = NAME
> login_form["password"] = PASS
> good_response = login_form.submit()
> 
> When I get my good_response back, it's a 302 redirect to "next".  Is
> there any way to get it follow that redirect automatically like
> Django's Client allows for .get(..., follow=True)?  Or do I have to
> manually follow every redirect?  I didn't see anything in the docs
> for WebTest/django-webtest nor did I turn up anything
> obvious-but-undocumented when poking around in the source for a
> couple minutes.

As far as I know, you have to call `.follow()` on the response, which
follows the redirect and returns the next response. Is that what you
mean by "manually follow every redirect"? It's manual in a sense, but
there is a convenience method to make it very easy. You can just change
`login_form.submit()` to `login_form.submit().follow()`, you don't even
need another line of code.

I think this is better than the Django client's `follow=True`, because
it means your test specifies the expected number of redirects, and will
fail if the number of redirects changes unexpectedly. (You can check
this with Django's test client by using the `redirect_chain` attribute,
but usually people don't.)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/56461C5E.30007%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Best practices for writing functional tests to exercise forms?

[Carl Meyer]

Hi Tim,

On 11/04/2015 05:36 PM, Tim Chase wrote:
> Beginning a new project, I'd like to include functional testing in
> addition to my unit tests.  What are recommended best practices for
> exercising form-classes?  I was hoping to do something like
> 
>   c = django.test.Client()
>   results = c.get(page_url)
>   frm = SomeForm()
>   # pseudo-code for desired functional testing follows
>   self.assertFormIn(frm, results) # check the form is used
>   frm["name"] = "Pat Smith"
>   frm["email"] = "psm...@example.com"
>   results = c.post(page_url, frm) # submit the filled-out form
>   # check results for error messages and/or success here
> 
> Is there some piece of Django that I've missed that would facilitate
> higher-level tests like this?

I recommend the WebTest package [1], along with django-webtest [2] to
adapt it for use with Django.

It would replace your use of the Django test client (which isn't a
problem; it's a replacement with more features); you'd create and use a
WebTest client instead. It will parse the HTML of a response and pull
out any forms, allowing you to programmatically "fill out" the form
fields present in the markup and submit them. This allows for
integration tests that will catch errors like forgetting to include a
formset's management form in your template.

I use WebTest rather than the Django test client for all my
request/response integration tests.

Carl

[1] https://pypi.python.org/pypi/WebTest
[2] https://pypi.python.org/pypi/django-webtest

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/563BEFC5.4050508%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: include template which extends something else

[Carl Meyer]

On 10/28/2015 10:25 AM, Sven Wanner wrote:
> Yes, sorry I was confused... No I don't get what you mean. I think thats
> exactly what I tried to do. 
> 
> I have kind of a main.html doing that:
> 
> {% block my-wall %}
> 
> 
> {% for obj in objects %}
> 
> {% include 'wall_item.html' with object=obj %}
> 
> {% endfor %}
> 
> 
> {% endblock %}
> 
> wall_item.html looks like that:
> 
> {% extends "wallitem.html" %}
> 
> {% block wall_item_header %}
> something
> {% endblock %}
> 
> and wallitem.html like that:
> 
> 
> 
> {% block wall_item_header %}
> {% endblock %}
> 
> 

That template structure should work just fine. I just tested here with a
similar set of templates and it worked. So there's something else going
on. Can you be more specific about how it fails? Do you get an error?
Unexpected output in some way?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5630F788.90202%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: include template which extends something else

[Carl Meyer]

On 10/28/2015 10:11 AM, Sven Wanner wrote:
> Ok, it's a pitty, but you're totally right. Using if in my template
> should be also fine. Not of that elegance I planned  ;) , but will
> definitely work, too.

I'm not sure what you mean. I thought I just explained how you can use
`extends` in an included template -- I think that should work just fine
for your use case (and I just tested to verify that it does work).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5630F421.2020906%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: include template which extends something else

[Carl Meyer]

Hi Sven,

On 10/28/2015 09:50 AM, Sven Wanner wrote:
> Thanks for your quick reply. Yes, this is also what I found, so I was
> quite shure that my code will never work. What I try to do is to render
> a list of database objects. But not only on one page but on different
> pages and with slightly different designs in each of the lists. So my
> plan was, to avoid writing individual templates for each list, to create
> one base template for the general appearance of such list items and only
> adapt blocks on each individual list.
> 
> Explained the other way around, how would I render a template.html which
> needs a database object  in a for loop and which extends a base_template
> html?
> 
> Hope I achieved to explain the problem well enough...

If I'm understanding right, I think blocks do work fine for your situation.

What you can't do, as Tim explained, is have blocks in an included
template interact with blocks in the including template. They are two
totally separate renders, with totally separate inheritance hierarchies.
But there's no problem with included templates having their own
independent inheritance hierarchy, with blocks.

So you can have an `item_base.html` that contains `{% block someblock
%}{% endblock %}`, and then define `item_foo.html` that `{% extends
"item_base.html" %}` and has `{% block someblock %}foo content{%
endblock %}`, and you can have a `main.html` that has `{% include
"item_foo.html" %}`, and all that should work fine -- the included
portion should display "foo content".

What you can't do is expect the `{% block someblock %}` in
`item_base.html` or `item_foo.html` to interact in any way with a `{%
block someblock %}` in `main.html` (or its parent). The included
template and the including template have separate and unrelated
inheritance and blocks.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5630F1F6.5090100%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: How to create a global management command?

[Carl Meyer]

Hi,

On 10/27/2015 03:40 PM, bwv549 wrote:
> I have some commands I'd like to include in the manage command (i.e.,
> like 'coverage').  How to make a global command?  (I've written many
> commands for apps, but wonder how to make one that isn't tied to any app).

You can't. Management commands are always tied to an app.

That said, many people have an overly-narrow view of what an "app" must
be. A Django "app" can be any Python package at all that you include a
reference to in your INSTALLED_APPS setting. It doesn't need to have
models, or urls, or views. For instance, I often list my top-level
project package in INSTALLED_APPS as an "app", just so I can put e.g.
management commands or template tags there, if they are really
project-wide utilities that don't naturally fit in any sub-package/app.

Note of course that which app contains a management command has no
practical effect on its use (except when it comes to overriding
management commands and the ordering of `INSTALLED_APPS`).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/562FF06B.5000504%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: duplicated selenium test fails

[Carl Meyer]

On 10/21/2015 02:55 AM, Ulrich Laube wrote:
> Am 21.10.2015 um 00:18 schrieb Carl Meyer:
>> On 10/19/2015 03:03 AM, Ulrich Laube wrote:
>>> It boils down to this:
>>> [...]
>>> Is it a known problem?
>>
>> Not really a problem - you just need to be aware that you can't rely on
>> consistent database IDs in tests.
> 
> So there is no test isolation where I assumed it to be.

There is test isolation, it just doesn't extend to sequences. Tables are
cleared of all data in between tests, so your second test still sees
only a single Post in the database, it's just a Post with ID 2 instead
of ID 1.

> My assumption was, that each
> 
> class younameit(TestCase)
> 
> gets its own fresh database to work with,

A completely fresh database would be the ideal situation from an
isolation standpoint, but it is prohibitively slow to actually create a
new test database for each test or test case.

The actual behavior is pretty close to "a fresh database for each test",
though -- ID numbering is really the only exception you're likely to
commonly see.

Really since auto ID numbers are chosen by the database, not your code,
it's better practice for your tests not to rely on them regardless.

> so that all
> 
> def test_foobar(self):
> 
> within, would work against the same DB.

No, the isolation is per test method, not per test class. Every single
test method effectively runs in a clean database (modulo the fact that
sequences aren't reset).

The details of how this isolation is implemented vary by the type of
test. For normal TestCases, each test method runs within a transaction,
and the test runner rolls back the transaction at the end of each test
so none of its changes survive. This is the fastest method, but it means
that the code under test cannot start or commit/rollback transactions.

For tests which may need to test transaction behavior, we have
TransactionTestCase (which LiveServerTestCase inherits from); in this
case the isolation is implemented by truncating all tables after each
test. This is slower, but doesn't interfere with transactions in the
code under test.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5627A0A3.60902%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: duplicated selenium test fails

[Carl Meyer]

Hi Ulrich,

On 10/19/2015 03:03 AM, Ulrich Laube wrote:
> It boils down to this:
> 
> I have a selenium test in a django project against the admin page. All
> green.
> If I duplicate the test class via copy and paste + renaming, then one is
> green and one is red, despite containing the same code. I expected both
> to be green.
[snip]
> from django.contrib.auth.models import User
> from django.contrib.staticfiles.testing import StaticLiveServerTestCase
> 
> from blog.models import Post
> 
> from selenium import webdriver
> 
> class AdminTest(StaticLiveServerTestCase):
> def login_to_django_admin(self):
> self.credentials = {
> 'username': 'admin',
> 'email': 'admin@localhost',
> 'password': 'admin'
> }
> User.objects.create_superuser(**self.credentials)
> # go to the admin site
> self.browser.get(self.live_server_url + '/admin/')
> # we are not logged in, it redirects us to the login page
> self.assertEquals(self.browser.current_url, self.live_server_url
> + '/admin/login/?next=/admin/')
> 
> # login in
> inputbox = self.browser.find_element_by_id('id_username')
> inputbox.send_keys('admin')
> inputbox = self.browser.find_element_by_id('id_password')
> inputbox.send_keys('admin')
> submit = 'input[type="submit"]'
> submit = self.browser.find_element_by_css_selector(submit)
> submit.click()
> self.browser.find_element_by_class_name('app-blog')
> 
> def test_insert_Post_via_django_admin(self):
> self.browser = webdriver.Firefox()
> self.login_to_django_admin()
> 
> # now that we are logged in, look at the Post model add page to
> get a csrftoken
> self.browser.get(self.live_server_url + '/admin/blog/post/add/')
> inputbox = self.browser.find_element_by_id('id_text')
> inputbox.send_keys('barfoo')
> submit = 'input[type="submit"][name="_save"]'
> submit = self.browser.find_element_by_css_selector(submit)
> submit.click()
> 
> # we should see it on the other side via python db access
> saved_beiträge = Post.objects.all()
> self.assertEquals(saved_beiträge.count(), 1)
> self.assertEquals(saved_beiträge[0].text, 'barfoo')
> 
> # try to look at it
> self.browser.get(self.live_server_url + '/admin/blog/post/1/')

This line is the problem. It hardcodes the ID 1 in the URL. The
database-isolation method used by Django tests does not restore ID
sequences to a fixed value, so you cannot ever rely on fixed ID values.
In your case, when you only have the one test, it always creates the
first Post in the database, with ID 1; when you copy-paste the test, the
second test creates a Post with ID 2 instead, breaking this line. You
should use the actual ID value (you query the object just above, so you
have it here) instead of hardcoding `1`.

> # modify it
> inputbox = self.browser.find_element_by_id('id_text')
> inputbox.send_keys('text')
> submit = 'input[type="submit"][name="_save"]'
> submit = self.browser.find_element_by_css_selector(submit)
> submit.click()
> 
> # we should see the change at the other end
> saved_posts = Post.objects.all()
> self.assertEquals(saved_posts.count(), 1)
> self.assertEquals(saved_posts[0].text, 'barfootext')
> 
> Now run it via:
> 
>> manage test
> 
> Selenium opens Firefox and walks through the admin page as it should.
> Test green.
> Now duplicate the whole class and rename it and run the tests again.
> 
>> manage test
> 
> One is green the other is not. In fact Selenium opens a second Firefox
> instance as it should. But this time
> 
>  self.browser.get(self.live_server_url + '/admin/blog/post/1/')
> 
> results in a 404 not found.

Right, because in this test there is no Post with ID 1, just one with ID 2.

> Now delete the whole class that is passing and run the test again.
> 
>> manage test
> 
> Now the failing one is green as well.

Yes, because now you're back down to just one test, so it is again
creating a Post with ID 1.

> I can't wrap my head around this. What am I missing?
> Is it a Django issue or a Selenium issue?
> Is it a known problem?

Not really a problem - you just need to be aware that you can't rely on
consistent database IDs in tests.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5626BDA7.2050803%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: 

Re: Using South in Django 1.7+

[Carl Meyer]

Hi Remco,

On 10/08/2015 04:26 AM, Remco Gerlich wrote:
[snip]
> Can South run on Django 1.7+ at all?

Nope. At least not last I tried, and I don't think anyone has done
anything on it since. It looked like it would need some major updating,
since South integrates closely with the database backends and they
changed in a big way in 1.7.

> If not, maybe we have to keep some 1.6 installation running somewhere on
> the backend to upgrade a file to the current situation, and then apply
> new migrations from there...

Yes, I think that's probably your best bet. You could try your hand at
porting South to 1.7, but I wouldn't expect it to be easy.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/56167ABB.1030102%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: How to run all tests in INSTALLED_APPS with the new test runner 1.6?

[Carl Meyer]

On 10/07/2015 06:57 AM, jordi.gutierrez.herm...@ecometrica.org wrote:
> Okay, I'll try grabbing the old test runner from git, thanks.

That should work fine.

You should be aware, though, that specifically in the case of the
django.contrib app tests, we make no promise that they will pass under
your project's configuration, and pull requests that complexify the
tests solely to make them pass under a wider variety of configurations
will not be accepted. It is our policy that they should pass when run as
part of the Django test suite, but not necessarily in an arbitrary project.

You may find that a similar de facto policy exists for an increasing
number of third-party apps' test suites, too.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/56152907.1040400%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Invalid HTTP_HOST, can someone explain why I am getting this?

[Carl Meyer]

On 09/21/2015 04:22 PM, François Schiettecatte wrote:
> Not likely, all that is happening is that you are getting requests
> where the ‘Host:’ HTTP header is not set or set to something other
> than what is accepted by your site. Most likely a buggy client. I get
> that all the time, I just ignore it.
> 
> Cheers
> 
> François
> 
>> On Sep 21, 2015, at 6:16 PM, frocco  wrote:
>> 
>> I am still getting this invalid host from time to time. Does this
>> mean that someone is trying to hack my site?
>> 
>> www.g3suprimentos.com.br is not anything I own.
>> 
>> For now, I am just ignoring this.

The best way to solve this for good and never get those errors again is
to fix it in your front-end webserver configuration, so that it ignores
requests for the wrong Host and doesn't even pass them on to Django in
the first place. If you're using nginx that means setting your
`server_name` directive correctly. If you're using Apache that means
using a name-based (non-default) VirtualHost. Any webserver should
provide some way to do this.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/56008402.70309%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Oracle XMLType and AES_Encrypt() field

[Carl Meyer]

On 09/15/2015 12:44 PM, Dan Davis wrote:
> I'm wondering how I would get python to run an interactive shell similar
> to what I'd get from running ./manage.py shell.

Why not just run `./manage.py shell`?

In general, `DJANGO_SETTINGS_MODULE=myproject.settings python` is
effectively equivalent to `./manage.py shell` (except for the latter
will use IPython by default if installed).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55F867F0.7070005%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Oracle XMLType and AES_Encrypt() field

[Carl Meyer]

Hi Dan,

On 09/11/2015 09:39 PM, Dan Davis wrote:
...
> For both my EncryptedTextField and XMLTypeField, I want to to alter the
> SQL that will be generated to query, insert, and update the field so
> that XMLType field will be converted to a CLOB or text value with
> .getClobValue() function.   Similarly, I want the value to save to be
> generated from calling AES_Encrypt() with the project's secret key and
> to be decrypted by calling AES_Decrypt().   The benefit of using the
> database encryption/decryption functions should be clear enough -
> encryption and decryption is offloaded from the application, and the
> programmer is assured of being able to see the values by constructing a
> simple view.
> 
> May I alter the way a query is constructed using a custom Field class in
> this way?If so, can anyone provide an example?

Yes, this is possible, and not even that hard (at least in Django 1.8,
where Field classes gained more control over SQL generation; I suspect
it would be quite a bit more difficult if you need to support older
versions). It does require exploiting some undocumented internal APIs.
For an example, you can look at my django-pgcrypto-expressions project,
which does the same thing for Postgres' built-in symmetric encryption
functions: https://github.com/orcasgit/django-pgcrypto-expressions

You should be aware, though, that depending on your threat model,
encrypting/decrypting in the database can be significantly more risky.
It potentially exposes your encryption key in several additional places:
on the wire between the application server and the database server and
in database server query logs or activity monitors. If your threat model
includes the possibility of an attacker with access to the running
database server (who may not have gained access to the app server),
encryption on the application side is better than on the database side.

It's possible to make application-side encryption in Django almost as
transparent to the developer as database-side encryption; see
django-fernet-fields: https://github.com/orcasgit/django-fernet-fields

The main difference is that with application-side encryption you can no
longer perform lookups or ordering in the database against the encrypted
field. But this isn't really much of a difference, because even with
database-side encryption, lookups/ordering on an encrypted field cannot
be indexed, meaning they won't scale to large table sizes anyway.

HTH,

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55F46D5A.1060706%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django for data management in a no-UI application?

[Carl Meyer]

On 09/08/2015 04:06 PM, jsa...@nvidia.com wrote:
> I have been planning to develop a standard web application using Django.
> Due to a change in direction, the development plan has been changed to:
> 
>  1. Develop a suite of no-GUI applications that run in a command line
> window on a server. One other application will run on _clients _in a
> command line window, and will perform certain operations on the
> server through a REST API service. However,
>  2. All of the applications will use an SQL database that runs on the
> server.
>  3. In a later phase of the project, the server applications will be
> combined and modified to present a GUI.
> 
> My question is this: Is it feasible to make all of the server
> applications use Django to communicate with the database server in step
> 1, and then use it to implement the GUI in step 3? By doing so, I would
> avoid having to (1) code all of the database operations without Django
> in step 1, then recode them with Django in step 3, or (2) end up with a
> "hybrid" application that implements the GUI in Django but the database
> operations outside Django.

Sure, you can write the command-line server applications as Django
management commands, or just as Python scripts that use
`django.conf.settings.configure()` and call `django.setup()` before
using the ORM to communicate with the database.

And you can use the same ORM models to drive your REST API, built in Django.

How you approach step 3 depends on whether "browser-based GUI" is
acceptable, or you need to build an OS-native application. Django is
oriented towards the former, but you can certainly build an OS-native
Python application using a GUI toolkit like Qt or whatever, and still
use the Django ORM within it to talk to the database.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55EF5D03.4020503%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django 1.7 migrations: Adding one field to User`s model

[Carl Meyer]

On 08/26/2015 12:18 PM, Marcin Nowak wrote:
> 
> On 26 August 2015 at 20:09, Hugo Osvaldo Barrera  > wrote:
> 
> Oh, so the field you're adding in monkey patching contrib.auth.User?
> IMHO, the design problem isn't django, but rather monkey-patching
> models itself. Maybe you'd be better of using a custom user model.
> 
> 
> I've tried to use custom user model, but Django forces me to change all
> FK`s. This is not possible for now.
> 
> I need to add just one column / field to existing User model. Nothing
> more, nothing less.
> 
> AFAIK I should create profile for user and put this column into profile
> model, but there are two disadvantages: 
> 
>   * the profile is not necessary now and will contain just one column
> (well, three when incl. PK and FK to user)
>   * performance issues (joining table)
> 
> I think that for this use case "Custom user" will not work. I was wrong.
> I must forget about "Custom user". ;)

Yes. I think the most important Django limitation here is how difficult
it is to switch from built-in User to custom User. That's a real
problem, and it should be fixed (but at the moment I'm not sure how to
fix it).

Better support for monkeypatching is not very interesting.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55DE03E6.9090300%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Turn off migrations completely in Django 1.7

[Carl Meyer]

On 08/26/2015 08:15 AM, Carl Meyer wrote:
> I agree, and I already suggested a new feature (`managed=False` at the
> AppConfig level) for which I think a solid patch would most likely be
> accepted. This new feature would be capable of handling the "don't
> migrate anything" case as well as the wide variety of "migrate these
> apps but not those" cases. 

Another option might be to leverage the existing `MIGRATION_MODULES`
setting, allowing a value of `None` to mean "ignore migrations for this
app, I will manage its schema myself outside of Django."

At this point the discussion belongs on the django-developers list, to
enumerate the real-world use cases and sort out the pros and cons of the
various possibilities. Getting a change into Django is rarely
impossible, but it does require committing some time and energy to
discussion and evaluation of the options.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55DDD23D.5030301%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Turn off migrations completely in Django 1.7

[Carl Meyer]

Hi Grzegorz,

> On Aug 26, 2015, at 2:25 AM, Grzegorz Stalmierski 
>  wrote:
> 
> W dniu 25.08.2015 o 19:04, Marcin Nowak pisze:
>> 
>> I would like to provide solid patch as a pull request, but I'm afraid that 
>> it will be rejected due to some kind of spirit, politics or something else. 
>> 
>> BR,
>> Marcin
> Marcin, please - consider sharing your patch on GitHub (or similar)
> with a documentation, because:
> It is not possible to build a framework where _everything_ is easy. We
> aim for "Simple things should be easy, advanced things should be possible."
> 
> Carl
> this is not so advanced thing, and it looks almost 
> impossible to solve in Django. 
> With current architecture around migrations Django 
> is database owner, not one of the database services.
> 

I agree, and I already suggested a new feature (`managed=False` at the 
AppConfig level) for which I think a solid patch would most likely be accepted. 
This new feature would be capable of handling the "don't migrate anything" case 
as well as the wide variety of "migrate these apps but not those" cases. 

You quoted me out of context. My quote was in reply to Marcin complaining that 
even _that_ feature would still be too much work to use. I wasn't claiming the 
status quo is sufficient for this. 

FWIW I don't think a single new setting to switch off migrations for all apps 
is 100% out of the question, if it turns out there are a number of people who 
would use it. I'd like to hear a little more specifics from such people about 
their use case. I don't think a global flag is as generally useful as the above 
feature, and it would need to come with some pretty strong warnings in the 
docs. For instance, Django may modify models in contrib apps and ship 
migrations for them in future versions, and of course third party apps will 
commonly do the same. So anyone using such a global flag is accepting 
responsibility for watching out for such schema changes themselves, or their 
project may break when they upgrade Django or another app. 

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5770D7A2-FDEC-41C2-BA60-674CE7E974DA%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.

Re: Migrating old users into custom user model

[Carl Meyer]

Hi Ben,

On 08/25/2015 05:14 PM, Ben Wattie wrote:
> Thanks for your reply.
> If I figure it out I would surely submit doc, but it's more likely that
> I will just come up with a workaround.

Fair enough :-)

> Starting a project with a custom user model every time just to avoid
> this doesn't feel so great since the documentation just says things like
>  "probably don't use a custom user model, even though its supported, use
> 1-1 profile instead"
> But then later on the track requirements can change and you need a
> custom model but its very difficult to migrate.

And that's why the docs are simply wrong when they say that, and should
be changed. See https://code.djangoproject.com/ticket/24370

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55DCF751.9020204%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrating old users into custom user model

[Carl Meyer]

Hi Ben,

On 08/25/2015 03:33 PM, Ben Wattie wrote:
> I have an existing project with migrations, datamigrations and I need a
> custom user model now, not just 1-1 profile so I created one.

Unfortunately, while this is technically possible (I haven't done it
myself, but I know at least one person who says they have), it's very
complex and requires a fair amount of familiarity with the migrations
system and use of the advanced SeparateDatabaseAndState operation. And
nobody, AFAIK, has written documentation on how to do it.

I don't believe this is really an acceptable state of affairs, so I've
filed https://code.djangoproject.com/ticket/25313 to at least clearly
acknowledge that it _ought_ to be fixed. I don't know when I'll get a
chance to work on it myself (not real likely unless I revisit an old
project, because nowadays I _always_ start every project with a custom
User model, to avoid this problem). Maybe the ideal scenario here is
that you figure it out and then write the docs on how it's done :-)

Sorry I can't give you a better answer,

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55DCE528.5060605%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Turn off migrations completely in Django 1.7

[Carl Meyer]

On 08/25/2015 09:23 AM, Marcin Nowak wrote:
> On 25 August 2015 at 17:04, Carl Meyer <c...@oddbird.net
> <mailto:c...@oddbird.net>> wrote:
> 
> Hmm, yes, third-party apps are an issue.
> 
> My recollection from the last time you brought this up is that we
> decided an AppConfig-level setting to do the equivalent of managed=False
> for an entire app would be a reasonable feature request. Since you can
> provide your own AppConfig for third-party apps, this would cover that
> use case, too.
> 
> 
> That would be a better than nothing and it's flexible.
> 
> But in the most of my cases I will need to provide app configs for all
> apps. This can mean a lot of unnecessary work.
> Something like 'disable all' is simplest solution, but naturally limited.
> 
> Maybe there is a solution for both?

Sometimes when you have unusual requirements you have to do a little
more work to meet them. To my knowledge, out of the many hundreds of
thousands of Django users, you are the first and only one to request a
way to turn off migrations entirely. So I don't think there is a very
strong need to make that easy, when we can add a more flexible feature
that makes it possible.

It is not possible to build a framework where _everything_ is easy. We
aim for "Simple things should be easy, advanced things should be possible."

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55DC8971.9030308%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Turn off migrations completely in Django 1.7

[Carl Meyer]

On 08/25/2015 09:02 AM, Marcin Nowak wrote:
> 
> On 25 August 2015 at 16:53, Carl Meyer <c...@oddbird.net
> <mailto:c...@oddbird.net>> wrote:
> 
> I thought we already covered this topic, and `managed=False` should work
> for your use case. That's the exact meaning of this flag: "I don't want
> Django to manage my database, please never touch it." Have you tried
> adding that flag to your models? Does something about that not work
> for you?
> 
> 
> 
> Managed=False is not same. I.e. I can't set managed=False for 3rd party
> apps (including `django.contrib`)
> But to be honest I didn't tried iterate over all INSTALLED_APPS to patch
> meta options...

Hmm, yes, third-party apps are an issue.

My recollection from the last time you brought this up is that we
decided an AppConfig-level setting to do the equivalent of managed=False
for an entire app would be a reasonable feature request. Since you can
provide your own AppConfig for third-party apps, this would cover that
use case, too.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55DC841A.4070206%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Turn off migrations completely in Django 1.7

[Carl Meyer]

On 08/25/2015 08:42 AM, Marcin Nowak wrote:
> 
> On 25 August 2015 at 16:21, Tim Graham  > wrote:
> 
> What does "turning off migrations" mean in practice? Is it not
> enough to avoid the makemigrations and migrate management commands?
> 
> 
> 
> Currently I'm replacing mgmt commands to avoid accidential calls, and
> also replacing test runner.  But, for example, `runserver` complains
> about unapplied migrations (which is not true), and I don't know what
> else and when other db alterations can be executed. 
> 
> "Global turn off" should give us assurance that nothing would be changed
> in db automatically.  

I thought we already covered this topic, and `managed=False` should work
for your use case. That's the exact meaning of this flag: "I don't want
Django to manage my database, please never touch it." Have you tried
adding that flag to your models? Does something about that not work for you?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55DC817D.6070905%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Architectural suggestions for (Django + API)

[Carl Meyer]

Hi Julio,

On 08/16/2015 04:59 PM, julio.lace...@m2agro.com.br wrote:
> I created a project using Django (let's call this Project A) and now I
> decided to start a new one but this time it is an API using Django REST.
> So, for now I have to refactor Project A to consume data from API, but
> in order to do that I need to duplicate all my models files in both
> projects. The question is how can I avoid this? Is there another way to
> do that?

I don't quite understand why the situation you describe requires
duplicating models files. Is the canonical data storage now in Project B
instead of Project A, and Project A gets the data via Project B's REST
API? In that case, why does Project A need those models anymore?

> I see that /moving models /from Project A to API as something natural
> but to have an empty models file in Project A I need to change some
> concepts, for removing ModelForms and put all validation logic using Form.

Well, yes, but it seems to me that those types of changes in the Project
A codebase will be required regardless, since it now wants to save the
data via a REST API, not to its own database. How does a ModelForm help
with that?

> How do you guys see this? Would you recommend something different? Is
> that a good idea?
> 
>  1. Project A (Models) <---> API (same models??? How can I handle that?)
>  2. Project A (No models) <---> API (Models)

The latter. I don't understand how the first option would even work.

But if you _do_ want to share some models between two projects, for
whatever reason, you can do it the same way you'd share any other code:
put the models in a reusable app and install it in both projects.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55D16270.8010506%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: How to use Django 1.8.2 with Python 2.7?

[Carl Meyer]

Hi,

On 08/11/2015 11:56 AM, jsa...@nvidia.com wrote:
> I'm studying the Django Project tutorial
>  using Python
> 2.7 (because that's my department's current standard) and Django 1.8.2
> (because that's the current stable version).
> 
> The tutorial says, "If you are still using Python 2.7, you will need to
> adjust the code samples slightly, as described in comments." What
> comments does it mean?

Python code comments, in the code samples that need adjusting. The first
example I see is in the `polls/models.py` code sample in this section:
https://docs.djangoproject.com/en/1.8/intro/tutorial01/#playing-with-the-api

See the "__unicode__ on Python 2" comments?

> The section "Playing with the API" instructs me to run the command:
> 
> $ python manage.py shell
> or
 import django
 django.setup()
> 
> Being a methodical sort, I tried both. The first works. The second gives
> me a bunch of errors, beginning with:
> 
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "c:\Python27\lib\site-packages\django\__init__.py", line 17, in setup
> configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
> 
> I think it's complaining about an incompatibility between Python 2.7 and
> Python 3.2.

Can't say without seeing more of the traceback. I doubt it though, since
this only executes code in Django itself, which is fully compatible with
both Python 2 and 3.

> I can evade the immediate problem by using "python manage.py shell"
> instead, but I expect further problems in short order, and I need to
> know how to fix them.
> 
> So, where are these "comments" that tell me what to do? I don't see any
> in that part of the tutorial. I opened __init__.py, and there are no
> helpful comments there, either.

You don't need to adjust anything at all in Django itself to adapt to
Python 2; Django fully supports both 2 and 3. You just need to make sure
that the code you write yourself is Python 2 compatible. Where the
tutorial tells you to type some code in a file, it gives the Python 3
version, but has comments showing what needs to be adjusted for Python
2. There's nothing special you need to do until you get to those code
samples.

> I could figure this out myself, but I can't see debugging my way through
> the entire Django codebase. I'm being paid to write an application, not
> fix up Django.

You don't need to fix up Django.

> Either I'm missing the comments that the tutorial promised me, or it has
> left me in the lurch. Can someone explain, please?

Just continue with the tutorial, and read the code samples (including
comments) carefully, when you reach them.

Your issue when running `django.setup()` is probably entirely unrelated,
but in order to debug it we'd need to see the full traceback.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55CA31AA.4040707%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Question about squashing django migrations

[Carl Meyer]

Hi Richard,

On 08/01/2015 06:27 PM, Richard Brockie wrote:
> Can someone please explain the implications of this note about squashing
> migrations in the documentation?
> 
> From: https://docs.djangoproject.com/en/1.7/topics/migrations/
> 
> "Once you’ve squashed a migration, you should not then re-squash
> that squashed migration until you have fully transitioned it to a
> normal migration."
> 
> 
>  Is there a transition command to convert a squashed migration into a
> normal migration, or how should I go about this?

The transition steps are described immediately above the note you
quoted. See the two steps listed right after the text "you must then
transition the squashed migration to a normal initial migration, by:"

> Here is my current migration situation:
> 
> Migrations:
> 
>   * 0001_initial.py
>   * ...
>   * 0052_auto_20150711_1407.py
> 
> have been squashed into:
> 
>   * 0001_squashed_0052_auto_20150711_1407.py
> 
> Since then, I have created further migrations:
> 
>   * 0002_auto_20150731_0651.py
>   * 0003_auto_20150731_0659.py
> 
> I was a little surprised that the numbering continued from 0002 and not
> 0053. I am concerned that I will become confused with the overlapping
> numbering before/after the squash so I would like to streamline things a
> little. I'm ok to remove the first set of migrations 0001 to 0052 as
> they are correctly captured by the first squash.
> 
> What are your recommendations in a case like this?

To go ahead and remove the old migrations, along with the `replaces`
setting on the new squashed migration, as recommended in the docs. Just
first make sure every deployment of your project has run migrate since
the squash migration was created.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55C8E971.9050803%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Different auth user models withoout doing multi-table inheritance

[Carl Meyer]

Hello Ankit,

On 08/10/2015 09:38 AM, Ankit Agrawal wrote:
> I am working on a project which has two different sets of users -
> |Customer| and |Merchant|. Both of these users should be able to
> register and login to their respective profiles. The most obvious choice
> to implement this that came to my mind was to make two different models
> |Customer| and |Merchant| that inherit from a BaseUser model that will
> store the common fields i.e. Multi-table inheritance -
> https://docs.djangoproject.com/en/1.8/topics/db/models/#multi-table-inheritance
> 
> 
> Quoting |Two Scoops of Django| -
> 
> 
> |At all costs, everyone should avoid multi-table inheritance (see warning 
> above) since it adds both confusion and substantial overhead...
> Adds substantial overhead since each query on a child table requires joins 
> with all parent tables.
> 
> |
> 
> I would like to know if and why having an explicit |OneToOneField| is
> better than Multi-table inheritance.

In terms of the underlying database schema (and thus query efficiency),
a OneToOneField is exactly the same as MTI (MTI uses a OneToOneField
internally).

Personally, I agree with the _Two Scoops_ authors that the explicit
OneToOneField is preferable, because it means the ORM objects more
closely map to the actual database schema, making it easier to
understand what is actually happening at the database level. An explicit
OneToOneField also gives you more flexibility in several ways - you can
create / delete the base User instance and the linked CustomerProfile /
MerchantProfile instances separately from each other, and you can even
have a single User instance with both a CustomerProfile and a
MerchantProfile (though that may not be useful for your case).

> Also, are there any other better
> ways to model the above relationship?

That depends on how much different information you need to store about
customers vs merchants. Assuming it's significant, I think a common User
model with OneToOne-linked CustomerProfile and MerchantProfile is
probably best (I'm doing the same thing in my current project in a
similar situation). I also recommend having a `role` or similar field on
the User model, with 'merchant' and 'customer' as choices. This means
you can know what type of user a given user is while querying only the
User table (so you know which profile table to query for that user), and
it also means that you can have users with a known role who haven't
created their profile yet (this may or may not be useful for your
application, depending how/when profiles are created).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55C8D571.8010102%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: "RuntimeError: Error creating new content types."

[Carl Meyer]

On 07/26/2015 10:46 AM, Tobias McNulty wrote:
> I just ran into this issue as well when updating an old project to
> Django 1.8. The only suitable workaround I found was to update first to
> Django 1.7 and fake the initial migrations for my apps (presumably
> Django's as well, though it seems to handle that automatically). Only
> then can you safely update to Django 1.8. Otherwise, there is no way to
> fake only the initial migrations for the auth and contenttypes apps if
> you've already upgraded to Django 1.8 (perhaps there should be?).

I believe the way to do this is the `--fake-initial` flag to `manage.py
migrate`. It's mentioned in the migrations docs, as well as the 1.8
release notes.

In 1.7, the behavior enabled by `--fake-initial` (automatically fake any
migration where it looks like its tables already exist) was always on,
but we felt that was a bit too implicit and magical. Especially
considering that "tables already exist" is a very rough heuristic for
"is the initial migration already applied", which silently fails to
catch any kind of mismatch at the field/column level. So in 1.8 we
require the use of that heuristic to be explicitly requested.

`--fake-initial` should work equally well for your own apps or Django
contrib apps. Did you try it and it didn't work? Or were you not aware
of it? If the latter, is there someplace it ought to be documented but
isn't?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55B65BA9.5080709%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Database inserts with autocommit set to off.

[Carl Meyer]

Hi Maciej,

On 07/27/2015 07:03 AM, Maciej Gol wrote:
> I've been recently working (porting to Django 1.8) on a project where we
> need to use a few transactions in a single request, and these
> transactions cannot be correctly handled by the `atomic` decorator due
> to functions calls nesting. Basically, we are sending celery tasks at
> the end of some processing, and it requires the results to be visible in
> the database. I'm doing this for each element of a list, thus the celery
> task sending is done right after saving the data to the database. The
> commit should happen between the save and posting the task, and since
> the processing logic is complex, I can't use a decorator here.

I think a better solution to this situation is to use
transaction.atomic, and then use django-transaction-hooks [1] to delay
creation of the Celery task(s) until the transaction is successfully
committed. (Django 1.9 will have transaction-hooks integrated into core.)

 [1] http://django-transaction-hooks.readthedocs.org/en/latest/

> The issue is, when autocommit is set to off, whenever I try to
> `.update()` a `QuerySet` or `.save()` a `Model`, it results in
> `TransactionManagementError: The outermost 'atomic' block cannot use
> savepoint = False when autocommit is off.` error, which is kind of sad
> because I could handle the eventual rollback myself gracefully. Instead,
> django throws me this error in the face.

Yes, this is a known issue in 1.8:
https://code.djangoproject.com/ticket/24921

The ticket describes the needed solution in some detail, it just remains
for someone to code up the patch with a test. Since the issue is a
regression in 1.8, I think such a patch would be backported to the 1.8
branch and appear in the next 1.8.x release.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55B652E4.2040904%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrate a many-to-many field to an explicit intermediate ("through") model?

[Carl Meyer]

Hi Gergely,

On 07/23/2015 02:27 PM, Gergely Polonkai wrote:
> Yes, you are right, my attempt is not the solution to your problem; it
> seems that this m2m field really cannot be modified like this. With some
> slight modifications, though, it may be.
> 
> 1) create the through table
> 2) migrate data with RunPython — if you want to be portable, stay with
> RunPython instead of RunSQL

Yes, this is true. Don't use RunSQL if you want your code to be portable
between database backends. (For most of my non-reusable-apps code, I
take great advantage of Postgres-specific features and don't care at all
about cross-database portability.)

> 3) delete the ManyToManyField
> 4) recreate the ManyToManyField with the through option
> 
> All this can (and I think, should) go into the same migration file.

Hmm. I'm almost sure I've had problems in the past with trying to do
schema alterations and data migration in the same migration file (thus
same transaction), but with a simple test just now it seems to work OK
on Postgres. I'll have to start trying to combine migrations like this
more often and see how it goes.

> Meanwhile it might worth a question to the devs (or a more thorough
> search on the topic) to understand why you can't switch from a simple
> m2m field to one with a through option.

Just because it's a bit complex to implement, and nobody has implemented
it yet.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55B15070.8000707%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Allow users to submit django form once per day

[Carl Meyer]

On 07/23/2015 01:32 PM, Nkansah Rexford wrote:
> You can save the datetime of the last form submission and check
> whether its oneday or not in the view. If its one day then show pass
> the form or else dont.
> 
> Hope this helps.
> 
> 
> I do know I have to save the date stamp, but the question is Where? On
> the model class, the user class, or in a text file?

The database. Either a new field on a user or profile model, or on the
model the form is for, or a new model specifically for this purpose.
Hard to say with the info you've given so far which of those makes most
sense for your case, any of them could work. (E.g. "on the model the
form is for" only makes sense if that model is "owned" by a single user,
not edited by multiple users.)

Saving persistent data on a Python class (if that's what you meant by
"the model class" or "the user class") doesn't work; you'll lose the
data if the server restarts.

Text file would work, but that's a lot of hassle to no good end when you
have the database available, whose entire purpose is to store persistent
data for you.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55B14265.9000200%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Why is Admin login template at admin/login.html instead of registration/login.html?

[Carl Meyer]

Hi Carsten,

On 07/23/2015 11:41 AM, Carsten Fuchs wrote:
> Am 23.07.2015 um 18:49 schrieb Tim Graham:
>> The admin login only allows staff users to login so it makes sense to
>> me that if you
>> wanted to add regular user login to your site, it should have separate
>> URLs.
> 
> I think what confuses me is the fact that (in the Auth app) there is
> only one User model, and the only difference between regular and staff
> users is the User.is_staff flag.
> 
> For example, if a staff user logs out of the Admin, he is logged out as
> a regular user as well. If a regular user logs in via a custom login
> page, then browses to an Admin page, some kind of error report or
> redirect must occur.
> 
> Given this, authentication is like a user-centric, site-wide feature
> rather than an app-specific one, isn't it?

Sure, the logged-in status of a given session is site-wide. But that
doesn't imply that there must be only a single login page, that always
looks the same and behaves the same. Normally on a Django site (with the
defaults) you'd have an admin login page which only allows staff users
to log in, and redirects them by default to the admin post-login, and is
styled to look like the admin. And you'd have a public login page which
allows any user to login, and redirects them by default to somewhere
else (not the admin) post-login.

There's no contradiction between having two (or more!) such login pages,
and the fact that once a user logs in with either of those login pages,
they are logged in to the whole site.

Of course it's _possible_ to have just a single login page instead, if
you want that, but it's not at all clear to me that that's better. I
prefer to keep the admin relatively separate from the public site.

And I think the same is true for password-reset etc. I'd prefer to leave
the admin with its own pages, styled consistently with the rest of the
admin, and design my own pages for public users, consistent with the
design of the rest of the public site.

>> As for the template issue, it seems to me the
>> admin/template/registration templates
>> should be more like admin/login.html and namespaced under admin so
>> that if you want to
>> implement a non-admin password reset, you don't have a conflict in the
>> template names
>> (see the ticket below for an example).
>>
>> https://code.djangoproject.com/ticket/20372
>>
> 
> Well, I am quite happy about the admin using the registration/...
> templates by default: With the view that authentication is user- rather
> than app-specific, I recently made my admin and regular logins look
> identical, which worked very well.
> So ticket 20372 is quite the opposite of my view.  ;-)

Presuming we made the admin use its own templates for all of this, you
could achieve what you want by also overriding the admin templates and
just making them inherit everything from your templates. A tiny bit of
boilerplate, but not much.

I think the preferable default is to have the admin separate.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55B12D5A.50409%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Why is Admin login template at admin/login.html instead of registration/login.html?

[Carl Meyer]

On 07/23/2015 10:49 AM, Tim Graham wrote:
> The admin login only allows staff users to login so it makes sense to me
> that if you wanted to add regular user login to your site, it should
> have separate URLs.
> 
> As for the template issue, it seems to me the
> admin/template/registration templates should be more like
> admin/login.html and namespaced under admin so that if you want to
> implement a non-admin password reset, you don't have a conflict in the
> template names (see the ticket below for an example).
> 
> https://code.djangoproject.com/ticket/20372

I agree. The auth views expose the option to pass a different template
name. I think the admin should have its own templates namespaced under
`admin/` and pass those template names to the views, rather than
overriding the default auth templates.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55B12345.1060908%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrate a many-to-many field to an explicit intermediate ("through") model?

[Carl Meyer]

Hi Carsten,

On Thursday, July 16, 2015 at 12:44:55 PM UTC-6, Carsten Fuchs wrote:
>
> Am 2015-07-16 um 18:15 schrieb Gergely Polonkai: 
> > 1) create the "through" model, add it to the migration file 
> > 2) migrate all m2m instances (e.g. iterate over all Bereich objects then 
> > iterate through its users, creating a UserBereichAssignment object for 
> > each (all this done in a migrations.RunPython call) 
>
> Ok. I've never used migrations.RunPython before, but I'm quite confident 
> that I can manage it. 
>

You could also use RunSQL instead of RunPython for this data migration step 
- personally I often find that easier (and almost always more efficient) 
for a simple data migration, since all you need is a SQL query to copy some 
data from one table to another, you don't need any Python conveniences or 
model instances along the way.

But either can work, it really just depends how comfortable you are with 
raw SQL.
 

> Would the migration for step 2) be a separate migration file from step 
> 1), or is the migration file of step 1) edited to cover step 2) as well? 
>

Separate migration. In general you can't put schema alterations and data 
migrations in the same migration file, because then they'll try to run in 
the same transaction, and PostgreSQL at least doesn't like that.

In general, for complex changes, this three-step dance (one migration to 
add the new field/table, a second migration to copy the data, and a third 
migration to remove the old field/table) is very common and the right way 
to do things.
 

> > 3) change the m2m field by adding the through option, and add this 
> > change to the migrations file. 
>
> Same question: Is this supposed to go into a separate migration file or 
> into the common migrations file from above? 
>
> And won't this last step trigger the same error as before? ("... you 
> cannot alter to or from M2M fields, or add or remove through= on M2M 
> fields") ? 
>

This part I'm not sure about without trying it. I'm honestly not sure what 
exactly Gergely was recommending. Based on my reading of the code, the 
error is raised by the schema-editor backend, meaning if you try an 
`AlterField` with this change, you'd hit the error.

Another possibility might be to use the `SeparateDatabaseAndState` 
operation to create a migration that has no effect on the schema, but just 
updates the Python state for the field. Since you've already made the 
necessary db schema changes simply by adding the through model itself, this 
should work fine.

You could also go back and edit the field definition in whichever migration 
initially created it. This would probably work, but it would cause problems 
for any `RunPython` migrations since then that used that field (because now 
they'd try to use the through table instead), including your own data 
migration that you just created prior to this. So probably not a good 
option.
 

> (Not so important, but out of curiosity: This won't get us rid of the 
> old, implicit intermediate table, will it?) 
>

No. You could add another RunSQL migration to remove this table using raw 
SQL.

Overall I think it might be simpler to go with your initial approach. I'd 
first rename the old field to some other temporary name (you don't have to 
update any other code to use this name, as you'll commit all of these 
migrations in one commit, so nothing but the following data migration ever 
needs to know anything about this temporary name), then add the new field 
with through table (using the original field name), then migrate data 
(using either RunPython or RunSQL), then remove the old field. I think this 
will work fine, and it should also remove the old auto-created through 
table.

Good luck! I think this is definitely an area where the migrations system 
could help a bit more.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/e987cf79-358d-4fdc-bac5-5131a19a25c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: All posts from same day django

[Carl Meyer]

On 07/22/2015 02:26 PM, Nkansah Rexford wrote:
> Thanks Carl
> 
> Also got these approaches too, might help someone
> too: 
> http://stackoverflow.com/questions/31571607/all-posts-from-same-day-only-django

I just re-read your original post, and my suggestion was wrong - it gets
you all posts from the last 24 hours. Your accepted answer on SO is
right, except that really you should use `django.timezone.now()` instead
of `datetime.now()`.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55AFFD8C.7040301%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: All posts from same day django

[Carl Meyer]

Hi,

On 07/22/2015 12:54 PM, Nkansah Rexford wrote:
> I currently have:
> 
> |
> @classmethod
> defget_past_week(self):
> start_date =datetime.now().date()
> end_date =datetime.now().date()-timedelta(weeks=1)
>
> returnMyModel.objects.filter(pub_date__range=(end_date,start_date)).aggregate(Sum('off_hours'))
> |
> 
> which simply pulls all posts from the current date minus 7 days
> 
> I want to pull posts from within the same day factoring in the time at
> the moment. Thus if the time is 15:00 GMT now, I want all posts from
> 14:59:49 GMT back to 00:00:01 GMT of the same day.
> 
> How can I do something like that?

I'll assume `pub_date` is a `DateTimeField`. If it's a `DateField`, then
this query isn't possible unless you change it to a `DateTimeField`.

You want something like:

from datetime import timedelta
from django.utils import timezone

now = timezone.now()
one_day_ago = now - timedelta(days=1)

return MyModel.objects.filter(pub_date__range=(one_day_ago, now))

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55AFEF6C.7090600%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Testing related models without saving

[Carl Meyer]

Hi Roland,

On 07/16/2015 07:46 AM, Roland Swingler wrote:
>> i'll just say that Django tests don't favor too much into the "unit"
> test spectrum;
> 
> That is what I'm seeing - it is the same with rails out of the box as
> well. I guess I'm just curious to know whether:
> 
> a) the 'unit' test end of the spectrum is feasible if that's what you like
> b) what is available in the Django community (if anything, whether
> libraries, 'ways-of-doing-things' etc.) to support this if it is an
> approach one wants to take.

I also write model tests using unsaved models where possible. I don't
think it has significant test isolation benefits (your tests are still
integrating with most of the ORM), but it does have test-suite speed
benefits!

I understand why the change was made in 1.8 to disallow assigning an
unsaved object to a ForeignKey/OneToOneField attribute; in production
code that would almost always be a bug. Personally, though, I've never
been bitten by that bug, I'm confident I could easily find and fix such
a bug if I did write it, and I don't want to give up the ability to use
related unsaved models in tests. So I just use my own subclasses of
ForeignKey and OneToOneField with `allow_unsaved_instance_assignment =
True` set on them (essentially reverting the safety change in 1.8). I
haven't attempted to switch it on dynamically for testing; that should
be possible using a setting and a custom subclass, but I wouldn't choose
to do that; differences between test and production behavior should be
minimized.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55A7DDB4.9080506%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django formset hidden id field

[Carl Meyer]

Hi Peter,

On 07/04/2015 12:42 AM, Peter of the Norse wrote:
>> On Jul 2, 2015, at 7:46 PM, Carl Meyer <c...@oddbird.net> wrote:
>> 
>>> So what?  It’s quite likely that whoever is editing this row of
>>> the database, also has permissions to edit the other rows as
>>> well.  There’s no reason for them to go through the hassle of
>>> manually editing a hidden field when they can go to a different
>>> page and edit it there.
>> 
>> That's a bad answer. It's common in many systems for a user to
>> have access to edit some records in a table but not others (this is
>> often known as "object-level permissions"). If it was really
>> possible to edit any row in the table by just manually editing the
>> PK hidden field, that would be a serious security flaw in
>> formsets.
>> 
>> But it's not. Whatever queryset you pass to the model formset
>> limits the available rows for editing. The end user can edit the PK
>> to refer to any item in that queryset, but not any item in the
>> table.
>> 
>>> In general, primary keys are not security flaws.  While it’s a
>>> good idea to hide them from front-end pages, that’s mostly
>>> because they make URLs hard to read.  I have heard that you don’t
>>> want to use them publicly, because your competitors can use them
>>> to gauge your success, but that’s the kind of “Nice Problem to
>>> Have” that can wait until you’re bigger.
>> 
>> Exposing primary keys themselves (e.g. in URLs) is not necessarily
>> a security flaw. Exposing a primary key in a hidden field that can
>> be edited to change a form to edit any row in a table often would
>> be a serious security flaw.
> 
> You can’t have it both ways. Either exposing the PK is a security
> flaw or it isn’t. It’s just as easy for nefarious n’er-do-wells to
> edit the form’s URL as a hidden field. In either case, if you are
> using object-level permissions, more checks (not made automatic in
> Django) are necessary. Having the ID passed as a parameter doesn’t
> necessitate, hinder, or alleviate running these checks.

Yes, this is a good clarification, thanks. What I meant to say is that
simply exposing the ID information to the user isn't necessarily a
security issue. But any server-side use of a client-provided ID (whether
provided in the URL or in a form field) needs to be properly validated
on the server side (just like any other data received from the client).
Formsets do this validation; if they didn't, they would be insecure.

My main point is that it's not at all adequate to say that "if a user
has access to edit one object, they probably have access to edit all the
others anyway." They may or they may not; that's application-specific.
Formsets allow the developer to specify exactly what queryset of objects
the user should be allowed to edit via the formset, and they validate
that the user can only edit those objects, and no others (even if the
hidden ID field is manually modified.)

> If you can come up with a method that associates form data with a
> database row that doesn’t involve the PK, I would love to know. Keep
> in mind that most tables only have the one unique identifier.

I'm not saying that you shouldn't use unique IDs to associate form data
with rows in the database (though there are some valid reasons to prefer
e.g. a UUID or a unique slug to an auto-incrementing ID in some cases).
I'm just saying that you should never trust an ID value (or any data)
coming from the client; you should ensure on the server side that the
user actually should have access to the object they are trying to access.

Javier showed a good method for doing this in typical cases, by passing
a limited queryset rather than a model class to get_object_or_404.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/559ACC8C.8000104%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django model for non web applications

[Carl Meyer]

On 07/03/2015 07:46 AM, Vernon D. Cole wrote:
> Jeff:
>I think that Russell's answer might be more appropriate for your use
> case than Carl's.  django.setup() calls settings.configure(), but also
> tries to pull in other application modules, which you might not want.

No, `settings.configure()` and `django.setup()` are not alternatives.
They are _both_ needed for Jeff's use case (depending on how he wants to
handle settings). `settings.configure()` is one option for settings;
`django.setup()` is required no matter what (since Django 1.7).

Settings can be handled either via the usual Python settings module, in
which case the DJANGO_SETTINGS_MODULE env var needs to be set to the
import path to this module, or they can be handled by calling
`settings.configure()` with a dict of the desired settings.

One of those two things needs to be done before calling
`django.setup()`. `setup()` does not "call settings.configure()" - it
loads settings, which requires either that `settings.configure()` has
already been called, or that DJANGO_SETTINGS_MODULE is set.

And yes, `django.setup()` also loads installed applications. Since
Django 1.7, this is required in order to use the ORM (which is the core
of Jeff's use case).

Carl

> 
> 
> On Thursday, July 2, 2015 at 6:50:20 PM UTC-6, Carl Meyer wrote:
> 
> On 07/02/2015 05:49 PM, Russell Keith-Magee wrote:
> >
> > On Thu, Jul 2, 2015 at 12:50 AM, Jeff Fritz <jeff...@gmail.com
> 
> > <mailto:jeff...@gmail.com >> wrote:
> >
> > I'm fairly new to Django and making my way through the tutorial.
> >
> > Knowing what I've learned so far, I'd like to explore using
> > django.db.models.Model in a non-web application. I'd like to take
> > the entire model layer and use it for database access in a
> > standalone, multithreaded python 3.4 application. It is not a web
> > application. I would like to be able to take advantage of the
> > migration capabilities in manage.py.
> >
> > Questions:
> >
> > 1. Is it as simple as adding `from djanago.db import models`?
> Would
> > this bring in any unnecessary django packages/modules,
> considering I
> > will not be developing a web application, not using views,
> > templating, etc?
> >
> >
> > It's *almost* this simple - you will also need to configure your
> Django
> > environment before you start making database calls. If you're
> looking to
> > do this in a *completely* "non-web" way, this probably means a
> call to
> > settings.configure(); details here:
> >
> > https://docs.djangoproject.com/en/1.8/topics/settings/
> <https://docs.djangoproject.com/en/1.8/topics/settings/>
> 
> Don't forget that in Django 1.7+ you also need to call django.setup()
> yourself, if using Django outside the context of a WSGI server or
> management command:
> https://docs.djangoproject.com/en/1.8/ref/applications/#django.setup
> <https://docs.djangoproject.com/en/1.8/ref/applications/#django.setup>
> 
> Carl
> 
> >  
> >
> > 2. Is the django model layer thread safe? Specifically, when
> using
> > sqlite3 as a back end?
> >
> >
> > It should be - after all, web servers are frequently multi-threaded.
> > SQLite3's performance under multithreading, however, might leave
> > something to be desired.
> >  
> >
> > 3. Are there other clean ORM/database model layer
> > frameworks/packages that I should consider for python 3.4?
> >
> >  
> > The other obvious candidate would be SQLAlchemy; it's a perfectly
> fine
> > ORM - without any of the other web framework overhead. It's a lot
> more
> > like a "SQL building toolkit" than Django's ORM - whether this is
> a good
> > or bad thing depends on your own preferences and use case.
> >
> > Yours,
> > Russ Magee %-)
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users+unsubscr...@googlegroups.com
> <mailto:django-users+unsubscr...@googlegroups.com>.
> To post to this group, send email to django-users@googlegroups.com
> <mailto:django-users@googlegroups.com>.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on th

Re: Django formset hidden id field

[Carl Meyer]

On 06/28/2015 03:00 PM, Peter of the Norse wrote:
> On May 27, 2015, at 7:47 AM, Cheng Guo  > wrote:
>>
>> Hello,
>>
>> I have a formset and when I render it, Django would include this line
>> in the HTML:
>>
>> ||
>>
>> I am curious what is the purpose of having an id field here. 
>>
>> I mean in what situation would you use it. I did look through
>> Django's documentation on formset
>> but
>> cannot find much documentation on this.
>>
>> One answer I got is that this id field is the value of the primary key
>> of the model bound to this form. It is there so that when the formset
>> updates, people can use it to retrieve the corresponding record from
>> the database.
>>
>> Is the above explaination correct?
>>
>> If this explaination is correct, then my next question is, wouldn't it
>> be dangerous to expose the primary key like that? I can make a post
>> call to your server with a modified pk which can mess up your database.
> 
> So what?  It’s quite likely that whoever is editing this row of the
> database, also has permissions to edit the other rows as well.  There’s
> no reason for them to go through the hassle of manually editing a hidden
> field when they can go to a different page and edit it there.

That's a bad answer. It's common in many systems for a user to have
access to edit some records in a table but not others (this is often
known as "object-level permissions"). If it was really possible to edit
any row in the table by just manually editing the PK hidden field, that
would be a serious security flaw in formsets.

But it's not. Whatever queryset you pass to the model formset limits the
available rows for editing. The end user can edit the PK to refer to any
item in that queryset, but not any item in the table.

> In general, primary keys are not security flaws.  While it’s a good idea
> to hide them from front-end pages, that’s mostly because they make URLs
> hard to read.  I have heard that you don’t want to use them publicly,
> because your competitors can use them to gauge your success, but that’s
> the kind of “Nice Problem to Have” that can wait until you’re bigger.

Exposing primary keys themselves (e.g. in URLs) is not necessarily a
security flaw. Exposing a primary key in a hidden field that can be
edited to change a form to edit any row in a table often would be a
serious security flaw.


Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5595E969.1090205%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django model for non web applications

[Carl Meyer]

On 07/02/2015 05:49 PM, Russell Keith-Magee wrote:
> 
> On Thu, Jul 2, 2015 at 12:50 AM, Jeff Fritz  > wrote:
> 
> I'm fairly new to Django and making my way through the tutorial.
> 
> Knowing what I've learned so far, I'd like to explore using
> django.db.models.Model in a non-web application. I'd like to take
> the entire model layer and use it for database access in a
> standalone, multithreaded python 3.4 application. It is not a web
> application. I would like to be able to take advantage of the
> migration capabilities in manage.py.
> 
> Questions:
> 
> 1. Is it as simple as adding `from djanago.db import models`? Would
> this bring in any unnecessary django packages/modules, considering I
> will not be developing a web application, not using views,
> templating, etc?
> 
> 
> It's *almost* this simple - you will also need to configure your Django
> environment before you start making database calls. If you're looking to
> do this in a *completely* "non-web" way, this probably means a call to
> settings.configure(); details here:
> 
> https://docs.djangoproject.com/en/1.8/topics/settings/

Don't forget that in Django 1.7+ you also need to call django.setup()
yourself, if using Django outside the context of a WSGI server or
management command:
https://docs.djangoproject.com/en/1.8/ref/applications/#django.setup

Carl

>  
> 
> 2. Is the django model layer thread safe? Specifically, when using
> sqlite3 as a back end?
> 
> 
> It should be - after all, web servers are frequently multi-threaded.
> SQLite3's performance under multithreading, however, might leave
> something to be desired.
>  
> 
> 3. Are there other clean ORM/database model layer
> frameworks/packages that I should consider for python 3.4?
> 
>  
> The other obvious candidate would be SQLAlchemy; it's a perfectly fine
> ORM - without any of the other web framework overhead. It's a lot more
> like a "SQL building toolkit" than Django's ORM - whether this is a good
> or bad thing depends on your own preferences and use case.
> 
> Yours,
> Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5595DC21.5060600%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Atomic block termination question

[Carl Meyer]

Hi Mike,

On 06/25/2015 01:53 AM, Mike Dewhirst wrote:
> On 25/06/2015 2:34 AM, Carl Meyer wrote:
>> On 06/24/2015 02:16 AM, Mike Dewhirst wrote:
>>> On 24/06/2015 4:43 PM, Mike Dewhirst wrote:
>>>> When saving a model I'm getting a TransactionManagementError - You
>>>> can't
>>>> execute queries until the end of the 'atomic' block
>>>>
>>>> Ticket #21540 seems fairly explicit at least where Postgres is
>>>> concerned. TransactionManagementError prevents what I want to do and
>>>> I'm
>>>> not a nuclear expert.
>>>>
>>>> How do I terminate the save() method code in that atomic block and then
>>>> immediately execute my queries?
>>
>> I'm afraid this description of what you're trying to do is too vague to
>> be useful. Maybe some sample code would help?
>>
>> TransactionManagementError is a symptom, not a cause. It means that a
>> database error occurred inside a transaction, which leaves the
>> transaction in an unpredictable state, so Postgres wants you to roll
>> back the transaction (or roll back to a savepoint prior to the error)
>> before issuing any more database queries.
> 
> Ok. I thought from reading the ticket that I was trying to do something
> illegal in Postgres - that is issuing a query within a transaction which
> needed to be finalised or rolled back. I took it as a symptom or signal
> and think I understand that Postgres is somewhat more rigorous in this
> regard than MySQL.

Yes, that's right. (Well, the transaction needs to be rolled back. Once
there's been an error, there is no other "finalizing" possible besides a
rollback.) My point was that the transaction gets into this state
because some other query causes an error, so your efforts should first
focus on figuring out what that error was and making it not happen, if
possible.

>>
>> Possible solutions include:
>>
>> a) Figuring out why there was a database error, and fixing it so it
>> doesn't occur.
> 
> I separated out all the pre and post-save stuff without the offending
> queries and put them into ...
> 
> def save(self, *args, **kwargs):
> self._pre_save() # nothing tricky here
> super(Substance, self).save(*args, **kwargs)
> self._post_save() # nothing tricky here
> 
> ... which stopped the TransactionManagementError and everything worked
> on existing substances which already had the necessary 1:1 relations AND
> it kinda "worked" when I [saved as new] except obviously the 1:1
> relations were not created.
> 
> ... then did a _post_post_save() with the offending queries ...
> 
> def _post_post_save(self):
> if self.physical_state == SOLID:
> Solid.objects.get_or_create(substance=self)
> elif self.physical_state == LIQUID:
> Liquid.objects.get_or_create(substance=self)
> elif self.physical_state == GAS:
> Gas.objects.get_or_create(substance=self)
> elif self.physical_state == AEROSOL:
> Aerosol.objects.get_or_create(substance=self)
> Aquatic.objects.get_or_create(substance=self)
> Tox.objects.get_or_create(substance=self)
> Terrestrial.objects.get_or_create(substance=self)
> if self.terrestrial:
> # We can't do this in terrestrial.save() and it needs
> # to be recomputed on every save
> self.terrestrial.set_soil_m_factor()
> self.terrestrial.set_vertebrate_m_factor()
> self.terrestrial.set_invertebrate_m_factor()
> 
> ... which as I said is now called from substance.clean(). I realise
> clean() is called before save() but that's all I can think of at the
> moment. Those m_factors are unlikely to change once the concentration
> values (EC50, LD50 etc) upon which they are based are set.

I don't understand why you want to call this from clean() instead of
from save(), but it should work OK as long as you wrap it in an `if
self.pk:` so it doesn't try to run on an unsaved object. Of course that
means it will never run at all when saving a new object.

>> b) Wrapping the code that might cause a database error in its own
>> `transaction.atomic` block, so on error that bit of code is rolled back
>> and later queries within the same transaction can go forward.
> 
> That sounds like nuclear physics to me. I could probably follow a recipe
> but might have trouble figuring out when to use it.

Here's the general recipe. This code is problematic:

@transaction.atomic
def do_something():
do_the_first_thing()
try:
do_a_thing_that_might_cause_a_database_error()
except DatabaseError:

Re: Atomic block termination question

[Carl Meyer]

Hi Mike,

On 06/24/2015 02:16 AM, Mike Dewhirst wrote:
> On 24/06/2015 4:43 PM, Mike Dewhirst wrote:
>> When saving a model I'm getting a TransactionManagementError - You can't
>> execute queries until the end of the 'atomic' block
>>
>> Ticket #21540 seems fairly explicit at least where Postgres is
>> concerned. TransactionManagementError prevents what I want to do and I'm
>> not a nuclear expert.
>>
>> How do I terminate the save() method code in that atomic block and then
>> immediately execute my queries?

I'm afraid this description of what you're trying to do is too vague to
be useful. Maybe some sample code would help?

TransactionManagementError is a symptom, not a cause. It means that a
database error occurred inside a transaction, which leaves the
transaction in an unpredictable state, so Postgres wants you to roll
back the transaction (or roll back to a savepoint prior to the error)
before issuing any more database queries.

Possible solutions include:

a) Figuring out why there was a database error, and fixing it so it
doesn't occur.

b) Wrapping the code that might cause a database error in its own
`transaction.atomic` block, so on error that bit of code is rolled back
and later queries within the same transaction can go forward.

> I have implemented a workaround but not sure if it is the best way. Any
> comment appreciated ...
> 
> In the model's clean() method I test for self.pk and if true, execute
> the queries which previously caused the problem. This seems to work but
> hasn't had any testing in production.

Again it's hard to tell without seeing code or traceback, but it sounds
like probably what you've done here is fix the condition that was
causing the error in the first place (that is, solution (a) above). It
sounds like you were probably trying to do some related-model queries
with an unsaved model, and now you've guarded those queries to only
occur if the model is saved. If so, that's not a workaround, it's the
best solution.

>> I need the save() to complete so I can get_or_create some 1:1 records
>> belonging to the model being saved.

Again, sample code would really illuminate what you're trying to do.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/558ADC1E.6070709%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Inconsistency when importing views and models

[Carl Meyer]

On 06/19/2015 11:41 PM, James Schneider wrote:
> This is strictly a Python question, nothing Django-specific, but I've
> found this site to be helpful in explaining the different ways to import
> things:
> 
> http://effbot.org/zone/import-confusion.htm

This is good as far as it goes, but it is also quite old, and thus
doesn't cover explicit relative imports at all (because they didn't
exist yet when it was written).

> In general, using 'from blah import *' is frowned upon except in very
> specific cases. While it is easier up front, it makes the code hard to
> read later on, since you aren't explicitly dating where the functions or
> classes you are using are coming from.
> 
> In general, you'll want to be as specific as possible when importing
> things to keep your code clean and easy to read, even if it does make it
> a bit longer.
> 
> For example:
> 
> from CatAppA import *
> from CatAppB import *
> 
> my_cat = get_cat('Katy Purry')
> 
> If we were inspecting this code, we wouldn't know whether or not
> get_cat() comes from CatApp A or B, which makes troubleshooting more
> difficult, especially if both apps (or modules) define a get_cat()
> function. It would be much better to say:
> 
> from CatAppA import get_cat
> from CatAppB import say_meow
> 
> my_cat = get_cat('Katy Purry')
> 
> Now we know that get_cat() comes from CatAppA.

Yes, that's a great description of the reason to avoid `import *`.

> The other aspect to consider is memory use. If a module defines 500
> functions, doing 'import modulename' will load all 500 in memory, rather
> than just picking out the one or two you might use.

This isn't true. When you first import a module (no matter what syntax
you use), its full code is executed, resulting in a module object with
all of its top-level objects and attributes defined, which is placed
into the `sys.modules` dictionary. So every function or class defined in
the module will be in memory no matter what.

The import syntax you use determines only which functions/objects you
get references to in the current module, it doesn't change memory usage
at all.

(Not to mention that, apart from extreme cases, memory usage of your
code objects themselves will be negligible compared to your data.)

> Aside from that, all of your references would be prefixed with
> 'modulename', which may or may not make the code more readable.

This isn't related to `import *`, it depends on the distinction between

   from somemodule import some_func

vs

   import somemodule

In the latter case, you would use `somemodule.some_func` in your code;
in the former you'd use `some_func`. Both of those are equally explicit;
the choice between them is really a matter of preference, conciseness,
and avoiding naming clashes. If I am using only one or two things from a
module (especially if I am using them many times), I'll usually use
`import from` to make the references shorter. If I am using many things
from the module, I'll prefer the latter form (and sometimes shorten the
module name with an alias, via `import somemodule as sm`).

> In general, I import everything explicitly, and I use relative import
> paths for imports in files that are in the same directory:
> 
> from .models import Cat
> 
> Yes, it makes my import lines much longer, but having them more explicit
> makes the code much easier to read on larger chunks of code.
> 
> There it's some contention as to whether or not to use relative imports,
> some devs prefer to use the more explicit appname.models version. Both
> do the same thing.
> 
> Check out the Django source code to see how they handle imports, and
> definitely look at PEP8:
> 
> https://www.python.org/dev/peps/pep-0008/#imports

Good advice; PEP 8 gives useful guidance on these questions.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55876B8F.70705%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Inconsistency when importing views and models

[Carl Meyer]

Hi,

On 06/19/2015 08:32 PM, Peith Vergil wrote:
> It depends on how your project files are structured. If you have a file
> structure that looks like this:
> 
> app/
> __init__.py
> models.py
> views.py
> urls.py
> 
> Then, you can simply use relative imports.
> 
> In views.py:
> from models import *
> 
> In urls.py:
> from views import *

This style of import is called "implicit relative imports", and you
should never use it. It is deprecated in Python 2, and doesn't work at
all in Python 3. (You can opt in to the Python 3 behavior in Python 2
with "from __future__ import absolute_import".)

The reason it is "implicit" is because it is ambiguous - the "models"
and "views" modules you are importing from could be relative to the
current module location, or "absolute" (directly on sys.path), and it
opens the possibility of accidentally shadowing a top-level installed
module (even a stdlib module) with a local module (e.g. if you put a
`datetime.py` or a `django.py` into your package), causing hard-to-debug
issues.

(Also, as has been mentioned, you shouldn't use `import *` because it
leads to ambiguity as to the source of names in your code.)

The right way to do relative imports is with the explicit syntax:

from .models import MyModel
from .views import some_view

or to import a whole module:

from . import models
from . import views

Using the `.` make it explicit that you are importing from a local
module relative to your current module, not from a top-level module.

Carl

> If, for example, your models live in a separate Django app (i.e. a
> separate Python package):
> 
> app1/
> __init__.py
> views.py
> urls.py
> app2/
> __init__.py
> models.py
> 
> Then, you have to import your models using its full path so Python will
> know where to look for it.
> 
> In app1/views.py:
> from app2.models import *
> 
> You should familiarize yourself on how Python imports work since this is
> more of a Python issue rather than a Django issue.
> 
> On Jun 20, 2015 09:10,  > wrote:
> 
> This is mostly a cosmetic question, and I could be completely wrong
> because I'm fairly new to Django, or it could be that there is a
> perfectly logical explanation for this, but here goes:
> 
> It seems the code required to import views in urls.py and models in
> views.py is inconsistent (and in the case of urls.py perhaps redudant).
> 
> To import views in urls.py I have to use
> 
> |
> fromimportviews
> |
> 
> ...while in views.py I can simply write
> 
> |
> frommodels import*
> |
> 
> Why do I need to reference the appname in urls.py but not in
> views.py? Or is there a reason for this?
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to django-users+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-users@googlegroups.com
> .
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/django-users/03eb4740-6503-4757-ae09-9183ac2b8502%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-users@googlegroups.com
> .
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAMHY-KdkaS0RBdS4F4mi%3DaGHTP9uZThJVCesXM9%3DGbVbZoteaw%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55876924.4090409%40oddbird.net.
For more 

Re: Django 1.8 Transaction Begin

[Carl Meyer]

Hi Kate,

On 06/18/2015 04:05 AM, Katarzyna Łabędź wrote:
> Maybe i didn't understand it correctly. 
> 
> I know Django support transactions, but isn't it do it on commit? 
> I just tried to find the BEGIN statement inside them and i could not
> find it - searched through internet and what I found was a bunch of
> people asking the same question - where is the begin in the
> transactions? So i just figured i ask here. 
> Why Django don't execute Begin statement ? Isn't it necessary to have a
> fully correct transaction?

You are right to be confused, because this is confusing territory.

The default behavior of all Python database adapters is "implicit
transactions", meaning the underlying database adapter library itself
(Python-MySQL, psycopg2, etc) sends a BEGIN automatically as soon as you
send your first query. (At least if the library is compliant with PEP
249 [1], the Python database adapter standard, which all adapters used
by built-in Django backends are.)

Django overrides this and places the underlying connection into
autocommit mode (no transactions) by default. But when you request a
transaction (by using an 'atomic' block), Django doesn't need to
explicitly send BEGIN, it just puts the connection back into the default
PEP 249 "implicit transactions" mode, which means the database adapter
will send BEGIN automatically as soon as you send a query. So Django
never sends a BEGIN itself, just COMMIT or ROLLBACK.

In practice, though, you can ignore this implementation detail, and just
assume that a BEGIN is sent at the start of any (outer) 'atomic' block
(and COMMIT or ROLLBACK) is sent at the end, and you'll be correct.

I wrote a blog post [2] which discusses this in more detail. It's about
PostgreSQL instead of MySQL, but the part near the beginning about PEP
249 and Django applies equally well to MySQL (I think; I'm not very
familiar with transactions on MySQL).

Carl

[1] https://www.python.org/dev/peps/pep-0249/
[2] http://oddbird.net/2014/06/14/sqlalchemy-postgres-autocommit/

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5582E88F.7090709%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Squashed migration

[Carl Meyer]

Hi Cherie,

On 06/12/2015 06:32 AM, Cherie Pun wrote:
> Thanks for your replies! I am still on Django 1.7 but we are hoping to
> upgrade to 1.8 soon. I manage to use the squashed migrations when
> creating the test database now,

Do you know why it didn't work initially? That'd be useful to know, in
case it's something that might trip up other users, too.

> but there's syntax error in the
> automatically generated squash migration. The error occurs on the line
> on which it reference the user-defined functions in other migrations. It
> seems fine when it ran the first methods though.

That sounds like a probably-fixable bug (though it may already be fixed
in 1.8 and/or master). If you're able to reproduce the bug on master,
please do file a ticket for it.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/557AFE4C.7060403%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Squashed migration

[Carl Meyer]

On 06/12/2015 06:32 AM, aRkadeFR wrote:
> You need to delete your old migrations so it uses only the squashed
> one after.

No, the squashed migration should be used in place of the old ones for
any new database, even if the old ones are still present. This is the
point of the squashmigrations feature; that you can keep the old ones
around (which is necessary for any deployments that may not yet have
applied all of them) while still gaining the benefit of the new squashed
migration for new deployments (and tests).

I know this works, because I just did it recently myself. It sounds like
Cherie was able to get it working too, though we didn't get any
clarification on why it didn't seem to work originally.

> In the documentation:
> "This enables you to squash and not mess up systems currently in
> production that aren’t fully up-to-date yet. The recommended process is
> to squash, keeping the old files, commit and release, wait until all
> systems are upgraded with the new release (or if you’re a third-party
> project, just ensure your users upgrade releases in order without
> skipping any), and then remove the old files, commit and do a second
> release."

That's right. The length of time you need to wait before removing can
vary widely. For a third-party app, it may be a full release cycle or
two (as long as you can tell your users to upgrade version-by-version).
For a project with only a few deployments, all under your control, it
may be the same day. But regardless, the squashed migration will still
be used in tests immediately, before you remove the old migrations.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/557AFDF3.1090900%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Best Practices for Migrations in Reusable Apps?

[Carl Meyer]

Hi Jonathan,

On 06/10/2015 12:24 AM, Jonathan Barratt wrote:
> I'm interested to know what the community's opinions are on how best
> to package migrations with reusable apps (I'm thinking of the
> built-in migrations, but it seems the same principles would apply for
> pre-1.8 south migrations).
> 
> My current approach is to squash all schema migrations each time a
> new one is generated so that new installs are smooth;

You can do this, but you should keep the old migrations around as well,
for the sake of your existing upgrading users. You should only remove
replaced migrations once you are confident that all your users (or
whatever percentage of them you care about) have upgraded.

The migration-squashing system is designed to make it easy to squash
migrations for new users but also keep the old ones around for upgrading
users; it automatically takes care of making sure the right migrations
are used in each case.

> we've seen
> issues arise from just keeping the individual migrations that
> accumulate the long way (in terms of both speed and bugs).

Speed can be an issue with lots of migrations; squashing migrations (but
keeping the old ones) addresses this.

Not much that can be said about bugs without specific examples. The
migration system has certainly had its share of bugs! Almost all the
ones we know about have been fixed. Your old set of migrations should
result in the exact same schema as the new squashed one. If it doesn't,
that's certainly a problem that needs to be fixed, in Django and/or in
one or more of the migration files.

> I figure existing users who upgrade can be expected to run their own
> make migrations if necessary, but perhaps I'm placing the burden of
> work on the wrong party...

This is placing the burden on the wrong party (and on many parties,
instead of one party), and it actually puts your upgrading users in
quite a pickle. They can't add migrations directly to your codebase
without forking it, which means their only recourse to add the necessary
migration is to set MIGRATION_MODULES to ignore your migrations entirely
and replace them with their own.

MIGRATION_MODULES is intended for rare cases, not routine use. Your app
should contain the necessary migrations for all your users, new and
upgrading. Failing to provide migrations that will work for upgrading
users means you miss out on all the benefits of migrations; you're
essentially returning to the old syncdb world, where no provision is
made for upgrades that include schema changes.

> If initial data is necessary, then it gets its own separate data
> migration - this seems to keep the code (and git history) cleaner
> IMO.
> 
> I'd greatly appreciate and criticism, suggestions or alternate
> approaches that occurs to you.
> 
> Thanks in advance, Jonathan

Hope that helps,

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55788921.2020406%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Have mercy please!

[Carl Meyer]

Hi Jon,

Thanks for sharing your experience.

On 06/09/2015 07:25 PM, Jon Foster wrote:
> I've been involved with Django, on and off, since v0.96 or so. I love
> Django and think its the most productive way to build rich websites with
> custom defined content types. Throw in Django-CMS and things get pretty
> darn cool.
> 
> I was thrilled when Django reached v1.0 since it came with the promise
> of having a consistent and stable API, specifically a lack of backwards
> incompatible changes, unless required by security concerns.
> Unfortunately this promise is routinely violated for various excuses.
> The bottom line is none of the apps I've written have been able to
> survive any 2 point upgrades (v+0.2). Single point upgrades usually only
> cause minor breakage.

This last confuses me, so I'd like to get clarity. If "single point
upgrades usually only cause minor breakage", then a two point upgrade is
just two single point upgrades, one after the other, correct?

Trying to upgrade directly from e.g. 1.6 to 1.8 without first going
through 1.7 - by which I mean "having a fully working site that passes
its tests with no deprecation warnings in 1.7" -- is definitely not
recommended.

So is the problem here that you've been trying to do a two-version
upgrade directly, instead of version-by-version? Or that "upgrade with
minor breakage" + "upgrade with minor breakage" = "too much for the
project to survive"?

> I realize the desire to grow things and I applaud it. But there is a
> business issue here. I can't, in good conscience recommend Django as a
> site platform to many of my small clients as they simply could not
> afford the upkeep of a Django powered site. Especially if the site is
> e-commerce related, where PCI, and responsible site operation, will
> require that we stay current. In order to do so would require staying up
> with the constant flow of backwards incompatible changes, combined with
> the time and effort to reverse engineer and maintain contributed apps,
> which aren't keeping pace either.
> 
> With the current method of development on the Django platform, if I had
> just a dozen sites of moderate complexity, it would become a full time
> job just keeping them updated. Its complicated enough just finding the
> apps that will actually work with each other to construct a site. But
> the carefully constructed house of cards is virtually guaranteed to
> break with the next update.
> 
> So I ask, PLEASE return to and stick with the promise of API stability?
> You promised and routinely point to that statement, while making
> backwards incompatible changes. I want to spend more time working with
> Django, but I need to know that my clients can rely on painless and cost
> effective upgrades.
> 
> Thanks for reading my complaint,
> Jon
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-users@googlegroups.com
> .
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/11e78690-2b5f-4e99-a377-62c19b74e333%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/557886D1.9030608%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Upgrade Confusion with ForeignKey

[Carl Meyer]

Hi Tim,

On 06/05/2015 04:26 PM, Tim Sawyer wrote:
> Thanks. :-)
> 
> Did that, read the release notes, made sure it worked after each upgrade
> stage.  This is an esoteric corner of my app, and I didn't notice it not
> working.  The other foreign key stuff I have seems to work ok.

Ah, good! In that case, sorry for the unnecessary admonitions. :-)

> Looks like I have more investigation to do...any more clues over what
> *could* be the cause appreciated!

Well, I can give you a rundown of some things that crossed my mind as I
was thinking through what might cause that symptom. Some of them don't
make much sense, and none of them satisfy me as an explanation given the
data I have, but maybe they'll remind you of something in your codebase:

* Was your shell session pasted exactly as you ran it? No possibility
that queryset came from somewhere other than the reverse related manager?

* Any custom model managers in use on either of the relevant models?
(Didn't look like it from the code you pasted).

* Both the apps are in INSTALLED_APPS?

* No `to_field` on any relevant ForeignKeys?

Beyond that, if I were debugging this I would use PDB to step through
the entire process of accessing the reverse-relation attribute,
generating the related-manager, getting a queryset from it, etc, and
compare that to the same process for a reverse-related-set that's
working properly.

Good luck!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/557223E8.3060401%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Upgrade Confusion with ForeignKey

[Carl Meyer]

Hi Tim,

On 06/05/2015 03:44 PM, Tim Sawyer wrote:
> I've just upgraded from a very old version of Django (1.4) up to 1.7.8,
> and some behaviour isn't how I expect.  I may have missed something -
> I've not been keeping an eye on the framework changes.

I would strongly recommend that you upgrade version by version (that is,
first 1.4.x to 1.5.12, then 1.5.12 to 1.6.11, then 1.6.11 to 1.7.8),
each as a separate step, making sure that everything works as you expect
between each step, and that you carefully read the release notes (if you
haven't) for each version as you update to it.

Django tries to maintain backwards compatibility where we can, but we do
have to change and remove features sometimes. Those changes are
documented in the release notes for each version.

If you've upgraded all the way from 1.4 to 1.7 without reading the
release notes, which is what it sounds like, then I will be very
surprised if the below is the only problem you have.

> Two models, Venue and VenueAlias.  VenueAlias has a ForeignKey to Venue,
> one venue has many VenueAliases. (Full models pasted below.)
> 
> My problem is with accessing venuealias_set on my Venue, it doesn't seem
> to be limiting to Venue how it used to in the earlier version.
> 
> Here's an interactive session that shows the problem.  Venue 5854 has
> three aliases.
> 
 from bbr.contests.models import Venue
 lVenue = Venue.objects.filter(id=5854)[0]
 lVenue
>  Manchester, Greater Manchester, England, UK>
 lVenue.exact
> False
 lAliases = lVenue.venuealias_set.all()
 len(lAliases)
> 402
 print lAliases.query
> SELECT "venues_venuealias"."id", "venues_venuealias"."last_modified",
> "venues_venuealias"."created", "venues_venuealias"."name",
> "venues_venuealias"."alias_start_date",
> "venues_venuealias"."alias_end_date", "venues_venuealias"."venue_id",
> "venues_venuealias"."lastChangedBy_id", "venues_venuealias"."owner_id"
> FROM "venues_venuealias" INNER JOIN "contests_venue" ON (
> "venues_venuealias"."venue_id" = "contests_venue"."id" ) WHERE
> "contests_venue"."exact" = True ORDER BY "venues_venuealias"."name" ASC
> 
> Where's the SQL to limit by the foreign key to venue.id=5854?  Why is
> there a reference in here to contests_venue.exact?

That is very strange. I've never seen that, and I'm not aware of any
changes to Django that would cause that. Nor do I see anything in the
models code you pasted below that looks suspicious to me as a possible
cause.

I have to guess that there's some other code in your project causing
this problem, but I'm afraid I don't know what that might be.

I'm afraid the best I can advise is to go back to Django 1.4 and do the
upgrade again methodically, and see if you can track down at precisely
which version this problem first manifests.

Good luck!

Carl


> Approaching the same problem from the other direction works fine, and I
> get the correct number of aliases returned.
> 
 from bbr.venues.models import VenueAlias
 lVenueAliases = VenueAlias.objects.filter(venue_id=5854)
 len(lVenueAliases)
> 3
 print lVenueAliases.query
> SELECT "venues_venuealias"."id", "venues_venuealias"."last_modified",
> "venues_venuealias"."created", "venues_venuealias"."name",
> "venues_venuealias"."alias_start_date",
> "venues_venuealias"."alias_end_date", "venues_venuealias"."venue_id",
> "venues_venuealias"."lastChangedBy_id", "venues_venuealias"."owner_id"
> FROM "venues_venuealias" WHERE "venues_venuealias"."venue_id" = 5854
> ORDER BY "venues_venuealias"."name" ASC
> 
> On my old server, running the old django 1.4 code, both these approaches
> work as expected.  Venue and VenueAlias are actually in different
> models.py files, for historical reasons.
> 
> What have I missed?  Thanks for any input/education!

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55721CB8.9070202%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Bug in 1.7-1.8 migrations

[Carl Meyer]

Hi!

On 06/05/2015 05:29 AM, Владислав Пискунов wrote:
> In 1.7 auto-created migration with makemigrations. it contains: 
> 
> |
> related_name=b''
> |
> 
> Now in 1.8 while migrate, django tries Render model state and fails with:
> 
[snip]

>   File
> "/home/vladislav/.virtualenvs/furskru/local/lib/python2.7/site-packages/django/db/models/fields/related.py",
> line 1355, in is_hidden
> return self.related_name is not None and self.related_name[-1] == '+'
> IndexError: string index out of range
> |

Certainly looks like a bug! Would you be willing to report it at
https://code.djangoproject.com/newticket ?

Thanks!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5571E8BC.2000403%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Bug? Infinite migrations with empty help_text and ugettext_lazy

[Carl Meyer]

Hi Andy,

On 06/04/2015 09:44 AM, Andy Gimblett wrote:
> I've come across something which might possibly be a Django bug, but I wanted 
> to raise it here first in case I'm missing something, or this is expected 
> behaviour.
> 
> There's a fuller explanation and easily runnable example here:
> 
> https://github.com/gimbo/django-ugettext_lazy_migrations
> 
> In short, given the following model (note the empty help_text string),
> 
> from django.db import models
> from django.utils.translation import ugettext_lazy as _
> 
> class X(models.Model):
> y = models.CharField(help_text=_(''), max_length=10)
> 
> repeated calls of "manage.py makemigrations" will, after the initial 
> migration, produce an infinite series of new migrations, each of which 
> contains an AlterField, but which does nothing.  The expected behaviour would 
> be to produce an initial migration and then no further migrations, of course.
> 
> The problems goes away if you do any of the following:
> 
> * Remove the help_text parameter
> * Use a non-empty help_text
> * Don't internationalise the help_text value
> * Use ugettext instead of ugettext_lazy (which, I understand, you shouldn't)
> 
> I've reproduced it on all released 1.8.x Django versions up to 1.8.2, but it 
> doesn't exhibit on any 1.7.x versions I've tried.
> I haven't tried any development versions.
> 
> Can anyone provide any insight?  Is this worth raising a bug over?

That is most certainly a bug. It's not a super-high-priority bug
(because an empty translated help_text is an odd thing to do), but it's
a bug nonetheless and should be filed at
https://code.djangoproject.com/newticket and fixed.

I would guess the problem has to do with a careless `if help_text:`
somewhere in the migration detection code.

It would be worth checking whether the same problem exists with any
other similar field parameters (e.g. `label`).

Thanks for finding and reporting the bug!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55708551.1010702%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Help with customizing Django authentication

[Carl Meyer]

Hello Carlos,

On 05/29/2015 03:19 PM, Carlos Ribas wrote:
> Hello,
> 
> I have to confess that I did not understand your suggestion. How this
> will help me to reverse the logic of my system? I mean, instead of User
> with or without a profile (the Person class, in my case), I want a
> Person with or without a User.
> 
> Thanks anyway
> 
> Em quarta-feira, 27 de maio de 2015 12:54:59 UTC-3, Carlos Ribas escreveu:
> 
> Hello All,
> 
> I am currently extending the existing User model to store additional
> information. So, basically I have:
> 
> # models.py
> class Person(models.Model):
> user = models.OneToOneField(User, verbose_name=_('User'))
> zipcode = models.CharField(_('Zip Code'), max_length=9,
> blank=True, null=True)
> street = models.CharField(_('Address'), max_length=255,
> blank=True, null=True)
> ...
> 
> #admin.py
> class PersonAdmin(admin.StackedInline):
> model = Person
> ...
> 
> class UserAdmin(UserAdmin):
> inlines = (PersonAdmin, )
> ...
> 
> admin.site.unregister(User)
> admin.site.register(User, UserAdmin)   
> 
> 
> The problem is that my system should be able to register a new
> person, but this person may not need an account on the system (I
> just need to have his/her information). Right now, I can not create
> a person without an account. Besides that, first_name and last_name
> fields are not in the person class. 
> 
> I am wondering what is the best solution for my case. Probably, I
> will need to move the first_name and last_name fields to the Person
> class, and to do that, I will have to create custom users, is that
> right? 

Yes, the best solution for your case (and for all Django projects) is to
use a custom User model.

In order to have every User linked to a Person, but not all Persons
linked to Users, you need to place the OneToOneField on the User model
pointing to Person, rather than the other way around. And in order to do
that, you need to have control over the User model.

(You _could_ sort of do it without a custom User model by having a
ManyToMany relationship between User and Person, but that allows a wide
variety of situations you don't want to allow.)

> If that is the case, may I just copy and paste the lines shown here
> 
> (https://github.com/django/django/blob/master/django/contrib/auth/models.py
> 
> )
> and remove the lines about first_name and last_name? 

No, you should follow the documentation on custom User models.

You'll want to inherit from AbstractBaseUser instead of AbstractUser, so
as to not have first_name and last_name fields. You'll need to add the
PermissionsMixin and probably a few other fields (including a username
field). The docs should explain what you need to know.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5568DFAC.8020800%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Why the extra minute in PostgreSQL when using time zone info?

[Carl Meyer]

Hi aRkadeFR,

On 05/29/2015 02:09 AM, aRkadeFR wrote:
> Indeed, the problem comes from the representation on PG,
> cause I have only good results (0/30 for the minutes) from
> the python function.
[snip]

The OP already found and posted the solution (see below) and it is not
related to Postgres. Here's a fuller explanation:

A pytz timezone class does not represent a single offset from UTC, it
represents a geographical area which, over the course of history, has
probably gone through several different UTC offsets. The oldest offset
for a given zone, representing the offset from before time zones were
standardized (in the late 1800s, most places) is usually called "LMT"
(Local Mean Time), and it is often offset from UTC by an odd number of
minutes.

When you create a timezone using `pytz.timezone('Europe/London')`, pytz
doesn't yet know what datetime you plan to attach the timezone object
to, so it doesn't know which historical period's offset it should use,
and it defaults to the first one in the zoneinfo database, which is
usually LMT:

>>> import pytz
>>> uktz = pytz.timezone('Europe/London')
>>> uktz


You can see that this timezone object is Europe/London _LMT_, and you
can also see that it is one minute offset from UTC (that's the "23:59:00
STD" bit).

The _wrong_ way to attach a pytz timezone to a naive Python datetime is
by passing it to the `tzinfo` argument of the datetime constructor (or
by using `.replace(tzinfo=...)`). When you do this, pytz literally
attaches the exact timezone offset (London LMT, dating from the
mid-1800s) to your datetime:

>>> from datetime import datetime
>>> lmt_dt = datetime(2015, 6, 11, 13, 30, tzinfo=uktz)
datetime.datetime(2015, 6, 11, 13, 30, tzinfo=)

If you convert this to UTC, you'll see the effect of that mysterious one
minute offset:

>>> lmt_dt.astimezone(pytz.utc)
datetime.datetime(2015, 6, 11, 13, 31, tzinfo=)

The _right_ way to attach a pytz timezone to a naive Python datetime is
to call `tzobj.localize(dt)`. This gives pytz a chance to say "oh, your
datetime is in 2015, so I'll use the offset for Europe/London that's in
use in 2015, rather than the one that was in use in the mid-1800s":

>>> good_dt = uktz.localize(datetime(2015, 6, 11, 13, 30))
>>> good_dt
datetime.datetime(2015, 6, 11, 13, 30, tzinfo=)

You can see that this datetime object is in BST (one hour offset from
UTC, at least in June) instead of LMT - a much better choice for this
century. And we can convert it to UTC and get a more sensible result:

>>> good_dt.astimezone(pytz.utc)
datetime.datetime(2015, 6, 11, 12, 30, tzinfo=)

Since we're on the Django list here, I should also point out that
`django.utils.timezone` has a `make_aware` method which handles this
correctly for you automatically.

Interesting historical sidenote: the 1-minute offset for London LMT is
based on historical observations such as this [1]:

# From Peter Ilieve (1994-07-06):
#
# On 17 Jan 1994 the Independent, a UK quality newspaper, had a piece about
# historical vistas along the Thames in west London. There was a photo
# and a sketch map showing some of the sightlines involved. One paragraph
# of the text said:
#
# 'An old stone obelisk marking a forgotten terrestrial meridian stands
# beside the river at Kew. In the 18th century, before time and longitude
# was standardised by the Royal Observatory in Greenwich, scholars observed
# this stone and the movement of stars from Kew Observatory nearby. They
# made their calculations and set the time for the Horse Guards and
Parliament,
# but now the stone is obscured by scrubwood and can only be seen by walking
# along the towpath within a few yards of it.'
#
# I have a one inch to one mile map of London and my estimate of the stone's
# position is 51 degrees 28' 30" N, 0 degrees 18' 45" W. The longitude
should
# be within about +-2". The Ordnance Survey grid reference is TQ172761.
#
# [This yields GMTOFF = -0:01:15 for London LMT in the 18th century.]


Carl

[1] https://github.com/eggert/tz/blob/master/europe#L107


>> On Thu, May 28, 2015 at 11:42 PM, Daniel Grace > > wrote:
>>
>> I used this function, seemed to tidy up the problem:
>>
>> from datetime import datetime
>> from pytz import timezone
>>
>> def utcDtTm(self, dt_tm):
>> tz = timezone('Europe/London')
>> return
>> tz.normalize(tz.localize(dt_tm)).astimezone(timezone('UTC'))

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55688D0F.9040602%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.

Re: Adding a one field to user model: trying not to create custom one)

[Carl Meyer]

On Tuesday, May 19, 2015 at 8:18:24 AM UTC-6, Ilya Kazakevich wrote:
>
> > I am really unhappy with idea of using custom user model. 
>>
>> Why? 
>>
>> If it's because this is an existing project and the prospect of 
>> migrating your existing data to a custom user model is daunting, I 
>> totally understand that. It's doable, but hard. 
>>
> Yes, this project already deployed and has some data. Sure I can solve 
> this, but I feel that changing user model will make my app less reusable. 
>

I don't think "will make my app less reusable" is a valid reason to avoid a 
custom User model. A lot of thought went into the custom User model system 
to ensure that it preserves reusability. That's why things like 
`django.contrib.auth.get_user_model()` and friends exist.
 

> But I will probably stay with new model. There is something wrong with 
> Django in this area. There should be some easy and elegant way to add one 
> field to auth_user. I wonder why Django developers do not care about it.
>

Django developers do care about the problem, and the preferred solution 
(after many years of discussion) is the custom User model. I believe that 
it's a better solution than introducing a hacky way to monkeypatch fields 
onto a built-in model. I think every Django project should use a custom 
User model.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/d2f5e133-47a4-4255-a016-03d4b8a14d5d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Adding a one field to user model: trying not to create custom one)

[Carl Meyer]

Hello Ilya,

On 05/18/2015 04:58 PM, Ilya Kazakevich wrote:
> I want to add just a one field to my model, and this field participates
> in user representation ( I use monkeypatch to change __str__).
> 
> Django manual tells me to create profile with one-to-one relation in
> this case. Well, it may work. But if I have  with 250 users, I
> face 250 "SELECT" queries to my db (N+1). I need to tell "UserManager"
> somehow to use "select_related". But how can I do that with out of new
> ugly monkeypatching?
> Does there is an official (recommended) way to do that? If no, how  can
> I use one-to-one user profile if I have "list of users" webpage with out
> of N+1? How to add one field to user model not extending it?

I am not aware of a good solution to this problem other than manually
adding the .select_related() to your query on the list-of-users page.

> I am really unhappy with idea of using custom user model.

Why?

If it's because this is an existing project and the prospect of
migrating your existing data to a custom user model is daunting, I
totally understand that. It's doable, but hard.

If this is a new project, I think that there is no good reason to avoid
a custom user model. Every new project should always use a custom user
model, even if to begin with it just inherits from AbstractUser and
doesn't change or add anything. This way in the future you have control
over your user model and can modify it as needed without monkeypatching.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/555A72A0.2070105%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Invalid HTTP_HOST, can someone explain why I am getting this?

[Carl Meyer]

> On May 16, 2015, at 8:12 AM, Gerald Klein  wrote:
> 
> This looks like a cross domain request that is being received by your site 
> and the requestor is not listed in your sites allowed sites list, Sites 
> conforming with CORS will have a list of sites that can request data via http 
> from them, this is stating the requestor  is not on the list

No, this is not related to the problem. Django's ALLOWED_HOSTS setting has 
nothing to do with CORS or the domain the request is coming from, it's about 
the server's domain(s) and the Host header in the request. 

Carl

> Hope that helps
> 
>> On Sat, May 16, 2015 at 8:49 AM, bobhaugen  wrote:
>> I've gotten this error now and then when I have updated the software on a 
>> webfaction-hosted site and restarted Apache. Then it goes away after maybe a 
>> couple more restarts. It's like the software has not quite gotten itself 
>> together yet. I have no idea why this should be, and since it goes away, I 
>> have not opened a ticket on webfaction. 
>> 
>>> On Wednesday, May 13, 2015 at 5:45:43 PM UTC-5, Russell Keith-Magee wrote:
>>> 
>>> Is there a setting to turn of a warning that someone is attempting to 
>>> access your site in a potentially malicious way? No.
>>> 
>>> When you get a warning like this, you investigate the cause, and fix the 
>>> problem. You don't just silence the warning.
>>> 
>>> Yours,
>>> Russ Magee %-)
>>> 
 On Thu, May 14, 2015 at 3:44 AM, frocco  wrote:
 Thanks for getting back to me.
 I am on django 1.5. Is there a setting I can use to avoid getting these 
 emails?
 I get at least 5 a week.
 
 
 
> On Tuesday, March 17, 2015 at 6:58:00 PM UTC-4, Russell Keith-Magee wrote:
> Hi,
> 
> It's possible that you're getting this error for the exact reason that 
> the check was added - someone is submitting requests to your site with a 
> manipulated HTTP_HOST header. This may not be malicious - it could just 
> be a badly configured robot making web requests.
> 
> It might also point at a problem with your hosting provider - especially 
> if you're sharing an IP address with g3suprimentos.com.br, or if 
> g3suprimentos.com.br once had the same IP that you're currently using. 
> 
> Without more details, it's hard to say for certain. But is someone trying 
> to hack your site? Almost certainly not. If you were being hacked, you 
> wouldn't have described the problem as "I see this error from time to 
> time" - you would have said "I just received a thousand of these in the 
> last 10 minutes". Attackers aren't noted for their subtlety - they will 
> generally try every possible backdoor and exploit that they know of, all 
> at once. 
> 
> Yours,
> Russ Magee %-)
> 
>> On Tue, Mar 17, 2015 at 11:45 PM, frocco  wrote:
>> I am concerned because I have a value in ALLOWED_HOSTS, but it is not 
>> this value.
>> 
>> 
>>> On Tuesday, March 17, 2015 at 9:17:15 AM UTC-4, frocco wrote:
>>> SuspiciousOperation: Invalid HTTP_HOST header (you may need to set 
>>> ALLOWED_HOSTS): www.g3suprimentos.com.br
>>> 
>>> I keep getting this error from time to time.
>>> 
>>> Is someone trying to hack my site?
>>> 
>>> Thanks
>> 
>> -- 
>> You received this message because you are subscribed to the Google 
>> Groups "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send 
>> an email to django-users...@googlegroups.com.
>> To post to this group, send email to django...@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/5550dde0-4230-4068-a8b5-196f0bd844fc%40googlegroups.com.
>> 
>> For more options, visit https://groups.google.com/d/optout.
> 
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 "Django users" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to django-users...@googlegroups.com.
 To post to this group, send email to django...@googlegroups.com.
 Visit this group at http://groups.google.com/group/django-users.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/django-users/ab908cd5-8c5b-4d6e-9033-80fca60415f7%40googlegroups.com.
 
 For more options, visit https://groups.google.com/d/optout.
>>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at 

Re: Invalid HTTP_HOST, can someone explain why I am getting this?

[Carl Meyer]

On 05/15/2015 09:14 AM, Andreas Kuhne wrote:
> You are getting it because someone has come to your server by
> entering www.g3suprimentos.com.br  in
> a browser. If you don't have that address in the allowed hosts setting,
> you will get the error you have received. 

Correct.

> If you don't want your server to show something
> for www.g3suprimentos.com.br , then
> the problem is not on your end, but rather the clients (it can be an
> incorrect DNS entry, or that someone has edited their hosts file). So
> you can't do anything to fix it.

Not true - in most cases you can fix it on the server side, but the
details depend on your server configuration. You need to configure
whatever front-end server is listening for the web requests to ignore
any requests coming in for a host name that you don't want your server
to respond to. If you're using Apache, this meand making sure that your
Django site is running in a name-based virtualhost (and not the default
one). If you're using nginx, it just means setting the server_name of
the server block to the hostname(s) you actually want to serve. Etc.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5556116D.5090806%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django code of conduct - last line

[Carl Meyer]

Hi Graham,

On 05/08/2015 01:43 PM, Graham Oliver wrote:
> Thank you, so how about
> 
> 'Don’t forget that it is human to err and blaming each other doesn’t get
> us anywhere.
> Instead, focus on helping to resolve issues and learning from mistakes'

I think this is an improvement in clarity and style over the current
wording. Please see https://www.djangoproject.com/conduct/changes/ for
the process to make changes to the Code (short version: send a pull
request to http://github.com/django/djangoproject.com).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/554D1DA9.1090401%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Ensure an object is saved even when the atomic bock rollsback

[Carl Meyer]

Hi Marc,

On 05/08/2015 07:15 AM, Marc Aymerich wrote:
> I'm using atomic requests, but one of my views needs to save a model
> regardless of wheter the current transaction rolls back or not.
> 
> I'm confused about how to proceed with the current autocommit behaviour.
> Do I need to use a different db connection? or perhaps there is some way
> of telling django to ignore autocommit for some particular block ? 

You'd need to use a different db connection, since what you're trying to
do violates the very nature of a database transaction. If you were just
trying to run some raw SQL, you could establish the separate connection
yourself manually, but if you're trying to save a Django model, you'll
probably need a second connection defined in your DATABASES setting.

Possible alternatives:

- If you're using a task queuing system (like Celery or rq), queue a
task to perform the save; you can do this without making it conditional
on the transaction committing, as long as your task queue is using a
store other than your primary database (e.g. Redis or RabbitMQ). Usually
when people do this it's unintentional and a bug (they really wanted the
task to execute only if the current transaction committed), but in your
case it could be a feature.

- Switch from ATOMIC_REQUESTS to explicit use of transaction.atomic() in
your views, so that you can place this "no matter what" save in its own
transaction.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/554CDB37.7060503%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Why so complicated :O??

[Carl Meyer]

Hi Vesko,

On 05/06/2015 12:47 PM, Vesko Nikolov wrote:
> Hello everyone. So i have been trying to learn Django for the last
> couple of months but have been seeing it quite hard. I have a decent set
> of skills in PHP but i wanted to learn something newer and that is what
> brought me here. So on with the question. Where can i get a BASIC
> tutorial of how django works and how to perform simple tasks. All the
> tutorials I find are DATABASES MODELS TEMPLATING. I don't care about any
> of that sh*t (at least not at this stage). I want to make django
> calculate some math for me or some equally simple tasks so i can get an
> idea how basics work before i interact with databases and make really
> dynamic projects. 

Sure, that's reasonable. I would actually prefer it if the official
Django tutorial tackled things in the order you suggest, covering simple
views and HTTP request/response handling before covering models, the
admin, etc.

I think you'll find the starting point you're looking for in part 3 of
the official tutorial, at
https://docs.djangoproject.com/en/1.8/intro/tutorial03/#write-your-first-view

You'll still need to follow part 1 to get your project and app created,
but you can skip over the database and models stuff and come back to
that later, if you want.

Good luck!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/554A7E19.6040708%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: RemovedInDjango19Warning: Model class %s doesn't declare an explicit app_label and either isn't in an application in INSTALLED_APPS ..

[Carl Meyer]

Hi,

On 04/29/2015 11:13 AM, Bumyong Choi wrote:
> I am trying to understand this warning better. I understand that we want
> to make sure that a model has an explicit app_label or it is in
> INSTALLED_APPS. I spent some time trying to remove warnings from my code
> because I was getting these warning messages even though my model was
> defined in an app listed in INSTALLED_APPS. 
> 
> After scratching my head for quite some time, I put a breakpoint on the
> following line of the code in django.db.models.base:
> 
> # Look for an application configuration to attach the model to.
> 
> app_config =apps.get_containing_app_config(module)
> 
> And I checked to see what I get for "app_configs.values()" and I
> realized that the app is in INSTALLED_APPS but not yet been loaded. This
> was because one of the third party framework I use was trying to import
> models before they were loaded. 
> 
> My question is the following:
> 
> Is Django trying to prevent a model from being imported before this
> model is actually defined? If this is the case, is it possible to
> improve the warning so that it would check "INSTALLED_APPS" and warn the
> user that even if the app is listed there some other app is trying to
> import the model before the app is loaded?"

Yes, that is the current intention in Django 1.7+: you may not import a
models.py file before the app cache has been prepared. (I personally
think this is unfortunate, but to change it while maintaining consistent
and predictable semantics would require a deep reworking of how relation
fields are initialized, which nobody has so far attempted).

I'm surprised you are getting that particular error though - if a
models.py file is imported too soon, I would expect an "App registry not
ready" error instead. So it may be a bug that you are getting this error
message instead of that one; hard to know without seeing code to
reproduce the issue.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/554157B1.5000702%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django runserver needs restart on psql data changes

[Carl Meyer]

Hi Arnab,

On 04/29/2015 03:13 PM, Arnab Banerji wrote:
> Hi Javier - thanks for the response. Yes - I am storing the data in a
> dictionary - then having django_tables2 API render the dictionary as a
> table. How do I force a read of that variable for every browser refresh
> on the client side? 

Using a variable is fine; the problem is that (I'm guessing) you are
creating that variable at the top level of the module. Such code is only
run once, when the module is first imported.

You need to make sure that all database queries are run only inside view
functions (or methods of view classes), never at the top-level of a module.

carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5541568D.1060808%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: forcing user model to return first_name last_name if exists

[Carl Meyer]

On 04/28/2015 09:10 AM, Ilya Kazakevich wrote:
> I have many places in my app where user (from user model) is displayed:
> templates, forms, fliters, tables etc.
> It is displayed as username everywhere.
> 
> I want it to be displayed as first_name/last_name.
> 
> I can do that with monkey patching:
> 
> @receiver(request_started)
> def patch(*args, **kwargs):
> get_user_model().__str__ = my_smart_str_func
> 
> I use "request_started" to make sure all models are loaded. (BTW, does
> there is something like "models loaded" signal?).

Not a signal; the proper place for code like this in Django 1.7+ is in
an AppConfig.ready() method.

> How ever, I feel it a little bit hacky. There should be more pythonic
> way to do that. Am I miss something?

Of course, the best way to do this is a custom User model so you don't
have to monkeypatch at all. Sadly, switching to a custom User model is
difficult if the project didn't start with one.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/553FA98A.4060301%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrations During Development

[Carl Meyer]

On 04/25/2015 05:18 PM, Carl Meyer wrote:
> You can use the "squash" feature [1] to squash a set of historical
> migrations down to a single migration.

Forgot this link:
https://docs.djangoproject.com/en/1.8/topics/migrations/#squashing-migrations

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/553C210A.1050209%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrations During Development

[Carl Meyer]

On 04/25/2015 11:28 AM, Timothy W. Cook wrote:
> if you don't store migrations, then you have to generate them on each
> production update, right?  if so, wouldn't they count as untested
> code?  how about manually tweaked, or data migrations?
> 
> 
> ​That will certainly be true once you start using a staging or
> production server.  
> But before they they are just a kind left over creative ideas.  :-)  Not
> really useful for anything in the future, AFAICS. 

If you only have one deployment (your local machine), you can erase all
your migrations and "start over" anytime with a new initial migration,
with no problems.

Once you have additional deployments (either other developers' machines,
or staging or production deployments), you shouldn't remove or change
any migrations that have been pushed to those other deployments.

You can use the "squash" feature [1] to squash a set of historical
migrations down to a single migration. This requires a phased
deployment, where you first push out a version including both the new
squashed migration and all the old replaced migrations, and then later
(once you are confident all deployments are up to date) you can safely
remove the old replaced migrations.

It's also true that if you have a series of new migrations locally that
haven't yet been pushed to any other deployment, you can delete and
replace them with a single migration before pushing.

It's very much like VCS commits: it's ok to edit your local history, but
you have to be a lot more careful with history that you've already
shared with others.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/553C20AE.9080609%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrations During Development

[Carl Meyer]

On 04/25/2015 11:15 AM, Javier Guerra Giraldez wrote:
> On Sat, Apr 25, 2015 at 12:08 PM, Timothy W. Cook 
> wrote:
>> 
>> On Sat, Apr 25, 2015 at 9:48 AM, Jorge Andrés Vergara Ebratt
>>  wrote:
>>> 
>>> Are you using version control like GIT? I save the migration
>>> folder with the __init__.py in GIT, nothing else, because al the
>>> migrations will be diferent in the diferent servers.
>>> 
>> 
>> This is probably what I should have done. I have been saving them
>> all to the git repo.
> 
> 
> with South i used to save them to the repo too and push whole to 
> production.  haven't tried Django migrations yet; is there any 
> difference that make this more appropriate?
> 
> if you don't store migrations, then you have to generate them on
> each production update, right?  if so, wouldn't they count as
> untested code?  how about manually tweaked, or data migrations?

Migrations should be committed to VCS in both South and in Django
migrations.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/553C1F83.60105%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Migrations During Development

[Carl Meyer]

On 04/25/2015 06:48 AM, Jorge Andrés Vergara Ebratt wrote:
> Are you using version control like GIT? I save the migration folder with
> the __init__.py in GIT, nothing else, because al the migrations will be
> diferent in the diferent servers.

This is not a good idea. Migrations should be committed to git, and
should be the same on all deployments. Otherwise they do not serve their
purpose of ensuring that all deployments end up with the same database
schema.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/553C1F55.10904%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: How do I manage a Non-default Django PostgreSQL database

[Carl Meyer]

On 04/09/2015 07:12 AM, Stephen M wrote:
> If Django migrations with PostgreSQL is a bit flaky (as suggested by
> S.O. comment) which backend database is reliably supported by the Django
> migrations mechanism?

The premise is false. Postgres is almost certainly the best-supported
database for Django migrations.

I'm not entirely sure what's going on in your example, but the issue is
most likely related to multiple-databases, not related to Postgres. Did
you read the documentation at
http://django.readthedocs.org/en/latest/topics/db/multi-db.html#synchronizing-your-databases
(really you should carefully read that whole page if you're using
multiple databases)? Did you use the `--database` option to the
`migrate` command?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55269ADD.7090608%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Problem after Virtualenv Setup

[Carl Meyer]

Hi Kishan,

On 04/02/2015 09:41 AM, Kishan Mehta wrote:
> I have created my django project and worked on it . Recently I came to
> know about virtualenv and the problem it solves.
> I have installed virtualenv and activated that environment.
> Now django command is not recognized.
> * Will I have to install django again ? Or is there any other efficient
> way.*
> Thanks for help.

You will have to reinstall Django inside the virtual environment.

You should probably also uninstall the global Django you installed
(should just be a matter of "pip uninstall Django" without the
virtualenv activated; may need to preface that with "sudo " depending on
your OS and configuration.)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/551D63D9.8070005%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Help with Dynamically Adding Forms to Formset!!

[Carl Meyer]

Hi Stephanie,

On 03/30/2015 02:37 PM, Stephanie Socias wrote:
> I'm having trouble creating a formset where I can dynamically add and
> delete items. I've tried many different approaches and can't get
> anything to work. I'm new to Django and have been at this for days. Can
> someone please help me!? I can send my code, post it, or do a screen
> share. Please let me know!! Any help would be greatly appreciated.

I'm afraid I don't have time in my day for a screen-share, but I'd be
happy to try to help, if you can link to a gist or paste showing your
current best effort, and explaining how it is not working.

FWIW, I've usually used this library for the JavaScript side of things:
https://github.com/jgerigmeyer/jquery-django-superformset

If you'd find real-time assistance more helpful, you could visit the
#django channel on FreeNode IRC; I and others are available to help out
there.

This is a challenging task you've tackled, even for experienced Django
developers, so it's not surprising that it's taking some time if you're
new to Django.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5519BD9B.9080105%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: QueryDict and its .dict() method - why does it do that?

[Carl Meyer]

Hi Gabriel,

On 03/27/2015 02:10 PM, Gabriel Pugliese wrote:
> I perfectly understand what you are saying. It was very clear and
> informative, but do not agree with the design chosen here. Below is just
> an opinion and you do not have to agree with it:

That's good :-) I probably won't continue with the design discussion
past this email, because I don't think we will reach a conclusion which
convinces the Django core team to change the design.

> My buddies have given PHP and Rails examples, but there are other
> frameworks from other languages that do that the same way.

I'm not personally aware of them, but I believe you. If this behavior
ever becomes part of an accepted RFC, I'm sure Django will implement it
in core :-)

> I mean,
> what's the advantage here doing differently from others?

I already tried to explain that: simple, predictable behavior that
exposes the underlying querystring data directly to the user, with no
munging of values which might be unexpected by some users.

Django provides the simple, predictable basic behavior. If you want this
extended special behavior, it's easy to implement it on top of what
Django provides.

> And I don't agree it follows KISS if I need to re-iterate on the result
> again to get a dict from it (one clear example usage is destructuring as
> named function parameters).

If you just want a dict, you don't need to iterate, you can just use the
.dict() method (or other options already explained earlier in this thread).

If you want a dict interpretation of the querystring *that handles keys
with brackets in their names in a special way*, then yes, you have to
implement that yourself. It wouldn't be hard to implement, and then that
code could be shared by all who want it.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5515BB4F.3010502%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: QueryDict and its .dict() method - why does it do that?

[Carl Meyer]

Hi Gabriel,

On 03/27/2015 01:34 PM, Gabriel Pugliese wrote:
> @Masklinn
> That input is from jQuery's default serializer. It sends the data with
> contentType 'application/x-www-form-urlencoded; charset=UTF-8'. I just
> need to pass a data parameter with my JS Object (JSON) and it does the
> magic.

Yes, jQuery has chosen to adopt the same non-standard that Rails and PHP
use for smuggling nested lists and objects through a flat querystring.

> I understand I can use .getlist() or any other non-optimal solution.

request.GET.getlist('key') is the correct solution in Django for getting
a list of values from a querystring.

> I
> also understand what's going on about keys here (that's why I asked
> about laziness nature of QueryDict).

I don't understand what you think is lazy about a QueryDict, or how that
would be related to this issue. QueryDicts are not lazy; they parse the
querystring immediately on instantiation.

> But following DRY, KISS and other
> philosophies, couldn't Django do the same as other frameworks assuming
> it's widely adopted that way? What is the advantage of doing things so
> differently about that?

Django's philosophy here follows KISS. We give you the querystring in a
parsed format that allows you to easily get the data out of it by the
actual key names found in the querystring. We don't assume that you are
using jQuery on the client side, and we don't add any magical
non-standard features to smuggle nested objects through the querystring.
Such behavior might "do the expected thing" for some new users of
Django, at the expense of being very confusing to other new users of
Django, who understand HTTP and HTML but haven't been exposed to the
jQuery/Rails/PHP behavior.

> Why should I need to implement a lib that re-iterates over the same
> string that QueryDict already parses

You don't need to go back to the raw querystring. It would not be hard
to implement a small function that would take a QueryDict, iterate over
it and interpret it in the jQuery/Rails/PHP style, and return a
dictionary. I'd guess that someone may have already done this, though
I'm not sure what terms I'd use to search for it. If nobody has, then
you could be the first! Put it on PyPI, advertise it, and see what kind
of demand there is.

> to achieve the result that is
> expected to everyone that is new to Django?

I am a counter-example to your assertion that the Rails/PHP/jQuery
behavior is "expected to everyone that is new to Django." I had never
heard of the Rails/PHP/jQuery behavior when I was new to Django, and
Django's handling made perfect sense to me. The first time I ran across
jQuery's behavior, I was very confused.

> Anyway, thanks for the clear answer and information. And sorry about any
> grammar errors, English is not my first language.

No problem. I hope my answer was clear and informative.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5515B49F.3010603%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Re-run only tests that failed last time

[Carl Meyer]

Hi Gergely,

On 03/24/2015 04:27 AM, Gergely Polonkai wrote:
> I have a pretty extended test suite for my application, which can run
> for about 15 minutes. Now when I introduce an error, I will be notified
> of it pretty late, and a one-liner fix needs another 15 minutes to
> check. Of course if only one of my tests fail, I can add the name of the
> test case as a parameter to manage.py test, but in case of many failures
> it’s pretty hard to do the same.
> 
> Is there a way to run only the tests that failed last time, even if I
> have to install a separate app/module for that? A quick Google search
> gave me no usable results.

I use py.test [1], another test runner which has plugins that offer this
feature. The pytest-django [2] plugin provides the necessary integration
to make py.test able to run a normal Django test suite. Then the
pytest-cache [3] plugin offers an ``--lf`` (last-failed) option to run
only the tests that failed in the last run, and the pytest-xdist [4]
plugin offers the even-more-advanced ``--looponfail`` option, which "run
your tests repeatedly in a subprocess. After each run py.test waits
until a file in your project changes and then re-runs the previously
failing tests. This is repeated until all tests pass after which again a
full run is performed."

If you're interested in advanced Python testing, I think py.test is near
the top of the list of tools you should at least take for a spin.

Carl

[1] http://pytest.org/latest/
[2] https://pypi.python.org/pypi/pytest-django/
[3] https://pypi.python.org/pypi/pytest-cache
[4] https://bitbucket.org/pytest-dev/pytest-xdist

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5511E627.8090409%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django 1.6's lifespan for security updates?

[Carl Meyer]

On 03/19/2015 01:01 AM, Russell Keith-Magee wrote:
> On Thu, Mar 19, 2015 at 12:05 PM, James Bennett  > wrote:
> Way, way, *way* back in the day, there was unofficial bugfix support
> for pre-magic-removal versions of Django run on the basis of "some
> people who need it step up and do all the work". It was solely done
> in SVN and never involved issuing new releases, so anyone running
> off the maintenance branch had to just update their checkout
> whenever a fix landed in it.
> 
> I'm not opposed to allowing that again -- and in fact I was one of
> the people who did extended maintenance for pre-m-r Django because
> my employer at the time needed it -- but I think we should carefully
> consider how it's going to work before trying it, since we no longer
> have the ability to give partial commit bits
> 
> 
> Agreed that this is an approach we could take - but I don't think the
> lack of a partial commit bit isn't a major issue IMHO.
> 
> At a technical level, Git gives anyone the ability to fork Django's
> repository. There's no *technical* reason that someone couldn't maintain
> a 1.6 branch *right now* - the only restrictions are that they couldn't
> call that repository official (as per BSD licensing terms), and they
> wouldn't have access to security patches until everyone else did.
> 
> However, both of these are solvable problems. 
> 
> The only thing that makes Django's Github repository "official" is
> because the core team says it is. If someone wants to take on the task
> of back porting all the security patches to 1.6, we (as a project) can
> point to that repository if we want. If we're interested in doing 1.6
> support as an "unofficial" subproject, then it seems appropriate to me
> that the repository *isn't* Django's core repository; picking a
> third-party vendor's repo sounds like the right way to do it. 
> 
> As for getting the patches ahead of time - I'd be more than happy for an
> appropriately qualified and trustworthy individual/company to be added
> to the security pre-notification list.
> 
> So - it really just comes down to someone having the enthusiasm to
> maintain the branch. Christian has been around the Django project for a
> long time (he was a contributor to Django Evolution from very early on,
> and took over maintenance of the project when I moved on). If he's got a
> commercial interest in continued 1.6 support, and he can spare a couple
> of hours to do a backport when a security issue is announced, I'd have
> no problem adding him to the pre-notification list, and adding some
> documentation pointing at his fork of Django as an "unofficial 1.6
> repository". If Christian hasn't got the time to do this, but he (and/or
> anyone else) can throw some funds at Tim, Tim is already on the security
> notification list.

I agree with all of this. I think that the Django core team will not
provide this extended support for 1.6 (because it gives the impression
that Python 2.6 is a supported and secure platform, which is only true
for the subset of our user base on certain extended-support distros),
and I don't think that extended-support releases of 1.6 should or will
ever appear on pypi.python.org/pypi/Django.

But I think that anyone who feels the need for this extended support can
feel free to raise the resources to make it happen and do it in a github
fork. If someone planning to maintain such a fork applied for
pre-notification of security releases, and could demonstrate significant
support for their effort, I would be in favor of that application.

And I think that if "that person" ends up being Tim Graham, as an
individual outside of his duties as Django Fellow, that's his choice and
I don't see any problem with it. We've already established a precedent,
several times over, for members of the core team to be paid for certain
tasks that parts of the community need or want that aren't getting done
otherwise. I would not support the DSF or Django core team sponsoring
the fundraising for this effort, but if a part of the community raises
the funds and Tim wants to accept said funds and do the work, more power
to him, as long as it doesn't detract from his other work as Django Fellow.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/550AF6BD.8000703%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Testing template context processors

[Carl Meyer]

Hi Gergely,

On 03/17/2015 10:52 AM, Gergely Polonkai wrote:
> I wrote a context processor which makes some global variables of mine
> available for templates. I cannot find a good (cough… any) article on
> how to test such processors. Could you give me some directions on where
> to start with writing one?

Writing a unit test for a context processor is quite simple. Construct a
fake request (you can use RequestFactory, though if your context
processor only accesses one or two attributes of the request, it might
be simpler to just build a custom fake or use a library like pretend
[1]), then call the context processor function in your test, passing in
the fake request, and assert that the returned dictionary contains what
you expect.

If your templates require the values from the context processor in order
to render correctly, you can also write a higher-level integration or
functional test by using the test client to access a view that renders a
template that needs the context processor, and then assert the right
value is found in the rendered HTML.

Carl

  [1] https://pypi.python.org/pypi/pretend

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55085D7A.5040007%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Using the eval() command

[Carl Meyer]

Hi Henry,

On 03/10/2015 07:37 PM, Henry Versemann wrote:
> So how does an expression like you suggested above (
> innerDict['+newinnrkey+'] = newinnrval ) work then?
> It seems like it wouldn't work without enclosing the expression
> with quotes or double-quotes, and even then it seems like it would only
> become some kind of string instead of a statement which would be
> automatically executed to produce a result. Please explain or point me
> to some documentation explaining this type of code or coding.
> Thanks.

I didn't suggest that line of code, Bill did. As Tom said, it was
probably a typo. The line of code I suggested, which is what you should
use, is simply:

innerDict[newinnrkey] = newinnrval

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/550067A9.6030903%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django 1.6's lifespan for security updates?

[Carl Meyer]

Hi Stephen,

On 03/11/2015 06:28 AM, Stephen Gallagher wrote:
> So, here's the basic problem. The distributions that are packaging
> python 2.6 are, basically Red Hat Enterprise Linux 6 and its clones
> (CentOS, Scientific Linux, Oracle, etc.) and SUSE Linux Enterprise
> Server 11. These two distributions make up a significant percentage of
> the enterprise deployment space. Neither of these distributions ships
> Django itself. For RHEL, the Fedora Project provides the EPEL add-on
> repository which is unsupported and *may* carry Django (though it has
> proven to be difficult, more on that in a minute). For SLES, there is
> OpenSUSE which acts similarly. These are community-sponsored and
> generally limited to only packaging what upstream provides, since with
> no corporate backing and engineering support, backporting patches from
> newer releases is unlikely to happen.
> 
> The reason that neither of these distributions carries Django is
> specifically because of the short lifecycle of Django. The LTM releases
> (1.4 and the upcoming 1.8) are somewhat better aligned with the goals of
> enterprise distributions, but even those approximately three-year
> lifespans are significantly shorter than the ten years that the
> enterprise distributions want to maintain. So the chances of Django ever
> being shipped by a company capable of supporting it for that length of
> time is basically zero. (Not unless Django someday becomes a critical
> part of the standard Linux platform).
> 
> In the Enterprise software space, there's an average of a two-year
> period between a new version of software becoming available and
> companies actually deploying it. This essentially means that an LTM
> release needs to be at minimum four years long, since it will require
> half of the supported lifecycle for consumers to convert over to it and
> they'll need the other half of it in order to migrate to the next
> version. (Also, most companies don't like constant migrations like this).

It certainly sounds like there's an opportunity here for someone to
provide extra-extended security-backport support for certain Django
releases (beyond the ~3.5 years we'll typically support an LTS release
under current policy), to accommodate these enterprise users on dead (to
upstream) Python versions.

I remain quite unconvinced that this "someone" should be the Django core
team. I think the core team's limited energies are better spent
continuing to move Django forward.

> So, I realize that Django 1.6 is not an LTM release, and so asking for
> an extension on it is probably unlikely to happen. I'd like to see 1.4
> supported for at least the two-year migration period to 1.8, though
> (with 1.8 planned to accommodate the future migration as well).

I think the current plan is for a 6-month support window for 1.4 after
the release of 1.8. (Tim can probably correct me if I'm wrong - I know
there was discussion about 3 months vs 6 months; I thought we settled on
six.)

> Also, is there any chance at all that python 2.6 support could be
> reintroduced to Django 1.8? That would make it plausible to migrate
> existing users to an LTM release at least, buying time to figure further
> plans out.

I think that's extremely unlikely.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/55006656.7070305%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django 1.6's lifespan for security updates?

[Carl Meyer]

Hi Christian,

On 03/10/2015 01:44 PM, Christian Hammond wrote:
> According to
> https://docs.djangoproject.com/en/1.7/internals/release-process/#supported-versions,
> it appears that when Django 1.8 is released, Django 1.6 will no longer
> receive security updates. I wanted to verify that this is true, and ask
> whether there's a possibility of an extension on this timeframe.

It is true, and I don't think it should be extended.

> I'll explain the situation we're in.
> 
> I manage Review Board, a code review tool currently in use by several
> thousand companies/organizations, many of whom (according to stats we
> have available) are on Python 2.6. From conversations we've had, many of
> these companies are on LTS releases of Linux distributions that bundle
> Python 2.6 by default (including their mod_wsgi support, etc.), and are
> likely to remain on it for the next year or two. Not to mention Amazon
> Linux and other variants are all sticking with 2.6 for now as well.
> 
> This puts us in a difficult position where we are unable to drop Python
> 2.6 support without affecting a large number of installs out there (12%
> of our base, or over 700 installs), meaning we haven't yet been able to
> make the transition to Django 1.7/1.8 (as much as we want it). (It also
> makes the lives of packagers easier who are trying to support software
> stuck in this situation, from what I'm being told, as they're
> responsible for security updates.)
> 
> As Django 1.6 is the last release to support Python 2.6, it would be
> very nice to have a longer-term security release plan while companies
> transition over. We see this happening, but slowly.
> 
> Is there any possibility of treating Django 1.6 as a special LTS release?

I sympathize with your situation, but Python 2.6 reached end-of-life on
October 29, 2013 (a year and a half ago now), and since then has been
unsupported and not receiving security updates. I don't think the Django
core team should set a precedent of extended support for Python versions
which are themselves unsupported by the core Python developers.

If some Linux distributions are backporting Python security patches to
2.6 themselves in order to extend its lifetime in their distribution,
perhaps it would make sense to ask them whether they will also backport
Django security patches to Django 1.6. (I would guess that some of them
may already be planning to do so, and may even have already done so for
previous Django releases in the past.)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54FF6991.7080909%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Using the eval() command

[Carl Meyer]

Hi Henry,

On 03/10/2015 03:25 PM, Henry Versemann wrote:
> I have a new dictionary that I want to build, using data from another
> dictionary. I have a view which is receiving a single key/value pair
> from the original dictionary. Then in the view I've defined the new
> dictionary like this:
> 
> innerDict = {}  
> 
> Now I want to make this as dynamic as possible so I'm trying to use the
> "eval()" statement below to add the new key/value pair to the new
> dictionary, which is declared above. Will the following code work to
> actually add the new key/value pair to the new dictionary?
> 
> innrDictCmnd = "innerDict['"+newinnrkey+"'] = newinnrval"
> eval(innrDictCmnd)
> 
> If not why not, and in lieu of the statements above not working, then
> how would I do it?

It doesn't work, because eval() only accepts expressions; assignment is
a statement. Using exec() instead of eval() will work (though the way
you have it written, it will always assign the string "newinnrval" --
perhaps you meant to end innrDictCmnd with '... = ' + newinnrval).

But regardless, you should not use either eval() or exec().

Since you say this code is in a view, I assume that newinnrkey comes
from request data (user input). Imagine what happens if I am a malicious
user and I call this view with newinnrkey set to:

'] = 0; import os; os.rm('/'); d = {}; d['

Oops.

Both exec() and eval() should be avoided. They are very rarely
necessary, they usually make code less readable and maintainable, and if
you ever accidentally pass user input to them, you've opened up a
security hole in your application that someone could drive a truck through.

For your case, what's wrong with just writing `innerDict[newinnerkey] =
newinnerval`? It's every bit as dynamic as the version using eval or
exec - the eval/exec gains you nothing.

Carl


-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54FF66E3.5050408%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Fixing a poorly written Django website (no unit tests)

[Carl Meyer]

On 03/06/2015 11:23 AM, Ilya Kazakevich wrote:
> You may start from highest level testing: 
> 1) create "usage scenarios" for your website. Like "customer opens page
> 'foo', and should see 'bar'". You use such scenarios for manual testing,
> right?
> 2) code such scenarios against Django project. You may use BDD testing
> tools (like lettuce or behave), or do it in pure python. You may call
> Django views and templates directly (using django unittesting support or
> lettuce/harvest), or do it through the browser using Selenium. One
> scenario -- one method. Yes, there is a lot of boilerplate code, but you
> only need to write it once. BDD tools are used to simplify this process
> by writing scenarios on DSL called Gherkin, which is easier. 
> 
> This is some kind of "acceptance testing", the one that helps you to be
> sure website works like customer expects it to work. Every project
> should have one, I believe.
> 
> After it, you may find that some units of your system are complex and
> error prone. What is unit? Unit is module, function or even package with
> _clear interface_. You then create unit tests against this module. Only
> units with complex logic need to be tested. No need to test simple view
> that just returns render(): this view is tested as part of high level
> testing. But if you have function "calc_user_amount" and complex
> business logic stands behind it, you may create unit test for it.
> 
> There are 3 mistakes people do about testing:
> 1) Testing each function, even private function, even 1 line function.
> It takes a lot of time, and such tests are fragile. You throw'em away
> after simple refactoring.
> 2) Testing "in the middle of abstraction": I mean testing functions with
> out of clear interface, like private functions. If you need to read
> function code before writing test (pydoc is not enough), you should not
> test this function. Try a higher level of abstraction.
> 3) Skipping high level testing: even if all your code is covered with
> low-level unit-tests, you still need high level testing like the one
> with Selenium to make sure everything works correctly, and when you
> refactor your code you use such testing to make sure you did not break
> anything.
> 
> So, you start with high level, and then cover _some_ units with unit
> test, if you believe they may contain error.

This is really excellent advice.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54F9FA6E.8050001%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Problem installing and using smart_selects

[Carl Meyer]

Hi Michael,

On 03/01/2015 01:11 AM, Michael Ackary wrote:
> Smart Selects looked so simple and promising for chained selects
> (https://github.com/digi604/django-smart-selects).  However, no matter
> how I install it, it doesn't seem to work.  Here is what I've tried:
> 
> 
> *Attempt #1 - putting it in django.contrib
> *
> Copied the smart_selects folder to /django/contrib/smart_selects
> 
> Added "django.contrib.smart_selects" to INSTALLED_APPS in settings.py
> 
> But within Django shell I got this error:
> 
  from django.contrib import smart_selects
  smart_selects.urls 
> AttributeError: 'module' object has no attribute 'urls'

You don't ever want to do this. The "django.contrib" module is for
modules distributed with Django; it's not a place to add your own
additional code.

It also doesn't work, because smart_select expects to be able to import
itself as "smart_select", not as "django.contrib.smart_select."

> *Attempt #2 - installed using pip
> *
> I removed the smart_selects folder from /django/contrib  folder (above)
> and then installed using pip... which put smart_selects in the Python
> site-packages folder.
> 
> But then trying to use smart_selects within urls.py produced an ugly
> "ImportError: cannot import name simplejson" which suggests
> smart_selects has some outdated dependencies. (I"m using Django 1.7).

This is the right way to make use of third-party code. The problem here,
as you say, is just that smart_select has some outdated code. It's
actually been fixed in master, just not posted to PyPI yet, so you'll
need to install it directly from git, as mentioned in the fourth comment
here: https://github.com/digi604/django-smart-selects/issues/70

> *Attemp #3 - putting smart_packages in the root folder of my Django project
> *
> When smart_selects is placed here, again I can import smart_selects from
> the Django shell  but not access smart-selects.urls :
> 
> Type "help", "copyright", "credits" or "license" for more information.
> (InteractiveConsole)
 import smart_selects
 dir(smart_selects)
> ['__builtins__', '__doc__', '__file__', '__name__', '__package__',
> '__path__', 'models']
 smart_selects.urls
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: 'module' object has no attribute 'urls'
> 
> 
> Finally it works if i do this:
> 
 import smart_selects.urls
 smart_selects.urls
>  '/Users/chas/Projects/switch/djangoroot/smart_selects/urls.pyc'>

The difference between the "doesn't work" and "works" versions here
doesn't have anything to do with how you've installed the code, it's
simply how submodule imports work in Python. Submodules are never
automatically imported unless you import them explicitly.

> However, I really don't like putting apps here since they clutter up my
> own development (and would rather keep 3rd party apps in django/contrib). 

Yes, this installation technique will work fine, but it clutters things
up and makes it harder to upgrade in the future.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/54F3C467.7080203%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature