Re: Selecting data from intermediate table in prefetch_related results in duplicate rows

[Tomáš Ehrlich]

I completely forgot about `extra` method and it seems it does solve the problem:

roles = Prefetch(
‘users',
queryset=User.objects.extra(select={'role': 'users_role.role'})
)
qs = Project.objects.prefetch_related(roles))

However, I’m still curious why `annotate` doesn’t work when used inside 
`prefetch_related`.


1. Is it bug? Should it work in the same way as when used outside 
`prefetch_related`?
2. Is there a way to avoid `extra` method? It should be avoided at all cost, 
after all.


Thank you in advance!
   Tom


> 20. 5. 2018 v 11:41, Tomáš Ehrlich <tomas.ehrl...@gmail.com>:
> 
> Hello,
> I have a two models (User, Project) in m2m relationship with intermediate 
> (Role) table.
> 
> When I’m selecting all users in project and I also want to select 
> corresponding role, I simply annotate one field from Role table using F 
> expression:
> 
> Users.objects.all().filter(projects__name=‘Django’).annotate(role=F(‘roles__role’))
> 
> Annotate in this case doesn’t create new join, because `roles` table is 
> already joined to filter on data from `projects` table. This works well.
> 
> 
> However, I tried to same in `prefetch_related` and I’m getting duplicate 
> rows, because there’s a new JOIN statement added. (Usecase: Selecting all 
> projects in DB with all users per project)
> 
> The SQL statement with `prefetch_related`, but without `annotate` looks like 
> this:
> 
> roles = Prefetch(
> ‘users',
> queryset=User.objects.all()
> )
> qs = Project.objects.prefetch_related(roles)
> 
> SELECT
>   ("users_role"."project_id") AS "_prefetch_related_val_project_id",
>   — other fields here
> FROM "users_user"
>   INNER JOIN "users_role" ON ("users_user"."id" = "users_role"."user_id")
> WHERE "users_role"."project_id” IN (1, 2, 3, 4, 5)
> 
> 
> As you can see, the table `users_role` is already joined, so I’m basically 
> looking for Django ORM expression which generates following SQL query:
> 
> SELECT
>   ("users_role"."project_id") AS "_prefetch_related_val_project_id”,
>   “users_role_.”role”,
>   — other fields here
> FROM "users_user"
>   INNER JOIN "users_role" ON ("users_user"."id" = "users_role"."user_id")
> WHERE "users_role"."project_id” IN (1, 2, 3, 4, 5)
> 
> 
> Unfortunatelly, following expression generates incorrect SQL:
> 
> roles = Prefetch(
> ‘users',
> queryset=User.objects.all().annotate(role=F('roles__role'))
> )
> qs = Project.objects.prefetch_related(roles)
> 
> SELECT
>   ("users_role"."project_id") AS "_prefetch_related_val_project_id",
>   "users_role"."role" AS “role”,
>   — other fields here
> FROM "users_user"
>   LEFT OUTER JOIN "users_role" ON ("users_user"."id" = "users_role"."user_id")
>   INNER JOIN "users_role" T3 ON ("users_user"."id" = T3."user_id")
> WHERE T3."project_id" IN
>   (1, 2, 3, 4, 5)
> 
> The extra `left outer join` causes duplicate entries.
> 
> 
> I’ve found one ticket (https://code.djangoproject.com/ticket/27144 
> <https://code.djangoproject.com/ticket/27144>) which seems to be relevant, 
> but it’s old and closed.
> 
> Any ideas? Is it bug or is there really a reason to include extra JOIN? I’m 
> not very skilled in relational algebra.
> 
> Thank you in advance!
> 
> 
> Cheers,
>Tom
> 
> 
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/EFFACF5E-06B4-4F68-B388-83E30D8A3453%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Selecting data from intermediate table in prefetch_related results in duplicate rows

[Tomáš Ehrlich]

Hello,
I have a two models (User, Project) in m2m relationship with intermediate 
(Role) table.

When I’m selecting all users in project and I also want to select corresponding 
role, I simply annotate one field from Role table using F expression:

Users.objects.all().filter(projects__name=‘Django’).annotate(role=F(‘roles__role’))

Annotate in this case doesn’t create new join, because `roles` table is already 
joined to filter on data from `projects` table. This works well.


However, I tried to same in `prefetch_related` and I’m getting duplicate rows, 
because there’s a new JOIN statement added. (Usecase: Selecting all projects in 
DB with all users per project)

The SQL statement with `prefetch_related`, but without `annotate` looks like 
this:

roles = Prefetch(
‘users',
queryset=User.objects.all()
)
qs = Project.objects.prefetch_related(roles)

SELECT
  ("users_role"."project_id") AS "_prefetch_related_val_project_id",
  — other fields here
FROM "users_user"
  INNER JOIN "users_role" ON ("users_user"."id" = "users_role"."user_id")
WHERE "users_role"."project_id” IN (1, 2, 3, 4, 5)


As you can see, the table `users_role` is already joined, so I’m basically 
looking for Django ORM expression which generates following SQL query:

SELECT
  ("users_role"."project_id") AS "_prefetch_related_val_project_id”,
  “users_role_.”role”,
  — other fields here
FROM "users_user"
  INNER JOIN "users_role" ON ("users_user"."id" = "users_role"."user_id")
WHERE "users_role"."project_id” IN (1, 2, 3, 4, 5)


Unfortunatelly, following expression generates incorrect SQL:

roles = Prefetch(
‘users',
queryset=User.objects.all().annotate(role=F('roles__role'))
)
qs = Project.objects.prefetch_related(roles)

SELECT
  ("users_role"."project_id") AS "_prefetch_related_val_project_id",
  "users_role"."role" AS “role”,
  — other fields here
FROM "users_user"
  LEFT OUTER JOIN "users_role" ON ("users_user"."id" = "users_role"."user_id")
  INNER JOIN "users_role" T3 ON ("users_user"."id" = T3."user_id")
WHERE T3."project_id" IN
  (1, 2, 3, 4, 5)

The extra `left outer join` causes duplicate entries.


I’ve found one ticket (https://code.djangoproject.com/ticket/27144 
) which seems to be relevant, but 
it’s old and closed.

Any ideas? Is it bug or is there really a reason to include extra JOIN? I’m not 
very skilled in relational algebra.

Thank you in advance!


Cheers,
   Tom



-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/FCD600AD-E1DE-495D-9C6B-B3E3A3F693D9%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: Window expression inside Subquery

[Tomáš Ehrlich]

It seems to be a bug in Django. Here's the ticket 
(https://code.djangoproject.com/ticket/29172) with
patch and tests if anyone is interested in this topic.

Cheers,
   Tom


Dne čtvrtek 1. března 2018 7:38:13 UTC+1 Tomáš Ehrlich napsal(a):
>
> Hey folks,
> I’m getting an AttributeError: 'NoneType' object has no attribute 
> ‘relabeled_clone'
> when using Window expression inside Subquery:
>
> Message.objects
> .filter(pk__in=Subquery(
> Message.objects
> .annotate(latest_pk=Window(
> expression=FirstValue('pk'),
> partition_by=[F('conversation_id')],
> order_by=F('date').desc(),
> ))
> .values('latest_pk')
> ))
>
> I would like to translate this SQL statement to Django ORM:
>
> SELECT
>   "conversations_message"."id",
>   "conversations_message"."conversation_id",
>   "conversations_message"."author_id",
>   "conversations_message"."content",
>   "conversations_message"."date"
> FROM "conversations_message"
> WHERE "conversations_message"."id" IN (
>   SELECT first_value("id") OVER (PARTITION BY "conversation_id" ORDER BY 
> "date" DESC)
>   FROM conversations_message
> )
>
> I tested SQL statement and it works. I’m trying to select all 
> conversations in DB
> and prefetch latest message in each conversation.
>
>
> I’ve found this note about using aggregates in Subqueries:
>
> https://docs.djangoproject.com/en/2.0/ref/models/expressions/#using-aggregates-within-a-subquery-expression
> but it doesn’t seem to be related to my case.
>
> I could however replace Window function with Max aggregate:
>
> Message.objects.filter(pk__in=Subquery(
> Message.objects
> .order_by()
> .values('conversation')
> .annotate(latest=Max('id'))
> .values('latest')
> )
> )
>
> This works too, but I don’t feel very comfortable using Max on `id`.
>
>
> Related question: Is there a better way to prefetch latest related items? 
> Both in Django and raw SQL.
>
> Thanks in advance!
>
>
> Cheers,
>Tom
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/9558fa54-089d-4dd3-8e60-09fdf63712ec%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Window expression inside Subquery

[Tomáš Ehrlich]

Hey folks,
I’m getting an AttributeError: 'NoneType' object has no attribute 
‘relabeled_clone'
when using Window expression inside Subquery:

Message.objects
.filter(pk__in=Subquery(
Message.objects
.annotate(latest_pk=Window(
expression=FirstValue('pk'),
partition_by=[F('conversation_id')],
order_by=F('date').desc(),
))
.values('latest_pk')
))

I would like to translate this SQL statement to Django ORM:

SELECT
  "conversations_message"."id",
  "conversations_message"."conversation_id",
  "conversations_message"."author_id",
  "conversations_message"."content",
  "conversations_message"."date"
FROM "conversations_message"
WHERE "conversations_message"."id" IN (
  SELECT first_value("id") OVER (PARTITION BY "conversation_id" ORDER BY "date" 
DESC)
  FROM conversations_message
)

I tested SQL statement and it works. I’m trying to select all conversations in 
DB
and prefetch latest message in each conversation.


I’ve found this note about using aggregates in Subqueries:
https://docs.djangoproject.com/en/2.0/ref/models/expressions/#using-aggregates-within-a-subquery-expression
 

but it doesn’t seem to be related to my case.

I could however replace Window function with Max aggregate:

Message.objects.filter(pk__in=Subquery(
Message.objects
.order_by()
.values('conversation')
.annotate(latest=Max('id'))
.values('latest')
)
)

This works too, but I don’t feel very comfortable using Max on `id`.


Related question: Is there a better way to prefetch latest related items? Both 
in Django and raw SQL.

Thanks in advance!


Cheers,
   Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/DD9FBE1F-E4FB-483E-B85B-BBC2BC36F6BB%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [django-channels] Testing events from post_save signal

[Tomáš Ehrlich]

Kudos for you and all contributors! It’s really amazing.

I’m going to send few PRs to django-channels as a payback :)

Cheers,
   Tom

> 25. 2. 2018 v 19:46, Andrew Godwin <and...@aeracode.org>:
> 
> No problem - glad everything else seems to work alright! Getting async 
> testing working well has been a long, hard road :)
> 
> Andrew
> 
> On Sun, Feb 25, 2018 at 10:37 AM, Tomáš Ehrlich <tomas.ehrl...@gmail.com 
> <mailto:tomas.ehrl...@gmail.com>> wrote:
> Of course!
> 
> It works now perfectly, thank you. Sorry I missed that in docs.
> 
> Cheers,
>Tom
> 
>> 25. 2. 2018 v 19:12, Andrew Godwin <and...@aeracode.org 
>> <mailto:and...@aeracode.org>>:
>> 
>> I think the change you need to make is swapping in database_sync_to_async 
>> rather than sync_to_async - see here: 
>> http://channels.readthedocs.io/en/latest/topics/databases.html 
>> <http://channels.readthedocs.io/en/latest/topics/databases.html>
>> 
>> Andrew
>> 
>> On Sun, Feb 25, 2018 at 7:14 AM, Tomáš Ehrlich <tomas.ehrl...@gmail.com 
>> <mailto:tomas.ehrl...@gmail.com>> wrote:
>> Here's the gist 
>> (https://gist.github.com/tricoder42/af3d0337c1b33d82c1b32d12bd0265ec 
>> <https://gist.github.com/tricoder42/af3d0337c1b33d82c1b32d12bd0265ec>) with 
>> consumer.
>> 
>> Dne neděle 25. února 2018 15:37:19 UTC+1 Tomáš Ehrlich napsal(a):
>> Hello,
>> I’ve just migrated my project to django-channels 2.x. Thanks to everyone 
>> involved!
>> 
>> I’m trying to write a test for a consumer.
>> I have a post_save signal receiver, which sends a message to a group. As I 
>> understand,
>> I need to wrap `group_send` with `async_to_sync` because django signals 
>> can’t be
>> async functions:
>> 
>> def notify_on_model_changes(model):
>> from django.contrib.contenttypes.models import ContentType
>> ct = ContentType.objects.get_for_model(model)
>> model_label = '.'.join([ct.app_label, ct.model])
>> 
>> channel_layer = get_channel_layer()
>> group_send = async_to_sync(channel_layer.group_send)
>> 
>> def receiver(sender, instance, **kwargs):
>> payload = {
>> 'type': 'model.changed',
>> 'pk': instance.pk <http://instance.pk/>,
>> 'model': model_label
>> }
>> group_send(f'django.{model_label}', payload)
>> 
>> post_save.connect(receiver, sender=model, weak=False,
>>   dispatch_uid=f'django.{model_label}’)
>> 
>> # in AppConfig.ready:
>> # notify_on_model_changes(Conversation)
>> 
>> 
>> 
>> My test suite, however, is async function:
>> 
>> @pytest.fixture
>> async def communicator():
>> communicator = WebsocketCommunicator(GraphqlSubcriptionConsumer, "/")
>> await communicator.connect()
>> yield communicator
>> await communicator.disconnect()
>> 
>> async def test_subscription_start(communicator):
>> def make_conversation():
>> return Conversation.objects.create()
>> 
>> # function body truncated
>> # subscribe for changes in Conversation model
>> await communicator.send_json_to(data)
>> 
>> conversation = await sync_to_async(make_conversation)()
>> response = await communicator.receive_json_from()
>> assert response['type'] == ‘data'
>> 
>> I can’t use `Conversation.objects.create()` directly, because it uses 
>> `async_to_sync`.
>> First, I need to convert it to async and await the result. I kinda feel this 
>> is hackish
>> jumping from async to sync and back to async, but so far everything works as 
>> expected
>> and test works.
>> 
>> 
>> Here comes the punchline:
>> The tests fail to teardown cleanly, because apparently there’s hanging DB 
>> connection
>> and after a while pytest just fails with `There is 1 other session using the 
>> database.`.
>> 
>> Breadcrumbs:
>> 1. If I comment out last three lines of test (make_conversations and waiting 
>> for result),
>> the test exits cleanly - seems like there’s no problem with passing 
>> `sync_to_async` function
>> to `post_save.connect`.
>> 
>> 2. If I create `async_make = sync_to_async(make_conversation)`, but don’t 
>> call it at all,
>> the test exists cleanly - I thought that there might be problem with calling 
>> `async_to_sync`
>> inside code wrapped with `sync_to_async`.
>> 
>> 
>> I suspect there’s

Re: [django-channels] Testing events from post_save signal

[Tomáš Ehrlich]

Of course!

It works now perfectly, thank you. Sorry I missed that in docs.

Cheers,
   Tom

> 25. 2. 2018 v 19:12, Andrew Godwin <and...@aeracode.org>:
> 
> I think the change you need to make is swapping in database_sync_to_async 
> rather than sync_to_async - see here: 
> http://channels.readthedocs.io/en/latest/topics/databases.html 
> <http://channels.readthedocs.io/en/latest/topics/databases.html>
> 
> Andrew
> 
> On Sun, Feb 25, 2018 at 7:14 AM, Tomáš Ehrlich <tomas.ehrl...@gmail.com 
> <mailto:tomas.ehrl...@gmail.com>> wrote:
> Here's the gist 
> (https://gist.github.com/tricoder42/af3d0337c1b33d82c1b32d12bd0265ec 
> <https://gist.github.com/tricoder42/af3d0337c1b33d82c1b32d12bd0265ec>) with 
> consumer.
> 
> Dne neděle 25. února 2018 15:37:19 UTC+1 Tomáš Ehrlich napsal(a):
> Hello,
> I’ve just migrated my project to django-channels 2.x. Thanks to everyone 
> involved!
> 
> I’m trying to write a test for a consumer.
> I have a post_save signal receiver, which sends a message to a group. As I 
> understand,
> I need to wrap `group_send` with `async_to_sync` because django signals can’t 
> be
> async functions:
> 
> def notify_on_model_changes(model):
> from django.contrib.contenttypes.models import ContentType
> ct = ContentType.objects.get_for_model(model)
> model_label = '.'.join([ct.app_label, ct.model])
> 
> channel_layer = get_channel_layer()
> group_send = async_to_sync(channel_layer.group_send)
> 
> def receiver(sender, instance, **kwargs):
> payload = {
> 'type': 'model.changed',
> 'pk': instance.pk <http://instance.pk/>,
> 'model': model_label
> }
> group_send(f'django.{model_label}', payload)
> 
> post_save.connect(receiver, sender=model, weak=False,
>   dispatch_uid=f'django.{model_label}’)
> 
> # in AppConfig.ready:
> # notify_on_model_changes(Conversation)
> 
> 
> 
> My test suite, however, is async function:
> 
> @pytest.fixture
> async def communicator():
> communicator = WebsocketCommunicator(GraphqlSubcriptionConsumer, "/")
> await communicator.connect()
> yield communicator
> await communicator.disconnect()
> 
> async def test_subscription_start(communicator):
> def make_conversation():
> return Conversation.objects.create()
> 
> # function body truncated
> # subscribe for changes in Conversation model
> await communicator.send_json_to(data)
> 
> conversation = await sync_to_async(make_conversation)()
> response = await communicator.receive_json_from()
> assert response['type'] == ‘data'
> 
> I can’t use `Conversation.objects.create()` directly, because it uses 
> `async_to_sync`.
> First, I need to convert it to async and await the result. I kinda feel this 
> is hackish
> jumping from async to sync and back to async, but so far everything works as 
> expected
> and test works.
> 
> 
> Here comes the punchline:
> The tests fail to teardown cleanly, because apparently there’s hanging DB 
> connection
> and after a while pytest just fails with `There is 1 other session using the 
> database.`.
> 
> Breadcrumbs:
> 1. If I comment out last three lines of test (make_conversations and waiting 
> for result),
> the test exits cleanly - seems like there’s no problem with passing 
> `sync_to_async` function
> to `post_save.connect`.
> 
> 2. If I create `async_make = sync_to_async(make_conversation)`, but don’t 
> call it at all,
> the test exists cleanly - I thought that there might be problem with calling 
> `async_to_sync`
> inside code wrapped with `sync_to_async`.
> 
> 
> I suspect there’s a hanging db connection which isn’t cleaned and/or garbage 
> collected.
> I would also appreciate any comments about structure of such tests - is there 
> cleaner way
> test django signals inside async test cases?
> 
> 
> Cheers,
>Tom
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users+unsubscr...@googlegroups.com 
> <mailto:django-users+unsubscr...@googlegroups.com>.
> To post to this group, send email to django-users@googlegroups.com 
> <mailto:django-users@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-users 
> <https://groups.google.com/group/django-users>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/af745c4f-7571-40d3-b738-1593d4d75bbb%40googlegroups.com
&

Re: [django-channels] Testing events from post_save signal

[Tomáš Ehrlich]

Here's the gist 
(https://gist.github.com/tricoder42/af3d0337c1b33d82c1b32d12bd0265ec) with 
consumer.

Dne neděle 25. února 2018 15:37:19 UTC+1 Tomáš Ehrlich napsal(a):
>
> Hello,
> I’ve just migrated my project to django-channels 2.x. Thanks to everyone 
> involved!
>
> I’m trying to write a test for a consumer.
> I have a post_save signal receiver, which sends a message to a group. As I 
> understand,
> I need to wrap `group_send` with `async_to_sync` because django signals 
> can’t be
> async functions:
>
> def notify_on_model_changes(model):
> from django.contrib.contenttypes.models import ContentType
> ct = ContentType.objects.get_for_model(model)
> model_label = '.'.join([ct.app_label, ct.model])
>
> channel_layer = get_channel_layer()
> group_send = async_to_sync(channel_layer.group_send)
>
> def receiver(sender, instance, **kwargs):
> payload = {
> 'type': 'model.changed',
> 'pk': instance.pk,
> 'model': model_label
> } 
> group_send(f'django.{model_label}', payload)
>
> post_save.connect(receiver, sender=model, weak=False,
>   dispatch_uid=f'django.{model_label}’)
>
> # in AppConfig.ready:
> # notify_on_model_changes(Conversation)
>
>
>
> My test suite, however, is async function:
>
> @pytest.fixture
> async def communicator():
> communicator = WebsocketCommunicator(GraphqlSubcriptionConsumer, "/")
> await communicator.connect()
> yield communicator
> await communicator.disconnect()
> 
> async def test_subscription_start(communicator):
> def make_conversation():
> return Conversation.objects.create()
>
> # function body truncated
> # subscribe for changes in Conversation model
> await communicator.send_json_to(data)
>
> conversation = await sync_to_async(make_conversation)()
> response = await communicator.receive_json_from()
> assert response['type'] == ‘data'
>
> I can’t use `Conversation.objects.create()` directly, because it uses 
> `async_to_sync`.
> First, I need to convert it to async and await the result. I kinda feel 
> this is hackish
> jumping from async to sync and back to async, but so far everything works 
> as expected
> and test works.
>
>
> *Here comes the punchline:*
> The tests fail to teardown cleanly, because apparently there’s hanging DB 
> connection
> and after a while pytest just fails with `There is 1 other session using 
> the database.`.
>
> *Breadcrumbs*:
> 1. If I comment out last three lines of test (make_conversations and 
> waiting for result),
> the test exits cleanly - seems like there’s no problem with passing 
> `sync_to_async` function
> to `post_save.connect`.
>
> 2. If I create `async_make = sync_to_async(make_conversation)`, but don’t 
> call it at all,
> the test exists cleanly - I thought that there might be problem with 
> calling `async_to_sync`
> inside code wrapped with `sync_to_async`.
>
>
> I suspect there’s a hanging db connection which isn’t cleaned and/or 
> garbage collected.
> I would also appreciate any comments about structure of such tests - is 
> there cleaner way
> test django signals inside async test cases?
>
>
> Cheers,
>Tom
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/af745c4f-7571-40d3-b738-1593d4d75bbb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[django-channels] Testing events from post_save signal

[Tomáš Ehrlich]

Hello,
I’ve just migrated my project to django-channels 2.x. Thanks to everyone 
involved!

I’m trying to write a test for a consumer.
I have a post_save signal receiver, which sends a message to a group. As I 
understand,
I need to wrap `group_send` with `async_to_sync` because django signals can’t be
async functions:

def notify_on_model_changes(model):
from django.contrib.contenttypes.models import ContentType
ct = ContentType.objects.get_for_model(model)
model_label = '.'.join([ct.app_label, ct.model])

channel_layer = get_channel_layer()
group_send = async_to_sync(channel_layer.group_send)

def receiver(sender, instance, **kwargs):
payload = {
'type': 'model.changed',
'pk': instance.pk,
'model': model_label
}
group_send(f'django.{model_label}', payload)

post_save.connect(receiver, sender=model, weak=False,
  dispatch_uid=f'django.{model_label}’)

# in AppConfig.ready:
# notify_on_model_changes(Conversation)



My test suite, however, is async function:

@pytest.fixture
async def communicator():
communicator = WebsocketCommunicator(GraphqlSubcriptionConsumer, "/")
await communicator.connect()
yield communicator
await communicator.disconnect()

async def test_subscription_start(communicator):
def make_conversation():
return Conversation.objects.create()

# function body truncated
# subscribe for changes in Conversation model
await communicator.send_json_to(data)

conversation = await sync_to_async(make_conversation)()
response = await communicator.receive_json_from()
assert response['type'] == ‘data'

I can’t use `Conversation.objects.create()` directly, because it uses 
`async_to_sync`.
First, I need to convert it to async and await the result. I kinda feel this is 
hackish
jumping from async to sync and back to async, but so far everything works as 
expected
and test works.


Here comes the punchline:
The tests fail to teardown cleanly, because apparently there’s hanging DB 
connection
and after a while pytest just fails with `There is 1 other session using the 
database.`.

Breadcrumbs:
1. If I comment out last three lines of test (make_conversations and waiting 
for result),
the test exits cleanly - seems like there’s no problem with passing 
`sync_to_async` function
to `post_save.connect`.

2. If I create `async_make = sync_to_async(make_conversation)`, but don’t call 
it at all,
the test exists cleanly - I thought that there might be problem with calling 
`async_to_sync`
inside code wrapped with `sync_to_async`.


I suspect there’s a hanging db connection which isn’t cleaned and/or garbage 
collected.
I would also appreciate any comments about structure of such tests - is there 
cleaner way
test django signals inside async test cases?


Cheers,
   Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/6F600904-9697-47A6-A813-9FBB6A69E7C9%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [django-channels] Running multiple daphne servers

[Tomáš Ehrlich]

Ah, problem solved.

This line forces db=0 in asgi_redis 1.x:
https://github.com/django/channels_redis/blob/1.x/asgi_redis/core.py#L566 
<https://github.com/django/channels_redis/blob/1.x/asgi_redis/core.py#L566>

in 2.x version it seems to be fixed.

Thanks again! :)

> 19. 2. 2018 v 21:02, Tomáš Ehrlich <tomas.ehrl...@gmail.com>:
> 
> You’re absolutely right, using different prefix solves the problem. 
> asgi_redis seems to be ignoring
> my connection_kwargs where I set the db.
> 
> Connection_kwargs should be set like this, right?
> 
> REDIS = {
> "hosts": [(os.environ.get('REDIS_HOST'), 6379)],
> "connection_kwargs": {
> "db": os.environ.get('REDIS_DB'),
> }
> }
> 
> CHANNEL_LAYERS = {
> "default": {
> "BACKEND": "asgi_redis.RedisChannelLayer",
> "ROUTING": "birdview_api.channels.routing.channel_routing",
> "CONFIG": REDIS,
> },
> }
> 
> I double checked the env vars and also source code of asgi_redis, but still 
> no luck.
> 
> Anyway, this question was already aswered. Different isntances need to have 
> different prefix or db.
> 
> 
> Thank you Andrew! going to update to channels 2.x soon
> 
> Cheers,
>Tom
> 
>> 19. 2. 2018 v 19:10, Andrew Godwin <and...@aeracode.org 
>> <mailto:and...@aeracode.org>>:
>> 
>> Presuming you are using Channels/Daphne 1, then the channel layer 
>> configuration is what determines what handles the requests.
>> 
>> If you're seeing environments answer each other's requests, check they 
>> really are using different Redis databases, and consider changing the prefix 
>> setting on the channel layer instead.
>> 
>> Andrew
>> 
>> On Mon, Feb 19, 2018 at 8:34 AM, Tomáš Ehrlich <tomas.ehrl...@gmail.com 
>> <mailto:tomas.ehrl...@gmail.com>>wrote:
>> Hello everyone,
>> I'm running two instances of Daphne on my server (one per environment, 
>> production/staging).
>> I'm using UNIX sockets behind nginx, but production requests are sent to 
>> staging and vice versa.
>> 
>> Workers and Daphne are using the same settings per environment (I'm using 
>> Redis as a channel
>> layer and each environment uses it's own DB), but how Daphne actually knows 
>> which workers
>> belongs to her?
>> 
>> 
>> Thank you in advance
>> 
>> 
>> Cheers,
>>Tom
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users+unsubscr...@googlegroups.com 
>> <mailto:django-users+unsubscr...@googlegroups.com>.
>> To post to this group, send email to django-users@googlegroups.com 
>> <mailto:django-users@googlegroups.com>.
>> Visit this group at https://groups.google.com/group/django-users 
>> <https://groups.google.com/group/django-users>.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/66fabb88-4350-4275-8d4e-25f30e4e3b00%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-users/66fabb88-4350-4275-8d4e-25f30e4e3b00%40googlegroups.com?utm_medium=email_source=footer>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
>> 
>> 
>> --
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "Django users" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/django-users/wfxiPfqUPnk/unsubscribe 
>> <https://groups.google.com/d/topic/django-users/wfxiPfqUPnk/unsubscribe>.
>> To unsubscribe from this group and all its topics, send an email to 
>> django-users+unsubscr...@googlegroups.com 
>> <mailto:django-users+unsubscr...@googlegroups.com>.
>> To post to this group, send email to django-users@googlegroups.com 
>> <mailto:django-users@googlegroups.com>.
>> Visit this group at https://groups.google.com/group/django-users 
>> <https://groups.google.com/group/django-users>.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/CAFwN1urrLN7xVc3EM0-7rEeB2DyLo21JmB9bRiYQioYKq6bEfA%40mail.gmail.com
>>  
>> <https://groups.google.com/d/msgid/django-users/CAFwN1urrLN7xVc3EM0-7rEeB2DyLo21JmB9bRiYQioYKq6bEfA%40mail.gmail.com?utm_medium=email_source=footer>.
>> For more options, visit https://groups.google.com/d/optout 
>> <https://groups.google.com/d/optout>.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/736810F0-AA38-499C-B371-E9C818B43B68%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: [django-channels] Running multiple daphne servers

[Tomáš Ehrlich]

You’re absolutely right, using different prefix solves the problem. asgi_redis 
seems to be ignoring
my connection_kwargs where I set the db.

Connection_kwargs should be set like this, right?

REDIS = {
"hosts": [(os.environ.get('REDIS_HOST'), 6379)],
"connection_kwargs": {
"db": os.environ.get('REDIS_DB'),
}
}

CHANNEL_LAYERS = {
"default": {
"BACKEND": "asgi_redis.RedisChannelLayer",
"ROUTING": "birdview_api.channels.routing.channel_routing",
"CONFIG": REDIS,
},
}

I double checked the env vars and also source code of asgi_redis, but still no 
luck.

Anyway, this question was already aswered. Different isntances need to have 
different prefix or db.


Thank you Andrew! going to update to channels 2.x soon

Cheers,
   Tom

> 19. 2. 2018 v 19:10, Andrew Godwin <and...@aeracode.org>:
> 
> Presuming you are using Channels/Daphne 1, then the channel layer 
> configuration is what determines what handles the requests.
> 
> If you're seeing environments answer each other's requests, check they really 
> are using different Redis databases, and consider changing the prefix setting 
> on the channel layer instead.
> 
> Andrew
> 
> On Mon, Feb 19, 2018 at 8:34 AM, Tomáš Ehrlich <tomas.ehrl...@gmail.com 
> <mailto:tomas.ehrl...@gmail.com>>wrote:
> Hello everyone,
> I'm running two instances of Daphne on my server (one per environment, 
> production/staging).
> I'm using UNIX sockets behind nginx, but production requests are sent to 
> staging and vice versa.
> 
> Workers and Daphne are using the same settings per environment (I'm using 
> Redis as a channel
> layer and each environment uses it's own DB), but how Daphne actually knows 
> which workers
> belongs to her?
> 
> 
> Thank you in advance
> 
> 
> Cheers,
>Tom
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users+unsubscr...@googlegroups.com 
> <mailto:django-users+unsubscr...@googlegroups.com>.
> To post to this group, send email to django-users@googlegroups.com 
> <mailto:django-users@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-users 
> <https://groups.google.com/group/django-users>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/66fabb88-4350-4275-8d4e-25f30e4e3b00%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/django-users/66fabb88-4350-4275-8d4e-25f30e4e3b00%40googlegroups.com?utm_medium=email_source=footer>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.
> 
> 
> --
> You received this message because you are subscribed to a topic in the Google 
> Groups "Django users" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/django-users/wfxiPfqUPnk/unsubscribe 
> <https://groups.google.com/d/topic/django-users/wfxiPfqUPnk/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to 
> django-users+unsubscr...@googlegroups.com 
> <mailto:django-users+unsubscr...@googlegroups.com>.
> To post to this group, send email to django-users@googlegroups.com 
> <mailto:django-users@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-users 
> <https://groups.google.com/group/django-users>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/CAFwN1urrLN7xVc3EM0-7rEeB2DyLo21JmB9bRiYQioYKq6bEfA%40mail.gmail.com
>  
> <https://groups.google.com/d/msgid/django-users/CAFwN1urrLN7xVc3EM0-7rEeB2DyLo21JmB9bRiYQioYKq6bEfA%40mail.gmail.com?utm_medium=email_source=footer>.
> For more options, visit https://groups.google.com/d/optout 
> <https://groups.google.com/d/optout>.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/8767E794-BAD3-4CF3-84CA-29A04BC6EC6A%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

[django-channels] Running multiple daphne servers

[Tomáš Ehrlich]

Hello everyone,
I'm running two instances of Daphne on my server (one per environment, 
production/staging). 
I'm using UNIX sockets behind nginx, but production requests are sent to 
staging and vice versa.

Workers and Daphne are using the same settings per environment (I'm using 
Redis as a channel
layer and each environment uses it's own DB), but how Daphne actually knows 
which workers
belongs to her?


Thank you in advance


Cheers,
   Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/66fabb88-4350-4275-8d4e-25f30e4e3b00%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [django-channels] post-process message from Group

[Tomáš Ehrlich]

Hey Andrew,
that's great! I'll take a look today. For some reason, I didn't receive 
your response so I came up with a bit inefficient solution described in 
this 
[gist](https://gist.github.com/tricoder42/af3d0337c1b33d82c1b32d12bd0265ec).

I basically have another store with subscribers and then broadcast message 
manually.

Anyway, going to wipe it out and start over using Channels 2 :)


Thanks for a great library!

Cheers,
   Tom

Dne středa 17. ledna 2018 19:43:23 UTC+1 Andrew Godwin napsal(a):
>
> Hi Tom,
>
> In Channels 1, because of the way Groups work you cannot do 
> post-processing. The only fix for this is a complete rearchitecture which 
> is underway in Channels 2, where you get the ability to act on group 
> messages before they get sent back to the consumer.
>
> Channels 2 is not yet ready for production, but is close, and if you want 
> to start playing around with it now you can: 
> http://channels.readthedocs.io/en/2.0/
>
> Andrew
>
> On Wed, Jan 17, 2018 at 5:42 AM, Tomáš Ehrlich <tomas@gmail.com 
> > wrote:
>
>> Hello everyone,
>> I’m trying to connect django-channels with GraphQL subscriptions (using 
>> graphene and underlying graphql-core library).
>>
>> I’m basically doing sth similar to example in [docs](
>> https://channels.readthedocs.io/en/latest/concepts.html#groups):
>>
>> - I setup listener of post_save signal, passing instance PK to Group
>> - When subscription is requested, I add reply_channel to the Group I want 
>> to listen
>>
>> The problem is that with GraphQL I don’t know in advance what data client 
>> requests. 
>> Compared to DRF where I could simply serialize instance in post_save 
>> listener and send
>> serialized data to Group, here I’m only sending instance PK (each client 
>> might request different fields from
>> model and I can’t pass instances through channel). Ideally, instead of 
>> adding reply_channel
>> to the Group directly, I would like to add custom callback, which 1) gets 
>> instance by PK 2) serialize it using
>> given GraphQL query and finally 3) send data to reply_channel.
>>
>>
>> Other possible workaround: list channels in a group
>>
>> I think I could save query (== serializer) to channel_session and execute 
>> it (== serialize instance) when message is received. 
>> Then I need to add a consumer for a Group and for that I would need to 
>> list all channels in a Group.
>>
>>
>> Any ideas? GraphQL is still new in Djangoland and especially 
>> subscriptions are sth which needs to be figured out…
>>
>>
>> Thank you in advance!
>>
>> Cheers,
>>Tom
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com .
>> To post to this group, send email to django...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/3506375B-E9B4-4F99-96D0-AD18F78B8CE9%40gmail.com
>>  
>> <https://groups.google.com/d/msgid/django-users/3506375B-E9B4-4F99-96D0-AD18F78B8CE9%40gmail.com?utm_medium=email_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/fd7ebbdf-c278-42a5-bd6c-f5d7ea3c9356%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[django-channels] post-process message from Group

[Tomáš Ehrlich]

Hello everyone,
I’m trying to connect django-channels with GraphQL subscriptions (using 
graphene and underlying graphql-core library).

I’m basically doing sth similar to example in 
[docs](https://channels.readthedocs.io/en/latest/concepts.html#groups 
):

- I setup listener of post_save signal, passing instance PK to Group
- When subscription is requested, I add reply_channel to the Group I want to 
listen

The problem is that with GraphQL I don’t know in advance what data client 
requests.
Compared to DRF where I could simply serialize instance in post_save listener 
and send
serialized data to Group, here I’m only sending instance PK (each client might 
request different fields from
model and I can’t pass instances through channel). Ideally, instead of adding 
reply_channel
to the Group directly, I would like to add custom callback, which 1) gets 
instance by PK 2) serialize it using
given GraphQL query and finally 3) send data to reply_channel.


Other possible workaround: list channels in a group

I think I could save query (== serializer) to channel_session and execute it 
(== serialize instance) when message is received.
Then I need to add a consumer for a Group and for that I would need to list all 
channels in a Group.


Any ideas? GraphQL is still new in Djangoland and especially subscriptions are 
sth which needs to be figured out…


Thank you in advance!

Cheers,
   Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/3506375B-E9B4-4F99-96D0-AD18F78B8CE9%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP

Re: Research: Translations od documentation

[Tomáš Ehrlich]

Hi Tim,
I'm aware that it's possible to translate Django documentation. I also know 
that it's translated to french only.

My question is one step behind: Does it makes sense to translate Django 
documentation and (more important) keep it updated? Would anyone use it? Is 
English language the common tongue for programmers? Why is Django docs 
translated to one language only? There isn't enough translators or users 
just don't find it useful?

I asked the same question in Czech django user group and most of them 
agreed that it doesn't make sense to translate whole Django documentation. 
It would make sense to translate materials for beginners, like tutorials, 
django book, etc.


So, at the moment, I don't care about the *tool* or *process*. I'm trying 
to figure it out if there is some kind of motivation to do this stuff.

Cheers,
   Tom

Dne čtvrtek 26. února 2015 13:25:38 UTC+1 Tim Graham napsal(a):
>
> Maybe you aren't aware of the existing opportunity to translate Django's 
> documentation?
>
>
> https://docs.djangoproject.com/en/dev/internals/contributing/localizing/#documentation
>
> On Thursday, February 26, 2015 at 5:11:21 AM UTC-5, Tomáš Ehrlich wrote:
>>
>> Hello, 
>> tonight is regular python/ruby meetup in Brno (Czech republic) about 
>> documentation. Last few months I’ve been working on project concerning 
>> localization of documents. I would like to know your opinion about 
>> localization of documentation: 
>>
>> Do you think it would be useful to translate documentation of: 
>> 1) Django 
>> 2) Python 
>> 3) Any project documented using Sphinx (like numpy, scipy, request, …) 
>>
>> Three simple answers Yes/No would help me a lot. If you write a short 
>> paragraph about your opinion, I will very appreciate it. 
>>
>> Thank you 
>>
>>
>> Cheers, 
>>Tom 
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/c81d59cb-f348-4bfc-85e9-2bfe0f273d09%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Research: Translations od documentation

[Tomáš Ehrlich]

Hello,
tonight is regular python/ruby meetup in Brno (Czech republic) about 
documentation. Last few months I’ve been working on project concerning 
localization of documents. I would like to know your opinion about localization 
of documentation:

Do you think it would be useful to translate documentation of:
1) Django
2) Python
3) Any project documented using Sphinx (like numpy, scipy, request, …)

Three simple answers Yes/No would help me a lot. If you write a short paragraph 
about your opinion, I will very appreciate it.

Thank you


Cheers,
   Tom


-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/29FE9B62-B01A-4843-B4EE-D4ED7B71ECD8%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Smoke tests

[Tomáš Ehrlich]

Hi Derek,
the speed of tests isn't a problem I'm trying to solve. The problem is, the 
tests run in different environment (obviously), but I would like to run a 
subset of my tests in production environment. This subset should include 
only "safe" tests, which doesn't create objects in DB for example.

Cheers,
   Tom


Dne pondělí 26. ledna 2015 9:32:38 UTC+1 Derek napsal(a):
>
> The same article you refer to says"
>
> "A frequent characteristic of a smoke test is that it runs quickly, often 
> in the order of a few minutes and thus provides much quicker feedback and 
> faster turnaround than the running of full test suites which can take hours 
> or even days."
>
> I would think that if your current unit & functional tests run completely 
> within the order of minutes, then adding smoke tests as well may be 
> redundant.  There are also tools to help speed up those tests, which might 
> be worth investigating before adding another test layer. 
>
> On Saturday, 24 January 2015 17:22:04 UTC+2, Tomáš Ehrlich wrote:
>>
>> Hello,
>> last few weeks I’ve been thinking about implementing smoke tests into my 
>> deployment process. Last week I wrote simple test runner (
>> https://github.com/djentlemen/django-smoked), but still I’m missing 
>> methodology *what* should I test and *how*. Since smoke test has very wide 
>> definition for different types of software — 
>> https://en.wikipedia.org/wiki/Smoke_testing_(software), my idea is: 
>> After every deployment run small subset of tests with *production* settings 
>> and just check, that app was deployed successfully. If not, rollback to 
>> previous version immediately.
>>
>>
>> Few such tests might be:
>>  — check responses of few URL endpoints (like homepage)
>>  — check database settings are valid (since most tests runs on 
>> testing/development database)
>>  — check cache, email settings, etc (for the same reasons as above)
>>
>>
>> I wonder how do you test your apps? Do you use some kind of „smoke tests“ 
>> described above?
>>
>>
>> Cheers,
>>Tom
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/4b5e6643-1f69-44dd-b5b0-9ba27e55d788%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Smoke tests

[Tomáš Ehrlich]

Hello Cal,
thank you for your answer.

I'm deploying my project to AWS, so I can't replicate whole production 
environment easily. I do deployment to "release" environment which is about 
the same as production one (hosted on AWS, created with the same scripts as 
production one), but still there are some tests I need to run in this 
environment.


For example: check that email is set up correctly. I need a simple test 
which sends an email. It doesn't matter whether I catch the possible 
exception in test or in Sentry. I just want to know if it's working quickly.

Checking URL endpoint is a different problem. I can't simply check URL (as 
I do it now in django-smoked btw), because my app could be behind 
load-balancer where endpoints are updated gradually. The app itself have to 
be able to report that it works and can be plugged into load balancer. 
Right now I'm trying to figure out how to send few requests to wsgi app 
directly.

Testing deployment (whole load-balancer -> nginx -> docker stack) is a 
completely problem and I don't want to solve it with these tests.

I understand that testing app in production is anti-pattern, but after 
every release I open browser and try few links. I would like to do it 
programmatically because when I create a bunch of nodes behind 
load-balancer, I'm screwed with this approach...


And yes, it is probably a buzz word :) I ignored it for a while but when I 
heard about it from third different source, I started to digging into it...

Cheers,
   Tom

UTC+1 Cal Leeming napsal(a):
>
> Hi Tom, 
>
> Personally I'm not convinced by the concept of smoke tests in 
> production, if you have a proper development workflow, and your build 
> works in dev, then you should be confident that your build will work 
> in prod. Testing URL endpoints in prod should be part of your devops 
> testing, and kept completely separate to your application. Testing 
> database settings, cache, email etc should be part of your 
> bootstrapping, and the application should report a failure via a 
> reliable tracking system (NR, Sentry etc) of such problems. This is 
> not smoke testing, it's just good practise. Deployment testing is not 
> smoke testing, it's deployment testing. The concept of testing your 
> application in production is an anti-pattern, but the concept of 
> testing your deployment in production is a necessity, and the two 
> should be kept completely separate. 
>
> Some people use BDD, which I'm personally not a fan of, and others 
> will use tools such as Selenium and Mocha to ensure pages are working 
> correctly. If you have set up your application correctly, then you 
> will be catching these errors as they happen (e.g. Raven JS). 
>
> I don't know why "smoke tests" are suddenly becoming the new buzz 
> phrase.... 
>
> Anyway, hope this helps a bit 
>
> Cal 
>
> On Sat, Jan 24, 2015 at 3:20 PM, Tomáš Ehrlich <tomas@gmail.com 
> > wrote: 
> > Hello, 
> > last few weeks I’ve been thinking about implementing smoke tests into my 
> > deployment process. Last week I wrote simple test runner 
> > (https://github.com/djentlemen/django-smoked), but still I’m missing 
> > methodology *what* should I test and *how*. Since smoke test has very 
> wide 
> > definition for different types of software — 
> > https://en.wikipedia.org/wiki/Smoke_testing_(software), my idea is: 
> After 
> > every deployment run small subset of tests with *production* settings 
> and 
> > just check, that app was deployed successfully. If not, rollback to 
> previous 
> > version immediately. 
> > 
> > 
> > Few such tests might be: 
> >  — check responses of few URL endpoints (like homepage) 
> >  — check database settings are valid (since most tests runs on 
> > testing/development database) 
> >  — check cache, email settings, etc (for the same reasons as above) 
> > 
> > 
> > I wonder how do you test your apps? Do you use some kind of „smoke 
> tests“ 
> > described above? 
> > 
> > 
> > Cheers, 
> >Tom 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "Django users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to django-users...@googlegroups.com . 
> > To post to this group, send email to django...@googlegroups.com 
> . 
> > Visit this group at http://groups.google.com/group/django-users. 
> > To view this discussion on the web visit 
> > 
> https://groups.google.com/d/msgid/django-users/467626FA-A50D-4AF9-991C-5BD05637693F%40gmail.com.
>  
>
> > For more options, visit https://groups.google.com/d/optout. 
>

Smoke tests

[Tomáš Ehrlich]

Hello,
last few weeks I’ve been thinking about implementing smoke tests into my 
deployment process. Last week I wrote simple test runner 
(https://github.com/djentlemen/django-smoked 
), but still I’m missing 
methodology *what* should I test and *how*. Since smoke test has very wide 
definition for different types of software — 
https://en.wikipedia.org/wiki/Smoke_testing_(software) 
, my idea is: After 
every deployment run small subset of tests with *production* settings and just 
check, that app was deployed successfully. If not, rollback to previous version 
immediately.


Few such tests might be:
 — check responses of few URL endpoints (like homepage)
 — check database settings are valid (since most tests runs on 
testing/development database)
 — check cache, email settings, etc (for the same reasons as above)


I wonder how do you test your apps? Do you use some kind of „smoke tests“ 
described above?


Cheers,
   Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/467626FA-A50D-4AF9-991C-5BD05637693F%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Dutch translation of Django Tutorial

[Tomáš Ehrlich]

HI Erwin,
It won't be more than 8 pages of documentation: Overview, Installation and 
six parts of tutorial. I don't think it's possible to finish more in three 
weeks from now.

Right now is being translated the 1st part of tutorial. I'll publish it 
somewhere for review once it's ready.

Thank you in advance!

Cheers,
   Tom


Dne neděle, 26. října 2014 13:23:43 UTC+1 Erwin Sprengers napsal(a):
>
> Hi,
>
> Depending on the size, I can help to review.
>
> Erwin
>
> Op zaterdag 25 oktober 2014 13:36:11 UTC+2 schreef Tomáš Ehrlich:
>>
>> Hello everyone,
>> there are two things that going to happen in next month:
>>
>> 1. Django Under the Hood conference in Amsterdam
>> 2. (Hopefully) I’ll finish SaaS for managing translations. The first 
>> frontend build on top of gettext message catalogues will be for Sphinx 
>> documentation.
>>
>> As a result, I would like make dutch translation of django documentation. 
>> Unfortunately, as Django docs are pretty comprehensive, I don’t think it’s 
>> possible to translate everything in three weeks. Therefore, I want to focus 
>> on tutorial which is a the best starting point for beginners.
>>
>> I’ve already found a English->Dutch translator who is available.
>>
>>
>> The questions are:
>>
>> 1. Is there any Dutch translation already available? I’m aware of French 
>> and Czech translation, there are few more in transifex 
>> https://www.transifex.com/projects/p/django-docs/, but I haven’t found 
>> any references about Dutch one.
>> 2. Are there any Dutch developers willing to review the translation?
>>
>>
>> Thank you in advance,
>>Tom
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/eb74e125-c504-49d2-ae63-e903b32aff8a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Dutch translation of Django Tutorial

[Tomáš Ehrlich]

Hello everyone,
there are two things that going to happen in next month:

1. Django Under the Hood conference in Amsterdam
2. (Hopefully) I’ll finish SaaS for managing translations. The first frontend 
build on top of gettext message catalogues will be for Sphinx documentation.

As a result, I would like make dutch translation of django documentation. 
Unfortunately, as Django docs are pretty comprehensive, I don’t think it’s 
possible to translate everything in three weeks. Therefore, I want to focus on 
tutorial which is a the best starting point for beginners.

I’ve already found a English->Dutch translator who is available.


The questions are:

1. Is there any Dutch translation already available? I’m aware of French and 
Czech translation, there are few more in transifex 
https://www.transifex.com/projects/p/django-docs/, but I haven’t found any 
references about Dutch one.
2. Are there any Dutch developers willing to review the translation?


Thank you in advance,
   Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/etPan.544b8aee.238e1f29.9f79%40issac.
For more options, visit https://groups.google.com/d/optout.

Re: Deployment of django project using setuptools

[Tomáš Ehrlich]

I've started experimenting with this idea and created simple app:

https://github.com/elvard/django-deploy

I would very appreciate any opinions, ideas or second thoughts.


Cheers,
   Tom


Dne pátek, 2. května 2014 19:13:30 UTC+2 Tomáš Ehrlich napsal(a):
>
> Hi there, 
> when I deploy my django projects I always run several commands after 
> pulling latest code from repository. I've started using setup.py few 
> weeks ago which adds manage.py script into $VENV/bin. 
>
> Basicaly, I always do: 
>
> source $VENV/bin/activate 
> export DJANGO_SETTINGS_MODULE=... 
>
> git pull 
> python setup.py install 
> manage.py syncdb --noinput 
> manage.py migrate 
> manage.py collectstatic --noinput 
> manage.py compilemessages 
>
>
> I wrote deployment scripts in fabric, salt, ansible, but now I'm 
> thinking that all these 'manage.py' actions should be part of setup 
> script. Anytime when would I run ``python setup.py install`` 
> the setup script would run all defined management commands so my 
> environment, database and other stuff stays up to date. 
>
>
> I've found that install command can be extended: 
>
> http://www.niteoweb.com/blog/setuptools-run-custom-code-during-install 
>
> I haven't found any references that anyone have used this method to 
> deploy django project, so I'm looking for anyone who tried this 
> approach before. 
>
>
> Thanks in advance 
>
> Cheers, 
>Tom 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/30a7b284-38aa-4f8d-86f2-603498e5fbb2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: db queries made twice

[Tomáš Ehrlich]

Other option is set SHOW_TEMPLATE_CONTEXT to False
in DEBUG_TOOLBAR_CONFIG.

See https://github.com/django-debug-toolbar/django-debug-toolbar#configuration

Dne sobota, 31. srpna 2013 13:24:54 UTC+2 Tomáš Ehrlich napsal(a):
>
> Hi, 
> disable Template panel in debug_toolbar and try again. 
>
> Debug toolbar hits database when evaluating template context. 
>
>
> Cheers, 
>Tom 
>
> Dne Sat, 31 Aug 2013 12:09:00 +0100 
> Marcin Szamotulski napsal(a): 
>
> > Hello, 
> > 
> > I am using django-1.6.b2 on a localserver (./manage.py runserver) and 
> > for every request I have the chain of db lookups made twice.  For 
> > example my log looks something like this (after enabling 
> > django.db.backands logger): 
> > 
> > # HERE VIEW IS CALLED 
> > (0.001) SELECT "django_session"."session_key", 
> "django_session"."session_data", "django_session"."expire_date" FROM 
> "django_session" WHERE ("django_session"."session_key" = 
> '2dh8ly6cfqkyiauv8co5h1vossjg70ru'  AND "django_session"."expire_date" > 
> '2013-08-31 11:04:33.597734+00:00' ); 
> args=('2dh8ly6cfqkyiauv8co5h1vossjg70ru', u'2013-08-31 
> 11:04:33.597734+00:00') 
> > (0.001) SELECT "auth_user"."id", "auth_user"."password", 
> "auth_user"."last_login", "auth_user"."is_superuser", 
> "auth_user"."username", "auth_user"."first_name", "auth_user"."last_name", 
> "auth_user"."email", "auth_user"."is_staff", "auth_user"."is_active", 
> "auth_user"."date_joined" FROM "auth_user" WHERE "auth_user"."id" = 1 ; 
> args=(1,) 
> > 
> > #... (AND SO ON) 
> > 
> > # HERE VIEW CALL IS FINISHED 
> > [31/Aug/2013 12:04:33] "GET /coot/ HTTP/1.1" 200 220974 
> > [31/Aug/2013 12:04:34] "GET /site_media/static/css/style.css HTTP/1.1" 
> 304 0 
> > [31/Aug/2013 12:04:34] "GET /site_media/static/js/jquery.js HTTP/1.1" 
> 304 0 
> > [31/Aug/2013 12:04:34] "GET /site_media/static/js/jquery-ui.js HTTP/1.1" 
> 304 0 
> > [31/Aug/2013 12:04:34] "GET /site_media/static/js/posts.js HTTP/1.1" 304 
> 0 
> > [31/Aug/2013 12:04:34] "GET /site_media/static/js/confirm.js HTTP/1.1" 
> 304 0 
> > [31/Aug/2013 12:04:34] "GET /site_media/static/css/posts.css HTTP/1.1" 
> 304 0 
> > [31/Aug/2013 12:04:34] "GET 
> /site_media/static/debug_toolbar/css/toolbar.min.css HTTP/1.1" 304 0 
> > [31/Aug/2013 12:04:34] "GET 
> /site_media/static/debug_toolbar/js/toolbar.min.js HTTP/1.1" 304 0 
> > [31/Aug/2013 12:04:34] "GET /site_media/static/pics/django_logo.png 
> HTTP/1.1" 304 0 
> > [31/Aug/2013 12:04:34] "GET /site_media/static/pics/python_logo.png 
> HTTP/1.1" 304 0 
> > [31/Aug/2013 12:04:34] "GET /site_media/static/css/whitey.png HTTP/1.1" 
> 304 0 
> > [31/Aug/2013 12:04:34] "GET /site_media/static/css/stressed_linen.png 
> HTTP/1.1" 304   
> > 
> > # DB LOOKUPS ONCE AGAIN: 
> > (0.001) SELECT "django_session"."session_key", 
> "django_session"."session_data", "django_session"."expire_date" FROM 
> "django_session" WHERE ("django_session"."session_key" = 
> '2dh8ly6cfqkyiauv8co5h1vossjg70ru'  AND "django_session"."expire_date" > 
> '2013-08-31 11:04:33.597734+00:00' ); 
> args=('2dh8ly6cfqkyiauv8co5h1vossjg70ru', u'2013-08-31 
> 11:04:33.597734+00:00') 
> > (0.001) SELECT "auth_user"."id", "auth_user"."password", 
> "auth_user"."last_login", "auth_user"."is_superuser", 
> "auth_user"."username", "auth_user"."first_name", "auth_user"."last_name", 
> "auth_user"."email", "auth_user"."is_staff", "auth_user"."is_active", 
> "auth_user"."date_joined" FROM "auth_user" WHERE "auth_user"."id" = 1 ; 
> args=(1,) 
> > 
> > ... (AND SO ON) 
> > 
> > 
> > any ideas why? 
> > 
> > The views (as it is not just one) returns using render().  This also 
> > happens when using DetailedView cbv. 
> > 
> > I also setup looking on postgres (which I use) and indeed the queries 
> > are made twice. 
> > 
> > Thanks for ideas, 
> > Marcin 
> > 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Namespace URLs in TestCase

[Tomáš Ehrlich]

As I said, it's trivial:

urlpatterns = patterns('', 
url(r'^sso/$', include(app_patterns, 'app', 'app')))

The regexp is wrong. It should be only ^sso/ because other urls are 
appended after it…

Fortunatelly, Django 1.6 has more helpful exception as it prints all url 
patterns which 'reverse' function tried.

Sorry for spam, but it took me a while to found it.

Cheers,
  Tom


Dne čtvrtek, 4. července 2013 16:53:34 UTC+2 Tomáš Ehrlich napsal(a):
>
> Hi there, 
> it's probably silly bug, but I can't figure it out: 
>
>
> I have one test: 
> ### 
> # app/tests_functional/test_app.py 
> from django.core.urlresolvers import reverse 
> from django.test import TestCase 
>
> class TestApp(TestCase): 
> urls = 'app.tests_functional.urls' 
>
> def test_app(self): 
> print(reverse('app:get'), reverse('app:retrieve')) 
> self.fail() 
>
>
> where urls are defined: 
> ### 
> # apps/tests_functional/urls.py 
> from django.conf.urls import patterns, url, include 
>
> # I use class-based views, which are tested and works. 
> def view(request): pass   
>
> app_patterns = patterns('', 
> url(r'^get/$', view, name='get'), 
> url(r'^retrieve/$', view, name='retrieve')) 
>
> urlpatterns = patterns('', 
> url(r'^sso/$', include(app_patterns, 'app', 'app'))) 
>
>
> Now, when I run tests, I got Exception: 
> NoReverseMatch: Reverse for 'get' with arguments '()' and keyword 
> arguments '{}' not found. 
>
>
> I tried to rename app_patterns to urlpatterns and reverse('get'). It 
> works, but my app uses namespaced urls, so I can't change code of view 
> to test it that way. 
>
> I believe it's really silly think. Can you see it? 
>
>
> Thank you :) 
>
> Tom 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.

Re: django multi db routing doesnt work wich multiple schemas

[Tomáš Ehrlich]

Sorry, I've just found that schema != tablespace, so my last response is 
irelevant...

Cheers,
 Tom

Dne pátek, 12. října 2012 19:26:53 UTC+2 Tomáš Ehrlich napsal(a):
>
> Hi Michał, 
> is "schema" the same as thing as "tablespace"? If so, you should define 
> them using 
> db_tablespace meta keyword. 
> https://docs.djangoproject.com/en/1.4/topics/db/tablespaces/ 
>
> DATABASES creates new database connection and it's really not 
> possible to make relations between databases. 
>
> Cheers, 
>  Tom 
>
> Dne Fri, 12 Oct 2012 17:19:42 +0100 
> Michał Nowotka <mmm...@gmail.com> napsal(a): 
>
> > First of all the error is not from django but from database. 
> > Secondly I'm not using different databases but different schemas so 
> > relations are possible. 
> > The error has nothing to do with foreign key or any relation. 
> > It just looks like django is ignoring router suggestion and directs 
> > sql query to wrong DB. 
> > 
>
>
>
> S pozdravem 
>   Tomáš Ehrlich 
>
> Email:  tomas.ehrl...@gmail.com 
> Tel:+420 608 219 889 
> Jabber: elv...@jabber.cz 
>
> "Půjdu kamkoliv, pokud je to kupředu." - J. London 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/SjxuNuNqF78J.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Unhandled exception during validation

[Tomáš Ehrlich]

Hello Oyvind,
that's weird, Django (or Python  in general) usually provide very long and 
descriptive trackback.

Could you please provide more information? What version of Python do you 
have? Does it fail when you run ./manage.py runserver? You've mentioned, 
the error is in local app, does that app load any thirdparty modules?

I've never came across unhandled exception which doesn't show trackback, so 
my guess is some c/c++ python extension (PIL, psycopg2, …). But that's just 
guess…

Cheers,
 Tom

Dne pátek, 12. října 2012 12:47:51 UTC+2 Oyvind Idland napsal(a):
>
> Hello,
>
> I am currently trying to upgrade a site from 1.2.x to 1.4.1. When I try to 
> start the app, I get this:
>
> Validating models...
> Unhandled exception in thread started by <__main__.NewThreadStartup 
> instance at 0x03D13738>
>
>
> Since the output is very economical about info, I spent some time to track 
> down the origins. 
> I found this in loading.py,  AppCache._populate():
>
> for app_name in settings.INSTALLED_APPS:
> if app_name in self.handled:
> continue
> self.load_app(app_name, True)< CRASH
>
> The app it tries to load, is a local one (made by a co-developer some time 
> ago). 
>
> Shouldn't there be better error handling in cases like this, at least 
> showing what happened, and where ?
> There is no exception handling what so ever..
>
> -- Oyvind
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/SbzMIfmJbZEJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: "Writing your first Django app" Tutorial Question

[Tomáš Ehrlich]

Hi Stefano,
your understanding is incorrect. Django (neither Python) doesn't load (or 
import) anything by himself. Everything what you import is everything what 
you get. There are few exceptions, like Python builtin module (and default 
tags/templates in Django templates), but that's definitely not this case.

Template context consist of:
 - your context dictionary (in this case {'poll': p})
 - dictionaries of all context processors (if you use RequestContext)
 - and default tags/filters

One don't have to load Choice class, because choice instances are returned 
with {% for choice in poll.choice_set.all %}. You can access Choice class 
in choice.__class__ if you want, but that's unnecessary in most cases (in 
templates).

I'm writing this because recently I've been working with PHP and Yii 
framework and that kind of autoloading, you've mentioned, *does* occur 
there. It doesn't work like that in Python (and Django doesn't add it).

Cheers,
 Tom

Dne pátek, 12. října 2012 14:12:10 UTC+2 Stefano Tranquillini napsal(a):
>
> Not an expert at all, but this is my understanding.
> In the view you only set up data that will be available in the template, 
> there's no relation between the imports there and what can be displayed in 
> the template. 
> the templates probably loads the classes when he need to display something.
> not sure.
>
>  
>
> On Fri, Oct 12, 2012 at 12:06 PM, Rick Chong  > wrote:
>
>> Hi, I have just started learning programming and I am following the 
>> creating a poll app tutorial at:
>> https://docs.djangoproject.com/en/1.4/intro/tutorial03/
>>
>> I hope that someone has attempted this tutorial and is able to help me on 
>> some very beginner questions:
>>
>> In this app, we defined 2 classes in the model - Poll & Choice:
>>
>>> import datetime
>>>
>>> from django.utils import timezone
>>>
>>> from django.db import models
>>>
>>>   
>>>
>>> # Create your models here.
>>>
>>> class Poll(models.Model):
>>>
>>>  question = models.CharField(max_length=200)
>>>
>>>  pub_date = models.DateTimeField('date published')
>>>
>>>  def __unicode__(self):
>>>
>>>  return self.question
>>>
>>>  def was_published_recently(self):
>>>
>>>  return self.pub_date >= timezone.now() - datetime.timedelta(days=1)
>>>
>>>  was_published_recently.admin_order_field = 'pub_date'
>>>
>>>  was_published_recently.boolean = True
>>>
>>>  was_published_recently.short_description = 'Published recently?'
>>>
>>>
 class Choice(models.Model):
>>>
>>>  poll = models.ForeignKey(Poll)
>>>
>>>  choice = models.CharField(max_length=200)
>>>
>>>  votes = models.IntegerField()
>>>
>>>  def __unicode__(self):
>>>
>>>  return self.choice
>>>
>>>  
>>
>> Then in views.py... I have the following code:
>>
>>> from django.shortcuts import render_to_response, get_object_or_404
>>>
>>> from polls.models import Poll
>>>
>>>
 def detail(request, poll_id):
>>>
>>> p = get_object_or_404(Poll, pk=poll_id)
>>>
>>>  return render_to_response('polls/detail.html', {'poll': p})
>>>
>>>
>>>
>>
>> Then in my template, detail.html. I have the following code:
>>
>>> {{ poll.question }}
>>>
>>> 
>>>
>>> {% for choice in poll.choice_set.all %}
>>>
>>>  {{ choice.choice }}
>>>
>>> {% endfor %}
>>>
>>> 
>>>
>>>
>>
>>  
>> The scripts run My question:
>> In views.py, I only import Poll class from model... why am I able to 
>> display choice, which is a variable under Choice class?
>>
>>
>> Thank you very much.
>>
>>
>>
>>
>> Rick
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msg/django-users/-/2vqJuWiYVuIJ.
>> To post to this group, send email to django...@googlegroups.com
>> .
>> To unsubscribe from this group, send email to 
>> django-users...@googlegroups.com .
>> For more options, visit this group at 
>> http://groups.google.com/group/django-users?hl=en.
>>
>
>
>
> -- 
> Stefano
>  

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/SDIG4V7RbnwJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: A very basic question with Django

[Tomáš Ehrlich]

Ehm, It's not disgus but http://www.disqus.com ;) Sorry...

Dne středa, 10. října 2012 8:09:07 UTC+2 Tomáš Ehrlich napsal(a):
>
> Hello,
> I wouldn't say that Django is not suitable for web development. Even when 
> someone says it's not, it's just an opinion, but in general, Django is 
> framework for web development.
>
> A1: Django has many batteries included. Database access through object 
> relational model and form processing are just two parts from whole bunch. 
> Checkout the documentation for latest stable version of Django 
> https://docs.djangoproject.com/en/1.4/
>
> A2: I admit that I don't know exactly what CGI means, but you actually can 
> deploy Django using FastCGI. I'm using WSGI, which is recommended method 
> right now. Please checkout documentation for more details 
> https://docs.djangoproject.com/en/1.4/howto/deployment/fastcgi/
>
> A3: I believe you can. Do you know pinterest.com, disgus.com, 
> instagram.com? They're all written in Django. Please checkout the 
> homepage of Djangoproject for more https://www.djangoproject.com/ (column 
> Sites using django) or directly http://www.djangosites.com. Some of these 
> sites have source code available.
>
> My conclusion is: Django is suitable for web development. It's a great 
> project with perfect documentation and community. Checkout the tutorial, 
> it's really helpful 
> https://docs.djangoproject.com/en/1.4/intro/tutorial01/
>
> Cheers,
>  Tom
>
> Dne středa, 10. října 2012 7:24:26 UTC+2 Sarbjit singh napsal(a):
>>
>> First of all, I am very sorry for asking this basic question. I am not 
>> sure if this is the right place to put this question but I am very confused.
>>
>> I am not having much experience with web development, so i don't know 
>> where the Django fits in here. I searched a lot on internet, few forums 
>> suggested to use Django for websites and few mentioned that Django is not 
>> for web development. So i have couple of basic questions and i want to be 
>> sure that i am on right track in learning Django.
>>
>> Q: There are simple websites which just serves static contents and other 
>> site which deals with forms and data base. I have once used PHP for form 
>> processing and using it with DB. If i have to design such websites using 
>> Python, Is Django suitable for the following or there are some other 
>> modules which needs to be used.
>>
>> Q: Is Django a substitute to CGI for dynamic web generation.
>>
>> Q: Can i use Django for development of a full fledged website.
>>
>>
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/bF89jeIqADIJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: A very basic question with Django

[Tomáš Ehrlich]

Hello,
I wouldn't say that Django is not suitable for web development. Even when 
someone says it's not, it's just an opinion, but in general, Django is 
framework for web development.

A1: Django has many batteries included. Database access through object 
relational model and form processing are just two parts from whole bunch. 
Checkout the documentation for latest stable version of 
Django https://docs.djangoproject.com/en/1.4/

A2: I admit that I don't know exactly what CGI means, but you actually can 
deploy Django using FastCGI. I'm using WSGI, which is recommended method 
right now. Please checkout documentation for more 
details https://docs.djangoproject.com/en/1.4/howto/deployment/fastcgi/

A3: I believe you can. Do you know pinterest.com, disgus.com, 
instagram.com? They're all written in Django. Please checkout the homepage 
of Djangoproject for more https://www.djangoproject.com/ (column Sites 
using django) or directly http://www.djangosites.com. Some of these sites 
have source code available.

My conclusion is: Django is suitable for web development. It's a great 
project with perfect documentation and community. Checkout the tutorial, 
it's really helpful https://docs.djangoproject.com/en/1.4/intro/tutorial01/

Cheers,
 Tom

Dne středa, 10. října 2012 7:24:26 UTC+2 Sarbjit singh napsal(a):
>
> First of all, I am very sorry for asking this basic question. I am not 
> sure if this is the right place to put this question but I am very confused.
>
> I am not having much experience with web development, so i don't know 
> where the Django fits in here. I searched a lot on internet, few forums 
> suggested to use Django for websites and few mentioned that Django is not 
> for web development. So i have couple of basic questions and i want to be 
> sure that i am on right track in learning Django.
>
> Q: There are simple websites which just serves static contents and other 
> site which deals with forms and data base. I have once used PHP for form 
> processing and using it with DB. If i have to design such websites using 
> Python, Is Django suitable for the following or there are some other 
> modules which needs to be used.
>
> Q: Is Django a substitute to CGI for dynamic web generation.
>
> Q: Can i use Django for development of a full fledged website.
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/ZvNZXaaDuWoJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Model translation

[Tomáš Ehrlich]

Yes, transmeta is straightforward and simple. This is the main
advantage, but also the biggest disadvantage. Moreover, i'm scared of
altering big tables with lots of columns every time I want to add
another language.

On 8 čnc, 21:06, smac...@flagstonesoftware.com wrote:
> +1 for transmeta but I only use it for short text fields which contain  
> user supplied data which translated in the admin and or reference data  
> or other fields which are constant. It quickly gets unmanagable as you  
> add fields.
>
>  From what you describe multilingual is probably better suited since  
> the different translations are easier to manage in the admin. However  
> it probably only fits the core of your requirements unless you stored  
> all translatable fields in their own tables in which case it would be  
> easier to add and manage all the other information about the  
> translations.
>
> Stuart MacKay
> Lisbon, Portgual

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Model translation

[Tomáš Ehrlich]

Hi there,
I just want to open discussion about this topic again, because I
didn't find any solution which 100% fits my needs.

What I want:
 - store translations in other table (1:N)
 - storing additional info, if necessary (translation_date,
translation_edit_date, translator_name, etc)
 - keeping the original model API with additional methods (missing
translations, translations count, etc)
 - group permissions for translators
 - per-site fallback language

Before developing my own library, I was searching this mailing list
and found:
http://code.google.com/p/django-transmeta/
http://code.google.com/p/django-multilingual/

I don't like transmeta with it's column_ln idea. Problems are:
 - translating big columns (like text column)
 - adding another language on production server

Multiligual app looks better but it lacks documentation.

Does anyone use one of these (especially multilingual app) in
production?



My idea is:

# Create model as usual with specific Meta options
class Article(model18n.Model):
pub_date = models.DateField()
category = models.ForeignField()
title = models.CharField()
slug = models.SlugField()
content = models.TextField()

class Meta:
translate = ('title', 'slug', 'content')
translate_with = ArticleTranslate  # optional specification of
translation model

# Specify translation model, if necessary
class ArticleTranslate(model18n.Translations):
# id, foreign_id, lang, title, slug, content are created
implicitly
date_translate = models.DateField()
translator = models.ForeignField() # user who done translation


These two models will create:
CREATE TABLE "articles_article" (
"id" serial NOT NULL PRIMARY KEY,
"pub_date" date NOT NULL,
"category_id" integer NOT NULL,  -- not sure about proper syntax,
nevermind
) -- yes, translated field are not in primary table. This table keeps
only structure and common data

CREATE TABLE "articles_article_translate" (
"id" serial NOT NULL PRIMARY KEY,
"article_id" integer NOT NULL,
"title" varchar(5) NOT NULL,
"date_translated" date NOT NULL,
"translator" integer NOT NULL,
"title" varchar(200) NOT NULL,
"slug" varchar(200) NOT NULL,
"content" text NOT NULL,
)


And working with them is like usual:

>>> article = Article(content="Dummy text", title="Dummy title", ...) # creates 
>>> default language entry
>>> article.save()
>>> article.add_language("cs", content="Textovy obsah", title="Textovy nazev", 
>>> ...)
>>> article.save()
...
>>> article = Article.objects.all() # implicitly join default language
>>> article.content
Dummy text
>>> translation.activate('cs')
>>> article.content # call another SQL query
Textovy nazev

Reasons for this are:
 - usually you need select from table only one language (only in admin
you need to select all related languages)
 - separating data structure from translated content

Problems are:
 - language variants - you are translating for en_US, en_GB, default
(fallback) language is cs_CZ. When US user access model entry only
with en_GB and cs_CZ, there shoudl be fallback to en_GB, rather than
cs_CZ. Query should respect ACCEPT_LANGUAGE preferences. For now, I
don't have idea how to solve this with "single" query.


===
Summary:

1. Does anyone use transmeta or multilingual (especially multilingual
app) in production?
2. How are you translating models?
3. What do you think about my solution?

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Using Jinja2

[Tomáš Ehrlich]

Hi there,
this looks interesting:
http://jinja.pocoo.org/docs/switching/#django

I'll try it soon

On 8 čnc, 16:43, Venkatraman S  wrote:
> On Fri, Jul 8, 2011 at 8:03 PM, akaariai  wrote:
>
> > Trying jinja2 in your project is hard - you are using some template
> > tags, and you would need to port them to jinja2. Then you would need
> > to rewrite your templates so that they work under jinja2. This would
> > take some time for sure.
>
> > What you could do is check out the profiler suggested in this thread.
> > Jinja2 is generally somewhere from 5x to 10x faster that Django
> > templates. So if you can see that half of your time is spent in
> > template rendering, then you could probably almost double the speed of
> > your application using Jinja2. If you don't want to do that, you can
> > check how fast your code is without any template rendering: just
> > return HttpResponse("") from your view. But remember to evaluate your
> > queries, querysets are evaluated only on access. This will give you
> > some clue how much Jinja2 would benefit you.
>
> All the benchmarking that i have seen till now has clearly placed Jinja2
> perform
> way better than django templates and this has made me think; as i like
> taking
> the juice our of every piece of code.
>
> Having said that, i want to brainstorm on how the djano ecosystem can be
> made in
> such a way that the apps that are written are template-engine agnostic.
> For eg, i use many 3rd party apps like endless-pagination, ajax-selects,
> uni-form etc which are mainly 'template-specific' apps. (And i *love* all
> these apps to the core!)
>
> # How easy/tough is it to move a site which already uses django templates
> over to jinja2?
> # What are the points that one should be aware of?
>
> -V 

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ANN: django-iadmin

[Tomáš Ehrlich]

Hi Sax,
I've found, where is a problem, but it's much more complicated. I've
just filled issue: https://github.com/saxix/django-iadmin/issues/2

Hope we'll fix it soon, need to go sleep now

On 5 čnc, 23:18, Tomáš Ehrlich <tomas.ehrl...@gmail.com> wrote:
> I'm using latest Django svn checkout.
>
> I've just solved this problem, although still don't understand it.
>
> Problem was with models simply registered to admin like:
>
> admin.site.register(TaggedItem)
>
> So, I created simple model admin and it works:
>
> class TaggedItemAdmin(admin.ModelAdmin):
>     pass
>
> admin.site.register(TaggedItem, TaggedItemAdmin)
>
> On 5 čnc, 23:05, sax <s...@os4d.org> wrote:
>
>
>
>
>
>
>
> > Ciao Tomáš,
> > which version of django are you using ? did you compare your settings with
> > the one into the testprj app ?
>
> > You can find the issue tracker 
> > athttps://github.com/saxix/django-iadmin/issues?sort=created=...
>
> > let me know as i can help you
>
> > sax
>
> > 2011/7/5 Tomáš Ehrlich <tomas.ehrl...@gmail.com>
>
> > > Hi Sax,
> > > thanks for your work, i've already put it into my project.
>
> > > Unfortunately, I've got this error when accessing change_list view.
> > > Some apps works fine, but some of them shows this exception:
> > > File "/usr/lib/python2.6/dist-packages/django/contrib/admin/views/
> > > main.py" in __init__
> > >  81.         self.query_set = self.get_query_set(request)
>
> > > Exception Type: TypeError at /admin/markup/textprocessor/
> > > Exception Value: get_query_set() takes exactly 1 argument (2 given)
>
> > > I'm trying to figure out, what's wrong. Just posting in case you know
> > > about this.
>
> > > Does github have any issue tracker?
>
> > > Thanks
>
> > > On 4 čnc, 14:55, Stefano Apostolico <s...@os4d.org> wrote:
> > > > Hi all,
>
> > > > I would like some feedback onhttps://
> > > github.com/saxix/djangoscreenshots-home-page-iadmin<
> > >https://github.com/saxix/django-iadmin>
> > > > .
>
> > > > Documentation in progress,  but demo application included.
>
> > > > Suggestions and comments are welcome, any help will be appreciated
>
> > > > sax
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "Django users" group.
> > > To post to this group, send email to django-users@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > django-users+unsubscr...@googlegroups.com.
> > > For more options, visit this group at
> > >http://groups.google.com/group/django-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ANN: django-iadmin

[Tomáš Ehrlich]

I'm using latest Django svn checkout.

I've just solved this problem, although still don't understand it.

Problem was with models simply registered to admin like:

admin.site.register(TaggedItem)

So, I created simple model admin and it works:

class TaggedItemAdmin(admin.ModelAdmin):
pass

admin.site.register(TaggedItem, TaggedItemAdmin)

On 5 čnc, 23:05, sax <s...@os4d.org> wrote:
> Ciao Tomáš,
> which version of django are you using ? did you compare your settings with
> the one into the testprj app ?
>
> You can find the issue tracker 
> athttps://github.com/saxix/django-iadmin/issues?sort=created=...
>
> let me know as i can help you
>
> sax
>
> 2011/7/5 Tomáš Ehrlich <tomas.ehrl...@gmail.com>
>
>
>
>
>
>
>
> > Hi Sax,
> > thanks for your work, i've already put it into my project.
>
> > Unfortunately, I've got this error when accessing change_list view.
> > Some apps works fine, but some of them shows this exception:
> > File "/usr/lib/python2.6/dist-packages/django/contrib/admin/views/
> > main.py" in __init__
> >  81.         self.query_set = self.get_query_set(request)
>
> > Exception Type: TypeError at /admin/markup/textprocessor/
> > Exception Value: get_query_set() takes exactly 1 argument (2 given)
>
> > I'm trying to figure out, what's wrong. Just posting in case you know
> > about this.
>
> > Does github have any issue tracker?
>
> > Thanks
>
> > On 4 čnc, 14:55, Stefano Apostolico <s...@os4d.org> wrote:
> > > Hi all,
>
> > > I would like some feedback onhttps://
> > github.com/saxix/djangoscreenshots-home-page-iadmin<
> >https://github.com/saxix/django-iadmin>
> > > .
>
> > > Documentation in progress,  but demo application included.
>
> > > Suggestions and comments are welcome, any help will be appreciated
>
> > > sax
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django users" group.
> > To post to this group, send email to django-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-users+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> >http://groups.google.com/group/django-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: ANN: django-iadmin

[Tomáš Ehrlich]

Hi Sax,
thanks for your work, i've already put it into my project.

Unfortunately, I've got this error when accessing change_list view.
Some apps works fine, but some of them shows this exception:
File "/usr/lib/python2.6/dist-packages/django/contrib/admin/views/
main.py" in __init__
  81. self.query_set = self.get_query_set(request)

Exception Type: TypeError at /admin/markup/textprocessor/
Exception Value: get_query_set() takes exactly 1 argument (2 given)

I'm trying to figure out, what's wrong. Just posting in case you know
about this.

Does github have any issue tracker?

Thanks

On 4 čnc, 14:55, Stefano Apostolico  wrote:
> Hi all,
>
> I would like some feedback 
> onhttps://github.com/saxix/djangoscreenshots-home-page-iadmin
> .
>
> Documentation in progress,  but demo application included.
>
> Suggestions and comments are welcome, any help will be appreciated
>
> sax

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

django.contrib.admin - custom css class in result_headers

[Tomáš Ehrlich]

Hi there,
as I want to add custom css class to table header in admin, I was
looking for template, which I should edit.

I found that method result_headers in django/contrib/admin/
templatetags/admin_list.py sets whole 'class="xxx"' string. Why is
there 'class=' thing ? It's quiet complicated to alter template with
this hardcoded string. (This part of question is more for developers,
maybe)

My question is, how can I override templatetag ( result_headers in
this case ) ?

Thank you!

Ps: My original idea was add css class according to field name and
type, eg: class="id integer", etc. I want to specify column width
according it's content. Is there any better solution than overriding
template tag?

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

How to add non-model field in admin

[Tomáš Ehrlich]

Hi there,
I wonder if is there any way to add extra input field in admin, which
I don't want to define in model.

For example, I want to import external files. I have simple blog model
(title, content, pub_date), but in admin I want to also have file
input field, to import files.

Then I can:
1) Import external file, parse it and save values (title, content,
pub_date) to database.
 or
2) Enter values (title, ...) as usuall directly in admin.

Is there any way how to do this without making admin template for this
application ?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---