Hello everyone. I am seeking some help with Django Channels and Redis.
I have a Django app running on Heroku. This app is rather simple: a Daphne
server, a background worker and a scheduling beat worker (the last two,
using Celery).
I noticed I hit *very often* Redi's MaxClientsError (every 15 minutes or
so) despite using the non-free plan of RedisCloud on Heroku, which has the
following parameters: Memory Size = 100 MB, Dedicated DB = 4, Connections =
256. I was using the free plan for a while with 30 connections, but the
MaxClientsError was popping all the time. Thus I increased to 256
connections, but since I have more bakground tasks, I hit again the error.
In the stacktrace below, you can see the problem occurs within
djangochannelsrestframework code. Line 233, there is this:
for group_name in group_names:
async_to_sync(channel_layer.group_send)(group_name, message)
But we never managed to understand if that was the cause of the error. Is
that way of doing the reason for opening so many connections ? My app is
not a chat. It has only background tasks that discover some data remotely,
and send them through websockets for a kind of "live events RSS".
Wouldn't it be better to use something like:
async2sync = async_to_sync(channel_layer.group_send)
for group_name in group_names:
async2sync(group_name, message)
Thanks a lot in advance!!!
---
My conf is the following:
Django==2.2.*
channels==2.2.*
channels-redis==2.3.*
daphne==2.3.*
CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": [os.environ.get('REDISCLOUD_URL',
'redis://localhost:6379')],
},
"symmetric_encryption_keys": [SECRET_KEY],
},
}
And the stacktrace:
MaxClientsError: ERR max number of clients reached
File "arcsecond/activities/tasks/archive_datarows.py", line 65, in
parse_archive_latest_rows
target_name=row.target_name)
File "django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "django/db/models/query.py", line 422, in create
obj.save(force_insert=True, using=self.db)
File "django/db/models/base.py", line 741, in save
force_update=force_update, update_fields=update_fields)
File "django/db/models/base.py", line 790, in save_base
update_fields=update_fields, raw=raw, using=using,
File "django/dispatch/dispatcher.py", line 175, in send
for receiver in self._live_receivers(sender)
File "django/dispatch/dispatcher.py", line 175, in <listcomp>
for receiver in self._live_receivers(sender)
File "djangochannelsrestframework/observer/observer.py", line 154, in
post_save_receiver
**kwargs
File "djangochannelsrestframework/observer/observer.py", line 221, in
post_change_receiver
**kwargs
File "djangochannelsrestframework/observer/observer.py", line 233, in
send_messages
async_to_sync(channel_layer.group_send)(group_name, message)
File "asgiref/sync.py", line 79, in __call__
return call_result.result()
File "concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "asgiref/sync.py", line 95, in main_wrap
result = await self.awaitable(*args, **kwargs)
File "channels_redis/core.py", line 559, in group_send
async with self.connection(self.consistent_hash(group)) as connection:
File "channels_redis/core.py", line 742, in __aenter__
self.conn = await self.pool.pop()
File "channels_redis/core.py", line 49, in pop
conns.append(await aioredis.create_redis(**self.host, loop=loop))
File "aioredis/commands/__init__.py", line 178, in create_redis
loop=loop)
File "aioredis/connection.py", line 129, in create_connection
await conn.auth(password)
File "aioredis/util.py", line 48, in wait_ok
res = await fut
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/dfe526ee-cd75-4481-9029-6f423996116d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
