Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+-
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Someday/Maybe
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Ülgen Sarıkavak):

 * cc: Ülgen Sarıkavak (added)

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018e5063f301-48816ebf-3097-4889-abec-c47513db537d-00%40eu-central-1.amazonses.com.

Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+-
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Someday/Maybe
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Andrew Chen Wang):

 Makes sense. Thanks for the comment!

 You can take a look at a practically broken but conceptual implementation
 here: https://github.com/Andrew-Chen-Wang/django-async-redis/pull/6

 The repository itself is designed to be a django-redis replacement that I
 recently migrated from aioredis to redis-py; in regards to this convo, the
 places to look are `django_async_redis.client.core.py` and the test file
 `tests/test_core.py`

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/010701859d8ce4d6-e06757bf-771e-43ef-b01b-d109a1f5bd2a-00%40eu-central-1.amazonses.com.

Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+-
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Someday/Maybe
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Carlton Gibson):

 Hey Andrew, thanks for the follow-up!

 If I've understood you correctly, I think the best way forward would be to
 prototype the async backend in a external module, so we can see what it
 would look like and install it and give it a run. I think we'd need to
 have a some discussion as to whether/how we'd adopt that then into Django,
 and the messaging we'd put around that.

 Make sense?

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/01070185764745a4-d51569a3-507c-42cc-8ced-84d69ebcae0c-00%40eu-central-1.amazonses.com.

Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+-
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Someday/Maybe
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Andrew Chen Wang):

 It doesn't seem like the Sans-IO approach will be adopted anytime soon as
 work and review has stalled.

 > I believe the connection poll should be able to be reused between the
 sync/async clients, but I am not an expert on Redis.

 Sorry if I misunderstood, but the ConnectionPool classes for the sync and
 async versions are different, and the Connection classes are also
 different.

 > As far as I'm aware this would require initializing two redis clients:
 async and non-async.

 Assuming you mean initialize `redis.Redis` and `redis.asyncio.Redis` in
 one class, I disagree; I think we should have two RedisCache classes, one
 for sync and another for async.

 1. Some users may want both a sync and async Redis client. However, the
 options presented to devs cater to only one of the variants. For instance,
 if you need to pass in a custom ConnectionPool or provide pool options for
 the default pools, those could be different for the different environments
 like an async enabled middleware uses the async Redis but a sync view uses
 the sync Redis. If a custom Connection class is provided to the pool, then
 we'd have a problem since the Connection class for the sync and async
 Redis are different.
 2. The actual network calls like `get` and `set_many` look like they will
 be copy-paste because of the numerous awaits required, so there won't be
 much if any re-use of code
 3. Not all clients have upgraded to at least Redis 4.2.0.
 4. It sets a paradigm for future client implementations. Regardless of
 whether packages like psycopg(3) implement both a sync and async variant
 or just one type like the memcached packages implementing just sync, there
 should be a separation of clients based on environment to set future
 expectations of clients not necessarily having a counterpart variant like
 an async package not having sync support.

 Let me know your thoughts. Thanks!

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/01070185697ac569-c049724e-b69a-4352-b601-b55595a62776-00%40eu-central-1.amazonses.com.

Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+-
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Someday/Maybe
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Christopher Bailey):

 4.2.0 is fully out now.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107017fb7af0497-d46307be-3409-4064-adbe-0659608dc56d-00%40eu-central-1.amazonses.com.

Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+-
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Someday/Maybe
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Christopher Bailey):

 Do you have an idea for implementation? As far as I'm aware this would
 require initializing two redis clients: async and non-async.

 I kind of started to implement it, but then realized it was likely going
 to be a bigger issue then I though. But there is already an abstract
 RedisClient class that is a wrapper around the base redis.Redis class. It
 maintains connection polls and handles all of the core commands. It has
 methods for `get`, `set`, `add`, `touch`, `incr` that just calls
 `get_client` and makes a new client using the existing connection pull any
 time it is called.

 For Async we can just add something like `get_async_client` and follow the
 same pattern within the class. I believe the connection poll should be
 able to be reused between the sync/async clients.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107017f88d1145c-89b0a27a-a0be-4490-96ba-6c85221133ef-00%40eu-central-1.amazonses.com.

Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+-
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Someday/Maybe
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Old description:

> Do you have an idea for implementation? As far as I'm aware this would
> require initializing two redis clients: async and non-async.
>
> I kind of started to implement it, but then realized it was likely going
> to be a bigger issue then I though. But there is already an abstract
> RedisClient class that is a wrapper around the base redis.Redis class. It
> maintains connection polls and handles all of the core commands. It has
> methods for `get`, `set`, `add`, `touch`, `incr` that just calls
> `get_client` and makes a new client using the existing connection pull
> any time it is called.
>
> For Async we can just add something like `get_async_client` and follow
> the same pattern within the class. I believe the connection poll should
> be able to be reused between the sync/async clients.

New description:

 The latest version of redis-py will support both sync and async clients so
 `sync_to_async` will no longer be necessary so it would be nice if the out
 of the box Redis cache backend supports both natively as well.

 https://github.com/redis/redis-py/releases/tag/v4.2.0rc1

 It may be a bit premature since 4.2.0 is still RC, but I wanted to get a
 ticket out there so it can be on someones radar.

--

Comment (by Christopher Bailey):

 Do you have an idea for implementation? As far as I'm aware this would
 require initializing two redis clients: async and non-async.

 I kind of started to implement it, but then realized it was likely going
 to be a bigger issue then I though. But there is already an abstract
 RedisClient class that is a wrapper around the base redis.Redis class. It
 maintains connection polls and handles all of the core commands. It has
 methods for `get`, `set`, `add`, `touch`, `incr` that just calls
 `get_client` and makes a new client using the existing connection pull any
 time it is called.

 For Async we can just add something like `get_async_client` and follow the
 same pattern within the class. I believe the connection poll should be
 able to be reused between the sync/async clients.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107017f88d0f83c-44feaa6a-fd9e-4b69-b860-95eb8f5d857e-00%40eu-central-1.amazonses.com.

Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+-
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Someday/Maybe
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Description changed by Christopher Bailey:

Old description:

> The latest version of redis-py will support both sync and async clients
> so `sync_to_async` will no longer be necessary so it would be nice if the
> out of the box Redis cache backend supports both natively as well.
>
> https://github.com/redis/redis-py/releases/tag/v4.2.0rc1
>
> It may be a bit premature since 4.2.0 is still RC, but I wanted to get a
> ticket out there so it can be on someones radar.

New description:

 Do you have an idea for implementation? As far as I'm aware this would
 require initializing two redis clients: async and non-async.

 I kind of started to implement it, but then realized it was likely going
 to be a bigger issue then I though. But there is already an abstract
 RedisClient class that is a wrapper around the base redis.Redis class. It
 maintains connection polls and handles all of the core commands. It has
 methods for `get`, `set`, `add`, `touch`, `incr` that just calls
 `get_client` and makes a new client using the existing connection pull any
 time it is called.

 For Async we can just add something like `get_async_client` and follow the
 same pattern within the class. I believe the connection poll should be
 able to be reused between the sync/async clients.

--

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107017f88ced1d4-87e29488-69ac-4ce6-beea-258f7bde32e8-00%40eu-central-1.amazonses.com.

Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+-
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Someday/Maybe
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Andrew Chen Wang):

 We are currently re-implementing the async client (one last time) in a way
 that allows for aioredis 1.3.1 performance.
 [https://github.com/seandstewart/redis-py-
 sansio/blob/main/benchmark/conftest.py] The RedisLab team will need to
 approve it first, so I would wait until our new client is either
 approved/denied [https://github.com/redis/redis-py/issues/2039]. This
 won't affect the non-async client as the non-async client will still
 default on the current redis-py interface with the option to also use the
 new API.

 This isn't to say no one should try implementing the async client now; I
 have high confidence the new client will be merged in with Redis-py as
 we're gunning for the much-needed performance gain. Just don't be
 surprised if a few things shift around.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107017f88a27e13-b394dcd4-8e48-4114-a055-da98b1c61ce5-00%40eu-central-1.amazonses.com.

Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+-
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Someday/Maybe
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Mariusz Felisiak):

 * stage:  Accepted => Someday/Maybe


Comment:

 Let's wait for the final release first.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107017f87880797-93194edb-6174-4a79-93d5-7afbb7ac3481-00%40eu-central-1.amazonses.com.

Re: [Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
-+
 Reporter:  Christopher Bailey   |Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+
Changes (by Mariusz Felisiak):

 * cc: Andrew Chen Wang, Andrew Godwin, Carlton Gibson (added)
 * stage:  Unreviewed => Accepted


Comment:

 Tentatively accepted. Do you have an idea for implementation? As far as
 I'm aware this would require initializing two redis clients: async and
 non-async.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107017f877e7b15-e7f14363-3c43-4687-b7bf-7f1a363f5572-00%40eu-central-1.amazonses.com.

[Django] #33573: Add native async support to redis cache backend

[Django]

#33573: Add native async support to redis cache backend
---+
   Reporter:  AngellusMortis   |  Owner:  nobody
   Type:  New feature  | Status:  new
  Component:  Core (Cache system)  |Version:  dev
   Severity:  Normal   |   Keywords:
   Triage Stage:  Unreviewed   |  Has patch:  0
Needs documentation:  0|Needs tests:  0
Patch needs improvement:  0|  Easy pickings:  0
  UI/UX:  0|
---+
 The latest version of redis-py will support both sync and async clients so
 `sync_to_async` will no longer be necessary so it would be nice if the out
 of the box Redis cache backend supports both natively as well.

 https://github.com/redis/redis-py/releases/tag/v4.2.0rc1

 It may be a bit premature since 4.2.0 is still RC, but I wanted to get a
 ticket out there so it can be on someones radar.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107017f79e0faf6-d34fbaf5-5ba6-4074-a358-797bea3bfd60-00%40eu-central-1.amazonses.com.