Re: Async Caching

[Andrew Wang]

Hey Adam and Andrew,

I can definitely make the naming scheme something like get_async() rather 
than just get().

> This section specifically says that the default implementations will back 
onto the sync methods by default, so built-in cache backends won't need to 
all be converted (at once?).

I'm slightly confused by what you mean (and what the dep means). Are you 
saying in case someone is using a built-in cache backend like LocMem for 
local development and then an async third-party cache backend for 
production, then the code is like:

class BaseCache:
def get(...):
NotImplementedError
async def get_async(...):
NotImplementedError

class LocMemCache:
def get(...):
...
async def get_async(...):
return await asgiref.sync_to_async(self.get)(*args, **kwargs)

>  It could be informative to have a working backend using this interface, 
and if you encounter any edge cases whilst creating it.

You can view my progress (once midterms are over :P) over here. 
 I'm planning on 
making the structure very similar to django-redis to make any migration 
process to an async Django easier. When I compared redis-py to aioredis, 
there weren't that many differences besides file structure (and passing the 
event loop around), so hopefully nothing weird pops up. But I will 
definitely post in this thread or in a ticket (if I do make a PR for 
BaseCache) if something odd does pop up.

> The main work to do here is to work how quite how possible it is to offer 
all of them and if everything can be made to work regardless of what mode 
(sync or async) it's in

In terms of the package or the other built-in backends or just everything 
in general like the ORM? If in terms of the package, most if not all 
methods would have to be awaited... I could also just be confusing all 
these words, so sorry about uhh me I guess!

Also just saw https://forum.djangoproject.com/. Any preference for where 
future talks should be held: Google or that forum?

Thanks for the suggestions! Have fun with the ORM!
Andrew
On Saturday, September 26, 2020 at 8:38:07 PM UTC-4 Andrew Godwin wrote:

> Agreed - there's no work on caching inside Django yet, since the ORM is my 
> next focus, but I would definitely suggest writing a new pluggable 
> third-party backend that somehow provides async versions of the methods.
>
> The main work to do here is to work how quite how possible it is to offer 
> all of them and if everything can be made to work regardless of what mode 
> (sync or async) it's in; hopefully there's a lot less long-lived-connection 
> issues than in the ORM, say.
>
> Andrew
>
> On Sat, Sep 26, 2020, at 3:56 PM, Adam Johnson wrote:
>
> Hi Andrew
>
> I don't believe any work has started on async caching in core. However 
> there is a plan in DEP 9, the document that outlines how asynchronous 
> caching will work, using suffixed methods like get_async() etc. See 
> https://github.com/django/deps/blob/master/accepted/0009-async.rst#caching 
> . This section specifically says that the default implementations will back 
> onto the sync methods by default, so built-in cache backends won't need to 
> all be converted (at once?).
>
> I think a good start would be making your aioredis-based backend in a 
> third party package, using the future naming scheme `*_async`. It could be 
> informative to have a working backend using this interface, and if you 
> encounter any edge cases whilst creating it.
>
> Thanks,
>
> Adam
>
> On Sat, 26 Sep 2020 at 05:55, Andrew Wang  wrote:
>
> Hey guys, I'd like to contribute to the effort to make Django more async 
> capable. I've started to write an aioredis based cache backend based on 
> django-redis, but I noticed the BaseCache in Django is still all 
> synchronous basically.
>
> I was wondering which backends I should make async capable and how would I 
> go about it?
>
> I was thinking instead of creating a new class, we could just add the 
> async methods (e.g. add, get) to BaseClass. And for the FileBased, Dummy, 
> and LocMem backends, the plan would be the same? Problem with those are 
> Python's local file IO is synchronous...
>
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-develop...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/68307bf9-e6f4-4dc3-9e0f-d6e660075a85o%40googlegroups.com
>  
> 
> .
>
>
>
> -- 
> Adam
>
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and 

Re: Async Caching

[Andrew Godwin]

Agreed - there's no work on caching inside Django yet, since the ORM is my next 
focus, but I would definitely suggest writing a new pluggable third-party 
backend that somehow provides async versions of the methods.

The main work to do here is to work how quite how possible it is to offer all 
of them and if everything can be made to work regardless of what mode (sync or 
async) it's in; hopefully there's a lot less long-lived-connection issues than 
in the ORM, say.

Andrew

On Sat, Sep 26, 2020, at 3:56 PM, Adam Johnson wrote:
> Hi Andrew
> 
> I don't believe any work has started on async caching in core. However there 
> is a plan in DEP 9, the document that outlines how asynchronous caching will 
> work, using suffixed methods like get_async() etc. See 
> https://github.com/django/deps/blob/master/accepted/0009-async.rst#caching . 
> This section specifically says that the default implementations will back 
> onto the sync methods by default, so built-in cache backends won't need to 
> all be converted (at once?).
> 
> I think a good start would be making your aioredis-based backend in a third 
> party package, using the future naming scheme `*_async`. It could be 
> informative to have a working backend using this interface, and if you 
> encounter any edge cases whilst creating it.
> 
> Thanks,
> 
> Adam
> 
> On Sat, 26 Sep 2020 at 05:55, Andrew Wang  wrote:
>> Hey guys, I'd like to contribute to the effort to make Django more async 
>> capable. I've started to write an aioredis based cache backend based on 
>> django-redis, but I noticed the BaseCache in Django is still all synchronous 
>> basically.
>> 
>> I was wondering which backends I should make async capable and how would I 
>> go about it?
>> 
>> I was thinking instead of creating a new class, we could just add the async 
>> methods (e.g. add, get) to BaseClass. And for the FileBased, Dummy, and 
>> LocMem backends, the plan would be the same? Problem with those are Python's 
>> local file IO is synchronous...
>> 

>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/68307bf9-e6f4-4dc3-9e0f-d6e660075a85o%40googlegroups.com
>>  
>> .
> 
> 
> -- 
> Adam
> 

> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAMyDDM00ourgfJs53v7NmXOGEV6_t8vV3%3D%2BP%2Bf61DHEhOcFvDw%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2efe271c-2626-45a9-a6f0-bd808562b2a2%40www.fastmail.com.

Re: Async Caching

[Adam Johnson]

Hi Andrew

I don't believe any work has started on async caching in core. However
there is a plan in DEP 9, the document that outlines how asynchronous
caching will work, using suffixed methods like get_async() etc. See
https://github.com/django/deps/blob/master/accepted/0009-async.rst#caching
. This section specifically says that the default implementations will back
onto the sync methods by default, so built-in cache backends won't need to
all be converted (at once?).

I think a good start would be making your aioredis-based backend in a third
party package, using the future naming scheme `*_async`. It could be
informative to have a working backend using this interface, and if you
encounter any edge cases whilst creating it.

Thanks,

Adam

On Sat, 26 Sep 2020 at 05:55, Andrew Wang  wrote:

> Hey guys, I'd like to contribute to the effort to make Django more async
> capable. I've started to write an aioredis based cache backend based on
> django-redis, but I noticed the BaseCache in Django is still all
> synchronous basically.
>
> I was wondering which backends I should make async capable and how would I
> go about it?
>
> I was thinking instead of creating a new class, we could just add the
> async methods (e.g. add, get) to BaseClass. And for the FileBased, Dummy,
> and LocMem backends, the plan would be the same? Problem with those are
> Python's local file IO is synchronous...
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/68307bf9-e6f4-4dc3-9e0f-d6e660075a85o%40googlegroups.com
> 
> .
>


-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM00ourgfJs53v7NmXOGEV6_t8vV3%3D%2BP%2Bf61DHEhOcFvDw%40mail.gmail.com.

Async Caching

[Andrew Wang]

Hey guys, I'd like to contribute to the effort to make Django more async 
capable. I've started to write an aioredis based cache backend based on 
django-redis, but I noticed the BaseCache in Django is still all 
synchronous basically.

I was wondering which backends I should make async capable and how would I 
go about it?

I was thinking instead of creating a new class, we could just add the async 
methods (e.g. add, get) to BaseClass. And for the FileBased, Dummy, and 
LocMem backends, the plan would be the same? Problem with those are 
Python's local file IO is synchronous...

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/68307bf9-e6f4-4dc3-9e0f-d6e660075a85o%40googlegroups.com.