Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

Hi, redis lock has own limitations and depending on your use case it may or may not be suitable [1]. If possible I would redefine problem and also considered: 1) create worker per specific resource type to avoid locking 2) optimistic locking 3) File system lock like in twisted, but not sure about

Re: [Async-sig] asyncio.Lock equivalent for multiple processes

Hi Dima, Thanks for your time and explanations :-) However, I have the intuition that it will take me more time to implement your idea compare to the builtin feature of PostgreSQL. Nevertheless, I keep your idea in mind in case of I have problems with PostgreSQL. Have a nice day. -- Ludovic

Re: [Async-sig] asyncio.Lock equivalent for multiple processes

Hi Ludovic, I believe it's relatively straightforward to implement the core functionality, if you can at first reduce it to: * allow only one coro to wait on lock at a given time (i.e. one user per process / event loop) * decide explicitly if you want other coros to continue (I assume so, as

Re: [Async-sig] [python-tulip] Re: asyncio.Lock equivalent for multiple processes

Hi Antoine & Chris, Thanks a lot for the advisory lock, I didn't know this feature in PostgreSQL. Indeed, it seems to fit my problem. The small latest problem I have is that we have string names for locks, but advisory locks accept only integers. Nevertheless, it isn't a problem, I will do a

Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

Hi Roberto, Thanks for the pointer, it's exactly the type of feedbacks I'm looking for: Ideas that are out-of-box of my confort zone. However, in our use case, we are using gunicorn, that uses forks instead of multiprocessing to my knowledge, I can't use multiprocessing without to remove

Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

Hi Nickolai, Thanks for your suggestions, especially for the file system lock: We don't have often locks, but we must be sure it's locked. For 1) and 4) suggestions, in fact we have several systems to sync and also a PostgreSQL transaction, the request must be treated by the same worker from

Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

If you're already using PostgreSQL, you might also look at "advisory locks": https://www.postgresql.org/docs/current/static/explicit-locking.html#ADVISORY-LOCKS --Chris On Tue, Apr 17, 2018 at 4:34 AM, Ludovic Gasc wrote: > Hi Nickolai, > > Thanks for your suggestions,

Re: [Async-sig] asyncio.Lock equivalent for multiple processes

Indeed, thanks for the suggestion :-) Le mer. 18 avr. 2018 à 01:21, Nathaniel Smith a écrit : > Pretty sure you want to add a try/finally around that yield, so you > release the lock on errors. > > On Tue, Apr 17, 2018, 14:39 Ludovic Gasc wrote: > >>

Re: [Async-sig] asyncio.Lock equivalent for multiple processes

You could simply use something like the first 64 bits of sha1("myapp:") Regards Antoine. On Tue, 17 Apr 2018 15:04:37 +0200 Ludovic Gasc wrote: > Hi Antoine & Chris, > > Thanks a lot for the advisory lock, I didn't know this feature in > PostgreSQL. > Indeed, it seems to

Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

Hi, I don't know if there is a third party solution for this. I think the closest you can get today using the standard library is using a multiprocessing.manager().Lock (which can be shared among processes) and call the lock.acquire() function with asyncio.run_in_executor(), using a

Re: [Async-sig] asyncio.Lock equivalent for multiple processes

Pretty sure you want to add a try/finally around that yield, so you release the lock on errors. On Tue, Apr 17, 2018, 14:39 Ludovic Gasc wrote: > 2018-04-17 15:16 GMT+02:00 Antoine Pitrou : > >> >> >> You could simply use something like the first 64 bits

Re: [Async-sig] asyncio.Lock equivalent for multiple processes

2018-04-17 15:16 GMT+02:00 Antoine Pitrou : > > > You could simply use something like the first 64 bits of > sha1("myapp:") > I have followed your idea, except I used hashtext directly, it's an internal postgresql function that generates an integer directly. For now, it