On Friday 30 September 2011 13:06:55 Roberto De Ioris wrote:
> I will commit an async_rpc function soon.
Good! Thanks.
> In the mean time i have re-thinked th aio_read api:
>
> fd = open('myhugefile.txt')
> data = uwsgi.wait_aio_read(fd.fileno(), 40000[,offset])
> uwsgi.suspend()
> print data
> fd.close()
>
> This is more compliant to the current scheme.
> 'data' will be preallocated based on the size of the second argument.
>
> when returning from suspend(), 'data' contains the data (a string/bytes
> object of 40000 bytes)
>
> I will probably only need to add a way to check for errors after suspend(),
> then it will be usable.
>
> What do you think about it ?
Now, it looks more similar to socket stuff, but still different.
And, I think it's kinda confusing. Python's built-in open() return
FileObjects, witch already has methods like read(), seek(), etc. and it
incapsulates data inside. It's unclear what happens with it when we use
functions on its file descriptor and what happens if after "wait_aio_read" we
will call fd.read()... I'd venture to guess, this will be a common mistake.
Of course, the example will look more straight if we replace open() to
os.open():
fd = os.open('myhugefile.txt')
data = uwsgi.wait_aio_read(fd, 40000[,offset])
uwsgi.suspend()
print data
os.close(fd)
But, it seems, uwsgi.wait_aio_read should accept either a file descriptor or a
file object and behave accordingly.
If we passed the file object:
fo = open('myhugefile.txt')
uwsgi.wait_aio_read(fo, 40000)
uwsgi.suspend()
data = fo.read(20000) # non blocking read
print data
data2 = fo.read(20000) # non blocking read
print data2
data3 = fo.read(20000) # will block (20000*3 > 40000)
print data3
uwsgi.wait_aio_read(fo, 20000)
uwsgi.suspend()
data4 = fo.read(30000) # will read 20000 and then block
# until remaining 10000 bytes become available
fo.close()
If we passed the file descriptor then we're dealing with functions like
uwsgi.aio_read cause os.read() will block:
fd = os.open('myhugefile.txt')
uwsgi.wait_aio_read(fd, 40000 [, offset])
uwsgi.suspend()
data = uwsgi.aio_read(fd[, nbytes = 40000])
print data
os.close(fd)
Now it looks similar to socket uwsgi.wait_fd_read/uwsgi.recv and similar to
Python's os.read() way, and moreover, it somehow simplify error checking (for
example, aio_read might throw an exception).
Unfortunately, I haven't got any real experience with POSIX AIO, and not sure
that my suggestions are feasible.
And, I'm in big doubt, that we can somehow overload the file object, that
open() returns, just by passing it to wait_aio_read function and nothing
return. Seems it's not possible. Then, probably, we also need some wrapper
class (like django has https://docs.djangoproject.com/en/dev/ref/files/file/).
wbr, Valentin V. Bartenev
_______________________________________________
uWSGI mailing list
[email protected]
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
