On 06.06.19 21:49, Andrew Piskorski wrote:
The easiest thing to do is call Tcl's open, puts, close every single
time I need to write data, but that's blocking, and I'm worried that
it could easily degrade the latency of my main processing thread.
by "main thread" you refer probably to the "current connection thread".
I don't necessarily need async IO to the files, but I WOULD like to get
the latency of writing the files out of the main thread.
Not sure, what your performance constraints are, but open/close
is pretty fast these days (measured under linux on openacs.org):
% time {set F [open /tmp/foo a]; puts $F x; close $F} 1000
55.333 microseconds per iteration
if this is too slow for you, one can experiment with ramdisks,
and other already mentioned approaches.
However, (particularly given my small writes to many files), clearly
asynchronous file IO should be more efficient, and scale better, so
I'd like to understand how to do that. The ease of simply calling
"fconfigure" on a file handle is attractive, but I don't yet
understand what else I'd need to do to make that seemingly simple
approach actually work in the NaviServer context.
whatever "work" means.
you can do that when you have at no time more then 1024 file descriptors
open under Linux. For windows, the maximum is probably lower [1].
Doing async I/O in a connection thread does not yield anything (every
thread has its own interpreter, therefore its own open files).
Furthermore you have
to consider write synchronization (handling concurrent writes from multiple
connection/background threads to the same file). The previously mentioned
approach with the AsyncDiskWriter cares about this.
all the best
-gn
[1]
https://stackoverflow.com/questions/47675410/python-asyncio-aiohttp-valueerror-too-many-file-descriptors-in-select-on-win
_______________________________________________
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
