Re: fsync()-in webdav PUT

2018-03-05 Thread Nagy, Attila
On 03/05/2018 12:54 PM, Reinis Rozitis wrote: Jesus, why? You start the fsync in a thread and you wait for it to be completed with the HTTP response. Until this happens, the main thread can service other requests. Have you ever seen an async program which uses threads to run blocking operations?

Re: fsync()-in webdav PUT

2018-03-05 Thread Nagy, Attila
On 03/04/2018 12:45 PM, Reinis Rozitis wrote: That's what really kills performance, because of the async nature of nginx. That's why I'm proposing an option to do the fsync at the end of the PUT (or maybe even the whole operation) in a thread(pool). Then again this way you make it again

Re: fsync()-in webdav PUT

2018-03-02 Thread Nagy, Attila
On 03/02/2018 11:42 AM, Aziz Rozyev wrote: man page quote is related to the Valery’s argument that fsync wont affect performance, forget it. Of course it affects performance. But as for how much: it depends on many factors. It's possible to build servers where the overall effect will be

Re: fsync()-in webdav PUT

2018-03-02 Thread Nagy, Attila
On 02/28/2018 10:41 PM, Aziz Rozyev wrote: Without fsyncing file's data and metadata a client will receive a positive reply before data has reached the storage, thus leaving non-zero probability that states of two systems involved into a web transaction end up inconsistent. I understand why

Re: fsync()-in webdav PUT

2018-03-02 Thread Nagy, Attila
On 02/28/2018 03:08 PM, Maxim Dounin wrote: The question here is - why you want the file to be on disk, and not just in a buffer? Because you expect the server to die in a few seconds without flushing the file to disk? How probable it is, compared to the probability of the disk to die? A more

Re: fsync()-in webdav PUT

2018-03-02 Thread Nagy, Attila
On 02/28/2018 11:33 PM, Peter Booth wrote: This discussion is interesting, educational, and thought provoking.  Web architects only learn “the right way” by first doing things “the wrong way” and seeing what happens. Attila and Valery asked questions that sound logical, and I think there's

Re: fsync()-in webdav PUT

2018-03-02 Thread Nagy, Attila
On 02/28/2018 11:04 AM, Aziz Rozyev wrote: While it’s not clear why one may need to flush the data on each http operation, I can imagine to what performance degradation that may lead of. I store data on HTTP servers in a distributed manner and have a catalog of where each file is. If I get back

fsync()-in webdav PUT

2018-02-27 Thread Nagy, Attila
Hi, I would like to make sure when a WebDAV (with ngx_http_dav) PUT returns the file is reliably on the storage. This needs an fsync() on the file. It would be easy to put that into the module, but it would block the whole nginx process. Now, that nginx supports running threads, are there