On 2015-09-12 06:30 PM, Florian Weimer wrote: >> On 09/06/2015 11:13 AM, Florian Weimer wrote: >>> Surely that's not true, and NFS and SMB are fine as long as there >>> is no concurrent access? >> And no program crashes, no network glitches, no optimisation in the >> protocols to deal with latency, nothing else futzing with the files, >> no programs futzing with them (backup agents, virus scanners etc), the >> protocols are 100% complete compared to local file access, the >> implementation of client and server for the protocol is 100% complete >> and bug free, the operating systems don't treat network filesystems >> sufficiently different to cause problems, you aren't using WAL, and >> the list goes on. >> >> In other words it can superficially appear to work. But one day >> you'll eventually notice corruption, post to this list, and be told >> not to use network filesystems. The only variable is how long it >> takes before you make that post. > Sorry, this all sounds a bit BS to me. Surely, as an fopen > replacement, SQLite works with network file systems, be it the home > NAS, or something in a typical datacenter. And if the SQLite locking > doesn't work in practice (which I doubt, remote file systems are > better at locking than they used to be), should we really fall back > on lock files with user overrides? I hope not.
Your assumption and doubt is very wrong. Being able to open a file over the network is one thing, being able to lock it is another. Of course SQLite can request the lock (and does) like any other file access method, but the Network file system lies about the the actual locked state (for various reasons that requires more space than I aim to use here). This is not an SQLite shortcoming, it is a network file-system shortcoming. Your assertion that "remote file systems are better at locking than they used to be" may be superficially true, but that doesn't mean they are any more reliable. They simply lie all the time, because they have to in order to not make the connected World a very very slow place. > (A lot of people run enterprise databases on NFS because that gives > them snapshot-able storage and other goodies. Not everyone uses iSCSI > and block devices.) That's like saying "A lot of people drive off-road vehicles in the bush, there is no reason why F1-cars should not also be driven in the bush." Enterprise DBs have servers on the same machine as the Files they access, they do not actually use the network file-system to access the DB data-files over the network from multiple clients, or even servers (unless the DBs are partitioned so and ONLY accessed by the single process so locking is permanent and moot). You cannot achieve a safe mechanism for opening, altering and locking files using fOpen over a network from other machines and multiple processes, but I invite you to try, maybe the exercise will be educational and remove some of the misconceptions. You might even post your findings and code here after the fact. Lots of people do not realize the workings and will benefit from reading such an exercise. Cheers, Ryan
