On 4/30/2013 12:33 AM, Ehsan Akhgari wrote:
On 2013-04-29 1:51 PM, Taras Glek wrote:
* How to robustly write/update small datasets?
#3 above is it for small datasets. The correct way to do this is to
write blobs of JSON to disk. End of discussion.
For an API that is meant to be used by add-on authors, I'm afraid the
situation is not as easy as this. For example, for a "simple"
key/value store which should be used for small datasets, one cannot
enforce the implicit requirement of this solution (the data fitting in
a single block on the disk. for example) at the API boundary without
creating a crappy API which would "fail" some of the times if the
value to be written violates those assumptions. In practice it's not
very easy for the consumer of the API to guarantee the size of the
data written to disk if the data is coming from the user, the network,
etc.
The "implicit requirement" of which you speak is not a hard "this will
break if you violate this" requirement but rather a "performance is
optimized for this use-case" requirement.
Writes of data <= ~64K should just be implemented as atomic whole-file
read/write operations. Those are almost always single blocks on disk.
Writing a whole file at once eliminates risk of data corruption.
Incremental updates are what makes sqlite do the WAL/fsync/etc dance
that causes much of the slowness.
Is that true even if the file is written to more than one physical
block on the disk, across all of the filesystems that Firefox can run on?
OS.File.writeAtomic works correctly regardless of file size (it's
basically "write to temporary, move temporary to real filename"), which
works correctly so long as the move is not a cross-filesystem move.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
