wohali commented on issue #1097: CouchDB 2.1.1: Compaction daemon - unable to
calculate free space for `/run/user/0`: `enoent`
It'd be much better if we could simply check the free space for the volume
on which the shard resides, rather than scanning all volumes for free space
first. I did a bunch of research when it became clear to me this isn't an easy
problem to solve.
Reminder: [we've been here
before](https://github.com/apache/couchdb/issues/732), which is when we
[introduced this warning](https://github.com/apache/couchdb/pull/803).
First stop: python, since I feel they have excellent cross-platform support
for this kind of thing. A simplified [code excerpt of finding the mount point
for a given file's
path = os.path.abspath(path) # or os.path.realpath()
while not os.path.ismount(path):
path = os.path.dirname(path)
Erlang has `filename:absname/1` and `filename:dirname/1`, but it doesn't
have an `os.path.ismount` equivalent, so I dug into how Python implements this.
Of course, there are different implementations for
with the Windows implementation requiring [a call out to a Win32
because Windows treats local drives, UNC paths, network-mounted drives and
junction points (symlinks) all differently.
We also need to be mindful of the difference between `abspath` and
`realpath`, i.e. when a relative path symlink is a parent directory; ascending
up the tree can lead to misleading results if a true absolute path isn't
determined in the process. `filename;absname/1` doesn't attempt to deal with
this problem. Amusingly, [mochiweb has solved this particular
For comparison, I checked JS/node/npm, and found
[diskusage](https://www.npmjs.com/package/diskusage) which appears to be the
most popular solution to this problem. It, too, calls out to a Win32 API
endpoint on Windows and relies on `statvfs` for POSIX. (It's not clear that
this module's Win32 implementation actually handles all the different
possibilities that the Python implementation does, incidentally.)
**Option 1**: implement all of the above. The Windows NIF is ugly, but there
are other reasons we might want to go down the path of a Windows-only util NIF
anyway. I can help here if this is desired.
Next stop, rabbitmq, as a fairly mature Erlang server-thing that definitely
does disk space checking. They use a much lower tech alternative, namely,
[shelling out and running cli commands to check free space for a given
Unfortunately, this is problematic, too: there have been
A comment is made in the that StackOverflow link that this parsing is fragile,
and could easily be broken by locale-specific changes, too; `df -kP` guarantees
POSIX-compatible output, but there's no guarantee we'll get any sort of
consistent report out of Windows without switching to [a PowerShell
or the aforementioned Win32 API call.
**Option 2**: Use the rabbitmq approach and shell out to the system rather
than parsing paths and calling `disksup:get_disk_data/0` once we find a mount
point we can use to match against Erlang's gathered stats. This feels easier
(no NIF!), but the devil's in the details, and technically speaking, rabbitmq's
is a ["Category B" weak-copyleft
license](https://www.apache.org/legal/resolved.html#category-b). We could
probably write to the rabbitmq people and get them to waive the MPL on this
particular part of code, or we could white-box our own implementation.
**Option 3**: Blacklist a bunch of paths that we know are bad: `/proc`,
`/run`, `/dev`. I hesitate to say all `tmpfs` volume types on Linux, because
running Couch out of a `tmpfs` volume as an all-in-memory CouchDB (prior to PSE
landing) isn't unreasonable. That said, if the `*stat*` calls fail on a `tmpfs`
volume, we still need to recover gracefully. @janl and I don't like this
because it's yet another weird thing to maintain with magic values.
**Option 4**: Do the least possible. Stay with the current approach and just
don't crash out when we can't stat a particular volume. If we can't actually
get the free space for the guessed mount point for the current volume, punt per
#803 and try our best anyway, free space or no. We'll probably end up with more
bugs like this in the future if we do this, but it'll let us close out this bug
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
Apache Git Services