-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 23/10/2014 8:15, Aleksander Morgado wrote: > On Thu, Oct 23, 2014 at 1:24 AM, Philip Van Hoof > <phi...@codeminded.be> wrote:
[cut] >>> This is fixable, by enforcing a size limit on the queue. As >>> the limit is hit the algorithm should coalesce queued events >>> based on subtrees. For example, if one event for /foo/bar/buzz >>> and one for /foo/bar/waldo is queued, and the queue hits its >>> limit, both are replaced by an entry for /foo/bar which is >>> marked with a new flag that some event was lost below this >>> subtree. For clients this would then mean that when they >>> receive this they must rescan that specific subtree again, but >>> not the whole tree. >>> >>> It's a simple algorithm, the max number of entries stays >>> fixed, but perfomance doesn't go completely horrible when the >>> limit is reached. >>> >>> Such a scheme should be implemented in fanotify on the kernel >>> side. [cut] >> But yes. The coalesce solution in fanotify would be a good idea >> to allow unprivileged processes. Probably better than what >> FSEvents does. > > That's a good one indeed; coalescing events in that way in the > kernel looks quite a sane approach. Still, one single process in > userspace doing all the control of what changed when (like FSEvents > does) may actually behave much nicer, as other processes could ask > for all changes coalesced since a specific timestamp (e.g. since > that process was run). Not thinking in Tracker here, think of a > program which runs sporadically, but when it runs it wants to know > what changed since the last time it was run (e.g. a backup app). I'd design that API to be a lot like how tracker:modified and several protocol's modseqs works: store a modification sequence in each new record of the journal and return it after each query by a client. Clients can then using the modification sequence ask for all changes since that number. However. This also means logging forever. So the same problems tracker-store has when you don't disable its journal: ie. always growing storage for it and privacy - the 'if I remove/rename/create a file, I don't want its existence to be nevertheless logged forever'-use-case. For example if the administrator takes away +rx from a directory for a user, the user could with the journal still know what used to be there. So we'd need a method for the admin to purge that. I think it's inevitable that such log files will need to be truncated, rotated and/or also size limited at some point. The coalesce-only solution doesn't have this problem. > Anyway, please remember that being privileged isn't the only > reason why Tracker can't use fanotify. It's API being fd-based, it > works on existing open files only; e.g. it won't notify file > deletes or move events, among other things. If we want some > recursirve monitoring approach with all CREATE/UPDATE/DELETE/MOVE > events, something new needs to be implemented, or inotify somehow > improved to handle that. Correct. Kind regards, Philip -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (MingW32) iQEcBAEBAgAGBQJUSLIdAAoJEEP2NSGEz4aDrEgIAML+jfWgzRrnnTop7jRrLXdv bRjibU37FaK45rauUxX395ye13hqUWQYUx9FVE4xh6y/yvJVFGCt1SfJFuGIu8+S XLaud2qY145ePIEfakD4z3iX9BsAWfsWIcvGu6QIz7tpT8Nd6SM+tJYwRx9V1dli vdspQh8Kf1Xg9DxUjmx2bnVYzV7KFM3dRgyuOGrxlHVjAGOH9HFAn1n0PJlO87Fu 8Hzr2vLgRU+zDnFY8jifavLKo4+a6/2UHeLbthhwRUUNBk9LZixblCs5wgAPJmb3 SZdIk5u+EqOYCAwWs6AgIEl/XQMbiC34DVS+rtckm0DNA8lmvzOTvvPar/ZkuzE= =Ldnv -----END PGP SIGNATURE----- _______________________________________________ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
