So, following the work to remove inotify and just use GIO, find attached
the list of steps we may need when getting GIO events in order to
translate them into TrackerMonitor events.
As background, GIO raw events are listed here:
http://library.gnome.org/devel/gio/stable/GFileMonitor.html#GFileMonitorEvent
And the TrackerMonitor events are just:
- "item-created"
- "item-updated"
- "item-deleted"
- "item-moved"
Note that for event translations, I considered files and directory
events in a different way internally, because different steps should be
considered in each case.
These steps should cover all cases, including the issues of getting
DELETE events for directory A when moving it from A to B (in addition to
the MOVE(A->B) event). Using a new HT for those, which is actually the
thing fully complicating the issue...
Anyway, my impression is that even if all these steps are done, the
miner-fs should process them "in the same order" as notified by
TrackerMonitor, not based on 4 different queues where delete events have
precedence (IIRC)... although then that's another issue...
Comments more than welcome, specially from Carlos and Martyn :-)
--
Aleksander
Assuming we have 3 different Hash Tables:
* pre-update-file, will store only 2 types of events:
- G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(file)
- G_FILE_MONITOR_EVENT_CREATED(file)
* pre-update-dir, will store only 1 type of events (Actually, this one may be
mixed with the previous one in a single pre-update HT, but anyway):
- G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(directory)
* pre-delete-dir, will store only 2 types of events:
- G_FILE_MONITOR_EVENT_DELETED(directory)
- G_FILE_MONITOR_EVENT_MOVED(source_dir->dest_dir)
*** FILE events
<FILE_UPDATED(A)> = ( N * G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A) +
N * G_FILE_MONITOR_EVENT_CHANGED(A) +
(1|0) * G_FILE_MONITOR_EVENT_CHANGES_DONE_HINT(A)) ||
( 1 * G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A))
<FILE_CREATED(A)> = 1 * G_FILE_MONITOR_EVENT_CREATED(A) +
1 * <FILE_UPDATED(A)>
<FILE_DELETED (A)> = 1 * G_FILE_MONITOR_EVENT_DELETED(A)
<FILE_MOVED(A->B)> = 1 * G_FILE_MONITOR_EVENT_MOVED(A->B)
- When a G_FILE_MONITOR_EVENT_CREATED(A) is received,
-- If there is a previous G_FILE_MONITOR_EVENT_CREATED(A) event in the
pre-update-file cache, issue warning, and leave the previous event in
the cache.
-- If there is a previous G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A) event in
the pre-update-file cache, issue warning, remove the
G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A), and add a new
G_FILE_MONITOR_EVENT_CREATED(A) to the cache.
- When receiving a G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A):
-- If there is a previous G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A) in the
pre-update-file cache, don't do anything.
-- If there is a previous G_FILE_MONITOR_EVENT_CREATED(A) in the
pre-update-file cache, don't do anything.
-- If there is no previous item in the pre-update-file cache, add it.
- When receiving a G_FILE_MONITOR_EVENT_CHANGED(A):
-- If there is a previous G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A) in the
pre-update-file cache, remove the event from the cache. We know that
after a G_FILE_MONITOR_EVENT_CHANGED event, there must always be a
G_FILE_MONITOR_EVENT_CHANGED_DONE_HINT which always ends the event. This
will avoid cache expiration of that previous event where only attributes
of the file were changed.
-- If there is a previous G_FILE_MONITOR_EVENT_CREATED(A) event in the
pre-update-file cache, don't do anything.
- When receiving a G_FILE_MONITOR_EVENT_CHANGES_DONE_HINT(A):
-- First of all, check if there is a G_FILE_MONITOR_EVENT_MOVED(B->C) in the
pre-delete-dir cache where C is a parent directory of A. In this case,
the parent directory of A was first moved, and then A was modified (A is
notified after the move, so A is inside C). So, first issue the
DIRECTORY_MOVED(B->C), remove it from pre-delete-dir cache, and then keep
on with the following steps.
-- If there is a G_FILE_MONITOR_EVENT_MOVED(B->C) in the pre-delete-dir
cache where B is a parent directory of A, issue a warning, then keep on
with the following steps.
-- If there was a G_FILE_MONITOR_EVENT_CREATED(A) in the pre-update-file
cache, notify FILE_CREATED(A), and remove from the cache.
-- If there was a G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A) in the
pre-update-file cache, notify FILE_UPDATED(A), and remove from the cache.
-- If there is no previous item in the pre-update-file cache, just notify
directly FILE_UPDATED(A).
- When receiving a G_FILE_MONITOR_EVENT_DELETED:
-- First of all, check if there is a G_FILE_MONITOR_EVENT_MOVED(C->D) in the
pre-delete-dir cache where D is a parent directory of A. In this case,
the parent directory of A was first moved, and then A was deleted (A is
notified after the move, so A is inside D). So, first issue the
DIRECTORY_MOVED(C->D), remove it from pre-delete-dir cache, and then keep
on with the following steps.
-- If there is a G_FILE_MONITOR_EVENT_MOVED(B->C) in the pre-delete-dir
cache where B is a parent directory of A, issue a warning, then keep on
with the following steps.
-- If there is any kind of event in the pre-update-file cache, remove it,
and then notify FILE_DELETED(A).
-- If there is no event in the pre-update-file cache, just notify
FILE_DELETED(A).
-- Note that in all previous cases, FILE_DELETED is notified directly, not
added to any cache.
- When receiving a G_FILE_MONITOR_EVENT_MOVED(A->B):
-- First of all, check if there is a G_FILE_MONITOR_EVENT_MOVED(C->D) in the
pre-delete-dir cache where D is either parent of A or B. In this case,
the parent directory of A|B was first moved, and then A was moved to B.
So, first issue the DIRECTORY_MOVED(C->D), remove it from pre-delete-dir
cache, and then keep on with the following steps.
-- If there is a G_FILE_MONITOR_EVENT_MOVED(B->C) in the pre-delete-dir
cache where B is a parent directory of A, issue a warning, then keep on
with the following steps.
-- If there is any kind of event in the pre-update-file cache for A, notify
it first, remove from cache, and then notify FILE_MOVED(A->B)
-- If there is any kind of event in the pre-update-file cache for B, issue
a warning, and then notify FILE_MOVED(A->B).
-- If there is no event in the pre-update-file cache, just notify
FILE_MOVED(A->B).
-- Note that in all previous cases, FILE_MOVED is notified directly, not
added to any cache.
- If pre-update-file cache expires for A:
-- First of all, check if there is a G_FILE_MONITOR_EVENT_MOVED(C->D) in the
pre-delete-dir cache where D is parent of. In this case, the parent
directory of A was first moved, so first issue the DIRECTORY_MOVED(C->D),
remove it from pre-delete-dir cache, and then keep on with the following
steps.
-- If there is a G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED event waiting there,
notify the FILE_UPDATED(A) event.
-- If there is a G_FILE_MONITOR_EVENT_CREATED(A) event waiting there, notify
the FILE_CREATED(A) event. Note that in this case, we will possibly get
still a G_FILE_MONITOR_EVENT_CHANGES_DONE_HINT(A) after this. This could
happen when for example downloading a big file from the internet, where
the final update event is quite after the original create event.
*** DIRECTORY events
<DIRECTORY_CREATED(A)> = 1 * G_FILE_MONITOR_EVENT_CREATED(A)
<DIRECTORY_UPDATED(A)> = N * G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A)
<DIRECTORY_DELETED(A)> = 1 * G_FILE_MONITOR_EVENT_DELETED(A)
<DIRECTORY_MOVED(A->B)> = ((1|0) * G_FILE_MONITOR_EVENT_DELETED(A) +
1 * G_FILE_MONITOR_EVENT_MOVED(A->B) ) ||
( 1 * G_FILE_MONITOR_EVENT_MOVED(A->B) +
(1|0) * G_FILE_MONITOR_EVENT_DELETED(A))
- When receiving a G_FILE_MONITOR_EVENT_CREATED(A) for a Directory item:
-- If pre-delete-dir cache has a G_FILE_MONITOR_EVENT_DELETED(A), we need to
issue first the DIRECTORY_DELETED(A) and then the DIRECTORY_CREATED(A).
This is because the original directory may have had lots of files inside,
while the new one is empty, so both events must be notified. Also, the
G_FILE_MONITOR_EVENT_DELETED(A) must be removed from the pre-delete
cache.
-- If pre-delete-dir cache has a G_FILE_MONITOR_EVENT_MOVED(A->B), we need
to issue first the DIRECTORY_MOVED(A->B) and then the
DIRECTORY_CREATED(A). Also, the G_FILE_MONITOR_EVENT_MOVED(A->B) must be
removed from the cache.
-- In both previous cases, if pre-update-dir cache has a
G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A) it must be removed and not
notified, and possibly a warning issued, as there shouldn't be any
previous
event for A.
-- If pre-delete-dir cache has a G_FILE_MONITOR_EVENT_MOVED(B->A), it is
quite
a weird situation which should not happen. Probably just issue a warning
here and don't notify DIRECTORY_CREATED(A).
-- If pre-update-file cache has any event for files which are in this A
directory, issue a warning, and notify DIRECTORY_CREATED(A).
-- Otherwise, just notify the DIRECTORY_CREATED directly.
-- Note that in all previous cases, DIRECTORY_CREATED is notified directly,
not added to any cache.
- When receiving a G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A):
-- If there is a previous event in the pre-delete-dir cache, it would be
quite a weird situation, so issue a warning and forget about the new
event.
-- If there is a previous event in the pre-update-dir cache, don't do
anything.
-- If there is no previous event in the pre-update-dir cache, just add it.
-- Note that these events may come in a series of events (for example, a
chmod a+rwx generates 2 of these raw events).
- If the G_FILE_MONITOR_EVENT_ATTRIBUTE_CHANGED(A) expires in the
pre-update-dir cache:
-- Just notify DIRECTORY_UPDATED(A)
- When receiving a G_FILE_MONITOR_EVENT_DELETED(A) for a Directory item:
-- If there is any previous event in the pre-update-file cache for files
inside directory A, notify them first, then keep on with the next steps.
This may happen when modifying files inside the directory, and then
moving the directory from A to B (as the G_FILE_MONITOR_EVENT_DELETED(A)
may come first in a move operation).
-- If there is a previous G_FILE_MONITOR_EVENT_MOVED(A->B) event in the
pre-delete-dir cache, directly notify the previous DIRECTORY_MOVED(A->B)
event, and forget about the DIRECTORY_DELETED(A) event.
-- If there is a previous G_FILE_MONITOR_EVENT_MOVED(B->A) event in the
pre-delete cache, it means we first moved the directory and then removed
the new directory. This can actually be translated into
DIRECTORY_DELETED(B). For example, "mv source_dir dest_dir &&
rm -rf dest_dir" will be equivalent in terms of events to
"rm -rf source_dir". In this case, the event for the dest dir,
DIRECTORY_DELETED(A), shouldn't be notified.
-- If there is a previous G_FILE_MONITOR_EVENT_DELETED(A), issue a warning
as this is a weird situation.
-- If there was no previous event in the pre-delete-dir cache, just add it.
- If the G_FILE_MONITOR_EVENT_DELETED(A) expires in the pre-delete-dir cache:
-- If there is an event in the pre-update-dir cache for A, remove it, then
keep on with following steps.
-- If there are events in the pre-update-file cache for files inside A,
remove them, then keep on with following steps.
-- Finally, notify DIRECTORY_DELETED(A).
- When receiving a G_FILE_MONITOR_EVENT_MOVED(A->B):
-- If there is any previous event in the pre-update-file cache for files
inside directory A, notify them first, then keep on with the next steps.
-- If there is any previous event in the pre-update-file cache for files
inside directory B, issue a warning and then keep on with the next steps.
-- If there is a previous G_FILE_MONITOR_EVENT_DELETED(A) event in the
pre-delete-dir cache, then notify the DIRECTORY_MOVED(A->B) event and
remove the G_FILE_MONITOR_EVENT_DELETED(A) from the cache.
-- If there is a previous G_FILE_MONITOR_EVENT_DELETED(B) event in the
pre-delete-dir cache, then notify the DIRECTORY_DELETED(B), and add
the G_FILE_MONITOR_EVENT_MOVED(A->B) to the pre-delete-dir cache. For
example in: "rm -rf dest_dir && mv source_dir dest_dir"
-- If there is a previous G_FILE_MONITOR_EVENT_MOVED(A->B) event in the
pre-delete-dir cache, then issue a warning as this is a weird situation.
-- If there is a previous G_FILE_MONITOR_EVENT_MOVED(B->A) event in the
pre-delete-dir cache, first notify the DIRECTORY_MOVED(B->A), then add
the G_FILE_MONITOR_EVENT_MOVED(A->B) to the pre-delete-dir cache.
-- If there is no previous event in the pre-delete-dir cache, just add it.
- If the G_FILE_MONITOR_EVENT_MOVED(A->B) expires in the pre-delete-dir cache:
-- If there is any previous event in the pre-update-file cache for files
inside directory A, notify them first, then keep on with the next steps.
-- If there is any previous event in the pre-update-file cache for files
inside directory B, issue a warning and then keep on with the next steps.
-- If there is an event in the pre-update-dir cache for A, notify that one
first, then keep on with the next steps.
-- If there is an event in the pre-update-dir cache for B, issue a warning,
then keep on with the next steps.
-- Finally, notify G_FILE_MONITOR_EVENT_MOVED(A->B).
As a side note, all the cases issuing warnings should never happen, but still, I
guess it may make sense to consider them because we may have missed some weird
scenario...
_______________________________________________
tracker-list mailing list
[email protected]
http://mail.gnome.org/mailman/listinfo/tracker-list
