On Wed, 10 Feb 2010 14:54:36 +0100, Carlos Garnacho <[email protected]> wrote: > Hi!, > > On mar, 2010-02-09 at 21:31 +0100, Adrien Bustany wrote: >> Hi all, >> >> this proposal is about automatically reindexing a mime type when a new >> extractor is added/updated. There's already a "Reindex" call in >> tracker-miner-fs. > > I think we should take several use cases into account here. > > * A new tracker-extractor module is installed > * As mbiebl pointed out, a tracker-extractor module has been added > some new capability. > * The library a tracker extractor relies on gains new capabilities > (i.e. GStreamer, poppler) > > IMHO the trickiest one is the 3rd, which either requires integration > from packagers, or some way for extractors to probe the file types > supported. The second would largely depend on whether the library is > able to tells us that, which I don't think happens often, so we might be > just forced to the first option. > > For the second usecase, we clearly need some way to version the > extractors, so it is known when to re-extract. The keyfile with version > info approach looks quite sane to me, we should provide some command > line tool to bump the version number for a given extractor. > >> >> Philip told me he'd like to keep tracker-extract as stupid as possible, >> so the logic here would be implemented in tracker-miner-fs, at init time. >> All extractors modules provide a function to know the mime types they can >> index, but we want to avoid loading all the modules at start. Therefore, >> a solution using desktop files is favoured. The desktop files would be >> installed in ${datadir}/tracker/extractors and would have the format > > This makes much sense to me, since the extractor could not run at all if > tracker-miner-fs thinks everything is up-to-date. Certainly, having > complete info about the mimetypes the extractors knows about in the > miner would avoid queries about unsupported files to the extractor, as > it happens right now. > >> >> [Extractor] >> Name=Foobar extractor >> MimeTypes=application/foobar > > What I propose is, having the mimetypes info separated from the > versioning info, I think this way we can provide reasonable support for > the 3 usecases mentioned above: > > * If tracker-extractor-foo 0.15 provides better information than > the previous release 0.14, the version file installed by the > package bumps its number, tracker-miner-fs notices the version > bump and does its job. > * if some gstreamer package is added, package scripts use some CLI > tracker tool if available to add new mimetypes for the gstreamer > extract module, tracker-miner-fs notices the changes in > mimetypes supported and does its job. The version number isn't > actually changed, since the extract module hasn't changed. > * A new extractor is installed, tracker-miner-fs notices no prior > info about it and does it's job, more or less like a version > bump. > > So, I guess there should be some $(datadir)/tracker/extractors/ with > version info and a $(datadir)/tracker/extractor-mimetypes/ with a > mimetype->extractor mapping. The main caveat I see here is how would the > initial mimetype mapping be done for certain modules (gstreamer yet > again in mind :), this could require yet again packagers help. Rather than having a CLI tool to add mimetypes, I'd allow "partial definition" of modules. Eg. we could have in two separate files
[Extractor] Name=Gstreamer base extractor MimeTypes=audio/ogg and [Extractor] Name=Gstreamer ffmpeg extractor MimeTypes=audio/mp3 we don't actually care of the mapping to the .so file, since it's done by tracker-extract. We're just enumerating the supported mimetypes here. I'm not totally sure either about separating version from MimeType... What happens for example if my old gstreamer-ffmpeg couldn't read the vorbis tags (only id3) for ogg files , and the new one can ? The extractor version doesn't change, and the mimetypes don't change either (ogg was always supported). Therefore, I'd prefer one Version key per desktop file. Cheers Adrien > > Besides, we also need to take into account restarting tracker-extract if > it's alive at the time of the update, and making things persistent so if > tracker-miner-fs shuts down pending file checks wouldn't be lost. This > is going to be tricky :) Good point, maybe we could add a "RequestShutdown" call to tracker-extract that would wait for any pending task to finish and shutdown it. > > Cheers, > Carlos > >> >> at startup, the FS miner loads all the description files and checks if a >> new extractor has been added, removed, or changed its mtime. If >> so, it calls the reindex method with the appropriate mime type. >> To detect a change in a desktop file, a list of each desktop file with >> their >> modification time is kept in cache by the FS miner. >> >> Ideas : >> - Describe several extractors in one file >> That makes is much more difficult to detect a change, since only one >> extractor >> in a file listing 10 might have changed when the file modification time >> changes. >> - Adding a version number in the desktop file, to avoid relying only on >> the mtime >> of the desktop file. >> >> Please tell me your thoughts >> >> Adrien >> _______________________________________________ >> tracker-list mailing list >> [email protected] >> http://mail.gnome.org/mailman/listinfo/tracker-list > > > _______________________________________________ > tracker-list mailing list > [email protected] > http://mail.gnome.org/mailman/listinfo/tracker-list _______________________________________________ tracker-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/tracker-list
