-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/09/2014 17:11, Martyn Russell wrote:
Hi Martyn, >> >> My proposal would be to keep tracker-store, ontology and >> libtracker-sparql together (as one project). > > The reason I didn't put libtracker-sparql with tracker-store / > core things is that logically it's quite a different thing for > applications wanting to JUST run SPARQL queries. Well, nope. They depend on tracker-store either way. Since that's the case, then it's better to make tracker-store an impl. detail of libtracker-sparql > I suppose it doesn't make all that much difference, you still > require the store and other bits for libtracker-sparql to operate > anyway. Precisely. Applications wanting to JUST run SPARQL queries, need tracker-store. Even for reading with direct-access, as tracker-store is the one that assembles meta.db for in case it's not there yet and it needs to be created from initial ontology. > I guess you could go as far as to say that you might want to JUST > use DBus and none of the libtracker-sparql support at all, another > reason for it being separate. The DBus-only non-fd-passing Query APIs on tracker-store are purely for development purposes. Nobody should really use them in production. The only meaningful API on the Resources DBus object is GraphUpdated signal. The Steroids or FD-passing DBus API can be used, but preferably applications always use libtracker-sparql and get the benefits of direct-access mode (process prioritization and no-ipc overhead). >> The reason for that is that the actual goal of libtracker-sparql >> was to provide what libsqlite is for a RDMBS and SQL, but for RDF >> and SPARQL. > > Yea, but libsqlite3 and sqlite are separate packages (actually I am > not sure if they're separate projects at all), but having > tracker-store does not mean you have to use libtracker-sparql. sqlite is a package providing the sqlite shell. That would be equivalent to having a package called tracker that provides just the tracker-sparql tool. All of sqlite is in libsqlite, and you can perfectly use libsqlite without the sqlite package, like how I would like libtracker-sparql to be too (tracker is an embedded db vs. an oracle or mysql or postgresql). >> Without tracker-store, libtracker-sparql can't work. > > It's more the other way round I had in mind, without > libtracker-sparql, tracker-store works. Actually, libtracker-data (the code of tracker-store) is also the direct-access implementation of libtracker-sparql. The tracker-store itself is a bunch of in Vala written D-Bus service wrappers around the same libtracker-data (that is used for libtracker-sparql-direct's implementation) plus an eventloop and checkpointing plus some initialization code and DBus daemonization. That's it, that's what tracker-store is :) Everything is in libtracker-data, which is used by libtracker-sparql >> By splitting the ontologies into a separate project, managed by >> the Nepomuk organization or not, we could someday refactor >> libtracker-sparql to support multiple instances of installed >> ontologies. > > I wouldn't split the ontologies out, the store is no good without > the database and without the schema it's even more useless :) Sure it is. There are already a few companies that are using Tracker with a completely different ontology. The tracker-ivi project comes to mind. Will run on the dashboards of some nice cars. > Besides, all the ontology validation and handling is in the > libraries the store depends on. Not correct. All ontology validation rules are in the ontology's own introspection. The libtracker-data's ontology validation rules are created out of that ontology introspection, and nothing about it is hardcoded in it. Meaning that data/ontologies is completely separate from libtracker-data You could install a complete different data/ontologies, and libtracker-data would just happily crunch that and assembly you a different meta.db with different ontology validation rules. In fact, several embedded-solution companies do this today w. tracker. >> Ideal in that scenario would be that tracker-store becomes an >> impl. detail of libtracker-sparql (it'll manage the instances of >> stores on a on-demand basis). >> >> The only real three reasons of existence of tracker-store are: >> >> - GraphUpdated - The fact that SQLite isn't MVCC and we need WAL >> journaling and checkpointing done by a separate 'writer' process > > Yea, single point of update. But whether it comes as a implementation detail of libtracker-sparql or is a central process of a Desktop session; shouldn't be of concern to whoever links with and uses libtracker-sparql. >> - Providing a ontology > > Not sure I follow you here, the reason for the store is not to > provide an ontology - at least libtracker-data does a lot of that > stuff - I would have to double check this. libtracker-data doesn't provide the ontology, data/ontologies does. The libtracker-data implementation (which is also used 1 on 1 by libtracker-sparql's direct-access implementation) doesn't care for 99% what the ontology's content is. I think just some base types like rdfs:Resource and xsd:* need to be there. That's because of the tracker:uri() and tracker:id() functions and the root-of-them-all 'Resources' table. > The store offers a lot of buffering, queueing and general > "management" of updates (and queries). Let's not forget the DBus > interface. Yes. And the exact same buffering resides in any process that links with libtracker-sparql(-direct): you have the same LRU for SQLite statements. Just that libtracker-sparql will reroute your write queries to tracker-store, because of WAL checkpointing and GraphUpdated emitting. It's also the only reason. If SQLite would have triggers and MVCC, Jürg and me would never have introduced tracker-store in the first place. >> When your process links with libtracker-sparql and uses >> direct-access mode, it effectively has everything it needs to >> deal with meta.db. > > Indeed. Voila :) So why is tracker-store a separate central desktop-session daemon? No reason. libtracker-sparql users shouldn't care. >> The eventual dependency tree would look like this: >> >> Your program depends on: | +- nepomuk-ontology | >> +-libtracker-sparql `- Internally uses >> libexec/tracker-sparql-store >> >> This API: >> >> https://developer.gnome.org/libtracker-sparql/stable/TrackerSparqlConnection.html >> >> >> >> Would look like: >> >> #include <nepomuk/2006-2008/ontology.h> #include >> <tracker-sparql.h> >> >> static void something () { GError *error = NULL; Ontology >> *ontology = nepomuk_ontology_2006_2008_new ("session id"); >> TrackerSparqlConnection *con = >> tracker_sparql_connection_get_for_ontology (ontology) >> TrackerSparqlCursor *cursor = tracker_sparql_connection_query >> (con, "select ?a { ?a nie:title 'something in nepomuk' }", NULL, >> &error); >> >> >> while (tracker_sparql_cursor_next (cursor, NULL, NULL)) { g_print >> ("%s\n", tracker_sparql_cursor_get_string()); } >> >> g_object_unref (cursor); g_object_unref (con); g_object_unref >> (ontology); } > > Interesting idea. It could also be simpler with a: TrackerSparqlConnection * tracker_sparql_connection_get_for_ontology (const char* ontology_path, const char *session_id) of course >> This would internally deal with starting and stopping a >> libexec/tracker-sparql-store, and no global "tracker-store" would >> be needed anymore (the fact that multiple users share the same >> "session id", creates the central storage for users of that >> "session id"). >> >> Storage could go to: >> ~/.tracker/sessions/$session-id/nepomuk-2006-2008/meta.db or >> something > > This is quite an addition to what we have now. Yep, it suddenly allows for as much different ontologies and per-app domain or per session storage of user's metadata. That also allows for tighter security between sharing of metadata. Ie. we could allow access to metadata of session_id to applications a,b and c but not to d, e and f. ie. Rythmbox could decide to share metadata with Banshee, but nut with Evolution Data Server. But EDS could decide to share metadata with Sylpheed, but also with Rythmbox. And they could all have their own ontology, although the vast majority on a GNOME desktop would obviously use Nepomuk. > I don't think it makes much difference because libtracker-sparql > will always depend on a tracker-store (or whatever alias you use > for grouping components that update the DB - libtracker-data, > libtracker-fts, etc.). Yes, there's no big technical difference. Just that tracker-store disappears from sight and as a public API, and only libtracker-sparql remains as a publicly exposed API. How tracker-store gets activated? Hopefully with systemd. Else we just start and stop it ourselves in tracker_sparql_connection_get(), and/or we like today use DBus service activation. Except that we'd have a service per such a session_id or app domain or group. Not sure how that works with DBus. I guess the desktop-nepomuk-ontology.deb could install the .service file to make use of DBus activation. There are many possibilities. > Consider the application that only wants to query and not update > the DB - they don't want to depend on all the crap needed for > updating the DB, just the raw libtracker-sparql part. Except that they already and always depend on tracker-store. You can't avoid it (read libtracker-sparql's initialization code: you always and necessarily depend on it). Kind regards, Philip -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (MingW32) iQEcBAEBAgAGBQJUDd1KAAoJEEP2NSGEz4aDSY4IAKl2JiJQO8a9EXGYVPCAvQE5 GnTRrQ7Xh9iAF4CBvzyrJ8AMPxvgmGQQMsG+Zle0SMx8iGOONKH9SklvklPbpK0X kBSoKlg2P6m+QVtK5kyh7JEi51L4JE+SioxDfFAgZZtdnsL8oH1ngU2nvzu4IJzQ HwVWBYOy/vR9BPvGW4YpYdVys154S9dMXaQX/Tu+J4WV1KybMEDgrtgg5pBo0Y9j 4C9dopDiN2n/7uoLb3mhmAZkE4pXlsd4ECkuOkIqyCUtaig1/Iv/htSWdKePs1pn O2VShf9JcEvTnNrl/Ugyj9eo28nwQpau2kv9Gl8oTAj7miy9c21+QBwUFCzbHtY= =ZSs6 -----END PGP SIGNATURE----- _______________________________________________ desktop-devel-list mailing list [email protected] https://mail.gnome.org/mailman/listinfo/desktop-devel-list
