Re: [Zeitgeist] [Merge] lp:~tdfischer/zeitgeist/event-caching into lp:zeitgeist

2012-03-30 Thread Trever Fischer
Updated. I'll start working on providing some analysis.
-- 
https://code.launchpad.net/~tdfischer/zeitgeist/event-caching/+merge/97450
Your team Zeitgeist Framework Team is subscribed to branch lp:zeitgeist.

___
Mailing list: https://launchpad.net/~zeitgeist
Post to : zeitgeist@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zeitgeist
More help   : https://help.launchpad.net/ListHelp


Re: [Zeitgeist] [Merge] lp:~tdfischer/zeitgeist/event-caching into lp:zeitgeist

2012-03-26 Thread Siegfried Gevatter
Review: Needs Fixing

Hi Trever,

Nice work, thanks for working on this!

Some comments:

 max_cache_size = 1024;
We have a (so far unused) constant for this in src/utils.vala: Utils.CACHE_SIZE.

+uncached_ids.resize(uncached_ids.length+1);
+uncached_ids[uncached_ids.length-1] = event_ids[i];
You can use += here (which looks nicer and does power-of-two allocations).

+cache = new EventCache();
  ^ space
+ for(int i = 0; i  event_ids.length; i++)
 ^ space
And more spaces missing in function declarations in event-cache.vala.

Also, see the comment:

// TODO: Consider if we still want the cache. [...]
//  It'd also benchmark it again first, we may have better options
//  to enhance the performance of SQLite now, and event processing
//  will be faster now being C.

Some numbers about how this affects performance for different queries (cold 
query vs hot query vs mixed query with and without cache).
-- 
https://code.launchpad.net/~tdfischer/zeitgeist/event-caching/+merge/97450
Your team Zeitgeist Framework Team is subscribed to branch lp:zeitgeist.

___
Mailing list: https://launchpad.net/~zeitgeist
Post to : zeitgeist@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zeitgeist
More help   : https://help.launchpad.net/ListHelp


[Zeitgeist] [Merge] lp:~tdfischer/zeitgeist/event-caching into lp:zeitgeist

2012-03-14 Thread Trever Fischer
Trever Fischer has proposed merging lp:~tdfischer/zeitgeist/event-caching into 
lp:zeitgeist.

Requested reviews:
  Zeitgeist Framework Team (zeitgeist)

For more details, see:
https://code.launchpad.net/~tdfischer/zeitgeist/event-caching/+merge/97450

Implements a LRU cache for events, to reduce hitting the database on every 
request.
-- 
https://code.launchpad.net/~tdfischer/zeitgeist/event-caching/+merge/97450
Your team Zeitgeist Framework Team is requested to review the proposed merge of 
lp:~tdfischer/zeitgeist/event-caching into lp:zeitgeist.
=== modified file 'extensions/fts++/Makefile.am'
--- extensions/fts++/Makefile.am	2012-02-12 20:17:52 +
+++ extensions/fts++/Makefile.am	2012-03-14 16:35:57 +
@@ -42,6 +42,7 @@
 	ontology-uris.vala \
 	mimetype.vala \
 	ext-dummies.vala \
+	event-cache.vala \
 	$(NULL)
 
 libzeitgeist_internal_la_SOURCES = \

=== modified file 'src/Makefile.am'
--- src/Makefile.am	2012-02-15 19:55:59 +
+++ src/Makefile.am	2012-03-14 16:35:57 +
@@ -48,6 +48,7 @@
 	ontology.vala \
 	ontology-uris.vala \
 	mimetype.vala \
+	event-cache.vala \
 	$(NULL)
 
 zeitgeist_daemon_SOURCES = \

=== modified file 'src/db-reader.vala'
--- src/db-reader.vala	2012-03-14 12:31:51 +
+++ src/db-reader.vala	2012-03-14 16:35:57 +
@@ -45,6 +45,8 @@
 protected TableLookup mimetypes_table;
 protected TableLookup actors_table;
 
+protected EventCache cache;
+
 public DbReader () throws EngineError
 {
 Object (database: new Zeitgeist.SQLite.Database.read_only ());
@@ -59,6 +61,8 @@
 manifestations_table = new TableLookup (database, manifestation);
 mimetypes_table = new TableLookup (database, mimetype);
 actors_table = new TableLookup (database, actor);
+
+cache = new EventCache();
 }
 
 protected Event get_event_from_row (Sqlite.Statement stmt, uint32 event_id)
@@ -117,7 +121,20 @@
 if (event_ids.length == 0)
 return new GenericArrayEvent? ();
 
-var sql_event_ids = database.get_sql_string_from_event_ids (event_ids);
+var results = new GenericArrayEvent? ();
+uint32[] uncached_ids = new uint32[0];
+for(int i = 0; i  event_ids.length; i++)
+{
+Event? e = cache.get_event (event_ids[i]);
+if (e == null) {
+results.set(i, e);
+} else {
+uncached_ids.resize(uncached_ids.length+1);
+uncached_ids[uncached_ids.length-1] = event_ids[i];
+}
+}
+
+var sql_event_ids = database.get_sql_string_from_event_ids (uncached_ids);
 string sql = 
 SELECT * FROM event_view
 WHERE id IN (%s)
@@ -149,12 +166,13 @@
 }
 
 // Sort events according to the sequence of event_ids
-var results = new GenericArrayEvent? ();
 results.length = event_ids.length;
 int i = 0;
 foreach (var id in event_ids)
 {
-results.set(i++, events.lookup (id));
+Event e = events.lookup (id);
+cache.cache_event (e);
+results.set(i++, e);
 }
 
 return results;

=== modified file 'test/direct/Makefile.am'
--- test/direct/Makefile.am	2012-02-15 18:55:42 +
+++ test/direct/Makefile.am	2012-03-14 16:35:57 +
@@ -11,6 +11,7 @@
 
 TESTS = \
 	datamodel-test \
+	event-cache-test \
 	marshalling-test \
 	mimetype-test \
 	query-operators-test \
@@ -36,8 +37,12 @@
 	$(top_srcdir)/src/ontology.vala \
 	$(top_srcdir)/src/ontology-uris.vala \
 	$(top_srcdir)/src/mimetype.vala \
+	$(top_srcdir)/src/event-cache.vala \
 	$(NULL)
 
+event-cache-test: event-cache-test.vala $(SRC_FILES)
+	$(VALA_V)$(VALAC) $(VALAFLAGS) -o $@ $^
+
 datamodel-test: datamodel-test.vala $(SRC_FILES)
 	$(VALA_V)$(VALAC) $(VALAFLAGS) -o $@ $^
 
@@ -66,11 +71,13 @@
 	query-operators-test \
 	table-lookup-test \
 	where-clause-test \
+	event-cache-test \
 	$(NULL)
 
 EXTRA_DIST = \
 	assertions.vapi \
 	datamodel-test.vala \
+	event-cache-test.vala \
 	marshalling-test.vala \
 	mimetype-test.vala \
 	query-operators-test.vala \

___
Mailing list: https://launchpad.net/~zeitgeist
Post to : zeitgeist@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zeitgeist
More help   : https://help.launchpad.net/ListHelp