Members present: sschaffert, westei, Wikier ---------------- Meeting summary: ----------------
1. Preface 2. triple store versioning component 3. branding 4. triplestore: sKWRL reasoner d. https://issues.apache.org/jira/browse/MARMOTTA-9 (sschaffert, 4) 5. any other business -------- Actions: -------- - add Jira issues for versioning component: deleting old versions and reverting/undoing a version (sschaffert, 09:17:16) - sKWRL goal (Monday, 21/01): redesign data model (sschaffert, 09:32:30) - sKWRL goal (Tuesday, 22/02): port the queries to new data model (sschaffert, 09:32:55) IRC log follows: # 1. Preface # 09:12:02 [sschaffert]: versioning component, reasoning component # 2. triple store versioning component # 09:12:48 [sschaffert]: so I'll give a brief report on the progress of the versioning 09:13:00 [Wikier]: good 09:13:08 [sschaffert]: the component is now implemented as it was planned, so I closed MARMOTTA-8 09:13:23 [sschaffert]: the features are; 09:13:45 [sschaffert]: - takes a log of all changes done to the triple store and stores them as versions when a transaction ends (i.e. connection.commit() is called) 09:14:10 [sschaffert]: - versions can be listed (all versions or between dates) to see the actual changes 09:14:46 [Wikier]: cool 09:14:53 [sschaffert]: - you can access a snapshot of the repository at any given point in history (using a Java Date) by requesting a special repository connection 09:15:09 [Wikier]: I think such features would be really useful in many scenarios 09:15:15 [sschaffert]: over this connection you have read-only access to the snapshot, but you can use the normal repository API including SPARQL 09:15:45 [sschaffert]: yes, especially for things like Permalinks to a version of the repository 09:16:08 [sschaffert]: now, things that are still missing in the versioning component, but not planned for now: 09:16:22 [sschaffert]: - deleting old versions (including garbage collecting deleted triples) 09:16:31 [sschaffert]: - reverting/undoing a version 09:16:52 [sschaffert]: both features can be implemented at a later stage, I'll add issues to Jira 09:17:16 [sschaffert]: #action add Jira issues for versioning component: deleting old versions and reverting/undoing a version 09:17:24 [Wikier]: perfect 09:17:53 [sschaffert]: I have written tests, the test coverage of the versioning component is about 80-90% 09:17:55 [westei]: That would be also a nice feature for the Apache Stanbol Entityhub 09:18:08 [sschaffert]: tested with PostgreSQL, MySQL, H2 09:18:31 [sschaffert]: with the typical limitation of MySQL, which cuts timestamps at the second (this makes it a bit tricky to query for snapshots) 09:18:53 [Wikier]: westei: I see many potential scenarios: LDP-C, Entity/Cont Hubs of Stanbol, etc. 09:19:01 [sschaffert]: yes, I think there are many scenarios where this kind of snapshot functionality is useful 09:19:31 [sschaffert]: the Marmotta implementation is also very efficient, there is no difference in performance if you access a snapshot compared to the latest version 09:19:46 [Wikier]: that's great 09:19:46 [sschaffert]: well, almost no difference 09:19:53 [Wikier]: almost xD 09:19:54 [sschaffert]: a different SQL statement 09:20:15 [sschaffert]: ok, so next topic 09:20:30 [westei]: but that also means that you can not delete removed triples that are still used in snapshot versions 09:20:38 [sschaffert]: yes 09:20:47 [sschaffert]: this is the garbage collection issue that I mentioned 09:20:54 [westei]: ok 09:21:08 [sschaffert]: I'll implement this at a later stage # 3. branding # 09:21:31 [Wikier]: my two tasks 09:21:45 [sschaffert]: in general, the garbage collection is a bit tricky, though, because it requires iterating over all triples and checking if they are used somewhere else 09:21:45 [sschaffert]: ok 09:22:38 [Wikier]: - source site is ready, awaiting for infra to setup the site at marmotta.incubator.apache.org (INFRA-5624) 09:22:53 [sschaffert]: I already wanted to ask - still getting 404 09:23:24 [Wikier]: - I contacted with trademarks@ regarding MARMOTTA-5, still awaiting an answer 09:24:01 [Wikier]: sschaffert: they need to setup the cms, so that VirtualHost would be based on the source code at the svn 09:24:15 [Wikier]: that's waht missing right now 09:24:23 [Wikier]: welcome, dglachs 09:24:30 [sschaffert]: welcome :) 09:24:53 [Wikier]: so, awaiting regarding both issues 09:25:08 [Wikier]: that's all for the moment about the project branding 09:25:08 [sschaffert]: ok 09:25:15 [Wikier]: next topic? # 4. triplestore: sKWRL reasoner # 09:25:40 [sschaffert]: so again my topic: the reasoner 09:25:53 [Wikier]: good 09:26:00 [sschaffert]: this refers to MARMOTTA-9 09:26:08 [sschaffert]: #link https://issues.apache.org/jira/browse/MARMOTTA-9 09:26:40 [sschaffert]: the goal is to port the existing sKWRL reasoner from the current implementation in the LMF over to the KiWi triplestore 09:27:01 [sschaffert]: this will be my task until the end of the week 09:27:11 [sschaffert]: because it is probably a bit tricky 09:27:33 [sschaffert]: the challenge is to remove the Hibernate dependency 09:27:38 [sschaffert]: this requires: 09:28:01 [sschaffert]: - designing a new data model for representing the reasoner; I am still not sure how to store reasoning programs in the database 09:28:23 [sschaffert]: the old version splitted up programs into their components 09:28:38 [sschaffert]: i.e. there was a table for the program, a table for the rules, a table for each pattern 09:29:08 [sschaffert]: and evaluating a reasoning rule was a join over the pattern table and the triple table 09:29:30 [sschaffert]: my idea now is to simplify this and only have one table where all rules are stored in their parsable syntax 09:29:53 [sschaffert]: but I am not yet sure if this has serious influence on the evaluation, so this needs to be checked 09:30:00 [sschaffert]: second requirement: 09:30:33 [sschaffert]: - translate query patterns into SQL queries; at the moment they are evaluated in a dynamically generated HQL query 09:31:15 [sschaffert]: this needs a lot of care, and could cause trouble with different database systems, so I need to think about a proper way to achieve database independence, maybe using JOOQ 09:31:22 [sschaffert]: third requirement: 09:31:30 [sschaffert]: - storing justifications properly and efficiently 09:31:45 [sschaffert]: this can probably be ported from the existing implementation 09:32:00 [sschaffert]: so goal for today is to develop the new datamodel 09:32:30 [sschaffert]: #action sKWRL goal (Monday, 21/01): redesign data model 09:32:55 [sschaffert]: #action sKWRL goal (Tuesday, 22/02): port the queries to new data model 09:33:31 [sschaffert]: and rest of the week the rest, including designing proper tests and integrating with the transaction system :) 09:33:38 [Wikier]: ambitious deadlines ;-) 09:33:56 [sschaffert]: doable, since I already know what I am doing :) 09:34:16 [sschaffert]: and this time, the code is mostly reusable, so I don't need to implement the reasoner from scratch 09:34:47 [sschaffert]: so, topic finished if there are no questions :) 09:35:52 [Wikier]: not now 09:36:22 [Wikier]: but at some point, do we plan to be able to use sKWRL on top of any sesame-based triple store? 09:36:37 [westei]: hehe wanted to ask exactly the same question ^^ 09:36:47 [Wikier]: ¬_¬ 09:36:53 [sschaffert]: yes, but it is of course tightly integrated with the KiWi implementation because it directly operates on the database 09:36:56 [Wikier]: ok 09:37:01 [sschaffert]: ah, so the answer is NO 09:37:08 [Wikier]: that's what I guess 09:37:08 [sschaffert]: unfortunately 09:37:24 [sschaffert]: the reason is that the reasoner stores information about the reasoning process in the database 09:37:31 [Wikier]: well, being possitive, one additional feature of the kiwi triple store 09:37:46 [Wikier]: not a really bad conclusion, I mean 09:37:47 [sschaffert]: maybe I can explain the data model briefly tomorrow so you see what I mean 09:37:54 [sschaffert]: no, not at all 09:38:08 [sschaffert]: this way it is a very efficient reasoner 09:38:39 [sschaffert]: so any other topic? # 5. any other business # 09:39:32 [sschaffert]: I have a small thing: 09:40:08 [sschaffert]: in the new Sesame 2.7.0beta1 there is a bug that I reported which is still not fixed (https://openrdf.atlassian.net/browse/SES-1702) 09:40:33 [sschaffert]: the problem is that certain operations on the repository result of a query do not close the result properly 09:40:40 [sschaffert]: which can lead to resource exhaustion 09:41:07 [sschaffert]: affected are the convenience methods .asList(), .asSet() and .addTo(...) 09:41:15 [Wikier]: at least bug identified 09:41:15 [sschaffert]: so don't use them right now :) 09:41:30 [Wikier]: hopefully the 2.7 final release will fix it 09:41:45 [sschaffert]: in the tests of the versioning component I have implemented a workaround method asList 09:41:47 [sschaffert]: yes 09:41:53 [Wikier]: ok, good to know it 09:42:16 [sschaffert]: I can maybe move this method to the result utilities in sesame-commons 09:43:00 [Wikier]: Thomas reported me by skype (he can't access irc now) that he is working on the admin user interface improvement (MARMOTTA-14) 09:43:37 [Wikier]: sschaffert: I'd be prefer to wait until the solve it directly in sesame, it not I agree it could go to the sesame-tools, right 09:43:45 [Wikier]: let's see... 09:43:54 [sschaffert]: yes 09:44:01 [Wikier]: ok, any other business? 09:44:10 [sschaffert]: not from my side at the moment 09:44:30 [Wikier]: ok 09:44:38 [Wikier]: next meeting tomorrow 10h00 am cet
