Members present: tkurz, westei, dglachs, sschaffert, jfrank, Wikier ---------------- Meeting summary: ----------------
1. Preface 2. reasoner 3. triplestore migration 4. linked data client and caching 5. other dev tasks 6. ldp 7. branding 8. aob -------- Actions: -------- - create sub-task for the LDP implementation (Wikier, 10:09:25) IRC log follows: # 1. Preface # 09:03:30 [Wikier]: ok, agenda? 09:04:00 [sschaffert]: my topics: 09:04:00 [Wikier]: I'd suggest the following topics: triple store, other dev tasks, ldp, branding, aob 09:04:00 [sschaffert]: - reasoner 09:04:07 [sschaffert]: - triple store 09:04:15 [sschaffert]: - linked data client and caching 09:04:52 [Wikier]: ok, go ahead 09:05:00 [sschaffert]: - merging triplestore branch back to main branch 09:05:07 [sschaffert]: ok, so quick things first # 2. reasoner # 09:05:45 [sschaffert]: just to inform you that I stopped working on the reasoner in favor of migrating the core system to the new purely Sesame-based triple store 09:05:52 [jfrank]: +1 09:06:16 [sschaffert]: this means that for the time being, reasoning will not be available in the LMF, 09:06:45 [sschaffert]: but it allows merging back the branch much faster so you all can help with the adaptations 09:07:00 [sschaffert]: which brings me to the next topic # 3. triplestore migration # 09:07:53 [sschaffert]: the goal here is to remove all direct dependencies to the (old) KiWi triple store from the LMF/Marmotta and replace them with backend-agnostic use of the Sesame Repository API 09:08:15 [sschaffert]: some steps have already been taken by moving the old KiWi API to Sesame for the 2.4 release 09:08:26 [sschaffert]: but the changes are still significant 09:08:45 [sschaffert]: most of lmf-core will disappear, especially the triple store and the transaction system 09:09:09 [Wikier]: of course 09:09:15 [tkurz]: did you replace the whole kiwi triple api? or wrap it? 09:09:15 [sschaffert]: the transaction system is replaced by the new Sesame 2.7 transaction API plus very lightweight KiWi extensions 09:09:45 [sschaffert]: so what does it mean for you: 09:10:07 [sschaffert]: - no direct references to the KiWi data model anywhere, ONLY use the Sesame model and API 09:11:00 [sschaffert]: - no direct use of the transaction system (TransactionService and related model) anywhere, instead it is mandatory that you use the Sesame RepositoryConnection properly 09:11:15 [sschaffert]: Sesame 2.7 has changed the Repository API in this regard 09:11:30 [Wikier]: ok, good to know it 09:11:45 [sschaffert]: it is now necessary to explicitly start the transaction with connection.begin() and end it with connection.commit() or connection.rollback() 09:12:00 [jfrank]: what happens if not? 09:12:00 [sschaffert]: and never forget to close the connection after you used it with connection.close() 09:12:00 [Wikier]: therefore I priorize the merge of this work to the default branch, so all of us we can switch to that new api 09:12:38 [sschaffert]: if you don't commit the connection, the underlying database transaction will also not be committed, so the data will not be in the database, simple ;-) 09:13:08 [jfrank]: and changes before connection.begin() 09:13:15 [sschaffert]: if you don't close the connection, the underlying JDBC connection will not be closed, so you will have a resource exhaustion problem quite quickly 09:13:45 [sschaffert]: @jfrank: don't do it, this is undefined in the future 09:14:08 [sschaffert]: at the moment, there is still the "autocommit" mode, but this is deprecated in Sesame 2.7 beta 09:14:23 [sschaffert]: ok, back to what the changes mean to you: 09:15:25 [sschaffert]: - since the new triple store does not use Hibernate and we want to become independent from the Sesame backend, you should not persist any entity via Hibernate that holds a reference to a triplestore object 09:16:15 [sschaffert]: this has serious consequences in many modules (e.g. lmf-cache stored "CacheEntry" entities with a reference to the resource), 09:16:16 [sschaffert]: for now I disabled all these modules 09:16:38 [sschaffert]: we need to find different solutions in these cases 09:16:53 [sschaffert]: in some cases it will be possible to store a reference to the string uri instead 09:17:08 [sschaffert]: in other cases we would probably use a completely different storage anyways 09:17:38 [sschaffert]: so let's summarize, changes you need to take into account: 09:17:47 [jfrank]: what about a "system graph", storing this extra things as triples? 09:17:48 [sschaffert]: - no direct use of KiWi RDF model, only use Sesame API 09:18:01 [sschaffert]: - no direct use of transaction system, only use Sesame API 09:18:24 [sschaffert]: - rewrite all modules that persist entities with reference to objects from the RDF model 09:18:48 [sschaffert]: system graph can be an option, yes 09:19:15 [sschaffert]: actually with the new kind of implementation it will be possible to have completely separate triple stores 09:19:31 [sschaffert]: one could be used for the system and others for different instances or even containers 09:20:07 [sschaffert]: but the limitation of the KiWi triple store at the moment is: one triplestore per database 09:20:22 [sschaffert]: so this would mean: two databases for the LMF 09:20:45 [sschaffert]: ok, but this is maybe a topic for tomorrow 09:20:52 [jfrank]: why? same db, only different context. 09:21:23 [sschaffert]: I would prefer completely separate triple stores to avoid any security problems (or SPARQL queries without restrictions) 09:21:45 [sschaffert]: but we can discuss this tomorrow or on Thursday 09:21:53 [jfrank]: ok 09:22:00 [sschaffert]: my timeline is as follows: 09:22:15 [sschaffert]: - today I'd like to have a running version of lmf-core with the new triple store 09:22:31 [sschaffert]: - this afternoon I'd like to merge back the triplestore branch to the default branch 09:22:52 [sschaffert]: - tomorrow you should be able to work on your modules then 09:23:07 [sschaffert]: but please note that everything will be extremely unstable, because there are many significant changes 09:23:24 [sschaffert]: especially, the removal of the current transaction system could have unpredictable consequences 09:23:52 [sschaffert]: this is it mainly regarding the triple store 09:24:00 [sschaffert]: now questions :) 09:24:32 [sschaffert]: ok, they will come once everything is broken :-P 09:24:45 [jfrank]: unstable system. hurray! 09:24:47 [sschaffert]: so then I have a related topic # 4. linked data client and caching # 09:25:22 [Wikier]: yes, because here I guess some major changes are required, right? 09:25:22 [sschaffert]: the linked data caching module (lmf-cache) is one of the central modules that are no longer working 09:25:46 [sschaffert]: the reason it is not working is that the caching system in the old version stored caching information in the database 09:26:07 [sschaffert]: many other modules depend on lmf-cache, even though they would be unproblematic themselves 09:26:22 [sschaffert]: so I have some ideas on this topic 09:26:39 [sschaffert]: I would like to separate the linked data client and the linked data caching functionality 09:26:53 [sschaffert]: and implement it in a similar way to ldpath 09:27:15 [sschaffert]: for this reason I have created in the triplestore branch two new top-level modules 09:27:16 [sschaffert]: ldclient 09:27:22 [sschaffert]: and ldcache 09:27:22 [Wikier]: I'd say that both LDPath and LDCache should use the same linked data clients 09:27:37 [sschaffert]: LDPath uses no linked data client 09:27:39 [sschaffert]: it has an abstraction 09:28:00 [Wikier]: but maybe at some point it should... 09:28:15 [sschaffert]: ok wait 09:28:15 [sschaffert]: let me first explain 09:28:17 [Wikier]: sure 09:28:54 [sschaffert]: so ldclient should provide basic linked data client functionality, i.e. retrieving resources and getting back triples for them 09:29:00 [sschaffert]: there would be 09:29:22 [sschaffert]: ldclient-api: provides interfaces and data model, particularly the model for DataProviders 09:29:47 [sschaffert]: ldclient-core: connection management and response handling, provides the main library (similar to LDPath) 09:30:17 [sschaffert]: ldclient-backend-rdf: data provider for accessing real/proper Linked Data resources that offer data directly in RDF 09:30:30 [sschaffert]: ldclient-backend-rdfa: data provider for accessing RDFa documents 09:30:38 [sschaffert]: sorry, s/backend/provider/ 09:31:00 [sschaffert]: ldclient-provider-xml: the current abstraction for XML-based datasets 09:31:22 [sschaffert]: and then there could be further providers, e.g. based on xml like ldclient-provider-youtube or ldclient-provider-vimeo 09:31:30 [westei]: so a ldclient-provider-ldap would access an LDAP server and provide data as RDF 09:31:37 [sschaffert]: yes 09:31:45 [sschaffert]: actually this already exists but at the moment is deep inside the LMF 09:32:01 [jfrank]: so what's the difference between ldclient-provider-X and sesame-rio-X? 09:32:07 [sschaffert]: the advantage would be that anyone could be able to use the library to access web resources as RDF, independently of Marmotta 09:32:15 [westei]: I would like to use such a system in Stanbol for the Entityhub 09:32:17 [sschaffert]: sesame-rio is just a parser 09:32:45 [sschaffert]: ldclient-provider-X must do more, e.g. URL handling 09:33:00 [sschaffert]: for example, the ldclient-provider-phpbb that you implemented yourself :) 09:33:32 [sschaffert]: @westei: yes, this would be the point of it, then it would also be possible to use it in other systems 09:33:45 [sschaffert]: and actually the change is not too big 09:34:00 [sschaffert]: it is just a bit refactoring what we already have 09:34:22 [Wikier]: aja 09:34:22 [sschaffert]: I will do this this afternoon before or after merging back 09:34:30 [Wikier]: cool 09:34:37 [jfrank]: and what is the entry-point for ldclient? 09:34:45 [sschaffert]: but then I need Jakob to take over and improve 09:35:00 [sschaffert]: not sure yet 09:35:09 [sschaffert]: probably new LDClient() ;-) 09:35:30 [sschaffert]: ok, so much regarding ldclient 09:35:37 [sschaffert]: now ldcache 09:35:45 [westei]: I can care about making the plugin system work nicely with OSGI 09:35:52 [sschaffert]: yes, great 09:36:02 [sschaffert]: ldcache would be structured in a similar way 09:36:46 [sschaffert]: - ldcache-api provides the interfaces for the caching backends 09:36:56 [sschaffert]: - ldcache-core provides the basic caching functionality 09:37:39 [sschaffert]: - ldcache-sail provides a StackableSail that triggers retrieving resources on demand (i.e. when the repository API is used to query for a subject URI) 09:37:56 [sschaffert]: - ldcache-backend-X provides different backend implementations for storing the caching information 09:38:18 [sschaffert]: the idea with the backend is that the caching system could then be used in different systems and with different backends 09:38:47 [sschaffert]: for example, we could still have a ldcache-backend-kiwi that would use the JDBC database connection of the KiWi triple store to directly store caching metadata 09:38:55 [jfrank]: why ldcache-core and -sail? is there a usecase for -core without -sail? 09:38:57 [sschaffert]: yes 09:39:47 [sschaffert]: retrieving a resource and caching the results, without integrating this into a repository with transparently triggered retrieval 09:39:55 [sschaffert]: I think Stanbol could benefit from this 09:40:17 [sschaffert]: westei requested this feature long ago 09:40:40 [jfrank]: but what do i get from retrieving a resource if not a repository? 09:40:57 [sschaffert]: these are two different repositories 09:41:12 [jfrank]: got it! 09:41:32 [sschaffert]: what you get from the cache is: a repository containing the triples for the requested resource + caching metadata (expiry etc) 09:41:47 [westei]: does "ldcache-sail" mean that ldcache is bound to the Sail API? I guess not 09:41:49 [sschaffert]: the StackableSail is a replacement for what we currently did with event handling 09:42:10 [sschaffert]: yes and no 09:42:32 [sschaffert]: ldcache for the moment will return a Sesame repository, because this is what we already have implemented 09:42:47 [jfrank]: ldcache-sail is to integrate ldcache with any SesameRepository. 09:42:57 [sschaffert]: at a later stage we might abstract away from this similarly to ldpath, but this requires careful thinking 09:43:17 [westei]: ok than +1 09:43:25 [jfrank]: why add an other abstraction layer there? 09:43:26 [sschaffert]: @jfrank: yes, you can wrap this around any Sail and extend it with transparent LD access 09:43:47 [sschaffert]: this is starting to get complicated 09:44:10 [jfrank]: ok, maybe postpone this discussion... 09:44:10 [sschaffert]: the other abstraction layer would be for letting the cache and client return other kinds of repositories, e.g. a Jena model 09:44:25 [sschaffert]: or a Clerezza whatever tripleset 09:44:41 [sschaffert]: this would be needed by Stanbol, at least as long as it still uses Clerezza ;) 09:44:47 [westei]: For me a "Cache" has a mich simpler interface than an TripleStore e.g. a Cache could be limited to resource based retrieval without support for #filter(subject, predicate,object), SPARQL ... 09:45:10 [sschaffert]: well, the cache should somehow return a collection of triples 09:45:28 [sschaffert]: the straightforward way (we implemented now) is to return a Sesame in-memory repository 09:45:34 [sschaffert]: even if we don't run SPARQL over it 09:45:42 [sschaffert]: but this is the abstraction that is needed 09:45:48 [sschaffert]: let us discuss this at a later point 09:45:50 [sschaffert]: for now 09:46:17 [sschaffert]: both ldclient and ldcache will still return a Sesame repository, because it is simpler and we cannot open too many new tasks 09:46:18 [westei]: I just want to be able to implement an cloud storage based Cache 09:46:27 [westei]: e.g. with HBase 09:46:41 [sschaffert]: yes, good point 09:46:57 [Wikier]: +1 09:47:02 [sschaffert]: can be solved by implementing a cloud storage Sesame repository :) 09:47:02 [Wikier]: but for >= 3.0 09:47:17 [jfrank]: >> 3.0 09:47:25 [Wikier]: xD 09:47:40 [sschaffert]: ok, I occupied enough of your time 09:47:40 [sschaffert]: move to the next topics 09:48:03 [westei]: if you think of the Sail interface than true, but if you think of a "URI -> List<Triple>" things get a lot easier 09:49:25 [sschaffert]: never List :) 09:49:40 [sschaffert]: semantically incorrect 09:49:40 [sschaffert]: Set maybe 09:49:57 [sschaffert]: but let's discuss this at a later point 09:50:10 [westei]: ok 09:50:12 [sschaffert]: we can add Jira issues once I moved the current implementation 09:50:25 [Wikier]: ok 09:50:26 [sschaffert]: just rest assured it moves in the right direction also for stanbol ;) 09:50:49 [Wikier]: so, next topic 09:53:17 [Wikier]: #topic: other dev tasks 09:53:47 [Wikier]: umm... ASFBot looks zombie # 5. other dev tasks # 09:54:04 [sschaffert]: we have again killed him 09:54:12 [sschaffert]: heap space ;) 09:54:13 [Wikier]: no, syntax issue 09:54:18 [Wikier]: ok 09:54:40 [Wikier]: tkurz is working on the new admin ui 09:55:10 [tkurz]: I created the new admin interface and started to go through any admin page to clean it 09:55:10 [Wikier]: see MARMOTTA-14 09:55:17 [Wikier]: I like it 09:55:25 [Wikier]: just two points: 09:55:25 [tkurz]: already done for core module 09:55:40 [Wikier]: - colors: can we try to keep the SW colors somehow/somewhere? 09:55:47 [tkurz]: today I will start to align the other modules 09:55:55 [Wikier]: - the ordering issue 09:56:26 [jfrank]: - the sub menu items 09:56:41 [jfrank]: ... we'll be scrolling again 09:57:10 [tkurz]: ordering is configurable 09:57:32 [Wikier]: so letÅ enable it by defaul 09:57:34 [tkurz]: scrolling is not the big issue anymore. In the former UI you have to scroll after opening a submenu 09:57:47 [Wikier]: archives can't be on top, don't you think? 09:58:10 [tkurz]: yes. I set it to enable 09:58:10 [tkurz]: by default 09:58:40 [jfrank]: ordering is configured by weight-param in kiwi-module.properties 09:59:17 [Wikier]: ok, good 09:59:25 [tkurz]: I know. But ordering is disablked by default 09:59:32 [tkurz]: I'll change this 10:00:02 [tkurz]: furthermore I cleaned the whole ui stuff (deleted images etc) 10:00:04 [Wikier]: going according the plan, thx tkurz 10:00:26 [Wikier]: related, we should try to update all textual content at every module 10:00:40 [Wikier]: for instance, the content at lmf-sparql is quite useless I think 10:00:47 [Wikier]: but first the ui 10:00:49 [Wikier]: cool 10:01:02 [Wikier]: any other dev topic? 10:01:03 [tkurz]: yes. I'll check this and write Jira issues 10:01:19 [jfrank]: Java6 is close to EOL 10:01:41 [sschaffert]: Java6: yes, we should switch to Java7 now 10:01:56 [sschaffert]: I think it is reasonable to do so 10:02:13 [sschaffert]: noone should be using Java6 once the first Marmotta release is out 10:02:20 [jfrank]: before/after 2.6? 10:02:41 [Wikier]: +1 after 10:02:49 [sschaffert]: ok 10:02:49 [tkurz]: +1 10:03:19 [Wikier]: ok, next topic 10:03:34 [jfrank]: ok # 6. ldp # 10:03:55 [Wikier]: yesterday I attend my first telco of the working group 10:04:10 [Wikier]: still many core issue being discussed 10:04:34 [sschaffert]: yes, this is what I guess from scanning the mailinglist 10:05:02 [Wikier]: for instance, they are discussing the model and interaction model 10:05:40 [Wikier]: so, to provide proper feedback, I'm starting to implement some ldp features 10:05:47 [Wikier]: such as LDP-C, see MARMOTTA-24 10:06:11 [Wikier]: probably tomorrow I'll open the discussion to [email protected] 10:06:40 [Wikier]: I guess they will not arrive to June 1st as planned 10:07:10 [Wikier]: much better for us, for introducing some discussion about some stuff (versioning and so on) 10:07:17 [sschaffert]: indeed 10:07:19 [Wikier]: that's it for the moment 10:07:32 [tkurz]: maybe we should start with a simple implementation and do not take too much care about the container stuff 10:07:47 [Wikier]: tkurz: maybe we can work together in MARMOTTA-24 10:08:02 [tkurz]: okay! 10:08:10 [Wikier]: later I'd like to check how our resource implementation fit with LDP-R 10:08:25 [sschaffert]: so then maybe divide MARMOTTA-24 into subtasks 10:08:32 [Wikier]: because the spec, of course, asserts that not every LD resource is a LDP-R 10:09:03 [Wikier]: so not sure about some implementation details (such as shared endpoint and so on) 10:09:25 [Wikier]: #action create sub-task for the LDP implementation 10:09:27 [Wikier]: ok 10:10:02 [Wikier]: for 2.6 I only plan to include a very early implementation 10:10:10 [sschaffert]: yes, I suggest to focus on LDP-R 10:10:13 [Wikier]: but at least something to be able to understand some things 10:11:02 [tkurz]: yes. we should also follow our ideas of meta and content 10:11:10 [Wikier]: well, there are some issue around LDP-C (aggregation vs. composition) that I'd like to experiment with an implementation 10:11:17 [Wikier]: but yes, LDP-R is mucho more usefull 10:11:25 [sschaffert]: well, for now at least 10:11:40 [sschaffert]: and as thomas said, include the meta/content thing of the LMF 10:11:55 [Wikier]: yes, at least point to it 10:12:25 [Wikier]: LDP will be very basic regarding those (actual) things 10:12:32 [Wikier]: ok, that's it # 7. branding # 10:12:42 [Wikier]: summary: 10:12:55 [Wikier]: still awaiting for infra about the site publication 10:13:19 [Wikier]: I got some good feeback from [email protected] 10:13:27 [Wikier]: later I'll send it to the list 10:13:47 [Wikier]: but basically we can proceed to use the logo # 8. aob # 10:14:27 [sschaffert]: ok, so we can tell Daniela today so she can create different versions 10:14:50 [Wikier]: wait until I get the full confirmation 10:14:55 [Wikier]: but yes 10:15:10 [Wikier]: what else? 10:15:47 [sschaffert]: nothing from me :) 10:16:10 [sschaffert]: it compiles! ;) 10:16:10 [Wikier]: tkurz, jfrank, dglachs? 10:16:10 [tkurz]: me neither 10:16:32 [Wikier]: ok, done for today 10:16:33 [Wikier]: 1h15 10:16:40 [Wikier]: :-S 10:16:47 [sschaffert]: we need to improve 10:16:55 [jfrank]: hwo was talking about 15-20 mins? 10:16:55 [Wikier]: next meeting tomorrow 10:17:02 [Wikier]: same time, same place ;-) 10:17:02 [westei]: thats less than 10% of a typical working day ... 10:17:11 [dglachs]: thx 10:17:20 [Wikier]: westei: this is just coordination, not actuall working ;-) 10:17:32 [Wikier]: so we should try to be much more executive
