Re: kio and frameworks 5
On Sat, 2012-05-19 at 10:52 -0400, Dawit A wrote: On Fri, May 18, 2012 at 11:31 PM, Ben Martin monke...@users.sourceforge.net wrote: I'm wondering if there is a timeframe for KIO refresh in KDE5? Getting improved metadata handling in there would be a great benefit IMHO. To the point where I might start attaching patches to that effect ;) What does improved meta-data handling mean ? Sending additional ioslave specific information to/from the ioslave and vise-versa ? If that is the case, then it is already possible to do that right now to some degree using KIO's meta data system. Excellent. I'll have to dig into this to see how much of libferris' metadata I can expose for read/write if it was a kio slave. I'm thinking about metadata like KFileMetaInfo can offer: a collection of key-value pairs each of which has schema data associated. The schema for KFileMetaInfoItem::properties() gives the primary type related stuff I'm thinking about. signature.asc Description: This is a digitally signed message part
Re: kio and frameworks 5
I'm wondering if there is a timeframe for KIO refresh in KDE5? Getting improved metadata handling in there would be a great benefit IMHO. To the point where I might start attaching patches to that effect ;)
Re: kio and frameworks 5
On Fri, May 18, 2012 at 11:31 PM, Ben Martin monke...@users.sourceforge.net wrote: I'm wondering if there is a timeframe for KIO refresh in KDE5? Getting improved metadata handling in there would be a great benefit IMHO. To the point where I might start attaching patches to that effect ;) What does improved meta-data handling mean ? Sending additional ioslave specific information to/from the ioslave and vise-versa ? If that is the case, then it is already possible to do that right now to some degree using KIO's meta data system.
Re: kio and frameworks 5
On Tue, 2012-05-15 at 12:22 +0200, Aaron J. Seigo wrote: On Friday, May 11, 2012 21:33:35 Casper Clemence wrote: Libferris is an awesome piece of technology. It provides not just the traditional features of a VFS but a uniform method of access for applications and users to a large and expanding range of things. It has others have asked the what does it do that KIO does not (or does poorly) question, and i'll add this to it: is this about changing the application-facing API of KIO? I would personally advocate for more metadata allowed to flow through to/from apps from KIO. the kioslave-facing API? (e.g. replacing current kio slaves with libferris as the 'one implementation to bind them all') I think perhaps this should be a separate issue in and of itself. If both VFS can offer good interfaces then surely they can also play well with each other, however folks decide to put the blocks together. There are likely to be some blocks that libferris can provide which are not offered by existing kio slaves so there is no duplication or swapping of implementation for those. For example, the ability to mount zoneminder as a filesystem. the implementation of the various kioslaves (such as file, http, ...)? does this only apply to a subset of kioslaves (e.g. file, but not http)? iow, what is the scope of this discussion? :) IMHO it might be best to separate the discussion into what changes might be seen as desirable in the application facing KIO interface, and a separate item as to if/where/how libferris stuff plays a role in the refresh. I don't want to tread on any toes here with implementations. Starting with a general exchange of ideas at the API level might be useful for all? signature.asc Description: This is a digitally signed message part
Re: kio and frameworks 5
On Tue, 2012-05-15 at 09:52 +0200, todd rme wrote: ... One question I think might be worth asking is what (if anything) is kio missing? We have applications that do device or file access, such as ark and k3b, that are not using kio slaves for that access. Why not? Is it just a lack of time, or is there something fundamental in kio that makes this impossible, inefficient, or too difficult? If it is the latter, it would be good to fix it. If it is the former, then that implies kio does not provide something compelling enough for them to want to switch. This is a great question! One of the things I think a VFS can do with composite files like archives, mbox, xml, etc is offer additional features above and beyond raw data access. For example, indexing (at the file level) and change monitoring (ala inotify etc). Many archives are indexed already, though for whatever reason tarballs are still the de facto and suffer from linear access at many times. If an index is created on first read (and maybe stored in soprano/EA) then if the user wants file foo, the VFS might be able to be read directly from the index offset. As an implemented example, libferris does this for mounting mbox, and stores the index itself in an extended attribute of the mbox file. An example with implementation in libferris of change tracking as a plus is mounted XML. If one process changes the XML then another process viewing that XML filesystem will see normal filesystem events telling it about the change. This notification is at the subfile level, ie which XML element has changed. The second question is, will libferris help with this, or can it be made to help? If you are mounting berkeley db or XML then it might help. libferris doesn't have support for creating/burning to optical media. So I think the first question to ask is not, should we switch to libferris?, but what is preventing kio from being used more in KDE applications? If libferris is a solution, or at least part of a solution, to the problem, then maybe we should consider switching. If not, then maybe another route might be better. I would also advocate at any level that the VFS and nepomuk/soprano being good friends also helps many apps. Eg, annotation metadata being available on anything that KIO can mount. Sorry about repeating this point a few times here and there, but hopefully even if nothing else makes it into the new KIO, this idea does :) Going further, in ferris I have also implemented smushing so that metadata bundles can be associated with one or more URLs. This allows you to access the same file through many NFS mounts with uniform metadata across all access paths to the file. Such things can also help handle the case where files are moved/renamed on a file server without the client machine's knowledge. signature.asc Description: This is a digitally signed message part
Re: kio and frameworks 5
On Fri, May 11, 2012 at 10:33 PM, Casper Clemence maninal...@gmail.com wrote: I'm want to start a discussion, put an idea out there. I am also being pushy, on someone else's behalf. Although it may be a difficult discussion I think it would be a great shame to miss the opportunity to have it. I'm not trying to tell anyone to do it this way. Please excuse me for saying we when I have not contributed to KDE as a developer, I tied myself in knots trying to write in the third person and gave up. Preamble over. The discussions is this what part should libferris take in the refresh of KIO in KDE Frameworks 5? Ben Martin really should be involved in any discussion of a redesign at the very least. He clearly has put a lot of work and thought into virtual filesystems and has created something quite unique. Libferris is an awesome piece of technology. It provides not just the traditional features of a VFS but a uniform method of access for applications and users to a large and expanding range of things. It has been demonstrated to work on maemo and as a Plasma DataEngine and has a web service interface. The rate at which Ben is able to turn out new capabilities for libferris also suggests it is well written and easy to develop. A uniform method of access for applications makes developing cool new stuff easier. A uniform method of access for users leads means a powerful tool to do what you want with your data, browsing a database in your file browser and dragging and dropping data around. So there are three points I want to suggest: (1) Speak to Ben about the possible developments of kio (2) learn from libferris (3) consider adopting libferris wholesale Possibly the biggest pain point with adopting libferris wholesale is that libferris indexes and KDE already has an indexer in strigi. But before the idea of adopting it wholesale into kdelibs is thrown out we should ask whether the difficulty of solving any issues we have with it really are greater than the difficulty of writing - and the advantage of those features libferris has which would never get written into kio. Libferris can already talk to soprano and Nepomuk, perhaps it should be considared whether allowing libferris to take over from strigi could actually be a good thing. Integration of the indexer into the VFS seems quite smart especially when the VFS knows how to read the structure of documents as well as index the text and metadata. I don't think that it is an easy question, whether or not it is worth it can only be answered by looking into it in technical detail, even trying it out. One question I think might be worth asking is what (if anything) is kio missing? We have applications that do device or file access, such as ark and k3b, that are not using kio slaves for that access. Why not? Is it just a lack of time, or is there something fundamental in kio that makes this impossible, inefficient, or too difficult? If it is the latter, it would be good to fix it. If it is the former, then that implies kio does not provide something compelling enough for them to want to switch. The second question is, will libferris help with this, or can it be made to help? So I think the first question to ask is not, should we switch to libferris?, but what is preventing kio from being used more in KDE applications? If libferris is a solution, or at least part of a solution, to the problem, then maybe we should consider switching. If not, then maybe another route might be better. -Todd
Re: kio and frameworks 5
On Friday, May 11, 2012 21:33:35 Casper Clemence wrote: Libferris is an awesome piece of technology. It provides not just the traditional features of a VFS but a uniform method of access for applications and users to a large and expanding range of things. It has others have asked the what does it do that KIO does not (or does poorly) question, and i'll add this to it: is this about changing the application-facing API of KIO? the kioslave-facing API? (e.g. replacing current kio slaves with libferris as the 'one implementation to bind them all') the implementation of the various kioslaves (such as file, http, ...)? does this only apply to a subset of kioslaves (e.g. file, but not http)? iow, what is the scope of this discussion? :) -- Aaron J. Seigo signature.asc Description: This is a digitally signed message part.
Re: kio and frameworks 5
On Friday 11 May 2012 21:33:35 Casper Clemence wrote: Libferris is an awesome piece of technology. It provides not just the traditional features of a VFS but a uniform method of access for applications and users to a large and expanding range of things What does it do exactly, and how? -- David Faure, fa...@kde.org, http://www.davidfaure.fr Sponsored by Nokia to work on KDE, incl. KDE Frameworks 5
Re: kio and frameworks 5
On domingo, 13 de maio de 2012 11.53.13, David Faure wrote: On Friday 11 May 2012 21:33:35 Casper Clemence wrote: Libferris is an awesome piece of technology. It provides not just the traditional features of a VFS but a uniform method of access for applications and users to a large and expanding range of things What does it do exactly, and how? I'm more interested in what it does better than KIO. Why should we consider switching? -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kio and frameworks 5
On Sunday, 2012-05-13, David Faure wrote: On Friday 11 May 2012 21:33:35 Casper Clemence wrote: Libferris is an awesome piece of technology. It provides not just the traditional features of a VFS but a uniform method of access for applications and users to a large and expanding range of things What does it do exactly, and how? There have been a couple of blog postings on planetkde showing what it can do [1], but I have no idea how it works. Cheers, Kevin [1] http://monkeyiq.blogspot.com/search/label/libferris -- Kevin Krammer, KDE developer, xdg-utils developer KDE user support, developer mentoring signature.asc Description: This is a digitally signed message part.
Re: kio and frameworks 5
On Sunday 13 May 2012 11:32:46 Casper Clemence wrote: * it mounts lots more than an ordinary VFS - including XML documents (so the tree of the document can be browsed), databases, X (so that window position and size for example can be read and modified as if they were data in the filesystem), web browser bookmarks... .. all of which leads to the uniform method of access I mentioned where all sorts of document data, document structure and metadata becomes available in a a uniform way. All this sounds synchronous (the data is always available right away). So it probably makes sense as a kioslave (so that all applications can access all of the above via KIO API), not as a replacement for KIO (which has to provide an asynchronous way of accessing network protocols). -- David Faure, fa...@kde.org, http://www.davidfaure.fr Sponsored by Nokia to work on KDE, incl. KDE Frameworks 5
Re: kio and frameworks 5
In short it is a virtual file system. However it takes the concept further in several respects (for a fuller answer read the about page): * it mounts lots more than an ordinary VFS - including XML documents (so the tree of the document can be browsed), databases, X (so that window position and size for example can be read and modified as if they were data in the filesystem), web browser bookmarks... * it extracts useful metadata from all of these things and treats it in a consistent way, it also allows the user to add meta-data. * an interesting feature it has is agents who will intelligently tag items (files etc) giving a certain confidence that they are such a thing, the user can then accept or reject these tags. I can imagine this interacting very usefully with the whole activities thing, where intelligent suggestions as to what should be a part of your activity could make the whole experience much more useful. * ... much more (I mentioned before it works on maemo, can talk to Nepomuk, has a REST interface) .. all of which leads to the uniform method of access I mentioned where all sorts of document data, document structure and metadata becomes available in a a uniform way. As to the previous question about MS Windows support, I am having a look at what would be required to compile on that platform. Regards, C On 13 May 2012 10:53, David Faure fa...@kde.org wrote: On Friday 11 May 2012 21:33:35 Casper Clemence wrote: Libferris is an awesome piece of technology. It provides not just the traditional features of a VFS but a uniform method of access for applications and users to a large and expanding range of things What does it do exactly, and how? -- David Faure, fa...@kde.org, http://www.davidfaure.fr Sponsored by Nokia to work on KDE, incl. KDE Frameworks 5
Re: kio and frameworks 5
I thought I would hihack a reply to the windows support response to this message. For me that hasn't ever really been a priority so there isn't much support for it right now. The main issue would be the Linux file monitoring used by the file:// module. Beyond that there are some POSIX calls which might not be available on windows, for example posix_fadvise() or copying files through madvise()ed mmaped locations. I don't see any of these as show stoppers though. Just some chunks of code which will need to be ifdefed or more cleanly replaced with alternate modules and some posix glue APIs. On Fri, 2012-05-11 at 21:33 +0100, Casper Clemence wrote: I'm want to start a discussion, put an idea out there. I am also being pushy, on someone else's behalf. Although it may be a difficult discussion I think it would be a great shame to miss the opportunity to have it. I'm not trying to tell anyone to do it this way. Please excuse me for saying we when I have not contributed to KDE as a developer, I tied myself in knots trying to write in the third person and gave up. Preamble over. The discussions is this what part should libferris take in the refresh of KIO in KDE Frameworks 5? Ben Martin really should be involved in any discussion of a redesign at the very least. He clearly has put a lot of work and thought into virtual filesystems and has created something quite unique. Libferris is an awesome piece of technology. It provides not just the traditional features of a VFS but a uniform method of access for applications and users to a large and expanding range of things. It has been demonstrated to work on maemo and as a Plasma DataEngine and has a web service interface. The rate at which Ben is able to turn out new capabilities for libferris also suggests it is well written and easy to develop. A uniform method of access for applications makes developing cool new stuff easier. A uniform method of access for users leads means a powerful tool to do what you want with your data, browsing a database in your file browser and dragging and dropping data around. So there are three points I want to suggest: (1) Speak to Ben about the possible developments of kio (2) learn from libferris (3) consider adopting libferris wholesale Possibly the biggest pain point with adopting libferris wholesale is that libferris indexes and KDE already has an indexer in strigi. But before the idea of adopting it wholesale into kdelibs is thrown out we should ask whether the difficulty of solving any issues we have with it really are greater than the difficulty of writing - and the advantage of those features libferris has which would never get written into kio. Libferris can already talk to soprano and Nepomuk, perhaps it should be considared whether allowing libferris to take over from strigi could actually be a good thing. Integration of the indexer into the VFS seems quite smart especially when the VFS knows how to read the structure of documents as well as index the text and metadata. I don't think that it is an easy question, whether or not it is worth it can only be answered by looking into it in technical detail, even trying it out. --- I hope this was a useful conversation to start and I am not just being an ignorant user butting in on the development mailing list. Regards to you all Casper
Re: kio and frameworks 5
On Fri, 2012-05-11 at 21:33 +0100, Casper Clemence wrote: ... The discussions is this what part should libferris take in the refresh of KIO in KDE Frameworks 5? Ben Martin really should be involved in any discussion of a redesign at the very least. He clearly has put a lot of work and thought into virtual filesystems and has created something quite unique. Thank you Casper for all these kind words! I am very happy to discuss VFS tech and would be delighted if I could help with a refresh of KIO. Even if only at the level of contributing some of the ideas and designs I've come up with over the lifetime of libferris. A few ideas from libferris which might be useful, to kick off potential discussions; * Using the extended attribute (EA) interface to provide access to all metadata. Schemas offered for all metadata (fwiw, I used xsd with extensions in ferris). With these two, one can show any metadata in directory listings and with schema, can sort the columns correctly. The schema is also very handy for index/search on data to allow correct resolution of range queries. It can be handy to have access to the ctime, atime, and mtime as columns in a file manager depending on circumstance. Also, I have found decorator EA useful, so mtime is the time_t as an int, but mtime-display is the modification time in a user configurable human readable string format. The file manager is then free to sort on mtime when I click the mtime-display header and I get what I want, readable content sorted in the correct order. * With an emphasis on EA as above, the filesystem model becomes much closer to a virtual XML DOM. With xpath/xquery this can be a useful API for data access. * Hookup between RDF and EA. One of the backing plugins for EA in libferris is a soprano one. This allows $ echo 5 /something/cool.txt@rating So users and developers don't have to care about RDF/SPARQL for everybody to reap it's benefits. An added benefit of RDF in this case is that EA can be attached to any VFS object, be it a flickr photo or a tuple in a database. * More to come... is this seen as a useful discussion? Unfortunately I won't be at Akademy, largely due to my distance from the event. So quick fire feature exchange over beer is off the table, so to speak :/ Feel free to list some ideas which are cool from the KIO world too. Perhaps this can be a useful thread for all VFS hackers... signature.asc Description: This is a digitally signed message part
