On 8/21/05, Dirk Meyer <[EMAIL PROTECTED]> wrote: > Hi, > > The last few days I was thinking about a new mediadb. We did several > rewrites in the past, so let us do this right this time. Martijn tried > to put sqlite in the freevo mediadb, but he wanted to keep the old > interface which turned out to be a bad idea. So this mail should > explain my ideas and I hope to get some comments. > > First: the interface. Tack wrote about a 'vfs' for mebox with > sqlitedb. I guess he is right. The new mediadb shouldn't be a mediadb, > it should be a vfs which can be used as mediadb. So (unlike the > subject of the mail), I suggest we call it kaa.vfs.
agreed > > It is similar to the normal os operations. You have a listdir and you > can open files. But there are major differences: listdir->ok. an abstraction layer opening files can be tricky, please explain. > > o You can always write a file. If it is not possible in the fs > (e.g. on a rom drive), an overlay dir similar to the current freevo > vfs will be used. > o listdir returnes a Listing object similar to the current freevo > mediadb code. Hidden files (.*) are ignored by the vfs. > o Similar to listdir is a query function. With that you can get a > Listing object from all files were e.g. Artist=Enya. > o The Listing object has no simple list of files as string, all files > are objects, holding it's name and metadata (e.g. the stuff from > kaa.metadata) > o You can store three types of variables in an object. > 1. Stuff you want to be stored as key in the db > 2. Stuff you will nevery query on (this will be saved as pickle > inside the db to save time) I'm not shure if the extra complexity is worth it. > 3. tmp Variables that are in the object but won't be added to the db > and will be lost when the object is deleted. > o Support of removable drives > o Everything is a file. A DVD with it's title listing is only a file > on the disc. So a disc may contain a dir (normal data disc) or only > one file (DVD/VCD/AudioCD). By doing this there is no difference > between a DVD image on hd or on disc. > o Even titles on a dvd/vcd/audiocd are files in the 'directory' > complete-disc-file. > > OK, that's the interface. Now to the db itself. Tack wants a > vfs-server, I prefer a thread. His ipc stuff may be fast, but it is > faster if you don't need it at all. And sqlite works without a server, > why should be destroy this feature. But thread and process are > similar. They have requests from the main loop (client) to the db > handling code (server). It would be easy to add process support to it. The mediadb could be in the main thread. But if the ipc stuff is really good i'll change my opinion;) , i'll try to catch up. metadata parsing should be done in another thread or subprocess of the main mediadb > > Next, I plan to have the following tables: > > o global: key/values stuff that can be stored as global informations > about the vfs. They do not belong on any files. > o dir: basic information about a dir, e.g. dirname and cover > o file: basic information about a file belonging to a dir, > e.g. basename, id of the dir, cover, title, mtime > o file_video, file_audio, file_image: additinal informations about the > media types I don't think we really need the file_video,file_image etc. I'm not shure if the extra complexity will enhance speed or slow it down. metadata-getting now needs 2 selects: 1 from attributes and 1 for the file_media table, this will not be faster then 100% attributes. (title+image from file_table already present) > o attributes: a table were attributes id/key/value can be stored for > a file which do not fit in any other table. > > So to get information about /foo/bar.avi you need one entry from dir > foo, the file bar.avi, the file_video with the same id and all > attributes with that id. Joining that much tables will be slow. > > But if you look at Freevo and possible other use cases, you don't need > all information at once. When Freevo creates a listing, it will show > the title, the filename or a cover of the item. And Freevo will sort > by title or mtime. It is unlikly that you will sort an image listing > by width, so we should optimize the db for the common use case. You > have all this in two tables: dir and file. When getting a dir listing, > you only need two selects and no join. That is much faster. > > BUT, for one item (the selected one), you need more information. No > problem. When requesting a listing, the thread will return a simple > basic information list. After that it will scan the db for the other > informations and adds it to the File objects in the Listing. When the > information about a File is requested before this is done, the object > will force an update on this information. In code: > > class File(object): > ... > > __getitem__(self, key): > if not has_all_informations: > request_info_from_db > notifier.step() And what if the file has not been indexed yet? *request_info_from_db forces indexing? *or if not has_been_indexed():index_file() > Thread: > > get request for listing > get data dir/file from db, return it, wake up main loop > start getting all other information about files and give it to the > main loop > if a request comes from one item, stop the getting-all-info stuff > and handle this item > continue with getting all informations > > > Comments? Done ;). Martijn. ------------------------------------------------------- SF.Net email is Sponsored by the Better Software Conference & EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf _______________________________________________ Freevo-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/freevo-devel
