Re: [Musicpd-dev-team] Database backend
Asking mpd to do an 'update' on its music directory has never completed for me - on an NFS-mounted 100GB+ music library. This points to a real problem that needs to be fixed. My library is about the same size, tho it likely has far fewer files (I keep it all in Flac format), but I've never had such problems updating the database (I mostly do it locally on a 64MB machine, but I've also done it over NFS/wifi several times). It may take a couple minutes, on small machines or slow networks but nowhere near what you're talking about. Maybe you're just experiencing a bug/misfeature that can be easily fixed. Stefan -- Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry App World(TM) will have the opportunity to enter the BlackBerry Developer Challenge. See full prize details at: http://p.sf.net/sfu/Challenge ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team
Re: [Musicpd-dev-team] Database backend
Thanks for the suggestions all. Rasmus: A full database update on my setup seems to take forever (literally: I've left it running for 3 or 4 days before). J. Alexander Treuman: Yes I've thought in the past of doing something like your suggestion, using symlinks. I also have an 'unsorted' directory solely for the purpose of, well, unsorted files. However, because of my laziness this directory is massive (probably a good 40-50% the size of the entire collection). My view is, though, that the MPD database should really allow to perform these basic operations on it without me having to resort to scripts, symlinks, manually playing with modification times/dates, etc. My original thinking was that it would be nice to use something like SQL so that it's easier to integrate MPD with other music systems/setups that also use SQL. Whether it would actually make a great difference in practice, I don't know... Jeffrey: I had no idea the database was in a plain-text format! Interesting :) -- Jonny Tyers -- ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team
Re: [Musicpd-dev-team] Database backend
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jonny wrote: Thanks for the suggestions all. Rasmus: A full database update on my setup seems to take forever (literally: I've left it running for 3 or 4 days before). hmm.. i am not sure what you are doing wrong then... an UPDATE should only apply CHANGES to the database. this measn if you have removed 2 folders, only 2 folders will be removed, the rest stays as it is. i was NOT talking about the create-db option of mpd. You should never have to use that anyway, since its automatically run on 1st start of mpd. afterwards only use the update function which should always be finished in a few seonds (I have 20.000 songs and it takes not longer then 5 seconds, depending on how much has changed) J. Alexander Treuman: Yes I've thought in the past of doing something like your suggestion, using symlinks. I also have an 'unsorted' directory solely for the purpose of, well, unsorted files. However, because of my laziness this directory is massive (probably a good 40-50% the size of the entire collection). My view is, though, that the MPD database should really allow to perform these basic operations on it without me having to resort to scripts, symlinks, manually playing with modification times/dates, etc. My original thinking was that it would be nice to use something like SQL so that it's easier to integrate MPD with other music systems/setups that also use SQL. Whether it would actually make a great difference in practice, I don't know... Jeffrey: I had no idea the database was in a plain-text format! Interesting :) -- Jonny Tyers -- ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkpMn5IACgkQswz/d6k9dsV5UQCghd5osjrQ3yhAd6lVCZADffCG V14AoLQHtZAp0hLpAfeOvOxOvNVpNAxy =HIpU -END PGP SIGNATURE- -- ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team
Re: [Musicpd-dev-team] Database backend
hmm.. i am not sure what you are doing wrong then... an UPDATE should only apply CHANGES to the database. this measn if you have removed 2 folders, only 2 folders will be removed, the rest stays as it is. 30k songs here, but NFS mounted, which is almost certainly the reason. Plus there are a lot of WMA/WAV/RealPlayer/etc files in there that MPD won't like, I'd guess - it may be scanning all of those every time I update because those files will never make it into the database (because it doesn't recognise them). (Note, I'm not actually sure if MPD can't read WAVs, WMAs, etc but I use those as an example - there are all sorts of filetypes in there, some of which I'm sure MPD can't read!) i was NOT talking about the create-db option of mpd. You should never have to use that anyway, since its automatically run on 1st start of mpd. afterwards only use the update function which should always be finished in a few seonds (I have 20.000 songs and it takes not longer then 5 seconds, depending on how much has changed) I'll install mpd on the box with the music on it and see how it performs. If it really only takes a few seconds to update (after first update, of course) then I'm an idiot for not trying it before :) -- Jonny Tyers -- ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team
Re: [Musicpd-dev-team] Database backend
Max once thought about using SQLite as backend. He even was clear that, if done right, there shouldnt be ANY regression in speed. Not sure what the plans are for now. I guess there are more important things :) Fair enough. :) In the meantime, is there any way of selectively removing songs/albums/etc from the MPD database? Or is the best way to delete the database files or make MPD recreate it via --createdb and repopulate from scratch? -- Jonny Tyers -- ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team
Re: [Musicpd-dev-team] Database backend
I agree that it's sometimes awkward that the update is modification time based - for example, if you add replaygain tags to your whole library you don't really want it all to end up with the same modification date. Perhaps it's worth looking into adding a feature there, since something a lot simpler than implementing SQLite would be possible. For now, I've never actually done this but I expect you could edit the database by hand, since it's stored as text. In particular, you could delete references to songs you want to update, or reset their mtime to something old and let mpd handle the rest. And of course, for move/delete cases the update path mechanism should be sufficient, with enough care in picking the path so that it doesn't have to scan all 100GB. Jeffrey On Wed, Jul 1, 2009 at 3:24 AM, Jonny jonnyt...@gmail.com wrote: In the meantime, is there any way of selectively removing songs/albums/etc from the MPD database? Or is the best way to delete the database files or make MPD recreate it via --createdb and repopulate from scratch? -- Jonny Tyers -- ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team
Re: [Musicpd-dev-team] Database backend
On Wed, Jul 01, 2009 at 09:31:43AM -0500, Jeffrey Middleton wrote: if you add replaygain tags to your whole library you don't really want it all to end up with the same modification date. Why not? If all files were modified at the same time, they should all have the same modification time. Tracking file changes is really the only reason the modification time is stored at all. Not updating it breaks more than just mpd too. For example, rsync uses the same technique to determine which files have changed. Of course it checks file size too, but that doesn't really help when doing things like editing ID3 tags, since doing so rarely changes the file size. On Wed, Jul 1, 2009 at 3:24 AM, Jonny jonnyt...@gmail.com wrote: In the meantime, is there any way of selectively removing songs/albums/etc from the MPD database? What I do is use a fake music directory, with symlinks to my actual music. My real music directory has only a few directories, plus a todo directory with music that still needs to be sorted. In my fake music directory, I have symlinks to everything in there *except* the todo directory, keeping it out of my database. It might be a bit hard to manage if you want to keep out, say, specific albums or songs, and not just one out of a few top level directories. But it wouldn't be too difficult to write a script to create symlinks to everything in a given directory, excluding a subset of it's contents. pgpzyOZp93AUS.pgp Description: PGP signature -- ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team
Re: [Musicpd-dev-team] Database backend
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Why dont you just move the files to a non-mpd directory and do a database update? Jonny wrote: Max once thought about using SQLite as backend. He even was clear that, if done right, there shouldnt be ANY regression in speed. Not sure what the plans are for now. I guess there are more important things :) Fair enough. :) In the meantime, is there any way of selectively removing songs/albums/etc from the MPD database? Or is the best way to delete the database files or make MPD recreate it via --createdb and repopulate from scratch? -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkpLyboACgkQswz/d6k9dsWFawCeJzEYf30250z9UBbU82I3vza+ M6QAn0ZjPEMykAY6+3egyFO/3bc+KgEV =erk+ -END PGP SIGNATURE- -- ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team
Re: [Musicpd-dev-team] Database backend
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Max once thought about using SQLite as backend. He even was clear that, if done right, there shouldnt be ANY regression in speed. Not sure what the plans are for now. I guess there are more important things :) Regards, Rasi Jonny wrote: I know that people have asked why MPD doesn't use an SQL database in the past, and I know also that the reason has been performance. I'm not disputing this, and I don't want to start any kind of flame war, but aside from performance the use of SQL does allow for more features (or for making such features much easier to implement): 1 - Smart / dynamic playlists via complex SQL queries 2 - Easy maintenance of the database by external utilities using standard SQL 3 - Housing database on different machine from MPD (likely to cause a performance hit, I know) 4 - MPD clustering (an unlikely use case for most of MPDs users, I admit) For me personally, the second option is actually a biggie - currently one cannot manage the MPD database at any kind of granular level. Asking mpd to do an 'update' on its music directory has never completed for me - on an NFS-mounted 100GB+ music library. When I add things to the database I add things by name (e.g. mpc update Artist), but if I move files around or want to remove music from MPD's library, there doesn't seem to be any command to do that. I can only do 'update' on parts of the library and hope that MPD picks up that files are deleted / changed. I notice that MPD will also only rescan files if their modification date has changed; there seems to be no way to force MPD to re-scan a file/directory regardless of its modification time (which is useful in some situations). If these commands/features were added, I'd probably have less of a need for SQL personally, although the first idea above would be nice. I should also emphasise that in my view this doesn't have to be an all-or-nothing decision. Why not use a database plugin, and default to the internal MPD database, but allow the option of using SQLite/MySQL/etc? Wouldn't this satisfy both those who don't care/need the performance and those who want the flexibility of SQL? Kind regards, Jonny -- ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkpIl9wACgkQswz/d6k9dsXBoACfbMpRUsSkiHQPfjpTS4i4uck+ mGAAnis2Vlx/QMUs9qTgYXPT3j3T0HIX =FNlU -END PGP SIGNATURE- -- ___ Musicpd-dev-team mailing list Musicpd-dev-team@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/musicpd-dev-team
