I'm in a bind with this - I ctrl-C camlistored and the kvIndexFile became corrupted (how do I cleanly stop it?) & re-indexing is taking days. This is 40GB of blobs on an external non-ssd drive. A previous experiment on my internal SSD with a store of 60gb takes less than a minute.
I'd like to start over on SSD, but this is going to be a large store. Losing the index isn't supposed to be a serious problem, but if I can't recalculate it in a reasonable amount of time then it is a problem. Maybe a mysql index would be safer, but I still have the problem of the slow initial import. Il giorno giovedì 4 maggio 2017 04:00:25 UTC+10, clive boulton ha scritto: > > Using the GCE launcher version of Camlistore. Are there any steps required > to use MySQL or another indexer? > > On Apr 30, 2017 7:52 AM, "Mathieu Lonjaret" <[email protected] > <javascript:>> wrote: > >> On 30 April 2017 at 02:28, Rhythmic Fistman <[email protected] >> <javascript:>> wrote: >> > I finally tried out camlistore by starting a local instance of >> camlistored >> > with blob & index store on an external drive and running camput on >> 200GB of >> > photos, of which are probably 50% duplicates: >> > >> > >> > camput file --permanode --title "some photos" --tag=photos >> ~/backups/photos/ >> > >> > >> > That was four days ago. >> > >> > >> > I forgot to add any progress indicator to camput, so I've had to >> estimate >> > progress. Judging from blobs written to disk (40GB) and bytes read by >> camput >> > (50GB) it looks like it's a quarter of the way through. >> > >> > >> > I now need to interrupt the operation - will the successfully stored >> blobs >> > still be there? Can I re-run camput & and start from day 4? >> >> Yes, you can interrupt without any problem. >> When you restart the camput command, it will restart from the >> beginning. But there's a layer of caching on the client-side, and even >> if there weren't camput tries to stat the blob on the server before >> trying to upload it, so in effect camput should "skip" all the blobs >> that it already uploaded. >> >> > Is there something I can do to speed my configuration up? cpu usage of >> both >> > client and server is minimal as is disk IO and (local network). >> >> as usual, SSD vs regular HDD has an impact. >> Also, some indexer implementations are better than others. MySQL is >> most likely the most reliable, and I'd guess among the fastest ones. >> >> > Where's the time being spent? >> > >> > >> > Thanks, >> > >> > RF >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "Camlistore" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to [email protected] <javascript:>. >> > For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Camlistore" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Camlistore" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
