Ah. OK, I am fine with GCE indexed, MYSQL. Somehow I've discontinued my local Camlistore or switched to GCE before running into bulk up slurping performance issues. Is there recommended doc on how to set up MySQL index for local Camlistore server, perhaps how to setup up Camlistore for performance?
On Wed, May 3, 2017 at 11:04 AM, Mathieu Lonjaret < [email protected]> wrote: > The launcher deploys with a MySQL-based indexer for Camlistore. > > Are you asking how to use another one? If yes, it might be a little > tricky, but I could try to figure it out. > > > On 3 May 2017 at 20:00, clive boulton <[email protected]> wrote: > > Using the GCE launcher version of Camlistore. Are there any steps > required > > to use MySQL or another indexer? > > > > On Apr 30, 2017 7:52 AM, "Mathieu Lonjaret" <[email protected]> > > wrote: > >> > >> On 30 April 2017 at 02:28, Rhythmic Fistman <[email protected]> wrote: > >> > I finally tried out camlistore by starting a local instance of > >> > camlistored > >> > with blob & index store on an external drive and running camput on > 200GB > >> > of > >> > photos, of which are probably 50% duplicates: > >> > > >> > > >> > camput file --permanode --title "some photos" --tag=photos > >> > ~/backups/photos/ > >> > > >> > > >> > That was four days ago. > >> > > >> > > >> > I forgot to add any progress indicator to camput, so I've had to > >> > estimate > >> > progress. Judging from blobs written to disk (40GB) and bytes read by > >> > camput > >> > (50GB) it looks like it's a quarter of the way through. > >> > > >> > > >> > I now need to interrupt the operation - will the successfully stored > >> > blobs > >> > still be there? Can I re-run camput & and start from day 4? > >> > >> Yes, you can interrupt without any problem. > >> When you restart the camput command, it will restart from the > >> beginning. But there's a layer of caching on the client-side, and even > >> if there weren't camput tries to stat the blob on the server before > >> trying to upload it, so in effect camput should "skip" all the blobs > >> that it already uploaded. > >> > >> > Is there something I can do to speed my configuration up? cpu usage of > >> > both > >> > client and server is minimal as is disk IO and (local network). > >> > >> as usual, SSD vs regular HDD has an impact. > >> Also, some indexer implementations are better than others. MySQL is > >> most likely the most reliable, and I'd guess among the fastest ones. > >> > >> > Where's the time being spent? > >> > > >> > > >> > Thanks, > >> > > >> > RF > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "Camlistore" group. > >> > To unsubscribe from this group and stop receiving emails from it, send > >> > an > >> > email to [email protected]. > >> > For more options, visit https://groups.google.com/d/optout. > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Camlistore" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to [email protected]. > >> For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Camlistore" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Camlistore" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Camlistore" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
