Thanks Ed. Yes you're right, we've been looking at many possibilities for our app. Unfortunately I think S3QL won't scale with us but I wanted to check as it had a large number of the features we were looking for ie caching, dedup, ecryption and compression.
The decision we're taking now is that rather than use someone else's solution S3QL or other. We want to develop our own to ensure that 'no magic' is involved in the process, and we can support what we do. Thanks again to Nikolaus for what seems to be an awesome product. Giles On Friday, 17 October 2014 15:18:04 UTC+1, [email protected] wrote: > > Hi Everyone. > > We're looking at this solution as part of a web app we're building, I > wanted to ask the community what their experience is with benchmarks, how > many reads/writes can this system manage at what point will it need to be > scaled out. Although this seems to have solved all the issues we're going > to come up against, I'm worried that outgrowing it will be a difficult > process if we're moving terabytes of data in a few years time because we > need to increase the speed... > > This brings me to my next question, I know Nikolaus has mentioned on a few > occasions that this doesn't scale above one mount.s3ql for writes, although > you can add a few extra read mounts through sshfs, as mentioned here, would > this improve read performance or would all reads go still go through the > one mount (ie creating a bottleneck and not improving performance...) > > Thinking about clustering and performance it seems as though the system is > built off the back of a sqlite db, would this be possible to separate from > the system thereby making it possible to have multiple upload/download > points all write to same db?? > > I'm just thinking aloud on this but I'd appreciate hearing anyone else's > feedback on using this system before we commit to it. > > Kind regards > > Giles > > > -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
