Jumping back as it's currently 4:00 am in NZ and I'm sure Glen is sleeping but I want to keep this conversation going...
We have actually just moved away from IIPImage, for some of the reasons you hint at. We ended up pre-processing tiles and putting them out on S3 and serving to the end user via Cloudfront CDN. This offered us a few advantages: custom image compression, color profiles applied correctly (I think IIP was losing them!), and actually leveraging a CDN for parallel downloads. It's (a lot) more work up front to cut the tiles rather than doing it dynamically, but it's automated and working well. IMO the only way to run IIPImage in the cloud is to use EC2 as an IIP server (we actually had this set up for a while) so you don't get hit with all the bandwidth of pulling the ptiff each time. It does work, but for us the dealbreakers were the lack of control over the final jpegs and the inability to spread the requests across multiple CDN domains. (although that's solvable). I don't think you could possibly have the full ptiffs on the cloud and serve the tiles locally, the overhead and bandwidth costs would kill you. Nate On Fri, Apr 19, 2013 at 10:14 AM, Jeremy Ottevanger <JOttevanger at iwm.org.uk>wrote: > Absolutely, thank you Glen. I'm wondering at the moment about high-res > images intended for delivery through an IIPImage server. If we store large > volumes of these we'll definitely want plenty of space at low cost, but > they need to be in, effectively, local storage, so that the server can > access them as part of the file system and then stream them out as tiles to > users in real time. I guess for this scenario block storage is actually > what we need? > > All the best, > > Jeremy Ottevanger > Technical Web Manager > Imperial War Museum > Lambeth Road > London SE1 6HZ > > > -----Original Message----- > From: mcn-l-bounces at mcn.edu [mailto:mcn-l-bounces at mcn.edu] On Behalf Of > Nate Solas > Sent: 19 April 2013 15:47 > To: Museum Computer Network Listserv > Subject: Re: [MCN-L] Advice About Cloud Storage > > Great summary of the difference between block & object storage, thanks > Glen! > Nate > > > > On Thu, Apr 18, 2013 at 4:24 PM, My Tours <glen at mytoursapp.com> wrote: > > > >> > > >> We are planning to use a cloud service to store large numbers of > > >> hi-res > > digital images for an online education platform. If any of you already > > use cloud services I'd be very grateful for recommendations or advice > > about what to look out for. > > >> > > >> Thank you, as always. > > > > snip, snip, snip... > > > > Background: Along with running My Tours where we store images and > > audio clips on S3 I also work for Realestate.co.nz where we store > > ~1.4TB of images and serve around 80GB of images a day to consumers. > > This is currently stored on block storage at a local 'cloudy' > > provider. We are currently in the process of selecting a vendor for > > cloud file storage and at the moment Amazon s3 with a Fastly CDN is our > preferred option. > > > > On 19/04/2013, at 12:00 AM, mcn-l-request at mcn.edu wrote: > > > > > > We use Rackspace. They have a new part to their cloud offer, which > > > I've > > not yet tried but which sounds helpful if you have large storage needs > > because it lets you buy what you need without scaling up the server as > > a whole. Cloud Blocks, I think it's called. Ah yes, here it is: > > > > > > > > http://www.rackspace.com/knowledge_center/article/cloud-block-storage- > > overview > > > > You have 2 basic types of cloud storage, object storage and block > storage. > > Block storage is similar to your normal hard disks you would find in > > your computer or server. They 'look' just like a regular disk and > > perform the same. They tend to be faster than object storage and fit > > into traditional apps just like a normal disk would. > > > > Object storage is aimed at storing a large number of objects (images, > > audio, backups, and other assets). It is not a traditional file system > > in that you don't just copy files to a folder but use an API to put, > > requests and delete files. > > > > Both types of storage can scale but object based storage (Cloud Files > > from Rackspace and s3 from Amazon) is the right approach to storing > > large numbers of hi-res images. > > > > Another type of storage that may be also be of interest is Amazon's > > Glacier storage. For long term archival the price comes down to 1c/GB > > a month and is a great replacement for tape backups and long term > > storage that doesn't need to be accessed a lot. > > > > Amazon also hooks into a lot of other services such as the Amazon > > storage gateway - http://aws.amazon.com/storagegateway/). Worth > > looking at depending on your needs. > > > > And also don't forget a CDN - This helps deliver the assets out to the > > users as quickly as possible (s3 can be a little slow if you need fast > > access to the assets). Again Amazon makes it easy with CloudFront > > although in NZ we're looking at Fastly which has a local point of > > presence and has a really nice interface. > > > > > ----------------------------------------------------------------------------------------------------------------------------------------- > This email message has been delivered safely and archived online by > Mimecast. > For more information please visit http://www.mimecast.com > > ----------------------------------------------------------------------------------------------------------------------------------------- > > _______________________________________________ > You are currently subscribed to mcn-l, the listserv of the Museum Computer > Network (http://www.mcn.edu) > > To post to this list, send messages to: mcn-l at mcn.edu > > To unsubscribe or change mcn-l delivery options visit: > http://mcn.edu/mailman/listinfo/mcn-l > > The MCN-L archives can be found at: > http://toronto.mediatrope.com/pipermail/mcn-l/ > -- Nate Solas Sr. New Media Developer and Head Technologist Walker Art Center 1750 Hennepin Ave MInneapolis, MN 55407 http://www.walkerart.org/
