Thanks.
Just getting this data online is a big deal, so waiting a further 1-2 weeks
won't be our issue, compared to the other factors. I'm not yet sure if we'll
still chunk the data. S3 file size limits are significantly large, but I
don't know what current EBS object size limits are, and I'm not sure we want
any operation that requires the server to manipulate a 10-20GB file,
regardless. I can see lots of ways that will go awry for everyone trying to
use the data.
ari
On Wed, Jun 15, 2011 at 5:06 PM, Peter Pinch <peter_pi...@wgbh.org> wrote:
> Being able to connect directly to the RAID should help. My other advice:
>
>
> - if you have enough data that you need to do multiple deliveries,
> invest in multiple drives so you can send one out and work on preparing the
> 2nd at the same time.
> - you may also need some available CPU near the drives, if you need to
> do bit checking. We also needed to chunk the files, but Amazon’s increased
> their file size limits (although your disks or host OS may have limits).
> - Keep a list of what you sent. =)
> - I think it generally took about 1-2 weeks to get data ingested
>
>
>
> --
> Peter Pinch
> Director of Technology
> WGBH Interactive
> v: 617-300-3961
> e: peter_pi...@wgbh.org
>
>
>
>
> On 6/15/11 4:29 PM, "Ari Davidow" <aridavi...@gmail.com> wrote:
>
> Well, that's not what I wanted to hear, Peter, but you guys have very big
> pipes to the outside world. We're sharing a T1 data line with our phones,
> whereas AWS moves things back and forth at significantly higher speeds. It
> took several months to back up a few GB on our network the first time. For
> this project we can directly hook that outbound hard disk drive to send to
> AWS directly to our RAID server where the files currently reside without
> going through our internal network, so that part of the exercise may be
> saner.
>
>
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Fedora-commons-users mailing list
> Fedora-commons-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
>
>
------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
