My own random thoughts about the storage pod and random ideas.
(1) The backblaze pod we built (Rayson has my bioteam.net URL in his
post below) cost roughly $12K for 100 terabytes of usable storage. Even
with all the downsides and negatives to this particular hardware rig the
"$12,000 for 100 usable terabytes" is still a disruptive price point and
there is all sorts of room for innovation in what you'd might want to do
with them
(2) Caveat: This is not the only game in town. If you are willing/able
to spend a bit more money for something more 'safe' in operational terms
there are tons of chassis on the market now that would give you a large
number of hot-swap drive bays. Go to siliconmechanics.com etc. and see
their 'storage server' section for an idea of what you can get when you
look at hardware that is a few levels above the super cheap pod concept
(3) The protocase people who did the hardware for this particular pod
are working on a version 2 that addresses many of the hardware
resiliency items I had blogged about. Their new chassis design will have
redundant power and mirrored boot drives. So anyone looking at this type
of unit should check with protocase to see what their plans are
Now to answer Rayson's question
-- I personally would like to combine 3 of these pods using object
storage software like the stuff over at www.swiftstack.com. An
individual pod is a dangerous and risky thing. Three or more pods with
automatic replication and geographic separation is far more interesting.
Hopefully I'll find someone to fund some of this work early in 2013 so I
can kick the tires and write/talk about it.
So perhaps for the future of grid engine I'd like to see some sort of
"understanding of object storage" or other storage systems that use
RESTful HTTP for data operations. Maybe expand the file staging and
"store my results in location X" to handle putting job output into an
object. Heck the next extension after that would be support for the AWS
S3 API so that we've got the IaaS cloud storage thing handled.
-Chris
Rayson Ho wrote:
Somewhat related to the email message below: I am going to play with a
StoragePod2.0 for the next project in 2013... The machine uses
off-the-shelf components like motherboard, CPU, 45 disks, memory,
power supply... and the real intellectual property that was released
by Backblaze is the 4U custom case (but it is still cool!).
http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/
When I first saw the machine, I thought that it was the Sun Fire x4500
painted in red!! To me, it is a whitebox, DIY version of the x4500,
but the StoragePod is much more affordable at less than US $10,000!
So my question is, besides the obvious functionalities like input file
staging& transfer, what are the useful features that can (and should)
be implemented in Grid Engine for data management? (Feel free to
contact me offline if your site has special requirements.)
BTW, while googling, found that Chris also has a StoragePod:
http://bioteam.net/2011/08/backblaze-storage-pod/
Rayson
_______________________________________________
users mailing list
users@gridengine.org
https://gridengine.org/mailman/listinfo/users
