On 5/6/15 2:34 AM, Bala wrote:
I am new to this list so please excuse me if I posted it in wrong list.
We have a tape library which we would like to integrate with OpenStack
Swift & Swift3 object storage service to provide S3 interface.
The current file system we have for the library has been integrated with
Swift storage service and manages changer robot & tapes.
This works well for writing.
However for reading, loading a tape takes longer when GET requests are
received, in some cases over 5 minutes and this causes timeout error.
Most of the data stored in these tapes are archival data. This get
worsen when multiple GET requests received (muti-user) for objects which
are stored in different tapes.
Due to the longer read times, we are looking to provide Amazon S3
Glacier like interface through Swift & Swift3 so that clients can issue
a POST OBJECT RESTORE request and wait for the data to be moved to
temporary store/cache.
I have come across a similar request
http://openstack-dev.openstack.narkive.com/kI72vk9l/ltfs-integration-with-openstack-swift-for-scenario-like-data-archival-as-a-service
and understand the suggestions.
We would like to provide S3 Glacier like interface than Swift Storage
policies if we can.
I would be great full if you could kindly advise
1. How hard is to change Swift & Swift3 code base to provide S3 Glacier
like interface
It's not easy, that's for sure. Swift's API is all synchronous: issue a
GET, receive the object; issue a PUT, create an object; et cetera.
Glacier-style asynchronous retrieval is something completely new and
different.
Some food for thought: where will you store pending retrieval requests?
How will you ensure that retrieval requests survive disk and machine
failures the way everything else in Swift does?
I'm not asking you to answer here (though you can if you want to, of
course); I'm just trying to nudge your thoughts in the right direction.
2. Can this be done through Swift storage policies alone.
No. A storage policy determines where and how its objects are stored,
which affects things like access times and storage cost. The API for
accessing those objects does not change based on the storage policy.
3. Do we have to modify Swift Auditor service to do a tape based
checking rather than object based.
You mean audit in order? Probably a good idea, otherwise your tapes will
spend all day seeking.
4. Would Swift replication service cause frequent Tape change request.
I'd guess that it would, but nobody knows for sure. As far as I know,
nobody has jammed tape-library support into Swift before. You're the
first. Report back and let everyone know how it goes. :)
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : [email protected]
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
