Re: [openstack-dev] [swift] auth migration and user data migration

2015-03-12 Thread Weidong Shao
Thanks for the info.

On Wed, Mar 11, 2015 at 10:50 AM, Clay Gerrard clay.gerr...@gmail.com
wrote:



 On Mon, Mar 9, 2015 at 12:27 PM, Weidong Shao weidongs...@gmail.com
 wrote:


 I noticed swauth project is not actively maintained. In my local testing,
 swauth did not work after I upgraded swift to latest.


 Hrm... I think gholt would be open to patches/support, I know of a number
 of deployers of Swauth - so maybe if there's issues we should try to
 enumerate them.


​With this, I think I will try to stick with swauth. I will do some further
testing and let you know.
​


 I want to migrate off swauth. What are the auth alternative beside
 tempauth?



 Keystone.  The only other systems I know about are proprietary - what are
 your needs?


 On account-to-account server-side copy, is there an operation that is
 similar to mv? i.e., I want the data associated with an account to assume
 ownership of  a new account, but I do not want to copy the actual data on
 the disks.



 The account url is encoded in the object hash - the only realistic way to
 change the location (account/container/object) of an entity to swift is to
 read from it's current location and write it to the new location the delete
 the old object.


​the url is encoded in the object hash​! This somehow entangles the data
storage/validity with its account and makes it difficult to migrate the
data. I guess it is too late to debate on the design of this. Do you know
the technical reasons for doing this?



 -Clay

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [swift] auth migration and user data migration

2015-03-11 Thread Clay Gerrard
On Wed, Mar 11, 2015 at 1:16 PM, Weidong Shao weidongs...@gmail.com wrote:


 ​the url is encoded in the object hash​! This somehow entangles the data
 storage/validity with its account and makes it difficult to migrate the
 data. I guess it is too late to debate on the design of this. Do you know
 the technical reasons for doing this?




Well, yeah - can't see much good coming of trying to debate the design :)

The history may well be an aside from the issue at hand, but...

Not having a lookup/indirection layer was a design principle for achieving
the desired scaling properties of Swift.  Before Swift some of the
developers that worked on it had built another system that had a lookup
layer and it was a huge pain in the ass after a half billion objects or so
- but as with anything it's not the only way to do it, just trying
something and it seemed to work out.

I'd guess at least some of the justification came from: uri's don't change
- people change them [1].

Without a lookup layer that you can update (i.e. name = resource =
new_name = resource) - you can either create a new resources that happens
to have the same content of the other and delete the old OR add some custom
namespace redirection to make the resource accessible from another name (a
vanity url middleware comes up from time to time - reseller prefix rewrite
may be as good a use-case as any).

I made sure I was watching the swauth repo [2] - if you open any issues
there I'll try to keep an eye on them.  Thanks!

-Clay

1. http://www.w3.org/Provider/Style/URI.html
2. https://github.com/gholt/swauth/issues
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [swift] auth migration and user data migration

2015-03-11 Thread Clay Gerrard
On Mon, Mar 9, 2015 at 12:27 PM, Weidong Shao weidongs...@gmail.com wrote:


 I noticed swauth project is not actively maintained. In my local testing,
 swauth did not work after I upgraded swift to latest.


Hrm... I think gholt would be open to patches/support, I know of a number
of deployers of Swauth - so maybe if there's issues we should try to
enumerate them.


 I want to migrate off swauth. What are the auth alternative beside
 tempauth?



Keystone.  The only other systems I know about are proprietary - what are
your needs?


 On account-to-account server-side copy, is there an operation that is
 similar to mv? i.e., I want the data associated with an account to assume
 ownership of  a new account, but I do not want to copy the actual data on
 the disks.



The account url is encoded in the object hash - the only realistic way to
change the location (account/container/object) of an entity to swift is to
read from it's current location and write it to the new location the delete
the old object.

-Clay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [swift] auth migration and user data migration

2015-03-09 Thread John Dickinson

 On Mar 9, 2015, at 9:46 AM, Weidong Shao weidongs...@gmail.com wrote:
 
 hi,
 
 I have a standalone swift cluster with swauth as the auth module. By 
 standalone, I mean the cluster is not in the context of OpenStack, or 
 keystone server.

That's completely fine (and not uncommon at all).

 
 Now I have moved ACL logic to application level and decided to have all data 
 in swift under one user account. I have a few questions on this change:
 
 1) is it possible to migrate swauth to the tempAuth? (assuming tempauth will 
 be supported in newer swift versions).

Why?

Yes, tempauth is still in swift. It's mostly there for testing. I wouldn't 
recommend using it in production.


 
 2) Is there a way to migrate data associated with one user account to another 
 user?

user account Do you mean the identity or the account part of the Swift URL? 
If the former, then changing the reference in the auth system should probably 
work. If the latter, then you'll need to copy from one account to the other 
(Swift supports account-to-account server-side copy).


 
 Thanks,
 Weidong
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [swift] auth migration and user data migration

2015-03-09 Thread Weidong Shao
John,

thanks for the reply. See questions inline.

thanks,
Weidong

On Mon, Mar 9, 2015 at 8:23 AM John Dickinson m...@not.mn wrote:


  On Mar 9, 2015, at 9:46 AM, Weidong Shao weidongs...@gmail.com wrote:
 
  hi,
 
  I have a standalone swift cluster with swauth as the auth module. By
 standalone, I mean the cluster is not in the context of OpenStack, or
 keystone server.

 That's completely fine (and not uncommon at all).

 
  Now I have moved ACL logic to application level and decided to have all
 data in swift under one user account. I have a few questions on this change:
 
  1) is it possible to migrate swauth to the tempAuth? (assuming tempauth
 will be supported in newer swift versions).

 Why?

 Yes, tempauth is still in swift. It's mostly there for testing. I wouldn't
 recommend using it in production.


I noticed swauth project is not actively maintained. In my local testing,
swauth did not work after I upgraded swift to latest.

I want to migrate off swauth. What are the auth alternative beside tempauth?



 
  2) Is there a way to migrate data associated with one user account to
 another user?

 user account Do you mean the identity or the account part of the Swift
 URL? If the former, then changing the reference in the auth system should
 probably work. If the latter, then you'll need to copy from one account to
 the other (Swift supports account-to-account server-side copy).



I think it is for both.

The former applies because I plan to change auth, I hope that all the data
access in intact with a auth url change, so that I do not need to migrate
the data after the auth change.

On account-to-account server-side copy, is there an operation that is
similar to mv? i.e., I want the data associated with an account to assume
ownership of  a new account, but I do not want to copy the actual data on
the disks.


 
  Thanks,
  Weidong
  
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
 unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev