Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-28 Thread Sergey Lukjanov
Matt, thanks for catching this.

BTW That's an interesting idea of supporting different tenants.


On Fri, Jan 24, 2014 at 11:04 PM, Matthew Farrellee m...@redhat.com wrote:

 thanks for all the feedback folks.. i've registered a bp for this...

 https://blueprints.launchpad.net/savanna/+spec/swift-url-proto-cleanup


 On 01/24/2014 11:30 AM, Sergey Lukjanov wrote:

 Looks like we need to review prefixes and cleanup them. After the first
 look I'd like the idea of using common prefix for swift data.


 On Fri, Jan 24, 2014 at 7:05 PM, Trevor McKay tmc...@redhat.com
 mailto:tmc...@redhat.com wrote:

 Matt et al,

Yes, swift-internal was meant as a marker to distinguish it from
 swift-external someday. I agree, this could be indicated by setting
 other fields.

 Little bit of implementation detail for scope:

In the current EDP implementation, SWIFT_INTERNAL_PREFIX shows up
 in
 essentially two places.  One is validation (pretty easy to change).

The other is in Savanna's binary_retrievers module where, as others
 suggested, the auth url (proto, host, port, api) and admin tenant from
 the savanna configuration are used with the user/passw to make a
 connection through the swift client.

Handling of different types of job binaries is done in
 binary_retrievers/dispatch.py, where the URL determines the treatment.
 This could easily be extended to look at other indicators.

 Best,

 Trev

 On Fri, 2014-01-24 at 07:50 -0500, Matthew Farrellee wrote:
   andrew,
  
   what about having swift:// which defaults to the configured
 tenant and
   auth url for what we now call swift-internal, and we allow for user
   input to change tenant and auth url for what would be
 swift-external?
  
   in fact, we may need to add the tenant selection in icehouse. it's
 a
   pretty big limitation to only allow a single tenant.
  
   best,
  
  
   matt
  
   On 01/23/2014 11:15 PM, Andrew Lazarev wrote:
Matt,
   
For swift-internal we are using the same keystone (and identity
 protocol
version) as for savanna. Also savanna admin tenant is used.
   
Thanks,
Andrew.
   
   
On Thu, Jan 23, 2014 at 6:17 PM, Matthew Farrellee
 m...@redhat.com mailto:m...@redhat.com
mailto:m...@redhat.com mailto:m...@redhat.com wrote:
   
what makes it internal vs external?
   
swift-internal needs user  pass
   
swift-external needs user  pass  ?auth url?
   
best,
   
   
matt
   
On 01/23/2014 08:43 PM, Andrew Lazarev wrote:
   
Matt,
   
I can easily imagine situation when job binaries are
 stored in
external
HDFS or external SWIFT (like data sources). Internal and
external swifts
are different since we need additional credentials.
   
Thanks,
Andrew.
   
   
On Thu, Jan 23, 2014 at 5:30 PM, Matthew Farrellee
m...@redhat.com mailto:m...@redhat.com
 mailto:m...@redhat.com mailto:m...@redhat.com
mailto:m...@redhat.com mailto:m...@redhat.com
 mailto:m...@redhat.com mailto:m...@redhat.com wrote:
   
 trevor,
   
 job binaries are stored in swift or an internal
 savanna db,
 represented by swift-internal:// and savanna-db://
respectively.
   
 why swift-internal:// and not just swift://?
   
 fyi, i see mention of a potential future version
 of savanna w/
 swift-external://
   
 best,
   
   
 matt
   
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 mailto:OpenStack-dev@lists.
 mailto:OpenStack-dev@lists.__openstack.org http://openstack.org
mailto:OpenStack-dev@lists.openstack.org

 mailto:OpenStack-dev@lists.openstack.org
   
 http://lists.openstack.org/cgi-bin/mailman/listinfo/
 openstack-dev
   
 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__
 openstack-dev
   
 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__
 openstack-dev
   
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
   
   
   
   
_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org
mailto:OpenStack-dev@lists.openstack.org
 

Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-24 Thread Matthew Farrellee

andrew,

what about having swift:// which defaults to the configured tenant and 
auth url for what we now call swift-internal, and we allow for user 
input to change tenant and auth url for what would be swift-external?


in fact, we may need to add the tenant selection in icehouse. it's a 
pretty big limitation to only allow a single tenant.


best,


matt

On 01/23/2014 11:15 PM, Andrew Lazarev wrote:

Matt,

For swift-internal we are using the same keystone (and identity protocol
version) as for savanna. Also savanna admin tenant is used.

Thanks,
Andrew.


On Thu, Jan 23, 2014 at 6:17 PM, Matthew Farrellee m...@redhat.com
mailto:m...@redhat.com wrote:

what makes it internal vs external?

swift-internal needs user  pass

swift-external needs user  pass  ?auth url?

best,


matt

On 01/23/2014 08:43 PM, Andrew Lazarev wrote:

Matt,

I can easily imagine situation when job binaries are stored in
external
HDFS or external SWIFT (like data sources). Internal and
external swifts
are different since we need additional credentials.

Thanks,
Andrew.


On Thu, Jan 23, 2014 at 5:30 PM, Matthew Farrellee
m...@redhat.com mailto:m...@redhat.com
mailto:m...@redhat.com mailto:m...@redhat.com wrote:

 trevor,

 job binaries are stored in swift or an internal savanna db,
 represented by swift-internal:// and savanna-db://
respectively.

 why swift-internal:// and not just swift://?

 fyi, i see mention of a potential future version of savanna w/
 swift-external://

 best,


 matt

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 mailto:OpenStack-dev@lists.__openstack.org
mailto:OpenStack-dev@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-24 Thread Erik Bergenholtz

On Jan 24, 2014, at 7:50 AM, Matthew Farrellee m...@redhat.com wrote:

 andrew,
 
 what about having swift:// which defaults to the configured tenant and auth 
 url for what we now call swift-internal, and we allow for user input to 
 change tenant and auth url for what would be swift-external?
I like this idea, then swift-internal/swift-external becomes unnecessary. In 
general, doing anything outside of the existing tenant is frowned upon, at 
least by existing customers that we’re engaged with.

 
 in fact, we may need to add the tenant selection in icehouse. it's a pretty 
 big limitation to only allow a single tenant.
 
 best,
 
 
 matt
 
 On 01/23/2014 11:15 PM, Andrew Lazarev wrote:
 Matt,
 
 For swift-internal we are using the same keystone (and identity protocol
 version) as for savanna. Also savanna admin tenant is used.
 
 Thanks,
 Andrew.
 
 
 On Thu, Jan 23, 2014 at 6:17 PM, Matthew Farrellee m...@redhat.com
 mailto:m...@redhat.com wrote:
 
what makes it internal vs external?
 
swift-internal needs user  pass
 
swift-external needs user  pass  ?auth url?
 
best,
 
 
matt
 
On 01/23/2014 08:43 PM, Andrew Lazarev wrote:
 
Matt,
 
I can easily imagine situation when job binaries are stored in
external
HDFS or external SWIFT (like data sources). Internal and
external swifts
are different since we need additional credentials.
 
Thanks,
Andrew.
 
 
On Thu, Jan 23, 2014 at 5:30 PM, Matthew Farrellee
m...@redhat.com mailto:m...@redhat.com
mailto:m...@redhat.com mailto:m...@redhat.com wrote:
 
 trevor,
 
 job binaries are stored in swift or an internal savanna db,
 represented by swift-internal:// and savanna-db://
respectively.
 
 why swift-internal:// and not just swift://?
 
 fyi, i see mention of a potential future version of savanna w/
 swift-external://
 
 best,
 
 
 matt
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 mailto:OpenStack-dev@lists.__openstack.org
mailto:OpenStack-dev@lists.openstack.org

 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-24 Thread Andrew Lazarev
what about having swift:// which defaults to the configured tenant and
auth url for what we now call swift-internal, and we allow for user input
to change tenant and auth url for what would be swift-external?

I like the proposal.

Andrew.


On Fri, Jan 24, 2014 at 4:50 AM, Matthew Farrellee m...@redhat.com wrote:

 andrew,

 what about having swift:// which defaults to the configured tenant and
 auth url for what we now call swift-internal, and we allow for user input
 to change tenant and auth url for what would be swift-external?

 in fact, we may need to add the tenant selection in icehouse. it's a
 pretty big limitation to only allow a single tenant.

 best,


 matt

 On 01/23/2014 11:15 PM, Andrew Lazarev wrote:

 Matt,

 For swift-internal we are using the same keystone (and identity protocol
 version) as for savanna. Also savanna admin tenant is used.

 Thanks,
 Andrew.


 On Thu, Jan 23, 2014 at 6:17 PM, Matthew Farrellee m...@redhat.com
 mailto:m...@redhat.com wrote:

 what makes it internal vs external?

 swift-internal needs user  pass

 swift-external needs user  pass  ?auth url?

 best,


 matt

 On 01/23/2014 08:43 PM, Andrew Lazarev wrote:

 Matt,

 I can easily imagine situation when job binaries are stored in
 external
 HDFS or external SWIFT (like data sources). Internal and
 external swifts
 are different since we need additional credentials.

 Thanks,
 Andrew.


 On Thu, Jan 23, 2014 at 5:30 PM, Matthew Farrellee
 m...@redhat.com mailto:m...@redhat.com
 mailto:m...@redhat.com mailto:m...@redhat.com wrote:

  trevor,

  job binaries are stored in swift or an internal savanna db,
  represented by swift-internal:// and savanna-db://
 respectively.

  why swift-internal:// and not just swift://?

  fyi, i see mention of a potential future version of savanna
 w/
  swift-external://

  best,


  matt

  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  mailto:OpenStack-dev@lists.__openstack.org
 mailto:OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/
 openstack-dev
 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__
 openstack-dev
 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__
 openstack-dev
 http://lists.openstack.org/cgi-bin/mailman/listinfo/
 openstack-dev




 _
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.__org
 mailto:OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__
 openstack-dev
 http://lists.openstack.org/cgi-bin/mailman/listinfo/
 openstack-dev



 _
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.__org
 mailto:OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-24 Thread Trevor McKay
Matt et al,

  Yes, swift-internal was meant as a marker to distinguish it from
swift-external someday. I agree, this could be indicated by setting 
other fields.

Little bit of implementation detail for scope:

  In the current EDP implementation, SWIFT_INTERNAL_PREFIX shows up in
essentially two places.  One is validation (pretty easy to change).

  The other is in Savanna's binary_retrievers module where, as others
suggested, the auth url (proto, host, port, api) and admin tenant from
the savanna configuration are used with the user/passw to make a
connection through the swift client.

  Handling of different types of job binaries is done in
binary_retrievers/dispatch.py, where the URL determines the treatment.
This could easily be extended to look at other indicators.

Best,

Trev

On Fri, 2014-01-24 at 07:50 -0500, Matthew Farrellee wrote:
 andrew,
 
 what about having swift:// which defaults to the configured tenant and 
 auth url for what we now call swift-internal, and we allow for user 
 input to change tenant and auth url for what would be swift-external?
 
 in fact, we may need to add the tenant selection in icehouse. it's a 
 pretty big limitation to only allow a single tenant.
 
 best,
 
 
 matt
 
 On 01/23/2014 11:15 PM, Andrew Lazarev wrote:
  Matt,
 
  For swift-internal we are using the same keystone (and identity protocol
  version) as for savanna. Also savanna admin tenant is used.
 
  Thanks,
  Andrew.
 
 
  On Thu, Jan 23, 2014 at 6:17 PM, Matthew Farrellee m...@redhat.com
  mailto:m...@redhat.com wrote:
 
  what makes it internal vs external?
 
  swift-internal needs user  pass
 
  swift-external needs user  pass  ?auth url?
 
  best,
 
 
  matt
 
  On 01/23/2014 08:43 PM, Andrew Lazarev wrote:
 
  Matt,
 
  I can easily imagine situation when job binaries are stored in
  external
  HDFS or external SWIFT (like data sources). Internal and
  external swifts
  are different since we need additional credentials.
 
  Thanks,
  Andrew.
 
 
  On Thu, Jan 23, 2014 at 5:30 PM, Matthew Farrellee
  m...@redhat.com mailto:m...@redhat.com
  mailto:m...@redhat.com mailto:m...@redhat.com wrote:
 
   trevor,
 
   job binaries are stored in swift or an internal savanna db,
   represented by swift-internal:// and savanna-db://
  respectively.
 
   why swift-internal:// and not just swift://?
 
   fyi, i see mention of a potential future version of savanna w/
   swift-external://
 
   best,
 
 
   matt
 
   ___
   OpenStack-dev mailing list
   OpenStack-dev@lists.openstack.org
   mailto:OpenStack-dev@lists.__openstack.org
  mailto:OpenStack-dev@lists.openstack.org
  
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
  
  http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
  _
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.__org
  mailto:OpenStack-dev@lists.openstack.org
  
  http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
  _
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.__org
  mailto:OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-24 Thread Sergey Lukjanov
Looks like we need to review prefixes and cleanup them. After the first
look I'd like the idea of using common prefix for swift data.


On Fri, Jan 24, 2014 at 7:05 PM, Trevor McKay tmc...@redhat.com wrote:

 Matt et al,

   Yes, swift-internal was meant as a marker to distinguish it from
 swift-external someday. I agree, this could be indicated by setting
 other fields.

 Little bit of implementation detail for scope:

   In the current EDP implementation, SWIFT_INTERNAL_PREFIX shows up in
 essentially two places.  One is validation (pretty easy to change).

   The other is in Savanna's binary_retrievers module where, as others
 suggested, the auth url (proto, host, port, api) and admin tenant from
 the savanna configuration are used with the user/passw to make a
 connection through the swift client.

   Handling of different types of job binaries is done in
 binary_retrievers/dispatch.py, where the URL determines the treatment.
 This could easily be extended to look at other indicators.

 Best,

 Trev

 On Fri, 2014-01-24 at 07:50 -0500, Matthew Farrellee wrote:
  andrew,
 
  what about having swift:// which defaults to the configured tenant and
  auth url for what we now call swift-internal, and we allow for user
  input to change tenant and auth url for what would be swift-external?
 
  in fact, we may need to add the tenant selection in icehouse. it's a
  pretty big limitation to only allow a single tenant.
 
  best,
 
 
  matt
 
  On 01/23/2014 11:15 PM, Andrew Lazarev wrote:
   Matt,
  
   For swift-internal we are using the same keystone (and identity
 protocol
   version) as for savanna. Also savanna admin tenant is used.
  
   Thanks,
   Andrew.
  
  
   On Thu, Jan 23, 2014 at 6:17 PM, Matthew Farrellee m...@redhat.com
   mailto:m...@redhat.com wrote:
  
   what makes it internal vs external?
  
   swift-internal needs user  pass
  
   swift-external needs user  pass  ?auth url?
  
   best,
  
  
   matt
  
   On 01/23/2014 08:43 PM, Andrew Lazarev wrote:
  
   Matt,
  
   I can easily imagine situation when job binaries are stored in
   external
   HDFS or external SWIFT (like data sources). Internal and
   external swifts
   are different since we need additional credentials.
  
   Thanks,
   Andrew.
  
  
   On Thu, Jan 23, 2014 at 5:30 PM, Matthew Farrellee
   m...@redhat.com mailto:m...@redhat.com
   mailto:m...@redhat.com mailto:m...@redhat.com wrote:
  
trevor,
  
job binaries are stored in swift or an internal savanna
 db,
represented by swift-internal:// and savanna-db://
   respectively.
  
why swift-internal:// and not just swift://?
  
fyi, i see mention of a potential future version of
 savanna w/
swift-external://
  
best,
  
  
matt
  
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
mailto:OpenStack-dev@lists.__openstack.org
   mailto:OpenStack-dev@lists.openstack.org
  
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
   
 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
   
 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
   
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  
  
  
   _
   OpenStack-dev mailing list
   OpenStack-dev@lists.openstack.__org
   mailto:OpenStack-dev@lists.openstack.org
  
 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
   
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  
  
   _
   OpenStack-dev mailing list
   OpenStack-dev@lists.openstack.__org
   mailto:OpenStack-dev@lists.openstack.org
  
 http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  
  
  
   ___
   OpenStack-dev mailing list
   OpenStack-dev@lists.openstack.org
   http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours,
Sergey Lukjanov
Savanna Technical Lead
Mirantis Inc.
___
OpenStack-dev mailing list

Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-24 Thread Matthew Farrellee

thanks for all the feedback folks.. i've registered a bp for this...

https://blueprints.launchpad.net/savanna/+spec/swift-url-proto-cleanup

On 01/24/2014 11:30 AM, Sergey Lukjanov wrote:

Looks like we need to review prefixes and cleanup them. After the first
look I'd like the idea of using common prefix for swift data.


On Fri, Jan 24, 2014 at 7:05 PM, Trevor McKay tmc...@redhat.com
mailto:tmc...@redhat.com wrote:

Matt et al,

   Yes, swift-internal was meant as a marker to distinguish it from
swift-external someday. I agree, this could be indicated by setting
other fields.

Little bit of implementation detail for scope:

   In the current EDP implementation, SWIFT_INTERNAL_PREFIX shows up in
essentially two places.  One is validation (pretty easy to change).

   The other is in Savanna's binary_retrievers module where, as others
suggested, the auth url (proto, host, port, api) and admin tenant from
the savanna configuration are used with the user/passw to make a
connection through the swift client.

   Handling of different types of job binaries is done in
binary_retrievers/dispatch.py, where the URL determines the treatment.
This could easily be extended to look at other indicators.

Best,

Trev

On Fri, 2014-01-24 at 07:50 -0500, Matthew Farrellee wrote:
  andrew,
 
  what about having swift:// which defaults to the configured
tenant and
  auth url for what we now call swift-internal, and we allow for user
  input to change tenant and auth url for what would be swift-external?
 
  in fact, we may need to add the tenant selection in icehouse. it's a
  pretty big limitation to only allow a single tenant.
 
  best,
 
 
  matt
 
  On 01/23/2014 11:15 PM, Andrew Lazarev wrote:
   Matt,
  
   For swift-internal we are using the same keystone (and identity
protocol
   version) as for savanna. Also savanna admin tenant is used.
  
   Thanks,
   Andrew.
  
  
   On Thu, Jan 23, 2014 at 6:17 PM, Matthew Farrellee
m...@redhat.com mailto:m...@redhat.com
   mailto:m...@redhat.com mailto:m...@redhat.com wrote:
  
   what makes it internal vs external?
  
   swift-internal needs user  pass
  
   swift-external needs user  pass  ?auth url?
  
   best,
  
  
   matt
  
   On 01/23/2014 08:43 PM, Andrew Lazarev wrote:
  
   Matt,
  
   I can easily imagine situation when job binaries are
stored in
   external
   HDFS or external SWIFT (like data sources). Internal and
   external swifts
   are different since we need additional credentials.
  
   Thanks,
   Andrew.
  
  
   On Thu, Jan 23, 2014 at 5:30 PM, Matthew Farrellee
   m...@redhat.com mailto:m...@redhat.com
mailto:m...@redhat.com mailto:m...@redhat.com
   mailto:m...@redhat.com mailto:m...@redhat.com
mailto:m...@redhat.com mailto:m...@redhat.com wrote:
  
trevor,
  
job binaries are stored in swift or an internal
savanna db,
represented by swift-internal:// and savanna-db://
   respectively.
  
why swift-internal:// and not just swift://?
  
fyi, i see mention of a potential future version
of savanna w/
swift-external://
  
best,
  
  
matt
  
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
mailto:OpenStack-dev@lists.
mailto:OpenStack-dev@lists.__openstack.org http://openstack.org
   mailto:OpenStack-dev@lists.openstack.org
mailto:OpenStack-dev@lists.openstack.org
  
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
  
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
  
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  
  
  
   _
   OpenStack-dev mailing list
   OpenStack-dev@lists.openstack.__org
   mailto:OpenStack-dev@lists.openstack.org
mailto:OpenStack-dev@lists.openstack.org
  
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
  
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  
  
   _
   OpenStack-dev mailing list
   

Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-23 Thread Andrew Lazarev
Matt,

I can easily imagine situation when job binaries are stored in external
HDFS or external SWIFT (like data sources). Internal and external swifts
are different since we need additional credentials.

Thanks,
Andrew.


On Thu, Jan 23, 2014 at 5:30 PM, Matthew Farrellee m...@redhat.com wrote:

 trevor,

 job binaries are stored in swift or an internal savanna db, represented by
 swift-internal:// and savanna-db:// respectively.

 why swift-internal:// and not just swift://?

 fyi, i see mention of a potential future version of savanna w/
 swift-external://

 best,


 matt

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-23 Thread Matthew Farrellee

what makes it internal vs external?

swift-internal needs user  pass

swift-external needs user  pass  ?auth url?

best,


matt

On 01/23/2014 08:43 PM, Andrew Lazarev wrote:

Matt,

I can easily imagine situation when job binaries are stored in external
HDFS or external SWIFT (like data sources). Internal and external swifts
are different since we need additional credentials.

Thanks,
Andrew.


On Thu, Jan 23, 2014 at 5:30 PM, Matthew Farrellee m...@redhat.com
mailto:m...@redhat.com wrote:

trevor,

job binaries are stored in swift or an internal savanna db,
represented by swift-internal:// and savanna-db:// respectively.

why swift-internal:// and not just swift://?

fyi, i see mention of a potential future version of savanna w/
swift-external://

best,


matt

_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev