Re: [Openstack] [Openstack-operators] Health & Monitoring Blueprints

[Angus Salkeld]

On 13/04/12 18:48 -0700, Stefano Maffulli wrote:

On 04/11/2012 02:52 PM, Stefano Maffulli wrote:

We're not yet sure to be able to offer this. Wait for an official
communication.


Here is a brief update regarding the live streaming situation.

Here is the good news:

Cisco kindly offered http://openstack.webex.com. If you get there you'll
see some meetings setup for the days of the Summit, one meeting per each
day, per room.

Zareason kindly offered 9 computers to be put in each room, connected to
the audio equipment available there. Each of Zareason's PC will act as a
'robot-presenter' for Webex.

The good news stop here.

There is bad news: at least 6 of the zareason PCs run 64bit systems, and
64bit Java on Linux is not compatible with Webex (audio issues). Two of
the remaining computers may be too slow to run Webex (I haven't tested
them).

Possible solutions:

- offer live streaming only from 3 rooms (at best, only one at worst)
(which ones?)
- spend a few hours on Sunday wiping out the 6 machines and put 32 bit
systems on
   - is there a simpler hack to run 32bit Java on 64bit Ubuntu 11.10
systems?
   - these hours will have to be on top of the hours necessary to setup
the rooms, make sure that audio capture really works with the hotel's
audio system etc
- give to session leaders the honor/burden to run Webex sessions from
their own laptops (given how Webex conferencing works, it may be nearly
impossible)

Other ideas? I'm inclined to go with the simplest option, #1. Unless
many (more than 10) people reply to this message asking me to provide
full coverage of the event, I'd go with it. For the other options, I'd
ask for somebody else to take the lead: that means, helping wipe and
install the machines, install java, test them and install them. I'll be
glad to provide all support needed.


Hi

I'de really like to watch these if possible. I am in Australia so
recordings would be great too (I have to sleep at some point:).

Any ways I really appreciate any work you do in this area.

-Angus



Let me know asap, so I can plan the weekend.
stef

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Openstack-operators] Health & Monitoring Blueprints

[Duncan McGreggor]

cc'ing Jorge Castro from Canonical (who will be at the conference...
and maybe the summit?).

Jorge, question below on how to deal with Webex on Ubuntu and limited
time/people resources.

On Fri, Apr 13, 2012 at 9:48 PM, Stefano Maffulli  wrote:
> On 04/11/2012 02:52 PM, Stefano Maffulli wrote:
>> We're not yet sure to be able to offer this. Wait for an official
>> communication.
>
> Here is a brief update regarding the live streaming situation.
>
> Here is the good news:
>
> Cisco kindly offered http://openstack.webex.com. If you get there you'll
> see some meetings setup for the days of the Summit, one meeting per each
> day, per room.
>
> Zareason kindly offered 9 computers to be put in each room, connected to
> the audio equipment available there. Each of Zareason's PC will act as a
> 'robot-presenter' for Webex.
>
> The good news stop here.
>
> There is bad news: at least 6 of the zareason PCs run 64bit systems, and
> 64bit Java on Linux is not compatible with Webex (audio issues). Two of
> the remaining computers may be too slow to run Webex (I haven't tested
> them).
>
> Possible solutions:
>
> - offer live streaming only from 3 rooms (at best, only one at worst)
> (which ones?)
> - spend a few hours on Sunday wiping out the 6 machines and put 32 bit
> systems on

What about running on a LiveCD? You should be able to install Webex,
as long as the file system is mounted.

Jorge, is that correct? It's been a while since I've used LiveCD.

d

>    - is there a simpler hack to run 32bit Java on 64bit Ubuntu 11.10
> systems?
>    - these hours will have to be on top of the hours necessary to setup
> the rooms, make sure that audio capture really works with the hotel's
> audio system etc
> - give to session leaders the honor/burden to run Webex sessions from
> their own laptops (given how Webex conferencing works, it may be nearly
> impossible)
>
> Other ideas? I'm inclined to go with the simplest option, #1. Unless
> many (more than 10) people reply to this message asking me to provide
> full coverage of the event, I'd go with it. For the other options, I'd
> ask for somebody else to take the lead: that means, helping wipe and
> install the machines, install java, test them and install them. I'll be
> glad to provide all support needed.
>
> Let me know asap, so I can plan the weekend.
> stef

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Just JSON, and extensibility

[Jorge Williams]

On Apr 13, 2012, at 3:20 PM, Justin Santa Barbara wrote:

My understanding is that the solution we have now is that any extension goes 
into its own namespace; we assign a prefix to the namespace and have a way to 
map that prefix to the full namespace.  (Similar to XML schemas).  Currently 
prefixes are hard-coded, but we may not be able to keep doing this forever (XML 
has pre-document prefixes to avoid the need for a central authority).

I see 3 questions:
1) Is my summary correct?

It's pretty close.  I did propose that we maintain a registry of prefixes, but 
that's never taken off.

I did a write up a while back on extensions, you can find it here:  
https://github.com/RackerWilliams/OpenStack-Extensions/blob/master/apix-intro.pdf

Take the document that's in GitHub with a grain of salt, it doesn't entirely 
reflect the reality of things as they stand now and I feel,  after working with 
extensions for a while, that we need to make some slight modifications.
  It wasn't clear in the beginning where extension would be most useful so we 
added extensibility to everything.   As Mark mentioned, it's clear now that we 
need to scale down the points of extensibility.  In some cases we may introduce 
barriers to devs if we make absolutely everything extensible. In other cases, 
defining our own extensibility doesn't make sense.  For example,  there's no 
need define a way of extending HTTP headers -- first because, no one is writing 
those kinds of extensions and also because there already exists a method of 
extending  headers in HTTP so there's no need us to reinvent the wheel.  Stuff 
like that.


2) Are there any problems with the solution?

Yes a couple.  Especially when you consider what happens when an extension gets 
promoted to a full feature in a new version of the API.  I'm now leaning 
towards keeping prefixes forever...and providing a way for folks to write 
extension without a prefix, provided that they register the extension with the 
PTL of the project -- lets face it this is the sort of stuff that's happening 
now anyway.   Mind you these are all just thoughts at the moment we should have 
a larger discussion.

3) Are there any other problems we're not addressing?

Probably  :-)

We're having a panel on extensions at the summit.  We should discuss in detail 
then.

As one of the two authors of the Java binding, I can tell you how I plan on 
dealing with extensions:


  *   Map the JSON/XML/HPSTR to a strongly-typed model (so the representation 
is irrelevant to the consumer of the Java library).
  *   Each (known) extension has its own strongly-typed model object.
  *   These are stored in a registry.
  *   When we come across an extension, we look it up in the registry and 
either bind or ignore it.
  *   Every model object has an "Extensions" collection, which can be queried 
by type, to see if that extension data was present.

(Note: this has mostly been tested with the XML)


When you say registry, do you mean maven?


The nice thing about this is that a consumer of the library can write a binding 
for an extension, and register it with the client library, and "it just works". 
 So, even if you have a private extension, which you tell nobody about and run 
only on your private cloud, you can use it with the stock Java library.

That sounds like a really great approach!  I'd love to check out the 
nitty-gritty details...Are you documenting this somewhere?


Now, how you would do something that awesome in Python, I don't know ;-)


I'm sure there are Pythonistas out there working to figure it out.

-jOrGe W.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Openstack-operators] Health & Monitoring Blueprints

[Stefano Maffulli]

On 04/11/2012 02:52 PM, Stefano Maffulli wrote:
> We're not yet sure to be able to offer this. Wait for an official
> communication.

Here is a brief update regarding the live streaming situation.

Here is the good news:

Cisco kindly offered http://openstack.webex.com. If you get there you'll
see some meetings setup for the days of the Summit, one meeting per each
day, per room.

Zareason kindly offered 9 computers to be put in each room, connected to
the audio equipment available there. Each of Zareason's PC will act as a
'robot-presenter' for Webex.

The good news stop here.

There is bad news: at least 6 of the zareason PCs run 64bit systems, and
64bit Java on Linux is not compatible with Webex (audio issues). Two of
the remaining computers may be too slow to run Webex (I haven't tested
them).

Possible solutions:

- offer live streaming only from 3 rooms (at best, only one at worst)
(which ones?)
- spend a few hours on Sunday wiping out the 6 machines and put 32 bit
systems on
- is there a simpler hack to run 32bit Java on 64bit Ubuntu 11.10
systems?
- these hours will have to be on top of the hours necessary to setup
the rooms, make sure that audio capture really works with the hotel's
audio system etc
- give to session leaders the honor/burden to run Webex sessions from
their own laptops (given how Webex conferencing works, it may be nearly
impossible)

Other ideas? I'm inclined to go with the simplest option, #1. Unless
many (more than 10) people reply to this message asking me to provide
full coverage of the event, I'd go with it. For the other options, I'd
ask for somebody else to take the lead: that means, helping wipe and
install the machines, install java, test them and install them. I'll be
glad to provide all support needed.

Let me know asap, so I can plan the weekend.
stef

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Recent changes to Ubuntu packaging wrt Glance, upgrading, database sync

[Joshua Harlow]

So a useful question,

What mistakes can we learn from this to make sure that this doesn't happen 
again and that if it does we catch it much earlier?

On 4/13/12 4:17 PM, "Adam Gandelman"  wrote:

Hi-

We've added a patch to the Glance package in Ubuntu to help improve the
experience for users who are counting on a smooth upgrade from Diablo.
Since the release of Essex, I've been mostly focused on upgrade
testing.  Things seem generally okay with the exception of Glance and
the way it manages its database and migrations.  To ensure users can
reliably upgrade from Diablo, we've made a couple of changes that people
should know about as the end result is currently a bit different than
what you'd expect comparing to devstack/vanilla git repos.

1. Disabled database auto-creation

Glance is the only project that I know of that currently allows
sqlalchemy to auto-create database schema to match its database
models.   This is great for the developer use-case, but really not ideal
for anyone who wishes to maintain their database for the long-term.  The
original issue is that, on startup, glance-registry creates a database
schema that reflects its most recent view of the database.  When it
does, the schema is created outside the view of migrate and the
migration repository cannot be used to incrementally upgrade/downgrade
the schema.

Recent changes [1] in Glance provides a work-around to this isssue,
where we rely on glance-registry to automatically put the database under
version control after auto-creation and set its version to the most
recent (currently 13).  This works fine for development but there are
important details in the migration code that don't actually end up in
the database schema when auto-created (FK constraints, for example)
because the migration scripts are not actually run.

To avoid this entirely, we've prevented Glance from auto-migrating its
database and now require users to migrate the database on new install
and upgrade (via db_sync) in the same way Nova does.

2. Sync'ing a new database.

The changes also now allow users to set a specific version when
initializing version_control on a database with glance-manage, eg:

glance-manage version_control 9
glance-manage db_sync

This would properly migrate an non-version controlled Diablo database to
Essex. This is great, but it is still possible for Diablo users to
mistakenly start glance or run db_sync with no version control, and end
up in a situation where glance + migrate think it is up to date. In
reality the database is missing 3 migrations (one of which, 012 IDs to
UUIDs, is very essential!)   To prevent this, the Ubuntu patch
additionally requires explicitly version_control'ing the database a
before db_sync on both new databases and during upgrades of databases
that are not already version_control'd.

For example, on a new database:

# Start from the beginning of the migration repositroy
glance-manage version_control 0
glance-manage db_sync

or to upgrade from Diablo:

# Was our database properly version controlled to begin with?
if ! glance-manage db_version ; then
   # Set its version if not
   glance-manage version_control 9
fi
# Run all migrations 10 to 13
glance-manage db_sync

I understand this may have come a bit late to any users of the Precise
packages and I apologize for the lack of a heads up.  I've filed a new
bug [2] about the split brain between glance-registry and and hope to
address this at the database management session [3] at ODS next week.
Anne Gentle has also proposed an update to the documentation with a note
about this.   Please understand that our aim was to keep our patch delta
as minimal as possible in Ubuntu, but with the yesterday's Final Freeze
for the cycle 12.04 we needed something in place that ensures a sane
user story for previous users of Diablo and future Essex users alike.

Cheers,
Adam

[1] https://review.openstack.org/#/c/6166/
https://review.openstack.org/#/c/5980/1
[2] https://bugs.launchpad.net/bugs/981081
[3] http://summit.openstack.org/sessions/view/36



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New Gerrit version (and server)

[James E. Blair]

Chris Behrens  writes:

> I'm noticing that core reviewers aren't tagged as a +2 anymore in the
> general review box.  Instead, it just shows a check.  (Except for my
> own reviews, which show as a +2).  I have to look at the review
> comments to see if others voted +2 now.

A check mark is equivalent to a +2 review.  I believe all +2 votes
should be displayed as check marks but it's not immediately clear to me
why only one of the +2 votes is currently displayed as a check mark.

Until we arrive at an explanation, or fix, just know that +2 == green
check.

-Jim

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New Gerrit version (and server)

[Chris Behrens]

I'm noticing that core reviewers aren't tagged as a +2 anymore in the general 
review box.  Instead, it just shows a check.  (Except for my own reviews, which 
show as a +2).   I have to look at the review comments to see if others voted 
+2 now.

- Chris

On Apr 12, 2012, at 4:23 PM, James E. Blair wrote:

> Hi,
> 
> We've just upgraded Gerrit to version 2.3.  There are a lot of changes
> behind the scenes that we've been looking forward to (like being able to
> store data in innodb rather than myisam tables for extra data
> longevity).  And there are a few visible changes that may be of interest
> to OpenStack developers.
> 
> One new addition in 2.3 is "draft changes".  The idea behind a draft
> change in Gerrit is that it is a change that is not ready for merging,
> or even general code review, but you would like to share it with some
> people to get early comments.  If you upload a change as a draft, by
> default, no one else can see it.  You must explicitly add each person
> you would like to share it with as a reviewer.  Reviewers you add can
> leave comments, but can not vote at this stage.  You can continue to
> upload new patchsets to the change as it evolves, and once it is ready
> for general review, you can click the "Publish" button.  It will then
> become a normal change in Gerrit that everyone can see, including the
> earlier reviews from the draft stage.  This is a one way transition;
> once a draft is published, it can't be made a draft again.
> 
> If you're using git-review from source or the latest version from PyPI
> (version 1.16, released today), you can easily upload a draft change by
> adding the "-D" option (eg, "git review -D").  Earlier versions of
> git-review also have the "-D" option, but the git ref that Gerrit uses
> to indicate a change should be a draft was changed between the 2.3
> release candidate and the final release; so if using "-D" results in an
> error, you may need to upgrade.
> 
> You may notice some changes to the diff view.  Notably, the header which
> contained all of the possible viewing options has been split up into
> several parts; you can switch between them by selecting options that
> show up under the menu at the top.  I recommend setting "Retain Header
> On File Switch" under the Preferences section, as it is a nicer
> experience when changing files.
> 
> Another notable new feature is the ability to add a group to the list of
> reviewers for a change.  Just type in the name of the group and click
> "Add Reviewer" and all of the individuals in the group will be added to
> the list of reviewers (and will see the change on their "review
> requests" list.
> 
> Finally, we've modified some of our local OpenStack style changes so
> that it is easier for us to track upstream changes in layout.  It should
> mean a little more consistency throughout the interface, though we
> weren't able to keep the alternating row colors on the main table
> without a disproportionate amount of effort.  Do note that you can click
> on a line in a table, and it will be highlighted to improve legibility.
> 
> We've tried to give this as much testing as possible before moving it
> into production.  If you encounter any issues, please let us know on IRC
> (mtaylor, jeblair, LinuxJedi), via email at
> , or you can file a bug at:
> 
>  https://bugs.launchpad.net/openstack-ci/
> 
> Thanks,
> 
> Jim
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Just JSON, and extensibility

[Justin Santa Barbara]

It's easy when each new version is defined by a central body.

The problem we face is that we want to allow HP, Rackspace, Nexenta etc to
define their own extensions, without serializing through a central body.
 Some extensions may only apply to private clouds and never be shared
publicly.

This is a bit like ZFS feature flags, to use an example that should be near
and dear to your heart!



On Fri, Apr 13, 2012 at 2:24 PM, Caitlin Bestler <
caitlin.best...@nexenta.com> wrote:

>  Exactly what do you see as the required “non-linear extensibility”?
>
> ** **
>
> These are ultimately requests to a server. Each new extension is coded in
> that server.
>
> There is no value in a client making up its own extensions that are not
> understood by the server.
>
> ** **
>
> What is relevant is a server continuing to support clients that have not
> yet been updated to
> understand a new format.
>
> ** **
>
> As I stated in my first post, that problem was solved in ANSI C.
> Python/JSON is trivial.
>
> ** **
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] Recent changes to Ubuntu packaging wrt Glance, upgrading, database sync

[Adam Gandelman]

Hi-

We've added a patch to the Glance package in Ubuntu to help improve the 
experience for users who are counting on a smooth upgrade from Diablo.  
Since the release of Essex, I've been mostly focused on upgrade 
testing.  Things seem generally okay with the exception of Glance and 
the way it manages its database and migrations.  To ensure users can 
reliably upgrade from Diablo, we've made a couple of changes that people 
should know about as the end result is currently a bit different than 
what you'd expect comparing to devstack/vanilla git repos.


1. Disabled database auto-creation

Glance is the only project that I know of that currently allows 
sqlalchemy to auto-create database schema to match its database 
models.   This is great for the developer use-case, but really not ideal 
for anyone who wishes to maintain their database for the long-term.  The 
original issue is that, on startup, glance-registry creates a database 
schema that reflects its most recent view of the database.  When it 
does, the schema is created outside the view of migrate and the 
migration repository cannot be used to incrementally upgrade/downgrade 
the schema.


Recent changes [1] in Glance provides a work-around to this isssue, 
where we rely on glance-registry to automatically put the database under 
version control after auto-creation and set its version to the most 
recent (currently 13).  This works fine for development but there are 
important details in the migration code that don't actually end up in 
the database schema when auto-created (FK constraints, for example) 
because the migration scripts are not actually run.


To avoid this entirely, we've prevented Glance from auto-migrating its 
database and now require users to migrate the database on new install 
and upgrade (via db_sync) in the same way Nova does.


2. Sync'ing a new database.

The changes also now allow users to set a specific version when 
initializing version_control on a database with glance-manage, eg:


glance-manage version_control 9
glance-manage db_sync

This would properly migrate an non-version controlled Diablo database to 
Essex. This is great, but it is still possible for Diablo users to 
mistakenly start glance or run db_sync with no version control, and end 
up in a situation where glance + migrate think it is up to date. In 
reality the database is missing 3 migrations (one of which, 012 IDs to 
UUIDs, is very essential!)   To prevent this, the Ubuntu patch 
additionally requires explicitly version_control'ing the database a 
before db_sync on both new databases and during upgrades of databases 
that are not already version_control'd.


For example, on a new database:

# Start from the beginning of the migration repositroy
glance-manage version_control 0
glance-manage db_sync

or to upgrade from Diablo:

# Was our database properly version controlled to begin with?
if ! glance-manage db_version ; then
  # Set its version if not
  glance-manage version_control 9
fi
# Run all migrations 10 to 13
glance-manage db_sync

I understand this may have come a bit late to any users of the Precise 
packages and I apologize for the lack of a heads up.  I've filed a new 
bug [2] about the split brain between glance-registry and and hope to 
address this at the database management session [3] at ODS next week.  
Anne Gentle has also proposed an update to the documentation with a note 
about this.   Please understand that our aim was to keep our patch delta 
as minimal as possible in Ubuntu, but with the yesterday's Final Freeze 
for the cycle 12.04 we needed something in place that ensures a sane 
user story for previous users of Diablo and future Essex users alike.


Cheers,
Adam

[1] https://review.openstack.org/#/c/6166/ 
https://review.openstack.org/#/c/5980/1

[2] https://bugs.launchpad.net/bugs/981081
[3] http://summit.openstack.org/sessions/view/36



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] glance-registry fails to start with latest ubuntu pkgs

[Thompson Lee]

Works!

On Apr 13, 2012, at 5:59 PM, David Kranz wrote:

> I ran into this a few hours ago. It seems you have to do
> 
> glance-manage version_control 0
> glance-manage db_sync
> 
> before restarting glance-registry. After I did that all was well.
> 
> -David
> 
> On 4/13/2012 5:03 PM, Lee Thompson wrote:
>> Fresh install or upgraded install, glance-registry fails.  Dropping the 
>> glance DB (postgreSQL) and recreating it doesn't help.  This is the error...
>> 
>> 
>> # cat registry.log
>> 2012-04-13 20:58:13 20717 INFO [sqlalchemy.engine.base.Engine] SELECT 
>> images.created_at AS images_created_at, images.updated_at AS 
>> images_updated_at, images.deleted_at AS images_deleted_at, images.deleted AS 
>> images_deleted, images.id  AS images_id, images.name 
>>  AS images_name, images.disk_format AS 
>> images_disk_format, images.container_format AS images_container_format, 
>> images.size AS images_size, images.status AS images_status, images.is_public 
>> AS images_is_public, images.location AS images_location, images.checksum AS 
>> images_checksum, images.min_disk AS images_min_disk, images.min_ram AS 
>> images_min_ram, images.owner AS images_owner, images.protected AS 
>> images_protected
>> FROM images
>> LIMIT %(param_1)s
>> 2012-04-13 20:58:13 20717 INFO [sqlalchemy.engine.base.Engine] 
>> {'param_1': 1}
>> 2012-04-13 20:58:13 20717 INFO [sqlalchemy.engine.base.Engine] ROLLBACK
>> 2012-04-13 20:58:13 20717ERROR [glance.registry.db.api] 
>> (ProgrammingError) relation "images" does not exist
>> LINE 2: FROM images
>> ^
>> 'SELECT images.created_at AS images_created_at, images.updated_at AS 
>> images_updated_at, images.deleted_at AS images_deleted_at, images.deleted AS 
>> images_deleted, images.id  AS images_id, images.name 
>>  AS images_name, images.disk_format AS 
>> images_disk_format, images.container_format AS images_container_format, 
>> images.size AS images_size, images.status AS images_status, images.is_public 
>> AS images_is_public, images.location AS images_location, images.checksum AS 
>> images_checksum, images.min_disk AS images_min_disk, images.min_ram AS 
>> images_min_ram, images.owner AS images_owner, images.protected AS 
>> images_protected \nFROM images \n LIMIT %(param_1)s' {'param_1': 1}
>> 2012-04-13 20:58:13 20717ERROR [glance.registry.db.api] Could not ensure 
>> database connection and consistency. Ensure database configuration and 
>> permissions are correct and database has been migrated since last upgrade by 
>> running 'glance-manage db_sync'
>> 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] glance-registry fails to start with latest ubuntu pkgs

[David Kranz]

I ran into this a few hours ago. It seems you have to do

glance-manage version_control 0
glance-manage db_sync

before restarting glance-registry. After I did that all was well.

 -David

On 4/13/2012 5:03 PM, Lee Thompson wrote:
Fresh install or upgraded install, glance-registry fails.  Dropping 
the glance DB (postgreSQL) and recreating it doesn't help.  This is 
the error...



# cat registry.log
2012-04-13 20:58:13 20717 INFO [sqlalchemy.engine.base.Engine] 
SELECT images.created_at AS images_created_at, images.updated_at AS 
images_updated_at, images.deleted_at AS images_deleted_at, 
images.deleted AS images_deleted, images.id  AS 
images_id, images.name  AS images_name, 
images.disk_format AS images_disk_format, images.container_format AS 
images_container_format, images.size AS images_size, images.status AS 
images_status, images.is_public AS images_is_public, images.location 
AS images_location, images.checksum AS images_checksum, 
images.min_disk AS images_min_disk, images.min_ram AS images_min_ram, 
images.owner AS images_owner, images.protected AS images_protected

FROM images
 LIMIT %(param_1)s
2012-04-13 20:58:13 20717 INFO [sqlalchemy.engine.base.Engine] 
{'param_1': 1}
2012-04-13 20:58:13 20717 INFO [sqlalchemy.engine.base.Engine] 
ROLLBACK
2012-04-13 20:58:13 20717ERROR [glance.registry.db.api] 
(ProgrammingError) relation "images" does not exist

LINE 2: FROM images
 ^
 'SELECT images.created_at AS images_created_at, images.updated_at AS 
images_updated_at, images.deleted_at AS images_deleted_at, 
images.deleted AS images_deleted, images.id  AS 
images_id, images.name  AS images_name, 
images.disk_format AS images_disk_format, images.container_format AS 
images_container_format, images.size AS images_size, images.status AS 
images_status, images.is_public AS images_is_public, images.location 
AS images_location, images.checksum AS images_checksum, 
images.min_disk AS images_min_disk, images.min_ram AS images_min_ram, 
images.owner AS images_owner, images.protected AS images_protected 
\nFROM images \n LIMIT %(param_1)s' {'param_1': 1}
2012-04-13 20:58:13 20717ERROR [glance.registry.db.api] Could not 
ensure database connection and consistency. Ensure database 
configuration and permissions are correct and database has been 
migrated since last upgrade by running 'glance-manage db_sync'




___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Just JSON, and extensibility

[Doug Davis]

Mark Nottingham wrote on 04/13/2012 12:56:46 PM:

> In particular, if people are actually using these tools to do data
> binding, it's going to lead them to place dependencies upon the
> structure of our interfaces, and unless the scheme is constructed
> *exactly* right, we'll get lots of bug reports in the future when
> these tools base their "contracts" on their interpretation of a
> Schema that got published on the Web site at some point in the past.

Be a bit careful here.  If you're suggesting that JSON is better because
it doesn't force (or even give you the option) of being precise in what
your interfaces looks like, implying that you won't get bug reports when
the JSON changes, I think this is probably wrong.  IMO, the stuff around
XML like XSDs (and I include in this topic things like IDLs) just makes
it easier for people to write tooling if they want. But in the end,
changing an interface is just as brittle regardless of whether its in
XML or JSON - both have extensibility points and both will have clients
that will break with the 'expected' data goes missing.

IMO, both JSON and XML have their places.  And I do agree with Mark
on one key thing... don't add all of the stuff to JSON that people
claimed was too heavy in XML.  And, to me, this includes namespaces.
In the end, once you start down that road you'll just end up with
a curl-braced version of XML and won't we look silly at that point.  :-)

Earlier you said:
> It's not that just "our dev community" isn't seeing much value in
> XML, it's a good portion of the world.

Again, this needs to be weighed very carefully. With emerging technologies
its easy to assume that the entire world agrees with you while things
are still too new to be part of the mainstream. This xml vs json fight has
been going on for a while, and I've heard a number of times from a number
of different companies that they will not touch JSON
simply because its too new, too unstructured, doesn't have the tooling
and validation capabilities of XML, blah blah blah.  Now, ideally these
same folks should be speaking up in the community to express their concerns
and desires, but for whatever reason they may not choose to - or can't.

Now, I'm not suggesting that we MUST keep XML just to satisfy
people who choose not to participate in OS, rather I want the decision to
be
made with all of the implications considered.  And to me, knowing that
Openstack
might not be as easy a sell to some folks because their shop is so heavily
invested in XML and not JSON (when, IMO, support for both isn't that hard -
its just a matter of making it part of the normal code development/approval
process) seems like it might be a mistake - at least given the current
state of the world.  After all, if XML were really as dead as some people
imply I doubt we would even be having this conversation.

-Doug___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Templates for slides?

[Michael Pittaro]

On Wed, Apr 11, 2012 at 5:47 PM, Stefano Maffulli  wrote:
> On Wed, 2012-04-11 at 17:21 -0700, Joshua Harlow wrote:
>> Are there any good powerpoint/keynote templates for slides that should
>> be used for the conference/summit (ie with openstack branding??)
>>
>> Or is it just show whatever u want for the template (if anything)?
>
> All speakers *for the conference* should be in contact with Laura Beck
> (I'll email you privately later with her email): she is your go-to point
> for all questions about logistics at the conference.
>
> For the summit, I don't think the standard is: use what you're familiar
> with :)
>

For the benefit of those of us who only know how to use text editors,
I just pushed an OpenStack themed template based on docutils / S5 to
my github repository. Feel free to use or modify it:

https://github.com/lhrc-mikeyp/Presentations/tree/master/OpenStackTemplate

tl;dr; Summary:

edit the presentation.rest file, and run rst2s5.py to generate
the slides.  More details in the readme.

mike

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Just JSON, and extensibility

[Caitlin Bestler]

Exactly what do you see as the required "non-linear extensibility"?

These are ultimately requests to a server. Each new extension is coded in that 
server.
There is no value in a client making up its own extensions that are not 
understood by the server.

What is relevant is a server continuing to support clients that have not yet 
been updated to
understand a new format.

As I stated in my first post, that problem was solved in ANSI C. Python/JSON is 
trivial.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] Essex, horizon, floating ip pool and compute node

[Jayashree(Jay) Beltur]

Hello,
I have installed lastest Essex version on 2 nodes. One controller node 
(nova-compute/nova-network/nova-api/nova-volume) and the other one is compute 
node (nova-compute/nova-api/nova-network) in flatDHCP mode.

I am able to launch VMs and associate private/floating ips and also ping and 
ssh to the VMs.

My controller node is on one network (16.78.*)  and my compute node is another 
network (16.89.*). So, i created 2 pools of floating ips, one for each network 
(so that I could associate guest VMs  appropriately based on the host they are 
launched on).
From the dashboard, i launched a VM. It was successful and showed the private 
ip address. But, the UI does not show on which node it was launched on. Without 
this information, I am not able allocate/associate the floating ip from 
appropriate pool. I have to go to the command line and run 
euca-describe-instances to check where it was launched and then, come back 
to dashboard to choose the floating ip from appropriate pool.

Is there a way on the horizon dashboard to find which host the guest 
VM actually got launched on? 

Thanks,
-Jay___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] glance-registry fails to start with latest ubuntu pkgs

[Lee Thompson]

Fresh install or upgraded install, glance-registry fails.  Dropping the
glance DB (postgreSQL) and recreating it doesn't help.  This is the error...


# cat registry.log
2012-04-13 20:58:13 20717 INFO [sqlalchemy.engine.base.Engine] SELECT
images.created_at AS images_created_at, images.updated_at AS
images_updated_at, images.deleted_at AS images_deleted_at, images.deleted
AS images_deleted, images.id AS images_id, images.name AS images_name,
images.disk_format AS images_disk_format, images.container_format AS
images_container_format, images.size AS images_size, images.status AS
images_status, images.is_public AS images_is_public, images.location AS
images_location, images.checksum AS images_checksum, images.min_disk AS
images_min_disk, images.min_ram AS images_min_ram, images.owner AS
images_owner, images.protected AS images_protected
FROM images
 LIMIT %(param_1)s
2012-04-13 20:58:13 20717 INFO [sqlalchemy.engine.base.Engine]
{'param_1': 1}
2012-04-13 20:58:13 20717 INFO [sqlalchemy.engine.base.Engine] ROLLBACK
2012-04-13 20:58:13 20717ERROR [glance.registry.db.api]
(ProgrammingError) relation "images" does not exist
LINE 2: FROM images
 ^
 'SELECT images.created_at AS images_created_at, images.updated_at AS
images_updated_at, images.deleted_at AS images_deleted_at, images.deleted
AS images_deleted, images.id AS images_id, images.name AS images_name,
images.disk_format AS images_disk_format, images.container_format AS
images_container_format, images.size AS images_size, images.status AS
images_status, images.is_public AS images_is_public, images.location AS
images_location, images.checksum AS images_checksum, images.min_disk AS
images_min_disk, images.min_ram AS images_min_ram, images.owner AS
images_owner, images.protected AS images_protected \nFROM images \n LIMIT
%(param_1)s' {'param_1': 1}
2012-04-13 20:58:13 20717ERROR [glance.registry.db.api] Could not
ensure database connection and consistency. Ensure database configuration
and permissions are correct and database has been migrated since last
upgrade by running 'glance-manage db_sync'
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Just JSON, and extensibility

[Justin Santa Barbara]

I don't think that works for non-linear extensibility...

I would be very happy if we could agree out how we're going to deal with
extensibility in JSON.  It is easy to support XML & JSON & any future
formats, and have them all be nice if there's willingness to do so, but
there's not, so let's drop it and focus on "stuff that matters"!

I believe we have a solution now for JSON extensibility (thanks Jorge),
it's not as "complete" as XML (read into that what you will) but it seems
to work fine.

My understanding is that the solution we have now is that any extension
goes into its own namespace; we assign a prefix to the namespace and have a
way to map that prefix to the full namespace.  (Similar to XML schemas).
 Currently prefixes are hard-coded, but we may not be able to keep doing
this forever (XML has pre-document prefixes to avoid the need for a central
authority).

I see 3 questions:
1) Is my summary correct?
2) Are there any problems with the solution?
3) Are there any other problems we're not addressing?


As one of the two authors of the Java binding, I can tell you how I plan on
dealing with extensions:


   - Map the JSON/XML/HPSTR to a strongly-typed model (so the
   representation is irrelevant to the consumer of the Java library).
   - Each (known) extension has its own strongly-typed model object.
   - These are stored in a registry.
   - When we come across an extension, we look it up in the registry and
   either bind or ignore it.
   - Every model object has an "Extensions" collection, which can be
   queried by type, to see if that extension data was present.


(Note: this has mostly been tested with the XML)

The nice thing about this is that a consumer of the library can write a
binding for an extension, and register it with the client library, and "it
just works".  So, even if you have a private extension, which you tell
nobody about and run only on your private cloud, you can use it with the
stock Java library.

Now, how you would do something that awesome in Python, I don't know ;-)

Justin



On Fri, Apr 13, 2012 at 11:53 AM, Caitlin Bestler <
caitlin.best...@nexenta.com> wrote:

> The argument that XML has better extensibility than JSON isn't very
> convincing to my ears.
>
> I'm an old war horse, and recall extending message formats in ANSI C so as
> to maintain backwards
> compatibility with existing clients (by versioning the struct name itself
> and always keeping the same
> fields up front).
>
> If this problem could be solved in ANSI C, the solutions available to
> Python with JSON can only be better.
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] ERROR: Can not find requested image (HTTP 400)

[Nicolas de BONFILS]

Hi,


In dashboard, in the images/snapshots area can you list the images ?
Can you put the output of glance-api, please ?

---
Nicolas



On Fri, Apr 13, 2012 at 18:07, Guilherme Birk  wrote:

>  I'm having problems when trying to boot a new instance using nova boot
> (ERROR: Can not find requested image (HTTP 400)).
>
> I can list all my images and instances using nova image-list and nova list
> without problems.
>
> Follow de --debug from the nova boot command that throws the error:
> http://pastebin.com/FusBZnfC
>
> Looks like the GET request on the image is being executed without problems
>
>1. send: u'GET
>
> /v2/2ad1fc162c254e59bea043560b7f73cb/images/ff0d6247-b40e-481e-803b-7c31124fbf82
>HTTP/1.1\r\nHost: 192.168.100.142:8774\r\nx-auth-project-id:
>openstackDemo\r\nx-auth-token:
>179ca0bb579748eb8a28d9240bc94b0d\r\naccept-encoding: gzip,
>deflate\r\naccept: application/json\r\nuser-agent:
>python-novaclient\r\n\r\n'
>2. reply: 'HTTP/1.1 200 OK\r\n'
>
>
>
> I don't know if this can be related to any other configuration. I can
> access and authenticate to the dashboard without problems.
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] EC2 compat.

[John Garbutt]

I got asked by E2C support by RealStatus who have a cool 3D modeling tool that 
works against EC2 and would love better APIs to make that work well with 
OpenStack:
http://www.youtube.com/watch?feature=player_embedded&v=SOkWRxDwTNI

Hopefully I should be able to lend a hand with some of this EC2 work. See you 
all at the summit session on EC2!

Cheers,
John

From: openstack-bounces+john.garbutt=eu.citrix@lists.launchpad.net 
[mailto:openstack-bounces+john.garbutt=eu.citrix@lists.launchpad.net] On 
Behalf Of Joshua Harlow
Sent: 11 April 2012 10:42
To: Duncan McGreggor
Cc: openstack
Subject: Re: [Openstack] EC2 compat.

Sweet,

Ideas and thoughts are welcome.

There is a lot of cleanup/testing/validation that needs to happen, but someones 
got to do it :-P

On 4/11/12 8:06 AM, "Duncan McGreggor"  wrote:
Lurking on the thread, but love what I'm seeing :-)

Nice work, guys!

d

On Tue, Apr 10, 2012 at 10:43 PM, Joshua Harlow  wrote:
> Very cool, glad to see that is being worked on, it looks pretty similar to
> what I was thinking of.
> I'm all for open dialogues.
> In fact.
> I was thinking of what is needed to make this work better.
>
> Open questions/thoughts/brainstorm (at least that I was thinking of):
>
> How strict do we want to be with the XSD? (there aren't a lot of tolerant
> xsd validators out there, which sux)
>
> Should we use something like jaxb for python, that should be more tolerant
> (unsure as what the best solution here is)
>
> How do we continuously measure the compatibility level?
>
> # of test cases passing, # of xml differences, # of xsd issues
>
> Should we use boto as a intermediate layer? (it is very tolerant)
>
> From what I understand there XML code is basically selecting certain
> attributes out of the XML using SAX, then adding any unknown attributes
> dynamically on to a object
>
> How do we make it repeatable?
>
> For a given test X, if there is a problem with test X and its response Y,
> how do we easily recreate that test X and response Y (so that dev's can fix
> it)?
> Do we have a "golden set" of responses that when test X is called it should
> match golden response Z (otherwise there is an issue)
>
> This is where the mock server maybe useful, in that we can point test X at
> the mock server; get the expected responses Z,
> Then point the test X at the real openstack server and get responses Y that
> should match Z (exactly, minus the request id?)
> EC2 seems to also already have some type of mocking, but I haven't used
> it... (http://bit.ly/HJkdh7)
>
>
> I like how there is a tests folder that u guys have, that seems like it
> could be a good location for the "content checking tests" which actually
> require code/logic to dig into the XML response. It might make sense to use
> another tool to verify the XSD's (how tolerant we want to be is an open
> question) and another tool that will show u the xml differences (some of
> which might be ok, some not). I have used in java xmlunit to do those kind
> of xml difference comparisons, it provides some nifty ways of ignoring
> certain differences and such. If say we had 3 levels of tests I think that
> would make sense (starting say from XSD validation, to difference
> comparisons to content comparisons), and would make a hell of a EC2 cool
> validation toolkit.
>
> The other usage of the site I was making was to list all the known error
> conditions, and any other incompatibilities that I am noticing with EC2
> (error conditions, features, parameters...). That seems really needed to
> allow for anyone to actually use the EC2 apis and handle all the cases which
> could be thrown at them.
>
> -Josh
>
>
>
>
> On 4/10/12 7:02 PM, "Eric Windisch"  wrote:
>
>
> Josh, as a follow-up, it would be good to keep an open dialogue on this.
> When/if you get a chance to review the aws-compat branch, I'd like to get
> your feedback as well.
>
> PS  I meant to write "assess", not "access". I only noticed when I read back
> my email. I'm too pedantic to not correct myself.
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Endpoints problems

[Adam Gandelman]

On 04/13/2012 10:50 AM, Dolph Mathews wrote:
While $(tenant_id)s is certainly the documented syntax, it appears 
that the SQL catalog backend (and *only* the SQL catalog backend, as 
far as I can tell) explicitly supports both $(tenant_id)s and 
%(tenant_id)s:


https://github.com/openstack/keystone/blob/master/keystone/catalog/backends/sql.py#L163

Perhaps Adam Gandelman has some insight?

-Dolph


Dolph-

No, the same is supported in the case of templated catalog as well, 
which is what the SQL catalog was largely based off:


https://github.com/openstack/keystone/blob/master/keystone/catalog/backends/templated.py#L115

Just tested that "sed -i 's/\$/%/g' 
/etc/keystone/default_catalog.templates" still produces a functional 
service catalog when configured to use the templated backend.


Seeing as both are supported, perhaps it would be better for docs to be 
updated to refer to the use of % instead of $ to avoid people running 
into problems with the $() sub-shell?


Adam
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] profiling nova-api

[Yuriy Taraday]

Great work!

About a year ago I found out than config was in the very top of
profiling table. It is there again.
That time there was a small misprint that led to constant config
reloading. This time i tried to add caching since our config became so
dynamic.
My efforts can be found here: https://review.openstack.org/6534 . My
benchmark showed time loss up to 40% in 'nova list' case. All three
lines from cfg.py have disappeared from top-10 of pstats. As well as
from top-40 :)
Looking forward to reviews.

Kind regards, Yuriy.



On Thu, Apr 12, 2012 at 00:48, Yun Mao  wrote:
> Hi Stackers, I spent some time looking at nova-api today.
>
> Setup: everything-on-one-node devstack, essex trunk. I setup 1 user
> with 10 tiny VMs.
> Client: 3 python threads each doing a loop of "nova list" equivalent
> for 100 times. So 300 API calls with concurrency=3.
> how to profile: python -m cProfile -s time
> /opt/stack/nova/bin/nova-api --flagfile=/etc/nova/nova.conf
> --logdir=/var/log/nova --nodebug
> The partial output is attached in the end.
>
> Observations:
> * It takes about 60 seconds to finish. CPU of nova-api is around 70% to 90%
>
> * Database access: Each "nova list" API call will issue 4 db APIs: 3
> instance_get_all_by_filters(), 1
> instance_fault_get_by_instance_uuids(), so 1200 db API calls total
> (note: not necessarily 1200 SQL statements, could be more). The 900
> instance_get_all_by_filters() calls took 30.2 seconds (i.e. 0.03s
> each)! The 300 instance_fault_get_by_instance_uuids() calls only took
> 1.129 seconds (0.004 each).
>
> You might think: MySQL sucks. Not so fast. Remember this is a tiny
> database with only 10 VMs. Profile also shows that the actual
> _mysql.connection.query() method only took 1.883 seconds in total. So,
> we pretty much spend 29 seconds out of 60 seconds doing either
> sqlalchemy stuff or our own wrapper. You can also see from the sheer
> volume of sqlalchemy library calls involved.
>
> * the cfg.py library inefficiency. During 300 API calls,
> common.cfg.ConfigOpts._get() is called 135005 times! and we paid 2.470
> sec for that.
>
> Hopefully this is useful for whoever wants to improve the performance
> of nova-api.
>
> Thanks,
> Yun
>
> ===
>
>         23355694 function calls (22575841 primitive calls) in 77.874 seconds
>
>   Ordered by: internal time
>
>   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>      812   25.725    0.032   25.725    0.032 {method 'poll' of
> 'select.epoll' objects}
>     2408    1.883    0.001    1.883    0.001 {method 'query' of
> '_mysql.connection' objects}
>    70380    1.667    0.000    7.187    0.000
> expression.py:2263(corresponding_column)
>   135005    1.254    0.000    2.470    0.000 cfg.py:1058(_get)
>    41027    1.043    0.000    1.907    0.000 schema.py:542(__init__)
>    38802    1.008    0.000    1.219    0.000 __init__.py:451(format)
>   162206    0.821    0.000    0.821    0.000 util.py:883(values)
>  1530666    0.773    0.000    0.774    0.000 {isinstance}
> 135046/134960    0.716    0.000    1.919    0.000 cfg.py:1107(_substitute)
>     1205    0.713    0.001    1.369    0.001 base.py:2106(__init__)
>   183600    0.690    0.000    0.796    0.000 interfaces.py:954(_reduce_path)
>    81002    0.687    0.000    2.492    0.000 compiler.py:312(visit_label)
>    38802    0.650    0.000    6.087    0.000 log.py:227(format)
>   319270    0.622    0.000    0.748    0.000 attributes.py:164(__get__)
> 890242/884229    0.608    0.000    1.885    0.000 {getattr}
>    40500    0.605    0.000    3.101    0.000 schema.py:955(_make_proxy)
> 120783/120738    0.603    0.000    0.605    0.000 {method 'sub' of
> '_sre.SRE_Pattern' objects}
>    81000    0.601    0.000    2.156    0.000
> interfaces.py:677(create_row_processor)
>    63000    0.590    0.000    0.707    0.000 times.py:44(DateTime_or_None)
>    98102    0.588    0.000    0.886    0.000 compiler.py:337(visit_column)
>   658098    0.580    0.000    0.581    0.000 {method 'intersection'
> of 'set' objects}
>   109802    0.562    0.000    0.562    0.000 expression.py:3625(_from_objects)
> 231610/1202    0.551    0.000    5.813    0.005
> visitors.py:58(_compiler_dispatch)
>   144002    0.510    0.000    0.693    0.000
> compiler.py:622(_truncated_identifier)
> 135005/134960    0.485    0.000    4.872    0.000 cfg.py:860(__getattr__)
>     2408    0.463    0.000    1.942    0.001 {built-in method fetch_row}
>    71100    0.460    0.000    0.580    0.000
> strategies.py:121(create_row_processor)
>   299031    0.437    0.000    0.437    0.000 {_codecs.utf_8_decode}
>     6000    0.437    0.000    1.799    0.000 models.py:93(iteritems)
> 36000/9000    0.409    0.000    4.791    0.001 mapper.py:2146(populate_state)
>    81002    0.393    0.000    1.104    0.000
> compiler.py:672(label_select_column)
> 45000/9000    0.390    0.000    5.148    0.001 mapper.py:2186(_instance)
>     1202    0.366    0.000    5.797    0.005 compiler.py:701(visit_select)
> 231610/1202    0

Re: [Openstack] Just JSON, and extensibility

[Mark Nottingham]

On 13/04/2012, at 2:04 PM, Anne Gentle wrote:

> Hi Mark, 
>  
> However, I question whether it's even being used in this fashion. Looking at 
> our docs for Nova 
> , the schema 
> isn't locatable, and the XML Namespace 
>  is a 404. How are they finding 
> the schema, and how is it getting into these tools?
> 
> 
> The schema is available at 
> http://docs.openstack.org/api/openstack-compute/2/xsd/
> 
> I am told that we don't need to redirect to the XML Namespace as it is just a 
> string not an actual URL. If you disagree, please log a doc bug at 
> http://bugs.launchpad.net/openstack-manuals/ and tag it with compute-api with 
> specifics about how you think it should work on the docs site.

OK.

> I'm not talking about *removing* XML from the current APIs that ship it, BTW; 
> at most I'd like to see us:
> 
> 1. Not require XML for new APIs, and
> 2. Talk about deprecating XML support in current ones (i.e., still support 
> it, but de-emphasise it in documentation, don't expend new effort on it), and
> 3. Put some considered caveats around our current use of WADL and XML Schema.
> 
> 
> One additional comment about WADL - basically one of those caveats. We had 
> little community traction documenting Compute API extensions with various 
> templates. A working solution now is to add extensions to api.openstack.org 
> using WADL and JSON samples. It's one of those examples of elbow grease and a 
> small group adding great value. Next week I'll be doing a "Show-N-Tell" of 
> the api.openstack.org site first thing Monday morning as well as on a panel 
> about TryStack Thursday afternoon and would love feedback.

Yes. I think the api.openstack.org is great and definitely moving in the right 
direction; should be an interesting discussion.

Cheers,

--
Mark Nottingham
http://www.mnot.net/





___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Just JSON, and extensibility

[Anne Gentle]

Hi Mark,


> However, I question whether it's even being used in this fashion. Looking
> at our docs for Nova <
> http://docs.openstack.org/api/openstack-compute/2/content/>, the schema
> isn't locatable, and the XML Namespace <
> http://docs.openstack.org/compute/api/v1.1> is a 404. How are they
> finding the schema, and how is it getting into these tools?
>
>
>
The schema is available at
http://docs.openstack.org/api/openstack-compute/2/xsd/

I am told that we don't need to redirect to the XML Namespace as it is just
a string not an actual URL. If you disagree, please log a doc bug at
http://bugs.launchpad.net/openstack-manuals/ and tag it with compute-api
with specifics about how you think it should work on the docs site.


I'm not talking about *removing* XML from the current APIs that ship it,
> BTW; at most I'd like to see us:
>
> 1. Not require XML for new APIs, and
> 2. Talk about deprecating XML support in current ones (i.e., still support
> it, but de-emphasise it in documentation, don't expend new effort on it),
> and
> 3. Put some considered caveats around our current use of WADL and XML
> Schema.
>
>
One additional comment about WADL - basically one of those caveats. We had
little community traction documenting Compute API extensions with various
templates. A working solution now is to add extensions to
api.openstack.orgusing WADL and JSON samples. It's one of those
examples of elbow grease and
a small group adding great value. Next week I'll be doing a "Show-N-Tell"
of the api.openstack.org site first thing Monday morning as well as on a
panel about TryStack Thursday afternoon and would love feedback.

Thanks,
Anne




> Cheers,
>
>
> --
> Mark Nottingham
> http://www.mnot.net/
>
>
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Just JSON, and extensibility

[Caitlin Bestler]

The argument that XML has better extensibility than JSON isn't very convincing 
to my ears.

I'm an old war horse, and recall extending message formats in ANSI C so as to 
maintain backwards
compatibility with existing clients (by versioning the struct name itself and 
always keeping the same
fields up front).

If this problem could be solved in ANSI C, the solutions available to Python 
with JSON can only be better.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] ERROR: Can not find requested image (HTTP 400)

[Guilherme Birk]

Hey Brian, I'm running a all-in-one installation. I have the flag 
--glance_api_servers=192.168.100.142:9292 on my nova.conf.

From: brian.wal...@rackspace.com
Subject: Re: [Openstack] ERROR: Can not find requested image (HTTP 400)
Date: Fri, 13 Apr 2012 11:18:17 -0700
To: guib...@hotmail.com



It sounds like your nova config might not be set up properly to talk to glance. 
Maybe you haven't properly set up glance servers in the config?

On Apr 13, 2012, at 9:07 AM, Guilherme Birk wrote:I'm having problems when 
trying to boot a new instance using nova boot (ERROR: Can not find requested 
image (HTTP 400)).

I can list all my images and instances using nova image-list and nova list 
without problems.

Follow de --debug from the nova boot command that throws the error: 
http://pastebin.com/FusBZnfC

Looks like the GET request on the image is being executed without problems 
send: u'GET 
/v2/2ad1fc162c254e59bea043560b7f73cb/images/ff0d6247-b40e-481e-803b-7c31124fbf82
 HTTP/1.1\r\nHost: 192.168.100.142:8774\r\nx-auth-project-id: 
openstackDemo\r\nx-auth-token: 
179ca0bb579748eb8a28d9240bc94b0d\r\naccept-encoding: gzip, deflate\r\naccept: 
application/json\r\nuser-agent: python-novaclient\r\n\r\n'reply: 'HTTP/1.1 200 
OK\r\n'

I don't know if this can be related to any other configuration. I can access 
and authenticate to the dashboard without problems.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

  ___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Endpoints problems

[Justin Shepherd]

So why would the preferred method be the one that you have to do all sort of 
magic stuff to protect from bash extrapolation.. and also given that the code 
converts those to %(tenant_id).. why wouldn't the "%(" way be the preferred way?

Sounds like an artifact of the CLI being a second class use case..

--shep
On Apr 13, 2012, at 8:48 AM, Dean Troyer wrote:

> On Fri, Apr 13, 2012 at 8:28 AM, Guilherme Birk  wrote:
>> keystone --token ADMIN --endpoint http://192.168.100.142:35357/v2.0
>> endpoint-create --region RegionOne
>> --service_id=1fd7b5f1add74aa4b6efc514fd153e72
>> --publicurl="http://192.168.100.142:8774/v2/$(tenant_id)s"
>> --adminurl="http://192.168.100.142:8774/v2/$(tenant_id)s"
>> --internalurl="http://192.168.100.142:8774/v2/$(tenant_id)s"
>> 
>> But I'm getting a "tenant_id: command not found". When I list the endpoints
>> all my url's are like "http://192.168.100.142:8774/v2/s"; for the created
>> endpoint.
>> Am I doing something wrong ?
> 
> You need to escape the '$'s in your URLs or change the double quotes
> around them to single quotes.  The shell sees the $(tenant_id) and
> attempts to execute a command called tenant_id.
> 
> dt
> 
> -- 
> 
> Dean Troyer
> dtro...@gmail.com
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Health & Monitoring Blueprints

[Duncan McGreggor]

Lyle replied "yes" -- sadly, we can't edit the sessions anymore.

Fortunately, there's the wiki :-) I've updated the session to point to
these blueprints:
  http://wiki.openstack.org/Summit/Folsom/DevOps

d

On Wed, Apr 11, 2012 at 4:41 PM, Duncan McGreggor  wrote:
> Lyle, would you mind if I add these to the "Instrumenting OpenStack" session?
>  http://summit.openstack.org/sessions/edit/163
>
> d
>
> On Wed, Apr 11, 2012 at 4:33 PM, Wilkinson, Lyle  
> wrote:
>> Hi folks,
>>
>>
>>
>> We’ve created a set of 3 blueprints on the topics of health and monitoring,
>> metrics data collection, and cloud inventory maintenance.
>>
>>
>>
>> https://blueprints.launchpad.net/nova/+spec/resource-monitor-alerts-and-notifications
>>
>> https://blueprints.launchpad.net/nova/+spec/utilizationdata
>>
>> https://blueprints.launchpad.net/nova/+spec/cloud-inventory-manager
>>
>>
>>
>> We’re hoping to discuss these at the design summit next week, but currently
>> don’t know if they’ll be part of an existing session or covered in a
>> separate session in one of the open space.  We’ve created some etherpad
>> pages to get the discussion going, so if you have questions, suggestions,
>> etc., feel free to contribute there.
>>
>>
>>
>> http://etherpad.openstack.org/ResourceMonitorAlertsandNotifications
>>
>> http://etherpad.openstack.org/utilizationdata
>>
>> http://etherpad.openstack.org/Cloud-Inventory-Manager
>>
>>
>>
>> Thanks in advance!
>>
>>
>>
>> Lyle Wilkinson
>>
>>
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Endpoints problems

[Dolph Mathews]

While $(tenant_id)s is certainly the documented syntax, it appears that the
SQL catalog backend (and *only* the SQL catalog backend, as far as I can
tell) explicitly supports both $(tenant_id)s and %(tenant_id)s:

https://github.com/openstack/keystone/blob/master/keystone/catalog/backends/sql.py#L163

Perhaps Adam Gandelman has some insight?

-Dolph

On Fri, Apr 13, 2012 at 10:12 AM, Kiall Mac Innes wrote:

> I would be surprised if you could simply interchange the $ with a %..
> Never tried it though!
>
> Anyway - $ is defiantly the "correct" character to use:
> https://github.com/openstack-dev/devstack/blob/master/files/default_catalog.templates
>
> Thanks,
> Kiall
>
>
>
> On Fri, Apr 13, 2012 at 2:48 PM, David Kranz wrote:
>
>>  As far as my experience goes, you have to use %(tenant_id)s. I ran into
>> this problem the first time I did it as well. $ makes the shell think it's
>> a variable.
>>
>> David Kranz
>> Quanta Research Cambridge
>>
>>
>>
>>
>> On 4/13/2012 9:28 AM, Guilherme Birk wrote:
>>
>>  I've tried to execute the following command:
>>
>> keystone --token ADMIN --endpoint 
>> http://192.168.100.142:35357/v2.0endpoint-create --region RegionOne
>> --service_id=1fd7b5f1add74aa4b6efc514fd153e72 --publicurl=
>> "http://192.168.100.142:8774/v2/$(tenant_id)s"--adminurl=
>> "http://192.168.100.142:8774/v2/$(tenant_id)s"--internalurl=
>> "http://192.168.100.142:8774/v2/$(tenant_id)s"
>>
>> But I'm getting a "tenant_id: command not found". When I list the
>> endpoints all my url's are like 
>> "http://192.168.100.142:8774/v2/s";for the 
>> created endpoint.
>> Am I doing something wrong ?
>>
>> Thanks.
>>
>>  --
>> From: a...@openstack.org
>> Date: Thu, 12 Apr 2012 15:28:21 -0500
>> Subject: Re: [Openstack] Endpoints problems
>> To: guib...@hotmail.com
>> CC: openstack@lists.launchpad.net
>>
>> Hi Guilherme -
>> Sorry you ran into a doc bug -
>> https://bugs.launchpad.net/openstack-manuals/+bug/977905.
>>
>> Basically, the bug states that the nova endpoint definition should be:
>>
>> keystone --token 012345SECRET99TOKEN012345 --endpoint
>> http://192.168.206.130:35357/v2.0 endpoint-create \
>>
>>--region RegionOne \
>>--service_id=abc0f03c02904c24abdcc3b7910e2eed \
>>--publicurl
>> http://192.168.206.130:8774/v2/$(tenant_id)s \
>>--adminurl
>> http://192.168.206.130:8774/v2/$(tenant_id)s \
>>--internalurl
>> http://192.168.206.130:8774/v2/$(tenant_id)s
>>
>> I haven't fixed this yet because I'm not sure if the $(tenant_id)s is
>> literal or which tenant_id specifically to use (the Service tenant for the
>> adminurl possibly)?
>>
>> If someone on the list could offer more input here and on the doc bug it
>> would be greatly appreciated!
>> Anne
>>
>> On Thu, Apr 12, 2012 at 12:25 PM, Guilherme Birk wrote:
>>
>>  I'm having problems setting up the nova endpoint. I've followed the
>> manual
>> http://docs.openstack.org/trunk/openstack-compute/install/content/setting-up-tenants-users-and-roles.html,
>> putting the tenant id on the url's, like the manual says to do. But when I
>> try execute "nova list" I got a malformed url error. When I set the
>> endpoint without the tenant id on the url's I got a 404 error. Anyone
>> having the same problem?
>>
>> I can access the dashboard normally, but I'm unable to retrieve instance
>> list.
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Just JSON, and extensibility

[Mark Nottingham]

On 13/04/2012, at 11:29 AM, Jorge Williams wrote:
> 
> There are a lot more clients than servers.  You have to weigh the cost of 
> lowering barriers for those clients at the server side vs the cost of getting 
> those client to successfully integrate with the system.  This is typically 
> the a argument I make against SOAP and towards REST.  In static language 
> circles creating a SOAP service is really, really  easy…

As long as the server and client are using the same tools, yes. SOAP doesn't 
provide any interoperability, it's just an envelope for a proprietary messaging 
protocol. Background: 

But I digress...


> but using soap typically introduces  high barriers to dynamic language folks. 
>  Making the move from SOAP to REST definitely introduces some complexity on 
> the dev side of the service in dev cycles, but you have to compare this cost 
> against the cost that's saved on the client side and multiply that by the 
> number of clients that benefit. 
> 
> Having said that,  I understand the cost of supporting a different media type 
> is not insignificant, but it's also not rocket science. I reckon that the 
> sate of XML support today may have to do more with the fact that our dev 
> community doesn't see much value in XML, than in does with what it will 
> actually takes to get it rightand like I said before, I can understand 
> that perspective. 

OK, we're going to have to disagree here. It's not that just "our dev 
community" isn't seeing much value in XML, it's a good portion of the world. 
The question here is how far we bend over to cater to those who are using these 
tools, and how we go about it. I'm suggesting that doing it in the protocol, 
rather than with tools, is buying us more long-term trouble than it's solving 
for them.

In particular, if people are actually using these tools to do data binding, 
it's going to lead them to place dependencies upon the structure of our 
interfaces, and unless the scheme is constructed *exactly* right, we'll get 
lots of bug reports in the future when these tools base their "contracts" on 
their interpretation of a Schema that got published on the Web site at some 
point in the past.

However, I question whether it's even being used in this fashion. Looking at 
our docs for Nova , 
the schema isn't locatable, and the XML Namespace 
 is a 404. How are they finding the 
schema, and how is it getting into these tools?


> It messes up some of the syntactic sugar that folks are used to using:
> 
> server.namevs   server["foo:name"]  

If you use colons, sure, but let's not try to reinvent Namespaces in XML in 
JSON.


>> Sure. The question I'd ask, then, is it worth making our APIs seriously more 
>> complex, hard to develop, understand, maintain, test, document, etc. in the 
>> meantime, just to allow static language users to have their IDEs help them. 
>> Something that the dynamic folks have gotten pretty used to living without.
> 
> Asking static language folks to just get used to not having compile time 
> checks is silly. That said, I don't want to turn this into a static vs 
> dynamic language debate.  I advocate just as hard for good JSON support when 
> I'm dealing with static language folksgo talk to the atlas guys.
> 
> Again, I'm generally agreeing with what your article is saying.  If we had 
> great/nontrivial static language binding support...that's extensible and 
> flexible enough to remove the need to go down to the HTTP level, then there 
> wouldn't be a need for this debate.


Let's get there, then!

We won't have them tomorrow, so what I think we need to think very carefully 
about is whether offering this audience XML at this point in time is doing more 
harm than good, both in terms of opportunity cost and in terms of causing 
problems down the road.

I'm not talking about *removing* XML from the current APIs that ship it, BTW; 
at most I'd like to see us:

1. Not require XML for new APIs, and
2. Talk about deprecating XML support in current ones (i.e., still support it, 
but de-emphasise it in documentation, don't expend new effort on it), and
3. Put some considered caveats around our current use of WADL and XML Schema.

Cheers,


--
Mark Nottingham
http://www.mnot.net/





___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Just JSON, and extensibility

[Jorge Williams]

On Apr 13, 2012, at 8:47 AM, Mark Nottingham wrote:

> [ Full disclosure -- I'm using my personal address with Launchpad, etc., but 
> I work for Rackspace. ]
> 
> On 12/04/2012, at 7:28 PM, Jorge Williams wrote:
> 
>> Generally, I agree with a lot of what you're saying, but I want to point out 
>> a couple of things:
>> 
>> 1.  Static language folks gravitate to XML, not simply because they're 
>> invested in it, but because it solves a real problem:
>> […]
>> and I should see those errors at compile time or while I'm authoring my code 
>> if a have a good editor or IDEI shouldn't have to wait until my program 
>> is run to catch those errors. 
>> […]
>> but at that point, there's very little benefit to having a static language, 
>> because I don't have the assurances and guarantees that the language 
>> provides. So I don't see a lot of Java folks dealing with JSON in that 
>> manner.  Most devs will  need to build a class before hand.  So, you 
>> decrease barriers for static language clients because there's a set of tools 
>> that can extract the relevant info from XML schema languages and generate a 
>> set of class files at compile time.   There's nothing saying you can't do 
>> something similar with JSON, but those sort of tools aren't there  yet.
> 
> Great -- I think it's good to level set here. We're talking about supporting 
> an entire serialisation, all of its supporting code, the QA cycle, schema 
> authoring, etc., and the #1 benefit is making the programming experience for 
> a subset of our audience more comfortable. 
> 

There are a lot more clients than servers.  You have to weigh the cost of 
lowering barriers for those clients at the server side vs the cost of getting 
those client to successfully integrate with the system.  This is typically the 
a argument I make against SOAP and towards REST.  In static language circles 
creating a SOAP service is really, really  easy...but using soap typically 
introduces  high barriers to dynamic language folks.  Making the move from SOAP 
to REST definitely introduces some complexity on the dev side of the service in 
dev cycles, but you have to compare this cost against the cost that's saved on 
the client side and multiply that by the number of clients that benefit. 

Having said that,  I understand the cost of supporting a different media type 
is not insignificant, but it's also not rocket science. I reckon that the sate 
of XML support today may have to do more with the fact that our dev community 
doesn't see much value in XML, than in does with what it will actually takes to 
get it rightand like I said before, I can understand that perspective. 

> 
>> 2.  Then there's the issue of extensibilityespecially distributed 
>> extensibility. XML has that notion built in
> 
> Heh; you mean bolted on with Namespaces in XML, and then completely messed up 
> by XML Schema 1.0, then only partially fixed by 1.1. But yes.
> 
>> JSON has no concept of it...and we are building extensible APIs. There are 
>> no standard way in JSON to introduce a  new property without guaranteeing 
>> that there won't be  clash.  You've mention the need for namespaces in JSON 
>> precisely to  deal with this sort of issue 
>> (http://www.mnot.net/blog/2011/10/12/thinking_about_namespaces_in_json). 
> 
> First of all, as we've discussed many times, I don't think "extensibility" == 
> "good" in all cases; if we allow/encourage too much extensibility, the 
> platform we're building will fragment, and people won't see its full value. 
> Extensions should be allowed where the make sense, not everywhere, and 
> extensions should be encouraged to eventually converge.

I totally agree, encouraging  convergence is important.

> 
>> In the absence of a standard method, we're been using prefixes, which has 
>> worked out well, but most JSON tools don't know how to deal with them
> 
> What does that *mean*? The prefix is an opaque part of the name -- what 
> should a tool do with it?
> 

It messes up some of the syntactic sugar that folks are used to using:

server.namevs   server["foo:name"]  


>> and they seem alien to folk that are used to using JSON day to day.
> 
> Perhaps, but doing something like RAX_FOO isn't that onerous. Or using a 
> registry. Or just leaving it to the community to coordinate and document; try 
> opening a shell, typing "set" and pondering how *that* name space is 
> managed...
> 
>> This is a big deal because dynamic language folks are more likely to deal 
>> with the JSON directly...Static language folks are generally not dealing 
>> with XML in the same way.  In XML, the notion of extensibility is build into 
>> parsers  and data binding tools directly.
> 
> Not really; it's a syntactic artefact of namespaces. I'd say that the tools 
> manage it really, really badly, given how hard it is to author extensible 
> schemas (and don't get me started on XML Schema).
> 

That hasn't entirely been my experience.  For the most part you can se

Re: [Openstack] New Gerrit version (and server)

[Anne Gentle]

Thanks James and team for the work here, this will be very useful for
documentation drafts.

Anne

On Fri, Apr 13, 2012 at 8:55 AM, Sandy Walsh wrote:

> Sounds awesome! Looking forward to Draft Changes, much needed.
>
> -S
>
>
> 
> From: 
> openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net[openstack-bounces+sandy.walsh=
> rackspace@lists.launchpad.net] on behalf of James E. Blair [
> cor...@inaugust.com]
> Sent: Thursday, April 12, 2012 8:23 PM
> To: OpenStack Mailing List
> Subject: [Openstack] New Gerrit version (and server)
>
> Hi,
>
> We've just upgraded Gerrit to version 2.3.  There are a lot of changes
> behind the scenes that we've been looking forward to (like being able to
> store data in innodb rather than myisam tables for extra data
> longevity).  And there are a few visible changes that may be of interest
> to OpenStack developers.
>
> One new addition in 2.3 is "draft changes".  The idea behind a draft
> change in Gerrit is that it is a change that is not ready for merging,
> or even general code review, but you would like to share it with some
> people to get early comments.  If you upload a change as a draft, by
> default, no one else can see it.  You must explicitly add each person
> you would like to share it with as a reviewer.  Reviewers you add can
> leave comments, but can not vote at this stage.  You can continue to
> upload new patchsets to the change as it evolves, and once it is ready
> for general review, you can click the "Publish" button.  It will then
> become a normal change in Gerrit that everyone can see, including the
> earlier reviews from the draft stage.  This is a one way transition;
> once a draft is published, it can't be made a draft again.
>
> If you're using git-review from source or the latest version from PyPI
> (version 1.16, released today), you can easily upload a draft change by
> adding the "-D" option (eg, "git review -D").  Earlier versions of
> git-review also have the "-D" option, but the git ref that Gerrit uses
> to indicate a change should be a draft was changed between the 2.3
> release candidate and the final release; so if using "-D" results in an
> error, you may need to upgrade.
>
> You may notice some changes to the diff view.  Notably, the header which
> contained all of the possible viewing options has been split up into
> several parts; you can switch between them by selecting options that
> show up under the menu at the top.  I recommend setting "Retain Header
> On File Switch" under the Preferences section, as it is a nicer
> experience when changing files.
>
> Another notable new feature is the ability to add a group to the list of
> reviewers for a change.  Just type in the name of the group and click
> "Add Reviewer" and all of the individuals in the group will be added to
> the list of reviewers (and will see the change on their "review
> requests" list.
>
> Finally, we've modified some of our local OpenStack style changes so
> that it is easier for us to track upstream changes in layout.  It should
> mean a little more consistency throughout the interface, though we
> weren't able to keep the alternating row colors on the main table
> without a disproportionate amount of effort.  Do note that you can click
> on a line in a table, and it will be highlighted to improve legibility.
>
> We've tried to give this as much testing as possible before moving it
> into production.  If you encounter any issues, please let us know on IRC
> (mtaylor, jeblair, LinuxJedi), via email at
> , or you can file a bug at:
>
>  https://bugs.launchpad.net/openstack-ci/
>
> Thanks,
>
> Jim
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] ERROR: Can not find requested image (HTTP 400)

[Guilherme Birk]

I'm having problems when trying to boot a new instance using nova boot (ERROR: 
Can not find requested image (HTTP 400)).

I can list all my images and instances using nova image-list and nova list 
without problems.

Follow de --debug from the nova boot command that throws the error: 
http://pastebin.com/FusBZnfC

Looks like the GET request on the image is being executed without problems 
send: u'GET 
/v2/2ad1fc162c254e59bea043560b7f73cb/images/ff0d6247-b40e-481e-803b-7c31124fbf82
 HTTP/1.1\r\nHost: 192.168.100.142:8774\r\nx-auth-project-id: 
openstackDemo\r\nx-auth-token: 
179ca0bb579748eb8a28d9240bc94b0d\r\naccept-encoding: gzip, 
deflate\r\naccept: application/json\r\nuser-agent: 
python-novaclient\r\n\r\n'reply: 'HTTP/1.1 200 OK\r\n'
I don't know if this can be related to any other configuration. I can access 
and authenticate to the dashboard without problems.
  ___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Multiple nova-networks with QuantumManager (Was : Role of "nova-manage network" commands when using QuantumManager)

[Dan Wendlandt]

On Fri, Apr 13, 2012 at 5:34 AM, Vaze, Mandar wrote:

>  Dan,
>
> ** **
>
> On similar lines – Currently if nova-network processes are running on *two
> * nodes, only way to set specific network on specific nova-network node
> is to execute “nova-manage network create” on respective nova-network node.
> Is that correct ?
>
> ** **
>
> How is the above setup  related to “multi-host” ? I found a code comment
> that “Quantum Manager doesn’t support multi-host” 
>
> ** **
>
> I’m copying Vish, as he seems to be expert on multi-host – Reading
> http://docs.openstack.org/trunk/openstack-compute/admin/content/existing-ha-networking-options.html,
>  it looks like above is Option 2/Multi-Nic, but not really sure)  If yes,
> how is multi-nic supported by Quantum ?
>
> ** **
>
> Additionally – in the above document Vish suggested that each nova-compute
> must run nova-network – Is that why with QuantumManager – nova-manage
> executes networking commands like iptables, dnsmasq locally ?
>

Yes, as documented Quantum Manager in Essex does not support multi-host.
 This is why the nova-manage limitation is workable, albiet somewhat
inconvenient.

In Folsom, we will support a model that provides the same HA benefits of
multi-host, though the implementation will be different, as the L3
forwarding + DHCP functions will be handled by Quantum, not by Nova.

Dan


> 
>
> ** **
>
> ** **
>
> Thanks !!
>
> -Mandar
>
> ** **
>
> ** **
>
> *From:* Dan Wendlandt [mailto:d...@nicira.com]
> *Sent:* Thursday, April 12, 2012 9:44 PM
> *To:* Vaze, Mandar
> *Cc:* openstack@lists.launchpad.net; netst...@lists.launchpad.net
> *Subject:* Re: [Netstack] Role of "nova-manage network" commands when
> using QuantumManager
>
> ** **
>
> Hi Mandar,
>
> ** **
>
> Thanks for bringing this up.  For Essex, nova-manage commands to
> create/delete Quantum networks must be run on the nova-network node.  For
> Folsom this will all change, as all networks will be created directly
> against the Quantum API, rather than using nova-manage.  I'll add a note to
> the administrator guide, as this is not called there.  Thanks!
>
> ** **
>
> Dan
>
>  
>
> ** **
>
> On Thu, Apr 12, 2012 at 1:38 AM, Vaze, Mandar 
> wrote:
>
> It is my understanding that in multi-node setup :
>
> · nova-manage can be executed from any machine which may not be
> running nova-network process.  (Or should nova-manage always be run on
> nova-network node ?)
>
> · nova-manage does DB operations and delegates the actual
> networking calls to nova-network process ?
>
>  
>
> Is this understanding correct ?
>
>  
>
> I traced “nova-manage network create” and nova-manage network delete”
> using FlatDHCPManager (default for devstack/stack.sh)
>
> Both these calls seem to be doing only DB operations.
>
> BTW, nova-network process was shutdown during both “network create” and
> “network delete” – Still both operations were successful.
>
>  
>
> But when using QuantumManager as network manager – nova-manage seems to be
> doing networking operations like “iptables-save” (during network create)
> and “kill_dhcp” (during network delete)
>
> (via linux-net L3 driver)
>
>  
>
> Since nova-manage command may be executed on a host which isn’t running
> nova-network – network commands like “iptables” and “kill -9  dnsmasq>” on host running nova-manage seems incorrect.
>
>  
>
> For the first scenario (iptables-save during “network create” – there is
> already a defect in LP : https://bugs.launchpad.net/nova/+bug/977738
>
> and review : https://review.openstack.org/6451
>
>  
>
> I would like your comments and opinions which would help me understand
>  “What nova-manage should and should NOT do”
>
>  
>
> Thanks,
>
> -Mandar
>
>  
>
>  
>
>  
>
>  
>
>
> __
> Disclaimer:This email and any attachments are sent in strictest confidence
> for the sole use of the addressee and may contain legally privileged,
> confidential, and proprietary data. If you are not the intended recipient,
> please advise the sender by replying promptly to this email and then delete
> and destroy this email and any attachments without any further use, copying
> or forwarding
>
>
> --
> Mailing list: https://launchpad.net/~netstack
> Post to : netst...@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~netstack
> More help   : https://help.launchpad.net/ListHelp
>
>
>
> 
>
> ** **
>
> --
> ~~~
> Dan Wendlandt 
>
> Nicira, Inc: www.nicira.com
>
> twitter: danwendlandt
> ~~~
>
> ** **
>
> __
> Disclaimer:This email and any attachments are sent in strictest confidence
> for the sole use of the addressee and may contain legally privileged,
> confidenti

Re: [Openstack] New Gerrit version (and server)

[Monty Taylor]

On 04/13/2012 04:36 AM, Kiall Mac Innes wrote:
> The new gerrit version also supports per user namespaces, if enabled.
> 
> These allow users to create private branches with full push privileges etc..
> 
> Have these been enabled?

They have not - we need to verify what the behavior looks like over the
event interface so that we're not all of a sudden attempting to devstack
a bunch of in-progress stuff without meaning to.

We'll put it on the list of new things to investigate our possible use of!

Monty

> On Apr 13, 2012 12:33 p.m., "Thierry Carrez"  > wrote:
> 
> Mark McLoughlin wrote:
> >> One new addition in 2.3 is "draft changes".  The idea behind a draft
> >> change in Gerrit is that it is a change that is not ready for
> merging,
> >> or even general code review, but you would like to share it with some
> >> people to get early comments.  If you upload a change as a draft, by
> >> default, no one else can see it.  You must explicitly add each person
> >> you would like to share it with as a reviewer.  Reviewers you add can
> >> leave comments, but can not vote at this stage.  You can continue to
> >> upload new patchsets to the change as it evolves, and once it is
> ready
> >> for general review, you can click the "Publish" button.  It will then
> >> become a normal change in Gerrit that everyone can see, including the
> >> earlier reviews from the draft stage.  This is a one way transition;
> >> once a draft is published, it can't be made a draft again.
> >
> > This sounds cool. Will the vulnerability management team use this for
> > embargoed security fixes?
> 
> I'll definitely look into that possibility... Depends a bit on the
> security model around drafts (not being listed is not the same as being
> private).
> 
> --
> Thierry Carrez (ttx)
> Release Manager, OpenStack
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> 
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] quantum launchpad download link

[Dan Wendlandt]

The launchpad gods have been generous.  For future reference, marking the
Diablo release as "obsolete" helps launchpad figure out that a release
dated in 2012 is more recent than one in 2011 :)

Sorry for the confusion around the bad link.

Dan

On Thu, Apr 12, 2012 at 2:34 PM, Dan Wendlandt  wrote:

> Hi folks,
>
> A note of caution and a request for help from the launchpad gods.
>
> The "latest release" tarball on the Quantum launchpad site (
> http://launchpad.net/quantum) actually points to the diablo release of
> Quantum, not the Essex release.  I'm guessing this has confused at least
> one person, as the diablo release was last downloaded today, which is a bit
> odd for an old (in OpenStack time) release.
>
> This link was correct for the Essex RC's, but something seems to have gone
> wrong with the final release of Essex.
>
> The correct release is labeled as 2012.1:
> https://launchpad.net/quantum/essex/2012.1
>
> Anyone with good launchpad knowledge know how to fix this?  The release
> dates seem correct on both release, so I'm confused why launchpad has the
> wrong notion of "latest".
>
> Thanks,
>
> Dan
>
>
> --
> ~~~
> Dan Wendlandt
> Nicira, Inc: www.nicira.com
> twitter: danwendlandt
> ~~~
>
>


-- 
~~~
Dan Wendlandt
Nicira, Inc: www.nicira.com
twitter: danwendlandt
~~~
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Image API v2 Draft 4

[Mark Nottingham]

I'm working on that. 

Sent from my iPhone

On 13/04/2012, at 9:55 AM, Kiall Mac Innes  wrote:

> Maybe RackSpace (or any other large operator..) can share some stats of the 
> user-agents they see in the wild?

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Endpoints problems

[Kiall Mac Innes]

I would be surprised if you could simply interchange the $ with a %.. Never
tried it though!

Anyway - $ is defiantly the "correct" character to use:
https://github.com/openstack-dev/devstack/blob/master/files/default_catalog.templates

Thanks,
Kiall


On Fri, Apr 13, 2012 at 2:48 PM, David Kranz  wrote:

>  As far as my experience goes, you have to use %(tenant_id)s. I ran into
> this problem the first time I did it as well. $ makes the shell think it's
> a variable.
>
> David Kranz
> Quanta Research Cambridge
>
>
>
>
> On 4/13/2012 9:28 AM, Guilherme Birk wrote:
>
>  I've tried to execute the following command:
>
> keystone --token ADMIN --endpoint 
> http://192.168.100.142:35357/v2.0endpoint-create --region RegionOne
> --service_id=1fd7b5f1add74aa4b6efc514fd153e72 --publicurl=
> "http://192.168.100.142:8774/v2/$(tenant_id)s"--adminurl=
> "http://192.168.100.142:8774/v2/$(tenant_id)s"--internalurl=
> "http://192.168.100.142:8774/v2/$(tenant_id)s"
>
> But I'm getting a "tenant_id: command not found". When I list the
> endpoints all my url's are like 
> "http://192.168.100.142:8774/v2/s";for the 
> created endpoint.
> Am I doing something wrong ?
>
> Thanks.
>
>  --
> From: a...@openstack.org
> Date: Thu, 12 Apr 2012 15:28:21 -0500
> Subject: Re: [Openstack] Endpoints problems
> To: guib...@hotmail.com
> CC: openstack@lists.launchpad.net
>
> Hi Guilherme -
> Sorry you ran into a doc bug -
> https://bugs.launchpad.net/openstack-manuals/+bug/977905.
>
> Basically, the bug states that the nova endpoint definition should be:
>
> keystone --token 012345SECRET99TOKEN012345 --endpoint
> http://192.168.206.130:35357/v2.0 endpoint-create \
>
>--region RegionOne \
>--service_id=abc0f03c02904c24abdcc3b7910e2eed \
>--publicurl
> http://192.168.206.130:8774/v2/$(tenant_id)s \
>--adminurl
> http://192.168.206.130:8774/v2/$(tenant_id)s \
>--internalurl
> http://192.168.206.130:8774/v2/$(tenant_id)s
>
> I haven't fixed this yet because I'm not sure if the $(tenant_id)s is
> literal or which tenant_id specifically to use (the Service tenant for the
> adminurl possibly)?
>
> If someone on the list could offer more input here and on the doc bug it
> would be greatly appreciated!
> Anne
>
> On Thu, Apr 12, 2012 at 12:25 PM, Guilherme Birk wrote:
>
>  I'm having problems setting up the nova endpoint. I've followed the
> manual
> http://docs.openstack.org/trunk/openstack-compute/install/content/setting-up-tenants-users-and-roles.html,
> putting the tenant id on the url's, like the manual says to do. But when I
> try execute "nova list" I got a malformed url error. When I set the
> endpoint without the tenant id on the url's I got a 404 error. Anyone
> having the same problem?
>
> I can access the dashboard normally, but I'm unable to retrieve instance
> list.
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Image API v2 Draft 4

[Kiall Mac Innes]

On Fri, Apr 13, 2012 at 1:28 AM, Jorge Williams <
jorge.willi...@rackspace.com> wrote:

> Having said all of that,  I realize that our devs are working in a dynamic
> language, and don't see a lot of value in XML.  It's important to take that
> into consideration, but we should also be asking whether there's value to
> our current clients and potential clients.  Like it or not, there's a lot
> of folks out there using static languages.
>

I actually think this is an important point (But in favor of JSON!).

The OpenStack developers are writing in a dynamic language, python. This
means the tooling and experience available will be biased towards JSON over
XML for the most part. This, IMHO, will result in a less than perfect XML
implementation. I would much prefer to see a single great XML or
JSON implementation rather than two poor implementations.

Combine that with the massive uptake of dynamic languages over recent
years, and I would bet JSON is a more natural fit for a higher percent of
developers and end users (Please notice I said "I would bet", not "I know").

Maybe RackSpace (or any other large operator..) can share some stats of the
user-agents they see in the wild?

Thanks,
Kiall (A, likely biased, dynamic language guy ;))
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] running HA cluster of guests within openstack

[Martin Gerhard Loschwitz]

Hi Ikke,

great work! :-)

Am 13.04.12 11:31, schrieb ikke:
> I likely am not the first one to ask this, but since I didn't find a
> thread about it I start one.
> 
> Is there any shared experience available what are the capabilities of
> OpenStack to run cluster of guests in the cloud? Do you have
> experience of the following questions, or links to more info? The
> questions relate to running a legacy HA cluster in virtual env, and
> moving it into cloud...
> 
> 1. Private networks between guests
> [...]
> 
> BR,
> 
>  Ilkka Tengvall
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

I think, as Major pointed out already, that the biggest problem right now
is that there is a certain lack of easy-to-use STONITH solutions to trigger
STONITH events from within virtual machines. I have something cooking here
using the latest version of Pacemaker; should this turn out to work, it
would make many things a lot easier. I'll elaborate a little bit more on
this once I have it working the way I want it.

Concerning the general subject of virtual machines (and clustered VMs for
that matter) within OpenStack, I think there is some stuff missing in Nova
that would be necessary (granted -- in one way or another, it would be
possible to make Pacemaker deal with VMs that have failed within Nova, but
in my eyes, that'd be crazy). Nova knows what VMs are supposed to be there
and Nova can find out which VMs are in fact running and which are not, so
I think Nova should make sure that those VMs that are supposed to run are,
well, running :)

Best regards
Martin

-- 
Martin Gerhard Loschwitz
Chief Brand Officer, Principal Consultant
hastexo Professional Services

CONFIDENTIALITY NOTICE: This e-mail and/or the accompanying documents
are privileged and confidential under applicable law. The person who
receives this message and who is not the addressee, one of his employees
or an agent entitled to hand it over to the addressee, is informed that
he may not use, disclose or reproduce the contents thereof. Should you
have received this e-mail (or any copy thereof) in error, please let us
know by telephone or e-mail without delay and delete the message from
your system. Thank you.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] running HA cluster of guests within openstack

[Jason Kölker]

On Fri, 2012-04-13 at 12:31 +0300, ikke wrote:

> 1. Private networks between guests
>   -> Doable now using Quantum
> 1.1. Defining VLANs visible to guest machines to separate clusters
> internal traffic,
>VLAN tags should not be stripped by host (QinQ)

VLANs and Quantum private networks are pretty much the same thing, why
would you want both?

> 1.2. Set pre-defined MAC addresses for the guests, needed by non-IP
>traffic within the guest cluster (layer2 addressing)
>   - will Melange do this, according to docs it's not in plans?

If you send the mac address to Melange when you create the interface it
will record it for that instance:

http://melange.readthedocs.org/en/latest/apidoc.html#interfaces

Happy Hacking!

7-11


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New Gerrit version (and server)

[Eoghan Glynn]

> We've just upgraded Gerrit to version 2.3.  There are a lot of
> changes
> behind the scenes that we've been looking forward to (like being able
> to
> store data in innodb rather than myisam tables for extra data
> longevity).  And there are a few visible changes that may be of
> interest
> to OpenStack developers.


Am I imagining things, or has a little bug crept into the way the
"Reviewer Verified Code-Review Approved" matrix is rendered?

i.e. if a reviewer +2's the patch but doesn't also approve it, 
this is now indicated with a green tick in the Code-Review column
and also a red x in Approved column.

Previously this would have been a green tick and a blank.

The new style seems a tad misleading, i.e. it suggests a
"Do not submit" on a first glance.

Cheers,
Eoghan

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Endpoints problems

[Pete Zaitcev]

On Fri, 13 Apr 2012 13:28:36 +
Guilherme Birk  wrote:

> keystone --token ADMIN 
> --publicurl="http://192.168.100.142:8774/v2/$(tenant_id)s"
> But I'm getting a "tenant_id: command not found".

Shell expands variables in the double quote (") strings, so you
have to use single quote (').

-- Pete

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] git review and the contributor agreement error message

[Francisco Souza]

Sure, I'll try to contact them.

Thanks,
Francisco

On Fri, Apr 13, 2012 at 10:47 AM, Kiall Mac Innes wrote:

> Always hitting send too early -_-
>
> Anyway - Its also possible the recent Gerrit upgrade has affected the
> launchpad CLA group sync.. Maybe ask the infrastructure guys in
> #openstack-infra on Freenode if you use IRC.. Otherwise I'm sure they will
> see this thread...
>
> Thanks,
> Kiall
>
>
>
> On Fri, Apr 13, 2012 at 2:45 PM, Kiall Mac Innes wrote:
>
>> I don't believe that is an issue - so long as you are using the right
>> launchpad ID to sign into Gerrit..
>>
>> I could be wrong though - it may match on LP username rather than the LP
>> user ID..
>>
>> Thanks,
>> Kiall
>>
>>
>>
>> On Fri, Apr 13, 2012 at 2:43 PM, Francisco Souza  wrote:
>>
>>> On Fri, Apr 13, 2012 at 10:35 AM, Kiall Mac Innes wrote:
>>>
 How long have you been inthe CLA group?

>>> 11 hours...
>>>
 There is a 15min or so lag between approval, and getting access.
 [...]
>>>
>>>
>>> One little detail that I didn't mention in the previous email: my gerrit
>>> username (fsouza) is not my Launchpad ID (francisco-franciscosouza), is
>>> this a problem?
>>>
>>> Thanks!
>>>
>>> --
>>> ~f
>>>
>>>
>>
>


-- 
~f
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New Gerrit version (and server)

[Sandy Walsh]

Sounds awesome! Looking forward to Draft Changes, much needed.

-S



From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net 
[openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of 
James E. Blair [cor...@inaugust.com]
Sent: Thursday, April 12, 2012 8:23 PM
To: OpenStack Mailing List
Subject: [Openstack] New Gerrit version (and server)

Hi,

We've just upgraded Gerrit to version 2.3.  There are a lot of changes
behind the scenes that we've been looking forward to (like being able to
store data in innodb rather than myisam tables for extra data
longevity).  And there are a few visible changes that may be of interest
to OpenStack developers.

One new addition in 2.3 is "draft changes".  The idea behind a draft
change in Gerrit is that it is a change that is not ready for merging,
or even general code review, but you would like to share it with some
people to get early comments.  If you upload a change as a draft, by
default, no one else can see it.  You must explicitly add each person
you would like to share it with as a reviewer.  Reviewers you add can
leave comments, but can not vote at this stage.  You can continue to
upload new patchsets to the change as it evolves, and once it is ready
for general review, you can click the "Publish" button.  It will then
become a normal change in Gerrit that everyone can see, including the
earlier reviews from the draft stage.  This is a one way transition;
once a draft is published, it can't be made a draft again.

If you're using git-review from source or the latest version from PyPI
(version 1.16, released today), you can easily upload a draft change by
adding the "-D" option (eg, "git review -D").  Earlier versions of
git-review also have the "-D" option, but the git ref that Gerrit uses
to indicate a change should be a draft was changed between the 2.3
release candidate and the final release; so if using "-D" results in an
error, you may need to upgrade.

You may notice some changes to the diff view.  Notably, the header which
contained all of the possible viewing options has been split up into
several parts; you can switch between them by selecting options that
show up under the menu at the top.  I recommend setting "Retain Header
On File Switch" under the Preferences section, as it is a nicer
experience when changing files.

Another notable new feature is the ability to add a group to the list of
reviewers for a change.  Just type in the name of the group and click
"Add Reviewer" and all of the individuals in the group will be added to
the list of reviewers (and will see the change on their "review
requests" list.

Finally, we've modified some of our local OpenStack style changes so
that it is easier for us to track upstream changes in layout.  It should
mean a little more consistency throughout the interface, though we
weren't able to keep the alternating row colors on the main table
without a disproportionate amount of effort.  Do note that you can click
on a line in a table, and it will be highlighted to improve legibility.

We've tried to give this as much testing as possible before moving it
into production.  If you encounter any issues, please let us know on IRC
(mtaylor, jeblair, LinuxJedi), via email at
, or you can file a bug at:

  https://bugs.launchpad.net/openstack-ci/

Thanks,

Jim

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Issue in KVM block migration

[heut2008]

Live block migration may need to be root.through I am not sure.a few day's
ago,I tested live block migration,and config hostA and  hostB to use ssh
key to access each other with nova  user; but it fails to migration.when I
run nova-compute with root ,and root ssh key to both hosts;the block
migration works well. then I config libvirt to use tls to remote access .
http://libvirt.org/remote.html.it also works well.

2012/4/13 Gurjar, Unmesh 

>  The user running the compute server is a member of libvirt group.
>
> I tried the migration via virsh, however that too returns the same result.
> I get similar logs in the qemu log file.
>
> ** **
>
> Thanks & Regards,
>
> *Unmesh Gurjar* | Lead Engineer | Vertex Software Private Ltd. | *w.* 
> +91.20.6604.1500
> x 379 | *m.* +91.982.324.7631 | unmesh.gur...@nttdata.com | Follow us on
> Twitter@NTTDATAAmericas**
>
> ** **
>
> *From:* Razique Mahroua [mailto:razique.mahr...@gmail.com]
> *Sent:* Friday, April 13, 2012 2:22 PM
>
> *To:* Gurjar, Unmesh
> *Cc:* openstack@lists.launchpad.net
> *Subject:* Re: [Openstack] Issue in KVM block migration
>
>  ** **
>
> Hi, 
>
> sorry for the late reply.
>
> Does the user nova belong to the libvirt group ?
>
> Can you try manually the migration via virsh ?
>
> $ virsh --migrate --live --copy-storage-all $domain
> qemu+tcp://user@server/sytem
>
> ** **
>
> thanks
>
> ** **
>
> ** **
>
> *Nuage & Co - Razique Mahroua *
>
> razique.mahr...@gmail.com
>
>
> 
>
> ** **
>
> Le 11 avr. 2012 à 16:44, Gurjar, Unmesh a écrit :
>
>
>
> 
>
> Thanks Razique for taking up this one.
>
>  
>
> Libvirt version on both Compute hosts:
>
> $ libvirtd --version
>
> libvirtd (libvirt) 0.9.2
>
> $ virsh --version
>
> 0.9.2
>
>  
>
> Here are my libvirtd.conf details:
>
> listen_tls = 0
>
> listen_tcp = 1
>
> unix_sock_group = "libvirtd"
>
> unix_sock_rw_perms = "0770"
>
> auth_unix_ro = "none"
>
> auth_unix_rw = "none"
>
> auth_tcp = "none"
>
>  
>
>  
>
> Thanks & Regards,
>
> *Unmesh Gurjar* | Lead Engineer | Vertex Software Private Ltd. | *w.* 
> +91.20.6604.1500
> x 379 | *m.* +91.982.324.7631 | unmesh.gur...@nttdata.com | Follow us on
> Twitter@NTTDATAAmericas
>
>  
>
> *From:* Razique Mahroua [mailto:razique.mahr...@gmail.com]
> *Sent:* Wednesday, April 11, 2012 7:33 PM
> *To:* Gurjar, Unmesh
> *Cc:* openstack@lists.launchpad.net
> *Subject:* Re: [Openstack] Issue in KVM block migration
>
>  
>
> Hi,
>
> it looks like the user under which libvirt is running doesn't have the
> rights to manage server's IF.
>
> What version of libvirt are you using ?
>
> Can I see the file /etc/libvirt/libvirtd.com ? (cat
> /etc/libvirt/libvirtd.conf | grep -v -e "#" -e "^$
>
>  
>
> Raz
>
>  
>
> *Nuage & Co - Razique Mahroua *
>
> razique.mahr...@gmail.com
>
>
> 
>
>  
>
> Le 11 avr. 2012 à 14:48, Gurjar, Unmesh a écrit :
>
>
>
>
> 
>
> Hi,
>
>  
>
> I have setup two Compute nodes (using Openstack master branch) and
> configured libvirt for block migration (by  following step #1 and #4
> mentioned 
> here).
> In addition, I have also disabled apparmor for libvirtd profile and have an
> entry in ‘/etc/hosts’ of both the Compute hosts.
>
>  
>
> From both the Compute hosts, I am able to connect and fetch the list of
> running instances on the other host (using the hostname), as follows:
>
> virsh # connect qemu+tcp://ubuntu-dev-001/system
>
> virsh # list
>
>  
>
> The issue is block migrating an instance between these hosts fails with
> the following error in the source Compute host console:
>
> libvir: QEMU error : operation failed: migration job: unexpectedly failed*
> ***
>
> 2012-04-11 03:15:54 DEBUG nova.rpc.amqp [-] Making asynchronous call on
> network ... from (pid=18487) multicall /opt/stack/nova/nova/rpc/amqp.py:318
> 
>
> 2012-04-11 03:15:54 DEBUG nova.rpc.amqp [-] MSG_ID is
> 8d91158236ae4de0bd8b89533f060892 from (pid=18487) multicall
> /opt/stack/nova/nova/rpc/amqp.py:321
>
> 2012-04-11 03:15:54 DEBUG nova.rpc.amqp [-] Making asynchronous cast on
> compute.ubuntu-dev-001... from (pid=18487) cast
> /opt/stack/nova/nova/rpc/amqp.py:343
>
> Traceback (most recent call last):
>
>   File "/usr/lib/python2.7/dist-packages/eventlet/hubs/poll.py", line 97,
> in wait
>
> readers.get(fileno, noop).cb(fileno)
>
>   File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line
> 192, in main
>
> result = function(*args, **kwargs)
>
>   File "/opt/stack/nova/nova/virt/libvirt/connection.py", line 2179, in
> _live_migration
>
> recover_method(ctxt, instance_ref, dest, block_migration)
>
>   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
>
> self.gen.n

Re: [Openstack] Endpoints problems

[David Kranz]

As far as my experience goes, you have to use %(tenant_id)s. I ran into 
this problem the first time I did it as well. $ makes the shell think 
it's a variable.


David Kranz
Quanta Research Cambridge



On 4/13/2012 9:28 AM, Guilherme Birk wrote:

I've tried to execute the following command:

keystone --token ADMIN --endpoint http://192.168.100.142:35357/v2.0 
endpoint-create --region RegionOne 
--service_id=1fd7b5f1add74aa4b6efc514fd153e72 
--publicurl="http://192.168.100.142:8774/v2/$(tenant_id)s" 
--adminurl="http://192.168.100.142:8774/v2/$(tenant_id)s" 
--internalurl="http://192.168.100.142:8774/v2/$(tenant_id)s"


But I'm getting a "tenant_id: command not found". When I list the 
endpoints all my url's are like "http://192.168.100.142:8774/v2/s"; for 
the created endpoint.

Am I doing something wrong ?

Thanks.


From: a...@openstack.org
Date: Thu, 12 Apr 2012 15:28:21 -0500
Subject: Re: [Openstack] Endpoints problems
To: guib...@hotmail.com
CC: openstack@lists.launchpad.net

Hi Guilherme -
Sorry you ran into a doc bug - 
https://bugs.launchpad.net/openstack-manuals/+bug/977905.


Basically, the bug states that the nova endpoint definition should be:

keystone --token 012345SECRET99TOKEN012345 --endpoint 
http://192.168.206.130:35357/v2.0 endpoint-create \


   --region RegionOne \
   --service_id=abc0f03c02904c24abdcc3b7910e2eed \
   --publicurl 
http://192.168.206.130:8774/v2/$(tenant_id)s 
 \
   --adminurl 
http://192.168.206.130:8774/v2/$(tenant_id)s 
 \
   --internalurl 
http://192.168.206.130:8774/v2/$(tenant_id)s 




I haven't fixed this yet because I'm not sure if the $(tenant_id)s is 
literal or which tenant_id specifically to use (the Service tenant for 
the adminurl possibly)?


If someone on the list could offer more input here and on the doc bug 
it would be greatly appreciated!

Anne

On Thu, Apr 12, 2012 at 12:25 PM, Guilherme Birk > wrote:


I'm having problems setting up the nova endpoint. I've followed
the manual

http://docs.openstack.org/trunk/openstack-compute/install/content/setting-up-tenants-users-and-roles.html,
putting the tenant id on the url's, like the manual says to do.
But when I try execute "nova list" I got a malformed url error.
When I set the endpoint without the tenant id on the url's I got a
404 error. Anyone having the same problem?

I can access the dashboard normally, but I'm unable to retrieve
instance list.

___
Mailing list: https://launchpad.net/~openstack

Post to : openstack@lists.launchpad.net

Unsubscribe : https://launchpad.net/~openstack

More help   : https://help.launchpad.net/ListHelp




___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Endpoints problems

[Dean Troyer]

On Fri, Apr 13, 2012 at 8:28 AM, Guilherme Birk  wrote:
> keystone --token ADMIN --endpoint http://192.168.100.142:35357/v2.0
> endpoint-create --region RegionOne
> --service_id=1fd7b5f1add74aa4b6efc514fd153e72
> --publicurl="http://192.168.100.142:8774/v2/$(tenant_id)s"
> --adminurl="http://192.168.100.142:8774/v2/$(tenant_id)s"
> --internalurl="http://192.168.100.142:8774/v2/$(tenant_id)s"
>
> But I'm getting a "tenant_id: command not found". When I list the endpoints
> all my url's are like "http://192.168.100.142:8774/v2/s"; for the created
> endpoint.
> Am I doing something wrong ?

You need to escape the '$'s in your URLs or change the double quotes
around them to single quotes.  The shell sees the $(tenant_id) and
attempts to execute a command called tenant_id.

dt

-- 

Dean Troyer
dtro...@gmail.com

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] git review and the contributor agreement error message

[Kiall Mac Innes]

Always hitting send too early -_-

Anyway - Its also possible the recent Gerrit upgrade has affected the
launchpad CLA group sync.. Maybe ask the infrastructure guys in
#openstack-infra on Freenode if you use IRC.. Otherwise I'm sure they will
see this thread...

Thanks,
Kiall


On Fri, Apr 13, 2012 at 2:45 PM, Kiall Mac Innes  wrote:

> I don't believe that is an issue - so long as you are using the right
> launchpad ID to sign into Gerrit..
>
> I could be wrong though - it may match on LP username rather than the LP
> user ID..
>
> Thanks,
> Kiall
>
>
>
> On Fri, Apr 13, 2012 at 2:43 PM, Francisco Souza  wrote:
>
>> On Fri, Apr 13, 2012 at 10:35 AM, Kiall Mac Innes wrote:
>>
>>> How long have you been inthe CLA group?
>>>
>> 11 hours...
>>
>>> There is a 15min or so lag between approval, and getting access.
>>> [...]
>>
>>
>> One little detail that I didn't mention in the previous email: my gerrit
>> username (fsouza) is not my Launchpad ID (francisco-franciscosouza), is
>> this a problem?
>>
>> Thanks!
>>
>> --
>> ~f
>>
>>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] Just JSON, and extensibility

[Mark Nottingham]

[ Full disclosure -- I'm using my personal address with Launchpad, etc., but I 
work for Rackspace. ]

On 12/04/2012, at 7:28 PM, Jorge Williams wrote:

> Generally, I agree with a lot of what you're saying, but I want to point out 
> a couple of things:
> 
> 1.  Static language folks gravitate to XML, not simply because they're 
> invested in it, but because it solves a real problem:
> […]
> and I should see those errors at compile time or while I'm authoring my code 
> if a have a good editor or IDEI shouldn't have to wait until my program 
> is run to catch those errors. 
> […]
> but at that point, there's very little benefit to having a static language, 
> because I don't have the assurances and guarantees that the language 
> provides. So I don't see a lot of Java folks dealing with JSON in that 
> manner.  Most devs will  need to build a class before hand.  So, you decrease 
> barriers for static language clients because there's a set of tools that can 
> extract the relevant info from XML schema languages and generate a set of 
> class files at compile time.   There's nothing saying you can't do something 
> similar with JSON, but those sort of tools aren't there  yet.

Great -- I think it's good to level set here. We're talking about supporting an 
entire serialisation, all of its supporting code, the QA cycle, schema 
authoring, etc., and the #1 benefit is making the programming experience for a 
subset of our audience more comfortable. 


> 2.  Then there's the issue of extensibilityespecially distributed 
> extensibility. XML has that notion built in

Heh; you mean bolted on with Namespaces in XML, and then completely messed up 
by XML Schema 1.0, then only partially fixed by 1.1. But yes.

> JSON has no concept of it...and we are building extensible APIs. There are no 
> standard way in JSON to introduce a  new property without guaranteeing that 
> there won't be  clash.  You've mention the need for namespaces in JSON 
> precisely to  deal with this sort of issue 
> (http://www.mnot.net/blog/2011/10/12/thinking_about_namespaces_in_json). 

First of all, as we've discussed many times, I don't think "extensibility" == 
"good" in all cases; if we allow/encourage too much extensibility, the platform 
we're building will fragment, and people won't see its full value. Extensions 
should be allowed where the make sense, not everywhere, and extensions should 
be encouraged to eventually converge.

> In the absence of a standard method, we're been using prefixes, which has 
> worked out well, but most JSON tools don't know how to deal with them

What does that *mean*? The prefix is an opaque part of the name -- what should 
a tool do with it?

> and they seem alien to folk that are used to using JSON day to day.

Perhaps, but doing something like RAX_FOO isn't that onerous. Or using a 
registry. Or just leaving it to the community to coordinate and document; try 
opening a shell, typing "set" and pondering how *that* name space is managed...

> This is a big deal because dynamic language folks are more likely to deal 
> with the JSON directly...Static language folks are generally not dealing with 
> XML in the same way.  In XML, the notion of extensibility is build into 
> parsers  and data binding tools directly.

Not really; it's a syntactic artefact of namespaces. I'd say that the tools 
manage it really, really badly, given how hard it is to author extensible 
schemas (and don't get me started on XML Schema).

> Most folks don't have to worry too much about it.  In fact extensible 
> protocols like XMPP and Atom Pub generally benefit from the extensibility 
> that's already  built in:  
> http://metajack.im/2010/02/01/json-versus-xml-not-as-simple-as-you-think/ 

You bring up some interesting examples; XMPP and Atom are both products of the 
fashion for XML in the IETF in the early part of the 00's… we've learned a LOT 
since then, and there's a clear move towards more no-nonsense JSON in the APPS 
area more recently. The extensibility of both XMPP and Atom turned out to not 
really need that much coordination. 

Is extensibility important? Sure, but we've learned a LOT about it since XML 
and PEP; it shouldn't be slathered over your protocols and formats like peanut 
butter, but instead only exposed where it's genuinely needed. Is that a 
judgement call? Absolutely. 

> Given that, if we're going to go the route of just picking one format, I 
> think the fact that our API is extensible means that we might want to ask 
> ourselves whether XML isn't a better fit :-)

I'm guessing you know I disagree with that by now.

> Having said all of that,  I realize that our devs are working in a dynamic 
> language, and don't see a lot of value in XML.  It's important to take that 
> into consideration, but we should also be asking whether there's value to our 
> current clients and potential clients.  Like it or not, there's a lot of 
> folks out there using static languages. 

Sure, but it's less cle

Re: [Openstack] git review and the contributor agreement error message

[Kiall Mac Innes]

I don't believe that is an issue - so long as you are using the right
launchpad ID to sign into Gerrit..

I could be wrong though - it may match on LP username rather than the LP
user ID..

Thanks,
Kiall


On Fri, Apr 13, 2012 at 2:43 PM, Francisco Souza  wrote:

> On Fri, Apr 13, 2012 at 10:35 AM, Kiall Mac Innes wrote:
>
>> How long have you been inthe CLA group?
>>
> 11 hours...
>
>> There is a 15min or so lag between approval, and getting access.
>> [...]
>
>
> One little detail that I didn't mention in the previous email: my gerrit
> username (fsouza) is not my Launchpad ID (francisco-franciscosouza), is
> this a problem?
>
> Thanks!
>
> --
> ~f
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Endpoints problems

[Guilherme Birk]

Thanks Kiall, that worked. Looks like my endpoints are working now, I can 
execute the nova list command without problems.
Thanks all.

From: ki...@managedit.ie
Date: Fri, 13 Apr 2012 14:40:59 +0100
Subject: Re: [Openstack] Endpoints problems
To: guib...@hotmail.com
CC: openstack@lists.launchpad.net

Use single quotes, instead of double quotes.
$() has a special meaning in bash.. it executes whatever is between the braces 
and substitutes the whole thing for the STDOUT of the command it ran...


Thanks,
Kiall



On Fri, Apr 13, 2012 at 2:28 PM, Guilherme Birk  wrote:






I've tried to execute the following command:

keystone --token ADMIN --endpoint http://192.168.100.142:35357/v2.0 
endpoint-create --region RegionOne 
--service_id=1fd7b5f1add74aa4b6efc514fd153e72 
--publicurl="http://192.168.100.142:8774/v2/$(tenant_id)s" 
--adminurl="http://192.168.100.142:8774/v2/$(tenant_id)s" 
--internalurl="http://192.168.100.142:8774/v2/$(tenant_id)s"



But I'm getting a "tenant_id: command not found". When I list the endpoints all 
my url's are like "http://192.168.100.142:8774/v2/s"; for the created endpoint.


Am I doing something wrong ?

Thanks.

From: a...@openstack.org
Date: Thu, 12 Apr 2012 15:28:21 -0500
Subject: Re: [Openstack] Endpoints problems


To: guib...@hotmail.com
CC: openstack@lists.launchpad.net

Hi Guilherme - 


Sorry you ran into a doc bug - 
https://bugs.launchpad.net/openstack-manuals/+bug/977905. 

Basically, the bug states that the nova endpoint definition should be:






keystone --token 012345SECRET99TOKEN012345 --endpoint 
http://192.168.206.130:35357/v2.0 endpoint-create \

   --region RegionOne \

   --service_id=abc0f03c02904c24abdcc3b7910e2eed \

   --publicurl http://192.168.206.130:8774/v2/$(tenant_id)s 
\

   --adminurl http://192.168.206.130:8774/v2/$(tenant_id)s \

   --internalurl 
http://192.168.206.130:8774/v2/$(tenant_id)s
I haven't fixed this yet because I'm not sure if the $(tenant_id)s is literal 
or which tenant_id specifically to use (the Service tenant for the adminurl 
possibly)?





If someone on the list could offer more input here and on the doc bug it would 
be greatly appreciated!
Anne

On Thu, Apr 12, 2012 at 12:25 PM, Guilherme Birk  wrote:








I'm having problems setting up the nova endpoint. I've followed the manual 
http://docs.openstack.org/trunk/openstack-compute/install/content/setting-up-tenants-users-and-roles.html,
 putting the tenant id on the url's, like the manual says to do. But when I try 
execute "nova list" I got a malformed url error. When I set the endpoint 
without the tenant id on the url's I got a 404 error. Anyone having the same 
problem?





I can access the dashboard normally, but I'm unable to retrieve instance list.  
  

___

Mailing list: https://launchpad.net/~openstack

Post to : openstack@lists.launchpad.net

Unsubscribe : https://launchpad.net/~openstack

More help   : https://help.launchpad.net/ListHelp



  

___

Mailing list: https://launchpad.net/~openstack

Post to : openstack@lists.launchpad.net

Unsubscribe : https://launchpad.net/~openstack

More help   : https://help.launchpad.net/ListHelp



  ___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] git review and the contributor agreement error message

[Francisco Souza]

On Fri, Apr 13, 2012 at 10:35 AM, Kiall Mac Innes wrote:

> How long have you been inthe CLA group?
>
11 hours...

> There is a 15min or so lag between approval, and getting access.
> [...]


One little detail that I didn't mention in the previous email: my gerrit
username (fsouza) is not my Launchpad ID (francisco-franciscosouza), is
this a problem?

Thanks!

-- 
~f
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Endpoints problems

[Kiall Mac Innes]

Use single quotes, instead of double quotes.

$() has a special meaning in bash.. it executes whatever is between the
braces and substitutes the whole thing for the STDOUT of the command it
ran...

Thanks,
Kiall


On Fri, Apr 13, 2012 at 2:28 PM, Guilherme Birk  wrote:

>  I've tried to execute the following command:
>
> keystone --token ADMIN --endpoint 
> http://192.168.100.142:35357/v2.0endpoint-create --region RegionOne
> --service_id=1fd7b5f1add74aa4b6efc514fd153e72 --publicurl="
> http://192.168.100.142:8774/v2/$(tenant_id)s" --adminurl="
> http://192.168.100.142:8774/v2/$(tenant_id)s" --internalurl="
> http://192.168.100.142:8774/v2/$(tenant_id)s"
>
> But I'm getting a "tenant_id: command not found". When I list the
> endpoints all my url's are like "http://192.168.100.142:8774/v2/s"; for
> the created endpoint.
> Am I doing something wrong ?
>
> Thanks.
>
> --
> From: a...@openstack.org
> Date: Thu, 12 Apr 2012 15:28:21 -0500
>
> Subject: Re: [Openstack] Endpoints problems
> To: guib...@hotmail.com
> CC: openstack@lists.launchpad.net
>
>
> Hi Guilherme -
> Sorry you ran into a doc bug -
> https://bugs.launchpad.net/openstack-manuals/+bug/977905.
>
> Basically, the bug states that the nova endpoint definition should be:
>
> keystone --token 012345SECRET99TOKEN012345 --endpoint
> http://192.168.206.130:35357/v2.0 endpoint-create \
>
>--region RegionOne \
>--service_id=abc0f03c02904c24abdcc3b7910e2eed \
>--publicurl
> http://192.168.206.130:8774/v2/$(tenant_id)s \
>--adminurl
> http://192.168.206.130:8774/v2/$(tenant_id)s \
>--internalurl
> http://192.168.206.130:8774/v2/$(tenant_id)s
>
> I haven't fixed this yet because I'm not sure if the $(tenant_id)s is
> literal or which tenant_id specifically to use (the Service tenant for the
> adminurl possibly)?
>
> If someone on the list could offer more input here and on the doc bug it
> would be greatly appreciated!
> Anne
>
> On Thu, Apr 12, 2012 at 12:25 PM, Guilherme Birk wrote:
>
>  I'm having problems setting up the nova endpoint. I've followed the
> manual
> http://docs.openstack.org/trunk/openstack-compute/install/content/setting-up-tenants-users-and-roles.html,
> putting the tenant id on the url's, like the manual says to do. But when I
> try execute "nova list" I got a malformed url error. When I set the
> endpoint without the tenant id on the url's I got a 404 error. Anyone
> having the same problem?
>
> I can access the dashboard normally, but I'm unable to retrieve instance
> list.
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] git review and the contributor agreement error message

[Kiall Mac Innes]

How long have you been inthe CLA group?

There is a 15min or so lag between approval, and getting access.

Thanks,
Kiall

Sent from my phone.
On Apr 13, 2012 2:24 p.m., "Francisco Souza"  wrote:

> Hey there,
> I'm trying to git review some changes, but I keep getting the "A
> Contributor Agreement must be completed before uploading" error.
>
> I've already signed in it [1], my name is in the contributors wiki entry
> [2] and I'm member of the openstack-cla group [3]. Am I missing something?
>
> [1] https://rackspace.echosign.com/public/audit?tx=LPXXJ54L6P797D
> [2] http://wiki.openstack.org/Contributors
> [3] http://launchpad.net/~openstack-cla
>
> Thanks,
> Francisco
>
> --
> ~f
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Endpoints problems

[Guilherme Birk]

I've tried to execute the following command:

keystone --token ADMIN --endpoint http://192.168.100.142:35357/v2.0 
endpoint-create --region RegionOne 
--service_id=1fd7b5f1add74aa4b6efc514fd153e72 
--publicurl="http://192.168.100.142:8774/v2/$(tenant_id)s" 
--adminurl="http://192.168.100.142:8774/v2/$(tenant_id)s" 
--internalurl="http://192.168.100.142:8774/v2/$(tenant_id)s"

But I'm getting a "tenant_id: command not found". When I list the endpoints all 
my url's are like "http://192.168.100.142:8774/v2/s"; for the created endpoint.
Am I doing something wrong ?

Thanks.

From: a...@openstack.org
Date: Thu, 12 Apr 2012 15:28:21 -0500
Subject: Re: [Openstack] Endpoints problems
To: guib...@hotmail.com
CC: openstack@lists.launchpad.net

Hi Guilherme - 
Sorry you ran into a doc bug - 
https://bugs.launchpad.net/openstack-manuals/+bug/977905. 

Basically, the bug states that the nova endpoint definition should be:




keystone --token 012345SECRET99TOKEN012345 --endpoint 
http://192.168.206.130:35357/v2.0 endpoint-create \

   --region RegionOne \

   --service_id=abc0f03c02904c24abdcc3b7910e2eed \

   --publicurl http://192.168.206.130:8774/v2/$(tenant_id)s 
\

   --adminurl http://192.168.206.130:8774/v2/$(tenant_id)s \

   --internalurl 
http://192.168.206.130:8774/v2/$(tenant_id)s
I haven't fixed this yet because I'm not sure if the $(tenant_id)s is literal 
or which tenant_id specifically to use (the Service tenant for the adminurl 
possibly)?



If someone on the list could offer more input here and on the doc bug it would 
be greatly appreciated!
Anne

On Thu, Apr 12, 2012 at 12:25 PM, Guilherme Birk  wrote:






I'm having problems setting up the nova endpoint. I've followed the manual 
http://docs.openstack.org/trunk/openstack-compute/install/content/setting-up-tenants-users-and-roles.html,
 putting the tenant id on the url's, like the manual says to do. But when I try 
execute "nova list" I got a malformed url error. When I set the endpoint 
without the tenant id on the url's I got a 404 error. Anyone having the same 
problem?



I can access the dashboard normally, but I'm unable to retrieve instance list.  
  

___

Mailing list: https://launchpad.net/~openstack

Post to : openstack@lists.launchpad.net

Unsubscribe : https://launchpad.net/~openstack

More help   : https://help.launchpad.net/ListHelp



  ___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] git review and the contributor agreement error message

[Francisco Souza]

Hey there,
I'm trying to git review some changes, but I keep getting the "A
Contributor Agreement must be completed before uploading" error.

I've already signed in it [1], my name is in the contributors wiki entry
[2] and I'm member of the openstack-cla group [3]. Am I missing something?

[1] https://rackspace.echosign.com/public/audit?tx=LPXXJ54L6P797D
[2] http://wiki.openstack.org/Contributors
[3] http://launchpad.net/~openstack-cla

Thanks,
Francisco

-- 
~f
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Issue in KVM block migration

[Gurjar, Unmesh]

The user running the compute server is a member of libvirt group.
I tried the migration via virsh, however that too returns the same result. I 
get similar logs in the qemu log file.

Thanks & Regards,
Unmesh Gurjar | Lead Engineer | Vertex Software Private Ltd. | w. 
+91.20.6604.1500 x 379 | m. +91.982.324.7631 | 
unmesh.gur...@nttdata.com | Follow us on 
Twitter@NTTDATAAmericas

From: Razique Mahroua [mailto:razique.mahr...@gmail.com]
Sent: Friday, April 13, 2012 2:22 PM
To: Gurjar, Unmesh
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] Issue in KVM block migration

Hi,
sorry for the late reply.
Does the user nova belong to the libvirt group ?
Can you try manually the migration via virsh ?
$ virsh --migrate --live --copy-storage-all $domain qemu+tcp://user@server/sytem

thanks


Nuage & Co - Razique Mahroua
razique.mahr...@gmail.com

[cid:image001.jpg@01CD199E.79A758F0]

Le 11 avr. 2012 à 16:44, Gurjar, Unmesh a écrit :


Thanks Razique for taking up this one.

Libvirt version on both Compute hosts:
$ libvirtd --version
libvirtd (libvirt) 0.9.2
$ virsh --version
0.9.2

Here are my libvirtd.conf details:
listen_tls = 0
listen_tcp = 1
unix_sock_group = "libvirtd"
unix_sock_rw_perms = "0770"
auth_unix_ro = "none"
auth_unix_rw = "none"
auth_tcp = "none"


Thanks & Regards,
Unmesh Gurjar | Lead Engineer | Vertex Software Private Ltd. | w. 
+91.20.6604.1500 x 379 | m. +91.982.324.7631 | 
unmesh.gur...@nttdata.com | Follow us on 
Twitter@NTTDATAAmericas

From: Razique Mahroua 
[mailto:razique.mahr...@gmail.com]
Sent: Wednesday, April 11, 2012 7:33 PM
To: Gurjar, Unmesh
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] Issue in KVM block migration

Hi,
it looks like the user under which libvirt is running doesn't have the rights 
to manage server's IF.
What version of libvirt are you using ?
Can I see the file /etc/libvirt/libvirtd.com ? (cat /etc/libvirt/libvirtd.conf 
| grep -v -e "#" -e "^$

Raz

Nuage & Co - Razique Mahroua
razique.mahr...@gmail.com



Le 11 avr. 2012 à 14:48, Gurjar, Unmesh a écrit :



Hi,

I have setup two Compute nodes (using Openstack master branch) and configured 
libvirt for block migration (by  following step #1 and #4 mentioned 
here).
 In addition, I have also disabled apparmor for libvirtd profile and have an 
entry in '/etc/hosts' of both the Compute hosts.

>From both the Compute hosts, I am able to connect and fetch the list of 
>running instances on the other host (using the hostname), as follows:
virsh # connect qemu+tcp://ubuntu-dev-001/system
virsh # list

The issue is block migrating an instance between these hosts fails with the 
following error in the source Compute host console:
libvir: QEMU error : operation failed: migration job: unexpectedly failed
2012-04-11 03:15:54 DEBUG nova.rpc.amqp [-] Making asynchronous call on network 
... from (pid=18487) multicall /opt/stack/nova/nova/rpc/amqp.py:318
2012-04-11 03:15:54 DEBUG nova.rpc.amqp [-] MSG_ID is 
8d91158236ae4de0bd8b89533f060892 from (pid=18487) multicall 
/opt/stack/nova/nova/rpc/amqp.py:321
2012-04-11 03:15:54 DEBUG nova.rpc.amqp [-] Making asynchronous cast on 
compute.ubuntu-dev-001... from (pid=18487) cast 
/opt/stack/nova/nova/rpc/amqp.py:343
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/eventlet/hubs/poll.py", line 97, in 
wait
readers.get(fileno, noop).cb(fileno)
  File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 192, in 
main
result = function(*args, **kwargs)
  File "/opt/stack/nova/nova/virt/libvirt/connection.py", line 2179, in 
_live_migration
recover_method(ctxt, instance_ref, dest, block_migration)
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
self.gen.next()
  File "/opt/stack/nova/nova/virt/libvirt/connection.py", line 2175, in 
_live_migration
FLAGS.live_migration_bandwidth)
  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 689, in migrateToURI
if ret == -1: raise libvirtError ('virDomainMigrateToURI() failed', 
dom=self)
libvirtError: operation failed: migration job: unexpectedly failed
Removing descriptor: 12

I find the following in the qemu log file 
(/var/log/libvirt/qemu/instance-0003.log ) on the destination Compute host:
2012-04-11 04:14:25.971: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin 
QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-0.14 -enable-kvm -m 512 -smp 
1,sockets=1,cores=1,threads=1 -name instance-0003 -uuid 
f58d5f32-6d55-43fb-89ed-c33ebf72d1ed -nodefconfig -nodefaults -chardev 
socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0003.monitor,server,nowait
 -mon chardev=charmonitor,id=monitor

Re: [Openstack] [Swift] Does there any exist blueprint or sub-project of user's storage space quota or counting method for Swift ?

[Dmitry Ukov]

Please explain, how did you measure replication time?

On Fri, Apr 13, 2012 at 7:23 AM, Alex Yang  wrote:

> In my view, the biggest problem of swift is not the new features but the
> improvement of performance.
>
> At first, we knew that container-updater, *-auditor, *-replicator will
> loop all the files in the disk during the running interval. When the number
> of files is very large, the process of replicator, updater and auditor will
> spend much time. So, the time of eventual consistency is very long.
>
> Our practice of  Sina Web Service Team https://launchpad.net/~sws:
>
> total accounts:  121,961;
>
> total containers:160,703;
>
> total objects: 14,291,519;
>
> total storage usage:  1.3T
>
> account replication time:10 hours;
>
> container replication time:  10 hours;
>
> object replication time:   48 hours;
>
> account audit time:   2 hours;
>
> container audit time: 9 hours;
>
> container update time:19 hours;
>
> This is terrible If we develop quota upon account db. There is long time
> for eventual consistency.
>
> Secondly,  there is a vicious circle during replication. The replicator
> will query the account-server, container-server and object-server to
> compare the metadata, and determine whether to sync. When the number of
> files is very large, the frequent query make the account-server,
> container-server and object-server become a bottle neck. This will
> influence the process of proxy-server to work with back-end servers. There
> a lot of Timeout(10s) ERROR in proxy-server's log and the load-average is
> very high.
> So, some PUT, POST operations failed, and the replicator to sync, and fail
> more, and sync more...
> .
> In my opinion, we need to improve the process of replication and container
> update by using event drive framework or something else..
> My leader may discuss this topic at Design Summit,
> http://openstackconferencespring2012.sched.org/speaker/huicheng
>
>
> 2012/4/13 John Dickinson 
>
>> Swift keeps total bytes, container, and object count (eventually)
>> up-to-date in the account metadata. There are also log processing tools
>> (like slogging - http://github.com/notmyname/slogging) that can provide
>> usage information (including bandwidth) based on swift logs.
>>
>> While I think that it's appropriate for swift to generate the usage
>> information (via internal processes or log processing), the appropriate
>> place for quotas is in whatever system handles the concept of a user
>> (normally the auth system). This way quotas are enforced by revoking or
>> limiting access of the auth token.
>>
>> --John
>>
>>
>> On Apr 12, 2012, at 11:53 AM, Frederik Van Hecke wrote:
>>
>> > Hi Kuo,
>> >
>> > One option would be to keep the usage information (num files, num
>> bytes, etc) per container / account in an sqlite DB, just like it is done
>> for account and container info.
>> >
>> > To avoid having to loop through all data at regular intervals (to
>> update the info), additional logic could be added to the api methods to
>> update the sqlite DB's when new files are added, files are deleted, etc.
>> Such approach will require more lines of code, but will be far less
>> stressful on performance.
>> >
>> > (the brute-force approach to loop through it at regular intervals will
>> be hell on performance on large deployments..)
>> >
>> >
>> > For data transfer billing based on download / upload amounts, a similar
>> approach could be used.
>> >
>> > If no one else is looking into this, I would certainly be willing to
>> help to help get this started.
>> >
>> >
>> > Kind regards,
>> > Frederik Van Hecke
>> >
>> > T:  +32487733713
>> > E:  frede...@cluttr.be
>> > W: www.cluttr.be
>> >
>> >
>> >
>> >
>> >
>> > This e-mail and any attachments thereto may contain information which
>> is confidential and/or protected by intellectual property rights and are
>> intended for the sole use of the recipient(s)named above. Any use of the
>> information contained herein (including, but not limited to, total or
>> partial reproduction, communication or distribution in any form) by persons
>> other than the designated recipient(s) is prohibited. If you have received
>> this e-mail in error, please notify the sender either by telephone or by
>> e-mail and delete the material from any computer. Thank you for your
>> cooperation.
>> >
>> >
>> >
>> > On Thu, Apr 12, 2012 at 17:45, Kuo Hugo  wrote:
>> > Hi folks ,
>> >
>> > I'm thinking about the better approach to manage "an user" or "an
>> account" space usage quota in swift.
>> > Is  there any related blueprint or sub-project even an idea around ?
>> > Any suggestion of benefits to be an external service or to be a
>> middle-ware in swift-proxy ?
>> >
>> > I'm concerning about such feature will reduce the performance of entire
>> Swift environment.
>> >
>> > Appreciate :>
>> >
>> >
>> >
>> > --
>> > +Hugo Kuo+
>> > tonyt...@gmail.com
>> > +886 935004793
>> >
>> >
>> > ___

[Openstack] Multiple nova-networks with QuantumManager (Was : Role of "nova-manage network" commands when using QuantumManager)

[Vaze, Mandar]

Dan,

On similar lines - Currently if nova-network processes are running on two 
nodes, only way to set specific network on specific nova-network node is to 
execute "nova-manage network create" on respective nova-network node. Is that 
correct ?

How is the above setup  related to "multi-host" ? I found a code comment that 
"Quantum Manager doesn't support multi-host"

I'm copying Vish, as he seems to be expert on multi-host - Reading  
http://docs.openstack.org/trunk/openstack-compute/admin/content/existing-ha-networking-options.html
 , it looks like above is Option 2/Multi-Nic, but not really sure)  If yes, how 
is multi-nic supported by Quantum ?

Additionally - in the above document Vish suggested that each nova-compute must 
run nova-network - Is that why with QuantumManager - nova-manage executes 
networking commands like iptables, dnsmasq locally ?


Thanks !!
-Mandar


From: Dan Wendlandt [mailto:d...@nicira.com]
Sent: Thursday, April 12, 2012 9:44 PM
To: Vaze, Mandar
Cc: openstack@lists.launchpad.net; netst...@lists.launchpad.net
Subject: Re: [Netstack] Role of "nova-manage network" commands when using 
QuantumManager

Hi Mandar,

Thanks for bringing this up.  For Essex, nova-manage commands to create/delete 
Quantum networks must be run on the nova-network node.  For Folsom this will 
all change, as all networks will be created directly against the Quantum API, 
rather than using nova-manage.  I'll add a note to the administrator guide, as 
this is not called there.  Thanks!

Dan


On Thu, Apr 12, 2012 at 1:38 AM, Vaze, Mandar 
mailto:mandar.v...@nttdata.com>> wrote:
It is my understanding that in multi-node setup :

* nova-manage can be executed from any machine which may not be running 
nova-network process.  (Or should nova-manage always be run on nova-network 
node ?)

* nova-manage does DB operations and delegates the actual networking 
calls to nova-network process ?

Is this understanding correct ?

I traced "nova-manage network create" and nova-manage network delete" using 
FlatDHCPManager (default for devstack/stack.sh)
Both these calls seem to be doing only DB operations.
BTW, nova-network process was shutdown during both "network create" and 
"network delete" - Still both operations were successful.

But when using QuantumManager as network manager - nova-manage seems to be 
doing networking operations like "iptables-save" (during network create) and 
"kill_dhcp" (during network delete)
(via linux-net L3 driver)

Since nova-manage command may be executed on a host which isn't running 
nova-network - network commands like "iptables" and "kill -9 " 
on host running nova-manage seems incorrect.

For the first scenario (iptables-save during "network create" - there is 
already a defect in LP : https://bugs.launchpad.net/nova/+bug/977738
and review : https://review.openstack.org/6451

I would like your comments and opinions which would help me understand  "What 
nova-manage should and should NOT do"

Thanks,
-Mandar





__
Disclaimer:This email and any attachments are sent in strictest confidence for 
the sole use of the addressee and may contain legally privileged, confidential, 
and proprietary data. If you are not the intended recipient, please advise the 
sender by replying promptly to this email and then delete and destroy this 
email and any attachments without any further use, copying or forwarding

--
Mailing list: https://launchpad.net/~netstack
Post to : netst...@lists.launchpad.net
Unsubscribe : https://launchpad.net/~netstack
More help   : https://help.launchpad.net/ListHelp



--
~~~
Dan Wendlandt
Nicira, Inc: www.nicira.com
twitter: danwendlandt
~~~


__
Disclaimer:This email and any attachments are sent in strictest confidence for 
the sole use of the addressee and may contain legally privileged, confidential, 
and proprietary data.  If you are not the intended recipient, please advise the 
sender by replying promptly to this email and then delete and destroy this 
email and any attachments without any further use, copying or forwarding___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Swift] Does there any exist blueprint or sub-project of user's storage space quota or counting method for Swift ?

[Oleg Gelbukh]

Alex,

Thank you for important point and interesting information on large-scale
Swift performance!

Can you please explain a little what these times stand for? Is this a
single process runtime, or the time needed to converge cluster in case of
device failure, or something else?

--
Best regards,
Oleg

On Fri, Apr 13, 2012 at 7:23 AM, Alex Yang  wrote:

> In my view, the biggest problem of swift is not the new features but the
> improvement of performance.
>
> At first, we knew that container-updater, *-auditor, *-replicator will
> loop all the files in the disk during the running interval. When the number
> of files is very large, the process of replicator, updater and auditor will
> spend much time. So, the time of eventual consistency is very long.
>
> Our practice of  Sina Web Service Team https://launchpad.net/~sws:
>
> total accounts:  121,961;
>
> total containers:160,703;
>
> total objects: 14,291,519;
>
> total storage usage:  1.3T
>
> account replication time:10 hours;
>
> container replication time:  10 hours;
>
> object replication time:   48 hours;
>
> account audit time:   2 hours;
>
> container audit time: 9 hours;
>
> container update time:19 hours;
>
> This is terrible If we develop quota upon account db. There is long time
> for eventual consistency.
>
> Secondly,  there is a vicious circle during replication. The replicator
> will query the account-server, container-server and object-server to
> compare the metadata, and determine whether to sync. When the number of
> files is very large, the frequent query make the account-server,
> container-server and object-server become a bottle neck. This will
> influence the process of proxy-server to work with back-end servers. There
> a lot of Timeout(10s) ERROR in proxy-server's log and the load-average is
> very high.
> So, some PUT, POST operations failed, and the replicator to sync, and fail
> more, and sync more...
> .
> In my opinion, we need to improve the process of replication and container
> update by using event drive framework or something else..
> My leader may discuss this topic at Design Summit,
> http://openstackconferencespring2012.sched.org/speaker/huicheng
>
>
> 2012/4/13 John Dickinson 
>
>> Swift keeps total bytes, container, and object count (eventually)
>> up-to-date in the account metadata. There are also log processing tools
>> (like slogging - http://github.com/notmyname/slogging) that can provide
>> usage information (including bandwidth) based on swift logs.
>>
>> While I think that it's appropriate for swift to generate the usage
>> information (via internal processes or log processing), the appropriate
>> place for quotas is in whatever system handles the concept of a user
>> (normally the auth system). This way quotas are enforced by revoking or
>> limiting access of the auth token.
>>
>> --John
>>
>>
>> On Apr 12, 2012, at 11:53 AM, Frederik Van Hecke wrote:
>>
>> > Hi Kuo,
>> >
>> > One option would be to keep the usage information (num files, num
>> bytes, etc) per container / account in an sqlite DB, just like it is done
>> for account and container info.
>> >
>> > To avoid having to loop through all data at regular intervals (to
>> update the info), additional logic could be added to the api methods to
>> update the sqlite DB's when new files are added, files are deleted, etc.
>> Such approach will require more lines of code, but will be far less
>> stressful on performance.
>> >
>> > (the brute-force approach to loop through it at regular intervals will
>> be hell on performance on large deployments..)
>> >
>> >
>> > For data transfer billing based on download / upload amounts, a similar
>> approach could be used.
>> >
>> > If no one else is looking into this, I would certainly be willing to
>> help to help get this started.
>> >
>> >
>> > Kind regards,
>> > Frederik Van Hecke
>> >
>> > T:  +32487733713
>> > E:  frede...@cluttr.be
>> > W: www.cluttr.be
>> >
>> >
>> >
>> >
>> >
>> > This e-mail and any attachments thereto may contain information which
>> is confidential and/or protected by intellectual property rights and are
>> intended for the sole use of the recipient(s)named above. Any use of the
>> information contained herein (including, but not limited to, total or
>> partial reproduction, communication or distribution in any form) by persons
>> other than the designated recipient(s) is prohibited. If you have received
>> this e-mail in error, please notify the sender either by telephone or by
>> e-mail and delete the material from any computer. Thank you for your
>> cooperation.
>> >
>> >
>> >
>> > On Thu, Apr 12, 2012 at 17:45, Kuo Hugo  wrote:
>> > Hi folks ,
>> >
>> > I'm thinking about the better approach to manage "an user" or "an
>> account" space usage quota in swift.
>> > Is  there any related blueprint or sub-project even an idea around ?
>> > Any suggestion of benefits to be an external service or to be a
>> middle-ware in swi

Re: [Openstack] running HA cluster of guests within openstack

[Major Hayden]

On Apr 13, 2012, at 4:31 AM, ikke wrote:

> 2.5. virtual HW manager (guest isolation)
>  - Any plans to integrate a piece from which a state of guest could
>be reliably queried, e.g. guaranteeing that if I ask to power
> off another
>guest, it get's done in given time (millisecs), and not
> pending on e.g. some tcp
>timeout, and thus leading to split brain case of running two
> similar guest
>simultaneously. E.g. starting another guest to replace shut
> down one, but
>due some communications error the first one didn't really shut
> before the
>new one is already up.
> - should be able to reliably cut down the guests network and disk access 
> to
>   guarantee the above case

This would be a huge win for clustering.

Having a reliable and immediate STONITH capability within a virtual environment 
would be really handy for environments which have sensitive needs for shared 
storage (whether it's remote iscsi storage or DRBD).  It would be relatively 
trivia to assemble a fencing daemon to make requests to the API to hard reboot 
a misbehaving member of a cluster.

Good points!

-- 
Major Hayden
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] running HA cluster of guests within openstack

[Pádraig Brady]

On 04/13/2012 10:31 AM, ikke wrote:
> I likely am not the first one to ask this, but since I didn't find a
> thread about it I start one.
> 
> Is there any shared experience available what are the capabilities of
> OpenStack to run cluster of guests in the cloud? Do you have
> experience of the following questions, or links to more info? The
> questions relate to running a legacy HA cluster in virtual env, and
> moving it into cloud...

I'll just point out two early stage projects
that used in combination can provide a HA solution.

http://wiki.openstack.org/Heat
http://wiki.openstack.org/ResourceMonitorAlertsandNotifications

These are similar to AWS CloudFormations and CloudWatch respectively.

cheers,
Pádraig.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New Gerrit version (and server)

[Kiall Mac Innes]

The new gerrit version also supports per user namespaces, if enabled.

These allow users to create private branches with full push privileges etc..

Have these been enabled?

Thanks,
Kiall

Sent from my phone.
On Apr 13, 2012 12:33 p.m., "Thierry Carrez"  wrote:

> Mark McLoughlin wrote:
> >> One new addition in 2.3 is "draft changes".  The idea behind a draft
> >> change in Gerrit is that it is a change that is not ready for merging,
> >> or even general code review, but you would like to share it with some
> >> people to get early comments.  If you upload a change as a draft, by
> >> default, no one else can see it.  You must explicitly add each person
> >> you would like to share it with as a reviewer.  Reviewers you add can
> >> leave comments, but can not vote at this stage.  You can continue to
> >> upload new patchsets to the change as it evolves, and once it is ready
> >> for general review, you can click the "Publish" button.  It will then
> >> become a normal change in Gerrit that everyone can see, including the
> >> earlier reviews from the draft stage.  This is a one way transition;
> >> once a draft is published, it can't be made a draft again.
> >
> > This sounds cool. Will the vulnerability management team use this for
> > embargoed security fixes?
>
> I'll definitely look into that possibility... Depends a bit on the
> security model around drafts (not being listed is not the same as being
> private).
>
> --
> Thierry Carrez (ttx)
> Release Manager, OpenStack
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New Gerrit version (and server)

[Thierry Carrez]

Mark McLoughlin wrote:
>> One new addition in 2.3 is "draft changes".  The idea behind a draft
>> change in Gerrit is that it is a change that is not ready for merging,
>> or even general code review, but you would like to share it with some
>> people to get early comments.  If you upload a change as a draft, by
>> default, no one else can see it.  You must explicitly add each person
>> you would like to share it with as a reviewer.  Reviewers you add can
>> leave comments, but can not vote at this stage.  You can continue to
>> upload new patchsets to the change as it evolves, and once it is ready
>> for general review, you can click the "Publish" button.  It will then
>> become a normal change in Gerrit that everyone can see, including the
>> earlier reviews from the draft stage.  This is a one way transition;
>> once a draft is published, it can't be made a draft again.
> 
> This sounds cool. Will the vulnerability management team use this for
> embargoed security fixes?

I'll definitely look into that possibility... Depends a bit on the
security model around drafts (not being listed is not the same as being
private).

-- 
Thierry Carrez (ttx)
Release Manager, OpenStack

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New Gerrit version (and server)

[Mark McLoughlin]

Hey,

On Thu, 2012-04-12 at 16:23 -0700, James E. Blair wrote:

> One new addition in 2.3 is "draft changes".  The idea behind a draft
> change in Gerrit is that it is a change that is not ready for merging,
> or even general code review, but you would like to share it with some
> people to get early comments.  If you upload a change as a draft, by
> default, no one else can see it.  You must explicitly add each person
> you would like to share it with as a reviewer.  Reviewers you add can
> leave comments, but can not vote at this stage.  You can continue to
> upload new patchsets to the change as it evolves, and once it is ready
> for general review, you can click the "Publish" button.  It will then
> become a normal change in Gerrit that everyone can see, including the
> earlier reviews from the draft stage.  This is a one way transition;
> once a draft is published, it can't be made a draft again.

This sounds cool. Will the vulnerability management team use this for
embargoed security fixes?

Cheers,
Mark.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] running HA cluster of guests within openstack

[ikke]

I likely am not the first one to ask this, but since I didn't find a
thread about it I start one.

Is there any shared experience available what are the capabilities of
OpenStack to run cluster of guests in the cloud? Do you have
experience of the following questions, or links to more info? The
questions relate to running a legacy HA cluster in virtual env, and
moving it into cloud...

1. Private networks between guests
  -> Doable now using Quantum
1.1. Defining VLANs visible to guest machines to separate clusters
internal traffic,
   VLAN tags should not be stripped by host (QinQ)
1.2. Set pre-defined MAC addresses for the guests, needed by non-IP
   traffic within the guest cluster (layer2 addressing)
  - will Melange do this, according to docs it's not in plans?
2. HA capabilities
2.1. Failure notification times need to be fast, i.e. no tcp timeout allowed
  - there seems to be some activity to integrate pacemaker
2.2. Failure notification of both guests and hosts needs to be included
2.3. Guest cluster controller should be able to monitor the states,
  and get fast notifications of the events.
  - rather in milliseconds than in seconds
  - basically the host should have parent of the guest pid notifying
of a child process failure.
  - Host should have a virtual watch-dog noticing of a guest being stuck
2.4. Failure recovery time, how fast can OS bring up failed guest?
  - any measurements of time from failure to noticing it,
and time that the guest is restarted and back up?
2.5. virtual HW manager (guest isolation)
  - Any plans to integrate a piece from which a state of guest could
be reliably queried, e.g. guaranteeing that if I ask to power
off another
guest, it get's done in given time (millisecs), and not
pending on e.g. some tcp
timeout, and thus leading to split brain case of running two
similar guest
simultaneously. E.g. starting another guest to replace shut
down one, but
due some communications error the first one didn't really shut
before the
new one is already up.
 - should be able to reliably cut down the guests network and disk access to
   guarantee the above case
2.6. Shared disks
 - Could there be a shared scsi device concept for the legacy HW
abstraction?
 - Qemu/KVM supports this, what would it take to make OS to understand
   such disk devices?
2.7. Isolation of redundant nodes
 - In some cases there are nodes that need to backup each others 2N, N+1,
   there should be a way to make sure they run on different host.
 - This project might be aiming for that?
http://wiki.openstack.org/DistributedScheduler

This was something from top of my head, it would be interesting to
hear your thoughts about the issues. This need is coming from the
telco world, which would need a telco-cloud with such more real-time
features in it. Certainly the same applies to many other legacy
environments too.

BR,

 Ilkka Tengvall

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] control user quota

[Anton Haldin]

>>"You can set quotas per tenant"
+1 there is no quota for user.

nova-manage example with Diablo Openstack.
#nova-manage project quota ACME

metadata_items: 128
instances: 10
injected_file_content_bytes: 10240
injected_files: 5
volumes: 10
gigabytes: 1000
cores: 20
ram: 51200
floating_ips: 10

#nova-manage project quota ACME --key=instances --value=7


metadata_items: 128
instances: 7
injected_file_content_bytes: 10240
injected_files: 5
volumes: 10
gigabytes: 1000
cores: 20
ram: 51200
floating_ips: 10

default values you can find
in /usr/lib/python2.6/site-packages/nova/quota.py

flags.DEFINE_integer('quota_instances', 10,
 'number of instances allowed per project')
flags.DEFINE_integer('quota_cores', 20,
 'number of instance cores allowed per project')
flags.DEFINE_integer('quota_ram', 50 * 1024,
 'megabytes of instance ram allowed per project')
flags.DEFINE_integer('quota_volumes', 10,
 'number of volumes allowed per project')
flags.DEFINE_integer('quota_gigabytes', 1000,
 'number of volume gigabytes allowed per project')
flags.DEFINE_integer('quota_floating_ips', 10,
 'number of floating ips allowed per project')
flags.DEFINE_integer('quota_metadata_items', 128,
 'number of metadata items allowed per instance')
flags.DEFINE_integer('quota_max_injected_files', 5,
 'number of injected files allowed')
flags.DEFINE_integer('quota_max_injected_file_content_bytes', 10 * 1024,
 'number of bytes allowed per injected file')
flags.DEFINE_integer('quota_max_injected_file_path_bytes', 255,
 'number of bytes allowed per injected file path')

Anton Haldin

2012/4/12 Kuo Hugo 

> nova-manage
> http://nova.openstack.org/runnova/nova.manage.html
>
> nova-manage project quota 
>
> example :
> #list xin-project 's quota
> $>nova-manage project quota xin-project
> Will return several key/value
>
> #modify a key with new value
>
> $>nova-manage project quota xin-project --key=???  --value=???
>
> https://github.com/openstack/nova/blob/master/nova/quota.py
>
> For configuration flags please refer to the above link of quota.py
>
>
> Xin Zhao  於 2012年4月13日上午12:15 寫道：
>
>>  Hi Kuo,
>>
>> Could you give more details, like the commands used, and settings in the
>> config file ? I can't find a good example for them.
>>
>> Thanks,
>> Xin
>>
>>
>> On 4/12/2012 12:04 PM, Kuo Hugo wrote:
>>
>> I did a quick test in Essex .
>> The process almost same as before(Cactus/Diablo)
>>
>>  1. Manage Quota for a specified "Tenant" from nova-manage .
>> 2. Manage Default Quota parameters from nova.conf with several flags
>> 3. Hacking Nova source code quota.py for default values.
>>
>>
>>
>>  Hope it helps.
>>
>>
>>
>> Xin Zhao  於 2012年4月12日下午10:27 寫道：
>>
>>> Hello,
>>>
>>> I try to assign quota to individual users, to control how many instances
>>> each user can run concurrently. But I don't see a doc describing how to do
>>> that. I use diablo release.
>>> Any help or doc pointer will be greatly appreciated.
>>>
>>> Xin
>>>
>>>
>>> ___
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
>>
>>  --
>> +Hugo Kuo+
>> tonyt...@gmail.com
>>  + 886 935004793
>>
>>
>>
>
>
> --
> +Hugo Kuo+
> tonyt...@gmail.com
> + 886 935004793
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp