Re: [openstack-dev] [Solum] Stackforge Repo Ready

[Noorul Islam K M]

Adrian Otto  writes:

> Team,
>
> Our StackForge code repo is open, so you may begin submitting code for 
> review. For those new to the process, I made a will page with links to the 
> repo and information about how to contribute:
>
> https://wiki.openstack.org/wiki/Solum/Contributing
>

1. .gitreview file is missing, so I submitted a patch 

https://review.openstack.org/#/c/54877

This patch also contains update to README to include relevant project
information.

2. My review request got rejected by Jenkins. A re-base against [1] is
   not helping.

3. Github repo [2] is behind [1]. It is not mirrored yet?

Regards,
Noorul

[1] git://git.openstack.org/stackforge/solum
[2] https://github.com/stackforge/solum

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Migrating to newer full projects from what used to be part of nova

[Dean Troyer]

On Thu, Oct 31, 2013 at 2:18 PM, Jesse Pretorius
wrote:

> On 31 October 2013 18:46, John Griffith wrote:
>
>> Upgrading Essex->Folsom introduced both the challenge of upgrading
>> nova-volume to cinder and the challenge of upgrading nova-network to
>> quantum. Upgrading Folsom->Grizzly presents the challenge of migrating
>> nova-network to quantum and assumes that nova-volume has already been
>> migrated to cinder. Upgrading Grizzly->Havana finally closes the door on
>> nova-network as far as I can see, although I may be wrong.
>>
>
To clarify a bit, while deprecated, nova-network has not lost any
functionality in Havana yet.  It also hasn't gained much (if any), but it
still works.  The majority of the testing is performed using nova-network
and not neutron (yet).

It comes down to the fact that while we cater for migrating between
> versions of a particular project we don't cater particularly well for
> migrating between projects when a project splits out from a parent project
> as was the case for both of the above.
>

I am not aware of any effort to do a migration from nova-network to
neutron, we certainly don't have it in grenade's near future.  The
differences are large and the development cost for that sort of migration
is significant for something that a given deployment is only going to use
once.  It isn't a technical problem but a resource problem.

Also, FWIW, I don't see another one of these situations coming anytime
soon.  All of the new project activity is around new services/features.

dt

-- 

Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Re : welcoming new committers (was Re: When is it okay for submitters to say 'I don't want to add tests' ?)

[Jeremy Stanley]

On 2013-10-31 22:45:56 + (+), Romain Hardouin wrote:
> Adding a message for new comers is a good idea.
> I am a new Horizon contributor, some of my fixes have been merged
> (thanks to Upstream University :-) and reviewers of course) but I
> still hesitate to do code review. To my mind, it is reserved to
> "known" developpers whose opinion matters...

Not at all. One of the best ways to become "known" within the
community is to review code and provide good recommendations. Even
something as simple as spotting typographical errors in changes to
user-facing messages and documentation provides value. The more
problems you can find (and ultimately help prevent) in a change, the
faster your reputation will grow.

As has been said many times already, OpenStack does not lack
developers... it lacks reviewers.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-dev][Nova][Discussion]Blueprint : Auto VM Discovery in OpenStack for existing workload

[Lingxian Kong]

one question here, if we have vm-discovery, what's differece between
discoveried vm and orphron vm already handled in the periodic task of
nova-compute?


2013/10/30 Swapnil Kulkarni 

> I had a discussion with russellb regarding this for yesterday, I would
> like to discuss this with the team regarding the blueprint mentioned in
> subject.
>
> https://blueprints.launchpad.net/nova/+spec/auto-vm-discovery-on-hypervisor
>
> Description: Organizations opting to use openstack can have varied amount
> of workload that they would like to be available directly with the use of
> some discovery workflows. One common usage of this would be exising virtual
> machines present on the hypervisors. If this instances can be disovered by
> the compute agent during discovery, it would help to use Openstack to
> manage the existing workload directly. Auto VM Discovery will enable this
> functionality initially for KVM guests, the widely used hypervisor
> configuration in OpenStack deployments and enhance it further for other
> hypervisors.
>
> Best Regards,
> Swapnil
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
*---*
*Lingxian Kong*
Huawei Technologies Co.,LTD.
IT Product Line CloudOS PDU
China, Xi'an
Mobile: +86-18602962792
Email: konglingx...@huawei.com; anlin.k...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Infra] [Tempest] [Ironic] Testing Ironic

[Monty Taylor]

On 10/31/2013 04:38 PM, Devananda van der Veen wrote:
> Hi all,
> 
> Ironic has reached a point where it is capable of being added to
> integration tests and the gate pipeline. Does that mean we /should/ add
> it? I think so, and I'd like to know what others think, but let me be
> more specific about what I'm talking about.

Yes, we should - with caveats.

> At this point, it is possible for devstack to start ironic-api and
> ironic-conductor services, register them with keystone, init the db,
> etc. Then, tempest [1] can perform basic actions like create, update,
> delete records. Testing the hardware drivers will come later, which is
> where we'll see integration tests with glance, neutron, etc, become more
> important.
> 
> Roman has been working on enabling these tests [2] and I'd like to nudge
> the reviewers of the tempest and infra teams to take a look at his
> patches. At the very least, I'd like to make folks aware of this work so
> we can all discuss it at the summit.

Basically, we've been wanting for a while to see gate integration before
something graduates incubation - but there is a chicken and egg sitch
where we can't REALLY add it to the gate until after graduation.

So, I think what we want to do is add the tests to tempest/devstack, and
then make a job that only runs on ironic changes that runs that. That
will show us that you're ready once graduation review comes up.

I'll look at the infra patches (it's been slow with jeblair out)

> [1]
> Tempest tests for Ironic API: https://review.openstack.org/#/c/48109
> 
> [2]
> Pre-cache Ironic to slaves: https://review.openstack.org/#/c/54569
> Enable Ironic in devstack-gate: https://review.openstack.org/#/c/53899
> Enable tempest tests in the experimental pipeline:
> https://review.openstack.org/#/c/53917
> 
> 
> Thanks!
> Devananda
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa][keystone] Help with XML Tempest API tests

[Adam Young]

On 10/31/2013 07:58 PM, Adam Young wrote:

On 10/31/2013 03:40 PM, Steven Hardy wrote:

Hi all,

So firstly, if you're an XML guru, I apologize, the questions below are
probably really basic, I always prefer JSON or YAML, because every 
time I

deal with XML, I get a week-long-headache ;)

So I'm writing Tempest API tests for the keystone OS-TRUST extension, as
was previously requested - it's going pretty well, & I'm finding some 
real

bugs.  Here's a WIP review:

https://review.openstack.org/#/c/54810/

However, I've got a bit stuck formatting the POST body for the trust 
create

in XML (all works fine via JSON).  The json body looks like:

{
 "trust": {
 "expires_at": "2013-02-27T18:30:59.99Z",
 "impersonation": true,
 "project_id": "ddef321",
 "roles": [
 {
 "name": "member"
 }
 ],
 "trustee_user_id": "86c0d5",
 "trustor_user_id": "a0fdfd"
 }
}

And looking at other XML tests which format a POST body, they do 
something

like:

post_body = Element("trust",
 xmlns=XMLNS,
 trustor_user_id=trustor_user_id,
 trustee_user_id=trustee_user_id,
 project_id=project_id,
 impersonation=impersonation,
 expires_at=expires_at)

This gives me a post body which looks weird (but seems to work):

http://docs.openstack.org/identity/api/v3";
trustor_user_id="efc6504105c54fbe95928a51459d06c9" expires_at=""
trustee_user_id="f55efd1d617e4367891d202a811d7728"
project_id="b5d498f9631244b59912ce2a0025cf8d"/>


So my questions are:
1. Why do we create a single element like this, instead of appending
subelements so the XML body looks more like the JSON request?


There are two sides. Some would say that the above is more like a JSON 
map


  ['a':'1','b':'2','c':'3']

compare

<'a':'1','b':'2','c':'3' />

to

<'a'='1'>,<'b'='2'>,<'c'='3'>

And is starts getting even wonkier when the nested element is a 
map...XML has two abstractions where JSON has one.  THere is no clean 
way to always map on to the other (IMHO).




2. If any elements have a None value, they are encoded as a zero-length
string, is that expected?

3. How do I go about encoding the list of roles, as in the sample 
request

(it's a list of dicts, where each dict has one key called "name")

That, I think is


<"name": "member">


However, all of this should be viewable with the existing unit tests.  
Each API gets run with both JSON and XML.  In the keystone proejct, 
the test is in tests/test_v3_auth.py


I still use run_test.sh even though it is deprecated

run_tests.sh  keystone.tests.test_v3_auth.TestAuthXML

If you run that in a debugger you should be able to grab the actual 
XML that gets marshalled.


I think it is safe to say that the trusts API is broken in XML.  I added 
the following test:


diff --git a/keystone/tests/test_v3_auth.py b/keystone/tests/test_v3_auth.py
index c0e191b..6a0c10c 100644
--- a/keystone/tests/test_v3_auth.py
+++ b/keystone/tests/test_v3_auth.py
@@ -2238,3 +2238,7 @@ class TestTrustAuth(TestAuthInfo):
 self.get('/OS-TRUST/trusts?trustor_user_id=%s' %
  self.user_id, expected_status=401,
  token=trust_token)
+
+
+class TestTrustAuthXML(TestTrustAuth):
+content_type = 'xml'

And, when running it, I got:


Ran 24 tests in 5.832s

FAILED (SKIP=1, errors=12)


https://bugs.launchpad.net/keystone/+bug/1246941






Any help, review feedback or pointers to docs/examples would be hugely
appreciated!

Thanks,

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Adam Young]

On 10/31/2013 03:24 PM, Jeremy Stanley wrote:

On 2013-10-31 13:30:32 + (+), Mark McLoughlin wrote:
[...]

In cases like that, I'd be of a mind to go "+2 Awesome! Thanks for
catching this! It would be great to have a unit test for this, but
it's clear the current code is broken so I'm fine with merging the
fix without a test". You could say it's now the reviewers
responsibility to merge a test, but if that requirement then turns
off reviewers even reviewing such a patch, then that doesn't help
either.

[...]

I get the impression it was one of my +2s in a situation like this
which spawned the thread (or at least was the straw which broke the
camel's back), so apologies for stirring the beehive. Someone was
trying to set up their own copy of several of our infrastructure
projects, spotted a place in one where we had hard-coded something
specific to our environment, and wanted to parameterize it upstream
rather than paper over it on their end.

The bug-reporter-turned-patch-contributor didn't know how to write a
test which would confirm that parameter got passed through and we
weren't performing tests yet for any similar parameters already in
the daemon which I could point to as an example. I agree it's a
judgement call between risking discouraging a new contributor vs.
reducing test coverage further, but I was probably still a bit too
hasty to suggest that we could add tests for those in a separate
commit.

In my case I didn't have the available time to instruct the
contributor on how to write tests for this, but that also probably
meant that I didn't really have time to be reviewing the change
properly to begin with. I'm quite grateful to Robert for stepping in
and helping walk them through it! We got more tests, I think they
got a lot of benefit from the new experience as well, and I was
appropriately humbled for my lax attitude over the situation which
nearly allowed us to miss a great opportunity at educating another
developer on the merits of test coverage.
It is probably the best approach to work with a new patch submitter to 
show them how to write the tests.  Another approach is to write a test 
yourself.  CHeck for some precondition that shows the patch is appalied, 
and if it is not met, throw a Skip exception.  That test should pass 
until their bug comes in.  Then, once htier patch comes in, test test 
should be triggered.  You can remove the skip code in a future commit.


I would like to see more of this kind of coding:  tests that show a 
feature is missing etc.  We discussed it a bit in the Keystone/Client 
discussions where we wanted to run tests against a live sever for a 
client change, but couldn't really check in new tests in the server as 
it was after code freeze.  I made the decision to push for Tempest tests 
there...and we broke some new ground in our workflow.


One tool that is really valuable is the test coverage tool.  It can show 
the lines of code that have no coverage, and will help confirm if a 
given patch is tested or not.   It used to run on each commit, or  so I 
was told.  I was running it periodically for Keystone tests: here is an 
old run http://admiyo.fedorapeople.org/openstack/keystone/coverage/










___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa][keystone] Help with XML Tempest API tests

[Adam Young]

On 10/31/2013 03:40 PM, Steven Hardy wrote:

Hi all,

So firstly, if you're an XML guru, I apologize, the questions below are
probably really basic, I always prefer JSON or YAML, because every time I
deal with XML, I get a week-long-headache ;)

So I'm writing Tempest API tests for the keystone OS-TRUST extension, as
was previously requested - it's going pretty well, & I'm finding some real
bugs.  Here's a WIP review:

https://review.openstack.org/#/c/54810/

However, I've got a bit stuck formatting the POST body for the trust create
in XML (all works fine via JSON).  The json body looks like:

{
 "trust": {
 "expires_at": "2013-02-27T18:30:59.99Z",
 "impersonation": true,
 "project_id": "ddef321",
 "roles": [
 {
 "name": "member"
 }
 ],
 "trustee_user_id": "86c0d5",
 "trustor_user_id": "a0fdfd"
 }
}

And looking at other XML tests which format a POST body, they do something
like:

post_body = Element("trust",
 xmlns=XMLNS,
 trustor_user_id=trustor_user_id,
 trustee_user_id=trustee_user_id,
 project_id=project_id,
 impersonation=impersonation,
 expires_at=expires_at)

This gives me a post body which looks weird (but seems to work):

http://docs.openstack.org/identity/api/v3";
trustor_user_id="efc6504105c54fbe95928a51459d06c9" expires_at=""
trustee_user_id="f55efd1d617e4367891d202a811d7728"
project_id="b5d498f9631244b59912ce2a0025cf8d"/>


So my questions are:
1. Why do we create a single element like this, instead of appending
subelements so the XML body looks more like the JSON request?


There are two sides. Some would say that the above is more like a JSON map

  ['a':'1','b':'2','c':'3']

compare

<'a':'1','b':'2','c':'3' />

to

<'a'='1'>,<'b'='2'>,<'c'='3'>

And is starts getting even wonkier when the nested element is a 
map...XML has two abstractions where JSON has one.  THere is no clean 
way to always map on to the other (IMHO).




2. If any elements have a None value, they are encoded as a zero-length
string, is that expected?

3. How do I go about encoding the list of roles, as in the sample request
(it's a list of dicts, where each dict has one key called "name")

That, I think is


<"name": "member">


However, all of this should be viewable with the existing unit tests.  
Each API gets run with both JSON and XML.  In the keystone proejct, the 
test is in tests/test_v3_auth.py


I still use run_test.sh even though it is deprecated

run_tests.sh  keystone.tests.test_v3_auth.TestAuthXML

If you run that in a debugger you should be able to grab the actual XML 
that gets marshalled.





Any help, review feedback or pointers to docs/examples would be hugely
appreciated!

Thanks,

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Re : welcoming new committers

[Stefano Maffulli]

On 10/31/2013 03:45 PM, Romain Hardouin wrote:
> Adding a message for new comers is a good idea.
> I am a new Horizon contributor, some of my fixes have been merged
> (thanks to Upstream University :-)

Indeed :) Can you share with us briefly what you found interesting in
the sessions of Upstream University? Which ones did you go to?

> and reviewers of course) but I still
> hesitate to do code review. To my mind, it is reserved to "known"
> developpers whose opinion matters...

Well, I think that anybody's opinion matters and you're not a new
OpenStack developer anymore. You have your own experience and your
reviews may definitely help somebody even newer than you to get his/her
patch refined before the more experienced developers get to it. I'm sure
your comments, even without a vote, would help. Chime in then :)

/stef

-- 
Ask and answer questions on https://ask.openstack.org

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] welcoming new committers (was Re: When is it okay for submitters to say 'I don't want to add tests' ?)

[Tom Fifield]

On 01/11/13 08:35, Mark McLoughlin wrote:

On Thu, 2013-10-31 at 11:49 -0700, Stefano Maffulli wrote:


Another idea that Tom suggested is to use gerrit automation to send back
to first time committers something in addition to the normal 'your patch
is waiting for review' message. The message could be something like:


thank you for your first contribution to OpenStack. Your patch will
now be tested automatically by OpenStack testing frameworks and once
the automatic tests pass, it will be reviewed by other friendly
developers. They will give you comments and may require you to refine
it.

Nobody gets his patch approved at first try so don't be concerned
when someone will require you to do more iterations.

Patches usually take 3 to 7 days to be approved so be patient and be
available on IRC to ask and answer questions about your work. The
more you participate in the community the more rewarding it is for
you. You may also notice that the more you get to know people and get
to be known, the faster your patches will be reviewed and eventually
approved. Get to know others and be known by doing code reviews:
anybody can and should do it.

...

Very nicely done. I really like it.



Cool to see this happening :)

Another idea I had was to encourage new-ish people to engage in the 
review process. For example if #patches > N and #reviews < M, post a 
message. This might be getting too complicated though :)



Regards,


Tom

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Solum] Stackforge Repo Ready

[Adrian Otto]

Team,

Our StackForge code repo is open, so you may begin submitting code for review. 
For those new to the process, I made a will page with links to the repo and 
information about how to contribute:

https://wiki.openstack.org/wiki/Solum/Contributing

Regards,

Adrian




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Re : welcoming new committers (was Re: When is it okay for submitters to say 'I don't want to add tests' ?)

[Romain Hardouin]

Hi all, Adding a message for new comers is a good idea. I am a 
new Horizon contributor, some of my fixes have been merged (thanks to Upstream 
University :-) and reviewers of course) but I still hesitate to do code review. 
To my mind, it is reserved to "known" developpers whose opinion matters... 
- Romain___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][scheduler]The database access in the scheduler filters

[Jiang, Yunhong]

I noticed several filters (AggregateMultiTenancyIsoaltion, ram_filter, 
type_filter, AggregateInstanceExtraSpecsFilter) have DB access in the 
host_passes(). Some will even access for each invocation.

Just curios if this is considered a performance issue? With a 10k nodes, 60 VM 
per node, and 3 hours VM life cycle cloud, it will have more than 1 million DB 
access per second. Not a small number IMHO.

Thanks
--jyh

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Abdicating the PTL Position

[Gabriel Hurley]

Thanks for the well-wishes (from everyone)! As I said, I'm not gone for good.

By way of explanation, it's a combination of things: personal items I've put 
off too long (startup life, right?) and new responsibilities at Nebula (which 
will keep me at least partly engaged in OpenStack, just not on the Horizon 
side).

All in all I hold myself to a high standard in anything I do, and looking at 
the next six months I saw my attention being split too many ways. At the same 
time I saw strong people in Horizon Core and knew the project would be in good 
hands so I chose to ensure that everyone and everything gets the attention it 
deserves, even if not all from me personally.

In truth I probably care about Horizon more than I ought to. It's been my baby 
for the last 2+ years, long before I was PTL officially. I don't intend to see 
it decline or degrade at all, and given what the team has been talking about 
(and will discuss at the summit) I think the best is yet to come!


-  Gabriel

From: Endre Karlson [mailto:endre.karl...@gmail.com]
Sent: Thursday, October 31, 2013 2:52 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Horizon] Abdicating the PTL Position

@Gabriel, thanks for an awesome time leading Horizon in towards what is now a 
awesome Framework / UI for OpenStack! I'm sure you'll bring awesome to whatever 
you're doing next!

It's very sad to see good people leave the PTL positions but hey, time goes by 
and people do new things.

Good luck and see you around dude, it was awesome to meet you in Portland in 
the last summit :)

Endre.

2013/10/31 Endre Karlson 
mailto:endre.karl...@gmail.com>>
Nevermind and ignore the last e-mail, it was wrongly intended.

Endre

2013/10/31 Endre Karlson 
mailto:endre.karl...@gmail.com>>
Hey, I'm just curious. You quitting the PTL position to focus on Nebula efforts 
or ?

2013/10/31 Gabriel Hurley 
mailto:gabriel.hur...@nebula.com>>
Hi all,

It saddens me to say that for a mix of reasons I have decided to abdicate my 
position as PTL for Horizon. If anything, the reasons are all good ones 
overall, I just have to make the right decision for both myself and the project.

In the interim David Lyle will be the acting PTL. The Horizon core team has all 
weighed in with their confidence in his abilities, and he has confirmed his 
ability and interest in doing so. There will be a nomination period in coming 
weeks to determine the PTL for the full release cycle, should anyone else wish 
to run for the job as well. Thierry will announce more information about that 
soon.

Rest assured, you're not rid of me entirely; I'm just making a decision to 
focus in on other areas for a time. Horizon is blessed to have other phenomenal 
candidates willing and able to lead, and I would rather see the project in the 
hands of someone who can devote their full attention and energy to it.

I will still be around and engaged (though not in Hong Kong). I'll catch you 
all next time around.

All the best,

- Gabriel Hurley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Solum] HOLD THE DATES: Design Workshop @SFO

[Adrian Otto]

Jay,

It is November 19 and 20. Looks like because it was sent as a calendar invite 
that the notes were scrubbed. Here is the text:

Workshop sessions
Nov 19 Tuesday 9:30 am - 5 pm
Nov 20 Wednesday 9:30 am - 3:00 pm (we break early on Wednesday if some of us 
want to catch an evening return flight)

Etherpad for event planning-
https://etherpad.openstack.org/p/SolumSFOCommunityWorkshop

Cheers,

Adrian

On Oct 31, 2013, at 3:02 PM, Jay Pipes 
 wrote:

> On 10/31/2013 05:41 PM, Roshan Agrawal wrote:
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> I think you probably want to say which particular dates people should hold. :)
> 
> -jay
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Solum] HOLD THE DATES: Design Workshop @SFO

[Jay Pipes]

On 10/31/2013 05:41 PM, Roshan Agrawal wrote:

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


I think you probably want to say which particular dates people should 
hold. :)


-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Abdicating the PTL Position

[Endre Karlson]

@Gabriel, thanks for an awesome time leading Horizon in towards what is now
a awesome Framework / UI for OpenStack! I'm sure you'll bring awesome to
whatever you're doing next!

It's very sad to see good people leave the PTL positions but hey, time goes
by and people do new things.

Good luck and see you around dude, it was awesome to meet you in Portland
in the last summit :)

Endre.


2013/10/31 Endre Karlson 

> Nevermind and ignore the last e-mail, it was wrongly intended.
>
> Endre
>
>
> 2013/10/31 Endre Karlson 
>
>> Hey, I'm just curious. You quitting the PTL position to focus on Nebula
>> efforts or ?
>>
>>
>> 2013/10/31 Gabriel Hurley 
>>
>>> Hi all,
>>>
>>> It saddens me to say that for a mix of reasons I have decided to
>>> abdicate my position as PTL for Horizon. If anything, the reasons are all
>>> good ones overall, I just have to make the right decision for both myself
>>> and the project.
>>>
>>> In the interim David Lyle will be the acting PTL. The Horizon core team
>>> has all weighed in with their confidence in his abilities, and he has
>>> confirmed his ability and interest in doing so. There will be a nomination
>>> period in coming weeks to determine the PTL for the full release cycle,
>>> should anyone else wish to run for the job as well. Thierry will announce
>>> more information about that soon.
>>>
>>> Rest assured, you're not rid of me entirely; I'm just making a decision
>>> to focus in on other areas for a time. Horizon is blessed to have other
>>> phenomenal candidates willing and able to lead, and I would rather see the
>>> project in the hands of someone who can devote their full attention and
>>> energy to it.
>>>
>>> I will still be around and engaged (though not in Hong Kong). I'll catch
>>> you all next time around.
>>>
>>> All the best,
>>>
>>> - Gabriel Hurley
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Solum] HOLD THE DATES: Design Workshop @SFO

[Roshan Agrawal]

BEGIN:VCALENDAR
METHOD:REQUEST
PRODID:Microsoft Exchange Server 2010
VERSION:2.0
BEGIN:VTIMEZONE
TZID:Pacific Standard Time
BEGIN:STANDARD
DTSTART:16010101T02
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=11
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T02
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=2SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
ORGANIZER;CN=Roshan Agrawal:MAILTO:roshan.agra...@rackspace.com
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=openstack-
 d...@lists.openstack.org:MAILTO:openstack-dev@lists.openstack.org
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Rhett Dill
 ingham:MAILTO:rhett.dilling...@rackspace.com
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Josh Odom:
 MAILTO:josh.o...@rackspace.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Keith Bray
 :MAILTO:keith.b...@rackspace.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN='Murali Ba
 lcha':MAILTO:murali.bal...@triliodata.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Adrian Ott
 o:MAILTO:adrian.o...@rackspace.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Ed Leafe:M
 AILTO:ed.le...@rackspace.com
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=Shiv Haris
 :MAILTO:sha...@brocade.com
DESCRIPTION;LANGUAGE=en-US:\n\nBlocking 
 time on the calendars for the 2 day design workshop at SFO. More details t
 o follow\, but here is the current plan:\n\nWorkshop sessions\nNov 19 Tues
 day 9:30 am - 5 pm\nNov 20 Wednesday 9:30 am - 3:00 pm (we break early on 
 Wednesday if some of us want to catch an evening return flight)\n\n\nEther
 pad for event planning-\nhttps://etherpad.openstack.org/p/SolumSFOCommunit
 yWorkshop\n\n\n\n
SUMMARY;LANGUAGE=en-US:[Solum] HOLD THE DATES: Design Workshop @SFO
DTSTART;TZID=Pacific Standard Time:20131119T093000
DTEND;TZID=Pacific Standard Time:20131120T15
UID:04008200E00074C5B7101A82E008117CE6277ED6CE01000
 01000EB5D62ED45B251458DF7C660180FDDBA
CLASS:PUBLIC
PRIORITY:5
DTSTAMP:20131031T214150Z
TRANSP:OPAQUE
STATUS:CONFIRMED
SEQUENCE:1
LOCATION;LANGUAGE=en-US:Rackspace SFO office
X-MICROSOFT-CDO-APPT-SEQUENCE:1
X-MICROSOFT-CDO-OWNERAPPTID:2111502609
X-MICROSOFT-CDO-BUSYSTATUS:TENTATIVE
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
X-MICROSOFT-CDO-ALLDAYEVENT:FALSE
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-INSTTYPE:0
X-MICROSOFT-DISALLOW-COUNTER:FALSE
BEGIN:VALARM
ACTION:DISPLAY
DESCRIPTION:REMINDER
TRIGGER;RELATED=START:-P1D
END:VALARM
END:VEVENT
END:VCALENDAR
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Abdicating the PTL Position

[Endre Karlson]

Nevermind and ignore the last e-mail, it was wrongly intended.

Endre


2013/10/31 Endre Karlson 

> Hey, I'm just curious. You quitting the PTL position to focus on Nebula
> efforts or ?
>
>
> 2013/10/31 Gabriel Hurley 
>
>> Hi all,
>>
>> It saddens me to say that for a mix of reasons I have decided to abdicate
>> my position as PTL for Horizon. If anything, the reasons are all good ones
>> overall, I just have to make the right decision for both myself and the
>> project.
>>
>> In the interim David Lyle will be the acting PTL. The Horizon core team
>> has all weighed in with their confidence in his abilities, and he has
>> confirmed his ability and interest in doing so. There will be a nomination
>> period in coming weeks to determine the PTL for the full release cycle,
>> should anyone else wish to run for the job as well. Thierry will announce
>> more information about that soon.
>>
>> Rest assured, you're not rid of me entirely; I'm just making a decision
>> to focus in on other areas for a time. Horizon is blessed to have other
>> phenomenal candidates willing and able to lead, and I would rather see the
>> project in the hands of someone who can devote their full attention and
>> energy to it.
>>
>> I will still be around and engaged (though not in Hong Kong). I'll catch
>> you all next time around.
>>
>> All the best,
>>
>> - Gabriel Hurley
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Abdicating the PTL Position

[Endre Karlson]

Hey, I'm just curious. You quitting the PTL position to focus on Nebula
efforts or ?


2013/10/31 Gabriel Hurley 

> Hi all,
>
> It saddens me to say that for a mix of reasons I have decided to abdicate
> my position as PTL for Horizon. If anything, the reasons are all good ones
> overall, I just have to make the right decision for both myself and the
> project.
>
> In the interim David Lyle will be the acting PTL. The Horizon core team
> has all weighed in with their confidence in his abilities, and he has
> confirmed his ability and interest in doing so. There will be a nomination
> period in coming weeks to determine the PTL for the full release cycle,
> should anyone else wish to run for the job as well. Thierry will announce
> more information about that soon.
>
> Rest assured, you're not rid of me entirely; I'm just making a decision to
> focus in on other areas for a time. Horizon is blessed to have other
> phenomenal candidates willing and able to lead, and I would rather see the
> project in the hands of someone who can devote their full attention and
> energy to it.
>
> I will still be around and engaged (though not in Hong Kong). I'll catch
> you all next time around.
>
> All the best,
>
> - Gabriel Hurley
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] welcoming new committers (was Re: When is it okay for submitters to say 'I don't want to add tests' ?)

[Mark McLoughlin]

On Thu, 2013-10-31 at 11:49 -0700, Stefano Maffulli wrote:

> Another idea that Tom suggested is to use gerrit automation to send back
> to first time committers something in addition to the normal 'your patch
> is waiting for review' message. The message could be something like:
> 
> > thank you for your first contribution to OpenStack. Your patch will
> > now be tested automatically by OpenStack testing frameworks and once
> > the automatic tests pass, it will be reviewed by other friendly
> > developers. They will give you comments and may require you to refine
> > it.
> > 
> > Nobody gets his patch approved at first try so don't be concerned
> > when someone will require you to do more iterations.
> > 
> > Patches usually take 3 to 7 days to be approved so be patient and be
> > available on IRC to ask and answer questions about your work. The
> > more you participate in the community the more rewarding it is for
> > you. You may also notice that the more you get to know people and get
> > to be known, the faster your patches will be reviewed and eventually
> > approved. Get to know others and be known by doing code reviews:
> > anybody can and should do it.
...

Very nicely done. I really like it.

Thanks,
Mark.

> PS I put the text on
> https://etherpad.openstack.org/p/welcome-new-committers for refinements.
> 



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Solum] HOLD THE DATES: Design Workshop @SFO

[Roshan Agrawal]

BEGIN:VCALENDAR
METHOD:REQUEST
PRODID:Microsoft Exchange Server 2010
VERSION:2.0
BEGIN:VTIMEZONE
TZID:Central Standard Time
BEGIN:STANDARD
DTSTART:16010101T02
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=11
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T02
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=2SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
ORGANIZER;CN=Roshan Agrawal:MAILTO:roshan.agra...@rackspace.com
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN=openstack-
 d...@lists.openstack.org:MAILTO:openstack-dev@lists.openstack.org
DESCRIPTION;LANGUAGE=en-US:When: Tuesday\, November 19\, 2013\, 9:30 AM to 
 Wednesday\, November 20\, 2013\, 3:00 PM. (UTC-06:00) Central Time (US & C
 anada)\nWhere: Rackspace SFO office\n\n*~*~*~*~*~*~*~*~*~*\n\nBlocking tim
 e on the calendars for the 2 day design workshop at SFO. More details to f
 ollow\, but here is the current plan:\n\n\nWorkshop sessions\nNov 19 Tuesd
 ay 9:30 am - 5 pm\nNov 20 Wednesday 9:30 am - 3:00 pm (we break early on W
 ednesday if some of us want to catch an evening return flight)\n\n\nEtherp
 ad for event planning-\nhttps://etherpad.openstack.org/p/SolumSFOCommunity
 Workshop\n\n
SUMMARY;LANGUAGE=en-US:[Solum] HOLD THE DATES: Design Workshop @SFO
DTSTART;TZID=Central Standard Time:20131119T093000
DTEND;TZID=Central Standard Time:20131120T15
UID:04008200E00074C5B7101A82E008117CE6277ED6CE01000
 01000EB5D62ED45B251458DF7C660180FDDBA
CLASS:PUBLIC
PRIORITY:5
DTSTAMP:20131031T211416Z
TRANSP:OPAQUE
STATUS:CONFIRMED
SEQUENCE:0
LOCATION;LANGUAGE=en-US:Rackspace SFO office
X-MICROSOFT-CDO-APPT-SEQUENCE:0
X-MICROSOFT-CDO-OWNERAPPTID:2111502609
X-MICROSOFT-CDO-BUSYSTATUS:TENTATIVE
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
X-MICROSOFT-CDO-ALLDAYEVENT:FALSE
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-INSTTYPE:0
X-MICROSOFT-DISALLOW-COUNTER:FALSE
BEGIN:VALARM
ACTION:DISPLAY
DESCRIPTION:REMINDER
TRIGGER;RELATED=START:-P1D
END:VALARM
END:VEVENT
END:VCALENDAR
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] A question about atomicity of repository commits

[Robert Collins]

On 1 November 2013 09:20, Birdsall, Dave  wrote:
> Hi,
>
> I'm new to the Openstack community. I have what is probably a naïve question.
>
> Suppose source code for a project is in Git repository A, while the Jenkins 
> test suite for it is in Git repository B.
>
> First question: Does this happen in Openstack? (For example, are there nova 
> tests in the tempest repository that get executed in Jenkins when nova is 
> changed?)

Yes, in the devstack-gate-*-tempest jobs.

> Given my assumption above, suppose I'm a contributor and I want to change 
> some source in repository A. However, my source change affects the test 
> results of a test in repository B. So I want to change that as well.
>
> My understanding is that I'd clone both repositories, and commit changes to 
> each independently, and each would go to Gerrit as a separate change.

Yes.

> Second question: How does one handle the chicken-and-egg problem? If the 
> source change to A goes through Jenkins first, the automated tests would 
> fail. But if the test change to B goes through Jenkins first, it would also 
> fail because the old code in A has the old behavior.

There are a few nuances here.

Firstly, tests in separate repositories are usually loosely coupled,
so most such changes should not need changes to tests in concert with
the code in the other repo.

Secondly, we hold backwards compatibility as an important value, so
where such changes might need changes in concert, we often will choose
to make the change backwards compatible, avoiding any issue.

In the rare care where we can't make it backwards compatible, and the
test will be broken, we can disable the test, land the new change,
enable the test, or - better - make the test handle both behaviours,
land the new change, make the test only handle the new behaviour.

> Thanks and kind regards,
>
> Dave
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] A question about atomicity of repository commits

[Clark Boylan]

On Thu, Oct 31, 2013 at 1:20 PM, Birdsall, Dave  wrote:
> Hi,
>
> I'm new to the Openstack community. I have what is probably a naïve question.
>
> Suppose source code for a project is in Git repository A, while the Jenkins 
> test suite for it is in Git repository B.
>
> First question: Does this happen in Openstack? (For example, are there nova 
> tests in the tempest repository that get executed in Jenkins when nova is 
> changed?)
>
> Given my assumption above, suppose I'm a contributor and I want to change 
> some source in repository A. However, my source change affects the test 
> results of a test in repository B. So I want to change that as well.
>
> My understanding is that I'd clone both repositories, and commit changes to 
> each independently, and each would go to Gerrit as a separate change.
Correct. You will probably want to drop a comment on each review
referring to the other so that reviewers understand what is going on.
>
> Second question: How does one handle the chicken-and-egg problem? If the 
> source change to A goes through Jenkins first, the automated tests would 
> fail. But if the test change to B goes through Jenkins first, it would also 
> fail because the old code in A has the old behavior.
You have to make the changes in backward and forward compatible ways,
this will allow the tests to pass for at least one of the changes
which will then unblock the other change.
>
> Thanks and kind regards,
>
> Dave
>

Clark

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat] [savanna] [trove] Place for software configuration

[Clint Byrum]

Excerpts from Alexander Kuznetsov's message of 2013-10-31 10:51:54 -0700:
> Hi Heat, Savanna and Trove teams,
> 
> All this projects have common part related to software configuration
> management.  For creation,  an environment  user should specify a hardware
> parameter for vms:  choose flavor, decide use cinder or not, configure
> networks for virtual machines, choose topology for hole deployment. Next
> step is linking of software parameters with hardware specification. From
> the end user point of view, existence of three different places and three
> different ways (HEAT Hot DSL, Trove clustering API and Savanna Hadoop
> templates) for software configuration is not convenient, especially if user
> want to create an environment simultaneously involving components from
> Savanna, Heat and Trove.
> 

I'm having a hard time extracting the problem statement. I _think_ that
the problem is:

As a user I want to tune my software for my available hardware.

So what you're saying is, if you select a flavor that has 4GB of RAM
for your application, you want to also tell your application that it
can use 3GB of RAM for an in-memory cache. Likewise, if one has asked
Trove for an 8GB flavor, they will want to tell it to use 6.5GB of RAM
for InnoDB buffer cache.

What you'd like to see is one general pattern to express these types
of things?

> I can suggest two approaches to overcome this situations:
> 
> Common library in oslo. This approach allows a deep domain specific
> customization. The user will still have 3 places with same UI where user
> should perform configuration actions.
> 
> Heat or some other component for software configuration management. This
> approach is the best for end users. In feature possible will be some
> limitation on deep domain specific customization for configuration
> management.

Can you maybe be more concrete with your proposed solutions? The lack
of a clear problem statement combined with these vague solutions has
thoroughly confused me.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Infra] [Tempest] [Ironic] Testing Ironic

[Devananda van der Veen]

Hi all,

Ironic has reached a point where it is capable of being added to
integration tests and the gate pipeline. Does that mean we /should/ add it?
I think so, and I'd like to know what others think, but let me be more
specific about what I'm talking about.

At this point, it is possible for devstack to start ironic-api and
ironic-conductor services, register them with keystone, init the db, etc.
Then, tempest [1] can perform basic actions like create, update, delete
records. Testing the hardware drivers will come later, which is where we'll
see integration tests with glance, neutron, etc, become more important.

Roman has been working on enabling these tests [2] and I'd like to nudge
the reviewers of the tempest and infra teams to take a look at his patches.
At the very least, I'd like to make folks aware of this work so we can all
discuss it at the summit.


[1]
Tempest tests for Ironic API: https://review.openstack.org/#/c/48109

[2]
Pre-cache Ironic to slaves: https://review.openstack.org/#/c/54569
Enable Ironic in devstack-gate: https://review.openstack.org/#/c/53899
Enable tempest tests in the experimental pipeline:
https://review.openstack.org/#/c/53917


Thanks!
Devananda
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] welcoming new committers (was Re: When is it okay for submitters to say 'I don't want to add tests' ?)

[Kyle Mestery (kmestery)]

On Oct 31, 2013, at 3:21 PM, Monty Taylor  wrote:
> 
> 
> On 10/31/2013 04:15 PM, Kyle Mestery (kmestery) wrote:
>> On Oct 31, 2013, at 1:49 PM, Stefano Maffulli  wrote:
>>> On 10/31/2013 07:05 AM, Kyle Mestery (kmestery) wrote:
>>> [...]
 If we want to grow the committer base and help people to become
 better reviewers, taking the time to show them the ropes is part of
 the game.
>>> 
>>> hijacking the thread using Kyle's comment as an excuse.
>>> 
>> Hey, glad to provide you an opening Stef!
>> 
>>> It's not an 'if' but a 'since': since we are growing the committer base
>>> at an incredible pace we should help them become also good reviewers as
>>> rapidly possible.
>>> 
>>> One thing I already mentioned and I'll start doing this week in the
>>> weekly Newsletter is to give a shoutout to those that do their first
>>> review this week.
>>> 
>>> Another idea that Tom suggested is to use gerrit automation to send back
>>> to first time committers something in addition to the normal 'your patch
>>> is waiting for review' message. The message could be something like:
>>> 
 thank you for your first contribution to OpenStack. Your patch will
 now be tested automatically by OpenStack testing frameworks and once
 the automatic tests pass, it will be reviewed by other friendly
 developers. They will give you comments and may require you to refine
 it.
 
 Nobody gets his patch approved at first try so don't be concerned
 when someone will require you to do more iterations.
 
 Patches usually take 3 to 7 days to be approved so be patient and be
 available on IRC to ask and answer questions about your work. The
 more you participate in the community the more rewarding it is for
 you. You may also notice that the more you get to know people and get
 to be known, the faster your patches will be reviewed and eventually
 approved. Get to know others and be known by doing code reviews:
 anybody can and should do it.
>>> 
>>> With links to the wiki for more details, of course. This sort of
>>> messaging may help all the people that contribute tactically, those that
>>> are asked by their manager to land a patch in here and are simply
>>> lightly involved (not committed) in OpenStack. These are the ones that
>>> may have an incorrect perception of how easy it is to have patches
>>> landed in OpenStack as opposed to other large projects, like the kernel
>>> or android and complain about our time to traverse the review system.
>>> 
>>> What do you think? How can we instruct gerrit to do this?
>>> 
>> I think this is a really good idea. I've seen occasions were new committers
>> get antsy after waiting a few days (some even a few hours) and wondering
>> why their patch isn't getting reviewed. Something like this would set the
>> expectation for them correctly, and help to guide them to IRC to engage.
> 
> I agree! I think this is an excelent idea, and I think it's totally
> implementable. I'm not sure what all the details will be of that
> implementation, but I'm certain it can be done.
> 
Awesome, thanks Monty!

> We're all crazy with summit - could you file a bug at
> bugs.launchpad.net/openstack-ci so that we don't lose track of it?
> 
Done:

https://bugs.launchpad.net/openstack-ci/+bug/1246879

Thanks,
Kyle

> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] welcoming new committers (was Re: When is it okay for submitters to say 'I don't want to add tests' ?)

[Monty Taylor]

On 10/31/2013 04:15 PM, Kyle Mestery (kmestery) wrote:
> On Oct 31, 2013, at 1:49 PM, Stefano Maffulli  wrote:
>> On 10/31/2013 07:05 AM, Kyle Mestery (kmestery) wrote:
>> [...]
>>> If we want to grow the committer base and help people to become
>>> better reviewers, taking the time to show them the ropes is part of
>>> the game.
>>
>> hijacking the thread using Kyle's comment as an excuse.
>>
> Hey, glad to provide you an opening Stef!
> 
>> It's not an 'if' but a 'since': since we are growing the committer base
>> at an incredible pace we should help them become also good reviewers as
>> rapidly possible.
>>
>> One thing I already mentioned and I'll start doing this week in the
>> weekly Newsletter is to give a shoutout to those that do their first
>> review this week.
>>
>> Another idea that Tom suggested is to use gerrit automation to send back
>> to first time committers something in addition to the normal 'your patch
>> is waiting for review' message. The message could be something like:
>>
>>> thank you for your first contribution to OpenStack. Your patch will
>>> now be tested automatically by OpenStack testing frameworks and once
>>> the automatic tests pass, it will be reviewed by other friendly
>>> developers. They will give you comments and may require you to refine
>>> it.
>>>
>>> Nobody gets his patch approved at first try so don't be concerned
>>> when someone will require you to do more iterations.
>>>
>>> Patches usually take 3 to 7 days to be approved so be patient and be
>>> available on IRC to ask and answer questions about your work. The
>>> more you participate in the community the more rewarding it is for
>>> you. You may also notice that the more you get to know people and get
>>> to be known, the faster your patches will be reviewed and eventually
>>> approved. Get to know others and be known by doing code reviews:
>>> anybody can and should do it.
>>
>> With links to the wiki for more details, of course. This sort of
>> messaging may help all the people that contribute tactically, those that
>> are asked by their manager to land a patch in here and are simply
>> lightly involved (not committed) in OpenStack. These are the ones that
>> may have an incorrect perception of how easy it is to have patches
>> landed in OpenStack as opposed to other large projects, like the kernel
>> or android and complain about our time to traverse the review system.
>>
>> What do you think? How can we instruct gerrit to do this?
>>
> I think this is a really good idea. I've seen occasions were new committers
> get antsy after waiting a few days (some even a few hours) and wondering
> why their patch isn't getting reviewed. Something like this would set the
> expectation for them correctly, and help to guide them to IRC to engage.

I agree! I think this is an excelent idea, and I think it's totally
implementable. I'm not sure what all the details will be of that
implementation, but I'm certain it can be done.

We're all crazy with summit - could you file a bug at
bugs.launchpad.net/openstack-ci so that we don't lose track of it?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] A question about atomicity of repository commits

[Birdsall, Dave]

Hi,

I'm new to the Openstack community. I have what is probably a naïve question.

Suppose source code for a project is in Git repository A, while the Jenkins 
test suite for it is in Git repository B.

First question: Does this happen in Openstack? (For example, are there nova 
tests in the tempest repository that get executed in Jenkins when nova is 
changed?)

Given my assumption above, suppose I'm a contributor and I want to change some 
source in repository A. However, my source change affects the test results of a 
test in repository B. So I want to change that as well.

My understanding is that I'd clone both repositories, and commit changes to 
each independently, and each would go to Gerrit as a separate change.

Second question: How does one handle the chicken-and-egg problem? If the source 
change to A goes through Jenkins first, the automated tests would fail. But if 
the test change to B goes through Jenkins first, it would also fail because the 
old code in A has the old behavior.

Thanks and kind regards,

Dave

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon] Abdicating the PTL Position

[Monty Taylor]

On 10/31/2013 04:06 PM, Gabriel Hurley wrote:
> Hi all,
> 
> It saddens me to say that for a mix of reasons I have decided to
> abdicate my position as PTL for Horizon. If anything, the reasons are
> all good ones overall, I just have to make the right decision for
> both myself and the project.
> 
> In the interim David Lyle will be the acting PTL. The Horizon core
> team has all weighed in with their confidence in his abilities, and
> he has confirmed his ability and interest in doing so. There will be
> a nomination period in coming weeks to determine the PTL for the full
> release cycle, should anyone else wish to run for the job as well.
> Thierry will announce more information about that soon.
> 
> Rest assured, you're not rid of me entirely; I'm just making a
> decision to focus in on other areas for a time. Horizon is blessed to
> have other phenomenal candidates willing and able to lead, and I
> would rather see the project in the hands of someone who can devote
> their full attention and energy to it.
> 
> I will still be around and engaged (though not in Hong Kong). I'll
> catch you all next time around.

Thank you for your amazing work as PTL! I'm glad we're not rid of you
entirely, but I am sad I'll miss you in Hong Kong.

Monty


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] welcoming new committers (was Re: When is it okay for submitters to say 'I don't want to add tests' ?)

[Kyle Mestery (kmestery)]

On Oct 31, 2013, at 1:49 PM, Stefano Maffulli  wrote:
> On 10/31/2013 07:05 AM, Kyle Mestery (kmestery) wrote:
> [...]
>> If we want to grow the committer base and help people to become
>> better reviewers, taking the time to show them the ropes is part of
>> the game.
> 
> hijacking the thread using Kyle's comment as an excuse.
> 
Hey, glad to provide you an opening Stef!

> It's not an 'if' but a 'since': since we are growing the committer base
> at an incredible pace we should help them become also good reviewers as
> rapidly possible.
> 
> One thing I already mentioned and I'll start doing this week in the
> weekly Newsletter is to give a shoutout to those that do their first
> review this week.
> 
> Another idea that Tom suggested is to use gerrit automation to send back
> to first time committers something in addition to the normal 'your patch
> is waiting for review' message. The message could be something like:
> 
>> thank you for your first contribution to OpenStack. Your patch will
>> now be tested automatically by OpenStack testing frameworks and once
>> the automatic tests pass, it will be reviewed by other friendly
>> developers. They will give you comments and may require you to refine
>> it.
>> 
>> Nobody gets his patch approved at first try so don't be concerned
>> when someone will require you to do more iterations.
>> 
>> Patches usually take 3 to 7 days to be approved so be patient and be
>> available on IRC to ask and answer questions about your work. The
>> more you participate in the community the more rewarding it is for
>> you. You may also notice that the more you get to know people and get
>> to be known, the faster your patches will be reviewed and eventually
>> approved. Get to know others and be known by doing code reviews:
>> anybody can and should do it.
> 
> With links to the wiki for more details, of course. This sort of
> messaging may help all the people that contribute tactically, those that
> are asked by their manager to land a patch in here and are simply
> lightly involved (not committed) in OpenStack. These are the ones that
> may have an incorrect perception of how easy it is to have patches
> landed in OpenStack as opposed to other large projects, like the kernel
> or android and complain about our time to traverse the review system.
> 
> What do you think? How can we instruct gerrit to do this?
> 
I think this is a really good idea. I've seen occasions were new committers
get antsy after waiting a few days (some even a few hours) and wondering
why their patch isn't getting reviewed. Something like this would set the
expectation for them correctly, and help to guide them to IRC to engage.

Thanks,
Kyle

> /stef
> 
> PS I put the text on
> https://etherpad.openstack.org/p/welcome-new-committers for refinements.




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] Abdicating the PTL Position

[Gabriel Hurley]

Hi all,

It saddens me to say that for a mix of reasons I have decided to abdicate my 
position as PTL for Horizon. If anything, the reasons are all good ones 
overall, I just have to make the right decision for both myself and the project.

In the interim David Lyle will be the acting PTL. The Horizon core team has all 
weighed in with their confidence in his abilities, and he has confirmed his 
ability and interest in doing so. There will be a nomination period in coming 
weeks to determine the PTL for the full release cycle, should anyone else wish 
to run for the job as well. Thierry will announce more information about that 
soon.

Rest assured, you're not rid of me entirely; I'm just making a decision to 
focus in on other areas for a time. Horizon is blessed to have other phenomenal 
candidates willing and able to lead, and I would rather see the project in the 
hands of someone who can devote their full attention and energy to it.

I will still be around and engaged (though not in Hong Kong). I'll catch you 
all next time around.

All the best,

- Gabriel Hurley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] HK Summit - Image Creation and Customization "unconference"?

[Ian McLeod]

On Thu, 2013-10-31 at 12:50 -0700, Mark Washenberger wrote:
> 
> 
> 
> On Thu, Oct 31, 2013 at 12:20 PM, Ian McLeod 
> wrote:
> On Thu, 2013-10-31 at 11:56 -0700, Clint Byrum wrote:
> > Excerpts from Ian McLeod's message of 2013-10-31 11:27:39
> -0700:
> > > Hello,
> > >
> > > Would any of you attending the summit be interested in
> snagging an
> > > unconference session to discuss the state of play with
> image creation,
> > > customization and import?
> > >
> > > I can contribute an overview and demonstration of our
> Nova-native image
> > > building tool.
> > >
> > > I'd be interested in exploring integration points with
> Disk Image
> > > Builder and Glance.
> > >
> > > Any takers?

> 
> Sounds like a good opportunity!
>  

> 
> So any time Wednesday onward?
> 
> 
> That broad schedule constraint is good for Glance as well, as all of
> our sessions are on Tuesday.

Cool.  I'm not arriving until late Monday.  I've asked russellb to try
to grab an unconference slot.  However, I suppose I should just extend
this request to anyone who is interested.

So, could I ask that any interested parties look at the unconference
signup sheet when you arrive and, if you don't already see:

"Image Building and Customization"

please grab a time slot from Wednesday onward?

-Ian

> 
> 
>  
> 
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> >
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Closes-Bug and Launchpad

[Jeremy Stanley]

On 2013-10-31 12:43:19 -0700 (-0700), Gary Kotton wrote:
> Over the last few days I have noticed that bug fixes posted to
> gerrit are not updated in Launchpad.
[...]

Links to one or two example changes might help. There are a number
of other subtle ways this can break down (wrong branch/no
corresponding series task on the bug, missing bug supervisor
permissions on that project for the Launchpad account used by our
automation, transient API failures due to network or server issues,
et cetera). I can probably spot the specific issue with a bit more
context to go on.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] HK Summit - Image Creation and Customization "unconference"?

[Mark Washenberger]

On Thu, Oct 31, 2013 at 12:20 PM, Ian McLeod  wrote:

> On Thu, 2013-10-31 at 11:56 -0700, Clint Byrum wrote:
> > Excerpts from Ian McLeod's message of 2013-10-31 11:27:39 -0700:
> > > Hello,
> > >
> > > Would any of you attending the summit be interested in snagging an
> > > unconference session to discuss the state of play with image creation,
> > > customization and import?
> > >
> > > I can contribute an overview and demonstration of our Nova-native image
> > > building tool.
> > >
> > > I'd be interested in exploring integration points with Disk Image
> > > Builder and Glance.
> > >
> > > Any takers?
>

Sounds like a good opportunity!


> >
> > I am certain that one of the tripleo-core devs will want to be there,
> > if not many of us. We don't have that many sessions scheduled so I'd
> > suggest just trying to aim at when we're not having Deployment sessions.
>
> Those are exclusively on Tuesday, yes?
>
> http://icehousedesignsummit.sched.org/overview/type/tripleo+%
> 28deployment%29#.UnKs1x8u2al
>
> So any time Wednesday onward?
>

That broad schedule constraint is good for Glance as well, as all of our
sessions are on Tuesday.



>
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Closes-Bug and Launchpad

[Gary Kotton]

Hi,
Over the last few days I have noticed that bug fixes posted to gerrit are not 
updated in Launchpad. Am I doing something wrong? I think that the commit 
message is the correct format: Closes-Bug: #.
Any ideas?
Thanks
Gary
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa][keystone] Help with XML Tempest API tests

[Steven Hardy]

Hi all,

So firstly, if you're an XML guru, I apologize, the questions below are
probably really basic, I always prefer JSON or YAML, because every time I
deal with XML, I get a week-long-headache ;)

So I'm writing Tempest API tests for the keystone OS-TRUST extension, as
was previously requested - it's going pretty well, & I'm finding some real
bugs.  Here's a WIP review:

https://review.openstack.org/#/c/54810/

However, I've got a bit stuck formatting the POST body for the trust create
in XML (all works fine via JSON).  The json body looks like:

{
"trust": {
"expires_at": "2013-02-27T18:30:59.99Z",
"impersonation": true,
"project_id": "ddef321",
"roles": [
{
"name": "member"
}
],
"trustee_user_id": "86c0d5",
"trustor_user_id": "a0fdfd"
}
}

And looking at other XML tests which format a POST body, they do something
like:

post_body = Element("trust",
xmlns=XMLNS,
trustor_user_id=trustor_user_id,
trustee_user_id=trustee_user_id,
project_id=project_id,
impersonation=impersonation,
expires_at=expires_at)

This gives me a post body which looks weird (but seems to work):

http://docs.openstack.org/identity/api/v3";
trustor_user_id="efc6504105c54fbe95928a51459d06c9" expires_at=""
trustee_user_id="f55efd1d617e4367891d202a811d7728"
project_id="b5d498f9631244b59912ce2a0025cf8d"/>


So my questions are:
1. Why do we create a single element like this, instead of appending
subelements so the XML body looks more like the JSON request?

2. If any elements have a None value, they are encoded as a zero-length
string, is that expected?

3. How do I go about encoding the list of roles, as in the sample request
(it's a list of dicts, where each dict has one key called "name")

Any help, review feedback or pointers to docs/examples would be hugely
appreciated!

Thanks,

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [sqlalchemy-migrate] Blueprint for review: Add DB2 10.5 Support

[Matt Riedemann]

I've got a sqlalchemy-migrate blueprint up for review to add DB2 support 
in migrate.

https://blueprints.launchpad.net/sqlalchemy-migrate/+spec/add-db2-support 

This is a pre-req for getting DB2 support into Nova so I'm targeting 
icehouse-1.  We've been running with the migrate patches internally since 
Folsom, but getting them into migrate was difficult before OpenStack took 
over maintenance of the project.

Please let me know if there are any questions/issues or something I need 
to address here.

Thanks,

Matt Riedemann
Cloud Solutions and OpenStack Development
Email: mrie...@us.ibm.com
Office Phone: 507-253-7622___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Jeremy Stanley]

On 2013-10-31 13:30:32 + (+), Mark McLoughlin wrote:
[...]
> In cases like that, I'd be of a mind to go "+2 Awesome! Thanks for
> catching this! It would be great to have a unit test for this, but
> it's clear the current code is broken so I'm fine with merging the
> fix without a test". You could say it's now the reviewers
> responsibility to merge a test, but if that requirement then turns
> off reviewers even reviewing such a patch, then that doesn't help
> either.
[...]

I get the impression it was one of my +2s in a situation like this
which spawned the thread (or at least was the straw which broke the
camel's back), so apologies for stirring the beehive. Someone was
trying to set up their own copy of several of our infrastructure
projects, spotted a place in one where we had hard-coded something
specific to our environment, and wanted to parameterize it upstream
rather than paper over it on their end.

The bug-reporter-turned-patch-contributor didn't know how to write a
test which would confirm that parameter got passed through and we
weren't performing tests yet for any similar parameters already in
the daemon which I could point to as an example. I agree it's a
judgement call between risking discouraging a new contributor vs.
reducing test coverage further, but I was probably still a bit too
hasty to suggest that we could add tests for those in a separate
commit.

In my case I didn't have the available time to instruct the
contributor on how to write tests for this, but that also probably
meant that I didn't really have time to be reviewing the change
properly to begin with. I'm quite grateful to Robert for stepping in
and helping walk them through it! We got more tests, I think they
got a lot of benefit from the new experience as well, and I was
appropriately humbled for my lax attitude over the situation which
nearly allowed us to miss a great opportunity at educating another
developer on the merits of test coverage.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Sandy Walsh]

On 10/30/2013 11:37 PM, Robert Collins wrote:
> This is a bit of a social norms thread
> 
> I've been consistently asking for tests in reviews for a while now,
> and I get the occasional push-back. I think this falls into a few
> broad camps:
> 
> A - there is no test suite at all, adding one in unreasonable
> B - this thing cannot be tested in this context (e.g. functional tests
> are defined in a different tree)
> C - this particular thing is very hard to test
> D - testing this won't offer benefit
> E - other things like this in the project don't have tests
> F - submitter doesn't know how to write tests
> G - submitter doesn't have time to write tests
> 
> Now, of these, I think it's fine not add tests in cases A, B, C in
> combination with D, and D.
> 
> I don't think E, F or G are sufficient reasons to merge something
> without tests, when reviewers are asking for them. G in the special
> case that the project really wants the patch landed - but then I'd
> expect reviewers to not ask for tests or to volunteer that they might
> be optional.
> 
> Now, if I'm wrong, and folk have different norms about when to accept
> 'reason X not to write tests' as a response from the submitter -
> please let me know!

I've done a lot of thinking around this topic [1][2] and really it comes
down to this: everything can be tested and should be.

There is an argument to A, but that goes beyond the scope of our use
case I think. If I hear B, I would suspect the tests aren't unit tests,
but are functional/integration tests (a common problem in OpenStack).
Functional tests are brittle and usually have painful setup sequences.
The other cases fall into the -1 camp for me. Tests required.

That said, recently I was -1'ed for not updating a test, because I added
code that didn't change the program flow, but introduced a new call.
According to my rules, that didn't need a test, but I agreed with the
logic that people would be upset if the call wasn't made (it was a
notification). So a test was added. Totally valid argument.

TL;DR: Tests are always required. We need to fix our tests to be proper
unit tests and not functional/integration tests so it's easy to add new
ones.

-S


[1]
http://www.sandywalsh.com/2011/06/effective-units-tests-and-integration.html
[2]
http://www.sandywalsh.com/2011/08/pain-of-unit-tests-and-dynamically.html


> -Rob
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Migrating to newer full projects from what used to be part of nova

[Jesse Pretorius]

On 31 October 2013 19:05, Russell Bryant  wrote:

> Right, Nova (and probably other projects, too) only supports N to N+1
> upgrades.  If you need to go to N+2, you'll need to go through N+1 to
> get there.
>

Dead right, and especially with the aggressive schedule we have this is the
only sane approach for now.

FWIW, improving upgrades is very high on the priority list across all of
> OpenStack.  We have sessions all over the place to talk about it.  It's
> getting a lot better.
>

Yes it is, and while the upgrades haven't been perfect so far the increased
focus on getting the grenade testing in place are sure to make a big
difference. I just feel that it's important to highlight the gap to bring
it into the same discussion and hopefully raise some awareness for those
who are in a prime position to make the difference.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] HK Summit - Image Creation and Customization "unconference"?

[Ian McLeod]

On Thu, 2013-10-31 at 11:56 -0700, Clint Byrum wrote:
> Excerpts from Ian McLeod's message of 2013-10-31 11:27:39 -0700:
> > Hello,
> > 
> > Would any of you attending the summit be interested in snagging an
> > unconference session to discuss the state of play with image creation,
> > customization and import?
> > 
> > I can contribute an overview and demonstration of our Nova-native image
> > building tool.
> > 
> > I'd be interested in exploring integration points with Disk Image
> > Builder and Glance.
> > 
> > Any takers?
> 
> I am certain that one of the tripleo-core devs will want to be there,
> if not many of us. We don't have that many sessions scheduled so I'd
> suggest just trying to aim at when we're not having Deployment sessions.

Those are exclusively on Tuesday, yes?

http://icehousedesignsummit.sched.org/overview/type/tripleo+%
28deployment%29#.UnKs1x8u2al

So any time Wednesday onward?

> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] Locking and ZooKeeper - a space oddysey

[Joshua Harlow]

Agreed, 

I don't think we should enforce a hard requirement for the same reason we
should avoid zookeeper.

To me they are the same, saying 'no ZK' is the same as saying 'only mysql
or rabbitmq'...

Both not so open.

On 10/31/13 11:04 AM, "Monty Taylor"  wrote:

>
>
>On 10/31/2013 01:32 PM, Joshua Harlow wrote:
>> In the spirt of openness, yes I do think they are all needed.
>> 
>> If they are not supported, then openstack is not open, it is a closed
>> system.
>> 
>> We should strive to innovate, not strive to be stuck with the status
>>quo.
>> 
>> To me it is a developers decision to pick the right solution, if that
>> solution involves some complexity then u make it a pluggable solution.
>> Your view of the right solution will likely not be the view of someone
>> elses right solution in the end anyway (and u likely can't predict
>>future
>> solutions that will be applicable anyway). If u just say no to that
>>plugin
>> then u are just excluding people from participating in your project and
>> this is imho against the spirt of openness in general. And those people
>> who would have contributed will just start looking elsewhere for a
>> solution which does work. This kills the openstack...
>
>Hrm. I certainly don't want to kill openstack, and I _certainly_ don't
>want to disallow a plugin. I apologize if it came across that way.
>
>What I was questioning is whether or not we wanted to add a hard
>requirement on zookeeper somewhere. Even Rabbit and MySQL are decisions
>with other options. It's entirely possible I misread the conversation of
>course...
>
>> On 10/31/13 10:17 AM, "Monty Taylor"  wrote:
>> 
>>> Sigh.
>>>
>>> Yay We've added more competing methods of complexity!!!
>>>
>>> Seriously. We now think that rabbit and zookeeper and mysql are ALL
>>> needed?
>>>
>>> Joshua Harlow  wrote:
>>>
 I'm pretty sure the cats out of the bag.

 
https://github.com/openstack/requirements/blob/master/global-requiremen
ts
 .t
 xt#L29

 https://kazoo.readthedocs.org/en/latest/

 -Josh

 On 10/31/13 7:43 AM, "Monty Taylor"  wrote:

>
>
> On 10/30/2013 10:42 AM, Clint Byrum wrote:
>> So, recently we've had quite a long thread in gerrit regarding
>>locking
>> in Heat:
>>
>> https://review.openstack.org/#/c/49440/
>>
>> In the patch, there are two distributed lock drivers. One uses SQL,
>> and suffers from all the problems you might imagine a SQL based
>> locking
>> system would. It is extremely hard to detect dead lock holders, so
>>we
>> end up with really long timeouts. The other is ZooKeeper.
>>
>> I'm on record as saying we're not using ZooKeeper. It is a little
>> embarrassing to have taken such a position without really thinking
>> things
>> through. The main reason I feel this way though, is not because
>> ZooKeeper
>> wouldn't work for locking, but because I think locking is a mistake.
>>
>> The current multi-engine paradigm has a race condition. If you have
>>a
>> stack action going on, the state is held in the engine itself, and
>>not
>> in the database, so if another engine starts working on another
>> action,
>> they will conflict.
>>
>> The locking paradigm is meant to prevent this. But I think this is a
>> huge mistake.
>>
>> The engine should store _all_ of its state in a distributed data
>>store
>> of some kind. Any engine should be aware of what is already
>>happening
>> with the stack from this state and act accordingly. That includes
>>the
>> engine currently working on actions. When viewed through this lense,
>> to me, locking is a poor excuse for serializing the state of the
>> engine
>> scheduler.
>>
>> It feels like TaskFlow is the answer, with an eye for making sure
>> TaskFlow can be made to work with distributed state. I am not well
>> versed on TaskFlow's details though, so I may be wrong. It worries
>>me
>> that TaskFlow has existed a while and doesn't seem to be solving
>>real
>> problems, but maybe I'm wrong and it is actually in use already.
>>
>> Anyway, as a band-aid, we may _have_ to do locking. For that,
>> ZooKeeper
>> has some real advantages over using the database. But there is
>> hesitance
>> because it is not widely supported in OpenStack. What say you,
>> OpenStack
>> community? Should we keep ZooKeeper out of our.. zoo?
>
> Yes. I'm strongly opposed to ZooKeeper finding its way into the
>already
> complex pile of things we use.
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>> 
>> 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://l

Re: [openstack-dev] Migrating to newer full projects from what used to be part of nova

[Jesse Pretorius]

On 31 October 2013 18:46, John Griffith  wrote:

> Is your disappointment that there isn't a path from Essex -->
> Grizzly/Havana, or are you unhappy with the content?


Upgrading Essex->Folsom introduced both the challenge of upgrading
nova-volume to cinder and the challenge of upgrading nova-network to
quantum. Upgrading Folsom->Grizzly presents the challenge of migrating
nova-network to quantum and assumes that nova-volume has already been
migrated to cinder. Upgrading Grizzly->Havana finally closes the door on
nova-network as far as I can see, although I may be wrong.

It comes down to the fact that while we cater for migrating between
versions of a particular project we don't cater particularly well for
migrating between projects when a project splits out from a parent project
as was the case for both of the above.

I'm not disappointed about the lack of ability to jump from Essex to
Grizzly or Havana, but rather the lack of ability to migrate between
projects.

I would find it entirely suitable to upgrade from Essex to Folsom, then
migrate from nova-volume to cinder and from nova-network to quantum, then
only to upgrade to Grizzly.


> So to be quite honest we never intended to make skips like you
> describe.


This misses the point somewhat, as I've made (hopefully) clear above. It's
not about the skipping, but rather the clean migration, even within the
same version, from something that used to be inherent (like nova-volume and
cinder) to something that is now a new project with its own code tree.


> Going forward maybe we can come up with something better.
>

The reason for raising this now is primarily to do just that.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Michael Bright]

I like Steve's suggestion:

> The approach the Heat team have sometimes taken in this situation is to
> merge the patch, but raise a bug (targetted at the next milestone)
> identifying the missing coverage.

I'm (almost!) a first time contributor and I've put a fix on the backburner
as I find the time to ramp up on the unit tests suite.  The fix was
"review"ed 2 weeks ago, requesting unit tests - very reasonable but I
needed help to even start (I got no response from requests on IRC, ML, or
e-mail to the bug reporter).

If it wasn't for the good Upstream University people mentoring me - heh,
they deserve a plug! - I'm sure I would have moved on.

Thankfully, a cunning commit message provoked the guidance I needed so I
just need a long weekend to get back to the tests - which I have now.

I'm not sure in which cases you would/wouldn't want to split the bug for
the implementation of tests but I'm sure it would help other first timers
and encourage new contributors.

Regards,
Mike





On 31 October 2013 19:51, Steven Hardy  wrote:

> On Thu, Oct 31, 2013 at 01:30:32PM +, Mark McLoughlin wrote:
> > On Thu, 2013-10-31 at 15:37 +1300, Robert Collins wrote:
> > > This is a bit of a social norms thread
> > >
> > > I've been consistently asking for tests in reviews for a while now,
> > > and I get the occasional push-back. I think this falls into a few
> > > broad camps:
> > >
> > > A - there is no test suite at all, adding one in unreasonable
> > > B - this thing cannot be tested in this context (e.g. functional tests
> > > are defined in a different tree)
> > > C - this particular thing is very hard to test
> > > D - testing this won't offer benefit
> > > E - other things like this in the project don't have tests
> > > F - submitter doesn't know how to write tests
> > > G - submitter doesn't have time to write tests
> >
> > Nice breakdown.
> >
> > > Now, of these, I think it's fine not add tests in cases A, B, C in
> > > combination with D, and D.
> > >
> > > I don't think E, F or G are sufficient reasons to merge something
> > > without tests, when reviewers are asking for them. G in the special
> > > case that the project really wants the patch landed - but then I'd
> > > expect reviewers to not ask for tests or to volunteer that they might
> > > be optional.
> >
> > I totally agree with the sentiment but, especially when it's a newcomer
> > to the project, I try to put myself in the shoes of the patch submitter
> > and double-check whether what we're asking is reasonable.
> >
> > For example, if someone shows up to Nova with their first OpenStack
> > contribution, it fixes something which is unquestionably a bug - think
> > typo like "raise NotFund('foo')" - and testing this code patch requires
> > more than adding a simple new scenario to existing tests ...
> >
> > That, for me, is an example of "-1, we need a test! untested code is
> > broken!" is really shooting the messenger, not valuing the newcomers
> > contribution and risks turning that person off the project forever.
> > Reviewers being overly aggressive about this where the project doesn't
> > have full test coverage to begin with really makes us seem unwelcoming.
>
> The approach the Heat team have sometimes taken in this situation is to
> merge the patch, but raise a bug (targetted at the next milestone)
> identifying the missing coverage.
>
> This has a few benefits (provided the patch in question is sufficiently
> simple that patches aren't deemed essential)
> - The bug gets fixed quickly
> - The coverage still gets added, and we keep track of the gaps identified
> - Less chance of discouraging new submitters
> - Provides some "low hanging fruit" bugs, which new contributors can pick
>   up
>
> I'm not saying we do this routinely, but it's a possible middle ground to
> consider between "-1 fix all our tests" and "+2 shipit!"
>
> I think something that's important to recognise is that sometimes (often?)
> writing tests is *hard*.  I mean, look at the barriers to entry, you need
> to have some idea about tox, nose, testr, mox, mock, testscenarios, etc etc
>
> The learning curve is really steep, and thats before you start considering
> project specific test abstractions/fixtures/mocking-patterns which can all
> be complex and non-obvious.
>
> So +1 on a bit of compassion when reviewing, particularly if the
> contributor is a new or casual contributor to the project.
>
> Steve
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] HK Summit - Image Creation and Customization "unconference"?

[Clint Byrum]

Excerpts from Ian McLeod's message of 2013-10-31 11:27:39 -0700:
> Hello,
> 
> Would any of you attending the summit be interested in snagging an
> unconference session to discuss the state of play with image creation,
> customization and import?
> 
> I can contribute an overview and demonstration of our Nova-native image
> building tool.
> 
> I'd be interested in exploring integration points with Disk Image
> Builder and Glance.
> 
> Any takers?

I am certain that one of the tripleo-core devs will want to be there,
if not many of us. We don't have that many sessions scheduled so I'd
suggest just trying to aim at when we're not having Deployment sessions.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Paul Belanger]

On 13-10-30 10:37 PM, Robert Collins wrote:

This is a bit of a social norms thread

I've been consistently asking for tests in reviews for a while now,
and I get the occasional push-back. I think this falls into a few
broad camps:

A - there is no test suite at all, adding one in unreasonable
B - this thing cannot be tested in this context (e.g. functional tests
are defined in a different tree)
C - this particular thing is very hard to test
D - testing this won't offer benefit
E - other things like this in the project don't have tests
F - submitter doesn't know how to write tests
G - submitter doesn't have time to write tests

Now, of these, I think it's fine not add tests in cases A, B, C in
combination with D, and D.

I don't think E, F or G are sufficient reasons to merge something
without tests, when reviewers are asking for them. G in the special
case that the project really wants the patch landed - but then I'd
expect reviewers to not ask for tests or to volunteer that they might
be optional.

Now, if I'm wrong, and folk have different norms about when to accept
'reason X not to write tests' as a response from the submitter -
please let me know!

-Rob

I recently hit option A for nodepool.  My patch was accepted, but I 
didn't know where to start for adding the testsuite for the project.


So, that said, if option A keeps coming up, then I think the obvious 
choice is development needs to refocus to take the option of the table.


--
Paul Belanger | PolyBeacon, Inc.
Jabber: paul.belan...@polybeacon.com | IRC: pabelanger (Freenode)
Github: https://github.com/pabelanger | Twitter: 
https://twitter.com/pabelanger


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Steven Hardy]

On Thu, Oct 31, 2013 at 01:30:32PM +, Mark McLoughlin wrote:
> On Thu, 2013-10-31 at 15:37 +1300, Robert Collins wrote:
> > This is a bit of a social norms thread
> > 
> > I've been consistently asking for tests in reviews for a while now,
> > and I get the occasional push-back. I think this falls into a few
> > broad camps:
> > 
> > A - there is no test suite at all, adding one in unreasonable
> > B - this thing cannot be tested in this context (e.g. functional tests
> > are defined in a different tree)
> > C - this particular thing is very hard to test
> > D - testing this won't offer benefit
> > E - other things like this in the project don't have tests
> > F - submitter doesn't know how to write tests
> > G - submitter doesn't have time to write tests
> 
> Nice breakdown.
> 
> > Now, of these, I think it's fine not add tests in cases A, B, C in
> > combination with D, and D.
> > 
> > I don't think E, F or G are sufficient reasons to merge something
> > without tests, when reviewers are asking for them. G in the special
> > case that the project really wants the patch landed - but then I'd
> > expect reviewers to not ask for tests or to volunteer that they might
> > be optional.
> 
> I totally agree with the sentiment but, especially when it's a newcomer
> to the project, I try to put myself in the shoes of the patch submitter
> and double-check whether what we're asking is reasonable.
> 
> For example, if someone shows up to Nova with their first OpenStack
> contribution, it fixes something which is unquestionably a bug - think
> typo like "raise NotFund('foo')" - and testing this code patch requires
> more than adding a simple new scenario to existing tests ...
> 
> That, for me, is an example of "-1, we need a test! untested code is
> broken!" is really shooting the messenger, not valuing the newcomers
> contribution and risks turning that person off the project forever.
> Reviewers being overly aggressive about this where the project doesn't
> have full test coverage to begin with really makes us seem unwelcoming.

The approach the Heat team have sometimes taken in this situation is to
merge the patch, but raise a bug (targetted at the next milestone)
identifying the missing coverage.

This has a few benefits (provided the patch in question is sufficiently
simple that patches aren't deemed essential)
- The bug gets fixed quickly
- The coverage still gets added, and we keep track of the gaps identified
- Less chance of discouraging new submitters
- Provides some "low hanging fruit" bugs, which new contributors can pick
  up

I'm not saying we do this routinely, but it's a possible middle ground to
consider between "-1 fix all our tests" and "+2 shipit!"

I think something that's important to recognise is that sometimes (often?)
writing tests is *hard*.  I mean, look at the barriers to entry, you need
to have some idea about tox, nose, testr, mox, mock, testscenarios, etc etc

The learning curve is really steep, and thats before you start considering
project specific test abstractions/fixtures/mocking-patterns which can all
be complex and non-obvious.

So +1 on a bit of compassion when reviewing, particularly if the
contributor is a new or casual contributor to the project.

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] welcoming new committers (was Re: When is it okay for submitters to say 'I don't want to add tests' ?)

[Stefano Maffulli]

On 10/31/2013 07:05 AM, Kyle Mestery (kmestery) wrote:
[...]
> If we want to grow the committer base and help people to become
> better reviewers, taking the time to show them the ropes is part of
> the game.

hijacking the thread using Kyle's comment as an excuse.

It's not an 'if' but a 'since': since we are growing the committer base
at an incredible pace we should help them become also good reviewers as
rapidly possible.

One thing I already mentioned and I'll start doing this week in the
weekly Newsletter is to give a shoutout to those that do their first
review this week.

Another idea that Tom suggested is to use gerrit automation to send back
to first time committers something in addition to the normal 'your patch
is waiting for review' message. The message could be something like:

> thank you for your first contribution to OpenStack. Your patch will
> now be tested automatically by OpenStack testing frameworks and once
> the automatic tests pass, it will be reviewed by other friendly
> developers. They will give you comments and may require you to refine
> it.
> 
> Nobody gets his patch approved at first try so don't be concerned
> when someone will require you to do more iterations.
> 
> Patches usually take 3 to 7 days to be approved so be patient and be
> available on IRC to ask and answer questions about your work. The
> more you participate in the community the more rewarding it is for
> you. You may also notice that the more you get to know people and get
> to be known, the faster your patches will be reviewed and eventually
> approved. Get to know others and be known by doing code reviews:
> anybody can and should do it.

With links to the wiki for more details, of course. This sort of
messaging may help all the people that contribute tactically, those that
are asked by their manager to land a patch in here and are simply
lightly involved (not committed) in OpenStack. These are the ones that
may have an incorrect perception of how easy it is to have patches
landed in OpenStack as opposed to other large projects, like the kernel
or android and complain about our time to traverse the review system.

What do you think? How can we instruct gerrit to do this?

/stef

PS I put the text on
https://etherpad.openstack.org/p/welcome-new-committers for refinements.

-- 
Ask and answer questions on https://ask.openstack.org

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Clint Byrum]

Excerpts from Khanh-Toan Tran's message of 2013-10-31 07:22:06 -0700:
> Hi all,
> 
> As a newbie of the community, I'm not familiar with unittest and how to use 
> it here. I've learned that Jenkins runs tests
> everytime we submit some code. But how to write the test and what is a 'good 
> test' and a 'bad test'? I saw some commits
> in gerrit but am unable to say if the written test is enough to judge the 
> code, since it is the author of the code who writes
> the test. Is there a framework to follow or some rules/pratices to respect?
> 
> Do you have some links to help me out?
> 

This is a nice synopsis of the concept of test driven development:

http://net.tutsplus.com/tutorials/python-tutorials/test-driven-development-in-python/

In OpenStack we always put tests in  _base_module_name_/tests, So if you
are working on nova, you can see the unit tests in:

nova/tests

You can generally always run the tests by installing the 'tox' python
module/command on your system and running 'tox' in the root of the git
repository.

Projects use various testing helpers to make tests easier to read and
write. The most common one is testtools. A typical test will look like
this:


import testtools

from basemodule import submodule


class TestSubmoduleFoo(testtools.TestCase):
def test_foo_apple(self):
self.assertEquals(1, submodule.foo('apple'))

def test_foo_banana(self):
self.assertEquals(0, submodule.foo('banana'))


Often unit tests will include mocks and fakes to hide real world
interfacing code from the unit tests. You would do well to read up on
how those concepts work as well, google for 'python test mocking' and
'python test fakes'.

Good luck, and #openstack-dev is always there to try and help. :)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Solum] Call for participation: Breakout meeting for API deep-dive

[Adrian Otto]

Team,

In the interest of providing adequate advance notice, we will not consider any 
of the timeslots for today. I have edited the poll accordingly. It also looks 
like a number of us did not notice that there are a lot of proposed times 
hiding under the accordion view of the poll.

So, please take a moment to re-visit the poll, and click the "Show all 18 
options" button, and please mark the (If-Needbe) options if you have 
conflicting commitments that you would be willing to reschedule to attend the 
breakout meeting(s).

Note: No date/time options were proposed for the week of the Icehouse OpenStack 
Design Summit next week in Hong Kong. Most of the proposed times are the week 
following next.

Note: I plan to close the poll at 5:00 PM US/Pacific today and select the 
date/time for these based on the feedback you provide. Please chime in now so 
we can do our best to include you.

Thanks,

Adrian

On Oct 30, 2013, at 6:03 PM, Adrian Otto  wrote:

> Team,
> 
> Those of us interested in participating in a series of API deep-dive design 
> discussions, are encouraged to provide date/time scheduling feedback at the 
> link shown on our Breakout Meetings wiki page:
> 
> https://wiki.openstack.org/wiki/Solum/BreakoutMeetings
> 
> We look forward to your input to select the date/time of this series. 
> 
> Thanks,
> 
> Adrian
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat] [savanna] [trove] Place for software configuration

[Jay Pipes]

On 10/31/2013 01:51 PM, Alexander Kuznetsov wrote:

Hi Heat, Savanna and Trove teams,

All this projects have common part related to software configuration
management.  For creation,  an environment  user should specify a
hardware parameter for vms:  choose flavor, decide use cinder or not,
configure networks for virtual machines, choose topology for hole
deployment. Next step is linking of software parameters with hardware
specification. From the end user point of view, existence of three
different places and three different ways (HEAT Hot DSL, Trove
clustering API and Savanna Hadoop templates) for software configuration
is not convenient, especially if user want to create an environment
simultaneously involving components from Savanna, Heat and Trove.

I can suggest two approaches to overcome this situations:

Common library in oslo. This approach allows a deep domain specific
customization. The user will still have 3 places with same UI where user
should perform configuration actions.

Heat or some other component for software configuration management. This
approach is the best for end users. In feature possible will be some
limitation on deep domain specific customization for configuration
management.


I think this would be my preference.

In other words, describe and orchestrate a Hadoop or Database setup 
using HOT templates and using Heat as the orchestration engine.


Best,
-jay


Heat, Savanna and Trove teams can you comment these ideas, what approach
are the best?

Alexander Kuznetsov.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] HK Summit - Image Creation and Customization "unconference"?

[Ian McLeod]

Hello,

Would any of you attending the summit be interested in snagging an
unconference session to discuss the state of play with image creation,
customization and import?

I can contribute an overview and demonstration of our Nova-native image
building tool.

I'd be interested in exploring integration points with Disk Image
Builder and Glance.

Any takers?

-Ian


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] Locking and ZooKeeper - a space oddysey

[Monty Taylor]

On 10/31/2013 01:32 PM, Joshua Harlow wrote:
> In the spirt of openness, yes I do think they are all needed.
> 
> If they are not supported, then openstack is not open, it is a closed
> system.
> 
> We should strive to innovate, not strive to be stuck with the status quo.
> 
> To me it is a developers decision to pick the right solution, if that
> solution involves some complexity then u make it a pluggable solution.
> Your view of the right solution will likely not be the view of someone
> elses right solution in the end anyway (and u likely can't predict future
> solutions that will be applicable anyway). If u just say no to that plugin
> then u are just excluding people from participating in your project and
> this is imho against the spirt of openness in general. And those people
> who would have contributed will just start looking elsewhere for a
> solution which does work. This kills the openstack...

Hrm. I certainly don't want to kill openstack, and I _certainly_ don't
want to disallow a plugin. I apologize if it came across that way.

What I was questioning is whether or not we wanted to add a hard
requirement on zookeeper somewhere. Even Rabbit and MySQL are decisions
with other options. It's entirely possible I misread the conversation of
course...

> On 10/31/13 10:17 AM, "Monty Taylor"  wrote:
> 
>> Sigh.
>>
>> Yay We've added more competing methods of complexity!!!
>>
>> Seriously. We now think that rabbit and zookeeper and mysql are ALL
>> needed?
>>
>> Joshua Harlow  wrote:
>>
>>> I'm pretty sure the cats out of the bag.
>>>
>>> https://github.com/openstack/requirements/blob/master/global-requirements
>>> .t
>>> xt#L29
>>>
>>> https://kazoo.readthedocs.org/en/latest/
>>>
>>> -Josh
>>>
>>> On 10/31/13 7:43 AM, "Monty Taylor"  wrote:
>>>


 On 10/30/2013 10:42 AM, Clint Byrum wrote:
> So, recently we've had quite a long thread in gerrit regarding locking
> in Heat:
>
> https://review.openstack.org/#/c/49440/
>
> In the patch, there are two distributed lock drivers. One uses SQL,
> and suffers from all the problems you might imagine a SQL based
> locking
> system would. It is extremely hard to detect dead lock holders, so we
> end up with really long timeouts. The other is ZooKeeper.
>
> I'm on record as saying we're not using ZooKeeper. It is a little
> embarrassing to have taken such a position without really thinking
> things
> through. The main reason I feel this way though, is not because
> ZooKeeper
> wouldn't work for locking, but because I think locking is a mistake.
>
> The current multi-engine paradigm has a race condition. If you have a
> stack action going on, the state is held in the engine itself, and not
> in the database, so if another engine starts working on another
> action,
> they will conflict.
>
> The locking paradigm is meant to prevent this. But I think this is a
> huge mistake.
>
> The engine should store _all_ of its state in a distributed data store
> of some kind. Any engine should be aware of what is already happening
> with the stack from this state and act accordingly. That includes the
> engine currently working on actions. When viewed through this lense,
> to me, locking is a poor excuse for serializing the state of the
> engine
> scheduler.
>
> It feels like TaskFlow is the answer, with an eye for making sure
> TaskFlow can be made to work with distributed state. I am not well
> versed on TaskFlow's details though, so I may be wrong. It worries me
> that TaskFlow has existed a while and doesn't seem to be solving real
> problems, but maybe I'm wrong and it is actually in use already.
>
> Anyway, as a band-aid, we may _have_ to do locking. For that,
> ZooKeeper
> has some real advantages over using the database. But there is
> hesitance
> because it is not widely supported in OpenStack. What say you,
> OpenStack
> community? Should we keep ZooKeeper out of our.. zoo?

 Yes. I'm strongly opposed to ZooKeeper finding its way into the already
 complex pile of things we use.

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
> 
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Savanna] Release 0.3 retrospective

[Alexander Ignatov]

Hi,

Here is the wiki page with Savanna release 0.3 retrospective:

https://wiki.openstack.org/wiki/Savanna/Release_0.3_Retrospective

Thanks everyone who sent your opinions. If someone wants to add more
thoughts you are welcome to edit above page!

--
Regards,
Alexander Ignatov
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [heat] [savanna] [trove] Place for software configuration

[Alexander Kuznetsov]

Hi Heat, Savanna and Trove teams,

All this projects have common part related to software configuration
management.  For creation,  an environment  user should specify a hardware
parameter for vms:  choose flavor, decide use cinder or not, configure
networks for virtual machines, choose topology for hole deployment. Next
step is linking of software parameters with hardware specification. From
the end user point of view, existence of three different places and three
different ways (HEAT Hot DSL, Trove clustering API and Savanna Hadoop
templates) for software configuration is not convenient, especially if user
want to create an environment simultaneously involving components from
Savanna, Heat and Trove.

I can suggest two approaches to overcome this situations:

Common library in oslo. This approach allows a deep domain specific
customization. The user will still have 3 places with same UI where user
should perform configuration actions.

Heat or some other component for software configuration management. This
approach is the best for end users. In feature possible will be some
limitation on deep domain specific customization for configuration
management.

Heat, Savanna and Trove teams can you comment these ideas, what approach
are the best?

Alexander Kuznetsov.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] Locking and ZooKeeper - a space oddysey

[Joshua Harlow]

In the spirt of openness, yes I do think they are all needed.

If they are not supported, then openstack is not open, it is a closed
system.

We should strive to innovate, not strive to be stuck with the status quo.

To me it is a developers decision to pick the right solution, if that
solution involves some complexity then u make it a pluggable solution.
Your view of the right solution will likely not be the view of someone
elses right solution in the end anyway (and u likely can't predict future
solutions that will be applicable anyway). If u just say no to that plugin
then u are just excluding people from participating in your project and
this is imho against the spirt of openness in general. And those people
who would have contributed will just start looking elsewhere for a
solution which does work. This kills the openstack...

On 10/31/13 10:17 AM, "Monty Taylor"  wrote:

>Sigh.
>
>Yay We've added more competing methods of complexity!!!
>
>Seriously. We now think that rabbit and zookeeper and mysql are ALL
>needed?
>
>Joshua Harlow  wrote:
>
>>I'm pretty sure the cats out of the bag.
>>
>>https://github.com/openstack/requirements/blob/master/global-requirements
>>.t
>>xt#L29
>>
>>https://kazoo.readthedocs.org/en/latest/
>>
>>-Josh
>>
>>On 10/31/13 7:43 AM, "Monty Taylor"  wrote:
>>
>>>
>>>
>>>On 10/30/2013 10:42 AM, Clint Byrum wrote:
 So, recently we've had quite a long thread in gerrit regarding locking
 in Heat:
 
 https://review.openstack.org/#/c/49440/
 
 In the patch, there are two distributed lock drivers. One uses SQL,
 and suffers from all the problems you might imagine a SQL based
locking
 system would. It is extremely hard to detect dead lock holders, so we
 end up with really long timeouts. The other is ZooKeeper.
 
 I'm on record as saying we're not using ZooKeeper. It is a little
 embarrassing to have taken such a position without really thinking
things
 through. The main reason I feel this way though, is not because
ZooKeeper
 wouldn't work for locking, but because I think locking is a mistake.
 
 The current multi-engine paradigm has a race condition. If you have a
 stack action going on, the state is held in the engine itself, and not
 in the database, so if another engine starts working on another
action,
 they will conflict.
 
 The locking paradigm is meant to prevent this. But I think this is a
 huge mistake.
 
 The engine should store _all_ of its state in a distributed data store
 of some kind. Any engine should be aware of what is already happening
 with the stack from this state and act accordingly. That includes the
 engine currently working on actions. When viewed through this lense,
 to me, locking is a poor excuse for serializing the state of the
engine
 scheduler.
 
 It feels like TaskFlow is the answer, with an eye for making sure
 TaskFlow can be made to work with distributed state. I am not well
 versed on TaskFlow's details though, so I may be wrong. It worries me
 that TaskFlow has existed a while and doesn't seem to be solving real
 problems, but maybe I'm wrong and it is actually in use already.
 
 Anyway, as a band-aid, we may _have_ to do locking. For that,
ZooKeeper
 has some real advantages over using the database. But there is
hesitance
 because it is not widely supported in OpenStack. What say you,
OpenStack
 community? Should we keep ZooKeeper out of our.. zoo?
>>>
>>>Yes. I'm strongly opposed to ZooKeeper finding its way into the already
>>>complex pile of things we use.
>>>
>>>___
>>>OpenStack-dev mailing list
>>>OpenStack-dev@lists.openstack.org
>>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Ben Nemec]

On 2013-10-31 09:05, Kyle Mestery (kmestery) wrote:

On Oct 31, 2013, at 8:56 AM, Clint Byrum  wrote:

Excerpts from Mark McLoughlin's message of 2013-10-31 06:30:32 -0700:

On Thu, 2013-10-31 at 15:37 +1300, Robert Collins wrote:

This is a bit of a social norms thread

I've been consistently asking for tests in reviews for a while now,
and I get the occasional push-back. I think this falls into a few
broad camps:

A - there is no test suite at all, adding one in unreasonable
B - this thing cannot be tested in this context (e.g. functional 
tests

are defined in a different tree)
C - this particular thing is very hard to test
D - testing this won't offer benefit
E - other things like this in the project don't have tests
F - submitter doesn't know how to write tests
G - submitter doesn't have time to write tests


Nice breakdown.


Now, of these, I think it's fine not add tests in cases A, B, C in
combination with D, and D.

I don't think E, F or G are sufficient reasons to merge something
without tests, when reviewers are asking for them. G in the special
case that the project really wants the patch landed - but then I'd
expect reviewers to not ask for tests or to volunteer that they 
might

be optional.


I totally agree with the sentiment but, especially when it's a 
newcomer
to the project, I try to put myself in the shoes of the patch 
submitter

and double-check whether what we're asking is reasonable.



Even with a long time contributor, empathy is an important part of
constructing reviews. We could make more robotic things that review 
for
test coverage, but we haven't, because this is a gray area. The role 
of
a reviewer isn't just get patches merged and stop defects. It is also 
to

grow the other developers.


For example, if someone shows up to Nova with their first OpenStack
contribution, it fixes something which is unquestionably a bug - 
think
typo like "raise NotFund('foo')" - and testing this code patch 
requires

more than adding a simple new scenario to existing tests ...



This goes back to my recent suggestion to help the person not with a 
-1

or a +2, but with an additional patch that fixes it.


That, for me, is an example of "-1, we need a test! untested code is
broken!" is really shooting the messenger, not valuing the newcomers
contribution and risks turning that person off the project forever.
Reviewers being overly aggressive about this where the project 
doesn't
have full test coverage to begin with really makes us seem 
unwelcoming.


In cases like that, I'd be of a mind to go "+2 Awesome! Thanks for
catching this! It would be great to have a unit test for this, but 
it's

clear the current code is broken so I'm fine with merging the fix
without a test". You could say it's now the reviewers responsibility 
to

merge a test, but if that requirement then turns off reviewers even
reviewing such a patch, then that doesn't help either.



I understand entirely why you choose this, and I think that is fine. 
I,
however, see this as a massive opportunity to teach. That code was 
only
broken because it was allowed it to be merged without tests. By 
letting
that situation continue, we only fix it today. The next major 
refactoring

has a high chance now of breaking that part of the code again.

So, rather than +2, I suggest -1 with compassion. Engage with the
submitter. If you don't know them, take a look at how hard it would be
to write a test for the behavior and give pointers to the exact test
suite that would need to be changed, or suggest a new test suite and
point at a good example to copy.


So, with all of this, let's make sure we don't forget to first
appreciate the effort that went into submitting the patch that lacks
tests.



I'm not going to claim that I've always practiced "-1 with 
compassion",

so thanks for reminding us all that we're not just reviewing code, we
are having a dialog with real live people.

I think this is the key thing here, thanks for bringing this up Clint. 
At the
end of the day, patches are submitted by real people. If we want to 
grow

the committer base and help people to become better reviewers, taking
the time to show them the ropes is part of the game. I'd argue that is 
in

fact part of what being a core is about in fact.


When in doubt, I tend to err on the side of "no score with comments".  
It's not ideal for every situation, but I like to think it gets my point 
across without making it sound like I disapprove of the change itself.  
If my suggestions are ignored completely, I've always got the -1 in my 
back pocket to press the issue. :-)


-Ben

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] Locking and ZooKeeper - a space oddysey

[Monty Taylor]

Sigh.

Yay We've added more competing methods of complexity!!!

Seriously. We now think that rabbit and zookeeper and mysql are ALL needed?

Joshua Harlow  wrote:

>I'm pretty sure the cats out of the bag.
>
>https://github.com/openstack/requirements/blob/master/global-requirements.t
>xt#L29
>
>https://kazoo.readthedocs.org/en/latest/
>
>-Josh
>
>On 10/31/13 7:43 AM, "Monty Taylor"  wrote:
>
>>
>>
>>On 10/30/2013 10:42 AM, Clint Byrum wrote:
>>> So, recently we've had quite a long thread in gerrit regarding locking
>>> in Heat:
>>> 
>>> https://review.openstack.org/#/c/49440/
>>> 
>>> In the patch, there are two distributed lock drivers. One uses SQL,
>>> and suffers from all the problems you might imagine a SQL based locking
>>> system would. It is extremely hard to detect dead lock holders, so we
>>> end up with really long timeouts. The other is ZooKeeper.
>>> 
>>> I'm on record as saying we're not using ZooKeeper. It is a little
>>> embarrassing to have taken such a position without really thinking
>>>things
>>> through. The main reason I feel this way though, is not because
>>>ZooKeeper
>>> wouldn't work for locking, but because I think locking is a mistake.
>>> 
>>> The current multi-engine paradigm has a race condition. If you have a
>>> stack action going on, the state is held in the engine itself, and not
>>> in the database, so if another engine starts working on another action,
>>> they will conflict.
>>> 
>>> The locking paradigm is meant to prevent this. But I think this is a
>>> huge mistake.
>>> 
>>> The engine should store _all_ of its state in a distributed data store
>>> of some kind. Any engine should be aware of what is already happening
>>> with the stack from this state and act accordingly. That includes the
>>> engine currently working on actions. When viewed through this lense,
>>> to me, locking is a poor excuse for serializing the state of the engine
>>> scheduler.
>>> 
>>> It feels like TaskFlow is the answer, with an eye for making sure
>>> TaskFlow can be made to work with distributed state. I am not well
>>> versed on TaskFlow's details though, so I may be wrong. It worries me
>>> that TaskFlow has existed a while and doesn't seem to be solving real
>>> problems, but maybe I'm wrong and it is actually in use already.
>>> 
>>> Anyway, as a band-aid, we may _have_ to do locking. For that, ZooKeeper
>>> has some real advantages over using the database. But there is hesitance
>>> because it is not widely supported in OpenStack. What say you, OpenStack
>>> community? Should we keep ZooKeeper out of our.. zoo?
>>
>>Yes. I'm strongly opposed to ZooKeeper finding its way into the already
>>complex pile of things we use.
>>
>>___
>>OpenStack-dev mailing list
>>OpenStack-dev@lists.openstack.org
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [State-Management] Cancel IRC meeting for next 2 weeks

[Joshua Harlow]

Hi all,

I just wanted to cancel the state-management IRC meeting for the next 2 weeks 
due to people traveling and the HK summit.

Since it won't be as productive as usual I think we can just skip it until 
after the summit :-)

Likely most people will be in #openstack-state-management anyway, feel free to 
join!

Much appreciated!

As always: https://wiki.openstack.org/wiki/TaskFlow

-Josh
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] Locking and ZooKeeper - a space oddysey

[Joshua Harlow]

I'm pretty sure the cats out of the bag.

https://github.com/openstack/requirements/blob/master/global-requirements.t
xt#L29

https://kazoo.readthedocs.org/en/latest/

-Josh

On 10/31/13 7:43 AM, "Monty Taylor"  wrote:

>
>
>On 10/30/2013 10:42 AM, Clint Byrum wrote:
>> So, recently we've had quite a long thread in gerrit regarding locking
>> in Heat:
>> 
>> https://review.openstack.org/#/c/49440/
>> 
>> In the patch, there are two distributed lock drivers. One uses SQL,
>> and suffers from all the problems you might imagine a SQL based locking
>> system would. It is extremely hard to detect dead lock holders, so we
>> end up with really long timeouts. The other is ZooKeeper.
>> 
>> I'm on record as saying we're not using ZooKeeper. It is a little
>> embarrassing to have taken such a position without really thinking
>>things
>> through. The main reason I feel this way though, is not because
>>ZooKeeper
>> wouldn't work for locking, but because I think locking is a mistake.
>> 
>> The current multi-engine paradigm has a race condition. If you have a
>> stack action going on, the state is held in the engine itself, and not
>> in the database, so if another engine starts working on another action,
>> they will conflict.
>> 
>> The locking paradigm is meant to prevent this. But I think this is a
>> huge mistake.
>> 
>> The engine should store _all_ of its state in a distributed data store
>> of some kind. Any engine should be aware of what is already happening
>> with the stack from this state and act accordingly. That includes the
>> engine currently working on actions. When viewed through this lense,
>> to me, locking is a poor excuse for serializing the state of the engine
>> scheduler.
>> 
>> It feels like TaskFlow is the answer, with an eye for making sure
>> TaskFlow can be made to work with distributed state. I am not well
>> versed on TaskFlow's details though, so I may be wrong. It worries me
>> that TaskFlow has existed a while and doesn't seem to be solving real
>> problems, but maybe I'm wrong and it is actually in use already.
>> 
>> Anyway, as a band-aid, we may _have_ to do locking. For that, ZooKeeper
>> has some real advantages over using the database. But there is hesitance
>> because it is not widely supported in OpenStack. What say you, OpenStack
>> community? Should we keep ZooKeeper out of our.. zoo?
>
>Yes. I'm strongly opposed to ZooKeeper finding its way into the already
>complex pile of things we use.
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Chris Friesen]

On 10/31/2013 06:04 AM, Rosa, Andrea (HP Cloud Services) wrote:


A - there is no test suite at all, adding one in unreasonable B - this thing 
cannot
be tested in this context (e.g. functional tests are defined in a different 
tree)
C - this particular thing is very hard to test

>> D - testing this won't offer benefit


In my opinion  C instead of being an acceptable reason for not having tests is 
a symptom of one of the two things:
1) F => Submitter doesn't know how to write tests, in this case someone else 
can help with suggestions
2) The code we are trying to test is too complicated so it's time to refactor it

And about D, In my opinion  tests always offer benefits, like code coverage or 
helping in understanding the code.


I think there are actually cases where C is valid.  It's difficult to 
test certain kinds of race conditions, for example, unless you have very 
low-level hooks into the guts of the system in order to force the 
desired conditions to reliably occur at exactly the right time.


Chris


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Migrating to newer full projects from what used to be part of nova

[Russell Bryant]

On 10/31/2013 12:46 PM, John Griffith wrote:
>> Issues at hand, for now, are:
>>
>> 1) Migrating from nova-volume to cinder
> 
> So to be quite honest we never intended to make skips like you
> describe.  Perhaps that wasn't such a good choice in retrospect.  I'm
> willing to take a look at putting some special migrations in, but
> honestly given the changes in Nova alone between Essex and Folsom
> making a full jump from something like Essex to Havana is going to be
> challenging not just for adding Cinder, but overall.  I know it's not
> ideal but right now the best/preferred solution may in fact be to do
> incremental updates as intended.

Right, Nova (and probably other projects, too) only supports N to N+1
upgrades.  If you need to go to N+2, you'll need to go through N+1 to
get there.

FWIW, improving upgrades is very high on the priority list across all of
OpenStack.  We have sessions all over the place to talk about it.  It's
getting a lot better.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions

[Qing He]

Dolph,
Thanks!  It is good that python example is provided!
Qing

From: Dolph Mathews [mailto:dolph.math...@gmail.com]
Sent: Thursday, October 31, 2013 9:25 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] distibuted caching system in front of mysql server 
for openstack transactions


On Mon, Oct 28, 2013 at 5:46 PM, Qing He 
mailto:qing...@radisys.com>> wrote:
In my hard drive-less use case, I need an in-core-db/cache that can be in the 
same db cluster with real db (with hard drive) with the same sql api so that 
the current openstack code do not need to be changed, instead, just a pluggin 
with some configurations.

This is pretty much the original use case that dogpile.cache grew out of, see:

  http://docs.sqlalchemy.org/en/rel_0_9/orm/examples.html#dogpile-caching


-Original Message-
From: Morgan Fainberg [mailto:m...@metacloud.com]
Sent: Monday, October 28, 2013 10:22 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] distibuted caching system in front of mysql server 
for openstack transactions
In light of what Dolph said with regards to Keystone, we are using 
dogpile.cache to implement memoization in front of our driver calls.
It it has the ability to cache directly as well, but it has been effective (so 
far) for our use-case.

That being said, I am unsure if caching in front of MySQL is really what we 
want.  I believe that we should be caching after processing work (hence 
memoization mechanism) instead of at the SQL layer.  This also means we can be 
measured in what we cache (oh hey, it makes no sense to cache X because it 
needs to be "real time" or there isn't a performance issue with that query / 
call, but Y does a ton of processing and is an expensive join/temptable query). 
 In my experience, unless the whole application is designed with caching in 
mind, caching something as broad as MySQL calls (or any SQL store) is likely 
going to net exactly what Shawn Hartsock stated, adding a second performance 
issue.

--Morgan

On Mon, Oct 28, 2013 at 10:05 AM, Shawn Hartsock 
mailto:hartso...@vmware.com>> wrote:
>>
>> I once heard a quote.. "I had a performance problem, so I added caching.
>> now I have two performance problems."
>>
> this. 1,000 times this.
>
> Just to float this thought ... make sure it's considered...
>
> I've seen a *lot* of people misuse caching when what the really want is 
> memoization.
>
> *
> http://stackoverflow.com/questions/1988804/what-is-memoization-and-how
> -can-i-use-it-in-python
> *
> http://stackoverflow.com/questions/10879137/how-can-i-memoize-a-class-
> instantiation-in-python
>
> ... I'm not sure what you're trying to do. So YMMV, TTFN, BBQ.
>
> # Shawn Hartsock
>
> - Original Message -
>> From: "Clint Byrum" mailto:cl...@fewbar.com>>
>> To: "openstack-dev" 
>> mailto:openstack-dev@lists.openstack.org>>
>> Sent: Monday, October 28, 2013 12:12:49 PM
>> Subject: Re: [openstack-dev] distibuted caching system in front of mysql 
>>  server for openstack transactions
>>
>> Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700:
>> > It's not specific to mysql (or sql at all), but keystone is using
>> > dogpile.cache around driver calls to a similar effect.
>> >
>> >   http://dogpilecache.readthedocs.org/en/latest/
>> >
>> > It can persist to memcache, redis, etc.
>> >
>>
>> I once heard a quote.. "I had a performance problem, so I added caching.
>> now I have two performance problems."
>>
>> Caching is unbelievably awesome in the jobs it can do well. When the
>> problem is straight forward and the requirements are few, it is just
>> the right thing to relieve engineering pressure to make an
>> application more scalable.
>>
>> However, IMO, more than narrow, well defined cache usage is a sign
>> that the application needs some reworking to scale.
>>
>> I like the principle of "let's use dogpile so we don't have to
>> reinvent multi-level caching". However, let's make sure we look at
>> each performance issue individually, rather than just throwing them
>> all in a cache box and waving the memcache wand.
>>
>> >
>> > https://github.com/openstack/keystone/blob/master/keystone/common/c
>> > ache/core.py
>> >
>> > On Fri, Oct 25, 2013 at 6:53 PM, Qing He 
>> > mailto:qing...@radisys.com>> wrote:
>> >
>> > >  All,
>> > >
>> > > Has anyone looked at the options of putting a distributed caching
>> > > system in front of mysql server to improve performance? This
>> > > should be similar to Oracle Coherence, or VMware VFabric
>> > > SQLFire.
>> > >
>> > > ** **
>> > >
>> > > Thanks,
>> > >
>> > > ** **
>> > >
>> > > Qing
>> > >
>> > > ___
>> > > OpenStack-dev mailing list
>> > > OpenStack-dev@lists.openstack.org
>> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> > >
>> > >
>> >
>>
>> 

Re: [openstack-dev] [nova] changing old migrations is verboten

[Johannes Erdfelt]

On Thu, Oct 31, 2013, Sean Dague  wrote:
> So there is a series of patches starting with -
> https://review.openstack.org/#/c/53417/ that go back and radically
> change existing migration files.
> 
> This is really a no-no, unless there is a critical bug fix that
> absolutely requires it. Changing past migrations should be
> considered with the same level of weight as an N-2 backport, only
> done when there is huge upside to the change.
> 
> I've -2ed the first 2 patches in the series, though that review
> applies to all of them (I figured a mailing list thread was probably
> more useful than -2ing everything in the series).
> 
> There needs to be really solid discussion about the trade offs here
> before contemplating something as dangerous as this.

The most important thing for DB migrations is that they remain
functionality identical.

Historically we have allowed many changes to DB migrations that kept
them functionally identical to how they were before.

Looking through the commit history, here's a sampling of changes:

- _ was no longer monkey patched, necessitating a new import added
- fix bugs causing testing problems
- change copyright headers
- remove unused code (creating logger, imports, etc)
- fix bugs causing the migrations to fail to function (on PostgreSQL,
  downgrade bugs, etc)
- style changes (removing use of locals(), whitespace, etc)
- make migrations faster
- add comments to clarify code
- improve compatibility with newer versions of SQLAlchemy

The reviews you're referencing seem to fall into what we have
historically allowed.

That said, I do agree there needs to be a higher burden of proof that
the change being made is functionally identical to before.

JE


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Solum] Separation of concerns for 0.1 git deploy blueprint

[Clayton Coleman]

- Original Message -
> - Original Message -
> > Hi Clayton,
> > 
> > Thank you for creating these diagrams. They are a great starting point for
> > discussions
> > around the git deploy blueprint.
> > 
> > Some questions/comments:
> > 
> > In both the diagrams, what do the arrows indicate?
> > Data flow, control-flow, or some kind of ordering relationship?
> 
> "Happens after" mostly.  It was intended to be less formal than a true
> sequence flow, just a vehicle iterate and discuss until we'd nailed down any
> mismatches in how each of us looks at the overall flow, then move on to more
> formal definitions.

Updated the image with comments, and added a "new deployment artifact 
available" flow (assumed to be the output of a build.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Migrating to newer full projects from what used to be part of nova

[John Griffith]

On Thu, Oct 31, 2013 at 10:26 AM, Jesse Pretorius
 wrote:
> Hi everyone,
>
> Migrations from Essex to Grizzly/Havana are starting to hit my radar of
> responsible tasks and I'm disappointed that beyond this old wiki note [1]

Is your disappointment that there isn't a path from Essex -->
Grizzly/Havana, or are you unhappy with the content?

> and a wealth of questions with very few answers [2], there is very little
> available to support the migrations from what used to be part of nova to the
> newer full projects.
>
> I really think that as Openstack grows and the projects split out, one of
> the focal areas really needs to be on ensuring that people using the older
> versions can migrate to the newer versions without needing to do all sorts
> of terrible hacks.
>
> Issues at hand, for now, are:
>
> 1) Migrating from nova-volume to cinder

So to be quite honest we never intended to make skips like you
describe.  Perhaps that wasn't such a good choice in retrospect.  I'm
willing to take a look at putting some special migrations in, but
honestly given the changes in Nova alone between Essex and Folsom
making a full jump from something like Essex to Havana is going to be
challenging not just for adding Cinder, but overall.  I know it's not
ideal but right now the best/preferred solution may in fact be to do
incremental updates as intended.

Going forward maybe we can come up with something better.

> 2) Migrating from nova-network to neutron
>
> It'd be great if we could pool efforts to figure out an effective way of
> handling these migrations. Whether they're handled in the 'db sync' process,
> or by a set of companion utilities instead is immaterial to a deployer... as
> long as something suitable and effective is available to cater for the need.
>
> [1] https://wiki.openstack.org/wiki/MigrateToCinder
> [2] https://www.google.co.za/search?q=migrate+'nova-network'+quantum
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack's OpenPGP Web of Trust

[Jeremy Stanley]

On 2013-10-31 12:24:49 +0100 (+0100), Thierry Carrez wrote:
> OK, made it official
[...]

And:

https://wiki.openstack.org/wiki/OpenPGP_Web_of_Trust/Icehouse_Summit

If you add yourself to the table there in the wiki by noon the day
before, I'll try to bring a stack of printouts we can use as
checklists. With only 30 minutes and places to get from/to between
sessions, it's going to be tight timing if many people want to
participate. Don't feel like you have to get through everyone on the
list... it would be nice, but as long as enough interested attendees
participate and we each get a few signatures out of the effort it
will still be a great start.

Just a reminder of how these things go, in case you haven't
participated before--this is really about meeting in person and
checking names/photos/key fingerprints. The actual key signing can
(and should, in the interest of time) happen later at your
convenience... probably after we get home from the summit for many
of us because it will be a very busy week already.

Also, be prepared that some participants will decide after reviewing
your identification no to sign your key. This is perfectly natural,
as key signing choices are a matter of personal preference (some
people might care about strict name matching, may be uncertain of
the validity of your ID, or could have a hard time recognizing your
old photo from when you still had long hair and didn't need glasses
yet). In my opinion that additional individuality only leads to a
healthier web of trust and is not something you should ever consider
an insult.

Also, I'll be around at the conference every day with my ID and key
fingerprint cards too (as I expect will some others). So if you
can't get to the Thursday slot for other reasons and want to
exchange ID with someone, just let them know and see if they might
be able to work out a quick face-to-face in the hallway, at lunch or
maybe during one of the evening parties.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Migrating to newer full projects from what used to be part of nova

[Denis Makogon]

Yes, OS as big product tries to split hierarchy of responsibilities.
Currently we have serveral tasks, i suppose.
First one is upgrading from one stable release to another.
Second one is migrating from one release to another.

Currently OpenStack has several deployment projects: TripleO, Fuel (<--- is
a bit different from TripleO, it performs bare-metal deployment of whole
OS).

As way of covering issue of upgrading/migrating, personaly i see, we could
suggest those projects to propose a way of solving it.


2013/10/31 Jesse Pretorius 

> Hi everyone,
>
> Migrations from Essex to Grizzly/Havana are starting to hit my radar of
> responsible tasks and I'm disappointed that beyond this old wiki note [1]
> and a wealth of questions with very few answers [2], there is very little
> available to support the migrations from what used to be part of nova to
> the newer full projects.
>
> I really think that as Openstack grows and the projects split out, one of
> the focal areas really needs to be on ensuring that people using the older
> versions can migrate to the newer versions without needing to do all sorts
> of terrible hacks.
>
> Issues at hand, for now, are:
>
> 1) Migrating from nova-volume to cinder
> 2) Migrating from nova-network to neutron
>
> It'd be great if we could pool efforts to figure out an effective way of
> handling these migrations. Whether they're handled in the 'db sync'
> process, or by a set of companion utilities instead is immaterial to a
> deployer... as long as something suitable and effective is available to
> cater for the need.
>
> [1] https://wiki.openstack.org/wiki/MigrateToCinder
> [2] 
> https://www.google.co.za/search?q=migrate+'nova-network'+quantum
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] changing old migrations is verboten

[Sean Dague]

Actually no confusion. :-) Joe Gordon just made me realize that I didn't
really explain why we had that policy.

That really should have been a follow up to my own post, not yours. Sorry
if I made it look like I was arguing with you, which I wasn't.. :-)

We're all good.

Sean Dague
http://dague.net
On Oct 31, 2013 12:12 PM, "Jay Pipes"  wrote:

> On 10/31/2013 11:56 AM, Sean Dague wrote:
>
>> On 10/31/2013 11:23 AM, Jay Pipes wrote:
>>
>>> On 10/31/2013 08:01 AM, Sean Dague wrote:
>>>
 So there is a series of patches starting with -
 https://review.openstack.org/#**/c/53417/that
  go back and radically
 change existing migration files.

 This is really a no-no, unless there is a critical bug fix that
 absolutely requires it. Changing past migrations should be considered
 with the same level of weight as an N-2 backport, only done when there
 is huge upside to the change.

 I've -2ed the first 2 patches in the series, though that review applies
 to all of them (I figured a mailing list thread was probably more useful
 than -2ing everything in the series).

 There needs to be really solid discussion about the trade offs here
 before contemplating something as dangerous as this.

>>>
>>> +1
>>>
>>
>> There is a very real reason why we have a firm stance on this. There are
>> a huge number of OpenStack instances out in the field, at all sorts of
>> different past versions. We really try to promise that you can always
>> forward upgrade your database.
>>
>> If you go back and change an old migration, you have not forked the
>> past. Some people will have already taken that migration, and they have
>> one view of the world, others haven't yet, they hit your updated
>> version, and they now have different database. So 2 people with Havana
>> would no longer be guaranteed to have the same data model set up by us.
>>
>> It's easy to believe that "this change is really straight forward, it
>> will be exactly the same model", but if it isn't, in any way, exactly
>> the same (even in a way that we didn't realize yet that it mattered),
>> you've forked the past. And that makes supporting users in these various
>> forked versions of the world impossible.
>>
>> Migrations are basically idempotent. If you want to clean things up, do
>> them in a new migration. Don't touch an old one unless it is causing
>> corruption to someone's data so that fixing it with a future migration
>> is not an option.
>>
>
> LOL, I was +1'ing your thoughts, not +1'ing the proposal to have a solid
> discussion about the trade-offs :)
>
> Sorry for the confusion!
> -jay
>
>
> __**_
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.**org 
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Migrating to newer full projects from what used to be part of nova

[Jesse Pretorius]

Hi everyone,

Migrations from Essex to Grizzly/Havana are starting to hit my radar of
responsible tasks and I'm disappointed that beyond this old wiki note [1]
and a wealth of questions with very few answers [2], there is very little
available to support the migrations from what used to be part of nova to
the newer full projects.

I really think that as Openstack grows and the projects split out, one of
the focal areas really needs to be on ensuring that people using the older
versions can migrate to the newer versions without needing to do all sorts
of terrible hacks.

Issues at hand, for now, are:

1) Migrating from nova-volume to cinder
2) Migrating from nova-network to neutron

It'd be great if we could pool efforts to figure out an effective way of
handling these migrations. Whether they're handled in the 'db sync'
process, or by a set of companion utilities instead is immaterial to a
deployer... as long as something suitable and effective is available to
cater for the need.

[1] https://wiki.openstack.org/wiki/MigrateToCinder
[2] 
https://www.google.co.za/search?q=migrate+'nova-network'+quantum
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions

[Dolph Mathews]

On Mon, Oct 28, 2013 at 5:46 PM, Qing He  wrote:

> In my hard drive-less use case, I need an in-core-db/cache that can be in
> the same db cluster with real db (with hard drive) with the same sql api so
> that the current openstack code do not need to be changed, instead, just a
> pluggin with some configurations.
>

This is pretty much the original use case that dogpile.cache grew out of,
see:

  http://docs.sqlalchemy.org/en/rel_0_9/orm/examples.html#dogpile-caching


>
> -Original Message-
> From: Morgan Fainberg [mailto:m...@metacloud.com]
> Sent: Monday, October 28, 2013 10:22 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] distibuted caching system in front of mysql
> server for openstack transactions
>
> In light of what Dolph said with regards to Keystone, we are using
> dogpile.cache to implement memoization in front of our driver calls.
> It it has the ability to cache directly as well, but it has been effective
> (so far) for our use-case.
>
> That being said, I am unsure if caching in front of MySQL is really what
> we want.  I believe that we should be caching after processing work (hence
> memoization mechanism) instead of at the SQL layer.  This also means we can
> be measured in what we cache (oh hey, it makes no sense to cache X because
> it needs to be "real time" or there isn't a performance issue with that
> query / call, but Y does a ton of processing and is an expensive
> join/temptable query).  In my experience, unless the whole application is
> designed with caching in mind, caching something as broad as MySQL calls
> (or any SQL store) is likely going to net exactly what Shawn Hartsock
> stated, adding a second performance issue.
>
> --Morgan
>
> On Mon, Oct 28, 2013 at 10:05 AM, Shawn Hartsock 
> wrote:
> >>
> >> I once heard a quote.. "I had a performance problem, so I added caching.
> >> now I have two performance problems."
> >>
> > this. 1,000 times this.
> >
> > Just to float this thought ... make sure it's considered...
> >
> > I've seen a *lot* of people misuse caching when what the really want is
> memoization.
> >
> > *
> > http://stackoverflow.com/questions/1988804/what-is-memoization-and-how
> > -can-i-use-it-in-python
> > *
> > http://stackoverflow.com/questions/10879137/how-can-i-memoize-a-class-
> > instantiation-in-python
> >
> > ... I'm not sure what you're trying to do. So YMMV, TTFN, BBQ.
> >
> > # Shawn Hartsock
> >
> > - Original Message -
> >> From: "Clint Byrum" 
> >> To: "openstack-dev" 
> >> Sent: Monday, October 28, 2013 12:12:49 PM
> >> Subject: Re: [openstack-dev] distibuted caching system in front of
> mysql  server for openstack transactions
> >>
> >> Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700:
> >> > It's not specific to mysql (or sql at all), but keystone is using
> >> > dogpile.cache around driver calls to a similar effect.
> >> >
> >> >   http://dogpilecache.readthedocs.org/en/latest/
> >> >
> >> > It can persist to memcache, redis, etc.
> >> >
> >>
> >> I once heard a quote.. "I had a performance problem, so I added caching.
> >> now I have two performance problems."
> >>
> >> Caching is unbelievably awesome in the jobs it can do well. When the
> >> problem is straight forward and the requirements are few, it is just
> >> the right thing to relieve engineering pressure to make an
> >> application more scalable.
> >>
> >> However, IMO, more than narrow, well defined cache usage is a sign
> >> that the application needs some reworking to scale.
> >>
> >> I like the principle of "let's use dogpile so we don't have to
> >> reinvent multi-level caching". However, let's make sure we look at
> >> each performance issue individually, rather than just throwing them
> >> all in a cache box and waving the memcache wand.
> >>
> >> >
> >> > https://github.com/openstack/keystone/blob/master/keystone/common/c
> >> > ache/core.py
> >> >
> >> > On Fri, Oct 25, 2013 at 6:53 PM, Qing He  wrote:
> >> >
> >> > >  All,
> >> > >
> >> > > Has anyone looked at the options of putting a distributed caching
> >> > > system in front of mysql server to improve performance? This
> >> > > should be similar to Oracle Coherence, or VMware VFabric
> >> > > SQLFire.
> >> > >
> >> > > ** **
> >> > >
> >> > > Thanks,
> >> > >
> >> > > ** **
> >> > >
> >> > > Qing
> >> > >
> >> > > ___
> >> > > OpenStack-dev mailing list
> >> > > OpenStack-dev@lists.openstack.org
> >> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >> > >
> >> > >
> >> >
> >>
> >> ___
> >> OpenStack-dev mailing list
> >> OpenStack-dev@lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman

Re: [openstack-dev] [horizon / keystone] Marker could not be found?

[Dolph Mathews]

On Thu, Oct 31, 2013 at 8:38 AM, Sebastian Porombka <
porom...@uni-paderborn.de> wrote:

>  Hello Folks.
>
>  I have a problem after grizzly->havana migration where i’m unable to
> rescue myself.
> When I open the "Admin -> Resource-Usage View i get no results – only a
> red error box with the message "*Error: *Unable to retrieve tenant list.“.
>
>  Horizon log:
>
> [Thu Oct 31 11:39:44 2013] [error] Creating a new keystoneclient
> connection to http://$controller:35357/v2.0.
>
> [Thu Oct 31 11:39:44 2013] [error] REQ: curl -i -X GET 
> http://$controller:35357/v2.0/tenants?marker=tenant_marker&limit=21
> -H "User-Agent: python-keystoneclient" -H "Forwarded:
> for=131.234.5.178;by=python-keystoneclient" -H "X-Auth-Token: 82[…]f46"
>
> [Thu Oct 31 11:39:44 2013] [error] REQ: curl -i -X GET 
> http://$controller:35357/v2.0/tenants?marker=tenant_marker&limit=21
> -H "User-Agent: python-keystoneclient" -H "Forwarded:
> for=131.234.5.178;by=python-keystoneclient" -H "X-Auth-Token: 82[…]46"
>
> [Thu Oct 31 11:39:44 2013] [error] INFO:urllib3.connectionpool:Starting
> new HTTP connection (1): $controller
>
> [Thu Oct 31 11:39:44 2013] [error] DEBUG:urllib3.connectionpool:"GET
> /v2.0/tenants?marker=tenant_marker&limit=21 HTTP/1.1" 400 88
>
> [Thu Oct 31 11:39:44 2013] [error] RESP: [400]
> CaseInsensitiveDict({'date': 'Thu, 31 Oct 2013 11:39:47 GMT', 'vary':
> 'X-Auth-Token', 'content-length': '88', 'content-type': 'application/json'})
>
> [Thu Oct 31 11:39:44 2013] [error] RESP BODY: {"error": {"message":
> "Marker could not be found", "code": 400, "title": "Bad Request"}}
>
> [Thu Oct 31 11:39:44 2013] [error]
>
> [Thu Oct 31 11:39:44 2013] [error] RESP: [400]
> CaseInsensitiveDict({'date': 'Thu, 31 Oct 2013 11:39:47 GMT', 'vary':
> 'X-Auth-Token', 'content-length': '88', 'content-type': 'application/json'})
>
> [Thu Oct 31 11:39:44 2013] [error] RESP BODY: {"error": {"message":
> "Marker could not be found", "code": 400, "title": "Bad Request"}}
>
> [Thu Oct 31 11:39:44 2013] [error]
>
> [Thu Oct 31 11:39:44 2013] [error] Request returned failure status: 400
>
> [Thu Oct 31 11:39:44 2013] [error] Request returned failure status: 400
>
> [Thu Oct 31 11:39:44 2013] [error] Recoverable error: Marker could not be
> found (HTTP 400)
>
>  Keystone Log:
> 2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Matched GET
> /tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
>
> 2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Route path:
> '{path_info:.*}', defaults: {'controller':
> } __call__
> /usr/lib/python2.7/dist-packages/routes/middleware.py:102
>
> 2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Match dict:
> {'controller':  0x4156f10>, 'path_info': '/tenants'} __call__
> /usr/lib/python2.7/dist-packages/routes/middleware.py:103
>
> 2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Matched GET
> /tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
>
> 2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Route path:
> '{path_info:.*}', defaults: {'controller':
> } __call__
> /usr/lib/python2.7/dist-packages/routes/middleware.py:102
>
> 2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Match dict:
> {'controller': ,
> 'path_info': '/tenants'} __call__
> /usr/lib/python2.7/dist-packages/routes/middleware.py:103
>
> 2013-10-31 12:39:47.354 17187 DEBUG routes.middleware [-] Matched GET
> /tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
>
> 2013-10-31 12:39:47.354 17187 DEBUG routes.middleware [-] Route path:
> '{path_info:.*}', defaults: {'controller':
> }
> __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:102
>
> 2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Match dict:
> {'controller':  0x41517d0>, 'path_info': '/tenants'} __call__
> /usr/lib/python2.7/dist-packages/routes/middleware.py:103
>
> 2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Matched GET
> /tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
>
> 2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Route path:
> '{path_info:.*}', defaults: {'controller':
> } __call__
> /usr/lib/python2.7/dist-packages/routes/middleware.py:102
>
> 2013-10-31 12:39:47.356 17187 DEBUG routes.middleware [-] Match dict:
> {'controller': ,
> 'path_info': '/tenants'} __call__
> /usr/lib/python2.7/dist-packages/routes/middleware.py:103
>
> 2013-10-31 12:39:47.356 17187 DEBUG routes.middleware [-] Matched GET
> /tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
>
> 2013-10-31 12:39:47.357 17187 DEBUG routes.middleware [-] Route path:
> '/tenants', defaults: {'action': u'get_all_projects', 'controller':
> } __call__
> /usr/lib/python2.7/dist-packages/routes/middleware.py:102
>
> 2013-10-31 12:39:47.357 17187 DEBUG routes.middleware [-] Match dict:
> {'action': u'get_all_projects', 'controller':
> } __call__
> /usr/lib/python2.7/dist-packages/routes/middleware.py:103
>
>

Re: [openstack-dev] [nova] changing old migrations is verboten

[Jay Pipes]

On 10/31/2013 11:56 AM, Sean Dague wrote:

On 10/31/2013 11:23 AM, Jay Pipes wrote:

On 10/31/2013 08:01 AM, Sean Dague wrote:

So there is a series of patches starting with -
https://review.openstack.org/#/c/53417/ that go back and radically
change existing migration files.

This is really a no-no, unless there is a critical bug fix that
absolutely requires it. Changing past migrations should be considered
with the same level of weight as an N-2 backport, only done when there
is huge upside to the change.

I've -2ed the first 2 patches in the series, though that review applies
to all of them (I figured a mailing list thread was probably more useful
than -2ing everything in the series).

There needs to be really solid discussion about the trade offs here
before contemplating something as dangerous as this.


+1


There is a very real reason why we have a firm stance on this. There are
a huge number of OpenStack instances out in the field, at all sorts of
different past versions. We really try to promise that you can always
forward upgrade your database.

If you go back and change an old migration, you have not forked the
past. Some people will have already taken that migration, and they have
one view of the world, others haven't yet, they hit your updated
version, and they now have different database. So 2 people with Havana
would no longer be guaranteed to have the same data model set up by us.

It's easy to believe that "this change is really straight forward, it
will be exactly the same model", but if it isn't, in any way, exactly
the same (even in a way that we didn't realize yet that it mattered),
you've forked the past. And that makes supporting users in these various
forked versions of the world impossible.

Migrations are basically idempotent. If you want to clean things up, do
them in a new migration. Don't touch an old one unless it is causing
corruption to someone's data so that fixing it with a future migration
is not an option.


LOL, I was +1'ing your thoughts, not +1'ing the proposal to have a solid 
discussion about the trade-offs :)


Sorry for the confusion!
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] changing old migrations is verboten

[Sean Dague]

On 10/31/2013 11:23 AM, Jay Pipes wrote:

On 10/31/2013 08:01 AM, Sean Dague wrote:

So there is a series of patches starting with -
https://review.openstack.org/#/c/53417/ that go back and radically
change existing migration files.

This is really a no-no, unless there is a critical bug fix that
absolutely requires it. Changing past migrations should be considered
with the same level of weight as an N-2 backport, only done when there
is huge upside to the change.

I've -2ed the first 2 patches in the series, though that review applies
to all of them (I figured a mailing list thread was probably more useful
than -2ing everything in the series).

There needs to be really solid discussion about the trade offs here
before contemplating something as dangerous as this.


+1


There is a very real reason why we have a firm stance on this. There are 
a huge number of OpenStack instances out in the field, at all sorts of 
different past versions. We really try to promise that you can always 
forward upgrade your database.


If you go back and change an old migration, you have not forked the 
past. Some people will have already taken that migration, and they have 
one view of the world, others haven't yet, they hit your updated 
version, and they now have different database. So 2 people with Havana 
would no longer be guaranteed to have the same data model set up by us.


It's easy to believe that "this change is really straight forward, it 
will be exactly the same model", but if it isn't, in any way, exactly 
the same (even in a way that we didn't realize yet that it mattered), 
you've forked the past. And that makes supporting users in these various 
forked versions of the world impossible.


Migrations are basically idempotent. If you want to clean things up, do 
them in a new migration. Don't touch an old one unless it is causing 
corruption to someone's data so that fixing it with a future migration 
is not an option.


-Sean

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][LBaaS] Thursday meeting follow-up

[Samuel Bercovici]

Hi,

I have created two document to discuss SSL termination and L7 Rules at:
SSL termination : 
https://docs.google.com/document/d/1qnoJLD1txY5wnjx4k480AtEGCOEtkPMvTzxPo3_DPcs/edit?usp=sharing
SSL BP: https://blueprints.launchpad.net/neutron/+spec/lbaas-ssl-termination

L7 Rules: 
https://docs.google.com/document/d/1Dm6fWxHmsgO6gXFWaYf-Ce36vYpMB0L13ASrB_BZ0Bc/edit?usp=sharing
L7 BP: https://blueprints.launchpad.net/neutron/+spec/lbaas-l7-rules

Please feel free to comment on the documents.
We can also discuss at the summit.

Regards,
-Sam.




From: Eugene Nikanorov [mailto:enikano...@mirantis.com]
Sent: Friday, October 25, 2013 6:09 PM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [Neutron][LBaaS] Thursday meeting follow-up

Hi folks,

Thanks to everyone who joined the meeting on Thursday.
We've discussed desired features and changes in LBaaS service and identified 
dependencies between them.

You can find them on etherpad: 
https://etherpad.openstack.org/p/neutron-icehouse-lbaas
Most of the features are also captured in the document shared by Sam: 
https://docs.google.com/document/d/1Vjm57lh7PnXDelOy-VxsJkzc8QRiNN368sS11ePs_vA/edit?pli=1#
IRC meeting logs: http://paste.openstack.org/show/49561/

Thanks,
Eugene.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] Comments on Steve Baker's Proposal on HOT Software Config

[Lakshminaraya Renganarayana]

-Mike Spreitzer/Watson/IBM@IBMUS wrote: ->To: "OpenStack Development Mailing List \(not for usage questions\)">>From: Mike Spreitzer/Watson/IBM@IBMUS>Date: 10/30/2013 03:56PM>Subject: Re: [openstack-dev] [Heat] Comments on Steve Baker's>Proposal on HOT Software Config>>Lakshminaraya Renganarayana/Watson/IBM@IBMUS wrote on>10/30/2013 03:35:32 PM:> Zane Bitter  wrote on 10/29/2013 08:46:21>AM:>>> ...>>> In this method >>> > (i.e. option (2) above) shouldn't we be building the dependency>graph in >>> > Heat rather than running through them sequentially as specified>by the >>> > user? In that case, we should use a dictionary not a list:>>> > >>> >    app_server:>>> >      type: OS::Nova::Server>>> >      properties:>>> >        components:>>> >          install_user_profile:>>> >            definition: InstallWasProfile>>> >            params:>>> >              user_id>>> >          install_admin_profile:>>> >            definition: InstallWasProfile>>> >            params:>>> >              admin_id>>> >>> I missed this implication of using a list! You are right, it should>be >>> a dictionary and Heat would be building the dependence graph. Using a dictionary instead of a list can work, but>I think we might be going overboard here.  Do we expect the component>invocations on a given VM instance to run concurrently?  I think that>has been dissed before.  Chef users are happy to let a role be a list>of recipes, not a DAG.  A list is simple; is there an actual problem>with it?Yes, there was some agreement on component invocations on a given VM instance being run sequentially. However, the issue here is slightly different. If, as a design principle, Heat analyzes dependences between component invocations,then it should do that irrespective of whether the component invocations are on the same VM or from different VMs. Given this, a list of component invocationswould imply an ordering which is in addition to the ordering induced by dependences.Whereas a dictionary would not impose any additional ordering.Thanks,LN


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] changing old migrations is verboten

[Jay Pipes]

On 10/31/2013 08:01 AM, Sean Dague wrote:

So there is a series of patches starting with -
https://review.openstack.org/#/c/53417/ that go back and radically
change existing migration files.

This is really a no-no, unless there is a critical bug fix that
absolutely requires it. Changing past migrations should be considered
with the same level of weight as an N-2 backport, only done when there
is huge upside to the change.

I've -2ed the first 2 patches in the series, though that review applies
to all of them (I figured a mailing list thread was probably more useful
than -2ing everything in the series).

There needs to be really solid discussion about the trade offs here
before contemplating something as dangerous as this.


+1

-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] Locking and ZooKeeper - a space oddysey

[Sandy Walsh]

On 10/31/2013 11:43 AM, Monty Taylor wrote:
> 
> Yes. I'm strongly opposed to ZooKeeper finding its way into the already
> complex pile of things we use.

Monty, is that just because the stack is very complicated now, or
something personal against ZK (or Java specifically)?

Curious.

-S


> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] Locking and ZooKeeper - a space oddysey

[Sandy Walsh]

On 10/30/2013 08:08 PM, Steven Dake wrote:
> On 10/30/2013 12:20 PM, Sandy Walsh wrote:
>>
>> On 10/30/2013 03:10 PM, Steven Dake wrote:
>>> I will -2 any patch that adds zookeeper as a dependency to Heat.
>> Certainly any distributed locking solution should be plugin based and
>> optional. Just as a database-oriented solution could be the default
>> plugin.

> Sandy,
> 
> Even if it is optional, some percentage of the userbase will enable it
> and expect the Heat community to debug and support it.

But, that's the nature of every openstack project. I don't support
HyperV in Nova or HBase in Ceilometer. The implementers deal with that
support. I can help guide someone to those people but have no intentions
of standing up those environments.

>> Re: the Java issue, we already have optional components in other
>> languages. I know Java is a different league of pain, but if it's an
>> optional component and left as a choice of the deployer, should we care?
>>
>> -S
>>
>> PS> As an aside, what are your issues with ZK?
>>
> 
> 
> I realize zookeeper exists for a reason.  But unfortunately Zookeeper is
> a server, rather then an in-process library.  This means someone needs
> to figure out how to document, scale, secure, and provide high
> availability for this component.  

Yes, that's why we would use it. Same goes for rabbit and mysql.

> This is extremely challenging for the
> two server infrastructure components OpenStack server processes depend
> on today (AMQP, SQL).  If the entire OpenStack community saw value in
> biting the bullet and accepting zookeeper as a dependency and taking on
> this work, I might be more ameniable.  

Why do other services need to agree on adopting ZK? If some Heat users
need it, they can use it. Nova shouldn't care.

> What we are talking about in the
> review, however, is that the Heat team bite that bullet, which is a big
> addition to the scope of work we already execute for the ability to gain
> a distributed lock.  I would expect there are simpler approaches to
> solve the problem without dragging the baggage of a new server component
> into the OpenStack deployment.

Yes, there probably are, and alternatives are good. But, as others have
attested, ZK is tried and true. Why not support it also?

> Using zookeeper as is suggested in the review is far different then the
> way Nova uses Zookeeper.  With the Nova use case, Nova still operates
> just dandy without zookeeper.  With zookeeper in the Heat usecase, it
> essentially becomes the default way people are expected to deploy Heat.

Why, if it's a plugin?

> What I would prefer is taskflow over AMQP, to leverage existing server
> infrastructure (that has already been documented, scaled, secured, and
> HA-ified).

Same problem exists, we're just pushing the ZK decision to another service.

> Regards
> -steve
> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] Locking and ZooKeeper - a space oddysey

[Monty Taylor]

On 10/30/2013 10:42 AM, Clint Byrum wrote:
> So, recently we've had quite a long thread in gerrit regarding locking
> in Heat:
> 
> https://review.openstack.org/#/c/49440/
> 
> In the patch, there are two distributed lock drivers. One uses SQL,
> and suffers from all the problems you might imagine a SQL based locking
> system would. It is extremely hard to detect dead lock holders, so we
> end up with really long timeouts. The other is ZooKeeper.
> 
> I'm on record as saying we're not using ZooKeeper. It is a little
> embarrassing to have taken such a position without really thinking things
> through. The main reason I feel this way though, is not because ZooKeeper
> wouldn't work for locking, but because I think locking is a mistake.
> 
> The current multi-engine paradigm has a race condition. If you have a
> stack action going on, the state is held in the engine itself, and not
> in the database, so if another engine starts working on another action,
> they will conflict.
> 
> The locking paradigm is meant to prevent this. But I think this is a
> huge mistake.
> 
> The engine should store _all_ of its state in a distributed data store
> of some kind. Any engine should be aware of what is already happening
> with the stack from this state and act accordingly. That includes the
> engine currently working on actions. When viewed through this lense,
> to me, locking is a poor excuse for serializing the state of the engine
> scheduler.
> 
> It feels like TaskFlow is the answer, with an eye for making sure
> TaskFlow can be made to work with distributed state. I am not well
> versed on TaskFlow's details though, so I may be wrong. It worries me
> that TaskFlow has existed a while and doesn't seem to be solving real
> problems, but maybe I'm wrong and it is actually in use already.
> 
> Anyway, as a band-aid, we may _have_ to do locking. For that, ZooKeeper
> has some real advantages over using the database. But there is hesitance
> because it is not widely supported in OpenStack. What say you, OpenStack
> community? Should we keep ZooKeeper out of our.. zoo?

Yes. I'm strongly opposed to ZooKeeper finding its way into the already
complex pile of things we use.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Keystone Concurrency & Races in SQL Assignment Backend

[Peter Feiner]

Okay Brant, sounds good. I'll start working on the SQL code today.

Please bear in mind that I'm going to be on vacation next week, so a
patch won't be ready until some time after November 11.

On Thu, Oct 31, 2013 at 9:46 AM, Brant Knudson  wrote:
> Peter -
>
> We discussed better use of transactions in irc, but I don't think anyone has
> had a chance to look at it. This would be a very useful thing to have
> someone look at. I'm fine with holding off on the oslo.db sessions work
> until we're sure the code is correct w/r/t multi-processing so that tempest
> is going to pass consistently.
>
> - Brant
>
> On Wed, Oct 30, 2013 at 5:08 PM, Peter Feiner  wrote:
>>
>> Hi Brant,
>>
>> In addition to the race you've fixed in
>> https://review.openstack.org/#/c/50767/, it looks like there are quite
>> a few more races in the SQL backend of keystone.assignment. I filed a
>> bug to this effect: https://bugs.launchpad.net/keystone/+bug/1246489.
>> The general problem is that transactions are used somewhat
>> indiscriminately. The fix (i.e., using transactions judiciously) is
>> straightforward and should be mostly independent of your ongoing
>> oslo.db sessions port in https://review.openstack.org/#/c/49460/. So,
>> unless you already have something in the works, I'll get started on
>> that tomorrow.
>>
>> I'm eager to fix these races so
>> https://review.openstack.org/#/c/42967/ can reliably pass tempest :-)
>>
>> Peter
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Solum] Separation of concerns for 0.1 git deploy blueprint

[Clayton Coleman]

- Original Message -
> Hi Clayton,
> 
> Thank you for creating these diagrams. They are a great starting point for
> discussions
> around the git deploy blueprint.
> 
> Some questions/comments:
> 
> In both the diagrams, what do the arrows indicate?
> Data flow, control-flow, or some kind of ordering relationship?

"Happens after" mostly.  It was intended to be less formal than a true sequence 
flow, just a vehicle iterate and discuss until we'd nailed down any mismatches 
in how each of us looks at the overall flow, then move on to more formal 
definitions.

> Especially, the arrow from 'respond to client' to 'HEAT' box in the top
> diagram
> is a bit confusing. Correct me if I am wrong, but my guess is that the
> meaning
> of that arrow is, 'stack create' will happen after responding to client has
> happened.
> Although, this may not be necessarily true (both those steps can happen in
> async manner).

You and Adrian are right - I'll go back and explicitly call out the 
asynchronicity.

> 
> Second, how do you envision tying together the two diagrams/high-level flows?
> Specifically, once we have a built artifact (second diagram), how do we go
> about deploying it?

I'll throw another one in that's really abstract and follow up in a response.  
I think the easy answer is "make a change to the 'plan', turn the plan into a 
HOT template, call HEAT to update the stack".  Over time Heat stack update then 
is the natural boundary for versions of an app.

> 
> We could either have, build abstraction -> REST API -> deploy abstraction,
> or, build abstraction -> deploy abstraction. What are your thoughts on this?
> 

Working backwards, builds usually create resources somewhere and sometimes 
those resources are large (e.g Jenkins and a Glance image, for instance).  The 
builder needs a clean endpoint to notify Solum that a new deployment artifact 
is available that replaces one of the existing deployment artifacts.  There's a 
big gray area here because it depends on how strongly Solum specifies this flow 
- the less strongly the flow is specified the more flexibility there is to 
integrate complex systems (Gerritt, Jenkins, existing corporate build 
procesess), the more strongly the flow is specified the easier it is to provide 
valuable outcomes.  My guess is that there probably is a "new artifact 
available" API call that updates an existing artifact which triggers a 
redeploy.  An artifact might be a zip, a WAR, a docker image, a glance image, a 
VM img file, etc.  I think this is a good topic for the build subgroup.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Khanh-Toan Tran]

Hi all,

As a newbie of the community, I'm not familiar with unittest and how to use it 
here. I've learned that Jenkins runs tests
everytime we submit some code. But how to write the test and what is a 'good 
test' and a 'bad test'? I saw some commits
in gerrit but am unable to say if the written test is enough to judge the code, 
since it is the author of the code who writes
the test. Is there a framework to follow or some rules/pratices to respect?

Do you have some links to help me out?

Thanks,

Toan

- Original Message -
From: "Kyle Mestery (kmestery)" 
To: "OpenStack Development Mailing List (not for usage questions)" 

Sent: Thursday, October 31, 2013 3:05:27 PM
Subject: Re: [openstack-dev] When is it okay for submitters to say 'I   don't   
want to add tests' ?

On Oct 31, 2013, at 8:56 AM, Clint Byrum  wrote:
> Excerpts from Mark McLoughlin's message of 2013-10-31 06:30:32 -0700:
>> On Thu, 2013-10-31 at 15:37 +1300, Robert Collins wrote:
>>> This is a bit of a social norms thread
>>> 
>>> I've been consistently asking for tests in reviews for a while now,
>>> and I get the occasional push-back. I think this falls into a few
>>> broad camps:
>>> 
>>> A - there is no test suite at all, adding one in unreasonable
>>> B - this thing cannot be tested in this context (e.g. functional tests
>>> are defined in a different tree)
>>> C - this particular thing is very hard to test
>>> D - testing this won't offer benefit
>>> E - other things like this in the project don't have tests
>>> F - submitter doesn't know how to write tests
>>> G - submitter doesn't have time to write tests
>> 
>> Nice breakdown.
>> 
>>> Now, of these, I think it's fine not add tests in cases A, B, C in
>>> combination with D, and D.
>>> 
>>> I don't think E, F or G are sufficient reasons to merge something
>>> without tests, when reviewers are asking for them. G in the special
>>> case that the project really wants the patch landed - but then I'd
>>> expect reviewers to not ask for tests or to volunteer that they might
>>> be optional.
>> 
>> I totally agree with the sentiment but, especially when it's a newcomer
>> to the project, I try to put myself in the shoes of the patch submitter
>> and double-check whether what we're asking is reasonable.
>> 
> 
> Even with a long time contributor, empathy is an important part of
> constructing reviews. We could make more robotic things that review for
> test coverage, but we haven't, because this is a gray area. The role of
> a reviewer isn't just get patches merged and stop defects. It is also to
> grow the other developers.
> 
>> For example, if someone shows up to Nova with their first OpenStack
>> contribution, it fixes something which is unquestionably a bug - think
>> typo like "raise NotFund('foo')" - and testing this code patch requires
>> more than adding a simple new scenario to existing tests ...
>> 
> 
> This goes back to my recent suggestion to help the person not with a -1
> or a +2, but with an additional patch that fixes it.
> 
>> That, for me, is an example of "-1, we need a test! untested code is
>> broken!" is really shooting the messenger, not valuing the newcomers
>> contribution and risks turning that person off the project forever.
>> Reviewers being overly aggressive about this where the project doesn't
>> have full test coverage to begin with really makes us seem unwelcoming.
>> 
>> In cases like that, I'd be of a mind to go "+2 Awesome! Thanks for
>> catching this! It would be great to have a unit test for this, but it's
>> clear the current code is broken so I'm fine with merging the fix
>> without a test". You could say it's now the reviewers responsibility to
>> merge a test, but if that requirement then turns off reviewers even
>> reviewing such a patch, then that doesn't help either.
>> 
> 
> I understand entirely why you choose this, and I think that is fine. I,
> however, see this as a massive opportunity to teach. That code was only
> broken because it was allowed it to be merged without tests. By letting
> that situation continue, we only fix it today. The next major refactoring
> has a high chance now of breaking that part of the code again.
> 
> So, rather than +2, I suggest -1 with compassion. Engage with the
> submitter. If you don't know them, take a look at how hard it would be
> to write a test for the behavior and give pointers to the exact test
> suite that would need to be changed, or suggest a new test suite and
> point at a good example to copy.
> 
>> So, with all of this, let's make sure we don't forget to first
>> appreciate the effort that went into submitting the patch that lacks
>> tests.
>> 
> 
> I'm not going to claim that I've always practiced "-1 with compassion",
> so thanks for reminding us all that we're not just reviewing code, we
> are having a dialog with real live people.
> 
I think this is the key thing here, thanks for bringing this up Clint. At the
end of the day, patches are submitted by real people. If we want

Re: [openstack-dev] [Solum] An early peek into the Solum.io website

[Swapnil Kulkarni]

Adrian ++.

I too believe its a very good initiative started with #Solum & to follow
up. Other projects (existing/incubating/future) will definitely have a
fruitful benefit following similar approach in the long run, thus
benefiting the OpenStack ecosystem :)

Best Regards,
Swapnil



On Thu, Oct 31, 2013 at 7:07 PM, Adrian Otto wrote:

> Joshua,
>
> Good question. It's a project specific initiative, rather than an
> OpenStack Foundation effort. One of the interesting aspects of the
> OpenStack ecosystem is that each OpenStack Related project can work
> individually on growing its community, which in turn feeds the OpenStack
> ecosystem. If this approach is successful, it could be replicated for other
> projects as well. We feel this is an important thing for Solum because of
> its "open from the beginning" approach. This is less important for projects
> that come to openstack after they have taken comprehensive form.
>
> Adrian
>
> On Oct 31, 2013, at 12:04 AM, Joshua Harlow 
>  wrote:
>
> > Just wondering,
> >
> > Will said infrastructure be made available to all other openstack
> projects? It'd be nice to have that for other projects to (a nova blog, a
> taskflow blog...). It seems/feels a little awkward to me to have solum be a
> special snowflake here.
> >
> > Sent from my really tiny device...
> >
> >> On Oct 30, 2013, at 9:55 PM, "Adrian Otto" 
> wrote:
> >>
> >> Noorul,
> >>
> >> Yes, it uses WordPress. The community website will be used primarily as
> a rally point for attracting new contributors. It will have a blog feature
> which is currently being configured so the project contributors can publish
> related content there. We expect to use this site to attract new developers
> who are not already part of the OpenStack ecosystem.
> >>
> >> Adrian
> >>
> >>
> >> On Oct 30, 2013, at 9:25 PM, Noorul Islam K M 
> >> wrote:
> >>
> >>> Roshan Agrawal  writes:
> >>>
>  The Solum community website is very close to be launched publicly.
> 
>  If you want to take an early peek into how it is coming along, here
> are the access details:
>  www.Solum.io User name: Solum, password:
> OpenStack
>  The Solum logo is still in works, what we have now is meant to be a
> placeholder till we finalize on an awesome looking logo.
> 
>  Comments/suggestions welcome. (Cc Solum list for now, till everyone
> have had a chance to migrate to the openstack list)
> >>>
> >>> Is there any dynamic content in this site? Why are we not utilizing
> >>> github pages or something that can be generated from git repository?
> >>>
> >>> Thanks and Regards
> >>> Noorul
> >>>
> >>> ___
> >>> OpenStack-dev mailing list
> >>> OpenStack-dev@lists.openstack.org
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>
> >> ___
> >> OpenStack-dev mailing list
> >> OpenStack-dev@lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Kyle Mestery (kmestery)]

On Oct 31, 2013, at 8:56 AM, Clint Byrum  wrote:
> Excerpts from Mark McLoughlin's message of 2013-10-31 06:30:32 -0700:
>> On Thu, 2013-10-31 at 15:37 +1300, Robert Collins wrote:
>>> This is a bit of a social norms thread
>>> 
>>> I've been consistently asking for tests in reviews for a while now,
>>> and I get the occasional push-back. I think this falls into a few
>>> broad camps:
>>> 
>>> A - there is no test suite at all, adding one in unreasonable
>>> B - this thing cannot be tested in this context (e.g. functional tests
>>> are defined in a different tree)
>>> C - this particular thing is very hard to test
>>> D - testing this won't offer benefit
>>> E - other things like this in the project don't have tests
>>> F - submitter doesn't know how to write tests
>>> G - submitter doesn't have time to write tests
>> 
>> Nice breakdown.
>> 
>>> Now, of these, I think it's fine not add tests in cases A, B, C in
>>> combination with D, and D.
>>> 
>>> I don't think E, F or G are sufficient reasons to merge something
>>> without tests, when reviewers are asking for them. G in the special
>>> case that the project really wants the patch landed - but then I'd
>>> expect reviewers to not ask for tests or to volunteer that they might
>>> be optional.
>> 
>> I totally agree with the sentiment but, especially when it's a newcomer
>> to the project, I try to put myself in the shoes of the patch submitter
>> and double-check whether what we're asking is reasonable.
>> 
> 
> Even with a long time contributor, empathy is an important part of
> constructing reviews. We could make more robotic things that review for
> test coverage, but we haven't, because this is a gray area. The role of
> a reviewer isn't just get patches merged and stop defects. It is also to
> grow the other developers.
> 
>> For example, if someone shows up to Nova with their first OpenStack
>> contribution, it fixes something which is unquestionably a bug - think
>> typo like "raise NotFund('foo')" - and testing this code patch requires
>> more than adding a simple new scenario to existing tests ...
>> 
> 
> This goes back to my recent suggestion to help the person not with a -1
> or a +2, but with an additional patch that fixes it.
> 
>> That, for me, is an example of "-1, we need a test! untested code is
>> broken!" is really shooting the messenger, not valuing the newcomers
>> contribution and risks turning that person off the project forever.
>> Reviewers being overly aggressive about this where the project doesn't
>> have full test coverage to begin with really makes us seem unwelcoming.
>> 
>> In cases like that, I'd be of a mind to go "+2 Awesome! Thanks for
>> catching this! It would be great to have a unit test for this, but it's
>> clear the current code is broken so I'm fine with merging the fix
>> without a test". You could say it's now the reviewers responsibility to
>> merge a test, but if that requirement then turns off reviewers even
>> reviewing such a patch, then that doesn't help either.
>> 
> 
> I understand entirely why you choose this, and I think that is fine. I,
> however, see this as a massive opportunity to teach. That code was only
> broken because it was allowed it to be merged without tests. By letting
> that situation continue, we only fix it today. The next major refactoring
> has a high chance now of breaking that part of the code again.
> 
> So, rather than +2, I suggest -1 with compassion. Engage with the
> submitter. If you don't know them, take a look at how hard it would be
> to write a test for the behavior and give pointers to the exact test
> suite that would need to be changed, or suggest a new test suite and
> point at a good example to copy.
> 
>> So, with all of this, let's make sure we don't forget to first
>> appreciate the effort that went into submitting the patch that lacks
>> tests.
>> 
> 
> I'm not going to claim that I've always practiced "-1 with compassion",
> so thanks for reminding us all that we're not just reviewing code, we
> are having a dialog with real live people.
> 
I think this is the key thing here, thanks for bringing this up Clint. At the
end of the day, patches are submitted by real people. If we want to grow
the committer base and help people to become better reviewers, taking
the time to show them the ropes is part of the game. I'd argue that is in
fact part of what being a core is about in fact.

Thanks,
Kyle

> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Full schedule for HongKong summit?

[Adam Young]

On 10/29/2013 06:29 AM, Thierry Carrez wrote:

Dina Belova wrote:

There is no such mixed calendar. It was made, as far as I remember,
because design and summit talks are very different and should be separated.

But still, it is not really comfortable.

Yes, it's a bit of a trade-off...

In the past we had a lot of people, attracted by catchy session titles,
wandering in the design summit area and be really surprised by the
absence of presentations and the chaotic nature of the discussions
there. For this time around we decided that further separating the two
events (while keeping compatible time slots) was less confusing, and
keeping two separate schedules was the simplest way of making that happen.

I agree it's inconvenient for people who want to juggle between the two
parts of the event. The best solution for those is probably to look at
both schedules and use the iCal/ics exports to build your personal calendar:

http://icehousedesignsummit.sched.org/mobile-site
http://openstacksummitnovember2013.sched.org/mobile-site

Hope this helps,



I would think that this is a common enough occurance for the sched.org 
people that they would have a "nested conference" mechanism.  However, I 
think that this might have a secondary effect of messing up people that 
really do want to participate in the design sessions, but are only aware 
of the main summit schedule. While it is true that we don't want the 
design sessions to be flooded, we do want participation, and balancing 
the two is going to allways err one way or the other.


Security is particularly impacted by this, however, as developers really 
should be at the security sessions, but they are not part of the 
developers schedule.


At a minimum, we should probably have a recurring theme among the early 
day presenters "if you are interested in the design sessions, remember 
that they are on a separate schedule" or some such.



Dealing with this really feels like dealing with design issues in 
Keystone.  What user should see which project...


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] Locking and ZooKeeper - a space oddysey

[Renat Akhmerov]

On 31 окт. 2013 г., at 2:37, Clint Byrum  wrote:

> My point
> is really that we should not care how serialization happens, we should
> just express the work-flow, and let the underlying mechanisms distribute
> and manage it as it is completed.


Sounds reasonable.

In this context, you may want to look at Mistral project we recently started. 
We just published a kind of a high-level description of the use case that is 
relevant for the problem that is being discussed here. It’s called Cloud 
Environment Deployment and you can find it at 
https://wiki.openstack.org/wiki/Mistral/Cloud_Environment_Deployment_details. 
We think it can be a very important application for Mistral. Any kinds of 
locking mechanism, imho, should be avoiding at all system levels unless it’s 
absolutely required for algorithm complexity reasons (when there’s no other 
way). So If we can represent what Heat does in its internals as a set of 
related tasks we can offload dependencies resolution to a system like Mistral 
that would do everything in a distributed manner.

Another interesting feature we’re planning to implement is data flow. That is, 
some state (sort of a context) associated with a workflow travels between nodes 
in a task graph so there’s no need to worry about a shared state in many cases. 
Data transfer itself is supposed to work on top of some HA transport itself 
(like rabbit) so it shouldn’t be a challenge to implement it. Not 100% sure 
that it’s all applicable for solving this Heat task (pardon me), but it 
definitely could be considered as a possibility. Unfortunately, we’ve not 
described this feature very well yet (only some pictures and sketches not 
published anywhere), but we’ll do soon.

I don’t really want it to look like an ad, sorry if it does ( :) ). It would be 
cool if we could collaborate on this. Once you look at our ideas, you could 
provide your input on what else should be taken into account in Mistral design 
in order to address your problem well.

Renat


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Solum] Separation of concerns for 0.1 git deploy blueprint

[Adrian Otto]

Uri,

Yes. We will need to offer a facility for customizing the CD pipeline(s). We 
have not contemplated all of the possibilities for that yet, so this is 
something that we will certainly discuss further. We will need to strike the 
right balance between options and simplicity.

We have thought through the possibility of generating a HOT[1] artifact in the 
case that one is not provided, and allowing a user-supplied one in the event 
that one is supplied with the code at the time of the deployment action. Of 
course if no HOT is provided, we can generate one (or a simple hierarchy of 
them). That option would allow you to cause anything that Heat knows how to do.

We will continue to contribute to Heat (and other downstream projects) to help 
it achieve its mission. From my view, Heat should allow us to connect various 
different imperative systems downstream. For example, if you want something to 
happen on a given node type, you might use Chef or Puppet to accomplish that. 
Features related to orchestration (the things offered in the HOT as 
declarations) should be provided by heat, or one of its resource types. That's 
where I see most features like rollback fitting in. 

The actual design of the logic that allows the promotion of a deployment 
through different Environments[2] will likely be triggered by Solum, but 
implemented downstream in Heat. A system level rollback (user requested, rather 
than initiated by a failure) may actually be achieved by a request to swap 
Environments, or modify one.

Adrian

References:
[1]: 
https://wiki.openstack.org/wiki/Heat/DSL#Heat_Orchestration_Template_.28HOT.29
[2]: https://wiki.openstack.org/wiki/Heat/DSL#Environment

On Oct 31, 2013, at 2:04 AM, Uri Cohen 
 wrote:

> Hi Adrian 
> 
> Sounds like there's a need for a true workflow engine here. I suspect that 
> for any use case beyond trivial, each user will have her own flavor of CD 
> (TripleO is a good example for this) - some will prefer to use a canary 
> server and test it manually before deploying to other servers, some will 
> deploy to specific servers based on the subset of users they're serving, some 
> will do a red/black type of deployment (Which maps well into environments 
> below), etc. But I think that there should be an user-configurable mechanism 
> to define and tweak these workflows. There will also be additional related 
> workflows - e.g. rolling back a deployment, or updating the infrastructure 
> components of a running application (e.g. database version upgrade). 
> I might be raising a topic that's already been discussed (sorry if I am), but 
> I was wondering how this would fit into the architecture. 
> 
> Thanks! 
> Uri 
> 
> On Oct 31, 2013, at 6:42 AM, Adrian Otto  wrote:
> 
>> Robert,
>> 
>> We would be interested in exploring your use case for CD, and help you judge 
>> what would be the best fit. I know that Monty Taylor has identified some 
>> parts of openstack-infra that we could potentially leverage in Solum to 
>> address our CI/CD goals. The design of that feature is still pending, so we 
>> are looking forward to sourcing its of input on that.
>> 
>> I don't anticipate any fundament gaps that would prevent you from using a 
>> future version of Solum to help narrow your scope. The whole point of our 
>> focus is to try to make these goals easier to achieve, and try to get 
>> maximum leverage of what the OpenStack ecosystem already offers.
>> 
>> The high level concept behind the GIt Integration feature currently 
>> conceived for Solum is to offer application developers a way to commit code 
>> to a Git repo and upon a git push command, have a remote system trigger a 
>> modular CI/CD pipeline that ends with that application running on an 
>> OpenStack cloud in one or more "environments" (dev/test/stage/prod/etc.). In 
>> a simple case that may be (in the case of a dynamic language like Python) 
>> the testing of the code, arrangement of the commit into a deployable bundle, 
>> triggering of the Solum API to trigger a deployment, automatic generation of 
>> a HOT artifact, and stimulation of the Heat API and any other API calls 
>> needed.
>> 
>> In a more complex scenario, you may want additional features and 
>> capabilities, such as a system like Gerrit to trigger the commit/push, 
>> integrate an artifact repository for storing the builds, etc. Using Zuul for 
>> that would be appropriate for sure.
>> 
>> There are also dev shops who already have a lot invested into their Jenkins 
>> setups, and will want to bypass portions of what I described above, so there 
>> will be a use case supported where you can just integrate with the solum 
>> API, and pass in a pre-built bundle artifact and bypass part or all of 
>> Solum's CI features, and just use it for deployment and re-deployment and 
>> promotion of a deployment between various environments (dev/test/pros/etc.). 
>> You would expect features like canary deploy in that case that might make it 
>> more attractive 

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Clint Byrum]

Excerpts from Mark McLoughlin's message of 2013-10-31 06:30:32 -0700:
> On Thu, 2013-10-31 at 15:37 +1300, Robert Collins wrote:
> > This is a bit of a social norms thread
> > 
> > I've been consistently asking for tests in reviews for a while now,
> > and I get the occasional push-back. I think this falls into a few
> > broad camps:
> > 
> > A - there is no test suite at all, adding one in unreasonable
> > B - this thing cannot be tested in this context (e.g. functional tests
> > are defined in a different tree)
> > C - this particular thing is very hard to test
> > D - testing this won't offer benefit
> > E - other things like this in the project don't have tests
> > F - submitter doesn't know how to write tests
> > G - submitter doesn't have time to write tests
> 
> Nice breakdown.
> 
> > Now, of these, I think it's fine not add tests in cases A, B, C in
> > combination with D, and D.
> > 
> > I don't think E, F or G are sufficient reasons to merge something
> > without tests, when reviewers are asking for them. G in the special
> > case that the project really wants the patch landed - but then I'd
> > expect reviewers to not ask for tests or to volunteer that they might
> > be optional.
> 
> I totally agree with the sentiment but, especially when it's a newcomer
> to the project, I try to put myself in the shoes of the patch submitter
> and double-check whether what we're asking is reasonable.
> 

Even with a long time contributor, empathy is an important part of
constructing reviews. We could make more robotic things that review for
test coverage, but we haven't, because this is a gray area. The role of
a reviewer isn't just get patches merged and stop defects. It is also to
grow the other developers.

> For example, if someone shows up to Nova with their first OpenStack
> contribution, it fixes something which is unquestionably a bug - think
> typo like "raise NotFund('foo')" - and testing this code patch requires
> more than adding a simple new scenario to existing tests ...
>

This goes back to my recent suggestion to help the person not with a -1
or a +2, but with an additional patch that fixes it.

> That, for me, is an example of "-1, we need a test! untested code is
> broken!" is really shooting the messenger, not valuing the newcomers
> contribution and risks turning that person off the project forever.
> Reviewers being overly aggressive about this where the project doesn't
> have full test coverage to begin with really makes us seem unwelcoming.
> 
> In cases like that, I'd be of a mind to go "+2 Awesome! Thanks for
> catching this! It would be great to have a unit test for this, but it's
> clear the current code is broken so I'm fine with merging the fix
> without a test". You could say it's now the reviewers responsibility to
> merge a test, but if that requirement then turns off reviewers even
> reviewing such a patch, then that doesn't help either.
> 

I understand entirely why you choose this, and I think that is fine. I,
however, see this as a massive opportunity to teach. That code was only
broken because it was allowed it to be merged without tests. By letting
that situation continue, we only fix it today. The next major refactoring
has a high chance now of breaking that part of the code again.

So, rather than +2, I suggest -1 with compassion. Engage with the
submitter. If you don't know them, take a look at how hard it would be
to write a test for the behavior and give pointers to the exact test
suite that would need to be changed, or suggest a new test suite and
point at a good example to copy.

> So, with all of this, let's make sure we don't forget to first
> appreciate the effort that went into submitting the patch that lacks
> tests.
> 

I'm not going to claim that I've always practiced "-1 with compassion",
so thanks for reminding us all that we're not just reviewing code, we
are having a dialog with real live people.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Keystone Concurrency & Races in SQL Assignment Backend

[Brant Knudson]

Peter -

We discussed better use of transactions in irc, but I don't think anyone
has had a chance to look at it. This would be a very useful thing to have
someone look at. I'm fine with holding off on the oslo.db sessions work
until we're sure the code is correct w/r/t multi-processing so that tempest
is going to pass consistently.

- Brant

On Wed, Oct 30, 2013 at 5:08 PM, Peter Feiner  wrote:

> Hi Brant,
>
> In addition to the race you've fixed in
> https://review.openstack.org/#/c/50767/, it looks like there are quite
> a few more races in the SQL backend of keystone.assignment. I filed a
> bug to this effect: https://bugs.launchpad.net/keystone/+bug/1246489.
> The general problem is that transactions are used somewhat
> indiscriminately. The fix (i.e., using transactions judiciously) is
> straightforward and should be mostly independent of your ongoing
> oslo.db sessions port in https://review.openstack.org/#/c/49460/. So,
> unless you already have something in the works, I'll get started on
> that tomorrow.
>
> I'm eager to fix these races so
> https://review.openstack.org/#/c/42967/ can reliably pass tempest :-)
>
> Peter
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon / keystone] Marker could not be found?

[Sebastian Porombka]

Hello Folks.

I have a problem after grizzly->havana migration where i’m unable to rescue 
myself.
When I open the "Admin -> Resource-Usage View i get no results – only a red 
error box with the message "Error: Unable to retrieve tenant list.“.

Horizon log:

[Thu Oct 31 11:39:44 2013] [error] Creating a new keystoneclient connection to 
http://$controller:35357/v2.0.

[Thu Oct 31 11:39:44 2013] [error] REQ: curl -i -X GET 
http://$controller:35357/v2.0/tenants?marker=tenant_marker&limit=21 -H 
"User-Agent: python-keystoneclient" -H "Forwarded: 
for=131.234.5.178;by=python-keystoneclient" -H "X-Auth-Token: 82[…]f46"

[Thu Oct 31 11:39:44 2013] [error] REQ: curl -i -X GET 
http://$controller:35357/v2.0/tenants?marker=tenant_marker&limit=21 -H 
"User-Agent: python-keystoneclient" -H "Forwarded: 
for=131.234.5.178;by=python-keystoneclient" -H "X-Auth-Token: 82[…]46"

[Thu Oct 31 11:39:44 2013] [error] INFO:urllib3.connectionpool:Starting new 
HTTP connection (1): $controller

[Thu Oct 31 11:39:44 2013] [error] DEBUG:urllib3.connectionpool:"GET 
/v2.0/tenants?marker=tenant_marker&limit=21 HTTP/1.1" 400 88

[Thu Oct 31 11:39:44 2013] [error] RESP: [400] CaseInsensitiveDict({'date': 
'Thu, 31 Oct 2013 11:39:47 GMT', 'vary': 'X-Auth-Token', 'content-length': 
'88', 'content-type': 'application/json'})

[Thu Oct 31 11:39:44 2013] [error] RESP BODY: {"error": {"message": "Marker 
could not be found", "code": 400, "title": "Bad Request"}}

[Thu Oct 31 11:39:44 2013] [error]

[Thu Oct 31 11:39:44 2013] [error] RESP: [400] CaseInsensitiveDict({'date': 
'Thu, 31 Oct 2013 11:39:47 GMT', 'vary': 'X-Auth-Token', 'content-length': 
'88', 'content-type': 'application/json'})

[Thu Oct 31 11:39:44 2013] [error] RESP BODY: {"error": {"message": "Marker 
could not be found", "code": 400, "title": "Bad Request"}}

[Thu Oct 31 11:39:44 2013] [error]

[Thu Oct 31 11:39:44 2013] [error] Request returned failure status: 400

[Thu Oct 31 11:39:44 2013] [error] Request returned failure status: 400

[Thu Oct 31 11:39:44 2013] [error] Recoverable error: Marker could not be found 
(HTTP 400)

Keystone Log:
2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Matched GET /tenants 
__call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100

2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Route path: 
'{path_info:.*}', defaults: {'controller': 
} __call__ 
/usr/lib/python2.7/dist-packages/routes/middleware.py:102

2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Match dict: 
{'controller': , 
'path_info': '/tenants'} __call__ 
/usr/lib/python2.7/dist-packages/routes/middleware.py:103

2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Matched GET /tenants 
__call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100

2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Route path: 
'{path_info:.*}', defaults: {'controller': 
} __call__ 
/usr/lib/python2.7/dist-packages/routes/middleware.py:102

2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Match dict: 
{'controller': , 
'path_info': '/tenants'} __call__ 
/usr/lib/python2.7/dist-packages/routes/middleware.py:103

2013-10-31 12:39:47.354 17187 DEBUG routes.middleware [-] Matched GET /tenants 
__call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100

2013-10-31 12:39:47.354 17187 DEBUG routes.middleware [-] Route path: 
'{path_info:.*}', defaults: {'controller': 
} __call__ 
/usr/lib/python2.7/dist-packages/routes/middleware.py:102

2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Match dict: 
{'controller': , 'path_info': '/tenants'} __call__ 
/usr/lib/python2.7/dist-packages/routes/middleware.py:103

2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Matched GET /tenants 
__call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100

2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Route path: 
'{path_info:.*}', defaults: {'controller': 
} __call__ 
/usr/lib/python2.7/dist-packages/routes/middleware.py:102

2013-10-31 12:39:47.356 17187 DEBUG routes.middleware [-] Match dict: 
{'controller': , 
'path_info': '/tenants'} __call__ 
/usr/lib/python2.7/dist-packages/routes/middleware.py:103

2013-10-31 12:39:47.356 17187 DEBUG routes.middleware [-] Matched GET /tenants 
__call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100

2013-10-31 12:39:47.357 17187 DEBUG routes.middleware [-] Route path: 
'/tenants', defaults: {'action': u'get_all_projects', 'controller': 
} __call__ 
/usr/lib/python2.7/dist-packages/routes/middleware.py:102

2013-10-31 12:39:47.357 17187 DEBUG routes.middleware [-] Match dict: 
{'action': u'get_all_projects', 'controller': 
} __call__ 
/usr/lib/python2.7/dist-packages/routes/middleware.py:103

2013-10-31 12:39:47.357 17187 DEBUG dogpile.core.dogpile [-] 
NeedRegenerationException _enter 
/usr/lib/python2.7/dist-packages/dogpile/core/dogpile.py:94

2013-10-31 12:39:47.358 17187 DEBUG dogpile.core.dogpile [-] no value, waiting 
for create lock _enter_create 
/u

Re: [openstack-dev] [Solum] An early peek into the Solum.io website

[Adrian Otto]

Joshua,

Good question. It's a project specific initiative, rather than an OpenStack 
Foundation effort. One of the interesting aspects of the OpenStack ecosystem is 
that each OpenStack Related project can work individually on growing its 
community, which in turn feeds the OpenStack ecosystem. If this approach is 
successful, it could be replicated for other projects as well. We feel this is 
an important thing for Solum because of its "open from the beginning" approach. 
This is less important for projects that come to openstack after they have 
taken comprehensive form.

Adrian

On Oct 31, 2013, at 12:04 AM, Joshua Harlow 
 wrote:

> Just wondering,
> 
> Will said infrastructure be made available to all other openstack projects? 
> It'd be nice to have that for other projects to (a nova blog, a taskflow 
> blog...). It seems/feels a little awkward to me to have solum be a special 
> snowflake here.
> 
> Sent from my really tiny device...
> 
>> On Oct 30, 2013, at 9:55 PM, "Adrian Otto"  wrote:
>> 
>> Noorul,
>> 
>> Yes, it uses WordPress. The community website will be used primarily as a 
>> rally point for attracting new contributors. It will have a blog feature 
>> which is currently being configured so the project contributors can publish 
>> related content there. We expect to use this site to attract new developers 
>> who are not already part of the OpenStack ecosystem. 
>> 
>> Adrian
>> 
>> 
>> On Oct 30, 2013, at 9:25 PM, Noorul Islam K M 
>> wrote:
>> 
>>> Roshan Agrawal  writes:
>>> 
 The Solum community website is very close to be launched publicly.
 
 If you want to take an early peek into how it is coming along, here are 
 the access details:
 www.Solum.io User name: Solum, password: OpenStack
 The Solum logo is still in works, what we have now is meant to be a 
 placeholder till we finalize on an awesome looking logo.
 
 Comments/suggestions welcome. (Cc Solum list for now, till everyone have 
 had a chance to migrate to the openstack list)
>>> 
>>> Is there any dynamic content in this site? Why are we not utilizing
>>> github pages or something that can be generated from git repository?
>>> 
>>> Thanks and Regards
>>> Noorul
>>> 
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Mark McLoughlin]

On Thu, 2013-10-31 at 15:37 +1300, Robert Collins wrote:
> This is a bit of a social norms thread
> 
> I've been consistently asking for tests in reviews for a while now,
> and I get the occasional push-back. I think this falls into a few
> broad camps:
> 
> A - there is no test suite at all, adding one in unreasonable
> B - this thing cannot be tested in this context (e.g. functional tests
> are defined in a different tree)
> C - this particular thing is very hard to test
> D - testing this won't offer benefit
> E - other things like this in the project don't have tests
> F - submitter doesn't know how to write tests
> G - submitter doesn't have time to write tests

Nice breakdown.

> Now, of these, I think it's fine not add tests in cases A, B, C in
> combination with D, and D.
> 
> I don't think E, F or G are sufficient reasons to merge something
> without tests, when reviewers are asking for them. G in the special
> case that the project really wants the patch landed - but then I'd
> expect reviewers to not ask for tests or to volunteer that they might
> be optional.

I totally agree with the sentiment but, especially when it's a newcomer
to the project, I try to put myself in the shoes of the patch submitter
and double-check whether what we're asking is reasonable.

For example, if someone shows up to Nova with their first OpenStack
contribution, it fixes something which is unquestionably a bug - think
typo like "raise NotFund('foo')" - and testing this code patch requires
more than adding a simple new scenario to existing tests ...

That, for me, is an example of "-1, we need a test! untested code is
broken!" is really shooting the messenger, not valuing the newcomers
contribution and risks turning that person off the project forever.
Reviewers being overly aggressive about this where the project doesn't
have full test coverage to begin with really makes us seem unwelcoming.

In cases like that, I'd be of a mind to go "+2 Awesome! Thanks for
catching this! It would be great to have a unit test for this, but it's
clear the current code is broken so I'm fine with merging the fix
without a test". You could say it's now the reviewers responsibility to
merge a test, but if that requirement then turns off reviewers even
reviewing such a patch, then that doesn't help either.

So, with all of this, let's make sure we don't forget to first
appreciate the effort that went into submitting the patch that lacks
tests.

Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] Duplicated test effort development

[Lingxian Kong]

a huge +1!

thanks, chris, noticed the spreadsheet before, and have add some contents I
have been working on. I think more people should know that, avoid
conflicting.


2013/10/31 Zhu Bo 

>  hi, Chris thanks for your work.
>
> It's a good way. And how about  creating a blue-print and putting these
> link on it? Some guys have been familiar with looking for work items from
> blue-print, and writing something in  hacking guide may be better.
>
>
> On 2013年10月30日 21:10, Christopher Yeoh wrote:
>
>  On Wed, Oct 30, 2013 at 11:18 PM, Steven Hardy  wrote:
>
>> On Wed, Oct 30, 2013 at 11:00:22PM +1030, Christopher Yeoh wrote:
>> 
>> > I don't think blueprints/bugs work so
>> > well at this, and I don't think we have anything else setup at the
>> moment,
>> > so as a temporary measure I've created an etherpad here:
>>
>>  Can I ask why?  Surely blueprints for new features (in this case the
>> feature is test coverage for $api) are exactly what the normal openstack
>> process dictates, and is what most folks are familiar with?
>>
>>
>  Just to be clear - Its not that I think that we shouldn't have
> blueprints which covers the work being done (we should!), but they don't
> work so well at allowing people to see a good summary of what test coverage
> for an API we have (some of which may have been done a long time ago), what
> needs to be done and the quite fine grained allocation of what people are
> working on.
>
>  For example, see the tempest coverage for the Nova v2 API spreadsheet:
> https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc#gid=0
>
>
>
>> Anyway, I added the keystone test I'm working on (which has a BP) to the
>> etherpad, and definitely +1 on not duplicating effort, by whatever means
>> ;)
>>
>> Steve
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> ___
> OpenStack-dev mailing 
> listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
*---*
*Lingxian Kong*
Huawei Technologies Co.,LTD.
IT Product Line CloudOS PDU
China, Xi'an
Mobile: +86-18602962792
Email: konglingx...@huawei.com; anlin.k...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Sean Dague]

On 10/30/2013 10:37 PM, Robert Collins wrote:

This is a bit of a social norms thread

I've been consistently asking for tests in reviews for a while now,
and I get the occasional push-back. I think this falls into a few
broad camps:

A - there is no test suite at all, adding one in unreasonable
B - this thing cannot be tested in this context (e.g. functional tests
are defined in a different tree)
C - this particular thing is very hard to test
D - testing this won't offer benefit
E - other things like this in the project don't have tests
F - submitter doesn't know how to write tests
G - submitter doesn't have time to write tests

Now, of these, I think it's fine not add tests in cases A, B, C in
combination with D, and D.

I don't think E, F or G are sufficient reasons to merge something
without tests, when reviewers are asking for them. G in the special
case that the project really wants the patch landed - but then I'd
expect reviewers to not ask for tests or to volunteer that they might
be optional.

Now, if I'm wrong, and folk have different norms about when to accept
'reason X not to write tests' as a response from the submitter -
please let me know!


First - providing tests with code is a basic design tenant of OpenStack 
- https://wiki.openstack.org/wiki/BasicDesignTenets (I realize we don't 
highlight that as much as we used to, but it really is baked in culture 
to people that have been here for a while).


This is pretty close to my thinking. I like to challenge people on C. 
There was a pretty good case of it late in grizzly or havana where I was 
reviewing some nova virt code and the sumbitter claimed it was too hard 
to test. A week later danpb wrote a 100 line patch that provided the 
entire test framework for the case. So "C" was a pretty BS answer.


Adding tests and test frameworks are a way of paying it forward in the 
project. Much like doing reviews, adding tests is a way to save other 
people on the project time. Good tests pay for themselves very quickly 
in time lost on the project.


So I generally find "I don't want to add tests" the same kind of 
selfishness as "why should I have to review code?".


A good example of the kind of savings we get, over the summer when were 
were regularlly breaking on requirements mismatch (and on the update 
script modifying things incorrectly), I went in and added a testing 
framework to requirements repo, so we knew all the parts were doing 
expected things. Without that we'd still be fighting these things on a 
weekly basis, now it's a solved problem, and we can go fight the next fire.


This was actually as C class problem (which is why no one else had done 
it yet), and required building new devstack job to complete. However the 
benefits were really high on it.


-Sean

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposal for the new service: MagnetoDB, DynamoDB API implementation for OpenStack

[Ilya Sviridov]

Hi Thierry,

Yes, you are right. AWS SimpleDB and DynamoDB look alike.
I think in API implementation itself there are a lot of common parts.

But exactly backend makes the difference.

Planning to engage Trove for datasource provisioning and management, the
key features of MagnetoDB is autoscaling and high-availability.
By design we plan to achieve the constant response time regardless of
amount of data.
In order to achieve such qualities the backend part should satisfy that
requirement itself and obviously not always can be applicable for general
cases.

However I believe that Soren have made huge progress, of course I would
like to discus it and maybe join our efforts.

I wrote him without response yet.

Thank you for input.

With best regards,
Ilya Sviridov




On Thu, Oct 31, 2013 at 12:01 PM, Thierry Carrez wrote:

> Ilya Sviridov wrote:
> > I would like to propose a new initiative to implement AWS DynamoDB API
> > for OpenStack.
> > [...]
>
> There is a parallel effort by Soren to create an equivalent of SimpleDB:
> http://blog.linux2go.dk/tag/basicdb/
>
> I'm not familiar enough with SimpleDB/DynamoDB, but I wonder if we
> shouldn't have a single service to handle both cases, as the backend
> side looks reusable to me.
>
> You guys might want to talk together.
>
> --
> Thierry Carrez (ttx)
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] When is it okay for submitters to say 'I don't want to add tests' ?

[Rosa, Andrea (HP Cloud Services)]

Hi all, 

>This is a bit of a social norms thread
>
>I've been consistently asking for tests in reviews for a while now, and I get 
>the
>occasional push-back. I think this falls into a few broad camps:
>
>A - there is no test suite at all, adding one in unreasonable B - this thing 
>cannot
>be tested in this context (e.g. functional tests are defined in a different 
>tree)
>C - this particular thing is very hard to test D - testing this won't offer 
>benefit E
>- other things like this in the project don't have tests F - submitter doesn't
>know how to write tests G - submitter doesn't have time to write tests
>
>Now, of these, I think it's fine not add tests in cases A, B, C in combination
>with D, and D.

In my opinion  C instead of being an acceptable reason for not having tests is 
a symptom of one of the two things:
1) F => Submitter doesn't know how to write tests, in this case someone else 
can help with suggestions
2) The code we are trying to test is too complicated so it's time to refactor it

And about D, In my opinion  tests always offer benefits, like code coverage or 
helping in understanding the code.

What do you think?

Regards
--
Andrea Rosa

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] changing old migrations is verboten

[Sean Dague]

So there is a series of patches starting with - 
https://review.openstack.org/#/c/53417/ that go back and radically 
change existing migration files.


This is really a no-no, unless there is a critical bug fix that 
absolutely requires it. Changing past migrations should be considered 
with the same level of weight as an N-2 backport, only done when there 
is huge upside to the change.


I've -2ed the first 2 patches in the series, though that review applies 
to all of them (I figured a mailing list thread was probably more useful 
than -2ing everything in the series).


There needs to be really solid discussion about the trade offs here 
before contemplating something as dangerous as this.


-Sean

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack's OpenPGP Web of Trust

[Thierry Carrez]

Flavio Percoco wrote:
> On 31/10/13 11:06 +0100, Thierry Carrez wrote:
>> The summit being a bit huge it can be difficult for the PGP-minded to
>> cross paths. We could informally designate one of the Design Summit
>> breaks (Thursday 10:30am ?) as a keysigning break and encourage people
>> interested to spend some time in the dev lounge during that break.
> 
> +1

OK, made it official:

http://icehousedesignsummit.sched.org/event/435822fb3258e31075c8b57d2d57dbf0

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev