[openstack-dev] [swiftclient]Implement swift service-list in python-swiftclient
Hi, In Horizon dashboard, under Admin- System Info we have service lists for Compute and Block Storage. I have filed a blueprint to populate the Swift services there. But while going through the implementation details of Compute Services and Block Storage Services i got to know that the details there is populated through api calls to python-novaclient and python-cinderclient respectively which in turn uses nova service-list and cinder service-list to return the details. Whereas no such method is implemented in python-swiftclient to get the list of services. So my question is, Do we have plans to include swift service-list in swiftclient ? If yes then I would be filing a blueprint in python-swiftclient to implement the same coz I require it to populate under the Admin - System Info - Object Storage Services. As a side note I can also see it has also not been implemented in some other services like glance and heat. Is it a design decision or the feature has not been simply impemented. Thanks and Regards Ashish Chandra ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting
Excerpts from Samuel Merritt's message of 2014-09-09 19:04:58 -0700: On 9/9/14, 4:47 PM, Devananda van der Veen wrote: On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt s...@swiftstack.com wrote: On 9/9/14, 12:03 PM, Monty Taylor wrote: [snip] So which is it? Because it sounds like to me it's a thing that actually does NOT need to diverge in technology in any way, but that I've been told that it needs to diverge because it's delivering a different set of features - and I'm pretty sure if it _is_ the thing that needs to diverge in technology because of its feature set, then it's a thing I don't think we should be implementing in python in OpenStack because it already exists and it's called AMQP. Whether Zaqar is more like AMQP or more like email is a really strange metric to use for considering its inclusion. I don't find this strange at all -- I had been judging the technical merits of Zaqar (ex-Marconi) for the last ~18 months based on the understanding that it aimed to provide Queueing-as-a-Service, and found its delivery of that to be lacking on technical grounds. The implementation did not meet my view of what a queue service should provide; it is based on some serious antipatterns (storing a queue in an RDBMS is probably the most obvious); and in fact, it isn't even queue-like in the access patterns enabled by the REST API (random access to a set != a queue). That was the basis for a large part of my objections to the project over time, and a source of frustration for me as the developers justified many of their positions rather than accepted feedback and changed course during the incubation period. The reason for this seems clear now... As was pointed out in the TC meeting today, Zaqar is (was?) actually aiming to provide Messaging-as-a-Service -- not queueing as a service! This is another way of saying it's more like email and less like AMQP, which means my but-its-not-a-queue objection to the project's graduation is irrelevant, and I need to rethink about all my previous assessments of the project. The questions now before us are: - should OpenStack include, in the integrated release, a messaging-as-a-service component? I certainly think so. I've worked on a few reasonable-scale web applications, and they all followed the same pattern: HTTP app servers serving requests quickly, background workers for long-running tasks, and some sort of durable message-broker/queue-server thing for conveying work from the first to the second. A quick straw poll of my nearby coworkers shows that every non-trivial web application that they've worked on in the last decade follows the same pattern. While not *every* application needs such a thing, web apps are quite common these days, and Zaqar satisfies one of their big requirements. Not only that, it does so in a way that requires much less babysitting than run-your-own-broker does. I think you missed the distinction. What you describe is _message queueing_. Not messaging. The difference being the durability and addressability of each message. As Devananda pointed out, a queue doesn't allow addressing the items in the queue directly. You can generally only send, receive, ACK, or NACK. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][neutron] default allow security group
Not arguing if it's suitable to implement this with security-group commands. To solve the problem, I guess no 20 rules are necessary at all. You can just add one rules like the following to allow all traffic going out of the vm. iptables -I neutron-openvswi-o9LETTERID -j RETURN Where the id part is the first 9 letters of the vm attached port id. This rule will bypass all security filtering for the outgoing traffic. On Fri, Sep 5, 2014 at 11:27 PM, Monty Taylor mord...@inaugust.com wrote: Hi! I've decided that as I have problems with OpenStack while using it in the service of Infra, I'm going to just start spamming the list. Please make something like this: neutron security-group-create default --allow-every-damn-thing Right now, to make security groups get the hell out of our way because they do not provide us any value because we manage our own iptables, it takes adding something like 20 rules. 15:24:05 clarkb | one each for ingress and egress udp tcp over ipv4 then ipv6 and finaly icmp That may be great for someone using my-first-server-pony, but for me, I know how the internet works, and when I ask for a server, I want it to just work. Now, I know, I know - the DEPLOYER can make decisions blah blah blah. BS If OpenStack is going to let my deployer make the absolutely assinine decision that all of my network traffic should be blocked by default, it should give me, the USER, a get out of jail free card. kthxbai ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best wishes! Baohua ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] convergence flow diagrams
Excerpts from Angus Salkeld's message of 2014-09-08 17:15:04 -0700: On Mon, Sep 8, 2014 at 11:22 PM, Tyagi, Ishant ishant.ty...@hp.com wrote: Hi All, As per the heat mid cycle meetup whiteboard, we have created the flowchart and sequence diagram for the convergence . Can you please review these diagrams and provide your feedback? https://www.dropbox.com/sh/i8qbjtgfdxn4zx4/AAC6J-Nps8J12TzfuCut49ioa?dl=0 Great! Good to see something. I was expecting something like: engine ~= like nova-conductor (it's the only process that talks to the db - make upgrading easier) This complicates things immensely. The engine can just be the workers too, we're just not going to do the observing and converging in the same greenthread. observer - purely gets the actual state/properties and writes then to the db (via engine) If you look closely at the diagrams, thats what it does. worker - has a job queue and grinds away at running those (resource actions) The convergence worker is just another set of RPC API calls that split out work into isolated chunks. Then engine then triggers on differences on goal vs. actual state and create a job and sends it to the job queue. Remember, we're not targeting continuous convergence yet. Just convergence when we ask for things. - so, on create it sees there is no actual state so it sends a create job for the first resource to the worker queue The diagram shows that, but confusingly says is difference = 1. In the original whiteboard this is 'if diff = DNE'. DNE stands for Does Not Exist. - when the observer writes the new state for that resource it triggers the next resource create in the dependency tree. Not the next resource create, but the next resource convergence. And not just one either. I think one of the graphs was forgotten, it goes like this: https://www.dropbox.com/s/1h2ee151iriv4i1/resolve_graph.svg?dl=0 That is what we called return happy because we were at hour 9 or so of talking and we got a bit punchy. I've renamed it 'resolve_graph'. - like any system that relies on notifications we need timeouts and each stack needs a periodic notification to make sure This is, again, the continuous observer model. https://review.openstack.org/#/c/100012/ that progress is been made or notify the user that no progress is being made. One question about the observer (in either my setup or the one in the diagram). - If we are relying on rpc notifications all the observer processes will receive a copy of the same notification Please read that spec. We talk about a filter. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All] Maintenance mode in OpenStack during patching/upgrades
It would be great if each OpenStack component could provide a maintenance mode like this… there was some work being considered on Cells https://blueprints.launchpad.net/nova/+spec/disable-child-cell-support which would have allowed parts of Nova to indicate they were in maintenance. Something generic would be very useful. Some operators have asked for ‘read-only’ modes also where query is OK but update is not permitted. Tim From: Mike Scherbakov [mailto:mscherba...@mirantis.com] Sent: 09 September 2014 23:20 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [All] Maintenance mode in OpenStack during patching/upgrades Sergii, Clint, to rephrase what you are saying - there are might be situations when our OpenStack API will not be responding, as simply services would be down for upgrade. Do we want to support it somehow? For example, if we know that Nova is going to be down, can we respond with HTTP 503 with appropriate Retry-After time in header? The idea is not simply deny or hang requests from clients, but provide them we are in maintenance mode, retry in X seconds Turbo Hipster was added to the gate great idea, I think we should use it in Fuel too You probably would want 'nova host-servers-migrate host' yeah for migrations - but as far as I understand, it doesn't help with disabling this host in scheduler - there is can be a chance that some workloads will be scheduled to the host. On Tue, Sep 9, 2014 at 6:02 PM, Clint Byrum cl...@fewbar.commailto:cl...@fewbar.com wrote: Excerpts from Mike Scherbakov's message of 2014-09-09 00:35:09 -0700: Hi all, please see below original email below from Dmitry. I've modified the subject to bring larger audience to the issue. I'd like to split the issue into two parts: 1. Maintenance mode for OpenStack controllers in HA mode (HA-ed Keystone, Glance, etc.) 2. Maintenance mode for OpenStack computes/storage nodes (no HA) For first category, we might not need to have maintenance mode at all. For example, if we apply patching/upgrade one by one node to 3-node HA cluster, 2 nodes will serve requests normally. Is that possible for our HA solutions in Fuel, TripleO, other frameworks? You may have a broken cloud if you are pushing out an update that requires a new schema. Some services are better than others about handling old schemas, and can be upgraded before doing schema upgrades. But most of the time you have to do at least a brief downtime: * turn off DB accessing services * update code * run db migration * turn on DB accessing services It is for this very reason, I believe, that Turbo Hipster was added to the gate, so that deployers running against the upstream master branches can have a chance at performing these upgrades in a reasonable amount of time. For second category, can not we simply do nova-manage service disable..., so scheduler will simply stop scheduling new workloads on particular host which we want to do maintenance on? You probably would want 'nova host-servers-migrate host' at that point, assuming you have migration set up. http://docs.openstack.org/user-guide/content/novaclient_commands.html On Thu, Aug 28, 2014 at 6:44 PM, Dmitry Pyzhov dpyz...@mirantis.commailto:dpyz...@mirantis.com wrote: All, I'm not sure if it deserves to be mentioned in our documentation, this seems to be a common practice. If an administrator wants to patch his environment, he should be prepared for a temporary downtime of OpenStack services. And he should plan to perform patching in advance: choose a time with minimal load and warn users about possible interruptions of service availability. Our current implementation of patching does not protect from downtime during the patching procedure. HA deployments seems to be more or less stable. But it looks like it is possible to schedule an action on a compute node and get an error because of service restart. Deployments with one controller... well, you won’t be able to use your cluster until the patching is finished. There is no way to get rid of downtime here. As I understand, we can get rid of possible issues with computes in HA. But it will require migration of instances and stopping of nova-compute service before patching. And it will make the overall patching procedure much longer. Do we want to investigate this process? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Mike Scherbakov #mihgen ___ OpenStack-dev mailing list
Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting
On 09/10/2014 01:47 AM, Devananda van der Veen wrote: On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt s...@swiftstack.com wrote: On 9/9/14, 12:03 PM, Monty Taylor wrote: [snip] So which is it? Because it sounds like to me it's a thing that actually does NOT need to diverge in technology in any way, but that I've been told that it needs to diverge because it's delivering a different set of features - and I'm pretty sure if it _is_ the thing that needs to diverge in technology because of its feature set, then it's a thing I don't think we should be implementing in python in OpenStack because it already exists and it's called AMQP. Whether Zaqar is more like AMQP or more like email is a really strange metric to use for considering its inclusion. I don't find this strange at all -- I had been judging the technical merits of Zaqar (ex-Marconi) for the last ~18 months based on the understanding that it aimed to provide Queueing-as-a-Service, and found its delivery of that to be lacking on technical grounds. The implementation did not meet my view of what a queue service should provide; it is based on some serious antipatterns (storing a queue in an RDBMS is probably the most obvious); and in fact, it isn't even queue-like in the access patterns enabled by the REST API (random access to a set != a queue). That was the basis for a large part of my objections to the project over time, and a source of frustration for me as the developers justified many of their positions rather than accepted feedback and changed course during the incubation period. The reason for this seems clear now... Let me clarify some points, again: 1. We have never said that our recommended driver is an RBDMS. That driver was added for reasons that were already explained in this thread.[0] 2. The get-message-by-id feature/bug was not added as 'queue' feature. When we added it, we were working on messages pagination, that is, a way to iterate and keep getting messages out of the queue by following the lead of the last message. When the pagination thing was completed, adding the get-message-by-id endpoint was easy and cheap, hence we added it. Our current API version is 1.1 and we *can't* remove that endpoint until the next major version. 3. You saying that we just justified our positions rather than accepting the feedback is what frustrates me the most and it just means, with all due respect, that you haven't followed the project close enough. We've addressed feedback from *EVERYONE* whenever that was possible. As far as I can see, your frustration is mostly related to the fact that we haven't removed the `get-message-by-id` endpoint but as I've mentioned, we can't do it right now. However, we have talked about it several times in meetings and a couple of times on the mailing list.[1][2] [0] http://lists.openstack.org/pipermail/openstack-dev/2014-March/030367.html [1] http://lists.openstack.org/pipermail/openstack-dev/2014-May/036131.html [2] http://lists.openstack.org/pipermail/openstack-dev/2014-August/044213.html As was pointed out in the TC meeting today, Zaqar is (was?) actually aiming to provide Messaging-as-a-Service -- not queueing as a service! This is another way of saying it's more like email and less like AMQP, which means my but-its-not-a-queue objection to the project's graduation is irrelevant, and I need to rethink about all my previous assessments of the project. Zaqar is a messaging service, we changed that almost right before the Juno summit to help clarify the project goals. [0] [0] https://wiki.openstack.org/wiki/Zaqar#Overview The questions now before us are: - should OpenStack include, in the integrated release, a messaging-as-a-service component? - is Zaqar a technically sound implementation of such a service? I believe there is. I've been collecting use-cases from integrated projects and other users. I've put them here[0] [0] https://etherpad.openstack.org/p/zaqar-integrated-projects-use-cases As an aside, there are still references to Zaqar as a queue in both the wiki [0], in the governance repo [1], and on launchpad [2]. I'm sorry about this, apparently the change was not spread enough throughout our community, again. Thanks for your feedback, Flavio [0] Multi-tenant queues based on Keystone project IDs https://wiki.openstack.org/wiki/Zaqar#Key_features [1] Queue service is even the official OpenStack Program name, and the mission statement starts with To produce an OpenStack message queueing API and service. http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n315 [2] Zaqar is a new OpenStack project to create a multi-tenant cloud queuing service https://launchpad.net/zaqar ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- @flaper87 Flavio Percoco
Re: [openstack-dev] [TripleO] Review metrics - what do we want to measure?
There was some discussion of this topic during today's meeting. Full notes are at http://eavesdrop.openstack.org/meetings/tripleo/2014/tripleo.2014-09-09-19.04.log.html starting around 19:24 To summarise, we had one action item, and one comment from dprince which attracted broad agreement, and which I think is worth repeating here: 19:34:26 tchaypo #action bnemec to look at adding a report of items that have a -1 from a core but no response for 14 days, as a first step towards possibly auto-WIPing these patches 19:41:54 dprince I sort of feel like all the focus on stats in our meetings is going to encourage people to game them (i.e. fly by reviews) 19:42:13 dprince when what we really need is ownershipt to push through the important things... 19:42:45 * dprince thinkgs stats are becoming a TripleO waste of time 19:44:12 jp_at_hp stats not important, being active and responsive enough to not push new contributors away does matter 19:44:51 lifeless jp_at_hp: I agree; the stats are a tool, not the thing itself. On Fri, Sep 5, 2014 at 9:47 PM, Sullivan, Jon Paul jonpaul.sulli...@hp.com wrote: -Original Message- From: Jay Dobies [mailto:jason.dob...@redhat.com] Sent: 04 September 2014 18:24 To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [TripleO] Review metrics - what do we want to measure? It can, by running your own... but again it seems far better for core reviewers to decide if a change has potential or needs to be abandoned--that way there's an accountable human making that deliberate choice rather than the review team hiding behind an automated process so that no one is to blame for hurt feelings besides the infra operators who are enforcing this draconian measure for you. The thing is that it's also pushing more work onto already overloaded core review teams. Maybe submitters don't like auto-abandon, but I bet they like having a core reviewer spending time cleaning up dead reviews instead of reviewing their change even less. TBH, if someone's offended by the bot then I can't imagine how incensed they must be when a human does the same thing. The bot clearly isn't making it personal, and even if the human isn't either it's much easier to have misunderstandings (see also every over- reaction to a -1 ever). I suppose it makes it easier for cores to ignore reviews, but from the other discussions I've read that hasn't gone away just because auto-abandon did, so I'm not convinced that's a solution anyway. +1, I don't think it'll come as much of a shock if a -1 review gets closed due to time without progress. +1 to auto-abandon. Taking an excerpt from Dereks mail: A patch got negative feedback and we're automating the process to prompt the submitter to deal with it. It may be more friendly if it was a 2 step process 1. (after a few days if inactivity) Add a comment saying you got negative feedback with suggestions of how to proceed and information that the review will be autoabandoned if nothing is done in X number of days. 2. Auto abandon patch, with as much information as possible on how to reopen if needed. This sounds like the best solution. A 1 week warning and a 2 week abandoning. It is about as welcoming as this sort of thing could be for a new committer, whilst also avoiding an ever-growing backlog of changes that will never see attention because the submitter has moved on to other things. There is also the benefit of prompting submitters into action when things get auto-abandoned, rather than them having sitting at priority #2 or #3 forever. I was never particularly upset over the auto-abandon, just click a single button in the ui and get on with it. /2 cents To make the whole process a little friendlier we could increase the time frame from 1 week to 2. snarkHow about just automatically abandon any new change as soon as it's published, and if the contributor really feels it's important they'll unabandon it./snark ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Thanks, Jon-Paul Sullivan ☺ Cloud Services - @hpcloud Postal Address: Hewlett-Packard Galway Limited, Ballybrit Business Park, Galway. Registered Office: Hewlett-Packard Galway Limited, 63-74 Sir John Rogerson's Quay, Dublin 2. Registered Number: 361933 The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender. To any recipient of this message within
[openstack-dev] [Neutron][QoS] Applying QoS as Quota
Hello everyone, Looking at the QoS blueprint (proposed for incubation), I suggest to consider adding some parameters to Neutron Quotas. Let's suppose using rate-limit for managing QoS. The quota parameters could be such as rate_limit (per port) and max_bandwidth (per tenant). In this way it is possible to set/manage QoS quotas from the admin side, and for instance set the maximum bandwidth allowed per tenant (cumulative). What do you think about it? Regards, Giuseppe Giuseppe Cossu CREATE-NET ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] i need some help on this bug Bug #1365892
After some research, i find the reason for the cycle reference. In closure, the
_fix_paswords.func_closre reference the _fix_passwords. So the
cycle reference happened.
And in https://thp.io/2012/python-gc/python_gc_final_2012-01-22.pdf page 5, it
says that
We observe that Python implementations with distinct GCs behave differently:
CPython does not even try to get the order of finalizers right, and
simply puts uncollectable objects into the global list of garbage for the
developer to deal with manually.
So the gc can not handle cycle reference, then the memory leak happened.
If there is something wrong, please fix it. Thanks
--
Best
Li Tianqing
在 2014-09-10 11:52:28，Li Tianqing jaze...@163.com 写道：
Hello,
I use backdoor of eventlet to enable gc.DEBUG_LEAK, and after wait a few
minutes, i can sure that there will some objects that can not be collected by
gc.collect in gc.garbage.
Those looks like this (catched in ceilometer-collector)
['_context_auth_token', 'auth_token', 'new_pass'],
(cell at 0x363bc58: list object at 0x361c170,
cell at 0x363bda8: function object at 0x361a5f0),
function _fix_passwords at 0x361a5f0,
cell at 0x363bde0: list object at 0x363c680,
cell at 0x363bd70: function object at 0x361a668,
['_context_auth_token', 'auth_token', 'new_pass'],
(cell at 0x363bde0: list object at 0x363c680,
cell at 0x363bd70: function object at 0x361a668),
function _fix_passwords at 0x361a668,
cell at 0x363bf30: list object at 0x361b680,
cell at 0x363e168: function object at 0x361a758,
['_context_auth_token', 'auth_token', 'new_pass'],
(cell at 0x363bf30: list object at 0x361b680,
cell at 0x363e168: function object at 0x361a758),
function _fix_passwords at 0x361a758,
cell at 0x363e1a0: list object at 0x363cdd0,
cell at 0x363e130: function object at 0x361a7d0,
and i suspect those code in oslo.messaging
def _safe_log(log_func, msg, msg_data):
Sanitizes the msg_data field before logging.
SANITIZE = ['_context_auth_token', 'auth_token', 'new_pass']
def _fix_passwords(d):
Sanitizes the password fields in the dictionary.
for k in d.iterkeys():
if k.lower().find('password') != -1:
d[k] = 'SANITIZED'
elif k.lower() in SANITIZE:
d[k] = 'SANITIZED'
elif isinstance(d[k], dict):
_fix_passwords(d[k])
return d
return log_func(msg, _fix_passwords(copy.deepcopy(msg_data)))
i can resolve this problem by add _fix_passwords = None before _safe_log
returns.
But i do not really understand why this problem happened, and in depth why the
gc can not collect those object. Although i can make those uncollectable
objects disappeared.
But this is not good enough, because if you do not understand it you will write
out some code like this in future, and then also has memory leak too.
So can some one helps me give some detailed on recursive closure used like the
code above, and on why gc can not collect them.
Thanks a lot lot ..
--
Best
Li Tianqing
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][QoS] Applying QoS as Quota
Using the quota system would be a nice option to have. Can you clarify what you mean by cumulative bandwidth for the tenant? It would be possible to rate limit at the tenant router, but having a cumulative limit enforced inside of a tenant would be difficult. On Wed, Sep 10, 2014 at 1:03 AM, Giuseppe Cossu giuseppe.co...@create-net.org wrote: Hello everyone, Looking at the QoS blueprint (proposed for incubation), I suggest to consider adding some parameters to Neutron Quotas. Let’s suppose using rate-limit for managing QoS. The quota parameters could be such as rate_limit (per port) and max_bandwidth (per tenant). In this way it is possible to set/manage QoS quotas from the admin side, and for instance set the maximum bandwidth allowed per tenant (cumulative). What do you think about it? Regards, Giuseppe Giuseppe Cossu CREATE-NET ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [third party] New driver - questions regarding the process
Hi guys, We (CloudFounders) have been working on developing a storage solution that we want to integrate in OpenStack (we will release the software component as open source). From what i saw in the mailing list here we missed the Juno release so i guess we'll have to wait until Kilo. The status of the integration is currently: - driver: minimal functionality required for J (95%, small issues with upload_to_image) - unit testing and automated testing: covered minimal functionality, validation on backend (failing test for upload) - certification tests: passing, except 4 tests for upload - cinder unit tests and integration tests: passing - blueprint/specs: in progress - third-party CI: almost complete - company CLA and user CLA: almost complete Now for some questions: - first, did i miss anything in the checklist above? - once functionality is complete and tests are passing, the next steps are to create the blueprint and create a blueprint topic branch and get the code reviewed? or is there something else required first? - should we include also the automated tests in the code commit? (we have tests that setup the backend and require our software component to be installed, this wouldn't make sense to run on other setups since all tests would be failing due to missing dependencies) The code is available at the moment on a bitbucket repo but we're moving to github as soon as possible. Thanks, Eduard *Eduard Biceri Mate, Senior Software Developer* www.cloudfounders.com | eduard.ma...@cloudfounders.com *CloudFounders, The Private Cloud Software Company* Disclaimer: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this email in error we request you to notify us by reply e-mail and to delete all electronic files of the message. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. E-mail transmission cannot be guaranteed to be secure or error free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the content of this message, and shall have no liability for any loss or damage suffered by the user, which arise as a result of e-mail transmission. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][neutron][cinder] Averting the Nova crisis by splitting out virt drivers
On Tue, Sep 09, 2014 at 05:14:43PM -0700, Stefano Maffulli wrote: To me, this means you don't really want a sin bin where you dump drivers and tell them not to come out until they're fit to be reviewed by the core; You want a trusted driver community which does its own reviews and means the core doesn't have to review them. I think we're going somewhere here, based on your comment and other's: we may achieve some result if we empower a new set of people to manage drivers, keeping them in the same repositories where they are now. This new set of people may not be the current core reviewers but other with different skillsets and more capable of understanding the driver's ecosystem, needs, motivations, etc. I have the impression this idea has been circling around for a while but for some reason or another (like lack of capabilities in gerrit and other reasons) we never tried to implement it. Maybe it's time to think about an implementation. We have been thinking about mentors https://wiki.openstack.org/wiki/Mentors, maybe that's a way to go? Sub-team with +1.5 scoring capabilities? I think that setting up subteams is neccessary to stop us imploding but I don't think it is enough. As long as we have one repo we're forever going to have conflict contention in deciding which features to accept, which is a big factor in problems today. I favour the strong split of the drivers into separate repositories to remove the contente between the teams as much as is practical. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Review metrics - what do we want to measure?
On Thu, Sep 04, 2014 at 01:54:20PM +, Jeremy Stanley wrote: On 2014-09-04 11:01:55 +0100 (+0100), Derek Higgins wrote: [...] How would people feel about turning [auto-abandon] back on? A lot of reviewers (myself among them) feel auto-abandon was a cold and emotionless way to provide feedback on a change. Especially on high-change-volume projects where core reviewers may at times get sucked into triaging other problems for long enough that the auto-abandoner kills lots of legitimate changes (possibly from new contributors who will get even more disgusted by this than the silence itself and walk away indefinitely with the impression that we really aren't a welcoming development community at all). I can understand this argument, and perhaps an auto-abandon with a long period like say a month for the submitter to address comments and reset the clock would be a reasonable compromise? The problem we have now, is there is a constantly expanding queue of zombie reviews, where the submitter got some negative feedback and, for whatever reason, has chosen not to address it. For example, in my Incoming reviews queue on the gerrit dashboard, I've got reviews I (or someone) -1'd over four months ago, with zero feedback from the submitter, what value is there to these reviews cluttering up the dashboard for every reviewer? To make matters worse Jenkins comes along periodically and rechecks or figures out the patch merge failed and bumps the zombie back up to the top of the queue - obviously I don't realise that it's not a human comment I need to read until I've expended effort clicking on the review and reading it :( From a reviewer perspective, it's impossible, and means the review dashboard is basically unusable without complex queries to weed out the active reviews from the zombies. Can it be done on a per project basis? It can, by running your own... but again it seems far better for core reviewers to decide if a change has potential or needs to be abandoned--that way there's an accountable human making that deliberate choice rather than the review team hiding behind an automated process so that no one is to blame for hurt feelings besides the infra operators who are enforcing this draconian measure for you. With all the threads about core-reviewer overload, I think it's unreasonable to expect us to scrub the review queue making daily judgements on whether a patch should be abandoned, and I'd argue that a human abandoning another human's patch has much more demotivating impact on contributors than a clearly documented automated process that you must address negative review comments within a set period or your review will expire. If you think about mailing list patch workflow, it's similar - you post your patch, get email review feedback, then post new reviews with fixes. If you fail to post new reviews with fixes, your review falls out the bottom of people's inboxes and you don't get your patch merged. To make the whole process a little friendlier we could increase the time frame from 1 week to 2. snarkHow about just automatically abandon any new change as soon as it's published, and if the contributor really feels it's important they'll unabandon it./snark I think that's a pretty unfair comment - all reviewers and most contributors are working really hard, all we're asking for is that contributors work with reviewers to get their patch into shape withing a reasonable time. :( As someone who's drinking from the firehose every day with a seemingly insurmountable review queue, I'd rather we worked towards processes which help us manage the workload in a sustainable way, rather than turning the review queue into the zombie-soup-of-dispair it currently is. All we need is two things: 1. A way to automatically escalate reviews which have no feedback at all from core reviewers within a reasonable time (say a week or two) 2. A way to automatically remove reviews from the queue which have core negative feedback with no resolution within a reasonable time (say a month or several weeks, so it's not percieved contributor-hostile). Note (2) still allows core reviewers to decide if a change has potential, it just becomes opt-in, e.g we have to address the issues which prevent us giving it positive feedback, either by directly working with the contributor, or delegating the work to an active contributor if the original patch author has decided not to continue work on the patch. Ultimately, it's not just about reviews - who's going to maintain all these features if the author is not committed enough to post just one patch a month while getting it through the review process? Oh wait, that'll be another job for the core team won't it? Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Zaqar graduation (round 2) [was: Comments on the concerns arose during the TC meeting]
On 09/09/2014 09:03 PM, Monty Taylor wrote: On 09/04/2014 01:30 AM, Clint Byrum wrote: Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700: Greetings, Last Tuesday the TC held the first graduation review for Zaqar. During the meeting some concerns arose. I've listed those concerns below with some comments hoping that it will help starting a discussion before the next meeting. In addition, I've added some comments about the project stability at the bottom and an etherpad link pointing to a list of use cases for Zaqar. Hi Flavio. This was an interesting read. As somebody whose attention has recently been drawn to Zaqar, I am quite interested in seeing it graduate. # Concerns - Concern on operational burden of requiring NoSQL deploy expertise to the mix of openstack operational skills For those of you not familiar with Zaqar, it currently supports 2 nosql drivers - MongoDB and Redis - and those are the only 2 drivers it supports for now. This will require operators willing to use Zaqar to maintain a new (?) NoSQL technology in their system. Before expressing our thoughts on this matter, let me say that: 1. By removing the SQLAlchemy driver, we basically removed the chance for operators to use an already deployed OpenStack-technology 2. Zaqar won't be backed by any AMQP based messaging technology for now. Here's[0] a summary of the research the team (mostly done by Victoria) did during Juno 3. We (OpenStack) used to require Redis for the zmq matchmaker 4. We (OpenStack) also use memcached for caching and as the oslo caching lib becomes available - or a wrapper on top of dogpile.cache - Redis may be used in place of memcached in more and more deployments. 5. Ceilometer's recommended storage driver is still MongoDB, although Ceilometer has now support for sqlalchemy. (Please correct me if I'm wrong). That being said, it's obvious we already, to some extent, promote some NoSQL technologies. However, for the sake of the discussion, lets assume we don't. I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't keep avoiding these technologies. NoSQL technologies have been around for years and we should be prepared - including OpenStack operators - to support these technologies. Not every tool is good for all tasks - one of the reasons we removed the sqlalchemy driver in the first place - therefore it's impossible to keep an homogeneous environment for all services. I whole heartedly agree that non traditional storage technologies that are becoming mainstream are good candidates for use cases where SQL based storage gets in the way. I wish there wasn't so much FUD (warranted or not) about MongoDB, but that is the reality we live in. With this, I'm not suggesting to ignore the risks and the extra burden this adds but, instead of attempting to avoid it completely by not evolving the stack of services we provide, we should probably work on defining a reasonable subset of NoSQL services we are OK with supporting. This will help making the burden smaller and it'll give operators the option to choose. [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/ - Concern on should we really reinvent a queue system rather than piggyback on one As mentioned in the meeting on Tuesday, Zaqar is not reinventing message brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack flavor on top. [0] I think Zaqar is more like SMTP and IMAP than AMQP. You're not really trying to connect two processes in real time. You're trying to do fully asynchronous messaging with fully randomized access to any message. Perhaps somebody should explore whether the approaches taken by large scale IMAP providers could be applied to Zaqar. Anyway, I can't imagine writing a system to intentionally use the semantics of IMAP and SMTP. I'd be very interested in seeing actual use cases for it, apologies if those have been posted before. It seems like you're EITHER describing something called XMPP that has at least one open source scalable backend called ejabberd. OR, you've actually hit the nail on the head with bringing up SMTP and IMAP but for some reason that feels strange. SMTP and IMAP already implement every feature you've described, as well as retries/failover/HA and a fully end to end secure transport (if installed properly) If you don't actually set them up to run as a public messaging interface but just as a cloud-local exchange, then you could get by with very low overhead for a massive throughput - it can very easily be run on a single machine for Sean's simplicity, and could just as easily be scaled out using well known techniques for public cloud sized deployments? So why not use existing daemons that do this? You could still use the REST API you've got, but instead of writing it to a mongo backend and trying to implement all of the things that already exist in
Re: [openstack-dev] [TripleO] Propose adding StevenK to core reviewers
+1 On 09/09/2014 08:32 PM, Gregory Haynes wrote: Hello everyone! I have been working on a meta-review of StevenK's reviews and I would like to propose him as a new member of our core team. As I'm sure many have noticed, he has been above our stats requirements for several months now. More importantly, he has been reviewing a wide breadth of topics and seems to have a strong understanding of our code base. He also seems to be doing a great job at providing valuable feedback and being attentive to responses on his reviews. As such, I think he would make a great addition to our core team. Can the other core team members please reply with your votes if you agree or disagree. Thanks! Greg ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][neutron][cinder] Averting the Nova crisis by splitting out virt drivers
Le 10/09/2014 10:44, Daniel P. Berrange a écrit : On Tue, Sep 09, 2014 at 05:14:43PM -0700, Stefano Maffulli wrote: To me, this means you don't really want a sin bin where you dump drivers and tell them not to come out until they're fit to be reviewed by the core; You want a trusted driver community which does its own reviews and means the core doesn't have to review them. I think we're going somewhere here, based on your comment and other's: we may achieve some result if we empower a new set of people to manage drivers, keeping them in the same repositories where they are now. This new set of people may not be the current core reviewers but other with different skillsets and more capable of understanding the driver's ecosystem, needs, motivations, etc. I have the impression this idea has been circling around for a while but for some reason or another (like lack of capabilities in gerrit and other reasons) we never tried to implement it. Maybe it's time to think about an implementation. We have been thinking about mentors https://wiki.openstack.org/wiki/Mentors, maybe that's a way to go? Sub-team with +1.5 scoring capabilities? I think that setting up subteams is neccessary to stop us imploding but I don't think it is enough. As long as we have one repo we're forever going to have conflict contention in deciding which features to accept, which is a big factor in problems today. I favour the strong split of the drivers into separate repositories to remove the contente between the teams as much as is practical. Regards, Daniel Well, both proposals can be done : we can create subteams and the Subteam-Approval Gerrit label right know before Kilo, and we could split the virt repos by later once the interfaces and prereqs are done. Having subteams would be even better for the virt split, as you could find whose halfcores (here, that's how I call subteam's people) are good for becoming virt cores once the repository is set up. -Sylvain ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [glance] 0.14.0 client - v2 requests broken
All, The 0.14.0 client breaks all v2 requests when pointed at a glance server which is running code which is more than a few days old (ie pretty much any production server). There's a band-aid patch here: https://review.openstack.org/#/c/120143/ and a longer term patch here: https://review.openstack.org/#/c/120300/ What would folks propose we do to avoid having a broken client for an extended period of time (days)? Would merging 120143 and releasing it as 0.14.1 be a reasonable short term solution which would allow less pressured reviewing of 120300? -Stuart ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [FUEL] Re: SSL in Fuel.
On Tue, Sep 9, 2014 at 5:53 PM, Stanislaw Bogatkin sbogat...@mirantis.com wrote: So I think that we need to start on [3]. As this is required for OSt public endpoint SSL and also for Fuel SSL it can be quicker to make a first stage where a self-signed certificate is managed from nailgun and a second stage with the docker CA... So, yes, I think that it is good idea, cause it will be good base for [2] and [3] [1] https://blueprints.launchpad.net/fuel/+spec/ca-deployment [2] https://blueprints.launchpad.net/fuel/+spec/fuel-ssl-endpoints [3] https://blueprints.launchpad.net/fuel/+spec/ssl-endpoints +1 I totally agree that we should start with [1] first and later focus on [2] and [3]. I'll also update [2] to adjust it to the plan you proposed. Best, Sebastian ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting
On Tue, 9 Sep 2014, Samuel Merritt wrote: On 9/9/14, 4:47 PM, Devananda van der Veen wrote: The questions now before us are: - should OpenStack include, in the integrated release, a messaging-as-a-service component? I certainly think so. I've worked on a few reasonable-scale web applications, and they all followed the same pattern: HTTP app servers serving requests quickly, background workers for long-running tasks, and some sort of durable message-broker/queue-server thing for conveying work from the first to the second. A quick straw poll of my nearby coworkers shows that every non-trivial web application that they've worked on in the last decade follows the same pattern. While not *every* application needs such a thing, web apps are quite common these days, and Zaqar satisfies one of their big requirements. Not only that, it does so in a way that requires much less babysitting than run-your-own-broker does. I don't think there's any question about the value of a durable message-broke/queue-server thing in the general case. The question is whether OpenStack is in the business of satisfying that case. Which leads inevitably to the existential questions of What is OpenStack in the business of? and How do we stay sane while being in that business? Every long thread over the last couple of months has trended towards those questions. It's getting pretty tiresome. We'd all be a lot more focused if we knew the answer. -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Propose adding StevenK to core reviewers
+1 -Original Message- From: Ladislav Smola [mailto:lsm...@redhat.com] Sent: 10 September 2014 10:26 To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [TripleO] Propose adding StevenK to core reviewers +1 On 09/09/2014 08:32 PM, Gregory Haynes wrote: Hello everyone! I have been working on a meta-review of StevenK's reviews and I would like to propose him as a new member of our core team. As I'm sure many have noticed, he has been above our stats requirements for several months now. More importantly, he has been reviewing a wide breadth of topics and seems to have a strong understanding of our code base. He also seems to be doing a great job at providing valuable feedback and being attentive to responses on his reviews. As such, I think he would make a great addition to our core team. Can the other core team members please reply with your votes if you agree or disagree. Thanks! Greg ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Thanks, Jon-Paul Sullivan ☺ Cloud Services - @hpcloud Postal Address: Hewlett-Packard Galway Limited, Ballybrit Business Park, Galway. Registered Office: Hewlett-Packard Galway Limited, 63-74 Sir John Rogerson's Quay, Dublin 2. Registered Number: 361933 The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender. To any recipient of this message within HP, unless otherwise stated, you should consider this message and attachments as HP CONFIDENTIAL. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Ceilometer] Adding Nejc Saje to ceilometer-core
Hi, Nejc has been doing a great work and has been very helpful during the Juno cycle and his help is very valuable. I'd like to propose that we add Nejc Saje to the ceilometer-core group. Please, dear ceilometer-core members, reply with your votes! -- Julien Danjou // Free Software hacker // http://julien.danjou.info signature.asc Description: PGP signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] 0.14.0 client - v2 requests broken
On 09/10/2014 11:52 AM, stuart.mcla...@hp.com wrote: All, The 0.14.0 client breaks all v2 requests when pointed at a glance server which is running code which is more than a few days old (ie pretty much any production server). There's a band-aid patch here: https://review.openstack.org/#/c/120143/ and a longer term patch here: https://review.openstack.org/#/c/120300/ What would folks propose we do to avoid having a broken client for an extended period of time (days)? Would merging 120143 and releasing it as 0.14.1 be a reasonable short term solution which would allow less pressured reviewing of 120300? Why do you think landing 120300 will take long? If both patches are already on gerrit, I'd rather review the real fix. Is there a reason you think landing 120143 first is better? Flavio -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Propose adding StevenK to core reviewers
Hi +1 Cheers, -- Chris Jones On 9 Sep 2014, at 19:32, Gregory Haynes g...@greghaynes.net wrote: Hello everyone! I have been working on a meta-review of StevenK's reviews and I would like to propose him as a new member of our core team. As I'm sure many have noticed, he has been above our stats requirements for several months now. More importantly, he has been reviewing a wide breadth of topics and seems to have a strong understanding of our code base. He also seems to be doing a great job at providing valuable feedback and being attentive to responses on his reviews. As such, I think he would make a great addition to our core team. Can the other core team members please reply with your votes if you agree or disagree. Thanks! Greg ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [FUEL] Re: SSL in Fuel.
Hello, Thanks for the detailed email, Stanislaw. Your suggestion of deploying a CA container is really interesting. Especially for OSTF and other testing since the tools only need to know about the root CA. Lets back up a bit and list the different options for Fuel users: 0/ The user is happy with plain HTTP. = Already supported :) 1/ The user wants HTTPS but doesn't want the burden associated with certificate management. = Fuel creates and manages the SSL certificates, be them self-signed or signed by some internal CA. = Using an internal CA instead of multiple self-signed certificates is cleaner as you explained. 2/ The user wants HTTPS and wants to use certificates which are generated by an external source (either some internal corporate PKI or some public certificate authority) = Fuel supports certificate + key uploads = It should be possible to tell Fuel which entity (Fuel, OSt environment) uses which certificate 3/ The user wants HTTPS and agrees to let Fuel generating certificates on behalf of some corporate PKI. = Fuel supports CA + key uploads I think that option 1 is the way to go for a first approach. Option 2 is definitely something that end-users would need at some point. I'm less convinced by option 3: if I were a PKI admin, I'll be reluctant to let Fuel generate certificates on its own. Also my gut feeling tells me that implementing 1 2 is already quite a lot of work. I've also added some questions/comments inline. BR, Simon On Tue, Sep 9, 2014 at 5:53 PM, Stanislaw Bogatkin sbogat...@mirantis.com wrote: I think that if we have 3 blueprints that realises some SSL stuff around themselves then we can discuss it here. My vision about SSL in Fuel split into 3 parts: A) We need to implement [1] blueprint, cause it is only one way to generate certificates. How i see that: 1.0 We sync puppet-openssl from upstream, adapt it for Fuel tasks. 1.1 We create docker container (we already have many, so containerized CA should work well) with OpenSSL and puppet manifests in it. 1.2 When container will start first time, it will create CA that will store on master node. Our workitems here is: - Create docker container - Sync upstream puppet-openssl and adapt it for Fuel - Write code to create CA First of all I think this blueprint should be submitted to fuel-specs ;) How do you see the exchanges between the CA container and the various services? For instance, how would nailgun asks for a signed certificate and get back the result? B) We need to implement [2] blueprint. How I see that: 1.3 When CA container start first time and creates CA, then it will check for keypair for master node (Fuel UI). If that keypair will not found, then CA create it, change nginx conffile properly and restart nginx on master node. I find awkward to have the CA container restarting nginx... Our workitems here is: - Write code to check if we already have generated certificate and generate new if we have not. C) Then we need to implement [3] blueprint For next step we have 2 options: First: 1.3 When we create new cluster then we know all information to create new keypair(s). When user press Deploy changes button, then we will create new keypair(s). Q: Main question here is - how many keypairs we will create? One for every service or one for all? As a first implementation, I assume that one certificate for all OSt services is good enough. 1.4 We will distribute key(s) with mcollective agent (similar to how we sync puppet manifests from master node to other nodes). After that private key(s) will deleted from master node. How will it work if the user modifies the configuration of the environment afterwards? Say he/she adds one controller node, how will it be able to copy the private key to the new node? 1.5 On nodes puppet will do all work. We need to write some code for that Pros of that method: + It's relative simple, we can create clean and lucid code that will be easy for support Cons of that method: - We need to send every private key over network. We can reduce this danger cause we will already have passwordless sync over network between master node and other nodes, cause we will generate ssh keys for nodes before we will distribute any data at deployment stage. Second: 1.3 When we create new cluster, we do all work same way as we do it now, but after provisioning we will create keypair on first node, make csr for every service (or for one, if we will create one certificate for all services) and send that csr to master node, where it will signed and certificate will send back. 1.4 Puppet will do all work on nodes. We, obviously, need to write some code for it. But we need to sync our keys over controllers all the same (and now we don't have reliable mechanism to do this) Pros of that method: + I don't see any Cons of that method: - Code will be not so obvious - To
Re: [openstack-dev] [zaqar] Juno Performance Testing (Round 2)
On 09/09/2014 09:19 PM, Kurt Griffiths wrote: Hi folks, In this second round of performance testing, I benchmarked the new Redis driver. I used the same setup and tests as in Round 1 to make it easier to compare the two drivers. I did not test Redis in master-slave mode, but that likely would not make a significant difference in the results since Redis replication is asynchronous[1]. As always, the usual benchmarking disclaimers apply (i.e., take these numbers with a grain of salt; they are only intended to provide a ballpark reference; you should perform your own tests, simulating your specific scenarios and using your own hardware; etc.). ## Setup ## Rather than VMs, I provisioned some Rackspace OnMetal[3] servers to mitigate noisy neighbor when running the performance tests: * 1x Load Generator * Hardware * 1x Intel Xeon E5-2680 v2 2.8Ghz * 32 GB RAM * 10Gbps NIC * 32GB SATADOM * Software * Debian Wheezy * Python 2.7.3 * zaqar-bench * 1x Web Head * Hardware * 1x Intel Xeon E5-2680 v2 2.8Ghz * 32 GB RAM * 10Gbps NIC * 32GB SATADOM * Software * Debian Wheezy * Python 2.7.3 * zaqar server * storage=mongodb * partitions=4 * MongoDB URI configured with w=majority * uWSGI + gevent * config: http://paste.openstack.org/show/100592/ * app.py: http://paste.openstack.org/show/100593/ * 3x MongoDB Nodes * Hardware * 2x Intel Xeon E5-2680 v2 2.8Ghz * 128 GB RAM * 10Gbps NIC * 2x LSI Nytro WarpDrive BLP4-1600[2] * Software * Debian Wheezy * mongod 2.6.4 * Default config, except setting replSet and enabling periodic logging of CPU and I/O * Journaling enabled * Profiling on message DBs enabled for requests over 10ms * 1x Redis Node * Hardware * 2x Intel Xeon E5-2680 v2 2.8Ghz * 128 GB RAM * 10Gbps NIC * 2x LSI Nytro WarpDrive BLP4-1600[2] * Software * Debian Wheezy * Redis 2.4.14 * Default config (snapshotting and AOF enabled) * One process As in Round 1, Keystone auth is disabled and requests go over HTTP, not HTTPS. The latency introduced by enabling these is outside the control of Zaqar, but should be quite minimal (speaking anecdotally, I would expect an additional 1-3ms for cached tokens and assuming an optimized TLS termination setup). For generating the load, I again used the zaqar-bench tool. I would like to see the team complete a large-scale Tsung test as well (including a full HA deployment with Keystone and HTTPS enabled), but decided not to wait for that before publishing the results for the Redis driver using zaqar-bench. CPU usage on the Redis node peaked at around 75% for the one process. To better utilize the hardware, a production deployment would need to run multiple Redis processes and use Zaqar's backend pooling feature to distribute queues across the various instances. Several different messaging patterns were tested, taking inspiration from: https://wiki.openstack.org/wiki/Use_Cases_(Zaqar) Each test was executed three times and the best time recorded. A ~1K sample message (1398 bytes) was used for all tests. ## Results ## ### Event Broadcasting (Read-Heavy) ### OK, so let's say you have a somewhat low-volume source, but tons of event observers. In this case, the observers easily outpace the producer, making this a read-heavy workload. Options * 1 producer process with 5 gevent workers * 1 message posted per request * 2 observer processes with 25 gevent workers each * 5 messages listed per request by the observers * Load distributed across 4[6] queues * 10-second duration Results * Redis * Producer: 1.7 ms/req, 585 req/sec * Observer: 1.5 ms/req, 1254 req/sec * Mongo * Producer: 2.2 ms/req, 454 req/sec * Observer: 1.5 ms/req, 1224 req/sec ### Event Broadcasting (Balanced) ### This test uses the same number of producers and consumers, but note that the observers are still listing (up to) 5 messages at a time[4], so they still outpace the producers, but not as quickly as before. Options * 2 producer processes with 25 gevent workers each * 1 message posted per request * 2 observer processes with 25 gevent workers each * 5 messages listed per request by the observers * Load distributed across 4 queues * 10-second duration Results * Redis * Producer: 1.4 ms/req, 1374 req/sec * Observer: 1.6 ms/req, 1178 req/sec * Mongo * Producer: 2.2 ms/req, 883 req/sec * Observer: 2.8 ms/req, 348 req/sec ### Point-to-Point Messaging ###
Re: [openstack-dev] [TripleO] Propose adding StevenK to core reviewers
On 09/09/14 20:32, Gregory Haynes wrote: Hello everyone! I have been working on a meta-review of StevenK's reviews and I would like to propose him as a new member of our core team. As I'm sure many have noticed, he has been above our stats requirements for several months now. More importantly, he has been reviewing a wide breadth of topics and seems to have a strong understanding of our code base. He also seems to be doing a great job at providing valuable feedback and being attentive to responses on his reviews. As such, I think he would make a great addition to our core team. Can the other core team members please reply with your votes if you agree or disagree. +1 Thanks! Greg ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [kesytone][multidomain] - Time to leave LDAP backend?
On Tue, Sep 9, 2014 at 8:25 AM, Nathan Kinder nkin...@redhat.com wrote: On 09/01/2014 01:43 AM, Marcos Fermin Lobo wrote: Hi all, I found two functionalities for keystone that could be against each other. Multi-domain feature (This functionality is new in Juno.) --- Link: http://docs.openstack.org/developer/keystone/configuration.html#domain-specific-drivers Keystone supports the option to specify identity driver configurations on a domain by domain basis, allowing, for example, a specific domain to have its own LDAP or SQL server. So, we can use different backends for different domains. But, as Henry Nash said “it has not been validated with multiple SQL drivers” https://bugs.launchpad.net/keystone/+bug/1362181/comments/2 Hierarchical Multitenancy Link: https://blueprints.launchpad.net/keystone/+spec/hierarchical-multitenancy This is nested projects feature but, only for SQL, not LDAP. So, if you are using LDAP and you want “nested projects” feature, you should to migrate from LDAP to SQL but, I you want to get multi-domain feature too you can’t use 2 SQL backends (you need at least one LDAP backend) because is not validated for multiple SQL drivers… Maybe I’m losing something, please, correct me if I’m wrong. Here my questions: - If I want Multi-domain and Hierarchical Multitenancy features, which are my options? What should I do (migrate or not migrate to SQL)? - Is LDAP going to deprecated soon? I think you need to keep in mind that there are two separate backends that support LDAP: identity and assignment. From everyone I have talked to on the Keystone team, SQL is preferred for the assignment backend. Storing assignment information in LDAP seems to be a non-standard use case. For the identity backend, LDAP is preferred. Many people have users and groups already in an LDAP server, and Keystone should be able to take advantage of those existing users and credentials for centralized authentication. In addition, every LDAP server I know have has better security features than the SQL identity backend offers, such as password policies and account lockout. The multiple domain support for multiple LDAP servers was really designed to allow for separate groups of users from separate identity LDAP servers to be usable in a single Keystone instance. Given that the Keystone team considers SQL as the preferred assignment backend, the hierarchical project blueprint was targeted against it. The idea is that you would use LDAP server(s) for your users and have hierarchical projects in SQL. My personal feeling is that the LDAP assignment backend should ultimately be deprecated. I don't think the LDAP assignment backend really offers any benefit of SQL, and you have to define some non-standard LDAP schema to represent projects, roles, etc., or you end up trying to shoehorn the data into standard LDAP schema that was really meant for something else. It would be interesting to create a poll like Morgan did for the Keystone token format to see how widely the LDAP assignments backend is. Even more interesting would be to know the reasons why people are using it over SQL. Please don't consider LDAP assignment backend as and outcast. It is used and we have use cases where it's the only way to go. Some enterprises with strict security policies require all security-related tasks to be done through AD, and project/roles assignment is one of them. LDAP assignment backend is a right fit here. Storing such info in AD provides additional benefit of providing not only single management point, but also an enterprise-ready cross-datacenter replication. (Galera or other MySQL replications arguably don't quite work for this) From what I see, the only obstruction here is need for a custom LDAP schema for AD (which doesn't fly with strict enterprise constraints). That can be mitigated by using AD-native objectClass'es for projects and groups instead of 'groupOfNames' and 'organizationalRole': 'organizationalUnit' and 'group'. These object can be managed by commonly used AD tools (not LDAP editor), but require some changes in Keystone to work. We've hacked together some patches to Keystone that should make it work and will propose them in Kilo cycle. Another missing feature is domains/hierarchical projects. It's not impossible to implement this in LDAP backend, but we need someone to step up here. With OUs it should be rather obvious how to store these in LDAP, but we'll need some algorithmic support as well. We shouldn't give up on LDAP backend. It's used by a lot of private clouds and some public ones. The problem is that its users usually aren't ready to make necessary changes to make it work and so have to bend their rules to make existing backend work. Some of them already are giving back: connection
Re: [openstack-dev] [Ceilometer] Adding Nejc Saje to ceilometer-core
Hi, +1 from me, and thanks for the nice work so far. Best Regards, Ildikó -Original Message- From: Julien Danjou [mailto:jul...@danjou.info] Sent: Wednesday, September 10, 2014 12:35 PM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [Ceilometer] Adding Nejc Saje to ceilometer-core Hi, Nejc has been doing a great work and has been very helpful during the Juno cycle and his help is very valuable. I'd like to propose that we add Nejc Saje to the ceilometer-core group. Please, dear ceilometer-core members, reply with your votes! -- Julien Danjou // Free Software hacker // http://julien.danjou.info ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [FUEL] Re: SSL in Fuel.
On 10 Sep 2014, at 12:54, Simon Pasquier spasqu...@mirantis.com wrote: Hello, Lets back up a bit and list the different options for Fuel users: 0/ The user is happy with plain HTTP. = Already supported :) 1/ The user wants HTTPS but doesn't want the burden associated with certificate management. = Fuel creates and manages the SSL certificates, be them self-signed or signed by some internal CA. = Using an internal CA instead of multiple self-signed certificates is cleaner as you explained. 2/ The user wants HTTPS and wants to use certificates which are generated by an external source (either some internal corporate PKI or some public certificate authority) = Fuel supports certificate + key uploads = It should be possible to tell Fuel which entity (Fuel, OSt environment) uses which certificate 3/ The user wants HTTPS and agrees to let Fuel generating certificates on behalf of some corporate PKI. = Fuel supports CA + key uploads I think that option 1 is the way to go for a first approach. Option 2 is definitely something that end-users would need at some point. I'm less convinced by option 3: if I were a PKI admin, I'll be reluctant to let Fuel generate certificates on its own. Also my gut feeling tells me that implementing 1 2 is already quite a lot of work. I've also added some questions/comments inline. Regarding After careful consideration, I think that for 6.0 we will only be able to implement [2] with limited functionality. In terms of certificate management, we could offer uploading customer generated cert (and maybe provide shot doc on how to spawn CA + sign certs) or if user does not want to do it, generate simple self signed cert and install it on Fuel http server and let user download it. After 6.0 we can concentrate on proper implementation of CA management, and then allow Fuel master node part to use it. [1] https://blueprints.launchpad.net/fuel/+spec/ca-deployment [2] https://blueprints.launchpad.net/fuel/+spec/fuel-ssl-endpoints [3] https://blueprints.launchpad.net/fuel/+spec/ssl-endpoints -- Tomasz 'Zen' Napierala Sr. OpenStack Engineer tnapier...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] 0.14.0 client - v2 requests broken
On 09/10/2014 11:52 AM, stuart.mclaren at hp.com wrote: All, The 0.14.0 client breaks all v2 requests when pointed at a glance server which is running code which is more than a few days old (ie pretty much any production server). There's a band-aid patch here: https://review.openstack.org/#/c/120143/ and a longer term patch here: https://review.openstack.org/#/c/120300/ What would folks propose we do to avoid having a broken client for an extended period of time (days)? Would merging 120143 and releasing it as 0.14.1 be a reasonable short term solution which would allow less pressured reviewing of 120300? Why do you think landing 120300 will take long? If both patches are already on gerrit, I'd rather review the real fix. Is there a reason you think landing 120143 first is better? 120300 is a biggish change (several hundred lines) and I was concerned it might take a while to go through the review cycle. However, the consensus seems to be to go directly with 120300 -- that it shouldn't take too long to get through. (And its on-demand schema behaviour is a nice bonus too!) Flavio -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] 0.14.0 client - v2 requests broken
On 09/10/2014 01:40 PM, stuart.mcla...@hp.com wrote: On 09/10/2014 11:52 AM, stuart.mclaren at hp.com wrote: All, The 0.14.0 client breaks all v2 requests when pointed at a glance server which is running code which is more than a few days old (ie pretty much any production server). There's a band-aid patch here: https://review.openstack.org/#/c/120143/ and a longer term patch here: https://review.openstack.org/#/c/120300/ What would folks propose we do to avoid having a broken client for an extended period of time (days)? Would merging 120143 and releasing it as 0.14.1 be a reasonable short term solution which would allow less pressured reviewing of 120300? Why do you think landing 120300 will take long? If both patches are already on gerrit, I'd rather review the real fix. Is there a reason you think landing 120143 first is better? 120300 is a biggish change (several hundred lines) and I was concerned it might take a while to go through the review cycle. However, the consensus seems to be to go directly with 120300 -- that it shouldn't take too long to get through. (And its on-demand schema behaviour is a nice bonus too!) +1 lets go with that! Flavio -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [FUEL] Re: SSL in Fuel.
Hi, Tomasz is right. Let's try not to complicate the things. For 6.0 we'll allow just upload key, csr, certificate (like 3 edit boxes), or these edit boxes will be greyed if customer allows to generate self-signed certificates. -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Wed, Sep 10, 2014 at 1:40 PM, Tomasz Napierala tnapier...@mirantis.com wrote: On 10 Sep 2014, at 12:54, Simon Pasquier spasqu...@mirantis.com wrote: Hello, Lets back up a bit and list the different options for Fuel users: 0/ The user is happy with plain HTTP. = Already supported :) 1/ The user wants HTTPS but doesn't want the burden associated with certificate management. = Fuel creates and manages the SSL certificates, be them self-signed or signed by some internal CA. = Using an internal CA instead of multiple self-signed certificates is cleaner as you explained. 2/ The user wants HTTPS and wants to use certificates which are generated by an external source (either some internal corporate PKI or some public certificate authority) = Fuel supports certificate + key uploads = It should be possible to tell Fuel which entity (Fuel, OSt environment) uses which certificate 3/ The user wants HTTPS and agrees to let Fuel generating certificates on behalf of some corporate PKI. = Fuel supports CA + key uploads I think that option 1 is the way to go for a first approach. Option 2 is definitely something that end-users would need at some point. I'm less convinced by option 3: if I were a PKI admin, I'll be reluctant to let Fuel generate certificates on its own. Also my gut feeling tells me that implementing 1 2 is already quite a lot of work. I've also added some questions/comments inline. Regarding After careful consideration, I think that for 6.0 we will only be able to implement [2] with limited functionality. In terms of certificate management, we could offer uploading customer generated cert (and maybe provide shot doc on how to spawn CA + sign certs) or if user does not want to do it, generate simple self signed cert and install it on Fuel http server and let user download it. After 6.0 we can concentrate on proper implementation of CA management, and then allow Fuel master node part to use it. [1] https://blueprints.launchpad.net/fuel/+spec/ca-deployment [2] https://blueprints.launchpad.net/fuel/+spec/fuel-ssl-endpoints [3] https://blueprints.launchpad.net/fuel/+spec/ssl-endpoints -- Tomasz 'Zen' Napierala Sr. OpenStack Engineer tnapier...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [MagnetoDB] Blueprint approvement process changes
Hello mdb contributors, As far as we are growing and not the one solid team as it was before, it is important to make step forward to openness of technical decisions and agreements. OpenStack community has a great experience in it, but I'm not sure if we need such a formal process with specs approval via gerrit workflow right now. I suggest to use wiki for drafting and IRC with ML for discussion and approvement. For this purpose the wiki section has been created [1]. The old drafts have been moved to new location as well. Feel free to share your thoughts about it. [1] https://wiki.openstack.org/wiki/MagnetoDB/specs/ Thanks -- Ilya Sviridov isviridov @ FreeNode ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][cells] Flavors and cells
Hi, we are working on bug-1211011 (Flavors created in parent cell are not propagated to the child cell).. we need a few pointers from the community (and people working with cells). The problem is that if the instance_type tables are not in sync in all cells we can spawn a different flavor than the one selected by the user. Example: flavor_A is selected and in the child cell flavor_B is spawned. This is because what is checked at spawn time is the instance_type_id in the child cell database, and there is not guarantee that this corresponds to the same flavor in the parent cell. A possible solution to this problem can be propagating the flavors to all child cells when it is created/deleted/changed in the parent cell. But since instance_type_id is an autoincrement field, it will be difficult to sync it. Also there may be a problem when applying this to an existing cell setup. We believe that the flavors should only live in the parent cell (API cell). In this case every time that this information is needed the child cell needs to query the information from the parents. The problem is that at the moment in the child cells the compute api not aware about cells. Should this be reconsidered? Also, having this mechanism to query the parent cell will allow to more easily add support for security groups and aggregates. Thoughts? Regards, Dheeraj ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Kilo Cycle Goals Exercise
Joe Gordon wrote: To that end, I would like to propose an exercise as discussed in the TC meeting yesterday [1]: Have anyone interested (especially TC members) come up with a list of what they think the project wide Kilo cycle goals should be and post them on this thread by end of day Wednesday, September 10th. After which time we can begin discussing the results. The goal of this exercise is to help us see if our individual world views align with the greater community, and to get the ball rolling on a larger discussion of where as a project we should be focusing more time. Thanks again to Joe for starting this important discussion. My personal list of Kilo goals goes as follows: #1: Fix our growth pains We grew a lot, and our recipes were designed for a smaller group where trust happens more naturally. With our community growing to a Dunbar order of magnitude above, some of those recipes don't work so great, so we need to revisit them... That includes the binary integrated release (introduce layers?), cross-project functional testing (push it at project level?), the programs concept, encouraging PTL delegation (the czar/liaisons proposal ?), scaling core reviewing in larger projects (Nova driver split ?), etc. We already started the conversation on those important topics, but Kilo is the timeframe for us to implement those changes, because I don't see our community wait for more than one cycle to see the light at the end of the tunnel. #2: Fix the end user experience Monty expressed it better than I can. We need consistent and better-designed APIs, client SDKs that provide useful primitives and actually abstract differences in implementations, etc. #3: Fix what we have: bugfixes, consistency, scaling up, scaling down, upgrading Rather than adding more features for the mid-size private cloud, let's make sure that what we have works well, provides a consistent experience across projects, scales up beautifully, can be easily used at smaller scale as well (simplicity), and allows seamless upgrades. This is another way of looking at paying our technical debt that others have mentioned as goals -- generally polishing what we have rather than building out new things. Regards, -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] 5.1 branch created
Hi, everyone, as a follow-up for discussion [1] we created stable/5.1 branch for all Fuel projects. Since now on all patches targeted for upcoming 5.1 release should go to master branch first and then should be cherry-picked to stable/5.1. CI Infrastructure branching is in progress. With any branching-related issues please contact devops team at #fuel-devops IRC channel. [1] https://www.mail-archive.com/openstack-dev%40lists.openstack.org/msg34607.html -- Aleksandra Fedorova bookwar ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Zaqar graduation (round 2) [was: Comments on the concerns arose during the TC meeting]
Flavio Percoco wrote: [...] Based on the feedback from the meeting[3], the current main concern is: - Do we need a messaging service with a feature-set akin to SQS+SNS? [...] I think we do need, as Samuel puts it, some sort of durable message-broker/queue-server thing. It's a basic application building block. Some claim it's THE basic application building block, more useful than database provisioning. It's definitely a layer above pure IaaS, so if we end up splitting OpenStack into layers this clearly won't be in the inner one. But I think IaaS+ basic application building blocks belong in OpenStack one way or another. That's the reason I supported Designate (everyone needs DNS) and Trove (everyone needs DBs). With that said, I think yesterday there was a concern that Zaqar might not fill the some sort of durable message-broker/queue-server thing role well. The argument goes something like: if it was a queue-server then it should actually be built on top of Rabbit; if it was a message-broker it should be built on top of postfix/dovecot; the current architecture is only justified because it's something in between, so it's broken. I guess I don't mind that much zaqar being something in between: unless I misunderstood, exposing extra primitives doesn't prevent the queue-server use case from being filled. Even considering the message-broker case, I'm also not convinced building it on top of postfix/dovecot would be a net win compared to building it on top of Redis, to be honest. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [FUEL] Re: SSL in Fuel.
On Wed, Sep 10, 2014 at 2:40 PM, Tomasz Napierala tnapier...@mirantis.com wrote: Regarding After careful consideration, I think that for 6.0 we will only be able to implement [2] with limited functionality. In terms of certificate management, we could offer uploading customer generated cert (and maybe provide shot doc on how to spawn CA + sign certs) After 6.0 we can concentrate on proper implementation of CA management, and then allow Fuel master node part to use it. [1] https://blueprints.launchpad.net/fuel/+spec/ca-deployment [2] https://blueprints.launchpad.net/fuel/+spec/fuel-ssl-endpoints [3] https://blueprints.launchpad.net/fuel/+spec/ssl-endpoints So if I understand correctly the idea is that we implement basic functionality for [2] and [3] especially for the management of the certificate. I guess that even if [2] and [3] are managing a self-signed certificate by their own the code that does this could be share. And then concentrate on the CA management. Thus I removed the dependency to [1] and I consider that in [3] we should implement a limited functionality as described above for [2]. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][QoS] Applying QoS as Quota
Using the quota system would be a nice option to have. Can you clarify what you mean by cumulative bandwidth for the tenant? It would be possible to rate limit at the tenant router, but having a cumulative limit enforced inside of a tenant would be difficult. On Wed, Sep 10, 2014 at 1:03 AM, Giuseppe Cossu giuseppe.co...@create-net.orgmailto:giuseppe.co...@create-net.org wrote: Hello everyone, Looking at the QoS blueprint (proposed for incubation), I suggest to consider adding some parameters to Neutron Quotas. Let’s suppose using rate-limit for managing QoS. The quota parameters could be such as rate_limit (per port) and max_bandwidth (per tenant). In this way it is possible to set/manage QoS quotas from the admin side, and for instance set the maximum bandwidth allowed per tenant (cumulative). What do you think about it? I’m cautious about this. We’d either need to allow a “Number of DSCP settings” and set them outside the quota or leave it out altogether. Let’s not forget that there’s more than just rate limiting in QoS, and we need to make sure all the options are included. Otherwise, there’s going to be a lot of user and operator confusion as to what is and isn’t considered part of the quota. -Anthony Regards, Giuseppe Giuseppe Cossu CREATE-NET ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Zaqar graduation (round 2) [was: Comments on the concerns arose during the TC meeting]
On 09/10/2014 09:58 AM, Flavio Percoco wrote: To clarify the doubts of what Zaqar is or it's not, let me quote what's written in the project's overview section[0]: Zaqar is a multi-tenant cloud messaging service for web developers. How are different tenants isolated from each other? Can different tenants access the same queue? If so, what does Zaqar do to prevent one tenant from negatively affecting the other? If not, how is communication with other tenants achieved. Most messaging systems allow authorisation to be used to restrict what a particular user can access and quotas to restrict their resource consumption. What does Zaqar do differently? It combines the ideas pioneered by Amazon's SQS product with additional semantics to support event broadcasting. The service features a fully RESTful API, which developers can use to send messages between various components of their SaaS and mobile applications, by using a variety of communication patterns. Underlying this API is an efficient messaging engine designed with scalability and security in mind. Other OpenStack components can integrate with Zaqar to surface events to end users and to communicate with guest agents that run in the over-cloud layer. I may be misunderstanding the last sentence, but I think *direct* integration of other OpenStack services with Zaqar would be a bad idea. Wouldn't this be better done through olso.messaging's notifications in some way? and/or through some standard protocol (and there's more than one to choose from)? Communicating through a specific, fixed messaging system, with its own unique protocol is actually a step backwards in my opinion, especially for things that you want to keep as loosely coupled as possible. This is exactly why various standard protocols emerged. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] Node renaming in Fuel
Hello Fuelers,
I've figured out that there is a blueprint regarding node naming (
https://blueprints.launchpad.net/fuel/+spec/node-naming). It looks like
this this feature really has not been started. That being said all
necessary preparations in Nailgun code have been already done. The only
thing that is needed to be done is to enable this feature (in fact 4 lines
of code).
--- a/nailgun/nailgun/objects/node.py
+++ b/nailgun/nailgun/objects/node.py
@@ -654,7 +654,10 @@ class Node(NailgunObject):
@classmethod
def make_slave_name(cls, instance):
-return unode-{node_id}.format(node_id=instance.id)
+if instance.name.startswith(Untitled):
+return unode-{node_id}.format(node_id=instance.id)
+else:
+return u{node_id}.format(node_id=instance.name.lower())
@classmethod
def make_slave_fqdn(cls, instance):
Should I consider this as a bug or this blueprint will be implemented in
the nearest future?
--
Kind regards
Dmitry Ukov
IT Engineer
Mirantis, Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Propose adding StevenK to core reviewers
+1 On 09/09/2014 02:32 PM, Gregory Haynes wrote: Hello everyone! I have been working on a meta-review of StevenK's reviews and I would like to propose him as a new member of our core team. As I'm sure many have noticed, he has been above our stats requirements for several months now. More importantly, he has been reviewing a wide breadth of topics and seems to have a strong understanding of our code base. He also seems to be doing a great job at providing valuable feedback and being attentive to responses on his reviews. As such, I think he would make a great addition to our core team. Can the other core team members please reply with your votes if you agree or disagree. Thanks! Greg ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Puppet elements support
On Mon, Sep 8, 2014 at 7:11 PM, Emilien Macchi
emilien.mac...@enovance.com wrote:
Hi TripleO community,
I would be really interested by helping to bring Puppet elements support
in TripleO.
So far I've seen this work:
https://github.com/agroup/tripleo-puppet-elements/tree/puppet_dev_heat
which is a very good bootstrap but really outdated.
After some discussion with Greg Haynes on IRC, we came up with the idea
to create a repo (that would be move in Stackforge or OpenStack git) and
push the bits from what has been done by HP folks with updates
improvements.
I started a basic repo
https://github.com/enovance/tripleo-puppet-elements that could be moved
right now on Stackforge to let the community start the work.
My proposal is:
* move this repo (or create a new one directly on
github/{stackforge,openstack?})
* push some bits from agroup original work.
* continue the contributions, updates improvements.
Any thoughts?
Sounds good to me. I'm +1 on seeing some integration between TripleO
and the existing openstack-puppet-modules. The tripleo-puppet-elements
repo under agroup was kind of a POC that mostly Red Hat folks were
working on. I can try and answer any questions about that if you have
any.
--
-- James Slagle
--
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Zaqar graduation (round 2) [was: Comments on the concerns arose during the TC meeting]
On 09/10/2014 01:51 PM, Thierry Carrez wrote: I think we do need, as Samuel puts it, some sort of durable message-broker/queue-server thing. It's a basic application building block. Some claim it's THE basic application building block, more useful than database provisioning. It's definitely a layer above pure IaaS, so if we end up splitting OpenStack into layers this clearly won't be in the inner one. But I think IaaS+ basic application building blocks belong in OpenStack one way or another. That's the reason I supported Designate (everyone needs DNS) and Trove (everyone needs DBs). With that said, I think yesterday there was a concern that Zaqar might not fill the some sort of durable message-broker/queue-server thing role well. The argument goes something like: if it was a queue-server then it should actually be built on top of Rabbit; if it was a message-broker it should be built on top of postfix/dovecot; the current architecture is only justified because it's something in between, so it's broken. What is the distinction between a message broker and a queue server? To me those terms both imply something broadly similar (message broker perhaps being a little bit more generic). I could see Zaqar perhaps as somewhere between messaging and data-storage. There are of course quite a lot of durable message-broker/queue-server things around already. I understood Zaqar to have been created to address perceived limitations in existing solutions (e.g. requiring less 'babysitting', being 'designed for the cloud' etc). All solutions certainly have their limitations. Zaqar has limitations with respect to existing solutions also. So while I agree that there is great value in a basic building block for 'messaging as a service' I think the ideal solution would allow different variations, tailored to different patterns of use with a common API for provisioning, managing and monitoring coupled with support for standard protocols. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Propose adding StevenK to core reviewers
On Tue, Sep 9, 2014 at 2:32 PM, Gregory Haynes g...@greghaynes.net wrote: Hello everyone! I have been working on a meta-review of StevenK's reviews and I would like to propose him as a new member of our core team. +1. Steven has also been doing great work on os-cloud-config, I think he'd make a good addition to the core team. -- -- James Slagle -- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] cirros 0.3.3 released
Hello all, cirros 0.3.3 has been released. It is available for download at http://download.cirros-cloud.net/0.3.3 . The changes since 0.3.2 are: - Wrap udhcpc to provide for easier passing of options including those required to get MTU set properly. (LP: #1301958) - Busybox: enable nc flags '-s ADDR', '-n', '-u', '-o FILE', -z. - kernel: update to latest released Ubuntu 12.04 kernel (3.2.0.68.81). Thanks to Harm Weites for doing the MTU work. If you find issues or have feature requests, please feel free to file those at http://bugs.launchpad.net/cirros Scott ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Propose adding StevenK to core reviewers
On 09/09/2014 08:32 PM, Gregory Haynes wrote: Hello everyone! I have been working on a meta-review of StevenK's reviews and I would like to propose him as a new member of our core team. As I'm sure many have noticed, he has been above our stats requirements for several months now. More importantly, he has been reviewing a wide breadth of topics and seems to have a strong understanding of our code base. He also seems to be doing a great job at providing valuable feedback and being attentive to responses on his reviews. As such, I think he would make a great addition to our core team. Can the other core team members please reply with your votes if you agree or disagree. Thanks! Greg +1 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] Deploy upstream OpenStack by Fuel
Hi all, let us give the latest status on deploy upstream OpenStack by Fuel [1]. We consider a several use cases: 1. I want to build Fuel ISO with latest pre-built OpenStack vanilla packages. In this case packages will be downloaded from repository, which is automatically built on daily basis. 2. I want to build Fuel ISO with upstream OpenStack code locally. So, all OpenStack packages will be build locally. 3. I want to build Fuel ISO with already built packages, but with the use of my own fork of one or more OpenStack components (e.g. Nova) 4. I want to build a package of my implementation of OpenStack component and make it available on already running Fuel Master node, so I can immediately start deployment of new environments with this new package - requires repo rebuild on master node 5. I want to patch already running OpenStack environment by my implementation of OpenStack component, doing it by OpenStack patching feature in Fuel [2] - In this case flow would look like: package build - repo update - patch environment in automated manner, delivering the code in minutes to the running environment #1 - #3 are covered by [1]. For #4, #5 blueprints to be created. Currently, cases #1-#3 are implemented for RPM packages of OpenStack, with one prerequisite - you have to apply changes from [4] to fuel-main repo before building a Fuel ISO with OpenStack packages from master. For more details on configuration parameters and make iso examples, please refer to [3] [1] https://blueprints.launchpad.net/fuel/+spec/openstack-from-master [2] https://blueprints.launchpad.net/fuel/+spec/patch-openstack [3] https://etherpad.openstack.org/p/fuel-build-rpms-from-anywhere [4] https://review.openstack.org/#/c/116667/ We will be very welcome to hear more ideas on this. -- Regards, Vitaly Parakhin. IRC: brain461 @ chat.freenode.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA] Meeting Thursday September 11th at 17:00 UTC
Hi Everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be this Thursday, September 11th at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to the agenda. It's also worth noting that last week we started having a regular dedicated Devstack topic during the meetings. So if anyone is interested in Devstack development please join the meetings to be a part of the discussion. To help people figure out what time 17:00 UTC is in other timezones tomorrow's meeting will be at: 13:00 EDT 02:00 JST 02:30 ACST 19:00 CEST 12:00 CDT 10:00 PDT -Matt Treinish pgpURrvc_UpiZ.pgp Description: PGP signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting
On 09/10/2014 12:47 AM, Devananda van der Veen wrote: As was pointed out in the TC meeting today, Zaqar is (was?) actually aiming to provide Messaging-as-a-Service -- not queueing as a service! This is another way of saying it's more like email and less like AMQP The glossary[1] describes a model that is much more like messaging-oriented-middleware (AMQP and other things like it) than email. To me, messaging-as-a-service is perhaps a more general purpose term. The concept of a 'queue' might not be exposed in such a system (e.g. MQTT), whereas the term queueing-as-a-service implies that perhaps it is. I don't see them as necessarily distinct things though. The model exposed by Zaqar certainly has 'queues' featured prominently. [1] http://docs.openstack.org/developer/marconi/glossary.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility
Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token expiration time Timeout thrown Operation fails Terrible 500 error sent back to user It seems like we should have a standard pattern that on token expiration the underlying code at least gives one retry to try to establish a new token to complete the flow, however as far as I can tell *no* clients do this. I know we had to add that into Tempest because tempest runs can exceed 1 hr, and we want to avoid random fails just because we cross a token expiration boundary. Anyone closer to the clients that can comment here? -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility
I think at least clients supporting keystone sessions that are configured to use the auth.Password mech supports this since re-auth is done by the session rather then the service client itself. 2014-09-10 16:14 GMT+02:00 Sean Dague s...@dague.net: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token expiration time Timeout thrown Operation fails Terrible 500 error sent back to user It seems like we should have a standard pattern that on token expiration the underlying code at least gives one retry to try to establish a new token to complete the flow, however as far as I can tell *no* clients do this. I know we had to add that into Tempest because tempest runs can exceed 1 hr, and we want to avoid random fails just because we cross a token expiration boundary. Anyone closer to the clients that can comment here? -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][policy] Group-based Policy next steps
On 9/9/14, 7:51 PM, Jay Pipes wrote: On 09/09/2014 06:57 PM, Kevin Benton wrote: Hi Jay, The main component that won't work without direct integration is enforcing policy on calls directly to Neutron and calls between the plugins inside of Neutron. However, that's only one component of GBP. All of the declarative abstractions, rendering of policy, etc can be experimented with here in the stackforge project until the incubator is figured out. OK, thanks for the explanation Kevin, that helps! I'll add that there is likely to be a close coupling between ML2 mechanism drivers and corresponding GBP policy drivers for some of the back-end integrations. These will likely share local state such as connections to controllers, and may interact with each other as part of processing core and GBP API requests. Development, review, and packaging of these would be facilitated by having them on the same branch in the same repo, but its probably not absolutely necessary. -Bob Best, -jay On Tue, Sep 9, 2014 at 12:01 PM, Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com wrote: On 09/04/2014 12:07 AM, Sumit Naiksatam wrote: Hi, There's been a lot of lively discussion on GBP a few weeks back and we wanted to drive forward the discussion on this a bit more. As you might imagine, we're excited to move this forward so more people can try it out. Here are the options: * Neutron feature branch: This presumably allows the GBP feature to be developed independently, and will perhaps help in faster iterations. There does seem to be a significant packaging issue [1] with this approach that hasn’t been completely addressed. * Neutron-incubator: This allows a path to graduate into Neutron, and will be managed by the Neutron core team. That said, the proposal is under discussion and there are still some open questions [2]. * Stackforge: This allows the GBP team to make rapid and iterative progress, while still leveraging the OpenStack infra. It also provides option of immediately exposing the existing implementation to early adopters. Each of the above options does not preclude moving to the other at a later time. Which option do people think is more preferable? (We could also discuss this in the weekly GBP IRC meeting on Thursday: https://wiki.openstack.org/__wiki/Meetings/Neutron_Group___Policy https://wiki.openstack.org/wiki/Meetings/Neutron_Group_Policy) Thanks! [1] http://lists.openstack.org/__pipermail/openstack-dev/2014-__August/044283.html http://lists.openstack.org/pipermail/openstack-dev/2014-August/044283.html [2] http://lists.openstack.org/__pipermail/openstack-dev/2014-__August/043577.html http://lists.openstack.org/pipermail/openstack-dev/2014-August/043577.html Hi all, IIRC, Kevin was saying to me in IRC that GBP really needs to live in-tree due to it needing access to various internal plugin points and to be able to call across different plugin layers/drivers inside of Neutron. If this is the case, how would the stackforge GBP project work if it wasn't a fork of Neutron in its entirety? Just curious, -jay _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Review metrics - what do we want to measure?
On 09/10/2014 03:57 AM, Steven Hardy wrote: On Thu, Sep 04, 2014 at 01:54:20PM +, Jeremy Stanley wrote: On 2014-09-04 11:01:55 +0100 (+0100), Derek Higgins wrote: [...] How would people feel about turning [auto-abandon] back on? A lot of reviewers (myself among them) feel auto-abandon was a cold and emotionless way to provide feedback on a change. Especially on high-change-volume projects where core reviewers may at times get sucked into triaging other problems for long enough that the auto-abandoner kills lots of legitimate changes (possibly from new contributors who will get even more disgusted by this than the silence itself and walk away indefinitely with the impression that we really aren't a welcoming development community at all). I can understand this argument, and perhaps an auto-abandon with a long period like say a month for the submitter to address comments and reset the clock would be a reasonable compromise? The problem we have now, is there is a constantly expanding queue of zombie reviews, where the submitter got some negative feedback and, for whatever reason, has chosen not to address it. For example, in my Incoming reviews queue on the gerrit dashboard, I've got reviews I (or someone) -1'd over four months ago, with zero feedback from the submitter, what value is there to these reviews cluttering up the dashboard for every reviewer? To make matters worse Jenkins comes along periodically and rechecks or figures out the patch merge failed and bumps the zombie back up to the top of the queue - obviously I don't realise that it's not a human comment I need to read until I've expended effort clicking on the review and reading it :( From a reviewer perspective, it's impossible, and means the review dashboard is basically unusable without complex queries to weed out the active reviews from the zombies. Can it be done on a per project basis? It can, by running your own... but again it seems far better for core reviewers to decide if a change has potential or needs to be abandoned--that way there's an accountable human making that deliberate choice rather than the review team hiding behind an automated process so that no one is to blame for hurt feelings besides the infra operators who are enforcing this draconian measure for you. With all the threads about core-reviewer overload, I think it's unreasonable to expect us to scrub the review queue making daily judgements on whether a patch should be abandoned, and I'd argue that a human abandoning another human's patch has much more demotivating impact on contributors than a clearly documented automated process that you must address negative review comments within a set period or your review will expire. If you think about mailing list patch workflow, it's similar - you post your patch, get email review feedback, then post new reviews with fixes. If you fail to post new reviews with fixes, your review falls out the bottom of people's inboxes and you don't get your patch merged. To make the whole process a little friendlier we could increase the time frame from 1 week to 2. snarkHow about just automatically abandon any new change as soon as it's published, and if the contributor really feels it's important they'll unabandon it./snark I think that's a pretty unfair comment - all reviewers and most contributors are working really hard, all we're asking for is that contributors work with reviewers to get their patch into shape withing a reasonable time. :( As someone who's drinking from the firehose every day with a seemingly insurmountable review queue, I'd rather we worked towards processes which help us manage the workload in a sustainable way, rather than turning the review queue into the zombie-soup-of-dispair it currently is. All we need is two things: 1. A way to automatically escalate reviews which have no feedback at all from core reviewers within a reasonable time (say a week or two) 2. A way to automatically remove reviews from the queue which have core negative feedback with no resolution within a reasonable time (say a month or several weeks, so it's not percieved contributor-hostile). A suggestion Robert made during that discussion was to have auto-WIP instead of auto-abandon. That should be less hostile to contributors and still makes it easy to filter out the reviews that aren't ready to merge anyway. For me personally, and for the sake of tracking the stats, that would be sufficient to address the problem. Note (2) still allows core reviewers to decide if a change has potential, it just becomes opt-in, e.g we have to address the issues which prevent us giving it positive feedback, either by directly working with the contributor, or delegating the work to an active contributor if the original patch author has decided not to continue work on the patch. Ultimately, it's not just about reviews - who's going to maintain all
Re: [openstack-dev] On an API proxy from baremetal to ironic
On 09/09/2014 11:22 PM, Russell Bryant wrote: On 09/09/2014 05:24 PM, Michael Still wrote: Hi. One of the last things blocking Ironic from graduating is deciding whether or not we need a Nova API proxy for the old baremetal extension to new fangled Ironic API. The TC has asked that we discuss whether we think this functionality is actually necessary. It should be noted that we're _not_ talking about migration of deployed instances from baremetal to Ironic. That is already implemented. What we are talking about is if users post-migration should be able to expect their previous baremetal Nova API extension to continue to function, or if they should use the Ironic APIs from that point onwards. Nova had previously thought this was required, but it hasn't made it in time for Juno unless we do a FFE, and it has been suggested that perhaps its not needed at all because it is an admin extension. To be super specific, we're talking about the baremetal nodes admin extension here. This extension has the ability to: - list nodes running baremetal - show detail of one of those nodes - create a new baremetal node - delete a baremetal node Only the first two of those would be supported if we implemented a proxy. So, discuss. I'm in favor of proceeding with deprecation without requiring the API proxy. In the case of user facing APIs, the administrators in charge of upgrading the cloud do not have full control over all of the apps using the APIs. In this particular case, I would expect that the cloud administrators have *complete* control over the use of these APIs. Assuming we have one overlap release (Juno) to allow the migration to occur and given proper documentation of the migration plan and release notes stating the fact that the old APIs are going away, we should be fine. In summary, +1 to moving forward without the API proxy requirement. The thing is, we have the patch - https://review.openstack.org/#/c/120433/, so it's not like there is a zomg run around to get the patch. I think we should FFE this patch as it provides a smoother transition from baremetal to ironic. The patch is extremely low risk to the rest of Nova, as it's a surface proxy feature, so lets just land it and move forward. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility
Do we know which versions of the clients do that? -Sean On 09/10/2014 10:22 AM, Endre Karlson wrote: I think at least clients supporting keystone sessions that are configured to use the auth.Password mech supports this since re-auth is done by the session rather then the service client itself. 2014-09-10 16:14 GMT+02:00 Sean Dague s...@dague.net mailto:s...@dague.net: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token expiration time Timeout thrown Operation fails Terrible 500 error sent back to user It seems like we should have a standard pattern that on token expiration the underlying code at least gives one retry to try to establish a new token to complete the flow, however as far as I can tell *no* clients do this. I know we had to add that into Tempest because tempest runs can exceed 1 hr, and we want to avoid random fails just because we cross a token expiration boundary. Anyone closer to the clients that can comment here? -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Kilo Cycle Goals Exercise
tl;dr I'm concerned we're conflating user concerns and contributor concerns. I'd like to see laser focus on two things that help both: 1) Define layers in our governance and integration efforts 2) Pay down technical debt with a focus on supporting users. more Great kick off Joe, thanks. I have been waiting to reply to this thread on purpose while I try to gather as much info as I can about what both users and contributors are interested in for our future. I have a pretty specific curiosity about how we're framing this request: I see a spilt between contributor concerns and user concerns. Example contributor concerns: scope definition incubation and integration requirements heavy-handed processes review pileups contributor license agreement concerns about preventing contributions and openness technical debt weighing a project down Example user concerns: how do I know what's tested; what's buggy how do I trust a part of OpenStack to put it into production and maintain over time why isn't there complete driver or hypervisor documentation for fill-in-the-blank logging security best practices high availability across OpenStack monitoring OpenStack monitoring apps on my OpenStack infrastructure my own users have needs; how can I get upstream to care about them These example concerns are not comprehensive but I worry about conflation causing us all to burnout and flail. I know we all work in the gray areas but there's a black-and-white problem due to our current governance. Since I write docs and work on technical committee concerns, I sit on a cross-project and cross-concern bridge all the time, advocating, prioritizing, standardizing, and weighing needs across many project teams. The user concerns are a higher priority for docs currently, because they're a support mechanism. I think the Kilo cycle should be dedicated to: - Fulfill the promise of layers for defining OpenStack integration gradients in governance and integration taxes such as consistency across projects, reviews on projects with many drivers/contributors, infra, docs, and testing - Pay down technical debt by sharply focusing on capabilities rather than drivers that fulfill them Two focus areas. Each of those will have a lot of work items. Both find more common ground for devs and users. Let's do this! Anne On Wed, Sep 3, 2014 at 10:37 AM, Joe Gordon joe.gord...@gmail.com wrote: As you all know, there has recently been several very active discussions around how to improve assorted aspects of our development process. One idea that was brought up is to come up with a list of cycle goals/project priorities for Kilo [0]. To that end, I would like to propose an exercise as discussed in the TC meeting yesterday [1]: Have anyone interested (especially TC members) come up with a list of what they think the project wide Kilo cycle goals should be and post them on this thread by end of day Wednesday, September 10th. After which time we can begin discussing the results. The goal of this exercise is to help us see if our individual world views align with the greater community, and to get the ball rolling on a larger discussion of where as a project we should be focusing more time. best, Joe Gordon [0] http://lists.openstack.org/pipermail/openstack-dev/2014-August/041929.html [1] http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-09-02-20.04.log.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] webnumbr bug graphs appear dead for bug triage
All the various bug triage graphs point out to webnumbr urls from our wiki - https://wiki.openstack.org/wiki/BugTriage All of webnumbr appears to be dead and not returning any data. Why this service was used predates me. Does anyone know why? Anyone know if it's going to come back? Or should we just purge those links? Thanks, -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Node renaming in Fuel
Hi Dmitry,
It's not as easy as you think. The node's name may be reseted during
some of our actions. For example, if you remove node from a cluster a
node's name will be reseted.
So the idea of the blueprint [1] is to provide mechanism when you once
set names and they will not be reseted.
[1]: https://blueprints.launchpad.net/fuel/+spec/node-naming
- Igor
On Wed, Sep 10, 2014 at 4:19 PM, Dmitry Ukov du...@mirantis.com wrote:
Hello Fuelers,
I've figured out that there is a blueprint regarding node naming
(https://blueprints.launchpad.net/fuel/+spec/node-naming). It looks like
this this feature really has not been started. That being said all necessary
preparations in Nailgun code have been already done. The only thing that is
needed to be done is to enable this feature (in fact 4 lines of code).
--- a/nailgun/nailgun/objects/node.py
+++ b/nailgun/nailgun/objects/node.py
@@ -654,7 +654,10 @@ class Node(NailgunObject):
@classmethod
def make_slave_name(cls, instance):
-return unode-{node_id}.format(node_id=instance.id)
+if instance.name.startswith(Untitled):
+return unode-{node_id}.format(node_id=instance.id)
+else:
+return u{node_id}.format(node_id=instance.name.lower())
@classmethod
def make_slave_fqdn(cls, instance):
Should I consider this as a bug or this blueprint will be implemented in the
nearest future?
--
Kind regards
Dmitry Ukov
IT Engineer
Mirantis, Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release] client release deadline - Sept 18th
On 9/9/2014 4:19 PM, Sean Dague wrote: As we try to stabilize OpenStack Juno, many server projects need to get out final client releases that expose new features of their servers. While this seems like not a big deal, each of these clients releases ends up having possibly destabilizing impacts on the OpenStack whole (as the clients do double duty in cross communicating between services). As such in the release meeting today it was agreed clients should have their final release by Sept 18th. We'll start applying the dependency freeze to oslo and clients shortly after that, all other requirements should be frozen at this point unless there is a high priority bug around them. -Sean Thanks for bringing this up. We do our own packaging and need time for legal clearances and having the final client releases done in a reasonable time before rc1 is helpful. I've been pinging a few projects to do a final client release relatively soon. python-neutronclient has a release this week and I think John was planning a python-cinderclient release this week also. -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] What's up doc? Sept. 10, 2014
Bug squash day worked well on 9/9 -- squashing and triaging bugs, bringing the openstack-manuals backlog to about 400, with 50 in progress even now! The openstack-api-site backlog went to about 200 with 15 in progress. We also had some fairly new docs contributors join in -- thank you everyone who helped out. __In review and merged this past week__ This week was about bug squashing! __High priority doc work__ Install Guide testing starts in earnest once packages are available and with feature freeze this should be in the next week or so. We've been keeping up with both CLI and config reference patches at milestone releases. __Ongoing doc work__ The separate Networking Guide is still ongoing. Nick Chase has been interviewing contractors to pick up the work that still needs to be done, thank you Nick. Shaun McCance has moved to other contracts, thank you Shaun for your efforts. __New incoming doc requests__ Security Guide authors, there is interest in specializing your book by Cisco, and I wanted to let everyone know how exciting this is. I've asked them to look at the bug backlog and see what they could contribute upstream also, and they're willing and interested. The vision is that upstream docs can be reused and repurposed so I'm happy to see this vision become reality. I'm finding out if there are any particular mechanics needed so stay tuned. __Doc tools updates__ We should have a new release of the clouddocs-maven-plugin in the next 2 weeks with these features, mostly API reference-focused: Fix css padding-top for Rackspace output https://review.openstack.org/108156 Add the title of the service to API reference https://review.openstack.org/107768 Add fanatiguy to Rackspace API Ref https://review.openstack.org/112703 Add support for yaml syntax highlighting: https://review.openstack.org/#/c/111219/1 Allow error phrase to be listed with response code: https://review.openstack.org/106173 __Other doc news__ I've been clarifying what it means to be incubated from a documentation perspective on this wiki page. Most recently the Manila project was incubated and they're looking into how docs are incubated and integrated. The training team will discuss whether a separate program incubation application makes sense for them at their next team meeting, held Mondays at 12:00 Central in #openstack-meeting. An end user interest group has been having a lot of discussion on the user-committee mailing list. I wanted to be sure docs people know that the Foundation is seeking input on developer.openstack.org and looking for an app ecosystem community manager. If you know someone doing great work on cloud apps on OpenStack, please join us on that mailing list thread and let us know. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes
Hi folks, Some of you may know that there is ongoing work to achieve kindof data-driven orchestration for Fuel. If this is new to you, please get familiar with spec: https://review.openstack.org/#/c/113491/ Knowing that running random command on nodes will be probably most usable type of orchestration extension, i want to discuss our solution for this problem. Plugin writer will need to do two things: 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is completely configurable, we just need to reach agreement) /etc/puppet/tasks/echo/task.yaml with next content: type: exec cmd: echo 1 2. Provide control plane with orchestration metadata /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 compute: - task: echo description: Simple echo for you priority: 1000 This is done in order to separate concerns of orchestration logic and tasks. From plugin writer perspective it is far more usable to provide exact command in orchestration metadata itself, like: /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec compute: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec I would prefer to stick to the first, because there is benefits of using one interface between all tasks executors (puppet, exec, maybe chef), which will improve debuging and development process. So my question is first - good enough? Or second is essential type of plugin to support? If you want additional implementation details check: https://review.openstack.org/#/c/118311/ https://review.openstack.org/#/c/113226/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Node renaming in Fuel
Also, how about checking for duplicated name? That it matches standard for
DNS?
return u{node_id}.format(node_id=instance.name.lower())
you could just return instance.name.lower() I believe..
On Wed, Sep 10, 2014 at 7:00 PM, Igor Kalnitsky ikalnit...@mirantis.com
wrote:
Hi Dmitry,
It's not as easy as you think. The node's name may be reseted during
some of our actions. For example, if you remove node from a cluster a
node's name will be reseted.
So the idea of the blueprint [1] is to provide mechanism when you once
set names and they will not be reseted.
[1]: https://blueprints.launchpad.net/fuel/+spec/node-naming
- Igor
On Wed, Sep 10, 2014 at 4:19 PM, Dmitry Ukov du...@mirantis.com wrote:
Hello Fuelers,
I've figured out that there is a blueprint regarding node naming
(https://blueprints.launchpad.net/fuel/+spec/node-naming). It looks like
this this feature really has not been started. That being said all
necessary
preparations in Nailgun code have been already done. The only thing that
is
needed to be done is to enable this feature (in fact 4 lines of code).
--- a/nailgun/nailgun/objects/node.py
+++ b/nailgun/nailgun/objects/node.py
@@ -654,7 +654,10 @@ class Node(NailgunObject):
@classmethod
def make_slave_name(cls, instance):
-return unode-{node_id}.format(node_id=instance.id)
+if instance.name.startswith(Untitled):
+return unode-{node_id}.format(node_id=instance.id)
+else:
+return u{node_id}.format(node_id=instance.name.lower())
@classmethod
def make_slave_fqdn(cls, instance):
Should I consider this as a bug or this blueprint will be implemented in
the
nearest future?
--
Kind regards
Dmitry Ukov
IT Engineer
Mirantis, Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Mike Scherbakov
#mihgen
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] webnumbr bug graphs appear dead for bug triage
On 10/09/14 22:51, Sean Dague wrote: All the various bug triage graphs point out to webnumbr urls from our wiki - https://wiki.openstack.org/wiki/BugTriage All of webnumbr appears to be dead and not returning any data. Why this service was used predates me. Does anyone know why? Anyone know if it's going to come back? Or should we just purge those links? Not sure why it was used either, but it has been very flaky for quite a while. The most recent downtime has been the longest, and I believe we should indeed use another service to replace webnumbr. I think we already have similar graphs from bugday (http://status.openstack.org/bugday/), but only for one day, and/or the activity portal (http://activity.openstack.org/dash/browser/its-repos.html). Regards, Tom ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] anyone using RabbitMQ with active/active mirrored queues?
Hi, I see that the OpenStack high availability guide is still recommending the active/standby method of configuring RabbitMQ. Has anyone tried using active/active with mirrored queues as recommended by the RabbitMQ developers? If so, what problems did you run into? Thanks, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] webnumbr bug graphs appear dead for bug triage
Sean Dague wrote: All the various bug triage graphs point out to webnumbr urls from our wiki - https://wiki.openstack.org/wiki/BugTriage All of webnumbr appears to be dead and not returning any data. Why this service was used predates me. Does anyone know why? Anyone know if it's going to come back? Or should we just purge those links? It used to be a convenient way to graph bug stats from launchpad over time. But I think the service died. So yes, links can probably be purged now. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] 凌志阳149759 不在办公室。
我将从 2014/09/10 开始离开办公室，到 2014/09/11 时返回。 我将在回来之后答复您的消息。 ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited. If you have received this mail in error, please delete it and notify us immediately. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Plugable solution for running abstract commands on nodes
Hi, as for execution of arbitrary code across the OpenStack cluster - I was thinking of mcollective + fact filters: 1) we need to start using mcollective facts [0] [2] - we don't use/configure this currently 2) use mcollective execute_shell_command agent (or any other agent) with fact filter [1] So, for example, if we have mcollective fact called node_roles: node_roles: compute ceph-osd Then we can execute shell cmd on all compute nodes like this: mco rpc execute_shell_command execute cmd=/some_script.sh -F node_role=/compute/ Of course, we can use more complicated filters to run commands more precisely. [0] https://projects.puppetlabs.com/projects/mcollective-plugins/wiki/FactsFacterYAML [1] https://docs.puppetlabs.com/mcollective/reference/ui/filters.html#fact-filters [2] https://docs.puppetlabs.com/mcollective/reference/plugins/facts.html On Wed, Sep 10, 2014 at 6:04 PM, Dmitriy Shulyak dshul...@mirantis.com wrote: Hi folks, Some of you may know that there is ongoing work to achieve kindof data-driven orchestration for Fuel. If this is new to you, please get familiar with spec: https://review.openstack.org/#/c/113491/ Knowing that running random command on nodes will be probably most usable type of orchestration extension, i want to discuss our solution for this problem. Plugin writer will need to do two things: 1. Provide custom task.yaml (i am using /etc/puppet/tasks, but this is completely configurable, we just need to reach agreement) /etc/puppet/tasks/echo/task.yaml with next content: type: exec cmd: echo 1 2. Provide control plane with orchestration metadata /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 compute: - task: echo description: Simple echo for you priority: 1000 This is done in order to separate concerns of orchestration logic and tasks. From plugin writer perspective it is far more usable to provide exact command in orchestration metadata itself, like: /etc/fuel/tasks/echo_task.yaml controller: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec compute: - task: echo description: Simple echo for you priority: 1000 cmd: echo 1 type: exec I would prefer to stick to the first, because there is benefits of using one interface between all tasks executors (puppet, exec, maybe chef), which will improve debuging and development process. So my question is first - good enough? Or second is essential type of plugin to support? If you want additional implementation details check: https://review.openstack.org/#/c/118311/ https://review.openstack.org/#/c/113226/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility
On Wed, Sep 10, 2014 at 10:14:32AM -0400, Sean Dague wrote: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token expiration time Timeout thrown Operation fails Terrible 500 error sent back to user We actually have this exact problem in Heat, which I'm currently trying to solve: https://bugs.launchpad.net/heat/+bug/1306294 Can you clarify, is the issue either: 1. Create novaclient object with username/password 2. Do series of operations via the client object which eventually fail after $n operations due to token expiry or: 1. Create novaclient object with username/password 2. Some really long operation which means token expires in the course of the service handling the request, blowing up and 500-ing If the former, then it does sound like a client, or usage-of-client bug, although note if you pass a *token* vs username/password (as is currently done for glance and heat in tempest, because we lack the code to get the token outside of the shell.py code..), there's nothing the client can do, because you can't request a new token with longer expiry with a token... However if the latter, then it seems like not really a client problem to solve, as it's hard to know what action to take if a request failed part-way through and thus things are in an unknown state. This issue is a hard problem, which can possibly be solved by switching to a trust scoped token (service impersonates the user), but then you're effectively bypassing token expiry via delegation which sits uncomfortably with me (despite the fact that we may have to do this in heat to solve the afforementioned bug) It seems like we should have a standard pattern that on token expiration the underlying code at least gives one retry to try to establish a new token to complete the flow, however as far as I can tell *no* clients do this. As has been mentioned, using sessions may be one solution to this, and AFAIK session support (where it doesn't already exist) is getting into various clients via the work being carried out to add support for v3 keystone by David Hu: https://review.openstack.org/#/q/owner:david.hu%2540hp.com,n,z I see patches for Heat (currently gating), Nova and Ironic. I know we had to add that into Tempest because tempest runs can exceed 1 hr, and we want to avoid random fails just because we cross a token expiration boundary. I can't claim great experience with sessions yet, but AIUI you could do something like: from keystoneclient.auth.identity import v3 from keystoneclient import session from keystoneclient.v3 import client auth = v3.Password(auth_url=OS_AUTH_URL, username=USERNAME, password=PASSWORD, project_id=PROJECT, user_domain_name='default') sess = session.Session(auth=auth) ks = client.Client(session=sess) And if you can pass the same session into the various clients tempest creates then the Password auth-plugin code takes care of reauthenticating if the token cached in the auth plugin object is expired, or nearly expired: https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/auth/identity/base.py#L120 So in the tempest case, it seems like it may be a case of migrating the code creating the clients to use sessions instead of passing a token or username/password into the client object? That's my understanding of it atm anyway, hopefully jamielennox will be along soon with more details :) Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Horizon] Licensing issue with using JSHint in build
I noticed that the build is using JSHint now, and before I consider syncing it with the proposed options from the JavaScript best practices (https://review.openstack.org/#/c/117595/), I wanted to double check and be sure Horizon got past the legal problem with the good/evil licensing. Some background for those who are not aware. JSHint was authored by Doug Crockford, and he added an extra line in the licensing, The software shall be used for good, not evil. The issue is in the definition of what is good and what is evil. It is too subjective, what is evil differs from person to person therefore ends up being a liability and leaving users open to frivolous lawsuits. Did Horizon get permission or find some way around the licensing issue? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][neutron][cinder] Averting the Nova crisis by splitting out virt drivers
On Wed, Sep 10, 2014 at 3:44 AM, Daniel P. Berrange berra...@redhat.com wrote: On Tue, Sep 09, 2014 at 05:14:43PM -0700, Stefano Maffulli wrote: To me, this means you don't really want a sin bin where you dump drivers and tell them not to come out until they're fit to be reviewed by the core; You want a trusted driver community which does its own reviews and means the core doesn't have to review them. I think we're going somewhere here, based on your comment and other's: we may achieve some result if we empower a new set of people to manage drivers, keeping them in the same repositories where they are now. This new set of people may not be the current core reviewers but other with different skillsets and more capable of understanding the driver's ecosystem, needs, motivations, etc. I have the impression this idea has been circling around for a while but for some reason or another (like lack of capabilities in gerrit and other reasons) we never tried to implement it. Maybe it's time to think about an implementation. We have been thinking about mentors https://wiki.openstack.org/wiki/Mentors, maybe that's a way to go? Sub-team with +1.5 scoring capabilities? I think that setting up subteams is neccessary to stop us imploding but I don't think it is enough. As long as we have one repo we're forever going to have conflict contention in deciding which features to accept, which is a big factor in problems today. I favour the strong split of the drivers into separate repositories to remove the contente between the teams as much as is practical. I would be cautious around sub-teams. Our experience in Neutron has been that we do a very good job of setting up sub-teams, but a terrible job at deciding when they should be spun-down and folded back in. Because in a lot of cases, a sub-team's existance should be for a short period of time. The other problem is that sub-teams can tend to wander away from the broader team, making it harder for their work to be integrated back into the whole. All of this is to say that sub-teams require coordination and lots of communication, and should be carefully watched, groomed, and culled when necessary. Thanks, Kyle Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release] client release deadline - Sept 18th
On Wed, Sep 10, 2014 at 10:01 AM, Matt Riedemann mrie...@linux.vnet.ibm.com wrote: On 9/9/2014 4:19 PM, Sean Dague wrote: As we try to stabilize OpenStack Juno, many server projects need to get out final client releases that expose new features of their servers. While this seems like not a big deal, each of these clients releases ends up having possibly destabilizing impacts on the OpenStack whole (as the clients do double duty in cross communicating between services). As such in the release meeting today it was agreed clients should have their final release by Sept 18th. We'll start applying the dependency freeze to oslo and clients shortly after that, all other requirements should be frozen at this point unless there is a high priority bug around them. -Sean Thanks for bringing this up. We do our own packaging and need time for legal clearances and having the final client releases done in a reasonable time before rc1 is helpful. I've been pinging a few projects to do a final client release relatively soon. python-neutronclient has a release this week and I think John was planning a python-cinderclient release this week also. Just a slight correction: python-neutronclient will have a final release once the L3 HA CLI changes land [1]. Thanks, Kyle [1] https://review.openstack.org/#/c/108378/ -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] On an API proxy from baremetal to ironic
I thought it might be helpful to show a sample of the output from the proxied commands: Please find the example here: http://paste.openstack.org/show/Em861wMwFvrFlsWkugfX Chris Krelle NobodyCam On Wed, Sep 10, 2014 at 7:33 AM, Sean Dague s...@dague.net wrote: On 09/09/2014 11:22 PM, Russell Bryant wrote: On 09/09/2014 05:24 PM, Michael Still wrote: Hi. One of the last things blocking Ironic from graduating is deciding whether or not we need a Nova API proxy for the old baremetal extension to new fangled Ironic API. The TC has asked that we discuss whether we think this functionality is actually necessary. It should be noted that we're _not_ talking about migration of deployed instances from baremetal to Ironic. That is already implemented. What we are talking about is if users post-migration should be able to expect their previous baremetal Nova API extension to continue to function, or if they should use the Ironic APIs from that point onwards. Nova had previously thought this was required, but it hasn't made it in time for Juno unless we do a FFE, and it has been suggested that perhaps its not needed at all because it is an admin extension. To be super specific, we're talking about the baremetal nodes admin extension here. This extension has the ability to: - list nodes running baremetal - show detail of one of those nodes - create a new baremetal node - delete a baremetal node Only the first two of those would be supported if we implemented a proxy. So, discuss. I'm in favor of proceeding with deprecation without requiring the API proxy. In the case of user facing APIs, the administrators in charge of upgrading the cloud do not have full control over all of the apps using the APIs. In this particular case, I would expect that the cloud administrators have *complete* control over the use of these APIs. Assuming we have one overlap release (Juno) to allow the migration to occur and given proper documentation of the migration plan and release notes stating the fact that the old APIs are going away, we should be fine. In summary, +1 to moving forward without the API proxy requirement. The thing is, we have the patch - https://review.openstack.org/#/c/120433/, so it's not like there is a zomg run around to get the patch. I think we should FFE this patch as it provides a smoother transition from baremetal to ironic. The patch is extremely low risk to the rest of Nova, as it's a surface proxy feature, so lets just land it and move forward. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] Bringing back auto-abandon (was: Review metrics - what do we want to measure?)
On 2014-09-10 09:57:17 +0100 (+0100), Steven Hardy wrote: I can understand this argument, and perhaps an auto-abandon with a long period like say a month for the submitter to address comments and reset the clock would be a reasonable compromise? Perhaps, but the old script lacked a bunch of things we'd need for this (it didn't understand/skip WIP changes, it also abandoned changes with no reviews at all, it didn't support for the version of Gerrit we're running now) and was tied to old cruft we've excised (particularly the Launchpad-Gerrit group membership sync code) so would likely need to be rewritten mostly from scratch. Probably if we were to do it right, it would be something like a new kind of periodic meta-job job in Zuul which could be selectively added to projects who want it. The problem we have now, is there is a constantly expanding queue of zombie reviews, where the submitter got some negative feedback and, for whatever reason, has chosen not to address it. I agree that's a problem, but I'm unconvinced it's because we lack some system to automatically make them go away. For example, in my Incoming reviews queue on the gerrit dashboard, I've got reviews I (or someone) -1'd over four months ago, with zero feedback from the submitter, what value is there to these reviews cluttering up the dashboard for every reviewer? The moment you, as a core reviewer, abandon one of those with a friendly note it will cease to clutter up the dashboard for anyone. But doing so also gives you an opportunity to possibly notice that this is actually a valuable bug fix/enhancement for your project and take it over. If it gets automatically abandoned because some random reviewer did a drive-by -1 about whitespace preferences a month ago, then you may never know. To make matters worse Jenkins comes along periodically and rechecks or figures out the patch merge failed and bumps the zombie back up to the top of the queue - obviously I don't realise that it's not a human comment I need to read until I've expended effort clicking on the review and reading it :( If you, again as a core reviewer, notice that there is a change which needs work but perhaps is not in a situation where outright abandonment is warranted, mark it with workflow -1 (work in progress) then that too will break the cycle. Also Zuul is only spontaneously leaving votes on changes which suddenly cease to be able to merge due to merge conflicts appearing in the target branch, and will only do that to a patchset once... it doesn't periodically retest any changes beyond that without some intervention. From a reviewer perspective, it's impossible, and means the review dashboard is basically unusable without complex queries to weed out the active reviews from the zombies. I would argue that the default Gerrit dashboard is effectively unusable for a variety of other reasons anyway, which is why there are custom dashboards proliferating (globally, per-project and per-user) to maximize reviewer efficiency and use cases. For that matter, it's also arguable that's one of the driving factors behind the increasing number of Gerrit clients and front-end replacements. All we need is two things: Well, three... 0. A clearly documented set of expectations of code contributors so that they know in advance their proposed changes will be discarded if they do not look after them fairly continually to address comments as they come. This should include the expectation that they may have to gratuitously submit new patchsets from time to time to clear drive-by negative reviews which make pointless demands or are based on misunderstandings of those patches (because core reviewers will not be able to clear other reviewer's -1 votes manually and the original reviewer may never revisit that change), or continue to un-abandon their changes while waiting for proper reviews to find them. Also some custom query they check from time to time to spot their auto-abandoned reviews so they can restore them (this was one of my standard searches back when we used the auto-abandoner). 1. A way to automatically escalate reviews which have no feedback at all from core reviewers within a reasonable time (say a week or two) As a view in reviewday? Or using something like next-review (which I believe has heuristics for selecting neglected reviews for you)? 2. A way to automatically remove reviews from the queue which have core negative feedback with no resolution within a reasonable time (say a month or several weeks, so it's not percieved contributor-hostile). Well, here's part of the challenge... core reviewer is merely a concept we invented. We have Gerrit ACLs which allow anyone to leave a -1 review and also allow *some* reviewers based on a somewhat flexible group membership system to leave -2 reviews. Gerrit itself does not see a -1 review from a so-called core reviewer as being any different from one left by any other reviewer and provides no general
Re: [openstack-dev] On an API proxy from baremetal to ironic
As far as I understand it, though, that's a patch for a read-only mode. It seems bizzare, and possibly dangerous, to proxy read commands, but not write commands. It gives the impression that everything's fine until it's not fine (because someone tried to use an existing script to do a create command). IMHO, it would be better to just tell people up front Update your scripts to use Ironic, because they won't work at all instead of leading people (through empirical evidence) to believe that their scripts will work, and then having them discover later that something broke because they tried to create a node. Best Regards, Solly Ross - Original Message - From: Sean Dague s...@dague.net To: openstack-dev@lists.openstack.org Sent: Wednesday, September 10, 2014 10:33:05 AM Subject: Re: [openstack-dev] On an API proxy from baremetal to ironic On 09/09/2014 11:22 PM, Russell Bryant wrote: On 09/09/2014 05:24 PM, Michael Still wrote: Hi. One of the last things blocking Ironic from graduating is deciding whether or not we need a Nova API proxy for the old baremetal extension to new fangled Ironic API. The TC has asked that we discuss whether we think this functionality is actually necessary. It should be noted that we're _not_ talking about migration of deployed instances from baremetal to Ironic. That is already implemented. What we are talking about is if users post-migration should be able to expect their previous baremetal Nova API extension to continue to function, or if they should use the Ironic APIs from that point onwards. Nova had previously thought this was required, but it hasn't made it in time for Juno unless we do a FFE, and it has been suggested that perhaps its not needed at all because it is an admin extension. To be super specific, we're talking about the baremetal nodes admin extension here. This extension has the ability to: - list nodes running baremetal - show detail of one of those nodes - create a new baremetal node - delete a baremetal node Only the first two of those would be supported if we implemented a proxy. So, discuss. I'm in favor of proceeding with deprecation without requiring the API proxy. In the case of user facing APIs, the administrators in charge of upgrading the cloud do not have full control over all of the apps using the APIs. In this particular case, I would expect that the cloud administrators have *complete* control over the use of these APIs. Assuming we have one overlap release (Juno) to allow the migration to occur and given proper documentation of the migration plan and release notes stating the fact that the old APIs are going away, we should be fine. In summary, +1 to moving forward without the API proxy requirement. The thing is, we have the patch - https://review.openstack.org/#/c/120433/, so it's not like there is a zomg run around to get the patch. I think we should FFE this patch as it provides a smoother transition from baremetal to ironic. The patch is extremely low risk to the rest of Nova, as it's a surface proxy feature, so lets just land it and move forward. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build
JSHint *isn't* Douglas Crockford. It was written by someone who (understandably) thought Douglas Crockford had some good ideas, but was overzealous. It does mostly the same things, but is more lenient with regards to style elements. The license is as such: https://github.com/jshint/jshint/blob/master/LICENSE You are thinking of JSLint, which is written by Douglas Crockford. - Original Message - From: Aaron Sahlin asah...@linux.vnet.ibm.com To: openstack-dev@lists.openstack.org Sent: Wednesday, September 10, 2014 11:56:37 AM Subject: [openstack-dev] [Horizon] Licensing issue with using JSHint in build I noticed that the build is using JSHint now, and before I consider syncing it with the proposed options from the JavaScript best practices (https://review.openstack.org/#/c/117595/), I wanted to double check and be sure Horizon got past the legal problem with the good/evil licensing. Some background for those who are not aware. JSHint was authored by Doug Crockford, and he added an extra line in the licensing, The software shall be used for good, not evil. The issue is in the definition of what is good and what is evil. It is too subjective, what is evil differs from person to person therefore ends up being a liability and leaving users open to frivolous lawsuits. Did Horizon get permission or find some way around the licensing issue? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] i need some help on this bug Bug #1365892
On Sep 10, 2014, at 4:11 AM, Li Tianqing jaze...@163.com wrote:
After some research, i find the reason for the cycle reference. In closure,
the _fix_paswords.func_closre reference the _fix_passwords. So the
cycle reference happened.
And in https://thp.io/2012/python-gc/python_gc_final_2012-01-22.pdf page 5,
it says that
We observe that Python implementations with distinct GCs behave differently:
CPython does not even try to get the order of finalizers right, and
simply puts uncollectable objects into the global list of garbage for the
developer to deal with manually.
So the gc can not handle cycle reference, then the memory leak happened.
An object is only uncollectable in Python if it has a __del__ method and is
part of an unreachable cycle. Based on a grep, there is only one class in
oslo.messaging that has a __del__ and that is ConnectionContext in
_drivers/amqp.py.
Removing the __del__ method from this object would be the best approach.
There’s nothing wrong with reference cycles in Python as long as we don’t use
__del__, which IMHO has no legitimate use case. Additionally, it is very
difficult to be 100% vigilant against reference cycles reappearing in many
cases, but it is extremely simple to be 100% vigilant about disallowing __del__.
In this case it appears to be a “safety” in case someone uses the
ConnectionContext object outside of being a context manager. I’d fix that and
require that it be used as a context manager only. Guessing that the user is
going to mis-use an object and provide __del__ to protect the user is a bad
idea - and if you genuinely need this pattern, you use a weakref callback
instead:
import weakref
class MyCleanupThing(object):
def __init__(self):
self.connection = connection = Some Connection
self._ref = weakref.ref(
# key: the weakref callback *cannot* refer to self
self, lambda ref: MyCleanupThing._cleanup(connection))
@staticmethod
def _cleanup(connection):
print(Cleaning up %s! % connection)
mc = MyCleanupThing()
print(about to clean up...)
del mc
print(should be clean!)
output:
about to clean up...
Cleaning up Some Connection!
should be clean!
If there is something wrong, please fix it. Thanks
--
Best
Li Tianqing
在 2014-09-10 11:52:28，Li Tianqing jaze...@163.com 写道：
Hello,
I use backdoor of eventlet to enable gc.DEBUG_LEAK, and after wait a few
minutes, i can sure that there will some objects that can not be collected by
gc.collect in gc.garbage.
Those looks like this (catched in ceilometer-collector)
['_context_auth_token', 'auth_token', 'new_pass'],
(cell at 0x363bc58: list object at 0x361c170,
cell at 0x363bda8: function object at 0x361a5f0),
function _fix_passwords at 0x361a5f0,
cell at 0x363bde0: list object at 0x363c680,
cell at 0x363bd70: function object at 0x361a668,
['_context_auth_token', 'auth_token', 'new_pass'],
(cell at 0x363bde0: list object at 0x363c680,
cell at 0x363bd70: function object at 0x361a668),
function _fix_passwords at 0x361a668,
cell at 0x363bf30: list object at 0x361b680,
cell at 0x363e168: function object at 0x361a758,
['_context_auth_token', 'auth_token', 'new_pass'],
(cell at 0x363bf30: list object at 0x361b680,
cell at 0x363e168: function object at 0x361a758),
function _fix_passwords at 0x361a758,
cell at 0x363e1a0: list object at 0x363cdd0,
cell at 0x363e130: function object at 0x361a7d0,
and i suspect those code in oslo.messaging
def _safe_log(log_func, msg, msg_data):
Sanitizes the msg_data field before logging.
SANITIZE = ['_context_auth_token', 'auth_token', 'new_pass']
def _fix_passwords(d):
Sanitizes the password fields in the dictionary.
for k in d.iterkeys():
if k.lower().find('password') != -1:
d[k] = 'SANITIZED'
elif k.lower() in SANITIZE:
d[k] = 'SANITIZED'
elif isinstance(d[k], dict):
_fix_passwords(d[k])
return d
return log_func(msg, _fix_passwords(copy.deepcopy(msg_data)))
i can resolve this problem by add _fix_passwords = None before _safe_log
returns.
But i do not really understand why this problem happened, and in depth why
the gc can not collect those object. Although i can make those uncollectable
objects disappeared.
But this is not good enough, because if you do not understand it you will
write out some code like this in future, and then also has memory leak too.
So can some one helps me give some detailed on recursive closure used like
the code above, and on why gc can not collect them.
Thanks a lot lot ..
--
Best
Li Tianqing
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build
On 2014-09-10 10:56:37 -0500 (-0500), Aaron Sahlin wrote: [...] Did Horizon get permission or find some way around the licensing issue? It's worth mentioning that he seems to consider the free software legal concerns around his license choice amusing and will apparently, upon request, provide an exception to an organization wishing to use his software for evil[1]. He has no interest in changing the situation[2] in JSLint or derivatives (like JSHint) even if that means that they can not be provided as part of Debian[3] or similar legally-concerned distributions. It's probably best to re-raise this question on the legal-disc...@lists.openstack.org mailing list as well. [1] http://dev.hasenj.org/post/3272592502/ibm-and-its-minions [2] https://github.com/jshint/jshint/issues/1234 [3] http://www.mail-archive.com/debian-legal%40lists.debian.org/msg40718.html -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build
On 2014-09-10 13:00:29 -0400 (-0400), Solly Ross wrote: JSHint *isn't* Douglas Crockford. It was written by someone who (understandably) thought Douglas Crockford had some good ideas, but was overzealous. [...] Overzealous enough to copy his code. The license is as such: https://github.com/jshint/jshint/blob/master/LICENSE Ahem. https://github.com/jshint/jshint/blob/master/src/jshint.js#L19 You are thinking of JSLint, which is written by Douglas Crockford. JSHint is a derivative project of JSLint. Sorry to burst your bubble. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build
- Original Message - From: Jeremy Stanley fu...@yuggoth.org To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, September 10, 2014 1:10:18 PM Subject: Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build On 2014-09-10 13:00:29 -0400 (-0400), Solly Ross wrote: JSHint *isn't* Douglas Crockford. It was written by someone who (understandably) thought Douglas Crockford had some good ideas, but was overzealous. [...] Overzealous enough to copy his code. ?? This sentence doesn't make much sense. I meant to say that Douglas Crockford was overzealous (which he is, IMO). The license is as such: https://github.com/jshint/jshint/blob/master/LICENSE Ahem. https://github.com/jshint/jshint/blob/master/src/jshint.js#L19 Fair enough. I stand corrected. I didn't catch that. The general license, however, is as stated. You are thinking of JSLint, which is written by Douglas Crockford. JSHint is a derivative project of JSLint. Sorry to burst your bubble. To be fair, it's been undergoing *major* revisions lately, making it resemble JSHint less and less in terms of what it checks for. Having used it in the past, functionality wise it's very different. While it maintains some backwards compatibility, it has added in new checks, doesn't complain about nearly the number of things that JSLint complains about (for good reasons). -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting
On 09/05/2014 12:36 PM, Tim Bell wrote: How can the average deployer know whether a stackforge is a. An early prototype which has completed (such as some of the early LBaaS packages) b. A project which has lost its initial steam and further investment is not foreseen c. A reliable project where there has not been a significant need to change recently but is a good long term bet This pops up often and to me it looks like a software procurement issue, something that the 'buyer' needs to be able to sort out either with the help of their vendors (distributions or system integrators), or if they go DIY, with other instruments to check code quality, level of support by the community, professional support etc (there are a few EU funded researches and tools, like http://qualipso.icmc.usp.br/OMM/). Why do you think any open source project should be in the business of providing such assurances? Isn't that a role more suitable for the commercial ecosystem? /stef -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][neutron][cinder] Averting the Nova crisis by splitting out virt drivers
On 09/10/2014 02:27 AM, Sylvain Bauza wrote: Well, both proposals can be done : we can create subteams and the Subteam-Approval Gerrit label right know before Kilo, and we could split the virt repos by later once the interfaces and prereqs are done. That's what I mean in fact: create sub team is fairly easy to do, we can start very soon. Splitting the repos will require the cleanup in internal interfaces, documentation and the other things that (while important and needed anyway) will require probably one cycle, if not more. On 09/10/2014 09:07 AM, Kyle Mestery wrote: I would be cautious around sub-teams. Our experience in Neutron has been that we do a very good job of setting up sub-teams, but a terrible job at deciding when they should be spun-down and folded back in. Because in a lot of cases, a sub-team's existance should be for a short period of time. The other problem is that sub-teams can tend to wander away from the broader team, making it harder for their work to be integrated back into the whole. All of this is to say that sub-teams require coordination and lots of communication, and should be carefully watched, groomed, and culled when necessary. This is great feedback. Maybe picking the leaders of the sub-teams based on their communication skills and their understanding of the bigger picture would help? How would you do things differently, based on your experience? /stef -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build
What you are finding is the same as I found, which raised my concern. Thanks for the pointer to legal-disc...@lists.openstack.org, I will post the question there (let the lawyers figure it out). On 9/10/2014 12:16 PM, Solly Ross wrote: - Original Message - From: Jeremy Stanley fu...@yuggoth.org To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, September 10, 2014 1:10:18 PM Subject: Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build On 2014-09-10 13:00:29 -0400 (-0400), Solly Ross wrote: JSHint *isn't* Douglas Crockford. It was written by someone who (understandably) thought Douglas Crockford had some good ideas, but was overzealous. [...] Overzealous enough to copy his code. ?? This sentence doesn't make much sense. I meant to say that Douglas Crockford was overzealous (which he is, IMO). The license is as such: https://github.com/jshint/jshint/blob/master/LICENSE Ahem. https://github.com/jshint/jshint/blob/master/src/jshint.js#L19 Fair enough. I stand corrected. I didn't catch that. The general license, however, is as stated. You are thinking of JSLint, which is written by Douglas Crockford. JSHint is a derivative project of JSLint. Sorry to burst your bubble. To be fair, it's been undergoing *major* revisions lately, making it resemble JSHint less and less in terms of what it checks for. Having used it in the past, functionality wise it's very different. While it maintains some backwards compatibility, it has added in new checks, doesn't complain about nearly the number of things that JSLint complains about (for good reasons). -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting
-Original Message- From: Stefano Maffulli [mailto:stef...@openstack.org] Sent: 10 September 2014 19:29 To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting On 09/05/2014 12:36 PM, Tim Bell wrote: How can the average deployer know whether a stackforge is a. An early prototype which has completed (such as some of the early LBaaS packages) b. A project which has lost its initial steam and further investment is not foreseen c. A reliable project where there has not been a significant need to change recently but is a good long term bet This pops up often and to me it looks like a software procurement issue, something that the 'buyer' needs to be able to sort out either with the help of their vendors (distributions or system integrators), or if they go DIY, with other instruments to check code quality, level of support by the community, professional support etc (there are a few EU funded researches and tools, like http://qualipso.icmc.usp.br/OMM/). Why do you think any open source project should be in the business of providing such assurances? Isn't that a role more suitable for the commercial ecosystem? There is an impression that open source software does not have procurement issues. Where we choose to invest manpower in testing, documentation, bug reporting, scaling is as much as commitment as those who choose to buy software. CERN does not have resources on its own to keep a project alive. Combined with others, I'm OK to assist. If there is significant doubt, I will not invest at all, no additional function for my users and the overall community loses. Tim /stef -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] BM provisioning through Horizon/CLI?
Hello OpenStack Team, I have a use case where I want to host/manage a mix environment with VM's and baremetal servers through Overcloud (Horizon/CLI). To be specific, I am looking for an ability to create a new server on baremetal machine (instead of Vm) through Horizon and manage the instance life-cycle through Overcloud agents. I am wondering can we do that with current Juno release? Any pointers on this implementation or proposed/prospective Blueprints would be really helpful. Thanks, Shree Bageshree Lokare Cloud Architect HP ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Kilo Cycle Goals Exercise
On Wed, Sep 10, 2014 at 7:13 AM, Thierry Carrez thie...@openstack.org wrote: Joe Gordon wrote: To that end, I would like to propose an exercise as discussed in the TC meeting yesterday [1]: Have anyone interested (especially TC members) come up with a list of what they think the project wide Kilo cycle goals should be and post them on this thread by end of day Wednesday, September 10th. After which time we can begin discussing the results. The goal of this exercise is to help us see if our individual world views align with the greater community, and to get the ball rolling on a larger discussion of where as a project we should be focusing more time. Thanks again to Joe for starting this important discussion. +100 I was going to propose my own list, but to be honest, Thierry sums it all up so eloquently below that I'd be hard pressed to come up with a different list. My personal list of Kilo goals goes as follows: #1: Fix our growth pains We grew a lot, and our recipes were designed for a smaller group where trust happens more naturally. With our community growing to a Dunbar order of magnitude above, some of those recipes don't work so great, so we need to revisit them... That includes the binary integrated release (introduce layers?), cross-project functional testing (push it at project level?), the programs concept, encouraging PTL delegation (the czar/liaisons proposal ?), scaling core reviewing in larger projects (Nova driver split ?), etc. We already started the conversation on those important topics, but Kilo is the timeframe for us to implement those changes, because I don't see our community wait for more than one cycle to see the light at the end of the tunnel. #2: Fix the end user experience Monty expressed it better than I can. We need consistent and better-designed APIs, client SDKs that provide useful primitives and actually abstract differences in implementations, etc. #3: Fix what we have: bugfixes, consistency, scaling up, scaling down, upgrading Rather than adding more features for the mid-size private cloud, let's make sure that what we have works well, provides a consistent experience across projects, scales up beautifully, can be easily used at smaller scale as well (simplicity), and allows seamless upgrades. This is another way of looking at paying our technical debt that others have mentioned as goals -- generally polishing what we have rather than building out new things. I agree with all of your goals here Thierry, and as I stated above, my list matches these one for one. I think we have to continue fixing the growing pains which are happening at a micro-level in each project, and at a macro-level overall. There are lots of solid ideas around this for sure, we just need to execute on the ones which we think will have the most benefit. To me, this is the biggest issue we have and the one we should tackle first. Thanks, Kyle Regards, -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] On an API proxy from baremetal to ironic
I don't get that logic at all. Proxying the read commands means that monitoring or report scripts will work fine. Actual state transition scripts, which can't really work correctly, as there are too many differences, won't. -Sean On 09/10/2014 12:57 PM, Solly Ross wrote: As far as I understand it, though, that's a patch for a read-only mode. It seems bizzare, and possibly dangerous, to proxy read commands, but not write commands. It gives the impression that everything's fine until it's not fine (because someone tried to use an existing script to do a create command). IMHO, it would be better to just tell people up front Update your scripts to use Ironic, because they won't work at all instead of leading people (through empirical evidence) to believe that their scripts will work, and then having them discover later that something broke because they tried to create a node. Best Regards, Solly Ross - Original Message - From: Sean Dague s...@dague.net To: openstack-dev@lists.openstack.org Sent: Wednesday, September 10, 2014 10:33:05 AM Subject: Re: [openstack-dev] On an API proxy from baremetal to ironic On 09/09/2014 11:22 PM, Russell Bryant wrote: On 09/09/2014 05:24 PM, Michael Still wrote: Hi. One of the last things blocking Ironic from graduating is deciding whether or not we need a Nova API proxy for the old baremetal extension to new fangled Ironic API. The TC has asked that we discuss whether we think this functionality is actually necessary. It should be noted that we're _not_ talking about migration of deployed instances from baremetal to Ironic. That is already implemented. What we are talking about is if users post-migration should be able to expect their previous baremetal Nova API extension to continue to function, or if they should use the Ironic APIs from that point onwards. Nova had previously thought this was required, but it hasn't made it in time for Juno unless we do a FFE, and it has been suggested that perhaps its not needed at all because it is an admin extension. To be super specific, we're talking about the baremetal nodes admin extension here. This extension has the ability to: - list nodes running baremetal - show detail of one of those nodes - create a new baremetal node - delete a baremetal node Only the first two of those would be supported if we implemented a proxy. So, discuss. I'm in favor of proceeding with deprecation without requiring the API proxy. In the case of user facing APIs, the administrators in charge of upgrading the cloud do not have full control over all of the apps using the APIs. In this particular case, I would expect that the cloud administrators have *complete* control over the use of these APIs. Assuming we have one overlap release (Juno) to allow the migration to occur and given proper documentation of the migration plan and release notes stating the fact that the old APIs are going away, we should be fine. In summary, +1 to moving forward without the API proxy requirement. The thing is, we have the patch - https://review.openstack.org/#/c/120433/, so it's not like there is a zomg run around to get the patch. I think we should FFE this patch as it provides a smoother transition from baremetal to ironic. The patch is extremely low risk to the rest of Nova, as it's a surface proxy feature, so lets just land it and move forward. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [zaqar] Juno Performance Testing (Round 2)
Thanks! Looks good. Only thing I noticed was that footnotes were still referenced, but did not appear at the bottom of the page. On 9/10/14, 6:16 AM, Flavio Percoco fla...@redhat.com wrote: I've collected the information from both performance tests and put it in the project's wiki[0] Please, double check :D ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Bringing back auto-abandon (was: Review metrics - what do we want to measure?)
On Wed, Sep 10, 2014 at 04:37:14PM +, Jeremy Stanley wrote: On 2014-09-10 09:57:17 +0100 (+0100), Steven Hardy wrote: I can understand this argument, and perhaps an auto-abandon with a long period like say a month for the submitter to address comments and reset the clock would be a reasonable compromise? Perhaps, but the old script lacked a bunch of things we'd need for this (it didn't understand/skip WIP changes, it also abandoned changes with no reviews at all, it didn't support for the version of Gerrit we're running now) and was tied to old cruft we've excised (particularly the Launchpad-Gerrit group membership sync code) so would likely need to be rewritten mostly from scratch. Probably if we were to do it right, it would be something like a new kind of periodic meta-job job in Zuul which could be selectively added to projects who want it. Ok, so lets collect requirements for a smarter auto-abandon script, and figure out how feasable it is. The problem we have now, is there is a constantly expanding queue of zombie reviews, where the submitter got some negative feedback and, for whatever reason, has chosen not to address it. I agree that's a problem, but I'm unconvinced it's because we lack some system to automatically make them go away. It is though, because the current system puts a never-ending increasing load on a finite and hard-to-grow resource (core reviewers). Clearly documented auto-abandon puts the load on the never-ending increasing group of new contributors. For example, in my Incoming reviews queue on the gerrit dashboard, I've got reviews I (or someone) -1'd over four months ago, with zero feedback from the submitter, what value is there to these reviews cluttering up the dashboard for every reviewer? The moment you, as a core reviewer, abandon one of those with a friendly note it will cease to clutter up the dashboard for anyone. But doing so also gives you an opportunity to possibly notice that this is actually a valuable bug fix/enhancement for your project and take it over. If it gets automatically abandoned because some random reviewer did a drive-by -1 about whitespace preferences a month ago, then you may never know. I think the issue is, in practice, this is not happening, for the following reasons: - There's no friendly way, as a core reviewer, of saying to a contributor that their patch is abandoned. I've never abandoned someone elses patch, because I see it as carrying the same message as -2 reviews - it's a strongly negative message to send to a contributor. - As a core reviewer, typically I have zero visibility of whether a contributor is actually working on the patch, so what's a reasonable time to wait before taking over or abandoning a patch? Not all folks are active on IRC and even if they are, it's more -core cycles spent tracking folks down and chasing them (often repeatedly) for status updates. A clearly communicated patch-expiry system solves both of these issues. To make matters worse Jenkins comes along periodically and rechecks or figures out the patch merge failed and bumps the zombie back up to the top of the queue - obviously I don't realise that it's not a human comment I need to read until I've expended effort clicking on the review and reading it :( If you, again as a core reviewer, notice that there is a change which needs work but perhaps is not in a situation where outright abandonment is warranted, mark it with workflow -1 (work in progress) then that too will break the cycle. Also Zuul is only spontaneously leaving votes on changes which suddenly cease to be able to merge due to merge conflicts appearing in the target branch, and will only do that to a patchset once... it doesn't periodically retest any changes beyond that without some intervention. Maybe the auto-WIP idea mentioned in the original thread is the way forward then, as again I don't see -core review cycles doing WIP-bot duty as particularly worthwhile personally. From a reviewer perspective, it's impossible, and means the review dashboard is basically unusable without complex queries to weed out the active reviews from the zombies. I would argue that the default Gerrit dashboard is effectively unusable for a variety of other reasons anyway, which is why there are custom dashboards proliferating (globally, per-project and per-user) to maximize reviewer efficiency and use cases. For that matter, it's also arguable that's one of the driving factors behind the increasing number of Gerrit clients and front-end replacements. All we need is two things: Well, three... 0. A clearly documented set of expectations of code contributors so that they know in advance their proposed changes will be discarded if they do not look after them fairly continually to address comments as they come. This should include the expectation that they may have to gratuitously submit new patchsets
Re: [openstack-dev] On an API proxy from baremetal to ironic
As far as I understand it, though, that's a patch for a read-only mode. It seems bizzare, and possibly dangerous, to proxy read commands, but not write commands. It gives the impression that everything's fine until it's not fine (because someone tried to use an existing script to do a create command). IMHO, it would be better to just tell people up front Update your scripts to use Ironic, because they won't work at all instead of leading people (through empirical evidence) to believe that their scripts will work, and then having them discover later that something broke because they tried to create a node. How is it dangerous? Most code making write commands would need to be pretty diligent about making sure that the thing being requested actually succeeded. Having the proxy allows us to return a reasonable code for those things (i.e. 403 Forbidden, perhaps) instead of just 500 Huh? What?. I was pro-proxy from the beginning, not because I think proxies are awesome, but because that's what we do when we move things out of Nova's API to other services. Some feel this is a purely admin API and that gives us license to break our own rules here, but I don't really understand where, when and why we draw that line. The code is written, it's minor, and it gives a much more graceful response to the move. We know there are people running this, with clusters in the thousands. We don't know who they all are, what they're doing with it, and we don't know that all of them are happy or expecting to immediately rewrite all of their tools. I don't really see why this is a big deal. --Dan signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][neutron][cinder] Averting the Nova crisis by splitting out virt drivers
On Wed, Sep 10, 2014 at 12:34 PM, Stefano Maffulli stef...@openstack.org wrote: On 09/10/2014 02:27 AM, Sylvain Bauza wrote: Well, both proposals can be done : we can create subteams and the Subteam-Approval Gerrit label right know before Kilo, and we could split the virt repos by later once the interfaces and prereqs are done. That's what I mean in fact: create sub team is fairly easy to do, we can start very soon. Splitting the repos will require the cleanup in internal interfaces, documentation and the other things that (while important and needed anyway) will require probably one cycle, if not more. On 09/10/2014 09:07 AM, Kyle Mestery wrote: I would be cautious around sub-teams. Our experience in Neutron has been that we do a very good job of setting up sub-teams, but a terrible job at deciding when they should be spun-down and folded back in. Because in a lot of cases, a sub-team's existance should be for a short period of time. The other problem is that sub-teams can tend to wander away from the broader team, making it harder for their work to be integrated back into the whole. All of this is to say that sub-teams require coordination and lots of communication, and should be carefully watched, groomed, and culled when necessary. This is great feedback. Maybe picking the leaders of the sub-teams based on their communication skills and their understanding of the bigger picture would help? How would you do things differently, based on your experience? I think that is absolutely the case: sub-team leaders need to be vetted based on their upstream communication skills. I also think what we're looking at in Neutron is giving sub-teams a shelf-life, and spinning them down rather than letting them live long-term, lose focus, and wander aimlessly. Thanks, Kyle /stef -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Kilo Cycle Goals Exercise
On 09/03/2014 11:37 AM, Joe Gordon wrote: As you all know, there has recently been several very active discussions around how to improve assorted aspects of our development process. One idea that was brought up is to come up with a list of cycle goals/project priorities for Kilo [0]. To that end, I would like to propose an exercise as discussed in the TC meeting yesterday [1]: Have anyone interested (especially TC members) come up with a list of what they think the project wide Kilo cycle goals should be and post them on this thread by end of day Wednesday, September 10th. After which time we can begin discussing the results. The goal of this exercise is to help us see if our individual world views align with the greater community, and to get the ball rolling on a larger discussion of where as a project we should be focusing more time. In OpenStack, we have no shortage of interest and enthusiasm on all fronts, including development contributors, deployers, and cloud end users. When looking at project wide-priorities, we need to make sure our tools, processes, and resulting technology facilitate turning all of that interest into real value. We need to identify which areas have the most pain, and work to improve them. A lot of this is certainly about Kilo, but it's longer term, too. It's the way we should always be thinking about this. 1) Dev community We clearly have a lot of growing pains here. What's quite encouraging is that we also have a lot of hard work going into several different proposals to figure out ways to help. The largest projects (Nova and Neutron) are overwhelmed and approaching breaking points. We have to find ways to relieve this pressure. This may involve aggressively pursing project splits or other code review workflow changes. I think the problems and solutions here are project-wide issues, as solutions put in place tend to rapidly spread to the rest of OpenStack. This is an area I'm especially concerned about and eager to help look for solutions. We should evaluate all potential improvements against how well they help us scale our teams and processes to remove bottlenecks to productivity in the dev communtiy. There are several other encouraging proposals related to easing pain in the dev community: - re-working how we do functional testing by making it more project focused - discussions like this one to discuss both priorities, but also how we turn priorities into real action (like the nova projects discussions around using priorities in development) - evolving project leadership (the PTL position) so that we can provide more guidance around delegation in a way that is reasonably consistent across projects - continued discussion about the contents of the integrated release and how we can continue to foster growth without sacrificing quality We are always going to have problems like this, and I hope we continue to think about, discuss, and improve the way we run our projects every release cycle to come. 2) End Users A few others have done a very nice job describing end user experience problems. Monty's description of getting an instance with an IP was painful and embarrassing to read. We've got to figure out ways to provide better focus on these sorts of usability issues. They're obviously not getting the attention they deserve. There have also been lots of good points about improving our API consistency. I totally agree. I'd love to see a group of people step up and emerge as leaders in this area across all projects. I feel like that's something we're sorely missing right now. 3) Deployers OpenStack is still painful to deploy, and even more painful to manage. I'm still quite pleased that we have a deployment program working on this space. I'd actually really like to see how we can facilitate better integration and discussion between TripleO and the rest of the project teams. I'm also very pleased with the progress we've made in Nova towards the initial support for live upgrades. We still have more work to do in Nova, but I'd also like to see more work done in other projects towards the same goal. For both deployers and the dev community, figuring out what went wrong when OpenStack breaks sucks. Others provided some good pointers to several areas we can improve that area (better logs, tooling, ...) and I hope we can make some real progress in this area in the coming cycle. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Zaqar graduation (round 2) [was: Comments on the concerns arose during the TC meeting]
Excerpts from Gordon Sim's message of 2014-09-10 06:18:52 -0700: On 09/10/2014 09:58 AM, Flavio Percoco wrote: To clarify the doubts of what Zaqar is or it's not, let me quote what's written in the project's overview section[0]: Zaqar is a multi-tenant cloud messaging service for web developers. How are different tenants isolated from each other? Can different tenants access the same queue? If so, what does Zaqar do to prevent one tenant from negatively affecting the other? If not, how is communication with other tenants achieved. Most messaging systems allow authorisation to be used to restrict what a particular user can access and quotas to restrict their resource consumption. What does Zaqar do differently? It combines the ideas pioneered by Amazon's SQS product with additional semantics to support event broadcasting. The service features a fully RESTful API, which developers can use to send messages between various components of their SaaS and mobile applications, by using a variety of communication patterns. Underlying this API is an efficient messaging engine designed with scalability and security in mind. Other OpenStack components can integrate with Zaqar to surface events to end users and to communicate with guest agents that run in the over-cloud layer. I may be misunderstanding the last sentence, but I think *direct* integration of other OpenStack services with Zaqar would be a bad idea. Wouldn't this be better done through olso.messaging's notifications in some way? and/or through some standard protocol (and there's more than one to choose from)? It's not direct, nobody is suggesting that. What people are suggesting is that a user would be able to tell Nova to put any messages that would want to deliver in a _user_ focused queue/inbox. This has nothing to do with oslo.messaging. Users don't want many options for backends. They want a simple message passing interface so they don't have to babysit one and choose one. Certainly the undercloud Zaqar API could be based on the existing oslo.messaging notifications. A simple daemon that sits between the oslo notifications firehose and Zaqar's user queues would be quite efficient. However, putting the whole burden of talking directly to a notification bus on the users is unnecessarily complex... especially if they use Java and have no idea what oslo is. Communicating through a specific, fixed messaging system, with its own unique protocol is actually a step backwards in my opinion, especially for things that you want to keep as loosely coupled as possible. This is exactly why various standard protocols emerged. You're thinking like an operator. Think like an application developer. They're asking you how do I subscribe to notifications about _just my instances_ from Nova?, not how do I pump 40,000 messages per second through a message bus that I fully control? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Treating notifications as a contract
On 09/03/2014 11:21 AM, Sandy Walsh wrote:
On 9/3/2014 11:32 AM, Chris Dent wrote:
I took some notes on this a few weeks ago and extracted what seemed
to be the two main threads or ideas the were revealed by the
conversation that happened in this thread:
* At the micro level have versioned schema for notifications such that
one end can declare I am sending version X of notification
foo.bar.Y and the other end can effectively deal.
Yes, that's table-stakes I think. Putting structure around the payload
section.
Beyond type and version we should be able to attach meta information
like public/private visibility and perhaps hints for external mapping
(this trait - that trait in CADF, for example).
CADF doesn't address the underlying problem that Chris mentions above:
that our notification events themselves needs to have a version
associated with them.
Instead of versioning the message payloads themselves, instead CADF
focuses versioning on the CADF spec itself, which is less than useful,
IMO, and a sympton of what I like to call XML-itis.
Where I *do* see some value in CADF is the primitive string codes it
defines for resource classifications, actions, and outcomes (Sections
A.2.5, A.3.5., and A.4.5 respectively in the CADF spec). I see no value
in the long-form XML-itis fully-qualified URI long-forms of the
primitive string codes.
For resource classifications, it defines things like compute,
storage, service, etc, as well as a structured hierarchy for
sub-classifications, like storage/volume or service/block. Actions
are string codes for verbs like create, configure or authenticate.
Outcomes are string codes for success, failure, etc.
What I feel we need is a library that matches a (resource_type, action,
version) tuple to a JSONSchema document that describes the payload for
that combination of resource_type, action, and version.
If I were king for a day, I'd have a standardized notification message
format that simply consisted of:
resource_class (string) -- From CADF, e.g. service/block
occurred_on (timestamp) -- when the event was published
action (string) -- From CADF, e.g. create
version (int or tuple) -- version of the (resource_class, action)
payload (json-encoded string) -- the message itself
outcome (string) -- Still on fence for this, versus just using payload
There would be an Oslo library that would store the codification of the
resource classes and actions, along with the mapping of (resource_class,
action, version) to the JSONSchema document describing the payload field.
Producers of messages would consume the oslo lib like so:
```python
from oslo.notifications import resource_classes
from oslo.notifications import actions
from oslo.notifications import message
from nova.compute import power_states
from nova.compute import task_states
...
msg = message.Message(resource_classes.compute.machine,
actions.update,
version=1)
# msg is now an object that is guarded by the JSONSchema document
# that describes the version 1.0 schema of the UPDATE action
# for the resource class representing a VM (compute.machine)
# This means that if the producer attempts to set an
# attribute of the msg object that is *not* in that JSONSchema
# document, then an AttributeError would be raised. This essentially
# codifies the message's resource_class and action attributes
# (as constants in the oslo.notifications.resource_classes and
# oslo.notifications.actions module) as well as codifies the
# schema of the (resource_class, action, version) combo.
# Assume the JSONSchema document for a
# (resource_class, action, version) of
# (compute.machine, update, 1) looks like this:
# {
#properties: {
# state: {
# type: string,
# description: The new power state of VM
# },
# state: {
# type: string,
# description: The old power state of VM
# },
# task_state: {
# type: string,
# description: The new task state of VM
# },
# old_task_state: {
# type: string,
# description: The old task state of VM
# }
# additionalProperties: false
# }
msg.old_state = power_states.RUNNING
msg.state = power_states.SHUTDOWN
msg.old_taskkk_state = None # --- would blow up with TypeError,
# since taskkk is mispelled.
# Send the message over the wire...
message.send(...)
```
Similarly, on the consumer side, the message for a particular
resource_class, action and version can be constructed from the
oslo.notifications library and the JSONSchema document could be
consulted for payload introspection:
```python
from oslo.notifications import resource_classes
from oslo.notifications import actions
from oslo.notifications
Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers
On Sep 4, 2014, at 3:24 AM, Daniel P. Berrange berra...@redhat.com wrote: Position statement == Over the past year I've increasingly come to the conclusion that Nova is heading for (or probably already at) a major crisis. If steps are not taken to avert this, the project is likely to loose a non-trivial amount of talent, both regular code contributors and core team members. That includes myself. This is not good for Nova's long term health and so should be of concern to anyone involved in Nova and OpenStack. For those who don't want to read the whole mail, the executive summary is that the nova-core team is an unfixable bottleneck in our development process with our current project structure. The only way I see to remove the bottleneck is to split the virt drivers out of tree and let them all have their own core teams in their area of code, leaving current nova core to focus on all the common code outside the virt driver impls. I, now, none the less urge people to read the whole mail. I am highly in favor of this approach (and have been for at least a year). Every time we have brought this up in the past there has been concern about the shared code, but we have to make a change. We have tried various other approaches and none of them have made a dent. +1000 Vish Background information == I see many factors coming together to form the crisis - Burn out of core team members from over work - Difficulty bringing new talent into the core team - Long delay in getting code reviewed merged - Marginalization of code areas which aren't popular - Increasing size of nova code through new drivers - Exclusion of developers without corporate backing Each item on their own may not seem too bad, but combined they add up to a big problem. Core team burn out -- Having been involved in Nova for several dev cycles now, it is clear that the backlog of code up for review never goes away. Even intensive code review efforts at various points in the dev cycle makes only a small impact on the backlog. This has a pretty significant impact on core team members, as their work is never done. At best, the dial is sometimes set to 10, instead of 11. Many people, myself included, have built tools to help deal with the reviews in a more efficient manner than plain gerrit allows for. These certainly help, but they can't ever solve the problem on their own - just make it slightly more bearable. And this is not even considering that core team members might have useful contributions to make in ways beyond just code review. Ultimately the workload is just too high to sustain the levels of review required, so core team members will eventually burn out (as they have done many times already). Even if one person attempts to take the initiative to heavily invest in review of certain features it is often to no avail. Unless a second dedicated core reviewer can be found to 'tag team' it is hard for one person to make a difference. The end result is that a patch is +2d and then sits idle for weeks or more until a merge conflict requires it to be reposted at which point even that one +2 is lost. This is a pretty demotivating outcome for both reviewers the patch contributor. New core team talent It can't escape attention that the Nova core team does not grow in size very often. When Nova was younger and its code base was smaller, it was easier for contributors to get onto core because the base level of knowledge required was that much smaller. To get onto core today requires a major investment in learning Nova over a year or more. Even people who potentially have the latent skills may not have the time available to invest in learning the entire of Nova. With the number of reviews proposed to Nova, the core team should probably be at least double its current size[1]. There is plenty of expertize in the project as a whole but it is typically focused into specific areas of the codebase. There is nowhere we can find 20 more people with broad knowledge of the codebase who could be promoted even over the next year, let alone today. This is ignoring that many existing members of core are relatively inactive due to burnout and so need replacing. That means we really need another 25-30 people for core. That's not going to happen. Code review delays -- The obvious result of having too much work for too few reviewers is that code contributors face major delays in getting their work reviewed and merged. From personal experience, during Juno, I've probably spent 1 week in aggregate on actual code development vs 8 weeks on waiting on code review. You have to constantly be on alert for review comments because unless you can respond quickly (and repost) while you still have the attention of the reviewer, they may not be look again for days/weeks. The
Re: [openstack-dev] On an API proxy from baremetal to ironic
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/10/2014 01:13 PM, Dan Smith wrote: As far as I understand it, though, that's a patch for a read-only mode. It seems bizzare, and possibly dangerous, to proxy read commands, but not write commands. It gives the impression that everything's fine until it's not fine (because someone tried to use an existing script to do a create command). IMHO, it would be better to just tell people up front Update your scripts to use Ironic, because they won't work at all instead of leading people (through empirical evidence) to believe that their scripts will work, and then having them discover later that something broke because they tried to create a node. How is it dangerous? Most code making write commands would need to be pretty diligent about making sure that the thing being requested actually succeeded. Having the proxy allows us to return a reasonable code for those things (i.e. 403 Forbidden, perhaps) instead of just 500 Huh? What?. I was pro-proxy from the beginning, not because I think proxies are awesome, but because that's what we do when we move things out of Nova's API to other services. Some feel this is a purely admin API and that gives us license to break our own rules here, but I don't really understand where, when and why we draw that line. The code is written, it's minor, and it gives a much more graceful response to the move. We know there are people running this, with clusters in the thousands. We don't know who they all are, what they're doing with it, and we don't know that all of them are happy or expecting to immediately rewrite all of their tools. I don't really see why this is a big deal. I wasn't aware that this was already written when I replied originally, and that fact does reduce my opposition somewhat. I still have issues though: 1) Is this tested anywhere? There are no unit tests in the patch and it's not clear to me that there would be any Tempest coverage of this code path. Providing this and having it break a couple of months down the line seems worse than not providing it at all. This is obviously fixable though. 2) If we think maintaining compatibility for existing users is that important, why aren't we proxying everything? Is it too difficult/impossible due to the differences between Baremetal and Ironic? And if they're that different, does it still make sense to allow one to look like the other? As it stands, this isn't going to let deployers use their existing tools without modification anyway. - -Ben --Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJUEKMKAAoJEDehGd0Fy7uqzNsH+gPv45+IKWJEVeQK4/bd5KAa SjUch6kmTaWkHxMCcIrm3E9bug0zU/64jhtlJuLWfDohqK4I3NRnaY7Ur5R6aEx7 QsXa74LS+cl2n8ydcVAGtmTYzRyLfF+Qvocu8pUZ3ZP+d4A73lkX09B3s07hwY02 e87blL9E6/T9/4ni+186RDrMEnD8TIY3oD4cnbAgib9tVNBMitqlGuFGqdp7gRDW q0GMzh2bmbQRTE2OpEtSbjsVm+qeVsbVACg+bsM/y62ZT3TTO5NyqbYZPJJfDDy4 ys4rbJT6fDZLz2L5G835jAHMwUc54vLdXAz/blo/TsI1LCiJTHvIaWLdgsMSqAY= =vcMK -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers
On Sep 4, 2014, at 8:33 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Sep 04, 2014 at 01:36:04PM +, Gary Kotton wrote: Hi, I do not think that Nova is in a death spiral. I just think that the current way of working at the moment is strangling the project. I do not understand why we need to split drivers out of the core project. Why not have the ability to provide Œcore review¹ status to people for reviewing those parts of the code? We have enough talented people in OpenStack to be able to write a driver above gerrit to enable that. The consensus view at the summit was that, having tried failed at getting useful changes into gerrit, it is not a viable option unless we undertake a permanent fork of the code base. There didn't seem to be any apetite for maintaining developing a large java app ourselves. So people we're looking to start writing a replacement for gerrit from scratch (albeit reusing the database schema). I don’t think this is a viable option for us, but if we were going to do it, we would probably be better off using https://code.google.com/p/rietveld/ as a base, since it is actually written in python. Vish signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] On an API proxy from baremetal to ironic
1) Is this tested anywhere? There are no unit tests in the patch and it's not clear to me that there would be any Tempest coverage of this code path. Providing this and having it break a couple of months down the line seems worse than not providing it at all. This is obviously fixable though. AFAIK, baremetal doesn't have any tempest-level testing at all anyway. However, I don't think our proxy code breaks, like, ever. I expect that unit tests for this stuff is plenty sufficient. 2) If we think maintaining compatibility for existing users is that important, why aren't we proxying everything? Is it too difficult/impossible due to the differences between Baremetal and Ironic? And if they're that different, does it still make sense to allow one to look like the other? As it stands, this isn't going to let deployers use their existing tools without modification anyway. Ideally we'd proxy everything, based on our current API guarantees. However, I think the compromise of just the show/index stuff came about because it would be extremely easy to do, provide some measure of continuity, and provide us a way to return something nicer for the create/update operations than a 500. It seemed like a completely fair and practical balance. --Dan signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] On an API proxy from baremetal to ironic
On 09/10/2014 03:14 PM, Ben Nemec wrote: On 09/10/2014 01:13 PM, Dan Smith wrote: As far as I understand it, though, that's a patch for a read-only mode. It seems bizzare, and possibly dangerous, to proxy read commands, but not write commands. It gives the impression that everything's fine until it's not fine (because someone tried to use an existing script to do a create command). IMHO, it would be better to just tell people up front Update your scripts to use Ironic, because they won't work at all instead of leading people (through empirical evidence) to believe that their scripts will work, and then having them discover later that something broke because they tried to create a node. How is it dangerous? Most code making write commands would need to be pretty diligent about making sure that the thing being requested actually succeeded. Having the proxy allows us to return a reasonable code for those things (i.e. 403 Forbidden, perhaps) instead of just 500 Huh? What?. I was pro-proxy from the beginning, not because I think proxies are awesome, but because that's what we do when we move things out of Nova's API to other services. Some feel this is a purely admin API and that gives us license to break our own rules here, but I don't really understand where, when and why we draw that line. The code is written, it's minor, and it gives a much more graceful response to the move. We know there are people running this, with clusters in the thousands. We don't know who they all are, what they're doing with it, and we don't know that all of them are happy or expecting to immediately rewrite all of their tools. I don't really see why this is a big deal. I wasn't aware that this was already written when I replied originally, and that fact does reduce my opposition somewhat. I still have issues though: 1) Is this tested anywhere? There are no unit tests in the patch and it's not clear to me that there would be any Tempest coverage of this code path. Providing this and having it break a couple of months down the line seems worse than not providing it at all. This is obviously fixable though. 2) If we think maintaining compatibility for existing users is that important, why aren't we proxying everything? Is it too difficult/impossible due to the differences between Baremetal and Ironic? And if they're that different, does it still make sense to allow one to look like the other? As it stands, this isn't going to let deployers use their existing tools without modification anyway. Because the world isn't black and white. The ready only proxy probably covers a bunch of cases, and is easy. The write proxy is much more time, and unclear that it would be semantically equivalent enough to be useful to anyone. So this is an 80/20 rule piece of code. And as it's already done, lets do it. -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers
On Sep 5, 2014, at 4:12 AM, Sean Dague s...@dague.net wrote: On 09/05/2014 06:40 AM, Nikola Đipanov wrote: Just some things to think about with regards to the whole idea, by no means exhaustive. So maybe the better question is: what are the top sources of technical debt in Nova that we need to address? And if we did, everyone would be more sane, and feel less burnt. Maybe the drivers are the worst debt, and jettisoning them makes them someone else's problem, so that helps some. I'm not entirely convinced right now. I think Cells represents a lot of debt right now. It doesn't fully work with the rest of Nova, and produces a ton of extra code paths special cased for the cells path. The Scheduler has a ton of debt as has been pointed out by the efforts in and around Gannt. The focus has been on the split, but realistically I'm with Jay is that we should focus on the debt, and exposing a REST interface in Nova. What about the Nova objects transition? That continues to be slow because it's basically Dan (with a few other helpers from time to time). Would it be helpful if we did an all hands on deck transition of the rest of Nova for K1 and just get it done? Would be nice to have the bulk of Nova core working on one thing like this and actually be in shared context with everyone else for a while. In my mind, spliting helps with all of these things. A lot of the cleanup related work is completely delayed because the review queue starts to seem like an insurmountable hurdle. There are various cleanups needed in the drivers as well but they are not progressing due to the glacier pace we are moving right now. Some examples: Vmware spawn refactor, Hyper-v bug fixes, Libvirt resize/migrate (this is still using ssh to copy data!) People need smaller areas of work. And they need a sense of pride and ownership of the things that they work on. In my mind that is the best way to ensure success. Vish signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
