Re: [openstack-dev] [glance] tasks (following "proposed priorities for Mitaka")
On 09/14/2015 02:41 PM, Monty Taylor wrote: On 09/14/2015 04:58 PM, Brian Rosmaita wrote: I'm not saying that there's no other way to do this -- e.g., you could do all sorts of alternative workflows and configurations in the "regular" upload process -- but the feedback I got can be summarized like this: Given the importance of a properly-functioning Glance for normal cloud operations, it is useful to have one upload/download workflow that is locked down and you don't have to worry about, and a completely different workflow that you can expose to end users and tinker with as necessary. IMHO - a cloud that does not allow me to upload images is not a usable cloud. A cloud that requires me to upload images differently than another cloud is a hardship on the users. A cloud that makes the user know the image format of the cloud is a hardship on the users, especially when there exist nowhere in any existing distro tools that can actually produce the image format in question. (yup, Im just going to sneak that one in there) NOW - I think that the task api and the image conversion tools itself if it's a behind the scenes kind of thing is potentially nice thing. If "glance import-from http://example.com/my-image.qcow2' always worked, and in the back end generated a task with the task workflow, and one of the task workflows that a deployer could implement was one to do conversions to the image format of the cloud provider's choice, that would be teh-awesome. It's still a bit annoying to me that I, as a user, need to come up with a place to put the image so that it can be imported, but honestly, I'll take it. It's not _that_ hard of a problem. I predicted the above problems with the tasks API as it was designed way back in 2013 and urged it be redesigned: http://lists.openstack.org/pipermail/openstack-dev/2013-May/009400.html http://lists.openstack.org/pipermail/openstack-dev/2013-May/009527.html with a followup here: http://lists.openstack.org/pipermail/openstack-dev/2013-November/019028.html where I actually agreed with George Reese about something... Guess I should have been more vocal. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] tasks (following "proposed priorities for Mitaka")
On 14/09/15 16:09 -0400, Doug Hellmann wrote: Excerpts from Monty Taylor's message of 2015-09-14 20:41:38 +0200: On 09/14/2015 04:58 PM, Brian Rosmaita wrote: [snip] If "glance import-from http://example.com/my-image.qcow2' always worked, and in the back end generated a task with the task workflow, and one of the task workflows that a deployer could implement was one to do conversions to the image format of the cloud provider's choice, that would be teh-awesome. It's still a bit annoying to me that I, as a user, need to come up with a place to put the image so that it can be imported, but honestly, I'll take it. It's not _that_ hard of a problem. This is more or less what I'm thinking we want, too. As a user, I want to know how to import an image by having that documented clearly and by using an obvious UI. As a deployer, I want to sometimes do things to an image as they are imported, and background tasks may make that easier to implement. As a user, I don't care if my image upload is a task or not. IMHO, as much as it's not a _hard_ problem to solve, as I user I'd hate to be asked to upload my image somewhere to create an image in my cloud account. Not all users create scripts and not all users have the same resources. Simplest scenario. I'm a new user and I want to test your cloud. I want to know how it works and I want to run *my* distro which is not available in the list of public images. If I were to be asked to upload the image somewhere to then use it, I'd be really sad and, at the very least, I'd expect this cloud to provide *the* place where I should put the image, which is not always the case. Cheers, Flavio -- @flaper87 Flavio Percoco pgpX7yWTEO7lx.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [glance] tasks (following "proposed priorities for Mitaka")
Apologies for forking the thread, but there was way too much in Doug's email (and Flavio's response) and I only want to make a few points about tasks. Please read Doug's original email and Flavio's reply at some point, preferably before you read this. I'm going to limit myself to 4 points. We'll be discussing Glance tasks during the Mitaka summit design session, so we'll be able to go into details and determine the future of tasks there. But I would like to make these points before discussion gets too far along. (1) DefCore So I see DefCore as a two-way street, in which the OpenStack projects need to be aware of what's going on with the DefCore process, and the DefCore people are paying attention to what's going on in the projects. Glance tasks are not a recent innovation, they date back at least to the Havana summit, April 15-18, 2013. There was a session on "Getting Glance Ready for Public Clouds" [1], resulting in a blueprint for "New Upload Download Workflow for Public Glance" [2], which was filed on 2013-04-22. This was pre-specs days, but there was lots of information about the design direction this was taking posted on the wiki (see [3] and [4], which contain links to most of the stuff). My point is simply that the need for tasks and the discussion around their development and structure was carried out in the open via the standard OpenStack practices, and if Glance was headed in a weird/nonstandard/deviant direction, some guidance would have been useful at that point. (I'm not implying that such guidance is not useful now, of course.) (2) Tasks as a Public API Well, that has been the whole point throughout the entire discussion. See [1]-[4]. (3) Tasks and Deployers I participated in some of the DefCore discussions around image upload that took place before the Liberty summit. It just so happened that I was on the program to give a talk about Glance tasks, and I left room for discussion about (a) whether two image upload workflows are confusing for end users, and (b) whether the flexibility of tasks (e.g., the "input" element defined as a JSON blob) is actually a problem. (You can look at the talk [5] or my slides [6] to see that I didn't pull any punches about this.) The feedback I got from the deployers present was that they weren't worried about (a), and that they liked (b) because it enabled them to make customizations easily for their particular situation. I'm not saying that there's no other way to do this -- e.g., you could do all sorts of alternative workflows and configurations in the "regular" upload process -- but the feedback I got can be summarized like this: Given the importance of a properly-functioning Glance for normal cloud operations, it is useful to have one upload/download workflow that is locked down and you don't have to worry about, and a completely different workflow that you can expose to end users and tinker with as necessary. (4) Interoperability In general, this is a worthy goal. The OpenStack cloud platform, however, is designed to handle many different deployment scenarios from small private clouds to enormous public clouds, and allowing access to the same API calls in all situations is not desirable. A small academic department, for example, may allow regular end users to make some calls usually reserved for admins, whereas in a public cloud, this would be a remarkably bad idea. So if DefCore is going to enforce interoperability via tests, it should revise the tests to meet the most restrictive reasonable case. Image upload is a good example, as some cloud operators do not want to expose this operation to end users, period, and for a myriad of reasons (security, user frustration when the image from some large non-open-source cloud doesn't boot, etc.). With respect to tasks: the cloud provider specifies the exact content of the 'input' element. It's going to differ from deployment to deployment. But that isn't significantly different from different clouds having different flavors with different capabilities. You can't reasonably expect that "nova boot --flavor economy-flavor --image optimized-centos myserver" is going to work in all OpenStack clouds, i.e., you need to figure out the appropriate values to replace 'economy-flavor' and 'optimized-centos' in the boot call. I think the 'input' element is similar. The initial discussion was that it should be defined via documentation as we saw how tasks would be used in real life. But there's no reason why it must be documentation only. It would be easy to make a schema available. I tried to be as concise as possible, but now my email has gotten too long! cheers, brian [1] https://etherpad.openstack.org/p/havana-getting-glance-ready-for-public-clo uds [2] https://blueprints.launchpad.net/glance/+spec/upload-download-workflow [3] https://wiki.openstack.org/wiki/Glance-tasks-api [4] https://wiki.openstack.org/wiki/Glance-tasks-api-product [5] http://youtu.be/ROXrjX3pdqw [6]
Re: [openstack-dev] [glance] tasks (following "proposed priorities for Mitaka")
Excerpts from Monty Taylor's message of 2015-09-14 20:41:38 +0200: > On 09/14/2015 04:58 PM, Brian Rosmaita wrote: > > Apologies for forking the thread, but there was way too much in Doug's > > email (and Flavio's response) and I only want to make a few points about > > tasks. Please read Doug's original email and Flavio's reply at some > > point, preferably before you read this. > > > > I'm going to limit myself to 4 points. We'll be discussing Glance tasks > > during the Mitaka summit design session, so we'll be able to go into > > details and determine the future of tasks there. But I would like to make > > these points before discussion gets too far along. Part of the point of me starting this discussion now is to influence the summit sessions the Glance team has, and the goals of those sessions. > > > > > > (1) DefCore > > So I see DefCore as a two-way street, in which the OpenStack projects need > > to be aware of what's going on with the DefCore process, and the DefCore > > people are paying attention to what's going on in the projects. > > > > Glance tasks are not a recent innovation, they date back at least to the > > Havana summit, April 15-18, 2013. There was a session on "Getting Glance > > Ready for Public Clouds" [1], resulting in a blueprint for "New Upload > > Download Workflow for Public Glance" [2], which was filed on 2013-04-22. > > > > This was pre-specs days, but there was lots of information about the > > design direction this was taking posted on the wiki (see [3] and [4], > > which contain links to most of the stuff). > > > > My point is simply that the need for tasks and the discussion around their > > development and structure was carried out in the open via the standard > > OpenStack practices, and if Glance was headed in a > > weird/nonstandard/deviant direction, some guidance would have been useful > > at that point. (I'm not implying that such guidance is not useful now, of > > course.) > > I'm very sorry that I did not participate in that. I apologize heartily. Indeed, as I've said elsewhere I am going to work this cycle on improving the two-way conversations with the DefCore committee and the contributor community. DefCore has been very good at listening to technical input when we make it available, and I think all of our project teams need to engage more with them now that their processes are settling down to something we can work with regularly. > > honestly, I do not think we realized how broken this system was until > Infra started trying to move to creating our base images once and > uploading them to each of our cloud regions, which then resulted in me > needing to write an entire compatibility library because the differences > in approaches were so radically and intractably different between the two. > > It's possible that humans could have picked up on the pain this would > cause for the user if more of us had been in the sessions - but it's > also possible we would have missed it. I TOTALLY understand the logic > and reasoning - sometimes things that make perfect sense on paper just > flat fall over when they see the light of day. That's not a condemnation > of the people who worked on it - we all try things, and sometimes they > work and sometimes they do not. > > This did not work, and it's time we all acknowledge that and make plans > to move forward. > > > > > (2) Tasks as a Public API > > Well, that has been the whole point throughout the entire discussion. See > > [1]-[4]. > > > > > > (3) Tasks and Deployers > > I participated in some of the DefCore discussions around image upload that > > took place before the Liberty summit. It just so happened that I was on > > the program to give a talk about Glance tasks, and I left room for > > discussion about (a) whether two image upload workflows are confusing for > > end users, and (b) whether the flexibility of tasks (e.g., the "input" > > element defined as a JSON blob) is actually a problem. (You can look at > > the talk [5] or my slides [6] to see that I didn't pull any punches about > > this.) > > > > The feedback I got from the deployers present was that they weren't > > worried about (a), and that they liked (b) because it enabled them to make > > customizations easily for their particular situation. > > Sure. But it's not the deployers that I care about on this point. I'm > sure it's awesome for them. > > As a person _using_ this I can tell you it is utterly living hell. It > is, without a doubt, and with no close rivals, the hardest thing to > interact with in all of OpenStack. > > > I'm not saying that there's no other way to do this -- e.g., you could do > > all sorts of alternative workflows and configurations in the "regular" > > upload process -- but the feedback I got can be summarized like this: > > Given the importance of a properly-functioning Glance for normal cloud > > operations, it is useful to have one upload/download workflow that is > > locked down and you
Re: [openstack-dev] [glance] tasks (following "proposed priorities for Mitaka")
On 09/14/2015 04:58 PM, Brian Rosmaita wrote: Apologies for forking the thread, but there was way too much in Doug's email (and Flavio's response) and I only want to make a few points about tasks. Please read Doug's original email and Flavio's reply at some point, preferably before you read this. I'm going to limit myself to 4 points. We'll be discussing Glance tasks during the Mitaka summit design session, so we'll be able to go into details and determine the future of tasks there. But I would like to make these points before discussion gets too far along. (1) DefCore So I see DefCore as a two-way street, in which the OpenStack projects need to be aware of what's going on with the DefCore process, and the DefCore people are paying attention to what's going on in the projects. Glance tasks are not a recent innovation, they date back at least to the Havana summit, April 15-18, 2013. There was a session on "Getting Glance Ready for Public Clouds" [1], resulting in a blueprint for "New Upload Download Workflow for Public Glance" [2], which was filed on 2013-04-22. This was pre-specs days, but there was lots of information about the design direction this was taking posted on the wiki (see [3] and [4], which contain links to most of the stuff). My point is simply that the need for tasks and the discussion around their development and structure was carried out in the open via the standard OpenStack practices, and if Glance was headed in a weird/nonstandard/deviant direction, some guidance would have been useful at that point. (I'm not implying that such guidance is not useful now, of course.) I'm very sorry that I did not participate in that. I apologize heartily. honestly, I do not think we realized how broken this system was until Infra started trying to move to creating our base images once and uploading them to each of our cloud regions, which then resulted in me needing to write an entire compatibility library because the differences in approaches were so radically and intractably different between the two. It's possible that humans could have picked up on the pain this would cause for the user if more of us had been in the sessions - but it's also possible we would have missed it. I TOTALLY understand the logic and reasoning - sometimes things that make perfect sense on paper just flat fall over when they see the light of day. That's not a condemnation of the people who worked on it - we all try things, and sometimes they work and sometimes they do not. This did not work, and it's time we all acknowledge that and make plans to move forward. (2) Tasks as a Public API Well, that has been the whole point throughout the entire discussion. See [1]-[4]. (3) Tasks and Deployers I participated in some of the DefCore discussions around image upload that took place before the Liberty summit. It just so happened that I was on the program to give a talk about Glance tasks, and I left room for discussion about (a) whether two image upload workflows are confusing for end users, and (b) whether the flexibility of tasks (e.g., the "input" element defined as a JSON blob) is actually a problem. (You can look at the talk [5] or my slides [6] to see that I didn't pull any punches about this.) The feedback I got from the deployers present was that they weren't worried about (a), and that they liked (b) because it enabled them to make customizations easily for their particular situation. Sure. But it's not the deployers that I care about on this point. I'm sure it's awesome for them. As a person _using_ this I can tell you it is utterly living hell. It is, without a doubt, and with no close rivals, the hardest thing to interact with in all of OpenStack. I'm not saying that there's no other way to do this -- e.g., you could do all sorts of alternative workflows and configurations in the "regular" upload process -- but the feedback I got can be summarized like this: Given the importance of a properly-functioning Glance for normal cloud operations, it is useful to have one upload/download workflow that is locked down and you don't have to worry about, and a completely different workflow that you can expose to end users and tinker with as necessary. IMHO - a cloud that does not allow me to upload images is not a usable cloud. A cloud that requires me to upload images differently than another cloud is a hardship on the users. A cloud that makes the user know the image format of the cloud is a hardship on the users, especially when there exist nowhere in any existing distro tools that can actually produce the image format in question. (yup, Im just going to sneak that one in there) NOW - I think that the task api and the image conversion tools itself if it's a behind the scenes kind of thing is potentially nice thing. If "glance import-from http://example.com/my-image.qcow2' always worked, and in the back end generated a task with the task workflow, and one of the
