Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
On Tue, Jan 14, 2014 at 8:06 PM, Sean Dague s...@dague.net wrote: On 01/14/2014 05:17 AM, Christopher Yeoh wrote: On Mon, Jan 13, 2014 at 10:38 PM, Sean Dague s...@dague.net This skirts the run time issue, but using twice as many resources. It doesn't however address the fact that real effort goes into maintaining and reviewing those clients. Or the review on the Nova side to get these things in. Or the double documentation duties. Or the fact that it inhibits certain things in JSON because they wouldn't be XML friendly. Or the extra code that we need to carry around to handle bad XML insertion. There is a real cost of our dual data payload strategy. A cost in real people that means we aren't doing other things, or doing other things better. The folks working on keeping XML alive aren't doing it because they have a dramatic passion for XML and wouldn't be contributing to the project otherwise, so I think we'd be able to cover far more important parts of the project if we dropped this load. Oh I totally agree with this, was just thinking of alternatives if we can't get consensus as I know this has been suggested before that we should remove the XML support. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. Discoverable is nice to a point but I think we need to be cautious how far we take it as I think at least the positive tests should be written based on the specification and not autogenerated because that will help pick up inconsistencies we have between what we claim our API is versus what it actually is (more automation of documentation will help reduce but I don't think it will eliminate that problem). I don't want discoverability for Tempest (not on the positive side), I 100% believe that we should remain explicit on function testing. I want discoverability for Users and SDKs. I want user toolkits that work with Nova to have a fully discoverable way to work with our API. As I think that would be a *great* API for our users, and would make life dramatically simpler for our SDK ecosystem. +1 And as I mentioned in another thread having a summit cross project session on APIs I think would be very valuable and should cover these sorts of issues. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
On 2014年01月14日 20:04, Ken'ichi Ohmichi wrote: Hi, 2014/1/14 Alex Xu x...@linux.vnet.ibm.com: +1 for drop xml. But if we can't drop it, can we think about use XMLDictSerializer instead of XmlTemplate? We spend a lot of time to maintain XmlTemplate, and it make xml format inconsistent(some of resouce's attribute is output as xml sub-element, some of them is output as xml element's attribute, no rule for them). If we just use XMLDictSerializer for all api, can we just test XMLDictSerializer, not all the api? I like the idea that we use the same serializer for xml. Tempest contains many specific deserializers for each nova APIs, and that makes tempest code complex now. If using the same serializer, I guess we can reduce Tempest code also. In addition, can we use the same deserializer for xml request body? I think we can, I know neutron only use XMLDictSerializer/Deserializer. If doing it, we will be able to generate xml response documents from jsonschema API definitions because of fixing the deserializing rule. Thanks Ken'ichi Ohmichi --- On 2014年01月13日 22:38, Sean Dague wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). -Sean ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
Is the impact of dropping XML understood for the users of the OpenStack APIs ? Tim -Original Message- From: Alex Xu [mailto:x...@linux.vnet.ibm.com] Sent: 15 January 2014 14:33 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely ... ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
On 01/15/2014 08:44 AM, Tim Bell wrote: Is the impact of dropping XML understood for the users of the OpenStack APIs ? That was the rationale for the openstack@ thread on this as well. I believe the impact on users will be more reliable Nova API for users. It will also mean a more clear surface for SDK implementers, which will mean better SDKs for OpenStack (because if someone wrote and SDK against the XML API today it would not work as well as against the JSON API). This is all about a more solid OpenStack for users. It is also part of the reason to introduce this change with a major version bump, so there would be a clear indication that this was now different. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
On Mon, Jan 13, 2014 at 10:38 PM, Sean Dague s...@dague.net wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). Yes if we can establish that no one really uses it (and from the number of bugs we've found in the V2 XML I wouldn't be surprised if large chunks of it aren't used) or the are willing to use the JSON interface instead it'd be worth it just from the reduced complexity in the Nova code. As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. So I think I've mentioned this before, but one additional/alternative would be to simply split the V2 and V3 tests into different jobs in the gate. With most of the V2 tests ported we're planning on adding more tests to the V3 API where there currently isn't any coverage and at least for some of those there may be interest in V2 backports. We should also at some point start doing scenario tests with the V3 rather than V2 API enabled. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. Discoverable is nice to a point but I think we need to be cautious how far we take it as I think at least the positive tests should be written based on the specification and not autogenerated because that will help pick up inconsistencies we have between what we claim our API is versus what it actually is (more automation of documentation will help reduce but I don't think it will eliminate that problem). Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
+1 for drop xml. But if we can't drop it, can we think about use XMLDictSerializer instead of XmlTemplate? We spend a lot of time to maintain XmlTemplate, and it make xml format inconsistent(some of resouce's attribute is output as xml sub-element, some of them is output as xml element's attribute, no rule for them). If we just use XMLDictSerializer for all api, can we just test XMLDictSerializer, not all the api? On 2014年01月13日 22:38, Sean Dague wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). -Sean ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
Hi, 2014/1/14 Alex Xu x...@linux.vnet.ibm.com: +1 for drop xml. But if we can't drop it, can we think about use XMLDictSerializer instead of XmlTemplate? We spend a lot of time to maintain XmlTemplate, and it make xml format inconsistent(some of resouce's attribute is output as xml sub-element, some of them is output as xml element's attribute, no rule for them). If we just use XMLDictSerializer for all api, can we just test XMLDictSerializer, not all the api? I like the idea that we use the same serializer for xml. Tempest contains many specific deserializers for each nova APIs, and that makes tempest code complex now. If using the same serializer, I guess we can reduce Tempest code also. In addition, can we use the same deserializer for xml request body? If doing it, we will be able to generate xml response documents from jsonschema API definitions because of fixing the deserializing rule. Thanks Ken'ichi Ohmichi --- On 2014年01月13日 22:38, Sean Dague wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). -Sean ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
On 01/13/2014 10:38 PM, Sean Dague wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). -Sean Sean, +1 I think this is a very good idea. Also, I'd like to point out that supporting XML adds more attack surface. In other words: we had CVEs because of some nasty XML attacks, and it's always better to remove useless code and complexity whenever possible. Just one thing: is the AWS EC2 API using Json as well? Just my 2 cents, Cheers, Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
On 01/14/2014 10:37 AM, Thomas Goirand wrote: On 01/13/2014 10:38 PM, Sean Dague wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). -Sean Sean, +1 I think this is a very good idea. Also, I'd like to point out that supporting XML adds more attack surface. In other words: we had CVEs because of some nasty XML attacks, and it's always better to remove useless code and complexity whenever possible. Just one thing: is the AWS EC2 API using Json as well? No, unless they do a really good job of hiding it. And Google Compute Engine uses only json. -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
On 01/14/2014 10:50 AM, David Kranz wrote: On 01/14/2014 10:37 AM, Thomas Goirand wrote: On 01/13/2014 10:38 PM, Sean Dague wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). -Sean Sean, +1 I think this is a very good idea. Also, I'd like to point out that supporting XML adds more attack surface. In other words: we had CVEs because of some nasty XML attacks, and it's always better to remove useless code and complexity whenever possible. Just one thing: is the AWS EC2 API using Json as well? No, unless they do a really good job of hiding it. And Google Compute Engine uses only json. This would only be about OS native APIs, i.e. the APIs we control. 3rd party API definitions are beyond our control, and the scope of this. Also note, none of them do the crazy thing of supporting 2 formats. :) -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] [rfc] drop XML from v3 API entirely
I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
+1 to drop XML from v3 API On Mon, Jan 13, 2014 at 9:38 AM, Sean Dague s...@dague.net wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Davanum Srinivas :: http://davanum.wordpress.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
I'm not really an active nova contributor as of yet, but I'll +1 this if nova's XML support is anything like what I see in trove (which I believe just cloned how nova did it in the first place). XML without a schema is terrible for a serialization format. In my experience, the only people who actually use XML really want SOAP or XMLRPC (mostly .net and Java developers), which both give mechanisms for defining the schema of the request/response so data types like arrays and dates and booleans are sane to work with. Doing XML in a generic fashion never adequately deals with those problems and is then rarely, if ever, actually used. Greg On Jan 13, 2014, at 8:38 AM, Sean Dague s...@dague.net wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
On 01/13/2014 09:38 AM, Sean Dague wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). Can you also pose this question on the main openstack@ list? I'd like to cast a wider net on this question. Thanks, -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [rfc] drop XML from v3 API entirely
On 01/13/2014 10:13 AM, Russell Bryant wrote: On 01/13/2014 09:38 AM, Sean Dague wrote: I know we've been here before, but I want to raise this again while there is still time left in icehouse. I would like to propose that the Nova v3 API removes the XML payload entirely. It adds complexity to the Nova code, and it requires duplicating all our test resources, because we need to do everything onces for JSON and once for XML. Even worse, the dual payload strategy that nova employed leaked out to a lot of other projects, so they now think maintaining 2 payloads is a good thing (which I would argue it is not). As we started talking about reducing tempest concurrency in the gate, I was starting to think a lot about what we could shed that would let us keep up a high level of testing, but bring our overall time back down. The fact that Nova provides an extremely wide testing surface makes this challenging. I think it would be a much better situation if the Nova API is a single payload type. The work on the jsonschema validation is also something where I think we could get to a fully discoverable API, which would be huge. If we never ship v3 API with XML as stable, we can deprecate it entirely, and let it die with v2 ( probably a year out ). Can you also pose this question on the main openstack@ list? I'd like to cast a wider net on this question. Thanks, Absolutely. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
