Re: [Openstack] [openstack-dev] [nova] Call for Help -- OpenStack API XML Support

[George Reese]

On Aug 10, 2012, at 2:52 PM, Jay Pipes  wrote:

> On 08/09/2012 11:05 PM, George Reese wrote:
>> On Aug 9, 2012, at 8:14 PM, Doug Davis > > wrote:
>>> Situations like this are always interesting to watch.  :-)
>>> 
>>> On the one hand its open-source, so if you care about something then
>>> put up the resources to make it happen.
>> 
>> This attitude always bothers me. This isn't some Open Source labor of
>> love. It's a commercial collaboration in which many of the contributors
>> have a significant economic interest. 
>> 
>> To be more blunt: if I'm writing code, it's for enStratus.
> 
> Patches always welcome, George. If you can't see that code you may write
> for enStratus might be globally useful, then you're missing the point of
> this open development community.
> 
> And although there are many in this OpenStack community that work for a
> commercial entity and contribute code as such, there are many who don't
> -- and dismissing their contributions as "some Open Source labor of
> love" is degrading and shows the type of opinion you have towards
> anything that doesn't fit nicely in your
> everything-is-a-commercial-agenda worldview.
> 
> If you care about something, then help to fix it.
> 
> 


Either you aren't reading what I am writing or you can't read.

I'm willing to bet I have written more lines of Open Source code over more 
years than you have. So don't start assigning some kind of evil capitalist 
motives on me.

The fact that I don't have code going into enStratus that is not of use to 
OpenStack is not an indication that a) I have some kind of 
"everything-is-a-commercial-agenda" worldview or b) that I don't do Open Source 
or c) I don't think that some Open Source is a labor of love.

OpenStack is a commercial agenda however. Pretending that it's a labor of love 
and people should feel lucky to get functionality is just daft.

-George

--
George Reese - Chief Technology Officer, enStratus
e: george.re...@enstratus.comSkype: nspollutiont: @GeorgeReesep: 
+1.207.956.0217
enStratus: Enterprise Cloud Management - @enStratus - http://www.enstratus.com
To schedule a meeting with me: http://tungle.me/GeorgeReese

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] [nova] Call for Help -- OpenStack API XML Support

[Jay Pipes]

On 08/09/2012 11:05 PM, George Reese wrote:
> On Aug 9, 2012, at 8:14 PM, Doug Davis  > wrote:
>> Situations like this are always interesting to watch.  :-)
>>
>> On the one hand its open-source, so if you care about something then
>> put up the resources to make it happen.
> 
> This attitude always bothers me. This isn't some Open Source labor of
> love. It's a commercial collaboration in which many of the contributors
> have a significant economic interest. 
> 
> To be more blunt: if I'm writing code, it's for enStratus.

Patches always welcome, George. If you can't see that code you may write
for enStratus might be globally useful, then you're missing the point of
this open development community.

And although there are many in this OpenStack community that work for a
commercial entity and contribute code as such, there are many who don't
-- and dismissing their contributions as "some Open Source labor of
love" is degrading and shows the type of opinion you have towards
anything that doesn't fit nicely in your
everything-is-a-commercial-agenda worldview.

If you care about something, then help to fix it.

-jay

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] [nova] Call for Help -- OpenStack API XML Support

[Kevin L. Mitchell]

On Fri, 2012-08-10 at 09:52 -0500, George Reese wrote:
> I personally believe (and this is described in great detail in my API
> book) that you never, ever auto-generate JSON and XML from model
> objects for your REST APIs. You need your models to be free to evolve
> independent from the API representation. 
> 
> If you take that step, you can make your XML and JSON independent be
> first-class citizens. There are other ways as well, but that's my
> recommended approach. 

As a point of information, most (if not all) of the core API uses a
concept of "views" to translate between the model and an intermediate
representation which is then further serialized into either JSON or XML.
Many of the extensions do not use this specific abstraction, but they
often do use something similar.  (The JSON and XML serializer code
probably could not handle having the actual model objects passed to
them…)
-- 
Kevin L. Mitchell 


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] [nova] Call for Help -- OpenStack API XML Support

[George Reese]

This doesn't fit into the "things that make me grumpy" bucket, but instead an 
opinion on how APIs should be done…

I personally believe (and this is described in great detail in my API book) 
that you never, ever auto-generate JSON and XML from model objects for your 
REST APIs. You need your models to be free to evolve independent from the API 
representation.

If you take that step, you can make your XML and JSON independent be 
first-class citizens. There are other ways as well, but that's my recommended 
approach.

It's really not that hard. As I've said in a couple of other emails, the 
enStratus API supports both XML and JSON and maintaining that support without 
auto-generated output that supports schema changes across versions just isn't 
that hard. It's definitely not where we spend our maintenance time on the APIs.

-George

On Aug 10, 2012, at 8:14 AM, Gabe Westmaas  wrote:
>> 
> 
> The biggest problem with this is that either json or XML ends up feeling 
> second class. 


--
George Reese - Chief Technology Officer, enStratus
e: george.re...@enstratus.comSkype: nspollutiont: @GeorgeReesep: 
+1.207.956.0217
enStratus: Enterprise Cloud Management - @enStratus - http://www.enstratus.com
To schedule a meeting with me: http://tungle.me/GeorgeReese

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] [nova] Call for Help -- OpenStack API XML Support

[Vishvananda Ishaya]

On Aug 9, 2012, at 6:28 PM, Daryl Walleck  wrote:

> As part of my work on Tempest, I've created an alternate backend 
> configuration to use XML requests/responses. This right now mostly covers 
> Nova, but could easily be extended to test other projects as well. I hadn't 
> pushed it yet because it seemed to be low priority, but I'd be more than glad 
> to accelerate and get this committed.
> 
> Daryl

That is awesome news Daryl! Yes, please get this up as soon as possible so 
others can help. There definitely seems to be enough interest to at least have 
a go at cleaning up XML support. It would be great to know where the gaps are.

Vish


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] [nova] Call for Help -- OpenStack API XML Support

[George Reese]

On Aug 9, 2012, at 8:14 PM, Doug Davis  wrote:

> 
> Situations like this are always interesting to watch.  :-) 
> 
> On the one hand its open-source, so if you care about something then put up 
> the resources to make it happen. 

This attitude always bothers me. This isn't some Open Source labor of love. 
It's a commercial collaboration in which many of the contributors have a 
significant economic interest. 

To be more blunt: if I'm writing code, it's for enStratus.


> On the other hand, that doesn't mean that as a developer you get to ignore 
> the bigger picture and only do 1/2 of the work because you don't care about 
> the other 1/2. 
> 
> Overall, I tend to agree with the attitude that as long as XML is officially 
> supported then all code changes need to make sure they run through both the 
> JSON and XML codepaths. And if this means twice the testcases then so be it.  
> People committing code shouldn't have a choice in this - its either you do 
> the full job or your code is rejected. 
> 

+10

> Having said that, it is a valid question to ask whether we want to continue 
> to support both JSON and XML going forward.  But, until that decision is 
> formally made letting 1/2 of the APIs atrophy makes the entire community look 
> bad and therefore should not be allowed to happen.   
> 

Actually, it's not a valid question. That ship has sailed. XML is part of the 
spec, and we're talking about Folsom not Bexar. The API is the heart of the 
ecosystem. You never, ever deprecate code unless there's some strong compelling 
reason like a significant security flaw that can be addressed only through 
incompatibility.

> My vote: from now on don't let any code change in unless if works for both.  
> I suspect we'll either see the XML side come up to speed really quickly or 
> it'll force an ugly vote.  But either way, this needs to be resolved before 
> the next release. 
> 
> thanks
> -Doug
> 
> STSM |  Standards Architect  |  IBM Software Group
> (919) 254-6905  |  IBM 444-6905  |  d...@us.ibm.com
> The more I'm around some people, the more I like my dog. 
> 
> 
> George Reese 
> 08/09/2012 07:02 PM
> Please respond to
> OpenStack Development Mailing List 
> 
> To
> OpenStack Development Mailing List 
> cc
> "openstack@lists.launchpad.net \(openstack@lists.launchpad.net\)" 
> 
> Subject
> Re: [openstack-dev] [nova] Call for Help -- OpenStack API XMLSupport
> 
> 
> 
> 
> 
> And this is why I go off on the developer-oriented mentality of the OpenStack 
> community. 
> 
> The fact that there is no one in the OpenStack developer community writing 
> XML stuff is not a reflection of the fact that there's no huge desire for 
> XML. 
> 
> It's in the spec for a reason: BECAUSE ENTERPRISES USE XML HEAVILY 
> 
> OpenStack developers aren't that audience. They use JSON. 
> 
> That the project can get to this point and not have tests for these things 
> shows a flaw in the development processes, not some grand illustration of 
> supply and demand. 
> 
> Do I really have to point out that if the spec calls for JSON and XML, you 
> should bloody well write integration tests to check for JSON and XML? 
> 
> You don't write whatever happens to please you. 
> 
> You know how I know all of this? I have an API that supports both XML and 
> JSON. I personally prefer JSON. Most of my friends and colleagues prefer and 
> use JSON. 
> 
> Most of my customers use XML. 
> 
> Thank $deity I actually write unit tests for each format. 
> 
> -George 
> 
> File under: 
> -> statistics 101 
> -> software development 101 
> 
> On Aug 9, 2012, at 5:52 PM, Vishvananda Ishaya  wrote: 
> 
> 
> On Aug 9, 2012, at 3:32 PM, George Reese  wrote: 
> 
> Why aren't the integration tests both XML and JSON? 
> 
> The simple answer is that no one has taken the time to write them. Our 
> devstack exercises use the python client bindings. Tempest has json clients 
> but no xml clients[1]. I think this demonstrates that there just isn't a huge 
> desire for xml. Users that I have chatted with just seem to care that the api 
> works and that they they have good bindings. 
> 
> I am definitely willing to be proven wrong on this point, but I'm secretly 
> hoping everyone agrees with me. It is a lot of work to maintain three APIs 
> (we are still maintaining EC2 as well) and keep them all functioning well, so 
> if people are happy without OpenStack XML I would be perfectly content to 
> deprecate it. 
> 
> Vish 
> 
> [1] 
> https://github.com/openstack/tempest/tree/master/tempest/services/nova/xml 
> 
> ___
> OpenStack-dev mailing list
> openstack-...@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> 
> -- 
> George Reese (george.re...@imaginary.com)
> t: @GeorgeReese   m: +1(207)956-0217   Skype: 
> nspollution
> cal: http://tungle.me/GeorgeReese   
> 
> 
> 
> ___

Re: [Openstack] [openstack-dev] [nova] Call for Help -- OpenStack API XML Support

[Daryl Walleck]

As part of my work on Tempest, I've created an alternate backend configuration 
to use XML requests/responses. This right now mostly covers Nova, but could 
easily be extended to test other projects as well. I hadn't pushed it yet 
because it seemed to be low priority, but I'd be more than glad to accelerate 
and get this committed.

Daryl

On Aug 9, 2012, at 8:14 PM, Doug Davis mailto:d...@us.ibm.com>>
 wrote:


Situations like this are always interesting to watch.  :-)

On the one hand its open-source, so if you care about something then put up the 
resources to make it happen.
On the other hand, that doesn't mean that as a developer you get to ignore the 
bigger picture and only do 1/2 of the work because you don't care about the 
other 1/2.

Overall, I tend to agree with the attitude that as long as XML is officially 
supported then all code changes need to make sure they run through both the 
JSON and XML codepaths. And if this means twice the testcases then so be it.  
People committing code shouldn't have a choice in this - its either you do the 
full job or your code is rejected.

Having said that, it is a valid question to ask whether we want to continue to 
support both JSON and XML going forward.  But, until that decision is formally 
made letting 1/2 of the APIs atrophy makes the entire community look bad and 
therefore should not be allowed to happen.

My vote: from now on don't let any code change in unless if works for both.  I 
suspect we'll either see the XML side come up to speed really quickly or it'll 
force an ugly vote.  But either way, this needs to be resolved before the next 
release.

thanks
-Doug

STSM |  Standards Architect  |  IBM Software Group
(919) 254-6905  |  IBM 444-6905  |  d...@us.ibm.com
The more I'm around some people, the more I like my dog.


George Reese mailto:george.re...@imaginary.com>>

08/09/2012 07:02 PM
Please respond to
OpenStack Development Mailing List 
mailto:openstack-...@lists.openstack.org>>




To
OpenStack Development Mailing List 
mailto:openstack-...@lists.openstack.org>>
cc
"openstack@lists.launchpad.net 
\(openstack@lists.launchpad.net\)" 
mailto:openstack@lists.launchpad.net>>
Subject
Re: [openstack-dev] [nova] Call for Help -- OpenStack API XML
Support







And this is why I go off on the developer-oriented mentality of the OpenStack 
community.

The fact that there is no one in the OpenStack developer community writing XML 
stuff is not a reflection of the fact that there's no huge desire for XML.

It's in the spec for a reason: BECAUSE ENTERPRISES USE XML HEAVILY

OpenStack developers aren't that audience. They use JSON.

That the project can get to this point and not have tests for these things 
shows a flaw in the development processes, not some grand illustration of 
supply and demand.

Do I really have to point out that if the spec calls for JSON and XML, you 
should bloody well write integration tests to check for JSON and XML?

You don't write whatever happens to please you.

You know how I know all of this? I have an API that supports both XML and JSON. 
I personally prefer JSON. Most of my friends and colleagues prefer and use JSON.

Most of my customers use XML.

Thank $deity I actually write unit tests for each format.

-George

File under:
-> statistics 101
-> software development 101

On Aug 9, 2012, at 5:52 PM, Vishvananda Ishaya 
mailto:vishvana...@gmail.com>> wrote:


On Aug 9, 2012, at 3:32 PM, George Reese 
mailto:george.re...@imaginary.com>> wrote:

Why aren't the integration tests both XML and JSON?

The simple answer is that no one has taken the time to write them. Our devstack 
exercises use the python client bindings. Tempest has json clients but no xml 
clients[1]. I think this demonstrates that there just isn't a huge desire for 
xml. Users that I have chatted with just seem to care that the api works and 
that they they have good bindings.

I am definitely willing to be proven wrong on this point, but I'm secretly 
hoping everyone agrees with me. It is a lot of work to maintain three APIs (we 
are still maintaining EC2 as well) and keep them all functioning well, so if 
people are happy without OpenStack XML I would be perfectly content to 
deprecate it.

Vish

[1] https://github.com/openstack/tempest/tree/master/tempest/services/nova/xml

___
OpenStack-dev mailing list
openstack-...@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
George Reese (george.re...@imaginary.com)
t: @GeorgeReese   m: +1(207)956-0217   Skype: 
nspollution
cal: http://tungle.me/GeorgeReese



___
OpenStack-dev mailing list
opensta

Re: [Openstack] [openstack-dev] [nova] Call for Help -- OpenStack API XML Support

[Doug Davis]

Situations like this are always interesting to watch.  :-)

On the one hand its open-source, so if you care about something then put 
up the resources to make it happen.
On the other hand, that doesn't mean that as a developer you get to ignore 
the bigger picture and only do 1/2 of the work because you don't care 
about the other 1/2.

Overall, I tend to agree with the attitude that as long as XML is 
officially supported then all code changes need to make sure they run 
through both the JSON and XML codepaths. And if this means twice the 
testcases then so be it.  People committing code shouldn't have a choice 
in this - its either you do the full job or your code is rejected.

Having said that, it is a valid question to ask whether we want to 
continue to support both JSON and XML going forward.  But, until that 
decision is formally made letting 1/2 of the APIs atrophy makes the entire 
community look bad and therefore should not be allowed to happen. 

My vote: from now on don't let any code change in unless if works for 
both.  I suspect we'll either see the XML side come up to speed really 
quickly or it'll force an ugly vote.  But either way, this needs to be 
resolved before the next release.

thanks
-Doug

STSM |  Standards Architect  |  IBM Software Group
(919) 254-6905  |  IBM 444-6905  |  d...@us.ibm.com
The more I'm around some people, the more I like my dog.



George Reese  
08/09/2012 07:02 PM
Please respond to
OpenStack Development Mailing List 


To
OpenStack Development Mailing List 
cc
"openstack@lists.launchpad.net \(openstack@lists.launchpad.net\)" 

Subject
Re: [openstack-dev] [nova] Call for Help -- OpenStack API XML   Support






And this is why I go off on the developer-oriented mentality of the 
OpenStack community.

The fact that there is no one in the OpenStack developer community writing 
XML stuff is not a reflection of the fact that there's no huge desire for 
XML.

It's in the spec for a reason: BECAUSE ENTERPRISES USE XML HEAVILY

OpenStack developers aren't that audience. They use JSON.

That the project can get to this point and not have tests for these things 
shows a flaw in the development processes, not some grand illustration of 
supply and demand.

Do I really have to point out that if the spec calls for JSON and XML, you 
should bloody well write integration tests to check for JSON and XML?

You don't write whatever happens to please you.

You know how I know all of this? I have an API that supports both XML and 
JSON. I personally prefer JSON. Most of my friends and colleagues prefer 
and use JSON.

Most of my customers use XML.

Thank $deity I actually write unit tests for each format.

-George

File under:
-> statistics 101
-> software development 101

On Aug 9, 2012, at 5:52 PM, Vishvananda Ishaya  
wrote:


On Aug 9, 2012, at 3:32 PM, George Reese  
wrote:

Why aren't the integration tests both XML and JSON?

The simple answer is that no one has taken the time to write them. Our 
devstack exercises use the python client bindings. Tempest has json 
clients but no xml clients[1]. I think this demonstrates that there just 
isn't a huge desire for xml. Users that I have chatted with just seem to 
care that the api works and that they they have good bindings.

I am definitely willing to be proven wrong on this point, but I'm secretly 
hoping everyone agrees with me. It is a lot of work to maintain three APIs 
(we are still maintaining EC2 as well) and keep them all functioning well, 
so if people are happy without OpenStack XML I would be perfectly content 
to deprecate it.

Vish

[1] 
https://github.com/openstack/tempest/tree/master/tempest/services/nova/xml

___
OpenStack-dev mailing list
openstack-...@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
George Reese (george.re...@imaginary.com)
t: @GeorgeReese   m: +1(207)956-0217   Skype: 
nspollution
cal: http://tungle.me/GeorgeReese 



___
OpenStack-dev mailing list
openstack-...@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] [nova] Call for Help -- OpenStack API XML Support

[Christopher B Ferris]

Has anyone surveyed those subscribed to openstack-operators@lists.openstack.org
 list for usage? While imperfect, at least it would be asking the 
constituency that might be most affected. You might also consider asking
whether they would prefer JSON or XML, regardless of what they use today.I agree with George that weighing developer interest in writing some test cases is likely not the best measure of how widely the XML API is used in practice.One last slightly unrelated point. There was discussion and a PPB decision not to maintain 3rd party APIs in core. Why does the EC2 API remain? Cheers,Christopher FerrisIBM Distinguished Engineer, CTO Industry and Cloud StandardsMember, IBM Academy of TechnologyIBM Software Group, Standards Strategyemail: chris...@us.ibm.comTwitter: christo4ferrisphone: +1 508 234 2986-openstack-bounces+chrisfer=us.ibm@lists.launchpad.net wrote: ->To: OpenStack Development Mailing List>>From: Vishvananda Ishaya >Sent by: openstack-bounces+chrisfer=us.ibm@lists.launchpad.net>Date: 08/09/2012 07:05PM>Cc: "openstack@lists.launchpad.net \(openstack@lists.launchpad.net\)">>Subject: Re: [Openstack] [openstack-dev] [nova] Call for Help -->OpenStack	API XML Support>>>On Aug 9, 2012, at 3:32 PM, George Reese >wrote:>Why aren't the integration tests both XML and JSON?>The simple answer is that no one has taken the time to write them.>Our devstack exercises use the python client bindings. Tempest has>json clients but no xml clients[1]. I think this demonstrates that>there just isn't a huge desire for xml. Users that I have chatted>with just seem to care that the api works and that they they have>good bindings.>>I am definitely willing to be proven wrong on this point, but I'm>secretly hoping everyone agrees with me. It is a lot of work to>maintain three APIs (we are still maintaining EC2 as well) and keep>them all functioning well, so if people are happy without OpenStack>XML I would be perfectly content to deprecate it.>>Vish>>[1] https://github.com/openstack/tempest/tree/master/tempest/services>/nova/xml> ___ Mailing list:>https://launchpad.net/~openstack Post to :>openstack@lists.launchpad.net Unsubscribe :>https://launchpad.net/~openstack More help   :>https://help.launchpad.net/ListHelp 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] [nova] Call for Help -- OpenStack API XML Support

[Vishvananda Ishaya]

On Aug 9, 2012, at 3:32 PM, George Reese  wrote:

> Why aren't the integration tests both XML and JSON?

The simple answer is that no one has taken the time to write them. Our devstack 
exercises use the python client bindings. Tempest has json clients but no xml 
clients[1]. I think this demonstrates that there just isn't a huge desire for 
xml. Users that I have chatted with just seem to care that the api works and 
that they they have good bindings.

I am definitely willing to be proven wrong on this point, but I'm secretly 
hoping everyone agrees with me. It is a lot of work to maintain three APIs (we 
are still maintaining EC2 as well) and keep them all functioning well, so if 
people are happy without OpenStack XML I would be perfectly content to 
deprecate it.

Vish

[1] https://github.com/openstack/tempest/tree/master/tempest/services/nova/xml

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp