Re: [openstack-dev] [horizon] bower
Richard, thanks for sharing this. I hope we can move to bower sooner rather than later. On Sat, Mar 7, 2015 at 5:26 PM, Richard Jones r1chardj0...@gmail.com wrote: On Sun, 8 Mar 2015 at 04:59 Michael Krotscheck krotsch...@gmail.com wrote: Anyone wanna hack on a bower mirror puppet module with me? BTW are you aware of this spec for bower/Horizon/infra? https://review.openstack.org/#/c/154297/ Richard __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] [swift] dependency on non-standardized, private APIs
David,
FYI, the last time I chatted with John Dickinson I learned there are
numerous API elements not documented. Not meant to be private but the docs
have not kept up. How should we handle that?
Thanks,
Matt Farina
On Sat, Mar 7, 2015 at 5:25 PM, David Lyle dkly...@gmail.com wrote:
I agree that Horizon should not be requiring optional headers. Changing
status of bug.
On Tue, Mar 3, 2015 at 5:51 PM, Jay Pipes jaypi...@gmail.com wrote:
Added [swift] to topic.
On 03/03/2015 07:41 AM, Matthew Farina wrote:
Radoslaw,
Unfortunately the documentation for OpenStack has some holes. What you
are calling a private API may be something missed in the documentation.
Is there a documentation bug on the issue? If not one should be created.
There is no indication that the X-Timestamp or X-Object-Meta-Mtime HTTP
headers are part of the public Swift API:
http://developer.openstack.org/api-ref-objectstorage-v1.html
I don't believe this is a bug in the Swift API documentation, either.
John Dickinson (cc'd) mentioned that the X-Timestamp HTTP header is
required for the Swift implementation of container replication (John,
please do correct me if wrong on that).
But that is the private implementation and not part of the public API.
In practice OpenStack isn't a specification and implementation. The
documentation has enough missing information you can't treat it this
way. If you want to contribute to improving the documentation I'm sure
the documentation team would appreciate it. The last time I looked there
were a number of undocumented public swift API details.
The bug here is not in the documentation. The bug is that Horizon is
coded to rely on HTTP headers that are not in the Swift API. Horizon should
be fixed to use DICT.get('X-Timestamp') instead of doing
DICT['X-Timestamp'] in its view pages for container details. There are
already patches up that the Horizon developers have, IMO erroneously,
rejected stating this is a problem in Ceph RadosGW for not properly
following the Swift API).
Best,
-jay
Best of luck,
Matt Farina
On Tue, Mar 3, 2015 at 9:59 AM, Radoslaw Zarzynski
rzarzyn...@mirantis.com mailto:rzarzyn...@mirantis.com wrote:
Guys,
I would like discuss a problem which can be seen in Horizon: breaking
the boundaries of public, well-specified Object Storage API in favour
of utilizing a Swift-specific extensions. Ticket #1297173 [1] may
serve
as a good example of such violation. It is about relying on
non-standard (in the terms of OpenStack Object Storage API v1) and
undocumented HTTP header provided by Swift. In order to make
Ceph RADOS Gateway work correctly with Horizon, developers had to
inspect sources of Swift and implement the same behaviour.
From my perspective, that practise breaks the the mission of
OpenStack
which is much more than delivering yet another IaaS/PaaS
implementation.
I think its main goal is to provide a universal set of APIs covering
all
functional areas relevant for cloud computing, and to place that set
of APIs in front as many implementations as possible. Having an open
source reference implementation of a particular API is required to
prove
its viability, but is secondary to having an open and documented API.
I have full understanding that situations where the public OpenStack
interfaces are insufficient to get the work done might exist.
However, introduction of dependency on implementation-specific
feature
(especially without giving the users a choice via e.g. some
configuration option) is not the proper way to deal with the problem.
From my point of view, such cases should be handled with adoption of
new, carefully designed and documented version of the given API.
In any case I think that Horizon, at least basic functionality,
should
work with any storage which provides Object Storage API.
That being said, I'm willing to contribute such patches, if we decide
to go that way.
Best regards,
Radoslaw Zarzynski
[1] https://bugs.launchpad.net/horizon/+bug/1297173
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:
unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe
Re: [openstack-dev] [horizon] JavaScript docs?
I'd like to step back for a moment as to the purpose of different kinds of documentation. Sphinx is great and it provides some forms of documentation. But, why do we document methods, classes, or functions in python? Should we drop that and rely on Sphinx? I don't think anyone would argue for that. Some documentation has a different purpose. For example, text editors and IDEs can introspect comments and help as we're writing the code. Or, say you have a bit of bit of code like the reusable wizard being written JavaScript. If someone is going to use it there should be a code comment about it. Just like you'd have for the tables class in Python because people are going to use it. JavaScript is a programming language just as much as Python is. If we should have comments in Python we should have comments in JavaScript. It's no different. JSDoc is the common format for JavaScript. It will help with text editors and IDEs. If we are going to move into an API docs site (which we don't have now) using ngDoc would be helpful. I think documenting in a useful manner is a different piece of scope from using that documentation to generate a site. Ideally, we would document JavaScript in a useful manner no matter if we are creating html docs from it or not. To document in a useful manner we should likely use JSDoc (or ngDoc) for the useful tools rather than do it our own way. We should use the wheel everyone else is using rather than do it our own way. Just my 2 cents. On Thu, Feb 5, 2015 at 1:56 AM, Pavel Karikh pkar...@mirantis.com wrote: On 02/05/2015 11:04 AM, Radomir Dopieralski wrote: Plus, the documentation generator that we are using already, Sphinx, supports JavaScript perfectly fine, so I see no reason to add another tool. I agree with Radomir, why we can't just use Sphinx? __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] JavaScript docs?
Ah, I had forgotten about the python module documentation. Sorry about that and thanks for pointing it out. Can we have that system parse and use JSDoc? I'd like it to be useful to both JS devs and the doc generation toolchain. On Thu, Feb 5, 2015 at 9:42 AM, Jeremy Stanley fu...@yuggoth.org wrote: On 2015-02-05 09:20:35 -0800 (-0800), Matthew Farina wrote: [...] But, why do we document methods, classes, or functions in python? Should we drop that and rely on Sphinx? I don't think anyone would argue for that. [...] Particularly since Sphinx collects the method/class/function docstrings from the source code to assemble them into documentation[1]. I see that as its primary reason for existing in fact. Why do you seem to think those choices are at odds with one another? [1] http://docs.openstack.org/developer/nova/api/nova.api.validator.html -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [horizon] JavaScript docs?
In python we have a style to document methods, classes, and so forth. But, I don't see any guidance on how JavaScript should be documented. I was looking for something like jsdoc or ngdoc (an extension of jsdoc). Is there any guidance on how JavaScript should be documented? For anyone who doesn't know, Angular uses ngdoc (an extension to the commonly used jsdoc) which is written up at https://github.com/angular/angular.js/wiki/Writing-AngularJS-Documentation. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] static files handling, bower/
Radomir and Matthias, Has anyone done an inventory of xstatic packages that are available as system packages? I ask because I started asking these questions after doing a cursory inventory and finding few xstatic packages as system packages. It appeared to me that the common case was the one Matthias noted where the xstatic package bundles the js libs. That we're actually getting them from pypi or a mirror. Can someone show me there really are system packages to replace the xstatic packages? On Thu, Jan 22, 2015 at 5:15 AM, Matthias Runge mru...@redhat.com wrote: On 22/01/15 09:48, Radomir Dopieralski wrote: All of the XStatic packages had to be packaged for the respective distributions in order to package Horizon. That was a lot of work, but it has been done my the packagers of the distributions. As far as I understand, most of those XStatic packages are just dummies, pointing to the actual system-wide JavaScript packages -- XStatic has such a capability. So while we are indeed maintaining some of the XStatic packages for our own convenience, the packages that contain actual code in the distributions are maintained by those distributions' packagers. Yupp, poniting to system packages is something, which makes the XStatic approach way more acceptable. But, if you don't have them (yet), xstatic packages will bundle js libs for you. Matthias __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] static files handling, bower/
I would like to add one more nuance to this discussion that I don't remember seeing. JavaScript libraries run in web browser in their JavaScript engines (like v8) rather than on the server. A version of a JS library may be fine on a system, without any security issues, but contain browser issues. The version used matters more to the application and the web browsers consuming the application than to the system it's on. Some of the libraries exist as packages. For example, there are some debian packages. These have older versions of libraries than those that will work in Horizon. The libraries need to integrate for horizon and the browsers. So, supporting varying versions of a js library, their interactions together, and creating a usable interface will be a real problem. For example the debian packages of old or varying versions will a problem for those of us attempting to craft a UI. What's there isn't practically usable today. Some things are missing. Does that help add clarity? On Thu, Jan 22, 2015 at 11:53 AM, Matthew Farina m...@mattfarina.com wrote: Radomir and Matthias, Has anyone done an inventory of xstatic packages that are available as system packages? I ask because I started asking these questions after doing a cursory inventory and finding few xstatic packages as system packages. It appeared to me that the common case was the one Matthias noted where the xstatic package bundles the js libs. That we're actually getting them from pypi or a mirror. Can someone show me there really are system packages to replace the xstatic packages? On Thu, Jan 22, 2015 at 5:15 AM, Matthias Runge mru...@redhat.com wrote: On 22/01/15 09:48, Radomir Dopieralski wrote: All of the XStatic packages had to be packaged for the respective distributions in order to package Horizon. That was a lot of work, but it has been done my the packagers of the distributions. As far as I understand, most of those XStatic packages are just dummies, pointing to the actual system-wide JavaScript packages -- XStatic has such a capability. So while we are indeed maintaining some of the XStatic packages for our own convenience, the packages that contain actual code in the distributions are maintained by those distributions' packagers. Yupp, poniting to system packages is something, which makes the XStatic approach way more acceptable. But, if you don't have them (yet), xstatic packages will bundle js libs for you. Matthias __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] static files handling, bower/
Jeremy, thanks for sharing this. I do have a couple more questions and comments based on this. When there is an update to our requirements, such as the recent version increment in the version of angular used, a new package version doesn't automatically show up as evident from that list. How would that process be kicked off so we don't end up with a missing dependency? Does that have to wait for a release cycle? Many of the xstatic packages are maintained by the OpenStack community. Just see the list in stackforge ( https://github.com/stackforge/?query=xstatic). Right now we need to update the xstatic package ourselves and then start using it. It also feels a little odd to maintain xstatic packages for pypi that we don't consume that way. This appears to affect the testing setup as well. When we start to use a new version of a JavaScript dependency no package will exist for it. I believe this would mean the testing environment needs to use the development setup, in the proposal, of bower. IIRC, bower goes out to the Internet and there isn't a mirror of packages (just a mirror of the registry). That means we'll loose the ability to run testing installs from local mirrors for these dependencies. I imagine a solution has been thought of for this. Can you share any details? Thanks, On Thu, Jan 22, 2015 at 2:35 PM, Jeremy Stanley fu...@yuggoth.org wrote: On 2015-01-22 11:53:10 -0500 (-0500), Matthew Farina wrote: Has anyone done an inventory of xstatic packages that are available as system packages? I ask because I started asking these questions after doing a cursory inventory and finding few xstatic packages as system packages. [...] https://packages.debian.org/xstatic Found 41 matching packages. I believe the majority are due to distro efforts around packaging OpenStack. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] static files handling, bower/
Richard, I'm quite familiar with node.js and browser development. I think some of the issue here may be a lack of detailed explanations and assumptions. By asking questions here I, and some others, have been learning details that we didn't know before. And, we're getting to follow from the intent from start to finish. I'm finding this to be incredibly revealing. Thanks, On Thu, Jan 22, 2015 at 3:47 PM, Richard Jones r1chardj0...@gmail.com wrote: On Fri Jan 23 2015 at 4:28:59 AM Matthew Farina m...@mattfarina.com wrote: I would like to add one more nuance to this discussion that I don't remember seeing. JavaScript libraries run in web browser in their JavaScript engines (like v8) rather than on the server. A version of a JS library may be fine on a system, without any security issues, but contain browser issues. The version used matters more to the application and the web browsers consuming the application than to the system it's on. I think you're confusing Javascript in the browser vs. Javascript for node.js Those are two separate things. Javascript used in the browser is rarely (never in our case) used also for node.js development. There are even two separate packaging systems: bower is all about components for browsers, whereas npm is all about packages for node.js. Our discussion is about bower, and definitely not about npm or the node.js programming language. Some of the libraries exist as packages. For example, there are some debian packages. These have older versions of libraries than those that will work in Horizon. Thanks to packagers working with OpenStack we are able to get newer versions of the components we need once we have pinned them in our code. The libraries need to integrate for horizon and the browsers. So, supporting varying versions of a js library, their interactions together, and creating a usable interface will be a real problem. For example the debian packages of old or varying versions will a problem for those of us attempting to craft a UI. What's there isn't practically usable today. Some things are missing. It's entirely up to us to specify a pinned set of component versions that we need, which we then need the system packagers to ensure are available as deb or rpm. Gaps will be filled as part of the usual packaging efforts that go on during the OpenStack release cycle. Richard __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] static files handling, bower/
Radomir, thanks for adding some clarity. I do have follow-on questions. In your example the packages are managed by xstatic. The proposal for horizon, as I understand it, is to move away from xstatic packages and instead use bower for development and system packages (for example, debian, rpm, and other packages) for production. Right now, we (the horizon community) is maintaining some of the xstatic packages. For many of these xstatic packages there is no corollary in a system package. Who will create and maintain the system packages for the JavaScript libraries? You noted that we get maintenance and updates for free. Since the system packages don't exist now and we don't know who will create or maintain them I'm not sure how to reconcile this. What am I missing? On Wed, Jan 21, 2015 at 3:04 AM, Radomir Dopieralski openst...@sheep.art.pl wrote: On 20/01/15 20:58, Matthew Farina wrote: Radomir, maybe you can help me better understand where this would go. I have a few questions. First, can you point me to a time when horizon used system packages successfully for JavaScript libraries? When I looked through the Debian and Ubuntu packages I couldn't find the libraries horizon is using. I'm curious to see this in action. Any distribution (perhaps except Ubuntu, which is a little funny in that regard) that has packaged the latest release of OpenStack, has those libraries. For instance, see http://pkgs.fedoraproject.org/cgit/python-django-horizon.git/tree/python-django-horizon.spec#n129 Front-end systems almost never use system packagers like this. Can you point me to applications like horizon that use system packages this way? If Horizon is going to go it virtually alone in this space, what will that mean for our level of work and ability to have updates? Certainly. The XStatic system itself is lifted from MoinMoin wiki, for example -- it was created to solve exactly this problem there, and is used by a couple of other projects too. As for our work and updates, using system-wide packages is an excellent solution in this regard, as we get maintenance and updates for free. For instance, if there is a security issue in one of the JavaScript libraries, we don't need to patch Horizon -- the patch that is prepared for that specific library and applied system-wide is sufficient. -- Radomir Dopieralski __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] static files handling, bower/
Martin, django_compressor does handles creating aggregated and compressed files for you. This isn't quite the same as C programs because it's not just due to file size. For example, if you have 2 files many browsers will make two separate connections to get each file. That mean negotiating a connection (can include tls), and pulling down the file dealing with TCP slow-start. This means one larger file is faster to download then multiple files that are smaller in overall size. To get a 304 Not Modified message you'll need to make sure your web server is handling that. I've encountered horizon instances where 304 Not Modified messages aren't happening. Cheers, Matt Farina On Wed, Jan 21, 2015 at 5:40 AM, Martin Geisler mar...@geisler.net wrote: Matthias Runge mru...@redhat.com writes: On 21/01/15 09:59, Martin Geisler wrote: This seems to imply that users will download at least one .js file per dependency. Not necessarily. We still use django-compressor, which copies all javascript into fewer files. E.g. here in my untweaked juno environment, I just get 3 instead of 10-15 following your logic above. (at least one js file per xstatic package). Cool, I did not know about this! Radomir said that this is run in a post-install hook for Horizon and that he's unsure how or if the distros re-run the scripts when the dependencies are installed. That is, if an updated version of the js-foo package is installed, all apps that depend on js-foo would need to have their JS bundles refreshed. Or maybe it's simply run every time Horizon is started? That's probably acceptable for an application like Horizon which users will be using again and again, but for most web applications, you don't want your users to download 10-20 small .js files, instead you want them to download one minified and compressed file with all the JavaScript code needed. see above :D I'm just mentioning this since it's one way that web apps differ from how normal Linux apps are typically deployed. Basically, web apps prefer static compilation and discourages dynamic linking. I'm not sure, I can really follow you here. I was trying to draw a parallel to how traditional (C) programs use dynamic when loading -- perfect for distributions since this allows them to patch a security problem just once and know that all programs that use the affected library will get the update since they dynamically load the library at runtime. Contrast this with static linking -- which is normally discouraged or forbidden by distributions since you'll have to recompile all dependant programs when you patch a library. I was thinking that web apps look like statically linked programs when they're deployed using compressed and minified sources for maximum performance. Let me add that this kind of optimization matters the most if your visitors who only visit the app once or rarely. With a file per dependency, they'll get poor performance since their browser won't have cached the files. For an app like Horizon it might not be as important: even if it loads a little slow on your first visit, you're likely to visit it many times as you admin your deployment and those subsequent visits will be faster. Still not as fast as they would be if the server could reply with a single 304 Not Modified, but probably fast enough. -- Martin Geisler http://google.com/+MartinGeisler __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] static files handling, bower/
Radomir, maybe you can help me better understand where this would go. I have a few questions. First, can you point me to a time when horizon used system packages successfully for JavaScript libraries? When I looked through the Debian and Ubuntu packages I couldn't find the libraries horizon is using. I'm curious to see this in action. Front-end systems almost never use system packagers like this. Can you point me to applications like horizon that use system packages this way? If Horizon is going to go it virtually alone in this space, what will that mean for our level of work and ability to have updates? Thanks, Matt On Mon, Jan 19, 2015 at 3:45 AM, Radomir Dopieralski openst...@sheep.art.pl wrote: On 16/01/15 18:55, Matthew Farina wrote: Doug, there still is one open question. Distributing JavaScript libraries via system packages is unusual. Because of that, most of the JavaScript libraries used by horizon don't have existing packages. Who will create and maintain the packages for these JavaScript libraries for production? For example, most of the libraries aren't available as debian or ubuntu packages. You are mistaken here. It's actually the other way around. Fedora and Debian packagers used to do heroic work with previous releases to unbundle the static files from Horizon and link to the system-wide JavaScript libraries installed from packages, because their packaging policies require that. The introduction of XStatic was supposed to merely simplify and formalize that process, but now it turns out that it is redundant, and we can cut a corner and save the packagers having to create all those dummy XStatic shims. -- Radomir Dopieralski __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] static files handling, bower/
Doug, there still is one open question. Distributing JavaScript libraries via system packages is unusual. Because of that, most of the JavaScript libraries used by horizon don't have existing packages. Who will create and maintain the packages for these JavaScript libraries for production? For example, most of the libraries aren't available as debian or ubuntu packages. Updating a JavaScript dependency, such as angular, would mean each of the system packages needs to be updated as well as the Horizon project. Would would this process look like with packages on different systems needing updates? Using bower for development and system packages for production doesn't appear to be ready to start using. On Fri, Jan 16, 2015 at 12:09 PM, Doug Hellmann d...@doughellmann.com wrote: On Jan 16, 2015, at 11:33 AM, Drew Fisher drew.fis...@oracle.com wrote: On 1/16/15 9:08 AM, Doug Hellmann wrote: We are, and as this conversation has veered off in a destructive direction, I think we should back up and look at the compromise Radomir posted [1] to see if that solves the original technical problem we all have. Does having the requirements specified in a JSON file, without requiring a specific build tool to install the files, solve the packaging, testing, and deployment issue on platforms where node.js isn’t supported natively right now? For Solaris, yes. We can make that work. Thanks, Radomir for the suggestion. Great! Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] function expressions vs function declarations in JavaScript
Thai, I'm still poking around at JavaScript things and did a little testing on function declarations vs function expressions. Seems Firefox is faster with function expressions by quite a bit at parse time. For reference see http://jsperf.com/mfer-function-types. I'm not suggesting what we do. I'm just sharing a data point I found surprising. I'm curious about the process for proposing changes to https://wiki.openstack.org/wiki/Horizon/Javascript. Is there any process? On Wed, Jan 14, 2015 at 1:15 PM, Thai Q Tran tqt...@us.ibm.com wrote: It was definitely an interesting the read, I never really noticed the subtle difference. Since then I have also read a few other thoughts on it. For me, function declarations can get convoluted very fast if not use correctly. Surely, you should never define functions inside of an if statement, and quite confusing to do it after a return statement. But they do have their uses when used correctly. I think it ultimately depends on what you are trying to do and whether it make sense to use declarations vs expressions. As long as people understand the difference, and take it case-by-case, its not really something I'm going to mull over too much. -Matthew Farina m...@mattfarina.com wrote: - To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org From: Matthew Farina m...@mattfarina.com Date: 01/14/2015 07:04AM Subject: [openstack-dev] [horizon] function expressions vs function declarations in JavaScript JavaScript has multiple ways to deal with functions. There are function declarations and function expressions (and named function expressions). In looking at some reviews and the current code I found some inconsistencies which leads me to ask, is there any guidance for handling this in Horizon? I noticed no documentation in https://wiki.openstack.org/wiki/Horizon/Javascript. Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [horizon] function expressions vs function declarations in JavaScript
JavaScript has multiple ways to deal with functions. There are function declarations and function expressions (and named function expressions). In looking at some reviews and the current code I found some inconsistencies which leads me to ask, is there any guidance for handling this in Horizon? I noticed no documentation in https://wiki.openstack.org/wiki/Horizon/Javascript. Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] static files handling, bower/
I think we're discussing two different situations with slightly different requirements. First, there is development and test. I believe the stated goal is to have node.js here. Would an environment not supporting node.js be needed for development or testing? Note, the JavaScript under test and to be executed by users is in browser rather than server side. The important execution environment is in their browser. The second environment is production. In that case the system packages would be used. What's still unclear is who creates and maintains those packages since many of them don't exist today. On Wed, Jan 14, 2015 at 10:14 AM, Drew Fisher drew.fis...@oracle.com wrote: On 1/14/15 6:25 AM, Anton Zemlyanov wrote: Solaris is supported by node.js: x86 is certainly supported. Always has been. That's not the issue in question. My point was that SPARC is not supported. Solaris 32-bit Binary: http://nodejs.org/dist/v0.10.35/node-v0.10.35-sunos-x86.tar.gz Solaris 64-bit Binary: http://nodejs.org/dist/v0.10.35/node-v0.10.35-sunos-x64.tar.gz I think Solaris is no longer relevant I won't stoop to comment on this statement other than to say Solaris is quite relevant to Oracle, Oracle's customers and Oracle's partners. -Drew __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] packaging problem production build question
Thanks for humoring me as I ask these questions. I'm just trying to connect the dots. How would system packages work in practice? For example, when it comes to ubuntu lucid (10.04 LTS) there is no system package meeting the jQuery requirement and for precise (12.04 LTS) you need precise-backports. This is for the most popular JavaScript library. There is only an angular package for trusty (14.04 LTS) and the version is older than the horizon minimum. private-bower would be a nice way to have a private registry. But, bower packages aren't packages in the same sense as system or pypi packages. If I understand it correctly, when bower downloads something it doesn't get it from the registry (bower.io or private-bower). Instead it goes to the source (e.g., Github) to download the code. private-bower isn't a package mirror but instead a private registry (of location). How could private-bower be used to negate network effects if you still need to go out to the Internet to get the packages? On Thu, Jan 8, 2015 at 5:11 PM, David Lyle dkly...@gmail.com wrote: Bower is not for use in production environments. There will continue to be two environment setup procedures, as there are today. For production, deploy Horizon and its dependencies via system packages. For development and testing leverage bower to pull the javascript resources, much as pip is used today and continue to use pip for python dependencies. For those running CI environments, remote access will likely be required for bower to work. Although, it seems something like private-bower [1] could be utilized to leverage a local mirror where access or network performance are issues. David [1] https://www.npmjs.com/package/private-bower On Thu, Jan 8, 2015 at 2:28 PM, Matthew Farina m...@mattfarina.com wrote: I've been going over the packaging problem in an effort to see how we can move to something better. Given the current proposal around bower I'm still left with a production deployment question. For a build environment sitting in isolation, unable to download from the Internet including Github, how would they be able to get all the bower controlled packages to create a system horizon package (e.g., rpm or deb)? These build environments currently use mirrors and controlled packages. For example, someone might have a pypi mirror with copies of the xstatic packages. This is tightly controlled. If bower is managing packages where, in theory, would it get them from for an environment like this? I may have missed something. If this has already been answered please excuse me and point me in the right direction. Thanks, Matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [horizon] packaging problem production build question
I've been going over the packaging problem in an effort to see how we can move to something better. Given the current proposal around bower I'm still left with a production deployment question. For a build environment sitting in isolation, unable to download from the Internet including Github, how would they be able to get all the bower controlled packages to create a system horizon package (e.g., rpm or deb)? These build environments currently use mirrors and controlled packages. For example, someone might have a pypi mirror with copies of the xstatic packages. This is tightly controlled. If bower is managing packages where, in theory, would it get them from for an environment like this? I may have missed something. If this has already been answered please excuse me and point me in the right direction. Thanks, Matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Horizon switching to the normal .ini format
Switching to an ini format would likely be painful to impossible. Horizon is built on django which is where the settings.py format comes from. It's part of a django app. For more info see the django docs. The settings information is at https://docs.djangoproject.com/en/1.6/topics/settings/ On Thu, Dec 25, 2014 at 1:25 PM, Timur Sufiev tsuf...@mirantis.com wrote: Thomas, I could only point you to the Radomir's patch https://review.openstack.org/#/c/100521/ It's still a work in progress, so you may ask him for more details. On Thu, Dec 25, 2014 at 1:59 AM, Thomas Goirand z...@debian.org wrote: Hi, There's been talks about Horizon switching to the normal .ini format that all other projects have been using so far. It would really be awesome if this could happen. Though I don't see the light at the end of the tunnel. Quite the opposite way: the settings.py is every day becoming more complicated. Is anyone at least working on the .ini switch idea? Or will we continue to see the Django style settings.py forever? Is there any blockers? Cheers, Thomas Goirand (zigo) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Timur Sufiev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] the future of angularjs development in Horizon
I would like to take a moment to point out that developing system software is different from developing web applications. The way systems software is developed and often deployed is different from web applications. Horizon as it sits today appears to be web application development by systems software developers. This raises the barrier to entry for web application developers. The approach being proposed moves horizon into the realm of web application technologies that web application people use today. The debate as I'm reading it is about taking web application development processes and turning them into systems development processes which are often foreign to web application developers. How is this going to work out? How will web app people be willing to get involved? Why should this be treated the same? Most of OpenStack is a systems problem. This piece is a little different. To make it successful should it get some wiggle room to work well in the space it's in? Note, I'm not saying it should be insecure or anything like that. There are just different approaches. On Thu, Nov 13, 2014 at 1:11 PM, Donald Stufft don...@stufft.io wrote: On Nov 13, 2014, at 12:38 PM, Thomas Goirand z...@debian.org wrote: On 11/13/2014 10:56 PM, Martin Geisler wrote: Maybe a silly question, but why insist on this? Why would you insist on installing a JavaScript based application using your package manager? I'm a huge fan of package managers and typically refuse to install anything globally if it doesn't come as a package. However, the whole JavaScript ecosystem seems to be centered around the idea of doing local installations. That means that you no longer need the package manager to install the software -- you only need a package manager to install the base system (NodeJs and npm for JavaScript). Yeah... Just like for Java, PHP, Perl, Python, you-name-it... In what way Javascript will be any different from all of these languages? Node.js, and npm in particular tends to solve the dependency hell problem by making it possible to install multiple versions of a particular thing and use them all within the same process. As far as I know the OS tooling doesn’t really handle SxS installs of the same thing in multiple versions very well, I think the closest that you can do is multiple separate packages with version numbers in the package name? In other words it’s entirely possible that a particular set of npm packages can not be resolved to a single version per library. Notice that Python has been moving rapidly in the same direction for years: you only need Python and pip to bootstrap yourself. After getting used to virtualenv, I've mostly stopped installing Python modules globally and that is how the JavaScript world expects you to work too. Fine for development. Not for deployments. Not for distributions. Or you just get a huge mess of every library installed 10 times, with 10 different versions, and then a security issue needs to be fixed… Eh, I wouldn’t say it’s not fine for deployments. Generally I find that the less I tie the things where I care about versions to my OS the happier my life gets. It’s not fine for distributions wanting to offer it though, that is correct. So maybe the Horizon package should be an installer package like the ones that download fonts or Adobe? This is a horrible design which will *never* make it to distributions. Please think again. What is it that makes Horizon so special? Answer: nothing. It's just a web app, so it doesn't need any special care. It should be packaged, just like the rest of everything, with .deb/.rpm and so on. That package would get the right version of node and which then runs the npm and bower commands to download the rest plus (importantly and much appreciated) puts the files in a sensible location and gives them good permissions. Fine for your development environment. But that's it. Also, does your $language-specific-package--manager has enough checks so that there's no man in the middle attack possible? Is it secured enough? Can a replay attack be done on it? Does it supports any kind of cryptography checks like yum or apt does? I'm almost sure that's not the case. pip is really horrible in this regard. I haven't checked, but I'm almost sure what we're proposing (eg: npm and such) have the same weakness. And here, I'm only scratching security concerns. There's other concerns, like how good is the dependency solver and such (remember: it took *years* for apt to be as good as it is right now, and it still has some defects). As far as I’m aware npm supports TLS the same as pip does. That secures the transport between the end users and the repository so you can be assured that there is no man in the middle. Security wise npm (and pip) are about ~95% (mad up numbers, but you can get the gist) of the effectiveness as the OS package managers.
[openstack-dev] [openstack-sdk-php] contribution process
When a group of us met recently we discussed documenting the contribution process to the PHP SDK. I've now done so and it's available at https://wiki.openstack.org/wiki/OpenStack-SDK-PHP/Process. If there are any questions or comments I'd be happy to talk about them. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack-sdk-php] Canceled Meeting and Future of weekly meetings
First I'd like to note that the weekly PHP SDK meeting this week is canceled. For the time being, unless someone has a good argument to the contrary, the meeting will be suspended. Those of us working on PHP can be found in the #openstack-sdks room in IRC. The meetings have stopped being useful at this point with few or no regular attenders. When we have a release and activity from others around the project picks up I expect to restart the meetings. If someone believes we should keep the meetings going I'm open to hearing the case. Please respond and share your stance. - Matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack-sdk-php] Meeting Canceled
The meeting for this Wednesday (8/20) is canceled. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack-sdk-php] Meeting canceled
The meeting for today is canceled. Sorry for the short notice. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack-sdk-php] Review of face to face meeting on direction for SDK.
We just completed a meeting on the OpenStack SDK for PHP and this email
attempts to sum up the discussion and action items to come from the
meeting. This was a face to face meeting that included discussions of
processes and architecture.
In attendance were,
- Glen Campbell, Rackspace
- Shaunak Kashyap, Rackspace
- Sam Choi, HP
- Matt Farina, HP
I want to thank everyone who came. This was a productive and good meeting.
The conversation opened with a discussion of process. We discussed the
Gerrit and review process used by the OpenStack community and how Gerrit
was configured for the PHP SDK codebase. Matt took an action item to
document the process.
In addition to the process we discussed reasons a patchset in review would
receive a -1 right away. These include:
- Duplicate effort
- Doesn't meet the coding standard
- The tests fail
- Counter to previously documented decision
- A work in progress (until we come up with a better way to handle that)
- Big feature not tied to a blueprint
We discussed the target audiences for PHP SDK work. The customers are, in
priority order:
1. SDK Consumer
2. Builders of the SDK
3. Extenders of the SDK
Note, #2 and #3 are very close to each other and a distance behind #1.
When we looked at the SDK Consumer we documented we noted a number of
projects that should be able to use the SDK easily. These include:
- Symfony
- Laravel
- Zend Framework
- Wordpress
- Drupal
- Joomla
- Spagetti Code
We also noted the difference between Products (that will be shipped) and
services that use the code.
As part of the process we looked at breaking ties. Since the project is
being entirely worked on by two companies it would be nice to have 3rd
parties as a tie breaker. We discussed using the community. We decided to
use a 3rd party agreed to arbitrator who is willing and we agree to. For
example, someone like Larry Garfield (who we have not contacted yet but is
an example of this type of person).
There were several topics where we came to technical direction.
The directory structure will be the following style...
identity/v2/
identity/v3/
objectStore/v1/
compute/v2/
compute/v3/
A service for a version will be the point of reference. In an attempt to
make extending or replacing the code for a service a viable option we're
looking into interfaces at the level of a service with additional interface
definition at the level of a service version. For example, compute could
have an interface with a reboot method. Then compute v3 could have an
interface that extends the compute interface and adds a compute v3
method/feature. Code consuming the these can check against the interfaces.
Glen took an action item to arrange feedback on the interface style from
some who have worked with this in the past for other SDKs.
When it comes to extending classes for functionality we came to a
conclusion to use the style...
class RackspaceObject extends Object {}
The idea is to use inheritance where needed. This style is quite common in
PHP. Scoping (public/protected/private) for methods and properties was
discussed. Common properties could be protected while uncommon ones, or
those we aren't sure of would be private. The only public methods would be
those defined by the interface.
An example of the scoping and using it in the inheritance model can be seen
at http://3v4l.org/peMiO.
At the top an OpenStack general object could be used to make the experience
easier. For example...
$service = $openstack-service(...);
The ... here is an array with a well documented structure for service
discovery from the service catalog. A session should be used to handle the
identity and make it easy for users.
The end user style discussed was based on 3 differing options. The options
discussed were..
1. An ORM style. For example,
$object = $objectStore-getContainer('foo')-getObject('bar');
2. Working with objects directly. For example,
$object = new Obejct();
$object-setRegion('region');
$object-setContainer('foo');
... (more sets)
$object-load();
3. A DSL Style. For example,
$object = $objectStorage-get(...)
Here ... is an array defining what to get.
The direction that clearly came forward was the #1 option. It was the one
everyone tended towards.
We noted that we might want to add -execute() to the end but that's an
optimization we can look at in the future. It could optimize to minimize
http requests.
To make it easier to understand end users, we want to have examples/tests
of real world code developers consuming the code could use. This will help
define APIs and real world needs. This should come before the implementing
code or along with it.
Action Items
- (Matt) Document the OpenStack/Gerrit workflow.
- Find 3rd party arbitrators.
- BDD or not to BDD decision.
- (Glen) Discussion into successes and failures of using interfaces.
- Document common properties to consider for protected scoping rather than
private scoping.
Decisions
Re: [openstack-dev] [openstack-sdk-php] Review of face to face meeting on direction for SDK.
Shaunak, I think it's safe to say I agree with all of your points. Thanks
for responding so quickly.
On Thu, Jul 31, 2014 at 7:01 PM, Shaunak Kashyap
shaunak.kash...@rackspace.com wrote:
Thanks for compiling this, Matt. I think you captured all the major
points here; I just have three additions/changes:
class RackspaceObject extends Object {}
I know this is the exact sample code we put up on the whiteboard and I
know we totally didn’t discuss namespaces at all at the time.
Looking at this snippet now, though, I’m wondering if using namespaces
here would be better so instead of RackspaceObject, the class name would be
just Object but in the Rackspace namespace. Presumably the name of the
class being extended would also be Object (as is the case in the sample
code) but that class would be part of the OpenStack (or Core or something)
namespace. Thoughts?
The idea is to use inheritance where needed. This style is quite common
in PHP.
I’ll add here something I mentioned during our meeting: We would favor
inheritance for scenarios such as extending an OpenStack-oriented class to
a Rackspace-oriented class. We would favor composition and dependency
injection for cross-cutting concerns such as logging.
Common properties could be protected while uncommon ones, or those we
aren't sure of would be private.
Based on the example we tried out in http://3v4l.org/ (thanks for
pointing us to this site, btw), I thought we were tending towards this
convention: all properties would be private, their accessors
(getters/setters) would either be public or protected; derived classes
would access these properties via these accessor methods, using
“parent::somePropertyAccessorMethod()” if necessary.
Thanks,
Shaunak
On Jul 31, 2014, at 6:40 PM, Matthew Farina m...@mattfarina.com wrote:
We just completed a meeting on the OpenStack SDK for PHP and this email
attempts to sum up the discussion and action items to come from the
meeting. This was a face to face meeting that included discussions of
processes and architecture.
In attendance were,
- Glen Campbell, Rackspace
- Shaunak Kashyap, Rackspace
- Sam Choi, HP
- Matt Farina, HP
I want to thank everyone who came. This was a productive and good
meeting.
The conversation opened with a discussion of process. We discussed the
Gerrit and review process used by the OpenStack community and how Gerrit
was configured for the PHP SDK codebase. Matt took an action item to
document the process.
In addition to the process we discussed reasons a patchset in review
would receive a -1 right away. These include:
- Duplicate effort
- Doesn't meet the coding standard
- The tests fail
- Counter to previously documented decision
- A work in progress (until we come up with a better way to handle that)
- Big feature not tied to a blueprint
We discussed the target audiences for PHP SDK work. The customers are,
in priority order:
1. SDK Consumer
2. Builders of the SDK
3. Extenders of the SDK
Note, #2 and #3 are very close to each other and a distance behind #1.
When we looked at the SDK Consumer we documented we noted a number of
projects that should be able to use the SDK easily. These include:
- Symfony
- Laravel
- Zend Framework
- Wordpress
- Drupal
- Joomla
- Spagetti Code
We also noted the difference between Products (that will be shipped) and
services that use the code.
As part of the process we looked at breaking ties. Since the project is
being entirely worked on by two companies it would be nice to have 3rd
parties as a tie breaker. We discussed using the community. We decided to
use a 3rd party agreed to arbitrator who is willing and we agree to. For
example, someone like Larry Garfield (who we have not contacted yet but is
an example of this type of person).
There were several topics where we came to technical direction.
The directory structure will be the following style...
identity/v2/
identity/v3/
objectStore/v1/
compute/v2/
compute/v3/
A service for a version will be the point of reference. In an attempt to
make extending or replacing the code for a service a viable option we're
looking into interfaces at the level of a service with additional interface
definition at the level of a service version. For example, compute could
have an interface with a reboot method. Then compute v3 could have an
interface that extends the compute interface and adds a compute v3
method/feature. Code consuming the these can check against the interfaces.
Glen took an action item to arrange feedback on the interface style from
some who have worked with this in the past for other SDKs.
When it comes to extending classes for functionality we came to a
conclusion to use the style...
class RackspaceObject extends Object {}
The idea is to use inheritance where needed. This style is quite common
in PHP. Scoping (public/protected/private
[openstack-dev] [openstack-sdk-php] Weekly Meeting Cancelations
The PHP SDK Meetings for 7/23 and 7/30 are canceled. The next meeting will be 8/6. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-php-sdk] pending reviews
Rob, This project is in PHP. We don't currently have a full test suite run for the project via the gates. When we set it up we were told not to go down that road for a PHP project, yet. Only recently have we been told to see about getting test suite gate testing going and I've started to look at doing it for things outside of python. That's why I wanted to run the full test suite locally. - Matt On Fri, Jun 20, 2014 at 1:44 AM, Robert Collins robe...@robertcollins.net wrote: What does that check that the gate doesn't? -Rob On 20 June 2014 14:20, Matthew Farina m...@mattfarina.com wrote: I'll dig into these on Friday. Thanks for the poke. I started to review them on Thursday and most of them are straight forward. But, something happened to my test environment and I want to run the full test suite on each of these. I should have that fixed and be able to finish working through these. On Thu, Jun 19, 2014 at 11:26 AM, Jamie Hannaford jamie.hannaf...@rackspace.com wrote: Hello all, Could we get some reviews through the door? This one is fairly straight-forward - we discussed and agreed on the action item in IRC: https://review.openstack.org/#/c/99633/ Here are some more trivial ones: https://review.openstack.org/#/c/100227/ https://review.openstack.org/#/c/99894/ https://review.openstack.org/#/c/99889/ I’m worried that the pace is slowing down. Thanks! Jamie Jamie Hannaford Software Developer III - CH [image: experience Fanatical Support] [image: LINE]Tel: +41434303908 Mob: +41791009767 [image: Rackspace] Rackspace International GmbH a company registered in the Canton of Zurich, Switzerland (company identification number CH-020.4.047.077-1) whose registered office is at Pfingstweidstrasse 60, 8005 Zurich, Switzerland. Rackspace International GmbH privacy policy can be viewed at www.rackspace.co.uk/legal/swiss-privacy-policy - Rackspace Hosting Australia PTY LTD a company registered in the state of Victoria, Australia (company registered number ACN 153 275 524) whose registered office is at Suite 3, Level 7, 210 George Street, Sydney, NSW 2000, Australia. Rackspace Hosting Australia PTY LTD privacy policy can be viewed at www.rackspace.com.au/company/legal-privacy-statement.php - Rackspace US, Inc, 5000 Walzem Road, San Antonio, Texas 78218, United States of America Rackspace US, Inc privacy policy can be viewed at www.rackspace.com/information/legal/privacystatement - Rackspace Limited is a company registered in England Wales (company registered number 03897010) whose registered office is at 5 Millington Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be viewed at www.rackspace.co.uk/legal/privacy-policy - Rackspace Benelux B.V. is a company registered in the Netherlands (company KvK nummer 34276327) whose registered office is at Teleportboulevard 110, 1043 EJ Amsterdam. Rackspace Benelux B.V privacy policy can be viewed at www.rackspace.nl/juridisch/privacy-policy - Rackspace Asia Limited is a company registered in Hong Kong (Company no: 1211294) whose registered office is at 9/F, Cambridge House, Taikoo Place, 979 King's Road, Quarry Bay, Hong Kong. Rackspace Asia Limited privacy policy can be viewed at www.rackspace.com.hk/company/legal-privacy-statement.php - This e-mail message (including any attachments or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com and delete the original message. Your cooperation is appreciated. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-sdk-php] Use of final and private keywords to limit extending
there will be disagreement to this. If someone has a different view please frame it in terms of the end users, use cases, and other context. This is about solving problems and enabling users to be successful over a particular sub-style in the PHP community. Cheers, Matt On Mon, Jun 16, 2014 at 5:16 PM, Matthew Farina m...@mattfarina.com wrote: Before I dig into the meat of my response, something occurred to me that's contextually significant. PHP is a flexible language. There are numerous ways to do the same thing and the language is open enough that those who have differing opinions on style and patterns can implement projects their way. When we often talk about software we talk about libraries to bring a feature. For example, if two or more people want to write a UUID library using different styles they can. The OpenStack PHP SDK isn't about the library itself but rather a pass through to the services. Ideally, where there are multiple UUID libraries with different patterns we wouldn't have that for the OpenStack SDK. It should be flexible enough to fit into multiple schemes rather than opinionated to one or two of the different philosophies. Entirely unrelated, there is an interesting post on structure that just came out at http://technosophos.com/2014/06/13/don-t-shrink-wrap-bananas.html (it's about software with shrink wrapping bananas as an illustration). We need to keep in mind that many of the uses of the SDK will not be open source projects. It might be something like a web app that takes advantage of OpenStack APIs but itself is not open source. The license allows for this. With that in mind and the two use cases I previously shared... I reached out to a number of different people in different software camps... No one suggested we use private or final. Even those who sometimes use it in their own code. If someone in their app wants to extend one of our classes that's fine and on them. Some folks do alterations that way so why not let them at their own risk. Larry Garfield had an interesting suggestion last week that someone else suggested to me today as well... have public methods for the public APIs to a class. For internal methods that are not public, mark them with the @internal annotation. Symfony and Drupal are doing this. It's an informal notation. I'm told that toolchains like .NET formally support this so you don't see the use of things like final and private so much anymore. If we wanted to protect internal methods/properties from outside calling make then protected. If they change something that's internal they do so at their own risk. If we define interfaces with clear ins and outs where everything holds to them this shouldn't be that much of a big deal either. If we keep things fairly simple, defined with interfaces, and reusable this shouldn't be a problem. When I asked a wide array of people about this topic without giving an opinion I was surprised how many had similar ideas. Thoughts? - Matt On Thu, Jun 12, 2014 at 4:12 PM, Matthew Farina m...@mattfarina.com wrote: I'm looking over this and reaching out to some folks (e.g., I just chatted with Larry Garfield). Want to let you know that I'm reading the articles, talking with others, and digging into this. I'll respond more after doing some more legwork. On Thu, Jun 12, 2014 at 4:37 AM, Jamie Hannaford jamie.hannaf...@rackspace.com wrote: To make some of my points less abstract, here’s the difference I’m talking about: https://gist.github.com/jamiehannaford/207c011a3cc45ff70e24 One uses inheritance, the other takes advantage of composition and interface polymorphism. The second is way more flexible because: the App class only uses what it needs to (as opposed to being lumbered with dozens of parent methods it doesn’t want), and secondly it can accept any implementor of the interface - so it’s not tightly coupled to a specific implementation. An example of how inheritance can go wrong is in our current codebase: the RemoteObject and Object classes. One of them inherits the other and has to override lots of methods just to get things to work. The result is confusion and incoherence because you have no freedom and flexibility when inheriting classes with dozens of methods that do a very specific thing. I’m guessing that the vast majority of developers will want to access SDK behavior, not augment it. For those that do want the latter, all they need to do is access the behavior and modify it outside of the class using composition - picking the methods they want and modifying behavior externally. IMO, inheritance is not a requirement for this. Jamie On June 12, 2014 at 2:49:55 AM, Matthew Farina (m...@mattfarina.com) wrote: We've been talking a bit theoretically here. Let me try to jump in with two use cases. 1. Someone needs to add functionality to a class but can't add it back upstream. That could be because they are in a hurry to get
Re: [openstack-dev] [openstack-php-sdk] pending reviews
I'll dig into these on Friday. Thanks for the poke. I started to review them on Thursday and most of them are straight forward. But, something happened to my test environment and I want to run the full test suite on each of these. I should have that fixed and be able to finish working through these. On Thu, Jun 19, 2014 at 11:26 AM, Jamie Hannaford jamie.hannaf...@rackspace.com wrote: Hello all, Could we get some reviews through the door? This one is fairly straight-forward - we discussed and agreed on the action item in IRC: https://review.openstack.org/#/c/99633/ Here are some more trivial ones: https://review.openstack.org/#/c/100227/ https://review.openstack.org/#/c/99894/ https://review.openstack.org/#/c/99889/ I’m worried that the pace is slowing down. Thanks! Jamie Jamie Hannaford Software Developer III - CH [image: experience Fanatical Support] [image: LINE]Tel: +41434303908 Mob: +41791009767 [image: Rackspace] Rackspace International GmbH a company registered in the Canton of Zurich, Switzerland (company identification number CH-020.4.047.077-1) whose registered office is at Pfingstweidstrasse 60, 8005 Zurich, Switzerland. Rackspace International GmbH privacy policy can be viewed at www.rackspace.co.uk/legal/swiss-privacy-policy - Rackspace Hosting Australia PTY LTD a company registered in the state of Victoria, Australia (company registered number ACN 153 275 524) whose registered office is at Suite 3, Level 7, 210 George Street, Sydney, NSW 2000, Australia. Rackspace Hosting Australia PTY LTD privacy policy can be viewed at www.rackspace.com.au/company/legal-privacy-statement.php - Rackspace US, Inc, 5000 Walzem Road, San Antonio, Texas 78218, United States of America Rackspace US, Inc privacy policy can be viewed at www.rackspace.com/information/legal/privacystatement - Rackspace Limited is a company registered in England Wales (company registered number 03897010) whose registered office is at 5 Millington Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be viewed at www.rackspace.co.uk/legal/privacy-policy - Rackspace Benelux B.V. is a company registered in the Netherlands (company KvK nummer 34276327) whose registered office is at Teleportboulevard 110, 1043 EJ Amsterdam. Rackspace Benelux B.V privacy policy can be viewed at www.rackspace.nl/juridisch/privacy-policy - Rackspace Asia Limited is a company registered in Hong Kong (Company no: 1211294) whose registered office is at 9/F, Cambridge House, Taikoo Place, 979 King's Road, Quarry Bay, Hong Kong. Rackspace Asia Limited privacy policy can be viewed at www.rackspace.com.hk/company/legal-privacy-statement.php - This e-mail message (including any attachments or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com and delete the original message. Your cooperation is appreciated. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-sdk-php] Use of final and private keywords to limit extending
Before I dig into the meat of my response, something occurred to me that's contextually significant. PHP is a flexible language. There are numerous ways to do the same thing and the language is open enough that those who have differing opinions on style and patterns can implement projects their way. When we often talk about software we talk about libraries to bring a feature. For example, if two or more people want to write a UUID library using different styles they can. The OpenStack PHP SDK isn't about the library itself but rather a pass through to the services. Ideally, where there are multiple UUID libraries with different patterns we wouldn't have that for the OpenStack SDK. It should be flexible enough to fit into multiple schemes rather than opinionated to one or two of the different philosophies. Entirely unrelated, there is an interesting post on structure that just came out at http://technosophos.com/2014/06/13/don-t-shrink-wrap-bananas.html (it's about software with shrink wrapping bananas as an illustration). We need to keep in mind that many of the uses of the SDK will not be open source projects. It might be something like a web app that takes advantage of OpenStack APIs but itself is not open source. The license allows for this. With that in mind and the two use cases I previously shared... I reached out to a number of different people in different software camps... No one suggested we use private or final. Even those who sometimes use it in their own code. If someone in their app wants to extend one of our classes that's fine and on them. Some folks do alterations that way so why not let them at their own risk. Larry Garfield had an interesting suggestion last week that someone else suggested to me today as well... have public methods for the public APIs to a class. For internal methods that are not public, mark them with the @internal annotation. Symfony and Drupal are doing this. It's an informal notation. I'm told that toolchains like .NET formally support this so you don't see the use of things like final and private so much anymore. If we wanted to protect internal methods/properties from outside calling make then protected. If they change something that's internal they do so at their own risk. If we define interfaces with clear ins and outs where everything holds to them this shouldn't be that much of a big deal either. If we keep things fairly simple, defined with interfaces, and reusable this shouldn't be a problem. When I asked a wide array of people about this topic without giving an opinion I was surprised how many had similar ideas. Thoughts? - Matt On Thu, Jun 12, 2014 at 4:12 PM, Matthew Farina m...@mattfarina.com wrote: I'm looking over this and reaching out to some folks (e.g., I just chatted with Larry Garfield). Want to let you know that I'm reading the articles, talking with others, and digging into this. I'll respond more after doing some more legwork. On Thu, Jun 12, 2014 at 4:37 AM, Jamie Hannaford jamie.hannaf...@rackspace.com wrote: To make some of my points less abstract, here’s the difference I’m talking about: https://gist.github.com/jamiehannaford/207c011a3cc45ff70e24 One uses inheritance, the other takes advantage of composition and interface polymorphism. The second is way more flexible because: the App class only uses what it needs to (as opposed to being lumbered with dozens of parent methods it doesn’t want), and secondly it can accept any implementor of the interface - so it’s not tightly coupled to a specific implementation. An example of how inheritance can go wrong is in our current codebase: the RemoteObject and Object classes. One of them inherits the other and has to override lots of methods just to get things to work. The result is confusion and incoherence because you have no freedom and flexibility when inheriting classes with dozens of methods that do a very specific thing. I’m guessing that the vast majority of developers will want to access SDK behavior, not augment it. For those that do want the latter, all they need to do is access the behavior and modify it outside of the class using composition - picking the methods they want and modifying behavior externally. IMO, inheritance is not a requirement for this. Jamie On June 12, 2014 at 2:49:55 AM, Matthew Farina (m...@mattfarina.com) wrote: We've been talking a bit theoretically here. Let me try to jump in with two use cases. 1. Someone needs to add functionality to a class but can't add it back upstream. That could be because they are in a hurry to get their app out and aren't concerned with contributing it or because the organization they work for won't let them release it in a timely manner or ever. 2. A vendor needs to extend a class to add functionality. For example, extending objet storage to add CDN support. I've personally run into cases in codebases to do both of these. While I was sitting here I came up with multiple
Re: [openstack-dev] [openstack-sdk-php] Use of final and private keywords to limit extending
We've been talking a bit theoretically here. Let me try to jump in with two use cases. 1. Someone needs to add functionality to a class but can't add it back upstream. That could be because they are in a hurry to get their app out and aren't concerned with contributing it or because the organization they work for won't let them release it in a timely manner or ever. 2. A vendor needs to extend a class to add functionality. For example, extending objet storage to add CDN support. I've personally run into cases in codebases to do both of these. While I was sitting here I came up with multiple examples of both that I've experienced and seen. When I look at this SDK I look at the library in it as a collection of building blocks that can be swapped out and used together with some helper code to make it easily work together. One can have access to all the low level parts and use them or use a high level API and brings it together. PHP is a popular language. According to w3techs.com PHP is used by 82.1% of all the websites whose server-side programming language we know. The style of extension you're talking about is used by a subset of PHP developers. Should we enable folks to extend this codebase the way they generally extend code or push them into extending this in a different manner from the other code they work with? You can allow for extension along multiple avenues and still have good design. A number of popular languages don't have the ability to use final or private. They have well designed apps and libraries without a problem. If we want to have something well designed we should have interfaces. Anything that implements those interfaces should be fine to use with anything that accepts them. We provide classes that handle the interactions with the services. They can be extended, replaced, or what not. As long as the class implements the interface with the other things that work with that interface it's fine. One thing did catch my attention. we need to understand how and why users want to interact with, or augment, the functionality of our SDK. I don't think we'll ever be able to know all the ways people may want to extend things in the ways they are comfortable extending things. There is a wide array of things that this can happen. Trying to know all these things and control them will be difficult for them to use and us to support. - Matt On Tue, Jun 10, 2014 at 4:10 AM, Choi, Sam sam.c...@hp.com wrote: To be clear, I don’t question his credentials as the blog post was quite thorough and informative. I’m just pointing out that he may be writing the post primarily from the viewpoint of an enterprise developer. About the “nobody else is using it” argument, I guess I should have clarified so my apologies. Linking back to my previous statement, I’m saying that there are occasionally different design philosophies and opinions based on the type of project. It has simply been my observation that the enterprise applications I’ve dealt with did use ‘final’ for a number of classes. As I mentioned, this makes sense since this is in the scope of a closed environment and the author is trying to enforce a constraint of future developers who may edit legacy code many years later. So in this context, sure, ‘final’ classes can prove to be useful. For open source APIs, final classes are used much more sparingly since the end goal should really be to allow other devs to make practical applications by using the API to suit their needs. Also, ‘better ways’ to allow extensions are fairly subjective and based on a dev’s past experience and current project. As Matt mentioned, we need to help the typical PHP dev, who may naturally gravitate towards extending based on past experience. On the other hand, the top devs will find clever workarounds to suit their needs. So again, why enforce the end users to follow a specific and opinionated design philosophy? From: Jamie Hannaford [mailto:jamie.hannaf...@rackspace.com] Sent: Tuesday, June 10, 2014 12:44 AM To: Matthew Farina; Choi, Sam Cc: Glen Campbell; OpenStack Development Mailing List (not for usage questions); Shaunak Kashyap; Farina, Matt Subject: RE: [openstack-sdk-php] Use of final and private keywords to limit extending I met Matthias a few weeks ago at PHP day in Verona and he really knows his stuff. I don’t think his work experience is relevance - just the clarity and applicability of his ideas. FWIW, he works closely with the folks at Inviqa, who are creating some of the best OSS in the PHP community. Like me, they value his opinion because it’s well-founded and generally applicable to any kind of software. Same goes with all the attendees who listen to his talks. I don’t understand the “well nobody else is using it” argument. We should examine the merits of this proposal based on whether we deem it a good design choice, not by looking over our shoulders. We are not constricting users here
Re: [openstack-dev] [openstack-sdk-php] Transport Clients, Service Clients, and state
that will hardly be used by the majority of our user base? I’ve worked on our current SDK for over a year - it has almost 40k downloads - and I’ve never been asked about custom transport configurations like proxy settings or whatever. - Related to the previous point: how are you going to lower the barriers for users who don’t want to inject their own HTTP clients? Will you offer a default transport client, for example? Jamie On June 7, 2014 at 9:58:07 PM, Matthew Farina (m...@mattfarina.com) wrote: My comments are inline below... On Fri, Jun 6, 2014 at 8:47 AM, Jamie Hannaford jamie.hannaf...@rackspace.com wrote: Whether the same one is used for each service or a new one is used for each service doesn't matter. Yes, it does matter IMO - and here are the reasons why: 1. By sharing a global transport object you’re introducing the risk of side effects. A transport object contains state that can be modified by its service object. Somewhere along the line, a Swift service could introduce a state modification that’s completely incompatible with a Nova service. What’s worse is that it will be a nightmare to debug - you’d have to trawl the entire service to see points where it interacts with the transport client. This is why people don’t use singletons - it’s incredibly risky and hard to debug. Object instantiations, on the other hand, are cheap and they offer protection through isolation. There are two things here. First, if the transport client has no state for the service than it doesn't get mixed up on state. A Swift client would never introduce state for swift to the transport client because the transport client has no state for this. It's for transporting. Second, it's not a singleton. You could have the same transport client for all of them, a different transport client for each, or any permutation in between. If the transport client contains no state to a service than it doesn't matter. To quote wikipedia, the singleton pattern is a design pattern that restricts the instantiation of a class to one object. A singleton is an intended restriction. This isn't a restriction. It's about options. If the service client is responsible for state for the service and the transport client is responsible for transporting information and the state of transport (e.g., is the info going through a proxy) than you don't run into issues where the transport client knows state of a service because that's the responsibility of the service client not the transport client. 2. Certain services rely on custom transport configurations. Each transport client has a base URL that is used for issuing HTTP requests - every time you execute a request, you’re effectively adding relevant paths for that API operation. A Swift service will have different URL endpoints from a Nova one - so there’s no point sharing. Another example is custom headers. Marconi requests custom headers to be sent, as does Glance. You save these as default headers on the transport client, that are sent for all requests that the service executes. These custom headers are not applicable to any other service except Marconi/Glance. If a transport client know the base URL than it knows state about the service. The separation of concerns is broken. Why does it need to know the URL? Why does it need to know about custom headers? Customizations and state for a service are the responsibility of the service client and not the transport client. Why does a service client and transport client need to both know the state of the service? The responsibility become blurred here. In the use-cases you mentioned, you’d easily handle that. You’d pass in proxy settings through the OpenStack entry point (like you do with your username and password), which would then percolate down into the transport clients as they’re created. These settings would be injected into each transport client. So if you require a different set-up for public clouds - that’s fine - you define different settings and fire up another $openstack object. How things get passed around isn't an issues. I don't think we need to debase how we pass settings around right now. The issue is separation of concerns between the service clients and the transport clients. -OR- you could define different transport settings for different services - by passing them into the $openstack-get(‘compute’, [‘custom_settings’ = true]); call. This is great because it gives users the ability to apply custom transport options to certain services. So if I want to interact with a private Compute instance, I’d pass in a custom transport configuration for that service; if I wanted to use a proxy with my Swift service, I can pass details into that service when creating it. You can only do this (provide custom transport settings for 1 service) if each transport client is isolated, i.e. if there’s a 1-to-1 relationship between service and transport client
Re: [openstack-dev] [openstack-sdk-php] Pending reviews
The reviews are in and they are both merged. Thanks for the reminder. On Tue, Jun 10, 2014 at 3:12 AM, Jamie Hannaford jamie.hannaf...@rackspace.com wrote: Hey folks, Could we get these two patches reviewed either today or tomorrow? The first is array syntax: https://review.openstack.org/#/c/94323 https://review.openstack.org/#/c/94323/5 The second is removing the “bin” and “scripts” directories from top-level tree, as discussed in last week’s meeting: https://review.openstack.org/#/c/98048/ Neither has code changes, so should be fairly simple to review. Thanks! Jamie Jamie Hannaford Software Developer III - CH [image: experience Fanatical Support] [image: LINE] Tel: +41434303908Mob: +41791009767 [image: Rackspace] Rackspace International GmbH a company registered in the Canton of Zurich, Switzerland (company identification number CH-020.4.047.077-1) whose registered office is at Pfingstweidstrasse 60, 8005 Zurich, Switzerland. Rackspace International GmbH privacy policy can be viewed at www.rackspace.co.uk/legal/swiss-privacy-policy - Rackspace Hosting Australia PTY LTD a company registered in the state of Victoria, Australia (company registered number ACN 153 275 524) whose registered office is at Suite 3, Level 7, 210 George Street, Sydney, NSW 2000, Australia. Rackspace Hosting Australia PTY LTD privacy policy can be viewed at www.rackspace.com.au/company/legal-privacy-statement.php - Rackspace US, Inc, 5000 Walzem Road, San Antonio, Texas 78218, United States of America Rackspace US, Inc privacy policy can be viewed at www.rackspace.com/information/legal/privacystatement - Rackspace Limited is a company registered in England Wales (company registered number 03897010) whose registered office is at 5 Millington Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be viewed at www.rackspace.co.uk/legal/privacy-policy - Rackspace Benelux B.V. is a company registered in the Netherlands (company KvK nummer 34276327) whose registered office is at Teleportboulevard 110, 1043 EJ Amsterdam. Rackspace Benelux B.V privacy policy can be viewed at www.rackspace.nl/juridisch/privacy-policy - Rackspace Asia Limited is a company registered in Hong Kong (Company no: 1211294) whose registered office is at 9/F, Cambridge House, Taikoo Place, 979 King's Road, Quarry Bay, Hong Kong. Rackspace Asia Limited privacy policy can be viewed at www.rackspace.com.hk/company/legal-privacy-statement.php - This e-mail message (including any attachments or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com and delete the original message. Your cooperation is appreciated. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-sdk-php] Use of final and private keywords to limit extending
points in different, often better, ways. Declaring explicitly what the extension points are is part of the contract your code has with the rest of the system. Final classes help to enforce this contract. To summarize, we have nothing to lose by favoring private and final keywords. We gain the above advantages, and if we later decide to open up that class as an extension point we can remove the keywords without any issues. Should a valid reason come up to open it up, it will be easy to do so, because nothing depends on it being closed. On the other hand, if you start by making everything open or inheritable, it will be very hard to close it later. Jamie On June 5, 2014 at 6:24:52 PM, Matthew Farina (m...@mattfarina.com) wrote: Some recent reviews have started to include the use of the private keyword for methods and talk of using final on classes. I don't think we have consistent agreement on how we should do this. My take is that we should not use private or final unless we can articulate the design decision to intentionally do so. To limit public the public API for a class we can use protected. Moving from protected to private or the use of final should have a good reason. In open source software code is extended in ways we often don't think of up front. Using private and final limits how those things can happen. When we use them we are intentionally limiting extending so we should be able to articulate why we want to put that limitation in place. Given the reviews that have been put forth I think there is a different stance. If there is one please share it. - Matt Jamie Hannaford Software Developer III - CH Tel: +41434303908 Mob: +41791009767 Rackspace International GmbH a company registered in the Canton of Zurich, Switzerland (company identification number CH-020.4.047.077-1) whose registered office is at Pfingstweidstrasse 60, 8005 Zurich, Switzerland. Rackspace International GmbH privacy policy can be viewed at www.rackspace.co.uk/legal/swiss-privacy-policy - Rackspace Hosting Australia PTY LTD a company registered in the state of Victoria, Australia (company registered number ACN 153 275 524) whose registered office is at Suite 3, Level 7, 210 George Street, Sydney, NSW 2000, Australia. Rackspace Hosting Australia PTY LTD privacy policy can be viewed at www.rackspace.com.au/company/legal-privacy-statement.php - Rackspace US, Inc, 5000 Walzem Road, San Antonio, Texas 78218, United States of America Rackspace US, Inc privacy policy can be viewed at www.rackspace.com/information/legal/privacystatement - Rackspace Limited is a company registered in England Wales (company registered number 03897010) whose registered office is at 5 Millington Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be viewed at www.rackspace.co.uk/legal/privacy-policy - Rackspace Benelux B.V. is a company registered in the Netherlands (company KvK nummer 34276327) whose registered office is at Teleportboulevard 110, 1043 EJ Amsterdam. Rackspace Benelux B.V privacy policy can be viewed at www.rackspace.nl/juridisch/privacy-policy - Rackspace Asia Limited is a company registered in Hong Kong (Company no: 1211294) whose registered office is at 9/F, Cambridge House, Taikoo Place, 979 King's Road, Quarry Bay, Hong Kong. Rackspace Asia Limited privacy policy can be viewed at www.rackspace.com.hk/company/legal-privacy-statement.php - This e-mail message (including any attachments or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com and delete the original message. Your cooperation is appreciated. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Stateful Applications on OpenStack
In my experience building apps that run in OpenStack, you don't give up state. You shift how you handle state. For example, instead of always routing a user to the same instance and that instance holding the session data there is a common session store for the app (possibly synced between regions). If you store session on each instance and loose an instance you'll run into problems. If sessions is more of a service for each instance than an instance coming and going isn't a big deal. A good database as a service, swift (object storage), and maybe a microservice architecture may be helpful. Legacy applications might have some issues with the architecture changes and some may not be a good fit for cloud architectures. One way to help legacy applications is to use block storage, keep the latest snapshot of the instance in glance (image service), and monitor an instance. If an instance goes offline you can easily create a new one from the image and mount block storage with the data. - Matt On Mon, Jun 9, 2014 at 7:27 AM, hossein zabolzadeh zabolza...@gmail.com wrote: Hi OpenStack Development Community, I know that the OpenStack interest is to become a cloud computing operating system. And this simple sentence means: Say goodbye to Statefull Applications. But, as you know we are in the transition phase from stateful apps to stateless apps(Remember Pets and Cattle Example). Legacy apps are still in used and how openstack can address the problems of running stateful applications(e.g. HA, DR, FT, R,...)? HA: High Availability DR: Disaster Recovery FT: Fault Tolerance R: Resiliancy! ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-sdk-php] Reasons to use Behat/behavior driven development in an SDK?
Jamie, thanks for sharing those links. They are quite useful and led me to a couple questions. 1. To quote the first link, To do this, we need a way to describe the requirement such that everyone – the business folks, the analyst, the developer and the tester – have a common understanding of the scope of the work. Where are the business folks, the analyst, and the tester? behat does things in human readable language that's really useful for the non-developer. Where do we have these in the development of this SDK? I ask because in that first post the idea of working with these different types of people is a central point. If you're working on a client project for non-technical clients, which is common in consulting circles, and in enterprise apps where you have analysts, product managers, and others this is definitely useful for them. Where are they in the loop on developing this SDK? 2. Can you point to end users of the SDK who aren't developers or engineers? That's not to say that someone developing an application that uses the SDK isn't working with non-developers. If they have a story about uploading a file to persistent storage the implementation by the developer might use the SDK. But, us using BDD doesn't help that process. This is really about the people involved in this project and consuming it. It's different from typical client consulting work or general consumer facing production. The audience is different. Can you explain how this technology is useful to this specific audience in a practical way? Thanks, Matt On Fri, Jun 6, 2014 at 12:55 PM, Jamie Hannaford jamie.hannaf...@rackspace.com wrote: Hey all, Sorry for the length of reply - but I want to provide as much information as possible about this topic. Instead of enumerating pros and cons, I want to give a bit of context first (about what “feature stories” actually are), and then respond to some common misconceptions about them. This way, the pros/cons of Behat are a bit more substantiated. Would Behat replace PHPUnit? No - they’re completely different. We’d still use phpunit for unit testing because it’s way better at xunit-like assertions. We’d use behat instead for functional testing - making sure that features work against a production API. Who’s using Behat and is it suitable for us? From what I’ve heard, we’re using it for some projects at Rackspace and possibly some OpenStack projects - but I need to double check that. I’ve reached out to some folks about their experiences with it - so I’ll post the findings when I hear back. What are BDD feature stories? Here’s a link to a fantastic article which explains the benefits of BDD feature stories: http://dannorth.net/whats-in-a-story/ tl;dr: BDD takes the position that you can turn an idea for a requirement into implemented, tested, production-ready code simply and effectively, as long as the requirement is specific enough that everyone knows what’s going on. To do this, we need a way to describe the requirement such that everyone – end-user, contributor, manager, technical lead (in short, anyone interested in using our SDK in their business) – have a common understanding of the scope of the work. You are showing them, in human-readable language, the features of the SDK and what it offers them. The result is that everyone — regardless of proficiency, skill level and familiarity with the codebase — is on the same level of understanding. From this they can agree a common definition of “done”, and we escape the dual gumption traps of “that’s not what I asked for” or “I forgot to tell you about this other thing”. This, then, is the role of a Story. It is a description of a requirement and a set of criteria by which we all agree that it is “done”. It helps us understand and satisfy customer use-cases in a well expressed and clear way. It also helps us track project progress by having well-established acceptance criteria for feature sets. 3 misconceptions about BDD (Inspired by http://www.thoughtworks.com/insights/blog/3-misconceptions-about-bdd) 1. End-users don’t care about this! They want code This is actually a completely misdirected point. The purpose of behat is not to serve as a public-facing repository of sample code. Its actual purpose is twofold: to serve as a functional test suite (i.e. make sure our SDK works against an API), and secondly to serve as a communication device - to codify features in a human-readable way. It’s the role of documentation to explain the concepts of the SDK with detailed code samples. Another good idea is to provide a “samples” folder that contains standalone scripts for common use-cases - this is what we offer for our current SDK, and users appreciate it. Both of these will allow developers to copy and paste working code for their requirements. 2. Contributors don’t want to write these specifications! My response is this: how can you implement a piece of functionality if you
Re: [openstack-dev] [openstack-sdk-php] Transport Clients, Service Clients, and state
about a service. A service client knows about a service but not about moving data. They have their own scope. A transport client is used to move data. Since it's scope is about transporting the different configurations for it are about the different ways an application needs to transport things. If it's scoped at transporting it doesn't need or care to know anything else. It doesn't know state on a service (and state includes a URL to a service). A service client knows about a service. So, for each service you connect to you'd need one. You'd need as many service clients as services you'd connect to. A singleton (forcing just one) would be a bad thing. A 1-to-1 relationship between the two where the transport client knows about state of the service breaks the separation of concerns. They have separate jobs. The number of each you need depends on what's happening in the scope of each space. Does that make sense? Jamie On June 5, 2014 at 6:33:34 PM, Matthew Farina (m...@mattfarina.com) wrote: My opinion is that we create a *new* transport client instance for every service client, not re-use existing instances. What’s your take on this? I'm not in agreement and here is why (with a use case). A transport client is concerned with transporting only. Whether the same one is used for each service or a new one is used for each service doesn't matter. An example of using two transport clients would be a case where an application communicates with two different OpenStack clouds. One is a public cloud and the application communicates through a proxy. A transport client would know how to talk through the proxy to the public cloud. A second OpenStack cloud is a private cloud that is on the same company network. A second transport client would know how to talk to that without communicating through the proxy. The service clients communicating with each cloud would use the appropriate transport client. The mapping of transport client to service client doesn't need to be 1:1 if they operate in different spaces. Only having instances of a transport client as needed decreases the use of resources or the time needed to manage those. If a transport client is only concerned with transporting than what is the need to have more than one per case to transport? - Matt On Thu, Jun 5, 2014 at 12:09 PM, Jamie Hannaford jamie.hannaf...@rackspace.com wrote: I completely agree with you regarding separation of concerns. I also agree with your definitions: a transport client is for managing HTTP transactions, a service client contains all the domain logic for an API service (Swift, Nova, etc.). A service knows nothing about HTTP, a transport client knows nothing about Swift. A transport client is injected into the service client, satisfying the type hint. So any transport client implementing our interface is fine. Up to this point I’m in 100% agreement. The area which I think I misunderstood was the *creation process* of service clients. My take was that you were advocating a shared transport client instance - i.e. a transport client instantiated once, and re-used for every service client. If we did that, there would be global state. My opinion is that we create a *new* transport client instance for every service client, not re-use existing instances. What’s your take on this? Jamie On June 5, 2014 at 5:17:57 PM, Matthew Farina (m...@mattfarina.com) wrote: We've started to talk about the interactions between transport clients, service clients, and state. I've noticed we're not on the same page so I wanted to start a dialog. Here's my starting point... A Transport Client is about transporting data. It sends and receives data. A Service Client handles the interactions with a service (e.g., swift, nova, keystone). A Service Client uses a Transport Client when it needs to transport data to and from a service. When it comes to state, a Transport Client knows about transporting things. That means it knows things like if there is a proxy and how to work with it. A Service Client knows about a service which includes and state for that service. In the realm of separation of concerns, a Service Client doesn't know about transport state and a Transport Client doesn't know about service state. They are separate. A Service Client doesn't care what Transport Client is used as long as the API (interface) is compliant. A Transport Client doesn't care what code calls it as long as it uses the public API defined by an interface. This is my take. If someone has a different take please share it with the reasoning. - Matt Jamie Hannaford Software Developer III - CH [image: experience Fanatical Support] [image: LINE]Tel: +41434303908 Mob: +41791009767 [image: Rackspace] Rackspace International GmbH a company registered in the Canton of Zurich, Switzerland (company identification number CH-020.4.047.077-1) whose registered office
[openstack-dev] [openstack-sdk-php] Transport Clients, Service Clients, and state
We've started to talk about the interactions between transport clients, service clients, and state. I've noticed we're not on the same page so I wanted to start a dialog. Here's my starting point... A Transport Client is about transporting data. It sends and receives data. A Service Client handles the interactions with a service (e.g., swift, nova, keystone). A Service Client uses a Transport Client when it needs to transport data to and from a service. When it comes to state, a Transport Client knows about transporting things. That means it knows things like if there is a proxy and how to work with it. A Service Client knows about a service which includes and state for that service. In the realm of separation of concerns, a Service Client doesn't know about transport state and a Transport Client doesn't know about service state. They are separate. A Service Client doesn't care what Transport Client is used as long as the API (interface) is compliant. A Transport Client doesn't care what code calls it as long as it uses the public API defined by an interface. This is my take. If someone has a different take please share it with the reasoning. - Matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack-sdk-php] Use of final and private keywords to limit extending
Some recent reviews have started to include the use of the private keyword for methods and talk of using final on classes. I don't think we have consistent agreement on how we should do this. My take is that we should not use private or final unless we can articulate the design decision to intentionally do so. To limit public the public API for a class we can use protected. Moving from protected to private or the use of final should have a good reason. In open source software code is extended in ways we often don't think of up front. Using private and final limits how those things can happen. When we use them we are intentionally limiting extending so we should be able to articulate why we want to put that limitation in place. Given the reviews that have been put forth I think there is a different stance. If there is one please share it. - Matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-sdk-php] Transport Clients, Service Clients, and state
My opinion is that we create a *new* transport client instance for every service client, not re-use existing instances. What’s your take on this? I'm not in agreement and here is why (with a use case). A transport client is concerned with transporting only. Whether the same one is used for each service or a new one is used for each service doesn't matter. An example of using two transport clients would be a case where an application communicates with two different OpenStack clouds. One is a public cloud and the application communicates through a proxy. A transport client would know how to talk through the proxy to the public cloud. A second OpenStack cloud is a private cloud that is on the same company network. A second transport client would know how to talk to that without communicating through the proxy. The service clients communicating with each cloud would use the appropriate transport client. The mapping of transport client to service client doesn't need to be 1:1 if they operate in different spaces. Only having instances of a transport client as needed decreases the use of resources or the time needed to manage those. If a transport client is only concerned with transporting than what is the need to have more than one per case to transport? - Matt On Thu, Jun 5, 2014 at 12:09 PM, Jamie Hannaford jamie.hannaf...@rackspace.com wrote: I completely agree with you regarding separation of concerns. I also agree with your definitions: a transport client is for managing HTTP transactions, a service client contains all the domain logic for an API service (Swift, Nova, etc.). A service knows nothing about HTTP, a transport client knows nothing about Swift. A transport client is injected into the service client, satisfying the type hint. So any transport client implementing our interface is fine. Up to this point I’m in 100% agreement. The area which I think I misunderstood was the *creation process* of service clients. My take was that you were advocating a shared transport client instance - i.e. a transport client instantiated once, and re-used for every service client. If we did that, there would be global state. My opinion is that we create a *new* transport client instance for every service client, not re-use existing instances. What’s your take on this? Jamie On June 5, 2014 at 5:17:57 PM, Matthew Farina (m...@mattfarina.com) wrote: We've started to talk about the interactions between transport clients, service clients, and state. I've noticed we're not on the same page so I wanted to start a dialog. Here's my starting point... A Transport Client is about transporting data. It sends and receives data. A Service Client handles the interactions with a service (e.g., swift, nova, keystone). A Service Client uses a Transport Client when it needs to transport data to and from a service. When it comes to state, a Transport Client knows about transporting things. That means it knows things like if there is a proxy and how to work with it. A Service Client knows about a service which includes and state for that service. In the realm of separation of concerns, a Service Client doesn't know about transport state and a Transport Client doesn't know about service state. They are separate. A Service Client doesn't care what Transport Client is used as long as the API (interface) is compliant. A Transport Client doesn't care what code calls it as long as it uses the public API defined by an interface. This is my take. If someone has a different take please share it with the reasoning. - Matt Jamie Hannaford Software Developer III - CH [image: experience Fanatical Support] [image: LINE] Tel: +41434303908Mob: +41791009767 [image: Rackspace] Rackspace International GmbH a company registered in the Canton of Zurich, Switzerland (company identification number CH-020.4.047.077-1) whose registered office is at Pfingstweidstrasse 60, 8005 Zurich, Switzerland. Rackspace International GmbH privacy policy can be viewed at www.rackspace.co.uk/legal/swiss-privacy-policy - Rackspace Hosting Australia PTY LTD a company registered in the state of Victoria, Australia (company registered number ACN 153 275 524) whose registered office is at Suite 3, Level 7, 210 George Street, Sydney, NSW 2000, Australia. Rackspace Hosting Australia PTY LTD privacy policy can be viewed at www.rackspace.com.au/company/legal-privacy-statement.php - Rackspace US, Inc, 5000 Walzem Road, San Antonio, Texas 78218, United States of America Rackspace US, Inc privacy policy can be viewed at www.rackspace.com/information/legal/privacystatement - Rackspace Limited is a company registered in England Wales (company registered number 03897010) whose registered office is at 5 Millington Road, Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be viewed at www.rackspace.co.uk/legal/privacy-policy - Rackspace Benelux B.V. is a company
[openstack-dev] [openstack-sdk-dotnet][openstack-cli-powershell] readme and contributing guide
It would be useful to have top level readme and contributing guides for these two projects. I was going to round out both of these and wanted to know what format would be good to use. Here's what I found. First, a lot of .NET projects don't have top level files. I understand how visual studio treats these and how useful they are in practice for working with and on a project. It doesn't fit well. That being said, when someone encounters one of these projects on the github mirror or elsewhere it would be useful for someone looking to get started. .NET projects on github test to have readmes. The most popular format I found, by far, was to use markdown. If a README.md and CONTRIBUTING.md (as markdown) are ok to go with I'm happy to craft the initial (or latest) versions of these. Markdown seems to fit the most here. RST is something I've not seen used by any .NET projects. Sound good? Other thoughts? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [golang-client] Tentative Design Decisions
The golang-client/openstack-sdk-go is an SDK written in Go to operate against OpenStack REST APIs. The project sits in stackforge and in one of a number of Go clients for OpenStack. This email is here to describe what we're doing and some base design decisions. First, if there are Go clients for OpenStack why is there another effort in Stackforge? The existing clients don't meet all the requirements and there are a number of cases where they fall short. For example, none of the existing clients support multiple REST API versions. They were written against the versions for a particular stack. We need an SDK that handles this cleanly. Ideally, we'd like to pull in working code from existing projects where the license will allow us to do so. Some tentative decisions (speak now if you don't agree). When it comes to directory/package structure we'd use something similar to the way python does it. That is a directory for each service (e.g., identity, compute, etc) with another directory in there for each version. This way we can handle versions like identity v2, v3, and v4 at the same time. Since Go is statically typed it would be useful to have a common interface for base service operations. For example, compute instances need to be created, destroyed, restarted, and a handful of other service. Each API version would/should implement some of these base operations, though in different ways. An interface that implements these would be useful for applications that use the SDK. The transport client would be pluggable and dependency injected but have a same default. It would likely be the built in system to go. But, we need flexibility here. For example, an application may communicate with a public cloud through a proxy while communicating with a private cloud compute endpoint bypassing the proxy. The flexibility to implement that needs to be in place. This means managers for service will be needed in packages so the same manager can be instantiated twice with different transport layers. The Python SDK is using an ORB style for working with services. This isn't something we've seen happen much in Go. Instead we're looking at the manager/resource style setup to be in a manner more common to the language. As I said earlier, where ever we can re-use code from existing projects we'd like to try and do so. Some places there is flexibility to do some. Some other places licenses (such at the GPLv3) will limit code reuse and we're going to pay attention to that. If you have any comments, questions, or want to get involved please jump in. - Matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Openstack access with Java SDKs
Vikas, That's a great question. I was on vacation so it took me a little time to respond. If you use the OpenStack provider in jclouds you should be able to work against OpenStack clouds from different providers. You won't have access to any proprietary extensions. The package that seems to be getting the most development and support is jclouds. See http://developer.openstack.org/ for more details. If you're curious about other languages I can speak to some of those as well. - Matt On Mon, May 5, 2014 at 10:50 PM, Vikas Kokare vikaskok...@gmail.com wrote: I am looking for a standard, seamless way to access OpenStack APIs , most likely using the Java SDKs that are summarized at https://wiki.openstack.org/wiki/SDKs#Software_Development_Kits There are various distributions of Openstack available today. Is this possible using these SDK's to write an application that works seamlessly across distributions? If the answer to the above is yes, then how does one evaluate the pros/cons of these SDK's? -Vikas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-sdk-php] discussion: json schema to define apis
Ken'ichi, thanks for the detail. I just added that summit session to my list to attend. I'm looking forward to it. On Thu, May 1, 2014 at 12:34 AM, Ken'ichi Ohmichi ken1ohmi...@gmail.comwrote: Hi, 2014-04-29 10:28 GMT+09:00 Matthew Farina m...@mattfarina.com: *3. Where would JSON schemas come from?* It depends on each OpenStack service. Glance and Marconi (soon) offer schemas directly through the API - so they are directly responsible for maintaining this - we'd just consume it. We could probably cache a local version to minimize requests. For services that do not offer schemas yet, we'd have to use local schema files. There's a project called Tempest which does integration tests for OpenStack clusters, and it uses schema files. So there might be a possibility of using their files in the future. If this is not possible, we'd write them ourselves. It took me 1-2 days to write the entire Nova API. Once a schema file has been fully tested and signed off as 100% operational, it can be frozen as a set version. Can we convert the schema files from Tempest into something we can use? just FYI Now Tempest contains schemas for Nova API only, and the schemas of request and response are stored into different directories. We can see request schema: https://github.com/openstack/tempest/tree/master/etc/schemas/compute response schema: https://github.com/openstack/tempest/tree/master/tempest/api_schema/compute In the future, the way to handle these schemas in Tempest is one of the topics in the next summit. http://junodesignsummit.sched.org/event/e3999a28ec02aa14b69ad67848be570a Nova also contains request schema under https://github.com/openstack/nova/tree/master/nova/api/openstack/compute/schemas/v3 These schemas are used only for Nova v3 API, there is nothing for v2 API(current) because v2 API does not use jsonschema. Thanks Ken'ichi Ohmichi ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-sdk-php] discussion: json schema to define apis
consequently use this validation error them is completely up to you: you could output it to STDOUT, you could save it to a local log on the filesystem, you could buffer it temporarily in memory. Any API data that does not validate successfully against a schema should not be presented to the end-user. So if a created_date property is returned, that isn't defined in our schema, it should not be populated in the resulting model. The model returned to the end-user would be a simple object that implements \ArrayAccess, meaning that it can be accessed like a simple array. *3. Where would JSON schemas come from?* It depends on each OpenStack service. Glance and Marconi (soon) offer schemas directly through the API - so they are directly responsible for maintaining this - we'd just consume it. We could probably cache a local version to minimize requests. For services that do not offer schemas yet, we'd have to use local schema files. There's a project called Tempest which does integration tests for OpenStack clusters, and it uses schema files. So there might be a possibility of using their files in the future. If this is not possible, we'd write them ourselves. It took me 1-2 days to write the entire Nova API. Once a schema file has been fully tested and signed off as 100% operational, it can be frozen as a set version. Can we convert the schema files from Tempest into something we can use? *4. What would the workflow look like?* I don't really understand what you mean: can you elaborate? For example, when would validation happen? Is that for testing or runtime for use in an application? *5. How does schema files handle business logic?* That's a really great question. I've written a brief write-up here: https://wiki.openstack.org/wiki/OpenStack-SDK-PHP/JSON-schema-business-logichttps://wiki.openstack.org/wiki/OpenStack-SDK-PHP/JSON-schema-business-logic#So_how_can_it_be_done_well.3F I think what you're proposing is that the methods map to API calls. There isn't any logic in these objects that isn't an API call. Jamie From: Matthew Farina m...@mattfarina.com Date: Thursday, April 24, 2014 at 5:42 PM To: Jamie Hannaford jamie.hannaf...@rackspace.com, OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Cc: sam.c...@hp.com sam.c...@hp.com Subject: [openstack-sdk-php] discussion: json schema to define apis Jamie (and whom ever else wants to jump in), It's been proposed to use json schema to describe the API calls rather than code. The operations to perform and what they do would be described rather than coded and then some code would use the schema to know how to act. Others are already doing this. For example, the AWS SDK for PHP. Take their S3 structure as an example https://github.com/aws/aws-sdk-php/blob/master/src/Aws/S3/Resources/s3-2006-03-01.php . The ability to do this goes beyond this one example. It just appears to be something similar to what we're considering. Given this in the scope of PHP I've got a number of questions. Several of these I've compiled while discussing this with others so they don't represent my point of view. Rather, they are just a list of outstanding questions. Since this is a different method for handling the API calls from the other SDKs being built the concept should be really vetted. Here are the questions: 1. Why use json schema rather than other reuse methods? I've discussed the use of json schemas with others and those working on the other languages have not been interested in json schema at the moment. Why do it differently given the context? Note, it might be worth looking at the python SDK which is doing things differently. If I understand it right they are moving aware from using managers and resources all together. 2. How will debugging work in practice? For example, a call is made from behind a proxy. The proxy alters the HTTP headers so the request fails and an exception is thrown. The schema and endpoint are valid. It's something in the middle that changed things. Walking through the code goes through magic methods to handle the schema. How would debugging that work to understand what's happening compared to what was expected. 3. Where would the json schemas for services come from and who would manage them? 4. What would the workflow look like for working with the schemas at both execution time for everyday use and for testing? 5. How would logic happen? Sometimes a request to an API is more than just a request and response. For example, calling to something in object storage where the object does not exist. The transport layer will throw an exception (this goes all the way down to Guzzle throwing one) that needs to be caught and managed. How should cases with some logic like this be handled and easy to understand? Thanks for looking into this. The topic has really sparked my interest. I for one
Re: [openstack-dev] [openstack-sdk-php] discussion: json schema to define apis
Jamie, thanks for going into so much detail.
On Mon, Apr 28, 2014 at 9:28 PM, Matthew Farina m...@mattfarina.com wrote:
While reading this it struck me that we should prioritize the experience
of end-user, that is application developers, over the experience of those
working on the SDK. I don't think we'd ever directly talked about this so I
wanted to take a moment and state it.
What I put in below isn't my full set of questions but I think it's enough
for now.
On Mon, Apr 28, 2014 at 11:34 AM, Jamie Hannaford
jamie.hannaf...@rackspace.com wrote:
Thanks Matt for bringing up these questions - I think having this kind
of discussion is essential for such a big idea. It also helps me clarify my
own thinking towards this issue.
Before I answer, I want to point out that I'm not staunchly for or
against any particular idea. I do think that schemas offer a lot of
advantages over writing user-land code, but I'm more than willing to
abandon the proposal if we all determine there's a stronger and more
compelling alternative.
*1. Why use schemas instead of userland code?*
I've put my answer to this question here:
https://wiki.openstack.org/wiki/OpenStack-SDK-PHP/JSON-schema
Can we look at this from the experience an end user would have? In the
Python SDK they are working on an ORM style system. It's sorta similar to
the system currently in the PHP SDK. For example you could do something
like this in Python,
o = Container.get_by_id('foo').get_object('bar/baz.awesome')
I would imagine something similar in PHP like,
$o = $objectStore-getContainer('foo')-getObject('bar/baz.awesome');
I don't think you can do this using the json schema code I've seen so far.
Can you touch on the experience for developers who are using the library?
For example, the coding style or ability to know what they have access to?
I was just thinking of how magic methods using a schema aren't going to
work for tools that do autocompletion.
I'm curious about blueprints for the schema support. Things on the mailing
list are great. I'm curious about plans and what's in the blueprints. Do
you have any info on that?
If the other SDKs aren't interested in using json schema, wouldn't that be
a lack of consistency?
*2. How will debugging work?*
I'll highlight two conceivable issues which might need debugging. The
first issue is the API rejecting a request for whatever reason (i.e. a
proxy modifying headers); the second issue is when a data structure
returned from the API fails to validate against a particular schema file.
*Issue 1: Malformed requests*
There are two reasons why a request would fail: if an end-user stocks it
with bad data, or if something in the middle deforms it. A very easy
solution to the first problem is using schemas to perform basic parameter
checking before a request is serialized. If we know, for example, that the
API is expecting a particular value - or a particular header - the schema
is in charge of making that happen. Performing basic validation catches
most errors - and debugging is very easy due to the exception thrown. If
you're ever in doubt, you just refer to the schema to see what was
serialized into a request in the same way you do for a concrete class
method.
If something in the middle deforms the request, the API will naturally
reject it. When it comes to debugging this issue, all you need to do is
wrap your original code in a try/catch block and use Guzzle's
BadResponseException to return the API's response. You can easily see
the type of failure through the HTTP status code, and the exact reason why
the request failed. So it doesn't matter where the failure happens - all
that matters is that there's a way to catch and spit out the API's response
and the originating request.
First, this assumes Guzzle. Since we aren't tightly coupled to Guzzle we
can't always assume that. But, for practical purposes we can assume it for
now.
I was curious how things would work in PHP, such as the stack trace. For
magic methods you'll have a call to the magic method and to __call() where
the logic actually sits. In a debugger you'll be able to step through this
just fine.
One thing that may be more difficult is that knowing how the json schema
system works to debug and understand what's going on. How the schemas work
and how something gets translated into a method. Walking through a few
methods that are extended would be less logic to understand in the process.
I'm curious how the debugging experience would be for an end user who
doesn't know the json schema system but is using the library. Out of
curiosity I might try to find some time to sit down with some PHP
developers and see how they handle the debugging experience.
*Issue 2: Incorrect API data *
Say we've defined that a Server has two properties: a name (which is a
string) and metadata (which is an object). If the API returns a name as an
array
Re: [openstack-dev] [openstack-php-sdk] Contributing guidelines
Doug, I have a few concerns with handling the PHP SDK this way. The authors of the novaclient are working on openstack, using python which has common community processes, and there is a bit of documentation covering the commonalities. Contributors to the PHP SDK are likely not going to know how the community works for creating an infrastructure. In PHP there will be some major process differences to fit into the PHP community and be native to PHP. The target audience for the PHP SDK is application developers, not those who write openstack or operate it. A contributors guide who really hits home with this audience is important. There are definitely things we can link off to. I just want to make sure we hit home with our target audiences. On Thu, Apr 24, 2014 at 5:14 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Thu, Apr 24, 2014 at 11:18 AM, Matthew Farina m...@mattfarina.com wrote: Doug, I think contributing to the PHP SDK is going to be a little different than a lot of other OpenStack contributions. For example, these are application developers rather than those who develop or standup OpenStack as infrastructure. There is a lot about OpenStack they don't need to know. We need to take that into account. Not to mention that this is a different language from the services. In my experience, a lot of PHP developers will never jump into contributing in other languages. Especially when it comes to processes, we can be consistent with our established community standards no matter what language we're writing in. The python-novaclient repository has something similar to what I'm suggesting, combining usage info with links to source, the bug tracker, and instructions for using gerrit (http://git.openstack.org/cgit/openstack/python-novaclient/tree/doc/source/index.rst). Doug We've also talked about having the documentation be usage documentation rather than contributing documentation. This is important as SDK users want a single package they download that gives them everything they need to get going with OpenStack. Pointing them to some other place to get started is a barrier to building against OpenStack. This is why I'm hesitant to suggest we do something similar to what you'd see with Swift, Nova, or something else. The context is quite different for someone approaching this project. Shaunak, this sounds like a good idea. Did you want to create a blueprint for it and we can work out the spec for what could or should be listed? Other SDKs in this space do something similar that helps folks get up to speed on the SDK quickly. One addition to your list would be, how to get started contributing to OpenStack. We need to guide folks to the CLA and anything else they need to get going. This would be great for the developer experience of contributing to the SDK. On Thu, Apr 24, 2014 at 10:29 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Wed, Apr 23, 2014 at 2:48 PM, Shaunak Kashyap shaunak.kash...@rackspace.com wrote: Hey PHP SDK folks (although others are welcome to chime in too), I am thinking of adding a CONTRIBUTING.rst to the root of our repo at http://git.openstack.org/cgit/stackforge/openstack-sdk-php/tree/. My immediate, selfish need is to have a single place where we capture any decisions around contribution process so I don’t have to remember them or rehash them often. Longer term I think this would be useful to all potential contributors - in making them feel welcome and less overwhelmed - especially as the project grows. If you think this would be a useful addition, please read on. Putting on a new contributor’s hat, here are some of the questions (in no particular order) that come to my mind when I encounter a new project: 1. What is the overall development process? 2. I see a bunch of directories and files in the source tree. What do these mean? 3. What do I need to setup in my development environment so I can contribute? 4. Are there any coding standards I should adhere to? 5. I'm ready to submit my first patch. What happens next? 6. How do I run the unit tests? 7. How do I run the integration tests? Can you think of more questions, ones you might’ve had in the past perhaps? I imagine the CONTRIBUTING.rst to be comprised of answers to these questions (but perhaps not necessarily in QA format). I realize that some of these answers would overlap with information that already exists elsewhere. We would link to those sources while still giving our contributors a single starting point within the context of our project. Please note that, at this time, I’m just soliciting approval for having a CONTRIBUTING.rst and coming up with the list of questions that it would answer. I am not (yet) looking for us to come up with all the answers and agree on them as a team. Thoughts? These are all good questions for helping out new and infrequent contributors. Many of the other projects answer
[openstack-dev] [openstack-sdk-php] discussion: json schema to define apis
Jamie (and whom ever else wants to jump in), It's been proposed to use json schema to describe the API calls rather than code. The operations to perform and what they do would be described rather than coded and then some code would use the schema to know how to act. Others are already doing this. For example, the AWS SDK for PHP. Take their S3 structure as an example https://github.com/aws/aws-sdk-php/blob/master/src/Aws/S3/Resources/s3-2006-03-01.php. The ability to do this goes beyond this one example. It just appears to be something similar to what we're considering. Given this in the scope of PHP I've got a number of questions. Several of these I've compiled while discussing this with others so they don't represent my point of view. Rather, they are just a list of outstanding questions. Since this is a different method for handling the API calls from the other SDKs being built the concept should be really vetted. Here are the questions: 1. Why use json schema rather than other reuse methods? I've discussed the use of json schemas with others and those working on the other languages have not been interested in json schema at the moment. Why do it differently given the context? Note, it might be worth looking at the python SDK which is doing things differently. If I understand it right they are moving aware from using managers and resources all together. 2. How will debugging work in practice? For example, a call is made from behind a proxy. The proxy alters the HTTP headers so the request fails and an exception is thrown. The schema and endpoint are valid. It's something in the middle that changed things. Walking through the code goes through magic methods to handle the schema. How would debugging that work to understand what's happening compared to what was expected. 3. Where would the json schemas for services come from and who would manage them? 4. What would the workflow look like for working with the schemas at both execution time for everyday use and for testing? 5. How would logic happen? Sometimes a request to an API is more than just a request and response. For example, calling to something in object storage where the object does not exist. The transport layer will throw an exception (this goes all the way down to Guzzle throwing one) that needs to be caught and managed. How should cases with some logic like this be handled and easy to understand? Thanks for looking into this. The topic has really sparked my interest. I for one am really curious about the practicalities of using json schema and the developer experience around it. - Matt Farina ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-sdk-php] Questions about user-facing documentation
Shaunak, these are some good questions. Input from the docs team would be useful for some of these as well. On Wed, Apr 16, 2014 at 10:06 PM, Shaunak Kashyap shaunak.kash...@rackspace.com wrote: Hi folks, As part of working on https://blueprints.launchpad.net/openstack-sdk-php/+spec/sphinx-docs, I’ve been looking at http://git.openstack.org/cgit/stackforge/openstack-sdk-php/tree/doc. Before I start making any changes toward that BP, however, I wanted to put forth a couple of overarching questions and proposals to the group: 1. Where and how should the user guide (i.e. Sphinx-generated docs) be published? For this I'll go by example first. If you look at something like the OpenStack Client docs (http://git.openstack.org/cgit/openstack/python-openstackclient/tree/doc) you'll see they are currently published to http://docs.openstack.org/developer/python-openstackclient/. The same is true of other projects as well. I'm not sure this is the ideal place but it is where things are published to now. There has been talk of producing a developers.openstack.org. The initial proposed content for that currently resides at api.openstack.org. If a more detailed portal comes together that would be a good place for this. I know there’s http://docs.openstack.org/. It seems like the logical place for these to be linked off of but where would that link go and what is the process of publishing our Sphinx-generated docs to that place? 2. How should the user guide(s) be organized? If I were a developer, I’m probably looking to use a particular OpenStack service (as opposed to learning about the PHP SDK without a particular orientation). So I propose organizing the PHP SDK user guide accordingly: as a set of user guides, each showing how to use the OpenStack PHP SDK for a particular service. Of course, Identity is common to all so it’s documentation would somehow be included in each user guide. This is similar to how OpenStack organizes its REST API user guides - one per service (e.g. http://docs.openstack.org/api/openstack-object-storage/1.0/content/). If you take a look at the general SDK development page (https://wiki.openstack.org/wiki/SDK-Development) there is a description of the target audience. This target audience is a little different from most of the other documentation so we should take that into account. We shouldn't expect a user of the SDK to understand the internals of OpenStack or even the names such as swift, nova, etc. An application developer will likely know little about OpenStack other than it's a cloud platform. The SDK should introduce them to OpenStack with the limited amount of knowledge a dev would need to know. From here I like your idea of a section for each service (e.g., identity). This make sense. Further, within each user guide, I propose ordering the content according to popularity of use cases for that service (with some other constraints such as introducing concepts first, grouping similar concepts, etc.). This ensures that the reader can get what they want, from their perspective. Particularly, beginners would get what they came for without having to read too far into the documentation. As an example, I think http://git.openstack.org/cgit/stackforge/openstack-sdk-php/tree/doc/oo-tutorial.md does a fine job of walking the user through common Object Store use cases. I would just extend it to gradually introduce the user to more advanced use cases as well, thereby completing the user guide for Object Store. Great. We want to help application developers get up to speed quickly. They're concerned with their app. I like the idea of common use cases near the front. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [OpenStack-SDK-PHP] First IRC meeting
To make the OpenStack APIs easily accessible via PHP, the most popular server side language of the web, we've been working on a PHP SDK in Stackforge. This week we are going to have our first IRC meeting on Wednesday. If you're interested in a PHP SDK please come join in the discussion. More information is available on the Wiki at https://wiki.openstack.org/wiki/Meetings#PHP_SDK_Team_Meeting ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [python-openstacksdk] First meeting scheduled
I would like to point out that the unified SDK has a different target audience from the existing clients. Application developers who want to work with OpenStack APIs are far different from those who are trying to build OpenStack. For example, application developers don't know what keystone, nova, or swift are and these names will confuse them creating a barrier for them to build great things against OpenStack. The language usage matters in a good SDK. Given the different target audiences there's room for overlap when trying to create great experiences for two segments. On Wed, Feb 12, 2014 at 10:32 AM, Jesse Noller jesse.nol...@rackspace.comwrote: On Feb 12, 2014, at 9:07 AM, Ed Leafe e...@openstack.org wrote: On Feb 11, 2014, at 3:30 PM, Jesse Noller jesse.nol...@rackspace.com wrote: I did propose in the original thread that we join efforts: in fact, we already have a fully functioning, unified SDK that *could* be checked in today - but without discussing the APIs, design and other items with the community at large, I don't think that that would be successful. [1] https://github.com/openstack/oslo-incubator/tree/master/openstack/common/apiclient While there are some differences, this library shares the same client/manager/resource class design and flow as the work we've done. IOW, I think the approaches are more alike than different, which is encouraging. -- Ed Leafe The projects are not exclusive to one another - they are completely complimentary. And as of yet, we have not solidified the client/manager/resource structure - we should talk about that at the meeting next week. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Ugly Hack to deal with multiple versions
It would really be nice to be able to hit an endpoint like https://hostname:35357/ and get a list of available versions back. It would make discovery easier. But, for scripting languages like PHP this would add a lot of extra http requests unless you cache the location between page views/requests. We'd need to be smart about it. I would mind the both and approach. if /v3 is appended it's smart enough in an SDK to know what to do. If it's not there it's smart enough to hit the top level where a list might be available of what's supported. On Tue, Feb 4, 2014 at 2:55 PM, Jesse Noller jesse.nol...@rackspace.comwrote: On Feb 4, 2014, at 1:28 PM, Sean Dague s...@dague.net wrote: On 02/05/2014 01:50 AM, Jesse Noller wrote: On Feb 4, 2014, at 10:31 AM, Sean Dague s...@dague.net wrote: On 02/05/2014 01:09 AM, Dean Troyer wrote: On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack the version. dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. I think the difference is this is the first API we have actually tried to deprecate and we don't have the option to hide it in an updated SC endpoint. The service catalog has hidden a lot of this pain for other services because the clients generally can use whatever endpoint the SC gives it. a) Version discovery needs to be rationalized across the services. We've talked about this at summits before, and proposals have been written. And here we are. We'll do it again in Atlanta, hopefully for the last time. b) Define a common structured endpoint and let the client assemble the components into the final URL. If the service catalog had a base URL for compute, and a list of versions, and the additional bits to be appended the client could make an intelligent choice and assemble the endpoint. It isn't like the client doesn't already have to know how the REST URLs are constructed. b-alt) Stop putting things like tenant IDs in the SC. This has the same issue as the auth URL in how to do this without instantly breaking the existing clients. Ok, much clearer now to me (though I'll still claim jetlag for some bits not sinking in). I think a really important thing to keep in mind is any solution that's implemented client side, is something that all the other OpenStack SDKs are going to have to implement as well. So an ugly hack isn't just python-keystone... and be done. It's also just hoisted doing that ugly hack on the php / go sdk teams, jclouds, deltacloud, etc. Something they may not be aware is going to break them, or their users. Do we have official openstack PHP / go SDKs? Official is a strong word, but we do have stackforge teams active on it:
Re: [openstack-dev] a common client library
It seems we have two target audiences here. Developers who work on OpenStack and developers who write apps to use it. In the long run I think we need to optimize for both of these groups. If we want developers to write applications to use OpenStack in python we likely need a common python SDK. Note, I'm not a fan of the term client because it's not the common language for this group of developers. On Fri, Jan 17, 2014 at 10:26 AM, John Dennis jden...@redhat.com wrote: Keeping them separate is awesome for *us* but really, really, really sucks for users trying to use the system. I agree. Keeping them separate trades user usability for developer usability, I think user usability is a better thing to strive for. I don't understand how multiple independent code bases with a lot of overlapping code/logic is a win for developers. The more we can move to single shared code the easier code comprehension and maintenance becomes. From a software engineering perspective the amount of duplicated code/logic in OpenStack is worrisome. Iterating towards common code seems like a huge developer win as well as greatly enhancing robustness in the process. -- John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
