Right now, this procedure is completely irrelevant for this discussion: do you know right now how many labs repos would we have? No.
Also that procedure could seem a long one but all those actions are likely to happen very separated in time. It will take time since an api/abstraction starts until it is in a good shape to be promoted, so I wouldn't take that procedure as a step by step one to follow. How often do we promote apis or providers? Do we really need a such a step by step procedure? We have done this many times in the past and there was no issue about how to do it. I think that the peocedure (which I also consider unnecessary) is irrelevant for this discussion. Let's focus on the subject of the thread and elaborate on that. El 16/12/2014 20:50, "Zack Shoylev" <zack.shoy...@rackspace.com> escribió: > Let's work out some examples about promoting APIs and how the repos should > work. > > Example: I have a provider-specific API that extends the compute > abstraction in a meaningful way. > Currently I would have to: > 1. Make a PR for the API in one of the labs > 2. Make a PR for the abstraction in jclouds/jclouds > 3. (Same as 1. in a way) Make a PR in other labs repos for other providers > (as it is likely new abstraction features would have to be supported in > other providers - some of these might be created by other people, obviously > - or these features might already be supported) > 4. Review and merge all of these. > 5. Make PRs to promote the provider-specific APIs to jclouds/jclouds and > merge those. > 6. Make PRs to clean up labs repos. > Does this make sense? Am I missing something? How can we improve this? > > ________________________________________ > From: Ignasi Barrera [n...@apache.org] > Sent: Monday, December 15, 2014 3:22 PM > To: dev@jclouds.apache.org > Subject: Re: [DISCUSS] What is the criteria for something to be in the > jclouds/jclouds repo? > > I can agree that giving full control over provider specific APIs can > be a good thing, but always as a secondary goal. As you say, > abstractions and working on the missing ones should be the high > priority. Adding provider specific features to complete the > abstractions is also a common sense thing; and at this concrete point > in time, keep growing or promoting the APIs that are not related to > any abstraction don't benefit the project as a whole. > > I think the entire point of this discussion is to set up a baseline > that allows us to "move forward". To do that, we need to have a common > criteria and a common vision of which should be the next steps to > evolve the project, and it seems we agree that the high priority > should be to work on the abstractions and focus on them. > > This means that specially the PMC and committers should work to make > this happen, and focus on this high priority tasks. Setting a rule > that we need an abstraction to promote the APIs will help us have > under control what we promote and help us focus on the tasks we have > defined as high priority. If we keep putting our major efforts in > things that are not the high priority (apis not related to an > abstraction), we won't achieve the main purpose of "moving forward". > > This is not black or white, and I don't think "we will never promote > an API that doesn't have an abstraction" is an statement we should do. > However, at this point in time, we should *really enforce* any > promoted provider and API to fit in an abstraction. We have limited > resources, and we need to focus in the high priority tasks. If we > don't dedicate these few resources to the high priority tasks, we > won't succeed. > > > So, "will having an abstraction be a mandatory requirement forever and > ever?" I'd say No. > But that will only make sense if we work *starting today* together on > the APIs that actually fit on those abstractions, and start building > and discussing the ones that could be added. We should prioritise > that, and make that a *real* thing, leaving the unrelated > apis/providers aside. Otherwise we will be at the same point and won't > be able to move forward, regardless of what we promote. > > > > > > On 15 December 2014 at 18:49, Zack Shoylev <zack.shoy...@rackspace.com> > wrote: > >>> Every single API in jclouds must be aligned with the project's > purpose, and > > that means *portability*. > > > > jclouds also tries to do portability "while giving you full control to > use cloud-specific features." > > > >>> if someone contributes a patch adding support for Google Drive > > > > I was not part of that discussion, but it should be possible to add a > blobstore abstraction for this, no? > > > >>> So, once again, jclouds is *not* a cloud aggregator for any API > labelled > > "cloud". It is about portable code, and every single API and provider > > should meet the project goals. > > > > jclouds allows you to write portable applications, but also allows you > the flexibility to use cloud-specific features. From a user perspective, > you need to have both. You want to be able to build infrastructure using > high-level code that works with all the clouds, but occasionally you will > need the flexibility to do provider-specific things (even just because this > is how providers differentiate themselves). > > > > > > I wanted to focus on the user perspective for this discussion. In the > end, users want things to work well, with less code, and with whatever > providers they choose. > > Because jclouds is modular, users don't care if jclouds "is not an API > aggregator of any API that is labelled "cloud"." - users want jclouds to > work well for whatever providers and clouds they choose. > > As such, figuring out the criteria of what should be in jclouds/jclouds > is not really something that affects users directly, but has to be done to > make development easier, saner, and to reduce current and future technical > debt. > > If we had infinite resources, it would make sense, for users, to include > and release all cloud APIs and their abstractions (where such abstractions > make sense) as part of jclouds. > > > > Part of this is about balance. Currently we have too many cloud-specific > APIs that could be abstracted, but are not. Clearly some of the > high-priority work in jclouds is working on abstractions. That's the case > at the moment. It might be a knee-jerk reaction to limit jclouds just to > abstractable APIs because of this situation, when in the future we might > also want to give users the flexibility to also use provider-specific > features or even provider-specific APIs that might be useful when used > together with higher-level, more abstract code. > > > > This is our current mission statement: > > "Apache jclouds® is an open source multi-cloud toolkit for the Java > platform that gives you the freedom to create applications that are > portable across clouds while giving you full control to use cloud-specific > features." > > > > Should this be morphed into: > > "Apache jclouds® is an open source multi-cloud toolkit for the Java > platform that gives you the freedom to create applications that are > portable across clouds"? Is this better for users? > > > > Thanks, and let's keep the discussion going. Figuring things out will > help both contributors and users. > > > > ________________________________________ > > From: Ignasi Barrera [n...@apache.org] > > Sent: Wednesday, December 10, 2014 12:59 AM > > To: dev@jclouds.apache.org > > Subject: RE: [DISCUSS] What is the criteria for something to be in the > jclouds/jclouds repo? > > > > I'd never do point 2 and would remove those APIs. As I've said several > > times in the thread, jclouds is not an API aggregator of any API that is > > labelled "cloud". > > > > Every single API in jclouds must be aligned with the project's purpose, > and > > that means *portability*. It does not make sense to keep adding APIs that > > don't fit in the project's purpose. If we do that we would be facing the > > exact same issues we are facing now (I mentioned them in my previous > email) > > regardless of the number of repos we use. > > > > To put a concrete example: jclouds is *not* the Rackspace SDK nor is the > > OpenStack java SDK. Perhaps Rackspace adopts jclouds as its SDK, but > > jclouds isn't the Rackspace SDK. And this difference is important. > Perhaps > > not every single API in Rackspace makes sense in jclouds, as it would > > hardly be modelled in a portable way. In that case, we shouldn't add it, > as > > it does not benefit jclouds as a whole and doesn't serve the jclouds > > purpose. > > > > Let's take another example: if someone contributes a patch adding support > > for Google Drive (which was discussed in the ML or IRC), providing an API > > to store documents, should we merge it? That API does not fit in the > > BlobStore abstraction as it does not manage blobs (raw object data) but > > documents. The answer is no, because jclouds is not about that (ir is > just > > a single, isolated API to store documents out there). Without alignment > > with what jclouds targets, we shouldn't add it. > > > > So, once again, jclouds is *not* a cloud aggregator for any API labelled > > "cloud". It is about portable code, and every single API and provider > > should meet the project goals. If not, we should not add them. > > El 10/12/2014 01:54, "Zack Shoylev" <zack.shoy...@rackspace.com> > escribió: > > > >> If we take this to its logical conclusion, it sounds like we can > separate > >> APIs into two categories: > >> > >> 1. Those that have an abstraction or can be abstracted in the future (to > >> be in jclouds/jclouds). Most APIs would be in here. > >> 2. Those that provide provider-specific features only, and cannot be > >> abstracted. There should only be a few here, but this means a few per > >> provider, so overall this category might contain a large number of APIs. > >> > >> If we were to organize the repos using the two categories above, we can > do > >> something like (suggestion): > >> > >> 1. jclouds/jclouds (Same as now, with more APIs - including beta APIs - > >> promoted. Regular release schedule) > >> 2. jclouds/providers (All provider-specific APIs that will never be > >> abstracted. Release together with 1. This also needs a better name, as > it > >> only contains APIs that are not abstractable). > >> 3. jclouds/labs (very experimental only - will not be released) > >> > >> Note that I am strongly in favor of fewer repos for us to manage. Maybe > we > >> should not even have 2. > >> > >> What would you change in the description above? > >> Thanks! > >> -Zack > >> ________________________________________ > >> From: Ignasi Barrera [ignasi.barr...@gmail.com] > >> Sent: Monday, December 08, 2014 4:19 PM > >> To: dev@jclouds.apache.org > >> Subject: Re: [DISCUSS] What is the criteria for something to be in the > >> jclouds/jclouds repo? > >> > >> 1. So once an api meets the criteria of having a steward, being > >> modern, and having a proper test suite, it can immediately be > >> promoted? > >> > >> I really think that those APIs need to have an abstraction, or a > >> potential abstraction that is being developed, but something real, > >> tangible, that ensures that it is aligned with the essence of what > >> jclouds is. > >> > >> 2. Can you elaborate on this point? Did you have some criteria in mind > >> for what “makes sense” means? > >> > >> There are several features more than one cloud provider offers > >> (networking, load balancers and databases, are the obvious examples) > >> that could have an abstraction. Not every feature in every cloud > >> provider may make sense in others, and thus are not suitable to have > >> an abstraction. In that case, we should be asking ourselves what value > >> adding such an "independent" API gives to jclouds. We have provider > >> specific APIs to *complete* the functionality "offered by the > >> abstractions", but I really don't think is a good idea to add "entire > >> independent apis" that won't have an abstraction because they're too > >> specific to a single cloud provider. When I say "makes sense" I'm > >> saying to ask ourselves if that API really makes sense in the global > >> jclouds picture. If it makes sense, let's work on getting those > >> features available in a portable way to other providers too. If that > >> doesn't make sense, then that API doesn't make sense to jclouds. > >> > >> 3. Are you saying that if there’s an api that would never be a good > >> fit for abstraction, that it has no chance of being promoted? > >> > >> Yes. I really believe we should be doing that. As I said, jclouds is > >> not an API placeholder to every single API that qualifies itself as > >> "cloud". jclouds is all about abstractions, and about providing users > >> a way to write portable code. Of course we have provider specific > >> features, but as I mentioned before, those should serve the purpose of > >> completing the abstractions, but never be independent and standalone > >> providers/apis. That has proven to be difficult to maintain. > >> considerably increase the codebase complexity and make it more > >> difficult to evolve jclouds itself. > >> > >> 4. Are you saying that if an abstraction doesn’t happen in some amount > >> of time, that any apis related to that lack-of-abstraction would be > >> removed? > >> > >> Well, this is open source, and "time" is relative. The complexity of > >> abstractions can be very different: databases seems to be a simple, > >> but networking can be really complex. What I'm saying is that we > >> should *work* on those abstractions and *work in a direction to make > >> those abstractions" something real. But we can't just keep adding more > >> "satellite" APIs and forget about the rest. We should be thinking > >> about abstractions and woking on them, and that's what matters, not > >> time. Take the database APIs as an example. It has been commented more > >> than once that it would be nice to have an abstraction. What actions > >> have been taken in that direction? None. Any proposal? Nope. That's > >> what we have to change. > >> > >> > >> > >> On 8 December 2014 at 16:01, Everett Toews <everett.to...@rackspace.com > > > >> wrote: > >> > On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <n...@apache.org> wrote: > >> > > >> >> This said, the absence of an abstraction (take network or databases > as > >> >> an example) shouldn't be a blocker. Abstractions can be proposed in > >> >> the mailing list, and even better discussed in a PR! We are all > >> >> looking to improve jclouds to make it better for the users, and that > >> >> means making it more portable. Let's work on adding an abstraction > >> >> when that makes sense! But things have to be *real* and that work has > >> >> to be done. Without it, it does not make sense to promote/keep such > >> >> apis/providers. > >> > > >> > I think having everything in jclouds/jclouds could possibly work but > we > >> would need to be more specific about the above. Otherwise I’m concerned > >> we’d wind up in the same place 6 months down the road. > >> > > >> > If this were to be the path we go down, I have some clarifying > >> questions. These are open questions for anyone to answer. > >> > > >> > “the absence of an abstraction shouldn’t be a blocker” > >> > > >> > 1. So once an api meets the criteria of having a steward, being > modern, > >> and having a proper test suite, it can immediately be promoted? > >> > > >> > > >> > “Let's work on adding an abstraction when that makes sense!” > >> > > >> > 2. Can you elaborate on this point? Did you have some criteria in mind > >> for what “makes sense” means? > >> > > >> > > >> > "it does not make sense to promote/keep such apis/providers.” > >> > > >> > These are two different concerns and should be broken out. > >> > > >> > "it does not make sense to promote such apis/providers.” > >> > > >> > 3. Are you saying that if there’s an api that would never be a good > fit > >> for abstraction, that it has no chance of being promoted? > >> > > >> > "it does not make sense to keep such apis/providers.” > >> > > >> > 4. Are you saying that if an abstraction doesn’t happen in some amount > >> of time, that any apis related to that lack-of-abstraction would be > removed? > >> > > >> > > >> > Everett > >> >
