Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
> On 22 Jun 2018, at 05:14, Vincent S Hou wrote: > > Great thanks to folks with votes and the comments. Wow, a lot happened on my travel day! > As a recap of current replies we have received, we have opened a list of > issues to be fixed for OpenWhisk in the coming release or further releases: I’m happy with items 1-5. As long as the ASF incubator people are happy with a release that doesn’t have the org.apache.openwhisk.* package name, it all seems fine. > Regarding how many repositories we are going to release, we decided to > continue with the release of 13 repositories, after my discussions with many > OpenWhiskers. All the 13 repos by far are great intelligent assets, which > have been evolving during the past months or even years. FWiW, I strongly disagree with this. Bertrand took a fairly cursory look over the first attempt at a release and came up with a laundry list of items to be addressed - none of which were related to the operation of the code itself or even the build process. It’s reasonable to assume that when it goes to the Incubator people, they are going to have another list of items to address that are again nothing to do with the operation of the code. It seems to me that it would be much easier and *polite* to get all the way through to a release tarball on the Apache servers with a single component that’s reasonably easy for the Incubator people to assess and check that we’ve got everything right. It really doesn’t matter what it is as it’s all about the release process details. Rodric suggested wskdeploy or the GoSDK. Either would work really well as they are small and easily buildable. I see no reason why once we successfully get the first tarball onto the Apache servers, we can’t start rolling the “big” product (the 13 inter-related tarballs) the following day as 0.9.1. If we really want 0.9.0 to be the full caboodle, then, we can do the “get-our-ducks-in-row” release of wskdeploy as 0.8.0. Regards, Rob -- (“-ra” just looks wrong!)
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Great thanks to folks with votes and the comments. As a recap of current replies we have received, we have opened a list of issues to be fixed for OpenWhisk in the coming release or further releases: 1. Add the tutorial for 0.9.0 to build and deploy locally with source code https://github.com/apache/incubator-openwhisk-release/issues/197, we will resolve it for 0.9.0 2. Add the instruction on how to verify the license header for each valid source code file https://github.com/apache/incubator-openwhisk-release/issues/196, we will resolve it for 0.9.0 3. Add scripts to make download, unzip and installation of source code easier https://github.com/apache/incubator-openwhisk-release/issues/198, we will resolve it for 0.9.0 4. Add the instruction to the private key and credentials https://github.com/apache/incubator-openwhisk/issues/3800, we will resolve it for 0.9.0 5. Renaming the package from whisk.* into org.apache.openwhisk.* https://github.com/apache/incubator-openwhisk/issues/3797 We need to defer it for next release, since all the repos depend on the naming convention so far. It takes great effort and collaboration, because it affects existing offerings. Regarding how many repositories we are going to release, we decided to continue with the release of 13 repositories, after my discussions with many OpenWhiskers. All the 13 repos by far are great intelligent assets, which have been evolving during the past months or even years. They all play important roles to make openwhisk complete, and users/contributors are longing to see them distributed. Contributors in OpenWhisk have done great work to all of them and we are confident with source code, and there will be more openwhisk repositories in future, as openwhisk attracts more contributors with good ideas. Based on my experiences with cooperating with people from Apache, I also believe that Apache members are passionate about technologies and desire to try out new projects, by fulfilling their duties with their evaluations and feedback. Except the issues we have above, does anyone have any other concerns we need to take into account for the 0.9.0 release? If so, this is the chance to raise it; if not, we shall proceed the, after we made the minor fixes to the above listed issues. Thank you. Best wishes. Vincent Hou (侯胜博) Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM Cloud Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com, Phone: +1(919)254-7182 Address: 4205 S Miami Blvd (Cornwallis Drive), Durham, NC 27703, United States -"Matt Rutkowski" wrote: - To: dev@openwhisk.apache.org From: "Matt Rutkowski" Date: 06/21/2018 12:08PM Cc: "Vincent S Hou" Subject: Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating Hi Bertrand, ...Plus a single repo. source is not usable by itself and its build dependent on the other parts as I mentioned earlier... >>Right, it if cannot be built that's a problem - but if you say that I >>suppose there's a build order that must be followed? >>If that's correct those overall build instructions should be included >>with the set of release archives. As required, the main openwhisk README (and supporting) docs include instructions on how to build (and tooling that makes it quite easy). We can open an issue to better document suggest manual build order. Will talk to Vincent to see if he has time today as I am leaving soon to return Monday. -mr From: Bertrand Delacretaz To: dev@openwhisk.apache.org Date: 06/21/2018 11:00 AM Subject:Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating Hi Matt, On Thu, Jun 21, 2018 at 5:27 PM Matt Rutkowski wrote: > ...Are you saying you believe the Incubator PMC > will fail us strictly due to having 13 tgz/tar files vs. 1 for a first > release?... I don't know (and someone's welcome to ask on the general@incubator.a.o list to find out), but that looks unusual to me, and more work for reviewers that might need to unpack 13 archives to find similar issues in several of them. That's why I focused on just one archive here, and found a few interesting things already - the other 12 archives have not been useful for my initial review. > ...Are you saying they need to be "eased into the concept" because we will > have 13 (now and more eventually); at some point the board will be exposed > to multiples... No, it's just a practical question and fairness for the reviewers, where multiple archives might not say much more than one about the readiness of OpenWhisk to make Apache Releases. The ASF Board is not involved with releases, it's just the Incubator PMC in this case, for a podling. > ...Plus a single repo. source is not usable by itself and its build dependent > on the other parts as I mentioned earlier... Right, it if cannot be built that's a problem - but if you say that I suppose there's a build order that must be followed? If that's correct those overall build
Re: Let's maintain and test our Swagger spec
Thanks Ben for looking into this, having a good API doc/spec and matching tests is very need it. +1 -cs On Thu, Jun 21, 2018 at 2:25 PM Ben Browning wrote: > Our Swagger spec > ( > https://github.com/apache/incubator-openwhisk/blob/92a64c291156a2cd3d6b304babc2a193a46d0699/core/controller/src/main/resources/apiv1swagger.json > ) > is incomplete and doesn't accurately reflect the actual Controller > API. It's manually updated without a full test suite which means it's > easy for changes in the API to happen without the spec getting > updated. > > An accurate Swagger spec will not only better document the OpenWhisk > API but also allow autogenerated clients in multiple languages to > supplement or eventually replace some of the existing client > implementations we have today. It also paves way for future compatible > server implementations, whether they be rewrites of the existing > Controller or stub test harnesses to facilitate end-to-end testing on > a developer's laptop. > > As I'm already working with autogenerating code from the Swagger spec > for other purposes, I'm happy to take the lead on this effort. I'd > like to take a two-pronged approach for a test suite: > > * Generate a server stub from the spec and ensure the wsk CLI can > communicate with it. > > * Generate a client stub from the spec and ensure it can communicate > with the existing API. > > There are a lot of details to figure out from those two statements. > And, this approach won't guarantee 100% correctness of the spec. The > only way to do that would be to generate all supported clients and the > Controller API from the spec. But, this should get us started in the > right direction. > > If anyone's gone down this path before and has some wisdom to share, > please speak up! > > Thanks, > > Ben >
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
+1 for 0.9.0-incubating time to write a book on it :) -- Michele Sciabarra openwh...@sciabarra.com On Thu, Jun 21, 2018, at 8:24 PM, James Thomas wrote: > +1 Release as Apache OpenWhisk 0.9.0-incubating. > > Good work on this everyone. Time to get the ready > > On 21 June 2018 at 17:35, Priti Desai wrote: > > > +1 for the release, its been a lot of hard work from the team, great job > > Matt, Vincent, and Daisy! > > > > Cheers > > Priti > > > > > > > -- > Regards, > James Thomas
Let's maintain and test our Swagger spec
Our Swagger spec (https://github.com/apache/incubator-openwhisk/blob/92a64c291156a2cd3d6b304babc2a193a46d0699/core/controller/src/main/resources/apiv1swagger.json) is incomplete and doesn't accurately reflect the actual Controller API. It's manually updated without a full test suite which means it's easy for changes in the API to happen without the spec getting updated. An accurate Swagger spec will not only better document the OpenWhisk API but also allow autogenerated clients in multiple languages to supplement or eventually replace some of the existing client implementations we have today. It also paves way for future compatible server implementations, whether they be rewrites of the existing Controller or stub test harnesses to facilitate end-to-end testing on a developer's laptop. As I'm already working with autogenerating code from the Swagger spec for other purposes, I'm happy to take the lead on this effort. I'd like to take a two-pronged approach for a test suite: * Generate a server stub from the spec and ensure the wsk CLI can communicate with it. * Generate a client stub from the spec and ensure it can communicate with the existing API. There are a lot of details to figure out from those two statements. And, this approach won't guarantee 100% correctness of the spec. The only way to do that would be to generate all supported clients and the Controller API from the spec. But, this should get us started in the right direction. If anyone's gone down this path before and has some wisdom to share, please speak up! Thanks, Ben
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
+1 Release as Apache OpenWhisk 0.9.0-incubating. Good work on this everyone. Time to get the ready On 21 June 2018 at 17:35, Priti Desai wrote: > +1 for the release, its been a lot of hard work from the team, great job > Matt, Vincent, and Daisy! > > Cheers > Priti > > -- Regards, James Thomas
Re: [Release] Preparing the release of OpenWhisk
Can we also write up the release process in markdown and store in in the repo to help future release managers (unless Vincent wants to do it forever :))? On 20 June 2018 at 20:59, Vincent S Hou wrote: > Give me the honor to the initiative as the first release manager of > OpenWhisk. > The first version is named after "0.9.0-incubating", based on the semantic > version 2.0. > I am preparing the email for VOTE now. I will send out the email by the > end of today. > > > > Best wishes. > Vincent Hou (侯胜博) > > Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM > Cloud > > Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com, > Phone: +1(919)254-7182 > Address: 4205 S Miami Blvd (Cornwallis Drive), Durham, NC 27703, United > States > > -"Matt Rutkowski" wrote: - > To: dev@openwhisk.apache.org > From: "Matt Rutkowski" > Date: 06/20/2018 03:05PM > Subject: Re: [Release] Preparing the release of OpenWhisk > > Agree, Vincent should be first Release Manager. Do we have a champagne > bottle somewhere? > > Kind regards, > Matt > > > > From: Carlos Santana > To: dev@openwhisk.apache.org > Date: 06/20/2018 01:36 PM > Subject:Re: [Release] Preparing the release of OpenWhisk > > > > Vincent, > > If it's not already clear :-), I think should do the honors, and be > release > manager for the first release :-) > I'm out most of the month of July (vacation). But will volunteer to do a > release in August > > Thread to dev list for vote should have the following Subject "[VOTE] > Release Apache OpenWhisk (incubating) version 0.9.0" > And include the details of the location of the RC, and the instructions > for > voting including the deadline of 72 hours. > > Release Candidate 1 should be located in > https://dist.apache.org/repos/dist/dev/incubator/openwhisk/0.9.0/RC1/ > > It's a requirement that artifacts need to include the string "incubating" > as part of the version. > Since we are trying to use semantic versioning "incubating" should be at > the end. > > dev/incubator/openwhisk/0.9.0/RC1/openwhisk-0.9.0-incubating.tar.gz > dev/incubator/openwhisk/0.9.0/RC1/openwhisk-apigateway-0.9. > 0-incubating.tar.gz > dev/incubator/openwhisk/0.9.0/RC1/openwhisk-cli-0.9.0-incubating.tar.gz > ... > > We should remove "incubator", and put "incubating" at the end. > Also I would remove "sources" from the name. Only sources are distributed > on apache servers. > After graduation, we stop using "-incubating" > > -cs > > On Wed, Jun 20, 2018 at 1:39 PM Vincent S Hou wrote: > > > So far, we can formally name the first version incubator-0.9.0 to > indicate > > the incubator status as well, and use subversion like rc1, rc2, etc, > before > > moving the artifacts to the final release SVN URL. > > > > For incubator-0.9.0-rc1, the package of source code openwhisk in the dev > > SVN URL is named after > openwhisk-apigateway-incubator-0.9.0-sources.tar.gz > > under the folder of apache-openwhisk-incubator-0.9.0-rc1. > > > > Shall we include the name "incubator" as part of the version name? Or it > > does not sound attractive. > > > > > > Best wishes. > > Vincent Hou (侯胜博) > > > > Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM > > Cloud > > > > Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com, > > Phone: +1(919)254-7182 <(919)%20254-7182> > > Address: 4205 S Miami Blvd > > < > https://maps.google.com/?q=4205+S+Miami+Blvd=gmail=g > > > > (Cornwallis Drive), Durham, NC 27703, United States > > > > -James Thomas wrote: - > > To: dev@openwhisk.apache.org > > From: James Thomas > > Date: 06/20/2018 01:17PM > > Subject: Re: [Release] Preparing the release of OpenWhisk > > > > 0.9 makes sense to me. > > > > Something to think about it - what would constitute a 1.0 release? > Whilst > > the platform is still evolving rapidly, it has been in production on > > multiple providers for over 12 months. What things would we like to tick > > off before reaching this stage? > > > > On 20 June 2018 at 17:38, Michele Sciabarra > > wrote: > > > > > I agree with 0.9.0 > > > > > > -- > > > Michele Sciabarra > > > openwh...@sciabarra.com > > > > > > On Wed, Jun 20, 2018, at 6:37 PM, Michele Sciabarra wrote: > > > > I agree with 0.9.0. > > > > > > > > -- > > > > Michele Sciabarra > > > > mich...@sciabarra.com > > > > > > > > On Wed, Jun 20, 2018, at 6:31 PM, Rob Allen wrote: > > > > > On 20 Jun 2018, at 16:24, Matt Rutkowski > > wrote: > > > > > > > > > > > > Can we go with 0.9.0? > > > > > > > > > > > > > > > > 0.9.0 is fine with me. > > > > > > > > > > Regards, > > > > > > > > > > Rob > > > > > > > > > > > > > > > > -- > > Regards, > > James Thomas > > > > > > > > > > -- Regards, James Thomas
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
+1 for the release, its been a lot of hard work from the team, great job Matt, Vincent, and Daisy! Cheers Priti
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Hi Bertrand, ...Plus a single repo. source is not usable by itself and its build dependent on the other parts as I mentioned earlier... >>Right, it if cannot be built that's a problem - but if you say that I >>suppose there's a build order that must be followed? >>If that's correct those overall build instructions should be included >>with the set of release archives. As required, the main openwhisk README (and supporting) docs include instructions on how to build (and tooling that makes it quite easy). We can open an issue to better document suggest manual build order. Will talk to Vincent to see if he has time today as I am leaving soon to return Monday. -mr From: Bertrand Delacretaz To: dev@openwhisk.apache.org Date: 06/21/2018 11:00 AM Subject:Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating Hi Matt, On Thu, Jun 21, 2018 at 5:27 PM Matt Rutkowski wrote: > ...Are you saying you believe the Incubator PMC > will fail us strictly due to having 13 tgz/tar files vs. 1 for a first > release?... I don't know (and someone's welcome to ask on the general@incubator.a.o list to find out), but that looks unusual to me, and more work for reviewers that might need to unpack 13 archives to find similar issues in several of them. That's why I focused on just one archive here, and found a few interesting things already - the other 12 archives have not been useful for my initial review. > ...Are you saying they need to be "eased into the concept" because we will > have 13 (now and more eventually); at some point the board will be exposed > to multiples... No, it's just a practical question and fairness for the reviewers, where multiple archives might not say much more than one about the readiness of OpenWhisk to make Apache Releases. The ASF Board is not involved with releases, it's just the Incubator PMC in this case, for a podling. > ...Plus a single repo. source is not usable by itself and its build dependent > on the other parts as I mentioned earlier... Right, it if cannot be built that's a problem - but if you say that I suppose there's a build order that must be followed? If that's correct those overall build instructions should be included with the set of release archives. -Bertrand
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Hi Matt, On Thu, Jun 21, 2018 at 5:27 PM Matt Rutkowski wrote: > ...Are you saying you believe the Incubator PMC > will fail us strictly due to having 13 tgz/tar files vs. 1 for a first > release?... I don't know (and someone's welcome to ask on the general@incubator.a.o list to find out), but that looks unusual to me, and more work for reviewers that might need to unpack 13 archives to find similar issues in several of them. That's why I focused on just one archive here, and found a few interesting things already - the other 12 archives have not been useful for my initial review. > ...Are you saying they need to be "eased into the concept" because we will > have 13 (now and more eventually); at some point the board will be exposed > to multiples... No, it's just a practical question and fairness for the reviewers, where multiple archives might not say much more than one about the readiness of OpenWhisk to make Apache Releases. The ASF Board is not involved with releases, it's just the Incubator PMC in this case, for a podling. > ...Plus a single repo. source is not usable by itself and its build dependent > on the other parts as I mentioned earlier... Right, it if cannot be built that's a problem - but if you say that I suppose there's a build order that must be followed? If that's correct those overall build instructions should be included with the set of release archives. -Bertrand
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
FWIW --- IF we had to pick one repo, the one with the fewest dependences that could be standalone, as a first release would the go sdk. Then wskdeploy? The runtimes and CLI are tricky there after because why self contained, for the most part, do share common bits with openwhisk repo for the tests. -r On Thu, Jun 21, 2018 at 11:27 AM Matt Rutkowski wrote: > Hi Bertrand, > > I am not sure I understand. Are you saying you believe the Incubator PMC > will fail us strictly due to having 13 tgz/tar files vs. 1 for a first > release? Again, it makes no sense to me as it is strictly a choice of > logical separation (representative of our architectural parts) and > packaging? Surely you see that can and it is technically not hard to > explain. > > Are you saying they need to be "eased into the concept" because we will > have 13 (now and more eventually); at some point the board will be exposed > to multiples. > > Plus a single repo. source is not usable by itself and its build dependent > on the other parts as I mentioned earlier. > > Kind regards, > Matt > > > > > From: Bertrand Delacretaz > To: dev@openwhisk.apache.org > Date: 06/21/2018 10:17 AM > Subject:Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating > > > > Hi Matt, > > On Thu, Jun 21, 2018 at 5:03 PM Matt Rutkowski > wrote: > > ...For now, I am quite happy with releasing all together > > We can try, but as I said I'm not sure if the Incubator PMC will > accept this for a first release. > > Even releasing a single module that's not usable by itself is progress > w.r.t. the incubation process, where it's the process and legal > aspects that count, for initial incubating releases, more than the > technical viability of the product. There's even no obligation to > "advertise" those releases, considering them training releases is > fine. > > But we can try if that's what the majority of the PPMC wants and if > the other mentors do not disagree. > > > ...BTW, I am more than happy to formalize and represent this position > (along with the > > history) to make it clear for others during the review process > > I think it's easy to understand the technical justification for > releasing multiple modules together - my angle is just the "incubation > training" one. > > -Bertrand > > > > > >
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Hi Bertrand, I am not sure I understand. Are you saying you believe the Incubator PMC will fail us strictly due to having 13 tgz/tar files vs. 1 for a first release? Again, it makes no sense to me as it is strictly a choice of logical separation (representative of our architectural parts) and packaging? Surely you see that can and it is technically not hard to explain. Are you saying they need to be "eased into the concept" because we will have 13 (now and more eventually); at some point the board will be exposed to multiples. Plus a single repo. source is not usable by itself and its build dependent on the other parts as I mentioned earlier. Kind regards, Matt From: Bertrand Delacretaz To: dev@openwhisk.apache.org Date: 06/21/2018 10:17 AM Subject:Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating Hi Matt, On Thu, Jun 21, 2018 at 5:03 PM Matt Rutkowski wrote: > ...For now, I am quite happy with releasing all together We can try, but as I said I'm not sure if the Incubator PMC will accept this for a first release. Even releasing a single module that's not usable by itself is progress w.r.t. the incubation process, where it's the process and legal aspects that count, for initial incubating releases, more than the technical viability of the product. There's even no obligation to "advertise" those releases, considering them training releases is fine. But we can try if that's what the majority of the PPMC wants and if the other mentors do not disagree. > ...BTW, I am more than happy to formalize and represent this position (along with the > history) to make it clear for others during the review process I think it's easy to understand the technical justification for releasing multiple modules together - my angle is just the "incubation training" one. -Bertrand
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Hi Matt, On Thu, Jun 21, 2018 at 5:03 PM Matt Rutkowski wrote: > ...For now, I am quite happy with releasing all together We can try, but as I said I'm not sure if the Incubator PMC will accept this for a first release. Even releasing a single module that's not usable by itself is progress w.r.t. the incubation process, where it's the process and legal aspects that count, for initial incubating releases, more than the technical viability of the product. There's even no obligation to "advertise" those releases, considering them training releases is fine. But we can try if that's what the majority of the PPMC wants and if the other mentors do not disagree. > ...BTW, I am more than happy to formalize and represent this position (along > with the > history) to make it clear for others during the review process I think it's easy to understand the technical justification for releasing multiple modules together - my angle is just the "incubation training" one. -Bertrand
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Hi Bertrand, I do not believe we should (or can) release just one repo. at this time (my vote is continue with current artifact granularity). In the future, we can work to enable individual repo. release over time as well (describe via process/tools), but most of these repos. must be released together for compatibility. IMO, we made a decision early in the project to separate component parts of our architecture into individual repos. for many good, well considered reasons. Primarily the project represents a complex PaaS platform with many moving parts (both client and server) with different requirements for skills and languages, but all interdependent on each other by inherent versioning of APIs and Service Provider Interfaces, as well as runtime conventions. In reality we cannot release just even the main OW repo. since it's build is dependent on the other repos.' images. It is disingenuous at best IMO. Releasing individual repos. without clear documentation of these still changing surfaces really does not work. It is something the community well knows, has been discussed several times, is documented as a future "goal". For now, I am quite happy with releasing all together. I look at it this way... we could simply TAR all the repos. into one big TAR (to no real benefit other than to get 1 file). Code is code, packaging is packaging; in the end that all it really represents. BTW, I am more than happy to formalize and represent this position (along with the history) to make it clear for others during the review process. /soapbox off -MR On 2018/06/21 13:08:23, Bertrand Delacretaz wrote: > Hi Vincent, > > On Thu, Jun 21, 2018 at 2:53 PM Vincent S Hou wrote: > > ...Does it mean we can try to release one of the 13 modules, like > > openwhisk, or openwhisk-cli, or consolidate > > all the 13 projects into one for release?... > > The former, I would say? > > It's probably more convenient for your users and w.r.t release cycles? > > For Apache Sling, as an example which is extremely modular, we do lots > of individual module releases all the time, and about once a year do a > "big bang" release that includes all core module. > > A model like that might be good for OpenWhisk, but as this stage as > mentioned for a first "training release" it's probably best to stick > to one typical module to refine the process. > > ... > > * The key can be accessed at > > https://dist.apache.org/repos/dist/dev/incubator/openwhisk/KEYS. You missed > > "dev/" in your link... > > Ah ok, sorry! Got it now. > > > ...* So far the header is not verified with RAT. We have a unitiy repo call > > openwhisk-utility(https://github.com/apache/incubator-openwhisk-utilities) > > to scan all the code. RAT has issues, > > since I have never got it running correctly in openwhisk. The Travis build > > uses this openwhisk-utility to verify the > > headers for every incoming commit > > Ok. The "how to I run the utility to verify the license headers" > question should be answerable with a URL, maybe the docs of that > utility? > People will need to be able to run it standalone to do their own > verifications. > > > * RSA private key should have some instructions. We will work on it... > > Great > > > * We do not release binary this time... > > Yes - I was checking for binaries that might have been leftover, saw > none and that's good! > > > * We will look at the .scala code files... > > Ok. If the package name change is too disruptive it can be postponed > for later during incubation, but that needs to be tracked. > > > * For README, let me make the build instruction more clear... > > Thanks! > > I suppose this means this vote is canceled until you have a new > release candidate? > > -Bertrand >
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Thanks Betrand, You saved me time today :yay: ! Yeah the renaming of the scala packages, should not be a show stopper but we should open an issue in the release repo to track. Also needs some coordination for own modules that depends on it and downstreams. We can release 0.9.0, and after release we can published to maven and repos and downtreams that depend on it we can pin in gradle to pull 0.9.0 until the package get's rename to org.apache.openwhisk.* I agree with Bertrand about license check, There should be a simple way that anyone outside the openwhisk community can download the tgz, extract and follow simple steps to run the license scanner against the content of the tgz -cs On Thu, Jun 21, 2018 at 10:33 AM Rodric Rabbah wrote: > Thanks Bertrand for the suggestion to modularize the release - I do think > that makes a lot of sense as well. > > The way we're vectoring is for the runtimes to be independent and can have > their own lifecycle. > Similarly the CLI and related tooling. > In the long run this will make a lot of sense. > > > -r > > > On Thu, Jun 21, 2018 at 9:08 AM, Bertrand Delacretaz < > bdelacre...@apache.org > > wrote: > > > Hi Vincent, > > > > On Thu, Jun 21, 2018 at 2:53 PM Vincent S Hou wrote: > > > ...Does it mean we can try to release one of the 13 modules, like > > openwhisk, or openwhisk-cli, or consolidate > > > all the 13 projects into one for release?... > > > > The former, I would say? > > > > It's probably more convenient for your users and w.r.t release cycles? > > > > For Apache Sling, as an example which is extremely modular, we do lots > > of individual module releases all the time, and about once a year do a > > "big bang" release that includes all core module. > > > > A model like that might be good for OpenWhisk, but as this stage as > > mentioned for a first "training release" it's probably best to stick > > to one typical module to refine the process. > > > > ... > > > * The key can be accessed at https://dist.apache.org/repos/ > > dist/dev/incubator/openwhisk/KEYS. You missed "dev/" in your link... > > > > Ah ok, sorry! Got it now. > > > > > ...* So far the header is not verified with RAT. We have a unitiy repo > > call > > > openwhisk-utility(https://github.com/apache/incubator- > > openwhisk-utilities) to scan all the code. RAT has issues, > > > since I have never got it running correctly in openwhisk. The Travis > > build uses this openwhisk-utility to verify the > > > headers for every incoming commit > > > > Ok. The "how to I run the utility to verify the license headers" > > question should be answerable with a URL, maybe the docs of that > > utility? > > People will need to be able to run it standalone to do their own > > verifications. > > > > > * RSA private key should have some instructions. We will work on it... > > > > Great > > > > > * We do not release binary this time... > > > > Yes - I was checking for binaries that might have been leftover, saw > > none and that's good! > > > > > * We will look at the .scala code files... > > > > Ok. If the package name change is too disruptive it can be postponed > > for later during incubation, but that needs to be tracked. > > > > > * For README, let me make the build instruction more clear... > > > > Thanks! > > > > I suppose this means this vote is canceled until you have a new > > release candidate? > > > > -Bertrand > > >
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Hi Bertrand, As you noted, the release process uses the Apache RAT to scan all the built TAR files and our own Scancode utility scans files at "build time" for both PR and release (master or named release) builds. We have endeavored to document our use of these scanning utilities within the context of our release process here: - "License Compliance": https://github.com/apache/incubator-openwhisk-release/blob/master/docs/license_compliance.md On this page, we describe our usage of both RAT and Scancode as well as detailing, in great depth, all file inclusions down to every file type we have across all repos. In addition (for convenience and to prove thoroughness), we have identified all known exclusions (all in accordance with Apache policy) by repo. here: - https://github.com/apache/incubator-openwhisk-release/blob/master/docs/license_exclusions.md and you surmised correctly that the Scancode utility usage is documented where it lives in the incubator-openwhisk-utility repo. here: - description/install/build/run basics: https://github.com/apache/incubator-openwhisk-utilities - full usage: https://github.com/apache/incubator-openwhisk-utilities/blob/master/scancode/README.md Fee free to ask any question about licenses and scanning as this has been my life for the last many months... Kind regards, Matt From: Bertrand Delacretaz To: dev@openwhisk.apache.org Date: 06/21/2018 08:08 AM Subject:Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating Hi Vincent, On Thu, Jun 21, 2018 at 2:53 PM Vincent S Hou wrote: > ...Does it mean we can try to release one of the 13 modules, like openwhisk, or openwhisk-cli, or consolidate > all the 13 projects into one for release?... The former, I would say? It's probably more convenient for your users and w.r.t release cycles? For Apache Sling, as an example which is extremely modular, we do lots of individual module releases all the time, and about once a year do a "big bang" release that includes all core module. A model like that might be good for OpenWhisk, but as this stage as mentioned for a first "training release" it's probably best to stick to one typical module to refine the process. ... > * The key can be accessed at https://dist.apache.org/repos/dist/dev/incubator/openwhisk/KEYS . You missed "dev/" in your link... Ah ok, sorry! Got it now. > ...* So far the header is not verified with RAT. We have a unitiy repo call > openwhisk-utility( https://github.com/apache/incubator-openwhisk-utilities ) to scan all the code. RAT has issues, > since I have never got it running correctly in openwhisk. The Travis build uses this openwhisk-utility to verify the > headers for every incoming commit Ok. The "how to I run the utility to verify the license headers" question should be answerable with a URL, maybe the docs of that utility? People will need to be able to run it standalone to do their own verifications. > * RSA private key should have some instructions. We will work on it... Great > * We do not release binary this time... Yes - I was checking for binaries that might have been leftover, saw none and that's good! > * We will look at the .scala code files... Ok. If the package name change is too disruptive it can be postponed for later during incubation, but that needs to be tracked. > * For README, let me make the build instruction more clear... Thanks! I suppose this means this vote is canceled until you have a new release candidate? -Bertrand
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Thanks Bertrand for the suggestion to modularize the release - I do think that makes a lot of sense as well. The way we're vectoring is for the runtimes to be independent and can have their own lifecycle. Similarly the CLI and related tooling. In the long run this will make a lot of sense. -r On Thu, Jun 21, 2018 at 9:08 AM, Bertrand Delacretaz wrote: > Hi Vincent, > > On Thu, Jun 21, 2018 at 2:53 PM Vincent S Hou wrote: > > ...Does it mean we can try to release one of the 13 modules, like > openwhisk, or openwhisk-cli, or consolidate > > all the 13 projects into one for release?... > > The former, I would say? > > It's probably more convenient for your users and w.r.t release cycles? > > For Apache Sling, as an example which is extremely modular, we do lots > of individual module releases all the time, and about once a year do a > "big bang" release that includes all core module. > > A model like that might be good for OpenWhisk, but as this stage as > mentioned for a first "training release" it's probably best to stick > to one typical module to refine the process. > > ... > > * The key can be accessed at https://dist.apache.org/repos/ > dist/dev/incubator/openwhisk/KEYS. You missed "dev/" in your link... > > Ah ok, sorry! Got it now. > > > ...* So far the header is not verified with RAT. We have a unitiy repo > call > > openwhisk-utility(https://github.com/apache/incubator- > openwhisk-utilities) to scan all the code. RAT has issues, > > since I have never got it running correctly in openwhisk. The Travis > build uses this openwhisk-utility to verify the > > headers for every incoming commit > > Ok. The "how to I run the utility to verify the license headers" > question should be answerable with a URL, maybe the docs of that > utility? > People will need to be able to run it standalone to do their own > verifications. > > > * RSA private key should have some instructions. We will work on it... > > Great > > > * We do not release binary this time... > > Yes - I was checking for binaries that might have been leftover, saw > none and that's good! > > > * We will look at the .scala code files... > > Ok. If the package name change is too disruptive it can be postponed > for later during incubation, but that needs to be tracked. > > > * For README, let me make the build instruction more clear... > > Thanks! > > I suppose this means this vote is canceled until you have a new > release candidate? > > -Bertrand >
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Hi Vincent, On Thu, Jun 21, 2018 at 2:53 PM Vincent S Hou wrote: > ...Does it mean we can try to release one of the 13 modules, like openwhisk, > or openwhisk-cli, or consolidate > all the 13 projects into one for release?... The former, I would say? It's probably more convenient for your users and w.r.t release cycles? For Apache Sling, as an example which is extremely modular, we do lots of individual module releases all the time, and about once a year do a "big bang" release that includes all core module. A model like that might be good for OpenWhisk, but as this stage as mentioned for a first "training release" it's probably best to stick to one typical module to refine the process. ... > * The key can be accessed at > https://dist.apache.org/repos/dist/dev/incubator/openwhisk/KEYS. You missed > "dev/" in your link... Ah ok, sorry! Got it now. > ...* So far the header is not verified with RAT. We have a unitiy repo call > openwhisk-utility(https://github.com/apache/incubator-openwhisk-utilities) to > scan all the code. RAT has issues, > since I have never got it running correctly in openwhisk. The Travis build > uses this openwhisk-utility to verify the > headers for every incoming commit Ok. The "how to I run the utility to verify the license headers" question should be answerable with a URL, maybe the docs of that utility? People will need to be able to run it standalone to do their own verifications. > * RSA private key should have some instructions. We will work on it... Great > * We do not release binary this time... Yes - I was checking for binaries that might have been leftover, saw none and that's good! > * We will look at the .scala code files... Ok. If the package name change is too disruptive it can be postponed for later during incubation, but that needs to be tracked. > * For README, let me make the build instruction more clear... Thanks! I suppose this means this vote is canceled until you have a new release candidate? -Bertrand
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Hi Bertrand, Thank you very much for your comments. Let me clarify what you mean by one module: Does it mean we can try to release one of the 13 modules, like openwhisk, or openwhisk-cli, or consolidate all the 13 projects into one for release? * The key can be accessed at https://dist.apache.org/repos/dist/dev/incubator/openwhisk/KEYS. You missed "dev/" in your link. * So far the header is not verified with RAT. We have a unitiy repo call openwhisk-utility(https://github.com/apache/incubator-openwhisk-utilities) to scan all the code. RAT has issues, since I have never got it running correctly in openwhisk. The Travis build uses this openwhisk-utility to verify the headers for every incoming commit. * RSA private key should have some instructions. We will work on it. * We do not release binary this time. * We will look at the .scala code files. * For README, let me make the build instruction more clear. Best wishes. Vincent Hou (侯胜博) Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM Cloud Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com, Phone: +1(919)254-7182 Address: 4205 S Miami Blvd (Cornwallis Drive), Durham, NC 27703, United States -Bertrand Delacretaz wrote: - To: dev@openwhisk.apache.org From: Bertrand Delacretaz Date: 06/21/2018 07:04AM Subject: Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating Hi Vincent, Thanks for your work in preparing this release! On Wed, Jun 20, 2018 at 11:16 PM Vincent S Hou wrote: > ...There are totally 13 OpenWhisk projects within this release As mentioned earlier I don't think it is a good idea to release multiple modules in your first Incubator release: if a single module has a problem the whole release will fail, and it's not convenient for Incubator PMC reviewers (who might not be very familiar with your code) to review multiple modules in one go. I'm not sure if the Incubator PMC would even accept voting on multiple artifacts with a single vote. I recommend releasing one module first, to validate the release voting process and to get feedback that's probably applicable for other modules as well. With this in mind I have just looked at openwhisk-0.9.0-incubating-sources.tar.gz so far, here are my comments: 1) The digests match. 2) The 22CC20CC key used to sign the release is not available at https://dist.apache.org/repos/dist/release/incubator/openwhisk/KEYS (that file doesn't exist) nor at https://people.apache.org/keys/group/openwhisk.asc 3) I don't find build instructions in the README (which is also at https://github.com/apache/incubator-openwhisk), for convenience I'm not very familiar with gradle, tried this: ./gradlew tasks doesn't show anything specific to OpenWhisk IIUC ./gradlew tasks --all shows many tasks, it's unclear where to start I usually expect to see clear build instructions in such a release archive, but maybe I missed something. 4) LICENSE, DISCLAIMER, NOTICE look good to me 5) The .scala code files are in whisk.* packages, that should be org.apache.openwhisk.* for an Apache project. 6) I suppose you used Apache Rat to validate the license headers, I don't see instructions on how to run it to make those checks myself. 7) There's an RSA private key in the source archive, if it's for testing purposes it should be clearly identified as such (ideally named test- something) to reassure people that it's not problematic to distribute it (./ansible/roles/nginx/files/openwhisk-server-key.pem). 8) I don't see binary files in the release archive which is good, except for ./gradle/wrapper/gradle-wrapper.jar which I think is acceptable - but its digest should be kept track of, maybe in a jira ticket so people can validate it if they want. Those are the types of comments that you might get when the Incubator PMC validates releases, I suppose many of them apply to multiple projects so it's easier to start with just one module, fix or clarify these things and then do the rest. -Bertrand (with my incubation mentor hat on)
Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating
Hi Vincent, Thanks for your work in preparing this release! On Wed, Jun 20, 2018 at 11:16 PM Vincent S Hou wrote: > ...There are totally 13 OpenWhisk projects within this release As mentioned earlier I don't think it is a good idea to release multiple modules in your first Incubator release: if a single module has a problem the whole release will fail, and it's not convenient for Incubator PMC reviewers (who might not be very familiar with your code) to review multiple modules in one go. I'm not sure if the Incubator PMC would even accept voting on multiple artifacts with a single vote. I recommend releasing one module first, to validate the release voting process and to get feedback that's probably applicable for other modules as well. With this in mind I have just looked at openwhisk-0.9.0-incubating-sources.tar.gz so far, here are my comments: 1) The digests match. 2) The 22CC20CC key used to sign the release is not available at https://dist.apache.org/repos/dist/release/incubator/openwhisk/KEYS (that file doesn't exist) nor at https://people.apache.org/keys/group/openwhisk.asc 3) I don't find build instructions in the README (which is also at https://github.com/apache/incubator-openwhisk), for convenience I'm not very familiar with gradle, tried this: ./gradlew tasks doesn't show anything specific to OpenWhisk IIUC ./gradlew tasks --all shows many tasks, it's unclear where to start I usually expect to see clear build instructions in such a release archive, but maybe I missed something. 4) LICENSE, DISCLAIMER, NOTICE look good to me 5) The .scala code files are in whisk.* packages, that should be org.apache.openwhisk.* for an Apache project. 6) I suppose you used Apache Rat to validate the license headers, I don't see instructions on how to run it to make those checks myself. 7) There's an RSA private key in the source archive, if it's for testing purposes it should be clearly identified as such (ideally named test- something) to reassure people that it's not problematic to distribute it (./ansible/roles/nginx/files/openwhisk-server-key.pem). 8) I don't see binary files in the release archive which is good, except for ./gradle/wrapper/gradle-wrapper.jar which I think is acceptable - but its digest should be kept track of, maybe in a jira ticket so people can validate it if they want. Those are the types of comments that you might get when the Incubator PMC validates releases, I suppose many of them apply to multiple projects so it's easier to start with just one module, fix or clarify these things and then do the rest. -Bertrand (with my incubation mentor hat on)
ArtifactStore shutdown handling and shared resources
ArtifactStore SPI exposes a shutdown method which is responsible for closing any resource owned by store implementation. Ccurrently for CouchDbRestStore it only shuts down ActorMaterializer which is created one per instance. It does not shutdown Http pool which is shared across 3 store instance. This is documented in PoolingRestClient. Now with CosmosDBArtifactStore we need to share a `DocumentClient` instance which owns the underlying Netty connection pool. As all the store instance talk to same db it makes sense to share the instance. However this sharing poses problem with shutdown. To handle that CosmosDB PR introduces a `CountedReference` [1] which keeps an open/close count and only closes when all references are closed. Wanted to check with team if that would be fine approach to take? Currently its bit tricky to manage components lifecycle and often we need to rely on shutdown hooks to close the resources properly. May be we revisit SPI/component lifecycle handling later and then review this shutdown method handling Chetan Mehrotra [1] https://github.com/apache/incubator-openwhisk/pull/3562/files#diff- 9d57de71410575fd70240ac974be407d
