Hi everyone, Just my two cents here...
I think the tendency nowadays is to add these tools to build the code to the repository itself. See Gitlab yamls or Github action file that can declare the compile+test phase for the CI. Gitlab docs perfectly sums up why I think it can be a better choice to include it next to the code: > Because .gitlab-ci.yml is in the repository and is version controlled, old > versions still build successfully, forks can easily make use of CI, > branches can have different pipelines and jobs, and you have a single > source of truth for CI. You can read more about the reasons why we are > using .gitlab-ci.yml in our blog about it > <https://about.gitlab.com/blog/2015/05/06/why-were-replacing-gitlab-ci-jobs-with-gitlab-ci-dot-yml/> > . Also I recommend reading the article. Regards, Adam On Wed, Apr 15, 2020 at 11:59 PM Weiwei Yang <[email protected]> wrote: > Hi Wilfred/Sunil > > Thanks for the feedback. > For a multi-repo project like YuniKorn, I think it makes sense to have a > separate one to manage them. > Put release things (script, doc, files) in any of the source code repo is a > bit weird to me. I have already created > https://github.com/apache/incubator-yunikorn-release/. *which is for the > release managers only.* This can avoid further confusion and also help us > to do future releases in minutes. > The docker file approach, which provides a build env without pollute > people's local env, this might be helpful in certain cases. Such as when > you need to install a lot of binaries to make a build. For YuniKorn, since > we just rely on Go for the build env, I don't see how useful it is at > present. I am open to adopting that approach as long as it helps. > > Weiwei > > > On Wed, Apr 15, 2020 at 8:06 AM Sunil Govindan <[email protected]> wrote: > > > Thanks Weiwei and Wilfred. > > > > +0 on this approach. I am overall feeling fine with the approach > suggested. > > However I may have personally prefer to do this in a create-release.sh > and > > Dockerfile like hadoop model. > > > > As Weiwei pointed out, its used in Opwnwhisk pjct as well, may be we can > > try this out and find the positiveness. > > > > I think, we should make the scripts or configs in this repo more dynamic > in > > nature agnostic to any release version. > > RM can supply this version etc as a run time param to these scripts. > > > > Thanks > > Sunil > > > > On Tue, Apr 14, 2020 at 12:26 PM Weiwei Yang <[email protected]> > wrote: > > > > > Hi Wilfred > > > > > > Thanks for the feedback. > > > > > > The reason to introduce a new repo is to have some tools/docs easier > for > > > release managers. I found this is a good practice when you have > multiple > > > repos. Please see Apache OpenWhisk as an example. > > > > > > If you look at the config file I shared, that helps to track source > code > > > revision of different repos, which is easy to assemble a release. > > > > > > I propose to do unified release, which means we alway release all repos > > > together. This is simpler to manage. > > > > > > If you look at the archive file, I should have put a script and a > README > > > about how to build docker images. Can you please take a look at that? > If > > > that is not there, pls see the script on github. > > > > > > Thanks for verifying the content. Could you please let me know what > files > > > are wrong? > > > > > > > > > > > > -- > > > Weiwei > > > On Apr 13, 2020, 11:30 PM -0700, Wilfred Spiegelenburg < > > > [email protected]>, wrote: > > > > Hi Weiwei, > > > > > > > > Do we really need another repo for three files? > > > > We should sleep track of this in the core repo not in another repo > > which > > > we > > > > need to release manage again. I think managing the release from the > > core > > > > repo will make it easier later on if we need to or want to change the > > > build > > > > process further. Now we need need to manage and track soo many repos > > that > > > > it becomes more and more difficult. > > > > We also need to keep in mind that version information is in the > module > > > > files. There might thus be more that needs to change for a release. > The > > > > other thing is that we might not want to release a new version of one > > of > > > > the components while updating another component. That would means > that > > we > > > > need to release manage 5 repositories for one release, including all > > the > > > > overhead. > > > > > > > > Apache releases are source releases. We still need to provide some > kind > > > of > > > > make etc over the source code also. I agree with the fact that we > need > > to > > > > provide one source archive that is signed. However with the current > > build > > > > process just the k8shim code is enough to build the docker image. The > > > other > > > > code repos will be pulled in from github. The mod file point there > for > > > all > > > > go dependencies including the core and SI. It does not provide any > > detail > > > > on the how and what for any of the repos. We need to provide some > build > > > > instructions in the root of the source archive. I would not know how > to > > > > build from the source package if we just add the checked out code > into > > > it. > > > > We need to provide some steps even if they are just pointers to > > existing > > > > docs. > > > > I also don't think we have the correct files in the archive with the > > > > current generated archive. > > > > > > > > Wilfred > > > > > > > > > > > > > On Mon, 13 Apr 2020 at 14:01, Weiwei Yang <[email protected]> wrote: > > > > > > > > > For 0.8 release, I did some work and I wanted to share the latest > > > status. I > > > > > think we should target for *docker-image-based* release mode. I > > > propose to > > > > > release a unified open-source tarball, we don't release a binary > > > tarball > > > > > (not a must [1]). Things have been *DONE*: > > > > > > > > > > 1. I have created a repo for release mgmt: > > > > > https://github.com/yangwwei/yunikorn-release, I think we need to > > move > > > > > this to apache repo too. > > > > > > > > > 2. This repo has the instructions and tools for a release. The tool > > > > > loads configs from > > > > > > > > > > > > > > > > https://github.com/yangwwei/yunikorn-release/blob/master/tools/release-configs.json > > > > > and > > > > > downloads source code from certain repo/branch/hash to assemble the > > > > > release > > > > > artifacts > > > > > > > > 3. This repo contains a *build-docker-image.sh* to build yunikorn > > docker > > > > > images (scheduler, admission-controller, and web) > > > > > 4. I have created *branch-0.8* for all 4 repos > > > > > 5. The generated tarball will also have the helm chart for user to > > > > > install and run yunikorn on an existing K8s cluster > > > > > 6. I tried to generate PGP key and sign the tarball > > > > > > > > > > Things *TODO* > > > > > > > > > > 1. Create a repo for yunikorn-release under ASF > > > > > 2. IIUC, https://issues.apache.org/jira/browse/YUNIKORN-79 is a > > > blocker > > > > > for 0.8. Can we get this fixed ASAP? > > > > > 3. Once #2 is done, create a tag for 0.8-rc1 and start the voting > > > thread > > > > > > > > > > Thanks! > > > > > > > > > > [1] https://infra.apache.org/release-publishing.html *The Apache > > > Software > > > > > Foundation exists to create open source software, so the > fundamental > > > > > requirement for a release is that it has the necessary source code > to > > > build > > > > > the project. A project may provide compiled binaries of each > release > > > for > > > > > the convenience of users.* > > > > > > > > > > >
