Re: [Pulp-dev] Pulp 3 Release Process Questions
I'd like to summarize where we landed on this after several off-list conversation. Current understanding: * All current stakeholders are good to consume the beta via PyPI * The build team will create automation to create RPMs from packages published on PyPI * The publishing of RPMs will start at some point in the future after Pulp developers and QE have put together a plan for where the RPMs will be hosted, what infrastructure will be used to test them, and how the packages will be signed. We invite stakeholders to tell us when they need RPMs. Robin On Tue, Apr 24, 2018 at 9:41 AM, Patrick Creech wrote: > On Mon, 2018-04-23 at 17:32 -0400, Brian Bouterse wrote: >> >> >> Whoever does the packaging needs to determine how/where they want to run >> them. > > So, this is a sentiment I thought was addressed by Robin's e-mail, but I'll > explicitly re-iterate here: > > The build teams role is not to be a black box that takes pulp's python > packages and produces an rpm, no questions asked. Our involvement in the > upstream Pulp project is as a support role to help Pulp > Engineering deliver on their desire to provide RPMs of pulp 3 to the upstream > pulp community. We are trying to work with Pulp Engineering to come up with > a process suitable for you to deliver RPMs to > your userbase. This is why we started asking questions around what your > teams expectations are with regards to rpm packages, where you want them > delivered, etc... > > Our primary goal is to work with you and QE to help design and implement a > process you can initiate at release time, and at the end have rpms, hopefully > eliminating the need for constant involvement > by a build team representative when it comes to release time. Especially if > it's pulps goal is to release more frequently. > >> >> > Pulp 2's userbase primarily uses rpm based distributions. If your support >> > statement is 'It works on travis!' without ensuring pulp 3 works on your >> > dominant userbase's machines, then I do not >> > think >> > that is a great attitude to have with regards to pulp's userbase. >> >> >> I don't think our claim has to do with Travis. The quality claim for pypi is >> the same approach as with Pulp2: "we ran X unit tests and Y smash tests and >> there were 0 errors so we released". Users >> can understand what our quality claim is by reading the functional tests. >> These quality checks will be run at pypi release time, and I think doing it >> at rpm release time would be consistent. Running >> the unit and smash tests at packaging time on a given distro will provide >> that consistent quality claim in the packaged assets. Untested assets I >> think come with no guarantees to their users. > > So, I want to somewhat revise your statment on pulp 2 w/r to the quality > statement: "we ran X unit tests and Y smash tests _on Z distributions_ and > there were 0 errors so we released". Each upstream > supported distribution is a first class citizen of the entire testing stack > to ensure proper functionality across the board. This has helped identify > problem areas across those distros that have > helped lead to a more robust Pulp 2 experience. I hope similar would be > provided for pulp 3 as well, even for the 'install from pypi' experience. > This would help identify distribution specific > issues, even in that use case. > >> >> > And while build team is happy to provide automation, configuration, and >> > setup to help package up pypi packages into rpms, I doubt we can commit to >> > doing the legwork to ensure the code itself works >> > specifically on said distributions. (i.e. selinux, systemd daemon scripts, >> > etc...). >> >> I think we're on the same page here. The systemd scripts are provided in the >> docs by the devs. SElinux is it's own special thing, for now the plan is to >> ship pypi with selinux disabled so rpms could >> do that too. I don't expect the build team to produce an SELinux policy. >> > > I hope there is a long term item to develop an selinux policy. I have a hard > time imagining people are going to want to install production workloads on > systems with selinux disabled > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
On Mon, 2018-04-23 at 17:32 -0400, Brian Bouterse wrote: > > > Whoever does the packaging needs to determine how/where they want to run them. So, this is a sentiment I thought was addressed by Robin's e-mail, but I'll explicitly re-iterate here: The build teams role is not to be a black box that takes pulp's python packages and produces an rpm, no questions asked. Our involvement in the upstream Pulp project is as a support role to help Pulp Engineering deliver on their desire to provide RPMs of pulp 3 to the upstream pulp community. We are trying to work with Pulp Engineering to come up with a process suitable for you to deliver RPMs to your userbase. This is why we started asking questions around what your teams expectations are with regards to rpm packages, where you want them delivered, etc... Our primary goal is to work with you and QE to help design and implement a process you can initiate at release time, and at the end have rpms, hopefully eliminating the need for constant involvement by a build team representative when it comes to release time. Especially if it's pulps goal is to release more frequently. > > > Pulp 2's userbase primarily uses rpm based distributions. If your support > > statement is 'It works on travis!' without ensuring pulp 3 works on your > > dominant userbase's machines, then I do not > > think > > that is a great attitude to have with regards to pulp's userbase. > > > I don't think our claim has to do with Travis. The quality claim for pypi is > the same approach as with Pulp2: "we ran X unit tests and Y smash tests and > there were 0 errors so we released". Users > can understand what our quality claim is by reading the functional tests. > These quality checks will be run at pypi release time, and I think doing it > at rpm release time would be consistent. Running > the unit and smash tests at packaging time on a given distro will provide > that consistent quality claim in the packaged assets. Untested assets I think > come with no guarantees to their users. So, I want to somewhat revise your statment on pulp 2 w/r to the quality statement: "we ran X unit tests and Y smash tests _on Z distributions_ and there were 0 errors so we released". Each upstream supported distribution is a first class citizen of the entire testing stack to ensure proper functionality across the board. This has helped identify problem areas across those distros that have helped lead to a more robust Pulp 2 experience. I hope similar would be provided for pulp 3 as well, even for the 'install from pypi' experience. This would help identify distribution specific issues, even in that use case. > > > And while build team is happy to provide automation, configuration, and > > setup to help package up pypi packages into rpms, I doubt we can commit to > > doing the legwork to ensure the code itself works > > specifically on said distributions. (i.e. selinux, systemd daemon scripts, > > etc...). > > I think we're on the same page here. The systemd scripts are provided in the > docs by the devs. SElinux is it's own special thing, for now the plan is to > ship pypi with selinux disabled so rpms could > do that too. I don't expect the build team to produce an SELinux policy. > I hope there is a long term item to develop an selinux policy. I have a hard time imagining people are going to want to install production workloads on systems with selinux disabled signature.asc Description: This is a digitally signed message part ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
Sure, there’s a couple things we’re trying to accomplish. First is to deploy to PyPI which forces us to be OS agnostic. The whole Pulp 2 codebase is tied to Fedora/CentOS/RHEL when I don’t think it has to be. The other goal is to make our build/testing processes simpler. Pulp 2 uses Jenkins AND Travis for things like testing PRs. Currently in Pulp 3, we’re just using Travis to test PRs. This offloads resources to Travis and means we don’t have to manage a CI environment. Lastly, Jenkins is not accessible outside the Red Hat network. This is bad for upstream community as it means our testing of PRs and our build process is a blackbox. David On Tue, Apr 24, 2018 at 8:22 AM, Bryan Kearney wrote: > On 04/23/2018 02:57 PM, Patrick Creech wrote: > > On Mon, 2018-04-23 at 14:01 -0400, Robin Chan wrote: > >> I feel like the Travis change is recent enough that I'm not exactly > sure how they differ from the Jenkins jobs. Are we all clear on these > terminology? Aren't there multiple jobs running at different > >> times? I am not comfortable enough without the assurance that we are > all talking about the same things to say we don't need "the Jenkins jobs". > I would like for some of these discussions around > >> release process and where (and what and when) different tests are run > be a little bit more collaborative from the beginning rather than reactive > post-change happening. > >> > >> I'm all for making quick incremental changes. I'm happy that there are > test results that are accessible to everyone. I don't want us to over > thinking these, but are we sure we don't have any tests > >> that need to stay private (do we test for any security type things that > might need to be embargoed before reported)? I want to make sure we think > thing through before we start tearing stuff down. > >> > > The build team, for one, has not had time yet to do an assessment on if > travis will be able to provide all the packaging related items needed to > generate RPMs. There are also infrastructure security > > implications we have yet to analyze as well. So, at least for rpm > packaging requirements, these are still a big ? till we have time to > evaluate. > > > > > Can someone summarize for me what there is Yet Another Process? > > -- bk > > > > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
On 04/23/2018 02:57 PM, Patrick Creech wrote: > On Mon, 2018-04-23 at 14:01 -0400, Robin Chan wrote: >> I feel like the Travis change is recent enough that I'm not exactly sure how >> they differ from the Jenkins jobs. Are we all clear on these terminology? >> Aren't there multiple jobs running at different >> times? I am not comfortable enough without the assurance that we are all >> talking about the same things to say we don't need "the Jenkins jobs". I >> would like for some of these discussions around >> release process and where (and what and when) different tests are run be a >> little bit more collaborative from the beginning rather than reactive >> post-change happening. >> >> I'm all for making quick incremental changes. I'm happy that there are test >> results that are accessible to everyone. I don't want us to over thinking >> these, but are we sure we don't have any tests >> that need to stay private (do we test for any security type things that >> might need to be embargoed before reported)? I want to make sure we think >> thing through before we start tearing stuff down. >> > The build team, for one, has not had time yet to do an assessment on if > travis will be able to provide all the packaging related items needed to > generate RPMs. There are also infrastructure security > implications we have yet to analyze as well. So, at least for rpm packaging > requirements, these are still a big ? till we have time to evaluate. > Can someone summarize for me what there is Yet Another Process? -- bk signature.asc Description: OpenPGP digital signature ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
On Mon, Apr 23, 2018 at 4:33 PM, Patrick Creech wrote: > On Mon, 2018-04-23 at 15:22 -0400, Brian Bouterse wrote: > > Travis CI runs a few OSes, but not RHEL and Fedora, or many others. > Travis is good for ensuring that Pulp's main release asset (the pypi > packages) are being tested continuously. > > > > Ensuring that Pulp runs on any given OS is a different kind of testing. > Whoever packages for a given distro (rhel, fedora, debian, etc) will need > to determine how they will test/track the > > correctness of the packaged asset itself. To that end the build team may > want to run pulp smash against built RPMs in a different environment other > than Travis and that would be fine. > > > > One thing to point out here, there is a difference in ensuring that pulp > installed from pypi runs on various distributions, and ensuring that > distribution specific packages run on successfully on > their respective distributions. The extent of whatever testing is done on > rpms would/should be done in a way that ensures the rpm packaged version > works the same as the pypi packaged version. > I agree, the pypi asset is quality checked with smash and unit tests, and any packaged assets (deb, msi, containers, etc) should also be. The unit and smash tests are available so that at packaging time quality can be ensured. Whoever does the packaging needs to determine how/where they want to run them. > > Pulp 2's userbase primarily uses rpm based distributions. If your support > statement is 'It works on travis!' without ensuring pulp 3 works on your > dominant userbase's machines, then I do not think > that is a great attitude to have with regards to pulp's userbase. > I don't think our claim has to do with Travis. The quality claim for pypi is the same approach as with Pulp2: "we ran X unit tests and Y smash tests and there were 0 errors so we released". Users can understand what our quality claim is by reading the functional tests. These quality checks will be run at pypi release time, and I think doing it at rpm release time would be consistent. Running the unit and smash tests at packaging time on a given distro will provide that consistent quality claim in the packaged assets. Untested assets I think come with no guarantees to their users. > > And while build team is happy to provide automation, configuration, and > setup to help package up pypi packages into rpms, I doubt we can commit to > doing the legwork to ensure the code itself works > specifically on said distributions. (i.e. selinux, systemd daemon scripts, > etc...). > I think we're on the same page here. The systemd scripts are provided in the docs by the devs. SElinux is it's own special thing, for now the plan is to ship pypi with selinux disabled so rpms could do that too. I don't expect the build team to produce an SELinux policy. ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
Do packages produced by Travis have to go straight to PyPI? Or is it possible to build artifacts (like wheels) and later retrieve those artifacts? Alternately, are reproductible builds possible for Pulp 3? That is, can you and I both clone the same source code, check out a particular commit, and run a one-liner to produce identical build artifacts? I'm not advocating for any particular solution with these questions. I'm just exploring the problem space. ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
On Mon, 2018-04-23 at 15:22 -0400, Brian Bouterse wrote: > Travis CI runs a few OSes, but not RHEL and Fedora, or many others. Travis is > good for ensuring that Pulp's main release asset (the pypi packages) are > being tested continuously. > > Ensuring that Pulp runs on any given OS is a different kind of testing. > Whoever packages for a given distro (rhel, fedora, debian, etc) will need to > determine how they will test/track the > correctness of the packaged asset itself. To that end the build team may want > to run pulp smash against built RPMs in a different environment other than > Travis and that would be fine. > One thing to point out here, there is a difference in ensuring that pulp installed from pypi runs on various distributions, and ensuring that distribution specific packages run on successfully on their respective distributions. The extent of whatever testing is done on rpms would/should be done in a way that ensures the rpm packaged version works the same as the pypi packaged version. Pulp 2's userbase primarily uses rpm based distributions. If your support statement is 'It works on travis!' without ensuring pulp 3 works on your dominant userbase's machines, then I do not think that is a great attitude to have with regards to pulp's userbase. And while build team is happy to provide automation, configuration, and setup to help package up pypi packages into rpms, I doubt we can commit to doing the legwork to ensure the code itself works specifically on said distributions. (i.e. selinux, systemd daemon scripts, etc...). signature.asc Description: This is a digitally signed message part ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
Travis CI runs a few OSes, but not RHEL and Fedora, or many others. Travis is good for ensuring that Pulp's main release asset (the pypi packages) are being tested continuously. Ensuring that Pulp runs on any given OS is a different kind of testing. Whoever packages for a given distro (rhel, fedora, debian, etc) will need to determine how they will test/track the correctness of the packaged asset itself. To that end the build team may want to run pulp smash against built RPMs in a different environment other than Travis and that would be fine. On Mon, Apr 23, 2018 at 2:27 PM, Preethi Thomas wrote: > > > On Mon, Apr 23, 2018 at 2:01 PM, Robin Chan wrote: > >> I feel like the Travis change is recent enough that I'm not exactly sure >> how they differ from the Jenkins jobs. Are we all clear on these >> terminology? Aren't there multiple jobs running at different times? I am >> not comfortable enough without the assurance that we are all talking about >> the same things to say we don't need "the Jenkins jobs". I would like for >> some of these discussions around release process and where (and what and >> when) different tests are run be a little bit more collaborative from the >> beginning rather than reactive post-change happening. >> >> I'm all for making quick incremental changes. I'm happy that there are >> test results that are accessible to everyone. I don't want us to over >> thinking these, but are we sure we don't have any tests that need to stay >> private (do we test for any security type things that might need to be >> embargoed before reported)? I want to make sure we think thing through >> before we start tearing stuff down. >> >> On Mon, Apr 23, 2018 at 1:37 PM, Brian Bouterse >> wrote: >> >>> >>> On Mon, Apr 23, 2018 at 1:25 PM, David Davis >>> wrote: >>> > Are those Travis jobs testing combinations of web servers, AMQP brokers, databases, etc? If not, is testing across those combinations a goal? This is definitely the goal. Right now, our build matrix tests against different databases, different Django versions, and different python versions. We could look at testing against different webservers but right now we just use django’s built-in webserver. AMQP is going away though[0]. > Let's say pulp_file is being tested. Is both it and pulpcore installed from source? Or is pulpcore installed from egg/wheel? Yes, we’re testing against source which also allows us to test against different PRs. Often, a pulpcore change will require a pulp_file change or vice versa and we just added some functionality recently that actually lets us test against PRs too.[1] pulp-smash is also running against source or PRs. IMO, testing against source lets us move faster than testing against releases. Maybe we can look at switching though once things have stabilized after the MVP. > Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? I don’t think we need them. We can trigger builds on an as-needed basis with Travis without making git commits. We’re also running builds nightly right now with Travis’ cron. >>> >>> I agree we don't need the Jenkins jobs for Pulp3 and they can be >>> deleted. We still need to keep them for Pulp2 because Pulp2 can't run on >>> Travis. Moving to Travis is a big step forward for the community since >>> Travis is readable by the whole Internet while to read Jenkins jobs, you >>> have to be a Red Hat employee. >>> >> > I admit that I know very little about Travis. So I did some reading up here > > https://docs.travis-ci.com/user/reference/overview/#For- > a-particular-.travis.yml-configuration > > Does Travis-ci run only on Ubuntu? > > If that is is true, how are we planning to test fedora/centos/rhel ? With > our history of SeLinux issues in pulp2 I would think we need to be running > tests in these as well. > > > >> >>> > What does this mean for goals like multi-host testing? It looks like Travis supports running docker containers[2] so maybe we could leverage containers to test this sort of setup. There are definitely limitations of Travis that might require us to move to something more robust in the future but short-term I think/hope Travis will fit our needs. [0] https://github.com/pulp/pulp/pull/3454 [1] https://github.com/pulp/pulp/pull/3435 [2] https://docs.travis-ci.com/user/docker/ David On Mon, Apr 23, 2018 at 12:10 PM, Jeremy Audet wrote: > Using Travis jobs instead of Jenkins jobs for testing Pulp 3 releases > begs several questions: > >- Are those Travis jobs testing combinations of web servers, AMQP >brokers, databases, etc? If not, is testing across those combinations a >goal? >- Let's say pulp_file is being tested. Is both it and pulpcore >installed from source? Or is pulpcore installe
Re: [Pulp-dev] Pulp 3 Release Process Questions
On Mon, 2018-04-23 at 14:01 -0400, Robin Chan wrote: > I feel like the Travis change is recent enough that I'm not exactly sure how > they differ from the Jenkins jobs. Are we all clear on these terminology? > Aren't there multiple jobs running at different > times? I am not comfortable enough without the assurance that we are all > talking about the same things to say we don't need "the Jenkins jobs". I > would like for some of these discussions around > release process and where (and what and when) different tests are run be a > little bit more collaborative from the beginning rather than reactive > post-change happening. > > I'm all for making quick incremental changes. I'm happy that there are test > results that are accessible to everyone. I don't want us to over thinking > these, but are we sure we don't have any tests > that need to stay private (do we test for any security type things that might > need to be embargoed before reported)? I want to make sure we think thing > through before we start tearing stuff down. > The build team, for one, has not had time yet to do an assessment on if travis will be able to provide all the packaging related items needed to generate RPMs. There are also infrastructure security implications we have yet to analyze as well. So, at least for rpm packaging requirements, these are still a big ? till we have time to evaluate. signature.asc Description: This is a digitally signed message part ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
On Mon, Apr 23, 2018 at 2:01 PM, Robin Chan wrote: > I feel like the Travis change is recent enough that I'm not exactly sure > how they differ from the Jenkins jobs. Are we all clear on these > terminology? Aren't there multiple jobs running at different times? I am > not comfortable enough without the assurance that we are all talking about > the same things to say we don't need "the Jenkins jobs". I would like for > some of these discussions around release process and where (and what and > when) different tests are run be a little bit more collaborative from the > beginning rather than reactive post-change happening. > > I'm all for making quick incremental changes. I'm happy that there are > test results that are accessible to everyone. I don't want us to over > thinking these, but are we sure we don't have any tests that need to stay > private (do we test for any security type things that might need to be > embargoed before reported)? I want to make sure we think thing through > before we start tearing stuff down. > > On Mon, Apr 23, 2018 at 1:37 PM, Brian Bouterse > wrote: > >> >> On Mon, Apr 23, 2018 at 1:25 PM, David Davis >> wrote: >> >>> > Are those Travis jobs testing combinations of web servers, AMQP >>> brokers, databases, etc? If not, is testing across those combinations a >>> goal? >>> >>> This is definitely the goal. Right now, our build matrix tests against >>> different databases, different Django versions, and different python >>> versions. We could look at testing against different webservers but right >>> now we just use django’s built-in webserver. AMQP is going away though[0]. >>> >>> > Let's say pulp_file is being tested. Is both it and pulpcore >>> installed from source? Or is pulpcore installed from egg/wheel? >>> >>> Yes, we’re testing against source which also allows us to test against >>> different PRs. Often, a pulpcore change will require a pulp_file change or >>> vice versa and we just added some functionality recently that actually lets >>> us test against PRs too.[1] pulp-smash is also running against source or >>> PRs. IMO, testing against source lets us move faster than testing against >>> releases. Maybe we can look at switching though once things have stabilized >>> after the MVP. >>> >>> > Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? >>> >>> I don’t think we need them. We can trigger builds on an as-needed basis >>> with Travis without making git commits. We’re also running builds nightly >>> right now with Travis’ cron. >>> >> >> I agree we don't need the Jenkins jobs for Pulp3 and they can be deleted. >> We still need to keep them for Pulp2 because Pulp2 can't run on Travis. >> Moving to Travis is a big step forward for the community since Travis is >> readable by the whole Internet while to read Jenkins jobs, you have to be a >> Red Hat employee. >> > I admit that I know very little about Travis. So I did some reading up here https://docs.travis-ci.com/user/reference/overview/#For-a-particular-.travis.yml-configuration Does Travis-ci run only on Ubuntu? If that is is true, how are we planning to test fedora/centos/rhel ? With our history of SeLinux issues in pulp2 I would think we need to be running tests in these as well. > >> >>> >>> > What does this mean for goals like multi-host testing? >>> >>> It looks like Travis supports running docker containers[2] so maybe we >>> could leverage containers to test this sort of setup. There are definitely >>> limitations of Travis that might require us to move to something more >>> robust in the future but short-term I think/hope Travis will fit our needs. >>> >>> [0] https://github.com/pulp/pulp/pull/3454 >>> [1] https://github.com/pulp/pulp/pull/3435 >>> [2] https://docs.travis-ci.com/user/docker/ >>> >>> >>> David >>> >>> On Mon, Apr 23, 2018 at 12:10 PM, Jeremy Audet >>> wrote: >>> Using Travis jobs instead of Jenkins jobs for testing Pulp 3 releases begs several questions: - Are those Travis jobs testing combinations of web servers, AMQP brokers, databases, etc? If not, is testing across those combinations a goal? - Let's say pulp_file is being tested. Is both it and pulpcore installed from source? Or is pulpcore installed from egg/wheel? - Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? (Possible answer: So that we can trigger test runs on an as-needed basis, without making new git commits.) - What does this mean for goals like multi-host testing? That is, what does this mean for the case where a single Pulp 3 application is deployed across multiple hosts? Can that be tested with Travis? Pulp Smash is architected with this in mind, and it's been a long standing goal. The biggest impediment has been the use of legacy Jenkins (1.x) and nodepool, and recent upgrades make this a more realistic testing target. If these
Re: [Pulp-dev] Pulp 3 Release Process Questions
On Mon, Apr 23, 2018 at 1:45 PM, Jeremy Audet wrote: > Thanks for answering my questions, all. > > > > Let's say pulp_file is being tested. Is both it and pulpcore > installed from source? Or is pulpcore installed from egg/wheel? > > > > Yes, we’re testing against source which also allows us to test against > different PRs. Often, a pulpcore change will require a pulp_file change or > vice versa and we just added some functionality recently that actually lets > us test against PRs too.[1] pulp-smash is also running against source or > PRs. IMO, testing against source lets us move faster than testing against > releases. Maybe we can look at switching though once things have stabilized > after the MVP. > > This seems fine in most cases. But when verifying whether a plugin can be > made into a wheel, this seems like a bad strategy. A brand new wheel should > work well when installed alongside other wheels from PyPI, and testing > against from-source installations gives no assurance that this is so. > > That is a great observation. When we automate releasing pulp_file or another plugin to PyPI using Travis, we will make sure to run tests against an already published build of pulpcore. > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
I feel like the Travis change is recent enough that I'm not exactly sure how they differ from the Jenkins jobs. Are we all clear on these terminology? Aren't there multiple jobs running at different times? I am not comfortable enough without the assurance that we are all talking about the same things to say we don't need "the Jenkins jobs". I would like for some of these discussions around release process and where (and what and when) different tests are run be a little bit more collaborative from the beginning rather than reactive post-change happening. I'm all for making quick incremental changes. I'm happy that there are test results that are accessible to everyone. I don't want us to over thinking these, but are we sure we don't have any tests that need to stay private (do we test for any security type things that might need to be embargoed before reported)? I want to make sure we think thing through before we start tearing stuff down. On Mon, Apr 23, 2018 at 1:37 PM, Brian Bouterse wrote: > > On Mon, Apr 23, 2018 at 1:25 PM, David Davis > wrote: > >> > Are those Travis jobs testing combinations of web servers, AMQP >> brokers, databases, etc? If not, is testing across those combinations a >> goal? >> >> This is definitely the goal. Right now, our build matrix tests against >> different databases, different Django versions, and different python >> versions. We could look at testing against different webservers but right >> now we just use django’s built-in webserver. AMQP is going away though[0]. >> >> > Let's say pulp_file is being tested. Is both it and pulpcore installed >> from source? Or is pulpcore installed from egg/wheel? >> >> Yes, we’re testing against source which also allows us to test against >> different PRs. Often, a pulpcore change will require a pulp_file change or >> vice versa and we just added some functionality recently that actually lets >> us test against PRs too.[1] pulp-smash is also running against source or >> PRs. IMO, testing against source lets us move faster than testing against >> releases. Maybe we can look at switching though once things have stabilized >> after the MVP. >> >> > Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? >> >> I don’t think we need them. We can trigger builds on an as-needed basis >> with Travis without making git commits. We’re also running builds nightly >> right now with Travis’ cron. >> > > I agree we don't need the Jenkins jobs for Pulp3 and they can be deleted. > We still need to keep them for Pulp2 because Pulp2 can't run on Travis. > Moving to Travis is a big step forward for the community since Travis is > readable by the whole Internet while to read Jenkins jobs, you have to be a > Red Hat employee. > > >> >> > What does this mean for goals like multi-host testing? >> >> It looks like Travis supports running docker containers[2] so maybe we >> could leverage containers to test this sort of setup. There are definitely >> limitations of Travis that might require us to move to something more >> robust in the future but short-term I think/hope Travis will fit our needs. >> >> [0] https://github.com/pulp/pulp/pull/3454 >> [1] https://github.com/pulp/pulp/pull/3435 >> [2] https://docs.travis-ci.com/user/docker/ >> >> >> David >> >> On Mon, Apr 23, 2018 at 12:10 PM, Jeremy Audet wrote: >> >>> Using Travis jobs instead of Jenkins jobs for testing Pulp 3 releases >>> begs several questions: >>> >>>- Are those Travis jobs testing combinations of web servers, AMQP >>>brokers, databases, etc? If not, is testing across those combinations a >>>goal? >>>- Let's say pulp_file is being tested. Is both it and pulpcore >>>installed from source? Or is pulpcore installed from egg/wheel? >>>- Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? >>>(Possible answer: So that we can trigger test runs on an as-needed basis, >>>without making new git commits.) >>>- What does this mean for goals like multi-host testing? That is, >>>what does this mean for the case where a single Pulp 3 application is >>>deployed across multiple hosts? Can that be tested with Travis? Pulp >>> Smash >>>is architected with this in mind, and it's been a long standing goal. The >>>biggest impediment has been the use of legacy Jenkins (1.x) and nodepool, >>>and recent upgrades make this a more realistic testing target. >>> >>> If these questions have already been considered and answered somewhere, >>> please point me to the correct resource. >>> >> >> > > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
Thanks for answering my questions, all. > > Let's say pulp_file is being tested. Is both it and pulpcore installed from source? Or is pulpcore installed from egg/wheel? > > Yes, we’re testing against source which also allows us to test against different PRs. Often, a pulpcore change will require a pulp_file change or vice versa and we just added some functionality recently that actually lets us test against PRs too.[1] pulp-smash is also running against source or PRs. IMO, testing against source lets us move faster than testing against releases. Maybe we can look at switching though once things have stabilized after the MVP. This seems fine in most cases. But when verifying whether a plugin can be made into a wheel, this seems like a bad strategy. A brand new wheel should work well when installed alongside other wheels from PyPI, and testing against from-source installations gives no assurance that this is so. ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
On Mon, Apr 23, 2018 at 1:25 PM, David Davis wrote: > > Are those Travis jobs testing combinations of web servers, AMQP > brokers, databases, etc? If not, is testing across those combinations a > goal? > > This is definitely the goal. Right now, our build matrix tests against > different databases, different Django versions, and different python > versions. We could look at testing against different webservers but right > now we just use django’s built-in webserver. AMQP is going away though[0]. > > > Let's say pulp_file is being tested. Is both it and pulpcore installed > from source? Or is pulpcore installed from egg/wheel? > > Yes, we’re testing against source which also allows us to test against > different PRs. Often, a pulpcore change will require a pulp_file change or > vice versa and we just added some functionality recently that actually lets > us test against PRs too.[1] pulp-smash is also running against source or > PRs. IMO, testing against source lets us move faster than testing against > releases. Maybe we can look at switching though once things have stabilized > after the MVP. > > > Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? > > I don’t think we need them. We can trigger builds on an as-needed basis > with Travis without making git commits. We’re also running builds nightly > right now with Travis’ cron. > I agree we don't need the Jenkins jobs for Pulp3 and they can be deleted. We still need to keep them for Pulp2 because Pulp2 can't run on Travis. Moving to Travis is a big step forward for the community since Travis is readable by the whole Internet while to read Jenkins jobs, you have to be a Red Hat employee. > > > What does this mean for goals like multi-host testing? > > It looks like Travis supports running docker containers[2] so maybe we > could leverage containers to test this sort of setup. There are definitely > limitations of Travis that might require us to move to something more > robust in the future but short-term I think/hope Travis will fit our needs. > > [0] https://github.com/pulp/pulp/pull/3454 > [1] https://github.com/pulp/pulp/pull/3435 > [2] https://docs.travis-ci.com/user/docker/ > > > David > > On Mon, Apr 23, 2018 at 12:10 PM, Jeremy Audet wrote: > >> Using Travis jobs instead of Jenkins jobs for testing Pulp 3 releases >> begs several questions: >> >>- Are those Travis jobs testing combinations of web servers, AMQP >>brokers, databases, etc? If not, is testing across those combinations a >>goal? >>- Let's say pulp_file is being tested. Is both it and pulpcore >>installed from source? Or is pulpcore installed from egg/wheel? >>- Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? >>(Possible answer: So that we can trigger test runs on an as-needed basis, >>without making new git commits.) >>- What does this mean for goals like multi-host testing? That is, >>what does this mean for the case where a single Pulp 3 application is >>deployed across multiple hosts? Can that be tested with Travis? Pulp Smash >>is architected with this in mind, and it's been a long standing goal. The >>biggest impediment has been the use of legacy Jenkins (1.x) and nodepool, >>and recent upgrades make this a more realistic testing target. >> >> If these questions have already been considered and answered somewhere, >> please point me to the correct resource. >> > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
> Are those Travis jobs testing combinations of web servers, AMQP brokers, databases, etc? If not, is testing across those combinations a goal? This is definitely the goal. Right now, our build matrix tests against different databases, different Django versions, and different python versions. We could look at testing against different webservers but right now we just use django’s built-in webserver. AMQP is going away though[0]. > Let's say pulp_file is being tested. Is both it and pulpcore installed from source? Or is pulpcore installed from egg/wheel? Yes, we’re testing against source which also allows us to test against different PRs. Often, a pulpcore change will require a pulp_file change or vice versa and we just added some functionality recently that actually lets us test against PRs too.[1] pulp-smash is also running against source or PRs. IMO, testing against source lets us move faster than testing against releases. Maybe we can look at switching though once things have stabilized after the MVP. > Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? I don’t think we need them. We can trigger builds on an as-needed basis with Travis without making git commits. We’re also running builds nightly right now with Travis’ cron. > What does this mean for goals like multi-host testing? It looks like Travis supports running docker containers[2] so maybe we could leverage containers to test this sort of setup. There are definitely limitations of Travis that might require us to move to something more robust in the future but short-term I think/hope Travis will fit our needs. [0] https://github.com/pulp/pulp/pull/3454 [1] https://github.com/pulp/pulp/pull/3435 [2] https://docs.travis-ci.com/user/docker/ David On Mon, Apr 23, 2018 at 12:10 PM, Jeremy Audet wrote: > Using Travis jobs instead of Jenkins jobs for testing Pulp 3 releases begs > several questions: > >- Are those Travis jobs testing combinations of web servers, AMQP >brokers, databases, etc? If not, is testing across those combinations a >goal? >- Let's say pulp_file is being tested. Is both it and pulpcore >installed from source? Or is pulpcore installed from egg/wheel? >- Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? >(Possible answer: So that we can trigger test runs on an as-needed basis, >without making new git commits.) >- What does this mean for goals like multi-host testing? That is, what >does this mean for the case where a single Pulp 3 application is deployed >across multiple hosts? Can that be tested with Travis? Pulp Smash is >architected with this in mind, and it's been a long standing goal. The >biggest impediment has been the use of legacy Jenkins (1.x) and nodepool, >and recent upgrades make this a more realistic testing target. > > If these questions have already been considered and answered somewhere, > please point me to the correct resource. > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
> > Are those Travis jobs testing combinations of web servers, AMQP brokers, > databases, etc? If not, is testing across those combinations a goal? The travis jobs currently text a matrix of Django version (2.0 and 1.1 LTS), database (sqlite and postgresql), and python version (3.5 and 3.6). Presumably the broker variable is about to go away, and we could theoretically easily add web server tests in later but we don't have the Apache / Nginx configs written yet. I feel like if we add much more to Travis though, it's going to start getting bogged down. It has already gotten pretty slow at times. We should consider how well (or poorly) it's going to scale if we start adding more and more tests and combinations to the Travis PR test runners. Maybe if some combos only ran nightly... Let's say pulp_file is being tested. Is both it and pulpcore installed from > source? Or is pulpcore installed from egg/wheel? Currently, source. Dennis added a nice feature where if you specify a Pulp Smash or Pulp File PR in your commit, it will check out that specific pull request and run tests against it. What does this mean for goals like multi-host testing? That is, what does > this mean for the case where a single Pulp 3 application is deployed across > multiple hosts? Can that be tested with Travis? I'm pretty sure we can't do that with Travis, although someone correct me if I'm wrong. On Mon, Apr 23, 2018 at 12:10 PM, Jeremy Audet wrote: > Using Travis jobs instead of Jenkins jobs for testing Pulp 3 releases begs > several questions: > >- Are those Travis jobs testing combinations of web servers, AMQP >brokers, databases, etc? If not, is testing across those combinations a >goal? >- Let's say pulp_file is being tested. Is both it and pulpcore >installed from source? Or is pulpcore installed from egg/wheel? >- Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? >(Possible answer: So that we can trigger test runs on an as-needed basis, >without making new git commits.) >- What does this mean for goals like multi-host testing? That is, what >does this mean for the case where a single Pulp 3 application is deployed >across multiple hosts? Can that be tested with Travis? Pulp Smash is >architected with this in mind, and it's been a long standing goal. The >biggest impediment has been the use of legacy Jenkins (1.x) and nodepool, >and recent upgrades make this a more realistic testing target. > > If these questions have already been considered and answered somewhere, > please point me to the correct resource. > > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
Using Travis jobs instead of Jenkins jobs for testing Pulp 3 releases begs several questions: - Are those Travis jobs testing combinations of web servers, AMQP brokers, databases, etc? If not, is testing across those combinations a goal? - Let's say pulp_file is being tested. Is both it and pulpcore installed from source? Or is pulpcore installed from egg/wheel? - Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs? (Possible answer: So that we can trigger test runs on an as-needed basis, without making new git commits.) - What does this mean for goals like multi-host testing? That is, what does this mean for the case where a single Pulp 3 application is deployed across multiple hosts? Can that be tested with Travis? Pulp Smash is architected with this in mind, and it's been a long standing goal. The biggest impediment has been the use of legacy Jenkins (1.x) and nodepool, and recent upgrades make this a more realistic testing target. If these questions have already been considered and answered somewhere, please point me to the correct resource. ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
Travis also supports secure signing so having travis handle releases seems viable. For signing, I think it's desirable to have 1 key to trust no matter how you are installing Pulp (pypi, rpm, debian, etc). To do that we should coordinate our keys before we start signing builds. We probably need to use subkeys. I hope we defer signing for several months. Feedback from the community or other stakeholders on their needs for signed builds would help inform this area. On Mon, Apr 23, 2018 at 9:07 AM, David Davis wrote: > I can only answer the Travis question and I think we tentatively want to > use Travis to push packages to PyPI. It looks like Travis can automatically > deploy tags[0] which is what I think we’ll leverage to do releases. > > [0] https://docs.travis-ci.com/user/deployment# > Conditional-Releases-with-on%3A > > > David > > On Fri, Apr 20, 2018 at 11:16 AM, Patrick Creech > wrote: > >> Thanks Robin! >> >> On Thu, 2018-04-19 at 16:34 -0400, Robin Chan wrote: >> > Dennis, Eric, & Patrick, >> > >> > Thanks for this additional information around this motivation behind >> some of the differences between Pulp 2 and Pulp 3 release options. I'm glad >> to hear that Pulp 2 had some constraints that Pulp 3 >> > does not have and that Pulp 3 can now be published on PyPI. This is >> great and a good change. A release process in Pulp 3 that allows for >> community contribution of a wide range of Linux >> > distributions would be fantastic and I fully support that goal. >> >> +1, The increase in accessibility of the pulp project with pulp3 to other >> user bases is a great thing. >> >> > When I see the term "derivative" packaging, I'm not clear on what is >> meant here. If it simply means that it is created after that, then I don't >> see is how the above goals or changes make RPM >> > deliverables something we don't have any commitments around. >> >> This is what we (the build team) believe it to mean. Whatever is >> published to PyPI becomes the source of reference here, and other packaging >> flows from these bits. As long as proper quality control >> measures are in place (before the publish to PyPI) that maintain the >> confidince in quality that pulp has earned during pulp 2, then the process >> of ensuring the availability of RPM packages from PyPI >> bits becomes much simpler and straightforward and easier to automate. >> >> > I suspect that Pulp 3.0 Core Beta consumers would be OK with taking >> just PyPI delivery of Pulp 3.0 core beta code even though we committed to >> RPM deliverables. >> >> Although there are other high priority items on our plate, we committed >> to having a software collection and beta rpm bits available on/near the >> beta date. Unless things change with those other >> priority items, I'm comfortable working to keep this commitment. >> >> > I also think some additional discussion of testing by various tools and >> teams would be good to have in a collaborative, open way. >> >> +1 >> >> --- >> >> With all of the above, I think where we (the build team) need immediate >> help to move forward is to figure out where rpms need to be hosted, and >> figuring out the signing processes. I don't think we >> can use Pulp 2's gpg keys with Pulp 3 (well, technically we could, but it >> was always the intention to change the GPG key with X releases on pulp). >> That way we can be ready to have 'something' hosted >> close to beta day. As far as a weekly release frequency for beta, I'm >> not sure we could keep up that pace with RPM packaging untill we iron out >> the process more. >> >> Also, it appears that Pulp has adopted the use of Travis upstream for >> pulp 3, I'm assuming that's how the pushes to pypi will happen? >> >> Thanks, >> Patrick >> ___ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> >> > > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
I can only answer the Travis question and I think we tentatively want to use Travis to push packages to PyPI. It looks like Travis can automatically deploy tags[0] which is what I think we’ll leverage to do releases. [0] https://docs.travis-ci.com/user/deployment#Conditional-Releases-with-on%3A David On Fri, Apr 20, 2018 at 11:16 AM, Patrick Creech wrote: > Thanks Robin! > > On Thu, 2018-04-19 at 16:34 -0400, Robin Chan wrote: > > Dennis, Eric, & Patrick, > > > > Thanks for this additional information around this motivation behind > some of the differences between Pulp 2 and Pulp 3 release options. I'm glad > to hear that Pulp 2 had some constraints that Pulp 3 > > does not have and that Pulp 3 can now be published on PyPI. This is > great and a good change. A release process in Pulp 3 that allows for > community contribution of a wide range of Linux > > distributions would be fantastic and I fully support that goal. > > +1, The increase in accessibility of the pulp project with pulp3 to other > user bases is a great thing. > > > When I see the term "derivative" packaging, I'm not clear on what is > meant here. If it simply means that it is created after that, then I don't > see is how the above goals or changes make RPM > > deliverables something we don't have any commitments around. > > This is what we (the build team) believe it to mean. Whatever is > published to PyPI becomes the source of reference here, and other packaging > flows from these bits. As long as proper quality control > measures are in place (before the publish to PyPI) that maintain the > confidince in quality that pulp has earned during pulp 2, then the process > of ensuring the availability of RPM packages from PyPI > bits becomes much simpler and straightforward and easier to automate. > > > I suspect that Pulp 3.0 Core Beta consumers would be OK with taking just > PyPI delivery of Pulp 3.0 core beta code even though we committed to RPM > deliverables. > > Although there are other high priority items on our plate, we committed to > having a software collection and beta rpm bits available on/near the beta > date. Unless things change with those other > priority items, I'm comfortable working to keep this commitment. > > > I also think some additional discussion of testing by various tools and > teams would be good to have in a collaborative, open way. > > +1 > > --- > > With all of the above, I think where we (the build team) need immediate > help to move forward is to figure out where rpms need to be hosted, and > figuring out the signing processes. I don't think we > can use Pulp 2's gpg keys with Pulp 3 (well, technically we could, but it > was always the intention to change the GPG key with X releases on pulp). > That way we can be ready to have 'something' hosted > close to beta day. As far as a weekly release frequency for beta, I'm not > sure we could keep up that pace with RPM packaging untill we iron out the > process more. > > Also, it appears that Pulp has adopted the use of Travis upstream for pulp > 3, I'm assuming that's how the pushes to pypi will happen? > > Thanks, > Patrick > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
Thanks Robin! On Thu, 2018-04-19 at 16:34 -0400, Robin Chan wrote: > Dennis, Eric, & Patrick, > > Thanks for this additional information around this motivation behind some of > the differences between Pulp 2 and Pulp 3 release options. I'm glad to hear > that Pulp 2 had some constraints that Pulp 3 > does not have and that Pulp 3 can now be published on PyPI. This is great and > a good change. A release process in Pulp 3 that allows for community > contribution of a wide range of Linux > distributions would be fantastic and I fully support that goal. +1, The increase in accessibility of the pulp project with pulp3 to other user bases is a great thing. > When I see the term "derivative" packaging, I'm not clear on what is meant > here. If it simply means that it is created after that, then I don't see is > how the above goals or changes make RPM > deliverables something we don't have any commitments around. This is what we (the build team) believe it to mean. Whatever is published to PyPI becomes the source of reference here, and other packaging flows from these bits. As long as proper quality control measures are in place (before the publish to PyPI) that maintain the confidince in quality that pulp has earned during pulp 2, then the process of ensuring the availability of RPM packages from PyPI bits becomes much simpler and straightforward and easier to automate. > I suspect that Pulp 3.0 Core Beta consumers would be OK with taking just PyPI > delivery of Pulp 3.0 core beta code even though we committed to RPM > deliverables. Although there are other high priority items on our plate, we committed to having a software collection and beta rpm bits available on/near the beta date. Unless things change with those other priority items, I'm comfortable working to keep this commitment. > I also think some additional discussion of testing by various tools and teams > would be good to have in a collaborative, open way. +1 --- With all of the above, I think where we (the build team) need immediate help to move forward is to figure out where rpms need to be hosted, and figuring out the signing processes. I don't think we can use Pulp 2's gpg keys with Pulp 3 (well, technically we could, but it was always the intention to change the GPG key with X releases on pulp). That way we can be ready to have 'something' hosted close to beta day. As far as a weekly release frequency for beta, I'm not sure we could keep up that pace with RPM packaging untill we iron out the process more. Also, it appears that Pulp has adopted the use of Travis upstream for pulp 3, I'm assuming that's how the pushes to pypi will happen? Thanks, Patrick signature.asc Description: This is a digitally signed message part ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
Dennis, Eric, & Patrick, Thanks for this additional information around this motivation behind some of the differences between Pulp 2 and Pulp 3 release options. I'm glad to hear that Pulp 2 had some constraints that Pulp 3 does not have and that Pulp 3 can now be published on PyPI. This is great and a good change. A release process in Pulp 3 that allows for community contribution of a wide range of Linux distributions would be fantastic and I fully support that goal. When I see the term "derivative" packaging, I'm not clear on what is meant here. If it simply means that it is created after that, then I don't see is how the above goals or changes make RPM deliverables something we don't have any commitments around. I would disagree that the build team can decide how often they build RPMs. Since the existing community has consumed Pulp 2 as RPMs, I was assuming that Pulp 3 (at the latest 3.0 GA) needed to be delivered to PyPI and also as RPMs. A deviation should be at least vetted with stakeholders before deciding the need? Is there feedback or some industry standard/example that I'm not aware of? Can we check with stakeholders and consumers if they are ok with taking Pulp 3.0 GA from PyPI only or propose a minimum commitment around how often RPMS would need to be made available? The build team can help support us and in our commitments, but I don't think they have a direct commitment to Pulp stakeholders - I think that's the rub here. I suspect that Pulp 3.0 Core Beta consumers would be OK with taking just PyPI delivery of Pulp 3.0 core beta code even though we committed to RPM deliverables. I also think some additional discussion of testing by various tools and teams would be good to have in a collaborative, open way. -Robin On Wed, Apr 18, 2018 at 4:05 PM, Dennis Kliban wrote: > One of the requirements for Pulp 3 is to be able to run on a wide range of > Linux distributions. Being a Python application, that means we can achieve > that goal by following good Python development practices. Part of those > good practices is releasing packages via the Python Package Index (PyPI). > > Pulp 2 could never be published on PyPI because it had dependencies that > were not available from PyPI. Pulp 3 was designed to be installed from > PyPI. It's dependencies were carefully selected to meet this requirement. > > On Wed, Apr 18, 2018 at 3:17 PM, Robin Chan wrote: > >> Since this is a change from Pulp 2, I think it would be helpful to >> outline the reasoning behind such a change and ask that spell those out >> here for transparency. In addition, are there any concerns we think others >> may have or new problems that such a change brings about that we need to >> work to answer? >> >> On Tue, Apr 17, 2018 at 11:57 AM, Dennis Kliban >> wrote: >> >>> On Tue, Apr 17, 2018 at 11:50 AM, Eric Helms wrote: >>> On Fri, Apr 13, 2018 at 4:40 PM, Dennis Kliban wrote: > On Fri, Apr 13, 2018 at 1:25 PM, Patrick Creech > wrote: > >> Pulp, >> >> So, while working on the packaging work, I figured it be nice to >> start talking about release process expectations around nightlies, beta, >> and GA. >> >> Generally, what is pulp's release plan? What are the expectations >> here? >> >> > The release process for Pulp 3 will be different from what we do for > Pulp 2. Our plan for publishing Pulp 3 with quality to PyPI is outlined on > our wiki[0]. We are hoping to be able to release to PyPI once a week > during > the beta cycle. After the packages are published to PyPI, any of the > derivative packaging (RPM, Debian, etc) can be performed. The build team > can decide how often the derivative packages need to be produced. > This implies that, for the Pulp developer team, Pypi is considered the release vector and that derivative release vectors (e.g. RPM, Deb, etc.) are considered community contributions that are not part of the core release process. Is that a fair summary of the position? Consumers of non-pypi release vectors will need to assume a delay between announced release and RPM release. Which then, unlike Pulp 2, means the team handling RPM for example would manage build and release announcement on our own schedule. I want to clarify so that we set expectations for developers and users and so that we can set our expectations for how we shift compared to Pulp 2. >>> You are correct in your understanding. >>> >>> If the above is the agreed workflow (and change for Pulp 3) I think the rest of the questions I'd ask related to the points below are answered and we can talk a bit further on these points above. - Eric > > [0] https://pulp.plan.io/projects/pulp/wiki/Continuous_Delivery_ > of_Pulp_3 > > >> And also, more specifically, >> >> Based on what we do for pulp 2, when will pulp 'c
Re: [Pulp-dev] Pulp 3 Release Process Questions
One of the requirements for Pulp 3 is to be able to run on a wide range of Linux distributions. Being a Python application, that means we can achieve that goal by following good Python development practices. Part of those good practices is releasing packages via the Python Package Index (PyPI). Pulp 2 could never be published on PyPI because it had dependencies that were not available from PyPI. Pulp 3 was designed to be installed from PyPI. It's dependencies were carefully selected to meet this requirement. On Wed, Apr 18, 2018 at 3:17 PM, Robin Chan wrote: > Since this is a change from Pulp 2, I think it would be helpful to outline > the reasoning behind such a change and ask that spell those out here for > transparency. In addition, are there any concerns we think others may have > or new problems that such a change brings about that we need to work to > answer? > > On Tue, Apr 17, 2018 at 11:57 AM, Dennis Kliban > wrote: > >> On Tue, Apr 17, 2018 at 11:50 AM, Eric Helms wrote: >> >>> >>> >>> On Fri, Apr 13, 2018 at 4:40 PM, Dennis Kliban >>> wrote: >>> On Fri, Apr 13, 2018 at 1:25 PM, Patrick Creech wrote: > Pulp, > > So, while working on the packaging work, I figured it be nice to start > talking about release process expectations around nightlies, beta, and GA. > > Generally, what is pulp's release plan? What are the expectations > here? > > The release process for Pulp 3 will be different from what we do for Pulp 2. Our plan for publishing Pulp 3 with quality to PyPI is outlined on our wiki[0]. We are hoping to be able to release to PyPI once a week during the beta cycle. After the packages are published to PyPI, any of the derivative packaging (RPM, Debian, etc) can be performed. The build team can decide how often the derivative packages need to be produced. >>> >>> This implies that, for the Pulp developer team, Pypi is considered the >>> release vector and that derivative release vectors (e.g. RPM, Deb, etc.) >>> are considered community contributions that are not part of the core >>> release process. Is that a fair summary of the position? Consumers of >>> non-pypi release vectors will need to assume a delay between announced >>> release and RPM release. Which then, unlike Pulp 2, means the team handling >>> RPM for example would manage build and release announcement on our own >>> schedule. I want to clarify so that we set expectations for developers and >>> users and so that we can set our expectations for how we shift compared to >>> Pulp 2. >>> >>> >> You are correct in your understanding. >> >> >>> If the above is the agreed workflow (and change for Pulp 3) I think the >>> rest of the questions I'd ask related to the points below are answered and >>> we can talk a bit further on these points above. >>> >>> - Eric >>> >>> [0] https://pulp.plan.io/projects/pulp/wiki/Continuous_Delivery_ of_Pulp_3 > And also, more specifically, > > Based on what we do for pulp 2, when will pulp 'code freeze'? What is > the expected turnaround from 'code freeze to 'packages shipped'. We > should > probably agree on some expectations of turnaround > time. > > The code will be frozen when it is published to PyPI. > Is there a staging process in place yet for packages (pypi or rpm)? Is > there testing expectations of these pre-release bits to ensure quality? > With pypi being a valid install location, are these > releases to be coordinated? > > As outlined on the wiki, we plan to ensure quality at merge time of every commit. > Where are pulp 3 bits expected to be hosted? How are we going to > handle signing packages? > Pulp 3 will always be published to PyPI. Any derivative packages can be hosted on fedorapeople.org. I'd like to defer to someone else to speak about the signing. > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev >>> >> >> ___ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> >> > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
Since this is a change from Pulp 2, I think it would be helpful to outline the reasoning behind such a change and ask that spell those out here for transparency. In addition, are there any concerns we think others may have or new problems that such a change brings about that we need to work to answer? On Tue, Apr 17, 2018 at 11:57 AM, Dennis Kliban wrote: > On Tue, Apr 17, 2018 at 11:50 AM, Eric Helms wrote: > >> >> >> On Fri, Apr 13, 2018 at 4:40 PM, Dennis Kliban >> wrote: >> >>> On Fri, Apr 13, 2018 at 1:25 PM, Patrick Creech >>> wrote: >>> Pulp, So, while working on the packaging work, I figured it be nice to start talking about release process expectations around nightlies, beta, and GA. Generally, what is pulp's release plan? What are the expectations here? >>> The release process for Pulp 3 will be different from what we do for >>> Pulp 2. Our plan for publishing Pulp 3 with quality to PyPI is outlined on >>> our wiki[0]. We are hoping to be able to release to PyPI once a week during >>> the beta cycle. After the packages are published to PyPI, any of the >>> derivative packaging (RPM, Debian, etc) can be performed. The build team >>> can decide how often the derivative packages need to be produced. >>> >> >> This implies that, for the Pulp developer team, Pypi is considered the >> release vector and that derivative release vectors (e.g. RPM, Deb, etc.) >> are considered community contributions that are not part of the core >> release process. Is that a fair summary of the position? Consumers of >> non-pypi release vectors will need to assume a delay between announced >> release and RPM release. Which then, unlike Pulp 2, means the team handling >> RPM for example would manage build and release announcement on our own >> schedule. I want to clarify so that we set expectations for developers and >> users and so that we can set our expectations for how we shift compared to >> Pulp 2. >> >> > You are correct in your understanding. > > >> If the above is the agreed workflow (and change for Pulp 3) I think the >> rest of the questions I'd ask related to the points below are answered and >> we can talk a bit further on these points above. >> >> - Eric >> >> >>> >>> [0] https://pulp.plan.io/projects/pulp/wiki/Continuous_Delivery_ >>> of_Pulp_3 >>> >>> And also, more specifically, Based on what we do for pulp 2, when will pulp 'code freeze'? What is the expected turnaround from 'code freeze to 'packages shipped'. We should probably agree on some expectations of turnaround time. >>> The code will be frozen when it is published to PyPI. >>> >>> Is there a staging process in place yet for packages (pypi or rpm)? Is there testing expectations of these pre-release bits to ensure quality? With pypi being a valid install location, are these releases to be coordinated? >>> As outlined on the wiki, we plan to ensure quality at merge time of >>> every commit. >>> >>> Where are pulp 3 bits expected to be hosted? How are we going to handle signing packages? >>> >>> Pulp 3 will always be published to PyPI. Any derivative packages can be >>> hosted on fedorapeople.org. I'd like to defer to someone else to speak >>> about the signing. >>> >>> ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev >>> >>> ___ >>> Pulp-dev mailing list >>> Pulp-dev@redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-dev >>> >>> >> > > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
On Tue, Apr 17, 2018 at 11:50 AM, Eric Helms wrote: > > > On Fri, Apr 13, 2018 at 4:40 PM, Dennis Kliban wrote: > >> On Fri, Apr 13, 2018 at 1:25 PM, Patrick Creech >> wrote: >> >>> Pulp, >>> >>> So, while working on the packaging work, I figured it be nice to start >>> talking about release process expectations around nightlies, beta, and GA. >>> >>> Generally, what is pulp's release plan? What are the expectations here? >>> >>> >> The release process for Pulp 3 will be different from what we do for Pulp >> 2. Our plan for publishing Pulp 3 with quality to PyPI is outlined on our >> wiki[0]. We are hoping to be able to release to PyPI once a week during the >> beta cycle. After the packages are published to PyPI, any of the >> derivative packaging (RPM, Debian, etc) can be performed. The build team >> can decide how often the derivative packages need to be produced. >> > > This implies that, for the Pulp developer team, Pypi is considered the > release vector and that derivative release vectors (e.g. RPM, Deb, etc.) > are considered community contributions that are not part of the core > release process. Is that a fair summary of the position? Consumers of > non-pypi release vectors will need to assume a delay between announced > release and RPM release. Which then, unlike Pulp 2, means the team handling > RPM for example would manage build and release announcement on our own > schedule. I want to clarify so that we set expectations for developers and > users and so that we can set our expectations for how we shift compared to > Pulp 2. > > You are correct in your understanding. > If the above is the agreed workflow (and change for Pulp 3) I think the > rest of the questions I'd ask related to the points below are answered and > we can talk a bit further on these points above. > > - Eric > > >> >> [0] https://pulp.plan.io/projects/pulp/wiki/Continuous_Delivery_of_Pulp_3 >> >> >>> And also, more specifically, >>> >>> Based on what we do for pulp 2, when will pulp 'code freeze'? What is >>> the expected turnaround from 'code freeze to 'packages shipped'. We should >>> probably agree on some expectations of turnaround >>> time. >>> >>> >> The code will be frozen when it is published to PyPI. >> >> >>> Is there a staging process in place yet for packages (pypi or rpm)? Is >>> there testing expectations of these pre-release bits to ensure quality? >>> With pypi being a valid install location, are these >>> releases to be coordinated? >>> >>> >> As outlined on the wiki, we plan to ensure quality at merge time of every >> commit. >> >> >>> Where are pulp 3 bits expected to be hosted? How are we going to handle >>> signing packages? >>> >> >> Pulp 3 will always be published to PyPI. Any derivative packages can be >> hosted on fedorapeople.org. I'd like to defer to someone else to speak >> about the signing. >> >> >>> ___ >>> Pulp-dev mailing list >>> Pulp-dev@redhat.com >>> https://www.redhat.com/mailman/listinfo/pulp-dev >>> >>> >> >> ___ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> >> > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
On Fri, Apr 13, 2018 at 4:40 PM, Dennis Kliban wrote: > On Fri, Apr 13, 2018 at 1:25 PM, Patrick Creech > wrote: > >> Pulp, >> >> So, while working on the packaging work, I figured it be nice to start >> talking about release process expectations around nightlies, beta, and GA. >> >> Generally, what is pulp's release plan? What are the expectations here? >> >> > The release process for Pulp 3 will be different from what we do for Pulp > 2. Our plan for publishing Pulp 3 with quality to PyPI is outlined on our > wiki[0]. We are hoping to be able to release to PyPI once a week during the > beta cycle. After the packages are published to PyPI, any of the > derivative packaging (RPM, Debian, etc) can be performed. The build team > can decide how often the derivative packages need to be produced. > This implies that, for the Pulp developer team, Pypi is considered the release vector and that derivative release vectors (e.g. RPM, Deb, etc.) are considered community contributions that are not part of the core release process. Is that a fair summary of the position? Consumers of non-pypi release vectors will need to assume a delay between announced release and RPM release. Which then, unlike Pulp 2, means the team handling RPM for example would manage build and release announcement on our own schedule. I want to clarify so that we set expectations for developers and users and so that we can set our expectations for how we shift compared to Pulp 2. If the above is the agreed workflow (and change for Pulp 3) I think the rest of the questions I'd ask related to the points below are answered and we can talk a bit further on these points above. - Eric > > [0] https://pulp.plan.io/projects/pulp/wiki/Continuous_Delivery_of_Pulp_3 > > >> And also, more specifically, >> >> Based on what we do for pulp 2, when will pulp 'code freeze'? What is the >> expected turnaround from 'code freeze to 'packages shipped'. We should >> probably agree on some expectations of turnaround >> time. >> >> > The code will be frozen when it is published to PyPI. > > >> Is there a staging process in place yet for packages (pypi or rpm)? Is >> there testing expectations of these pre-release bits to ensure quality? >> With pypi being a valid install location, are these >> releases to be coordinated? >> >> > As outlined on the wiki, we plan to ensure quality at merge time of every > commit. > > >> Where are pulp 3 bits expected to be hosted? How are we going to handle >> signing packages? >> > > Pulp 3 will always be published to PyPI. Any derivative packages can be > hosted on fedorapeople.org. I'd like to defer to someone else to speak > about the signing. > > >> ___ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> >> > > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
Re: [Pulp-dev] Pulp 3 Release Process Questions
On Fri, Apr 13, 2018 at 1:25 PM, Patrick Creech wrote: > Pulp, > > So, while working on the packaging work, I figured it be nice to start > talking about release process expectations around nightlies, beta, and GA. > > Generally, what is pulp's release plan? What are the expectations here? > > The release process for Pulp 3 will be different from what we do for Pulp 2. Our plan for publishing Pulp 3 with quality to PyPI is outlined on our wiki[0]. We are hoping to be able to release to PyPI once a week during the beta cycle. After the packages are published to PyPI, any of the derivative packaging (RPM, Debian, etc) can be performed. The build team can decide how often the derivative packages need to be produced. [0] https://pulp.plan.io/projects/pulp/wiki/Continuous_Delivery_of_Pulp_3 > And also, more specifically, > > Based on what we do for pulp 2, when will pulp 'code freeze'? What is the > expected turnaround from 'code freeze to 'packages shipped'. We should > probably agree on some expectations of turnaround > time. > > The code will be frozen when it is published to PyPI. > Is there a staging process in place yet for packages (pypi or rpm)? Is > there testing expectations of these pre-release bits to ensure quality? > With pypi being a valid install location, are these > releases to be coordinated? > > As outlined on the wiki, we plan to ensure quality at merge time of every commit. > Where are pulp 3 bits expected to be hosted? How are we going to handle > signing packages? > Pulp 3 will always be published to PyPI. Any derivative packages can be hosted on fedorapeople.org. I'd like to defer to someone else to speak about the signing. > ___ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > > ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
[Pulp-dev] Pulp 3 Release Process Questions
Pulp, So, while working on the packaging work, I figured it be nice to start talking about release process expectations around nightlies, beta, and GA. Generally, what is pulp's release plan? What are the expectations here? And also, more specifically, Based on what we do for pulp 2, when will pulp 'code freeze'? What is the expected turnaround from 'code freeze to 'packages shipped'. We should probably agree on some expectations of turnaround time. Is there a staging process in place yet for packages (pypi or rpm)? Is there testing expectations of these pre-release bits to ensure quality? With pypi being a valid install location, are these releases to be coordinated? Where are pulp 3 bits expected to be hosted? How are we going to handle signing packages? signature.asc Description: This is a digitally signed message part ___ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
