Are we running only a subset? I think currently we run all of them on Travis before each merge.
I think running all of them for now is the best way. If the runtime on Travis becomes unbearable we can look into solutions then. On Tue, Apr 17, 2018 at 4:22 PM, Daniel Alley <dal...@redhat.com> wrote: > bmbouter is fixing the docs link in a PR: https://github.com/pulp/pulp/p > ull/3446 > > My understanding is that the smoke tests are for PRs, but actual releases > would still be tested against the full test suite. > > > On Tue, Apr 17, 2018 at 3:25 PM, Preethi Thomas <ptho...@redhat.com> > wrote: > >> >> >> On Tue, Apr 17, 2018 at 8:32 AM, David Davis <davidda...@redhat.com> >> wrote: >> >>> A couple things. First, we’re only running pulpcore and pulp_file tests >>> against pulpcore PRs. Also, we’re not running all pulp-smash tests for each >>> PR—only a certain subset labeled as "smoke tests". That said, we’ll still >>> want to keep an eye on pulp-smash tests over time to make sure it doesn’t >>> become a bottleneck. >>> >> >> This raises another concern. >> >> If the "smoke tests" are the only tests run before releasing to PyPi, >> what would be plan to run the complete integration test suite? >> >> Also, looking at >> https://docs.pulpproject.org/en/3.0/nightly/contributing/con >> tinuous_integration.html >> >> "The unit tests for both pulpcore and pulpcore-plugin live in the tests >> <https://github.com/pulp/pulp/blob/3.0-dev/tests/> folder" >> >> The link seems to rendering a 404 >> >> >>> >>> David >>> >>> On Tue, Apr 17, 2018 at 8:24 AM, Preethi Thomas <ptho...@redhat.com> >>> wrote: >>> >>>> "The plan for quality involves the continuous delivery of Pulp 3 where >>>> both unit and integration are run with each PR and prior to each release to >>>> PyPI" >>>> >>>> My concern about the above statement is that as the number of tests >>>> increases, the time it will take to run the pulp-smash integration suite >>>> will also increase. If each PR is required to run the integration suite >>>> then pulp-smash will become the bottleneck. >>>> >>>> Just for comparison purpose, it takes about 2.5 hours to run pulp-smash >>>> for Pulp 2. >>>> >>>> >>>> >>>> On Mon, Apr 16, 2018 at 4:30 PM, Dennis Kliban <dkli...@redhat.com> >>>> wrote: >>>> >>>>> The documentation has been updated[0]. >>>>> >>>>> [0] https://docs.pulpproject.org/en/3.0/nightly/contributing/con >>>>> tinuous_integration.html >>>>> >>>>> On Fri, Apr 6, 2018 at 2:52 PM, Dennis Kliban <dkli...@redhat.com> >>>>> wrote: >>>>> >>>>>> I have updated the redmine issue[0] to include a page in the >>>>>> contributors guide on "Continuous Integration" as a deliverable. That >>>>>> page >>>>>> will include that quote. >>>>>> >>>>>> >>>>>> [0] https://pulp.plan.io/issues/3530 >>>>>> >>>>>> On Fri, Apr 6, 2018 at 2:06 PM, Robin Chan <rc...@redhat.com> wrote: >>>>>> >>>>>>> Dennis, >>>>>>> >>>>>>> Thanks for putting this together. I don't see any responses on this >>>>>>> thread and take that to mean there were no concerns about this proposal. >>>>>>> >>>>>>> Would this process/responsibility change need to go anywhere? (Side >>>>>>> questions, was this technically a PUP?) >>>>>>> "author of the PR would need to be responsible for making additional >>>>>>> PRs to pulp_file and/or pulp-smash to fix the smash tests" >>>>>>> >>>>>>> I think this is minor enough to skip some process, however I would >>>>>>> like to see the above quote go into the developers guide. A new >>>>>>> contributor >>>>>>> shouldn't have to glean through mail list archives to understand this >>>>>>> new >>>>>>> responsibility. >>>>>>> >>>>>>> Thanks, >>>>>>> Robin >>>>>>> >>>>>>> On Tue, Mar 27, 2018 at 8:22 PM, Dennis Kliban <dkli...@redhat.com> >>>>>>> wrote: >>>>>>> >>>>>>>> One of the requirements for this plan to work is making sure that >>>>>>>> pulp, pulp_file, and pulp-smash always continue working together. This >>>>>>>> means that any time we have a PR that breaks pulp-smash tests, the >>>>>>>> author >>>>>>>> of the PR would need to be responsible for making additional PRs to >>>>>>>> pulp_file and/or pulp-smash to fix the smash tests. To enforce this >>>>>>>> requirement using Travis, I have filed 2 tasks[0,1] in redmine that I >>>>>>>> would >>>>>>>> like to get groomed and added to the sprint. I plan on working on these >>>>>>>> tasks as I introduce a change that will result from resolving issue >>>>>>>> 3488[2]. >>>>>>>> >>>>>>>> [0] https://pulp.plan.io/issues/3530 >>>>>>>> [1] https://pulp.plan.io/issues/3531 >>>>>>>> [2] https://pulp.plan.io/issues/3488 >>>>>>>> >>>>>>>> On Fri, Mar 23, 2018 at 2:37 PM, Dennis Kliban <dkli...@redhat.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I've started putting together a Continuous Delivery of Pulp 3 >>>>>>>>> page[0] on our wiki. >>>>>>>>> >>>>>>>>> This page outlines a plan for how we can ensure and prove the >>>>>>>>> quality of Pulp 3 releases by relying on pulp-smash tests and unit >>>>>>>>> tests. >>>>>>>>> >>>>>>>>> This plan enables anyone to improve the quality of Pulp 3 releases >>>>>>>>> through contributions to pulp-smash and unit tests. >>>>>>>>> >>>>>>>>> Please take a look at the plan and provide feedback on this thread >>>>>>>>> or feel free to make edits directly on the page. >>>>>>>>> >>>>>>>>> [0] https://pulp.plan.io/projects/pulp/wiki/Continuous_Delivery_ >>>>>>>>> of_Pulp_3 >>>>>>>>> >>>>>>>>> -Dennis >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Pulp-dev mailing list >>>>>>>> Pulp-dev@redhat.com >>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Pulp-dev mailing list >>>>> Pulp-dev@redhat.com >>>>> https://www.redhat.com/mailman/listinfo/pulp-dev >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Pulp-dev mailing list >>>> Pulp-dev@redhat.com >>>> https://www.redhat.com/mailman/listinfo/pulp-dev >>>> >>>> >>> >> >> _______________________________________________ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> >> > > _______________________________________________ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev > >
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
