Fantastic, do we have some way to validate that the module was pulled in properly as part of fuel-library CI?
On Fri, Jul 17, 2015 at 2:48 PM Alex Schultz <aschu...@mirantis.com> wrote: > Hey All, > > I've figured it out without having to modify the fuel-main build code. > I've updated the fuel-library spec with a build action that invokes the > script to pull down external modules. Please take some time to review the > two reviews out there for this change to see if there are any issues with > the way it is implemented. > > https://review.openstack.org/#/c/202763/ > https://review.openstack.org/#/c/202767/ > > This is a first step towards being able to pull in unmodified external > puppet modules. > > Thanks, > -Alex > > On Fri, Jul 17, 2015 at 4:23 PM, Andrew Woodward <awoodw...@mirantis.com> > wrote: > >> >> >> On Fri, Jul 17, 2015 at 12:24 PM Vladimir Kozhukalov < >> vkozhuka...@mirantis.com> wrote: >> >>> Alex, >>> >>> Great that you did this. Now I think I can prepare fuel-main patch to >>> invoke this script right before building fuel-library package. I'll add you >>> to review it. Is it ok if I do this monday morning? >>> >> >> Keep in minde we agreeded to a deadline to get this sorted and in shape >> to land by EOD monday or we will have to retain the old, and crappy fork >> method. If possible please work out how this needs to work as early as >> possible so Alex can continue. >> >> >>> Vladimir Kozhukalov >>> >>> On Fri, Jul 17, 2015 at 5:51 PM, Alex Schultz <aschu...@mirantis.com> >>> wrote: >>> >>>> Hey Vladimir, >>>> >>>> On Fri, Jul 17, 2015 at 7:33 AM, Vladimir Kozhukalov < >>>> vkozhuka...@mirantis.com> wrote: >>>> >>>>> Alex, >>>>> >>>>> Gathering upstream modules certainly should be implemented as a >>>>> separate script so as to make it possible to use it wherever we need this >>>>> (tests, builds, etc.) According to builds there are two things >>>>> >>>>> 1) We have so called "perestroika" package build system (Dmitry >>>>> Burmistrov is a main contributor here). By the end of next week we are >>>>> going to switch building all the packages to "perestroika". And in order >>>>> to >>>>> gather upstream modules right before building fuel-library package, we >>>>> need >>>>> to change perestroika build scripts. >>>>> >>>>> 2) Currently we build packages using make system and you are right >>>>> about the place where you need to make changes. >>>>> https://github.com/stackforge/fuel-main/blob/master/packages/module.mk#L63-L82 >>>>> If you create shell script, I'll help you to add it to make code. >>>>> >>>>> >>>>> >>>> >>>> I have updated my review[0] to extract the update logic to it's own >>>> bash script that can be invoked by the build scripts. Let me know what >>>> would be the best way to wedge this in there. I think for the >>>> "perestroika" this would also be needed for the fuel-library build, so if >>>> you point me at that I can see if I can help make that change as well. >>>> >>>> Thanks, >>>> -Alex >>>> >>>> [0] https://review.openstack.org/#/c/202763/ >>>> >>>> >>>>> >>>>> Vladimir Kozhukalov >>>>> >>>>> On Fri, Jul 17, 2015 at 2:56 PM, Aleksandr Didenko < >>>>> adide...@mirantis.com> wrote: >>>>> >>>>>> I believe build_repo function is the best way to do this [0]. So for >>>>>> fuel-library we'll need to run a shell script right from the repo before >>>>>> 'touch $$@'. We can make it either conditional ( test -f >>>>>> ./path/additional_build_script.sh && bash >>>>>> ./path/additional_build_script.sh >>>>>> ) or as additional parameter to function and add it in fuel-library call >>>>>> [1] >>>>>> >>>>>> Regards, >>>>>> Alex >>>>>> >>>>>> [0] >>>>>> https://github.com/stackforge/fuel-main/blob/master/repos.mk#L16-L37 >>>>>> [1] https://github.com/stackforge/fuel-main/blob/master/repos.mk#L45 >>>>>> >>>>>> >>>>>> On Fri, Jul 17, 2015 at 2:37 PM, Alex Schultz <aschu...@mirantis.com> >>>>>> wrote: >>>>>> >>>>>>> Hey Alex, >>>>>>> >>>>>>> On Jul 17, 2015 4:32 AM, "Aleksandr Didenko" <adide...@mirantis.com> >>>>>>> wrote: >>>>>>> > >>>>>>> > Hi, >>>>>>> > >>>>>>> > I think that we should provide a separate script that will fetch >>>>>>> the upstream modules into fuel-library/deployment/puppet/ directory. It >>>>>>> will allow us to have everything in a single place and use this script >>>>>>> in >>>>>>> ISO build process and CI jobs. >>>>>>> > >>>>>>> >>>>>>> Right. That is what I'm going for. The issue I need help with is the >>>>>>> best way to execute this as part of the build process. From what i >>>>>>> understand of the build process is that we are using git archive for all >>>>>>> pieces so I'm not sure how to wedge in an extra script execution to do >>>>>>> the >>>>>>> module fetch. The creation of the script isn't the issue, the issue is >>>>>>> how >>>>>>> can I properly run it as part of the build process. >>>>>>> >>>>>>> >>>>>>> > Regards, >>>>>>> > Alex >>>>>>> > >>>>>>> >>>>>>> Thanks, >>>>>>> -Alex >>>>>>> >>>>>>> > On Thu, Jul 16, 2015 at 11:17 PM, Alex Schultz < >>>>>>> aschu...@mirantis.com> wrote: >>>>>>> >> >>>>>>> >> Hello everyone, >>>>>>> >> >>>>>>> >> I have committed the initial configuration required to start >>>>>>> leveraging librarian-puppet as part of the way we pull in upstream >>>>>>> puppet >>>>>>> modules[0]. Additionally, I have also committed a change that would >>>>>>> pull in >>>>>>> the openstack-ironic module[1]. The one piece that is missing from this >>>>>>> being a complete solution is the ability to run librarian-puppet as >>>>>>> part of >>>>>>> our build process for the fuel-library. I've looked into the fuel-main >>>>>>> build scripts and I think it's over my head to figure this out just by >>>>>>> looking. Can anyone explain to me or assist me in how I could go about >>>>>>> modifying the existing build system to be able to run librarian-puppet >>>>>>> to >>>>>>> prepare the source for the package? In my initial investigation, it >>>>>>> looks >>>>>>> like it would be a modification of the fuel-main/packages/module.mk[3] >>>>>>> file. I basically need to do the prepare_library[3] function from the >>>>>>> 202763 review[0] after we've pulled all the sources together to fetch >>>>>>> the >>>>>>> upstream modules. >>>>>>> >> >>>>>>> >> >>>>>>> >> Thanks, >>>>>>> >> -Alex >>>>>>> >> >>>>>>> >> [0] https://review.openstack.org/202763 >>>>>>> >> [1] https://review.openstack.org/202767 >>>>>>> >> [2] >>>>>>> https://github.com/stackforge/fuel-main/blob/master/packages/module.mk#L63-L82 >>>>>>> >> [3] >>>>>>> https://review.openstack.org/#/c/202763/1/utils/jenkins/fuel_noop_tests.rb >>>>>>> >> >>>>>>> >> >>>>>>> __________________________________________________________________________ >>>>>>> >> OpenStack Development Mailing List (not for usage questions) >>>>>>> >> Unsubscribe: >>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>> >> >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> __________________________________________________________________________ >>>>>>> > OpenStack Development Mailing List (not for usage questions) >>>>>>> > Unsubscribe: >>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>> > >>>>>>> >>>>>>> >>>>>>> __________________________________________________________________________ >>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>> Unsubscribe: >>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> __________________________________________________________________________ >>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>> Unsubscribe: >>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>> >>>>>> >>>>> >>>>> >>>>> __________________________________________________________________________ >>>>> OpenStack Development Mailing List (not for usage questions) >>>>> Unsubscribe: >>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>> >>>> >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> -- >> -- >> Andrew Woodward >> Mirantis >> Fuel Community Ambassador >> Ceph Community >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- -- Andrew Woodward Mirantis Fuel Community Ambassador Ceph Community
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
