Re: [OpenStack-Infra] What's the future for git-review?
Perhaps sending that email right before taking a few days off meaning I couldn't reply straight away wasn't the most helpful ;-) On Thu, 5 Jul 2018, 02:57 Jeremy Stanley, wrote: > On 2018-07-04 22:32:53 +0100 (+0100), Darragh Bailey wrote: > [...] > > Based on the comments at > > > https://git.openstack.org/cgit/openstack-infra/git-review/tree/CONTRIBUTING.rst#n5 > , > > git-review is considered feature complete, and as a consequence it > > seems that reviewers have mostly moved onto other projects so it > > can take quite some time to get reviews. Perfectly understandable, > > everyone can only do so much and needs to pick something(s) to > > prioritise. However this is such a useful tool for working with > > Gerrit from the command line it seems to be the defacto git > > subcommand for interfacing with Gerrit that it seems a shame to > > limit it. > > At one time git-review had started to suffer from scope creep and > was getting more random proposals for new features than actual > stability improvements. Its test coverage was, for quite a long > while, also sub-par so that some of the feature additions which did > get accepted introduced regressions that went unnoticed sometimes > for weeks or months before we'd discover they needed reverting. The > original authors intended for git-review to be primarily focused on > bootstrapping Gerrit connectivity from a cloned Git repository as > well as simplifying the basic Git commands for retrieving changes > from and pushing changes to a Gerrit. That update to the > CONTRIBUTING.rst file was intended to put the brakes on future scope > creep, especially in cases where an added feature would work just as > well as its own separate git subcommand. > You've made one suggestion below on that applying for the list behaviour. Made me think that an etherpad listing the current outstanding or abandoned changes and discussing which would be better off outside of git-review would be a useful starting place. > While I think there are a number of current reviews that would be > > beneficial to git-review, as well as some pieces that don't appear > > to be there currently, I'm reluctant to invest much time as it > > seems unlikely enhancements would be accepted due to the current > > state of feature complete. Instead of putting together various > > changes to see if they might be reviewed and accepted, hoping a > > chat about what paths might be available could save a bit of time. > > I've tried to go in and approve changes from time to time, but in > all honesty the negativity I've received in the past when attempting > to push back on feature additions has caused me to deprioritize > reviewing more changes for it. I should probably just buck up and go > in with a (very polite) machete anyway. > I think it's all due to needing to prioritise time spent and git-review has mostly done what's needed. > There are a couple of things that I would like to work towards: > > > > * Change the tests to use a single gerrit with separate projects > > instead of separate instances (faster testing) > > This seems reasonable if it doesn't introduce new races or odd test > interdependencies from the reduced fixture isolation. I really have > never been fond of the integration-testing-only model we ended up > with though. I originally recommended lower-level unit testing with > mocks for the Git and SSH interactions, but the one volunteer we got > to implement a testsuite chose to automate Gerrit installation so it > is what it is at the moment. > More mocks around ssh/http should work well, but I've found that it's not necessarily beneficial doing the same around git, as with some simple fixtures it can be tested very quickly. The existing tests are still useful, and changing to a single gerrit instance per runner would require some thought in the logging side (adding markers should be sufficient I think), but thetas probably the main issue. > * Allow the tests to run against multiple versions of Gerrit (ensure > > compatibility) > > This seems reasonable. We should have been bumping the Gerrit > versions in the tests and/or running more jobs for different > releases of it but the way version selection was implemented would > need a bit of an overhaul to accommodate that. > I'm thinking also for compatibility across a few releases would be good as well. > * Fix and land many of the changes making it easier to download > > changes, list changes ordered with their dependencies, stashing > > when downloading, etc > > The change listing feature really seems increasingly out of place to > me, and most of the "fixes" I saw related to it were about > supporting more and more of Gerrit's que
[OpenStack-Infra] What's the future for git-review?
Hi, Firstly, thanks for git-review, it's such a useful tool, and I use it all the time working with Gerrit, from working on some openstack projects (including the odd patch to git-review), various projects in work and the very rare patch to Gerrit or it's plugins itself. Based on the comments at https://git.openstack.org/cgit/openstack-infra/git-review/tree/CONTRIBUTING.rst#n5, git-review is considered feature complete, and as a consequence it seems that reviewers have mostly moved onto other projects so it can take quite some time to get reviews. Perfectly understandable, everyone can only do so much and needs to pick something(s) to prioritise. However this is such a useful tool for working with Gerrit from the command line it seems to be the defacto git subcommand for interfacing with Gerrit that it seems a shame to limit it. While I think there are a number of current reviews that would be beneficial to git-review, as well as some pieces that don't appear to be there currently, I'm reluctant to invest much time as it seems unlikely enhancements would be accepted due to the current state of feature complete. Instead of putting together various changes to see if they might be reviewed and accepted, hoping a chat about what paths might be available could save a bit of time. There are a couple of things that I would like to work towards: * Change the tests to use a single gerrit with separate projects instead of separate instances (faster testing) * Allow the tests to run against multiple versions of Gerrit (ensure compatibility) * Fix and land many of the changes making it easier to download changes, list changes ordered with their dependencies, stashing when downloading, etc * Have git-review auto configure refs/notes/review (assuming it's available) for fetching on setup (I find it very handy and I'm always forgetting to do this) And potentially controversially; support other workflows and options outside of the OpenStack workflow. Although maybe not directly, and still keeping the OpenStack one as the default. I think there are a couple of ways that could be achieved, but I can't see any of them working well without a decent amount of refactoring. * Have git-review provide the APIs so that someone may define a git-review- that can add their workflow * Add support for additional behaviour to be defined with refs/meta/config of projects * Allow extensions to be installed that allow additional options to be added to the git-review CLI and config file That last one might require being able to specify the additional required plugins to be listed in .gitreview, and providing the documentation might be trickier? Basically make it easier to add custom behaviour without it being builtin to git-review, and without needing to reimplement a whole load of functionality elsewhere. But I'm pretty sure that all requires a substantial rewrite. Thoughts? Is it worth putting a plan together around some of the initial changes? And then revisiting what would be needed to allow extensions around other workflows? -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Selecting New Priority Effort(s)
On 4 April 2018 at 03:33, David Moreau Simard <dmsim...@redhat.com> wrote: > It won't be very exciting but we really need to do one of the > following two things soon: > > 1) Ansiblify control plane [1] > 2) Update our puppet things to puppet 4 (or 5?) > > Puppet 3 has been end of life since Dec 31, 2016. [2] > > The longer we draw this out, the more work it'll be :( > > [1]: https://review.openstack.org/#/c/469983/ > [2]: https://groups.google.com/forum/#!topic/puppet-users/IdutL5FTW7w > > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] > > I would suggest that whether it's decided to switch to ansible for the control plane or update puppet modules, it will be well worth investing thought into performance when running across nodes that contain "different" services to perform "different functions". Ansible is very very good at running the same task across multiple machines, e.g. configuring homogeneous servers. But control planes have a tendency to have a lot of different services running on subsets, and this has a consequence of resulting in lots of time spent waiting on tasks to complete on some nodes and skip on the rest due to the synchronization of tasks across the entire set. When working on the precursor to https://github.com/ArdanaCLM (original was used as part of Helion OpenStack by HP(E)) we had a CI job testing the deployment of a small control plane and some services on a set of 6 VMs and the time cost was prohibitive at 1.5hrs ~ 2.5hrs (upgrade testing CI was double these figures). A lot of the time 50% or more of VMs were idle because tasks that involved a few nodes meant nothing else could be done on the others. There were some thoughts around adding a strategy plugin to ansible that could do a cross between the free-run and synchronized behaviour where you could free run to completion on nodes unless you encountered certain tasks. Other alternatives included nest ansible runs to have free runs done to a point before then performing the tasks that involved cluster style operations in synchronization or careful crafting of the playbooks to achieve the same. Never got around to solving these, and some of the problems were caused by us adopting an approach without necessarily having a deep understanding of the tooling. None of this is to say the same problems will exist here, but when you are managing systems/services that interact, and it's difficult to CI them in isolation at the project, potentially you'll want some way for developers and CI on changes to exercise a test env. The cost of developing/testing/integrating with either approach should probably be investigated for both in detail. Before you look at whether it's easy to replace the puppet modules with ansible or update to puppet 4/5, so it might be worth focusing on what approaches might be needed to extract the best experience first (stability, ease of writing/maintenance & speed of dev-env bring up come to mind as important)? Past experience with any config management suggests that when you start simple it's easy to incrementally improve on the existing approach, but reserving direction when you hit dead ends is almost impossible ;) -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] OpenStack and open infrastructure
Hi, I'm looking to spin out a testing code used for git-upstream ( http://git.openstack.org/cgit/openstack/git-upstream) into a separate project as a fixture to make it easier to use else where for building tests for tools that use git. Currently lining up a few commits using OpenStack's Gerrit as a holding place until I've worked out where the new project should reside, https://review.openstack.org/#/c/551445/ being the main one, and I've some tests to write. Based on http://lists.openstack.org/pipermail/foundation/2017-November/002532.html (OpenStack and open infrastructure) I'm wondering if that means it can continue to live within the OpenStack infrastructure/tooling? Or is this something that is still under discussion as to what it means? Created a change for this just in case this is a straight forward request and there was no need for this email: https://review.openstack.org/553978 -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Merging feature/zuulv3 into master in Zuul and Nodepool repos
Hi Clark, You might find the following series of commands help automate this a little more. git merge -s ours --no-commit feature/zuulv3 git read-tree -u --reset feature/zuulv3 git commit -m "replace mainline with feature/zuulv3" Basically it replaces the current branch contents with everything that came from feature/zuulv3, so no risk that changes get merged at all, whatever was on the master branch is ignored. Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" - unknown On 12 Jan 2018 22:43, "Clark Boylan" <cboy...@sapwetik.org> wrote: > Hello, > > I think we are very close to being ready to merge the zuulv3 feature > branch into master in both the Zuul and Nodepool repos. In particular we > merged https://review.openstack.org/#/c/523951/ which should prevent > breakages for anyone using that deployment method (single_node_ci) for an > all in one CI suite. > > One thing I've noticed is that we don't have this same handling in the > lower level individual service manifests. For us I don't think that is a > major issue, we'll just pin our builders to the nodepool 0.5.0 tag, do the > merge, then update our configs and switch back to master. But do we have > any idea if it is common for third part CI's to bypass single_node_ci and > construct their own like we do? > > As for the actual merging itself a quick test locally using `git merge -s > recursive -X theirs feature/zuulv3` on the master branch of nodepool > appears to work. I have to delete the files that the feature branch deleted > by hand but otherwise the merge is automated. The resulting tree does also > pass `tox -e pep8` and `tox -epy36` testing. > > We will probably want a soft freeze of both Zuul and Nodepool then do our > best to get both merged together so that we don't have to remember which > project has merged and which hasn't. Once that is done we will need to > repropose any open changes on the feature branch to the master branch, > abandon the changes on the feature branch then delete the feature branch. > Might be a good idea to merge as many feature branch changes as possible > before hand? > > Am I missing anything? > > Thank you, > Clark > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [jenkins-job-builder] When will jenkins-job-builder 2.0.0 be released?
Hi Artem, The delay has been caused by being consumed with other priorities. It's unfortunately, but it happens, of course any help around patch reviews and fixing up the patches in the backlog will help. If you'd like to drop into the irc chat #openstack-jjb on freenode room this Friday 17th @1400 UTC, there will be some discussion about v2 and what's left to be done. http://eavesdrop.openstack.org/#Jenkins_Job_Builder_Team_Meeting We've identified a bug in some of the changes we've landed http://eavesdrop.openstack.org/irclogs/%23openstack-jjb/%23openstack-jjb.2017-10-06.log.html#t2017-10-06T14:06:18, and once we have a working fix we should be ready to release v2. As I'm heading to a python conference this weekend, I'm hoping there will be a hacking space and I'll try to solve both that bug and the concern around use of a global config object everywhere mentioned in https://etherpad.openstack.org/p/jjb_api_v2.0 Fingers crossed we should be able to get the v2 release out in the next month, any help that you can give would be appreciated. Thanks, -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" On 17 October 2017 at 11:04, Artem Panchenko <artem@panchenko.email> wrote: > Hi folks! > > I wonder if there is a road map or some release schedule for > jenkins-job-builder project? Last time it was published on pypi/github more > than 6 months ago and that was the second beta of 2.0.0. AFAIU, OpenStack > infra doesn't use Jenkins anymore and doesn't invest into JJB development > (please correct me if I'm wrong). Though, I see that >150 changes have been > merged to master since that time, so the project is still maintained. > > I'll appreciate If someone shares any details about when the next major > version release of JJB is going to happen. > > Thanks in advance! > > --- > Best regards, > Artem Panchenko > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Zuul feature/zuulv3 branch to be rewound
On 21 Sep 2017 6:38 pm, "James E. Blair" <cor...@inaugust.com> wrote: Hi, Snipped 1) Abandon all open changes on feature/zuulv3. 2) Delete the feature/zuulv3 branch. 3) Re-create the feature/zuulv3 branch at the commit before the Sem-Ver change: 027ba992595d23e920a9cf84f67c87959a4b2a13. 4) Restore all the changes abandoned in step 1. The abandon/restore steps are required by Gerrit in order to delete the branch. We could force-push the branch tip, but this is the procedure we have asked and would ask any other project to use in a similar situation, in order to reduce the risk of error. Can you elaborate more on how this reduces risk/errors? Curious in case we run into a similar scenario in the future. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" - unknown ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [zuul] v3 UI enquiry
On 6 September 2017 at 01:26, James E. Blair <cor...@inaugust.com> wrote: > Darragh Bailey <daragh.bai...@gmail.com> writes: > > > Hi, > > > > Currently the main issue from end users, is the user experience around > the > > UI when looking at job results once the run is complete, and to a lesser > > extend looking at your jobs in the zuul status page when running. > > > > I'm wondering if there is a plan to have a full UI covering both > historical > > job runs for projects as well as live status reporting? > > Yes, Tristan has started work on that, and we plan to continue it after > we migrate openstack to Zuul v3. > Is there a link to where this is being tracked/discussed? > > I've noticed there is an SQL reporter, should that be leveraged to > provide > > access to the sort of information that needs to be tracked? Or should > > something else be used such as simple files to identify success/failure? > > Yes, we plan on using it to supply the historical data. > > > 1) Project status view: > > > > The main status page can be useful for those supporting zuul in an > > environment to have a quick overview of everything, however feedback > we've > > received locally suggests most developers are frequently only interested > in > > a single project and find this to be overwhelming This suggests there > > should be a per-project status view, showing only the pipelines and > changes > > of interest for a single project, along with any CRD that are linked. > > > > Possibly under an endpoint of "//" as the default view showing > any > > pipelines relevant to the project along with any changes going through > them. > > That sounds reasonable once we have the new web framework in place. > Are there stories on this area? Presumably the endpoint being queried would provide similar information so it might be possible for someone to work on this in parallel and then adjust without too much difficultly to use more appropriate API queries, or when you refer to a web framework do mean the client side web components? > > 2) Project build history view: > > > > Returning links at the end of a run in a comment to the raw log files as > > the only build history is somewhat jarring for people used to the > > Travis/Drone/Circle CI services available, there's the expectation that > it > > should be easily to see the previous runs, seeing whether they passed, > > which jobs within a set failed, and to display the detailed log when > > desired. Providing a reasonable UI is becoming a pre-requisite for > > adoption, no matter how good the functionality, it can be difficult to > get > > acceptance if the "form" is deemed limiting (non all follow this, but it > > does seem that some of the most vocal objectors can fall into this area). > > I'm a little confused by this one. I agree they should be available, > but I think they already are. Zuul currently provides (in a comment, as > you say) the overall pass/fail status of the build set, the pass/fail > status of individual jobs, that same information for previous runs on > the change, and links to the detailed logs. In Github, the presentation > of that available to us is somewhat limited, but I think in Zuul v3 > we're approaching a reasonable compromise. Note this may be > significantly different that what you are running. > I believe it provides this back to the corresponding PRs, but it would be nice if from the zuul interface if I'm looking at a failed job, that 'a' it doesn't suddenly disappear once all the other jobs in the set are complete, and 'b' it's possible to quickly look to another recent run that was executed due to another change within the same project. I haven't been running the zuulv3 code base yet, maybe you already have some of these improvements in place, and I'll see them as I start experimenting with it. > If you're suggesting that there should *also* be a way to find that > information from a main Zuul web page, yes, I think that would be > covered by the previous point. > Yes, I meant that if you are looking at your change and the job failed, finding/seeing previous runs of the same job for the same project is currently difficult, but if they could easily switch to seeing them, and also seeing others that passed since their change was tested it helps focus on the problem being due to the change at hand. > I would like to point out though that Zuul's primary user interface > always has been, and will continue to be, the code review system. > That's on purpose. The recognition that a developer should have all the > information about a change, including test results, at their fingertips > is o
Re: [OpenStack-Infra] [zuul] v3 UI enquiry
Hi, Sorry, got distracted and wanted to make some images available to help with the discussion as we're talking UI. On 5 September 2017 at 19:43, Clark Boylan <cboy...@sapwetik.org> wrote: > On Tue, Sep 5, 2017, at 09:25 AM, Darragh Bailey wrote: > > Hi, > > 1) Project status view: > > > > The main status page can be useful for those supporting zuul in an > > environment to have a quick overview of everything, however feedback > > we've > > received locally suggests most developers are frequently only interested > > in > > a single project and find this to be overwhelming This suggests there > > should be a per-project status view, showing only the pipelines and > > changes > > of interest for a single project, along with any CRD that are linked. > > > > Possibly under an endpoint of "//" as the default view showing > > any > > pipelines relevant to the project along with any changes going through > > them. > > > > This is already possible via the status page filter right? The > difference would be that the filter is cookie based and not path based. > Thinking about it, both provide nice features (with cookie based filter > I don't have to remember links I just always get things filtered the way > I want, with url it is easier to link to what you are seeing). > Yes, but at the same time it's not necessarily a great view for all cases A filter benefits in showing that there are many more items currently being hidden from view, but this can lead to the view being very cluttered when what the user wanted was just to see the state of the jobs for their change only. Examples: https://pasteboard.co/GL5fmnU.png <-- filter shows change with dependencies https://pasteboard.co/GL5qNmA.png <-- busy status view I'd prefer to revert back to keeping the filter as a way for users to adjust what is displayed, but to decide on what subset of data is going to be made available so they are only filtering a smaller set rather than the entire contents. I think a path based approach allows > > > > > > 2) Project build history view: > > > > Returning links at the end of a run in a comment to the raw log files as > > the only build history is somewhat jarring for people used to the > > Travis/Drone/Circle CI services available, there's the expectation that > > it > > should be easily to see the previous runs, seeing whether they passed, > > which jobs within a set failed, and to display the detailed log when > > desired. Providing a reasonable UI is becoming a pre-requisite for > > adoption, no matter how good the functionality, it can be difficult to > > get > > acceptance if the "form" is deemed limiting (non all follow this, but it > > does seem that some of the most vocal objectors can fall into this area). > > > > Users can be quick to blame the infrastructure when a chance working on > > their system fails in CI and seeing previous runs is non trivial. Making > > it easy to see that previous changes were working (or failing) can make > > it > > more likely that they will look closer at their own change for the cause > > of > > the failure. It also tends to be easier to understand a failing job, when > > you can easily view the output from a passing one. > > Yup, this is the reason the openstack health dashboard, > http://status.openstack.org/openstack-health/#/, exists. Being able to > see the bigger picture and status of a project is important. It also > provides rss feeds so you don't have to actively look at a screen to get > notifications which is nice. A lot of the health dashboard features are > probably fairly openstack specific, like the integration with > elastic-recheck and subunit, but my hunch is that we can probably make a > health dashboard that falls back gracefully to a base set of features > that a "normal" zuul v3 installation would support. > > Rather than start from scratch maybe it makes sense to build on this > health dashboard? > I find the current view very individual job based rather than change based. I'm not sure that requiring another service to be in place to display the historical information so users can see how previous build sets as well as ones currently running are performing, unless you meant to try and lift that and add it to the zuul webapp? Perhaps openstack health would be better focused on providing statistics and analysis enhancements to zuul? Time most jobs run at, details on failures, which job out of the build set is the most common failing job, timing comparisons over time, etc. >From a raw display of information can I find the last number of runs and see both tho
[OpenStack-Infra] Setting up a Jenkins Job Builder team meeting
Hi all, We're going to set up some bi-weekly meetings to help get the V2 API of JJB and then help plan subsequent improvements. Review request for the meeting: https://review.openstack.org/481664 For those not aware, there has been a new IRC channel set up to focus on JJB, so come join us on #openstack-jjb on freenode. Happy hacking. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Dell Ironic CI broken
Ah, shame, thought for a second you might have developed a greater interest
in it ;)
On 27 March 2017 at 19:38, <rajini@dell.com> wrote:
> *Dell - Internal Use - Confidential *
>
> Sorry we were in the process of upgrading the infra components and zuul
> layout configuration got messed up.
>
> We have fixed it
>
> *From:* Darragh Bailey [mailto:daragh.bai...@gmail.com]
> *Sent:* Monday, March 27, 2017 1:28 PM
> *To:* Ram, Rajini <rajini_...@dell.com>
> *Cc:* joshua.hesk...@gmail.com; a...@suse.com; openstack-ironic
> <openstack-iro...@dell.com>; Dearborn, Chris
> <christopher_dearb...@dell.com>; openstack infra <openstack-infra@lists.
> openstack.org>
>
> *Subject:* Re: [OpenStack-Infra] Dell Ironic CI broken
>
>
>
>
>
> Hi Rajini,
>
> Any idea why your CI started commenting on openstack/git-upstream changes?
>
> see https://review.openstack.org/#/c/408215/
>
>
>
>
>
> On 27 March 2017 at 15:20, <rajini@dell.com> wrote:
>
> *Dell - Internal Use - Confidential *
>
> Hi Josh
>
> The patch https://review.openstack.org/#/c/431691/ broke our Ironic CI
> builds. We are in the process of upgrading and fixing it.
>
> Will fix it soon
>
> Thanks
>
> Rajini
>
>
>
> *From:* Joshua Hesketh [mailto:joshua.hesk...@gmail.com]
> *Sent:* Monday, March 27, 2017 7:00 AM
> *To:* Andreas Jaeger <a...@suse.com>
> *Cc:* openstack-ironic <openstack-iro...@dell.com>; Ram, Rajini <
> rajini_...@dell.com>; Dearborn, Chris <christopher_dearb...@dell.com>;
> OpenStack Infra <openstack-infra@lists.openstack.org>
> *Subject:* Re: [OpenStack-Infra] Dell Ironic CI broken
>
>
>
> Hey,
>
>
>
> I've disabled the account. Once it is fixed we can re-enable it.
>
>
>
> Thanks,
> Josh
>
>
>
> On Mon, Mar 27, 2017 at 7:22 PM, Andreas Jaeger <a...@suse.com> wrote:
>
>
> Dell Ironic CI is reporting on unrelated changes like
> https://review.openstack.org/449385 - project-config
> https://review.openstack.org/#/c/450088/ - trove
>
> It reports in both cases:
> "Merge Failed.
> This change or one of its cross-repo dependencies was unable to be
> automatically merged with the current state of its repository. Please
> rebase the change and upload a new patchset."
>
> Please disable our CI immediately - and fix it and test on the
> sandbox-ci repo,
>
> Andreas
> --
> Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
>
>
> _______
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
>
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
>
>
>
> --
>
> Darragh Bailey
> "Nothing is foolproof to a sufficiently talented fool"
>
--
Darragh Bailey
"Nothing is foolproof to a sufficiently talented fool"
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Dell Ironic CI broken
Hi Rajini,
Any idea why your CI started commenting on openstack/git-upstream changes?
see https://review.openstack.org/#/c/408215/
On 27 March 2017 at 15:20, <rajini@dell.com> wrote:
> *Dell - Internal Use - Confidential *
>
> Hi Josh
>
> The patch https://review.openstack.org/#/c/431691/ broke our Ironic CI
> builds. We are in the process of upgrading and fixing it.
>
> Will fix it soon
>
> Thanks
>
> Rajini
>
>
>
> *From:* Joshua Hesketh [mailto:joshua.hesk...@gmail.com]
> *Sent:* Monday, March 27, 2017 7:00 AM
> *To:* Andreas Jaeger <a...@suse.com>
> *Cc:* openstack-ironic <openstack-iro...@dell.com>; Ram, Rajini
> <rajini_...@dell.com>; Dearborn, Chris <christopher_dearb...@dell.com>;
> OpenStack Infra <openstack-infra@lists.openstack.org>
> *Subject:* Re: [OpenStack-Infra] Dell Ironic CI broken
>
>
>
> Hey,
>
>
>
> I've disabled the account. Once it is fixed we can re-enable it.
>
>
>
> Thanks,
> Josh
>
>
>
> On Mon, Mar 27, 2017 at 7:22 PM, Andreas Jaeger <a...@suse.com> wrote:
>
>
> Dell Ironic CI is reporting on unrelated changes like
> https://review.openstack.org/449385 - project-config
> https://review.openstack.org/#/c/450088/ - trove
>
> It reports in both cases:
> "Merge Failed.
> This change or one of its cross-repo dependencies was unable to be
> automatically merged with the current state of its repository. Please
> rebase the change and upload a new patchset."
>
> Please disable our CI immediately - and fix it and test on the
> sandbox-ci repo,
>
> Andreas
> --
> Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
--
Darragh Bailey
"Nothing is foolproof to a sufficiently talented fool"
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [openstack-dev] [infra][security] Encryption in Zuul v3
On 22 March 2017 at 15:02, James E. Blair <cor...@inaugust.com> wrote: > Ian Cordasco <sigmaviru...@gmail.com> writes: > > > > > I suppose Barbican doesn't meet those requirements either, then, yes? > > Right -- we don't want to require another service or tie Zuul to an > authn/authz system for a fundamental feature. However, I do think we > can look at making integration with Barbican and similar systems an > option for folks who have such an installation and prefer to use it. > > -Jim > Sounds like you're going to make this plugable, is that a hard requirement that will be added to the spec? or just a possibility? -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] using custom tags: include-raw-escape
Hi Anil,
Sorry I didn't pick on this before.
On 23 February 2017 at 10:03, Anil SB <ask...@gmail.com> wrote:
> Hi,
>
> We use JJB version (1.6.1) and Jenkins 2.32.1 for all our jobs on ODL
> infrastructure. While testing the below code, I am trying to understand
> the expected behavior / differences of custom yaml tag
> `!include-raw-escape` (and !include-raw). From docs in [1.] suggests
> that escaping of any shell variables (${{VAR}} in the below script) is
> not required when custom tags !include-raw-escape used in builder.
>
>
> For instance, below code is taken from the example in docs from [1.]:
>
> // yaml file
> - job:
> name: raw-test
> builders:
> - shell:
> !include-raw-escape: include-raw001-vars.sh
>
The short answer is that "!include-raw-escape" is for use inside template
definitions or macros that require parameters, in which case variable
substitution is performed and the use of "!include-raw-escape" prevents
undesired substitution of anything inside curly braces.
> // include-raw001-vars.sh
> #!/bin/bash
> #
> # sample script to check that brackets aren't escaped
> # when using the include-raw application yaml tag
>
> VAR1="hello"
> VAR2="world"
> VAR3="${VAR1} ${VAR2}"
>
> [[ -n "${VAR3}" ]] && {
> # this next section is executed as one
> echo "${VAR3}"
> exit 0
> }
>
>
> Once the above job is pushed into Jenkins, this gets translated to:
>
> #!/bin/bash
> #
> # sample script to check that brackets aren't escaped
> # when using the include-raw application yaml tag
>
> VAR1="hello"
> VAR2="world"
> VAR3="${{VAR1}} ${{VAR2}}"
>
> [[ -n "${{VAR3}}" ]] && {{
> # this next section is executed as one
> echo "${{VAR3}}"
> exit 0
> }}
>
> Above script when run returns variable substitution errors when running
> the job. Is this a expected behavior or known issue ? We also noticed
> that the escaping of variables works only for some of the scripts and
> not all.
>
> Thanks,
> Anil
>
> [1.]
> https://docs.openstack.org/infra/jenkins-job-builder/
> definition.html?highlight=job%20definition
>
> "
>
> The tag !include-raw-escape: treats the given string or list of strings
> as filenames to be opened as one or more data blobs, which should be
> escaped before being read in as string data. This allows job-templates
> to use this tag to include scripts from files without needing to escape
> braces in the original file.
>
> "
>
Thanh mentioned in IRC that this needs clearing up to note the behaviour
around macros, since a macro that does not take parameters does not have
the string substitution performed on it within JJB and therefore should not
use !include-raw-escape.
This is because it is not expanded within the template before the
substitution is performed, but rather afterwards. Think of macros as kind
of like mini-job/mini-template-job definitions. If a macro does not take
parameters, it is like a standard job definition and no substitution is
performed. If it does take parameters then it's like job-template and
substitution takes place.
It would be nicer if it was possible to add something in that knew when to
perform the escaping automatically, maybe it's possible to have the code
that expands templates and does the substitution to be able to check if
something takes parameters then automatically escape anything that is of a
certain object type or responds to a specific method and then perform the
full substitution, otherwise if it doesn't take params just pass it through.
Needs a fair bit of thought and break down of the different scenarios to
support.
In the mean time Thanh has offered to help update the documentation to help
clarify
job defintion, macro without parameters -> use !include-raw for scripts to
just include the script as is
template job defintion, macro needing parameters -> use !include-raw-escape
to escape the script so that when substitution is performed the final
result is contents of the file provided.
Finally:
template job defintion, macro needing parameters -> Where you have a script
that you wish to embed a parameter to be substituted, you must pre-escape
the rest of the script and use !include-raw
Distinguishing between the last scenario and previous one for template
jobs/macros with parameters is the one that prevents the code from doing
everything for you automatically.
--
Darragh Bailey
"Nothing is foolproof to a sufficiently talented fool"
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Upcoming Nodepool changes
On 13 December 2016 at 00:25, James E. Blair <cor...@inaugust.com> wrote: > Hi, > > I'm happy to report that we have made significant progress on adding > ZooKeeper support to Nodepool, in preparation for supporting Zuul v3. > > I keep meaning to read up on what's planned for zuul v3, the recorded talk from the last summit was good in explaining some of the benefits for users of the new capabilities, but I'd also love to get up to speed on how some of the backend changes will improve things for operation. Is there some blog posts or something that goes into this? > This branch contains an update to the nodepool-builder process which > uses ZooKeeper rather than MySQL and gearman for coordinating image > builds and storing information about them (the main nodepool process > still uses MySQL and gearman -- updating it to use ZooKeeper is the next > section of work). > Will any of that break those still using zuul/nodepool with Jenkins? Or is that orthogonal? I still don't have a full handle on how Jenkins/Zuul/Nodepool interacts besides that it uses gearman. > Tomorrow, I also plan on merging the zuulv3 branch into master. If you > are interested in becoming an early adopter of this, that would be a > great time to start using it. We don't have much documentation in > Nodepool itself about this, but the short version is: install and run > ZooKeeper (a one node cluster is fine) and point Nodepool at it. There > is already (optional) support for it in puppet-openstackci and > puppet-nodepool. You can take a look at the OpenStack infra operational > documentation here: > > http://docs.openstack.org/infra/system-config/nodepool.html > > Once we tidy the master branch up a bit, we will release a new version, > 0.4.0 with the ZooKeeper based builder. At that point, it would be a > good idea for anyone running Nodepool to begin using it. > > Thanks, > > Jim > > Looking forward to checking it out, and hopefully getting involved around zuulv3 integration with GitHub! -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
Think you forgot to Reply-All ;-) Only spotted that now, while I was waiting for others to chime in and see different points of view. On 22 November 2016 at 07:25, Wayne Warren <wa...@puppet.com> wrote: > > > On Tue, Nov 15, 2016 at 8:04 AM, Darragh Bailey <daragh.bai...@gmail.com> > wrote: > >> >> So on that patch: >> https://review.openstack.org/358019 Support explicit API and simple >> config creation >> >> Does it make sense? I'd like to hear other people's opinion on whether it >> makes sense to have an explicit API as well as a from_config(cfg) method >> when someone would create the config object and then pass it around, or is >> it more of a YAGNI? >> > > I think that if you don't have an explicit use case for it that you intend > to use personally it's probably a YAGNI. > > Personally, I prefer to pass configuration values around with and access > them through an instance of a single configuration class that > > * can be more easily instantiated with reasonable default settings > * we can reason about and write tests for configuration > * discourages proliferation of scattered configuration value munging > throughout the code base which is one problem that JJBConfig was intended > to address > > Also, should all classes be instantiable without a JJBConfig? Why? If not, > why not? I'm curious why the focus on the Builder object for this behavior, > but I'm also interested in the principles at work here so I can change my > way of thinking if it's flawed; sorry if I've missed an explanation in the > past. > Previous experience with libcurl, which also uses this idea of configuration all in one object and then passed around, resulted in me experiencing that fun behaviour of needing to refer back to the config object in order to determine how to control the behaviour of the other API's I was calling. It was exceedingly frustrating, though that might be just due to poor API documentation on the curl project's behalf. I'm not sure it'll discourage "proliferation of scattered configuration", but rather have methods that reach into the config object to get behavioural controlling settings that are not then clearly reflected in the arguments into the method/class. End up needing to define config options used internally as part of classes/methods docstrings in order to help convey how to use the API. Btw, the builder was just the easiest to look at first, I didn't see any point in going through the rest of the objects and making changes to have an expressive constructor which used params that controlled the current objects behaviour, if it was clear this doesn't make sense as an approach. Of course for the plugin settings, these will have to be passed through as a single config object to the dispatch methods, so perhaps unless we plan to look at that as well, maybe there is no need to worry about the other classes having separate params for each setting controlling instantiated objects behaviour. > Since it impacts the API that will be published, seems like it would be a >> good idea to get this ironed out before releasing? >> > > I'd prefer to choose one approach for instantiating objects now based on > its technical merits, make sure it's not incompatible with whatever > alternative we are considering, and save alternatives as additive features > that can be implemented sometime in the future when a need becomes apparent. > Agreed, it's probably only because there is no overloading of constructors available in most dynamic languages that it's needing a closer look because I'm not sure it's possible to add support for creation without a config object without another breaking API change. I think the only solution would be simple subclasses with different __init__ methods. Hence I'm willing to look at this pretty closely. > -- >> Darragh Bailey >> "Nothing is foolproof to a sufficiently talented fool" >> > > Suggested to Thanh, that it might be worth cutting a beta release (maybe tomorrow) and advertise it to allow people to start installing from pypi by using '--pre'. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
(Starting a second reply for a different patch to make it easier to follow with threaded mail clients) I've a comment in https://review.openstack.org/#/c/319616/8/jenkins_jobs/builder.py that notes if an application made a call to 'JenkinsManager.update_jobs()' it would be a reasonable expectation that a subsequent call to 'JenkinsManager.delete_old_managed()' should not need to explicitly be passed the list of jobs that were just to the update method. For simplicity does it make sense to ensure the following would do the right thing by default: jm = JenkinsManager(cfg) jm.update_jobs([list of jobs]) jm.delete_old_unmanaged() This would require maintaining some internal state in JenkinsManager on the list of jobs updated and then reusing that list in delete_old_unmanaged() unless passed an explicit list of jobs to keep. If this were to be supported, should it take the form of: 1. Retain the list of the last set of jobs sent to update_jobs(), and overwrite on each call to update_jobs() 2. Continuously update an internal list of jobs of what has been updated since the JenkinsManager object was created? The second one would facilitate multiple calls to update_jobs, but tbh I suspect it's unnecessary complicated. But thought I'd throw it out there. At the very least I'm going to fix the method so that a call without passing a list of jobs doesn't cause an exception. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
Hi, On 11 November 2016 at 14:59, Thanh Ha <thanh...@linuxfoundation.org> wrote: > On Thu, Nov 10, 2016 at 12:07 PM, Wayne Warren <wa...@puppet.com> wrote: > >> On Wed, Nov 9, 2016 at 6:41 PM, Thanh Ha <thanh...@linuxfoundation.org> >> wrote: >> >> https://review.openstack.org/358019 Support explicit API and simple >>> config creation >>> >>> There's also some TODO actions in the etherpad that we need to decide if >>> we want to continue pushing for in the JJB 2.0 push. >>> >> >> I think the best forum for making a decision on these items would be here >> on the mailing list. If everyone else agrees let's start that in this >> thread. >> > > +1 that makes sense to me. > So on that patch: https://review.openstack.org/358019 Support explicit API and simple config creation Does it make sense? I'd like to hear other people's opinion on whether it makes sense to have an explicit API as well as a from_config(cfg) method when someone would create the config object and then pass it around, or is it more of a YAGNI? Since it impacts the API that will be published, seems like it would be a good idea to get this ironed out before releasing? -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
Hi, Quick reminder that today is the fourth planned sync up for JJB v2.0 api patches. https://wiki.openstack.org/wiki/VirtualSprints#JJB_V2.0_API_Sprint Time: August 19th 14:00 - 18:00 (UTC) - this friday Venue: #openstack-meeting (IRC) Not quite ready to land all of the remaining patches for the 2.0 API work during the upcoming sprint slot later today, but Thanh has been kind enough to spend some time to get some ready that can be landed. Additionally as suggested in the IRC room yesterday, can use this timeslot to focus on any TODO tasks and whatever else is needed to help get the remaining patches ready for the next session! https://etherpad.openstack.org/p/jjb_api_v2.0 has been updated with a list of patches that can be landed. Also included two additional patches in the etherpad that would benefit from feedback. These are design changes to the code where there are two reasonable directions. Thanh has helpfully moved things around so we can delay landing these without preventing most of the remaining work from being reviewed and approved. Chat to you in #openstack-sprint later this afternoon. Happy reviewing! > -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
Hi, Quick reminder that we're approaching the third planned sync up for JJB v2.0 api patches. https://wiki.openstack.org/wiki/VirtualSprints#JJB_V2.0_API_Sprint Time: August 5th 14:00 - 18:00 (UTC) - this friday Venue: #openstack-meeting (IRC) Seen some great work in the first two sync-ups with 11 patches landed, 8 from the original series and 3 additional changes. Hoping for another 8+ to land in this next session :-) The first two sync ups help define a sensible work flow, with the second one showing substantial productivity improvement in applying the learnings from the initial meeting. Here's a quick reminder of those: - Try to review the next set of patches before the sync up start - Use the meeting time to agree what changes need to be made if any - Document changes that can be added as follow up work - Then look to try and sub divide some patches to spread the load Updated https://etherpad.openstack.org/p/jjb_api_v2.0 with the next set of patches, 8 in total, that hopefully will get landed by end of this upcoming session. Feel free to comment on any in the entire set, this is just to help prioritise a part of the series that needs to land before the remainder. As there are a few TODO's listed, I would encourage anyone interested to help pick up the TODO's, as these are things that are needed, and can be proposed to be landed onto the existing tree right now. These were changes spotted where it was decided to rework it subsequently to balance the productivity penalty of rebasing the entire series against the benefit of making the change there. We still want these as part of the V2.0 API work, just make more sense to do it on top. Also included two additional patches in the etherpad that would benefit from feedback. They are design changes to the code where there are two reasonable directions, and will benefit from having time to step back and have a few days to consider, rather than trying to decide right before inclusion. So better to think about them before the sync up planned for the 19th, when they would be planned to be landed. Chat to you in #openstack-sprint on Friday. Happy reviewing! -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
Hi, Just realised I forgot to include the date/time in the email (I know it's available via the virtual sprint wiki page, should have still mentioned it) Next sprint/sync-up is schedule: 14:00 - 18:00 (UTC) 22nd July On 20 July 2016 at 12:45, Darragh Bailey <daragh.bai...@gmail.com> wrote: > Hi, > > > Quick reminder that we're approaching the second planned sync up for JJB > v2.0 api patches. > https://wiki.openstack.org/wiki/VirtualSprints#JJB_V2.0_API_Sprint > > > https://etherpad.openstack.org/p/jjb_api_v2.0 contains a list of 7 > patches that I'd like us to get close to merging. > > > The first sync up helped clarify a few things, and I've a couple of > suggestions to help us improve on the progress made: > >- Try to review the next set of patches before the sync up start >- Use the meeting time to agree what changes need to be made if any >- Document changes that can be added as follow up work (although >rebasing the set with invasive reworks can help you understand how >everything is working and help you better understand the entire series, >it's best if we can avoid doing that for the proposed patches for the sync >up, and do any more extensive rework outside of them). >- Then look to try and sub divide some patches to spread the load > > > I've pushed up a rebased set of Wayne's patches yesterday and this > morning, to make it easier to follow the changes added to the series by a > patch I've added in. > > Hopefully everyone gets a chance to review in time, and I think things > will get substantially easier now after this week, hopefully see more > landed outside of the sync-ups from next week. > > Chat to you in #openstack-sprint on Friday. > > -- > Darragh Bailey > "Nothing is foolproof to a sufficiently talented fool" > -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
Hi, Quick reminder that we're approaching the second planned sync up for JJB v2.0 api patches. https://wiki.openstack.org/wiki/VirtualSprints#JJB_V2.0_API_Sprint https://etherpad.openstack.org/p/jjb_api_v2.0 contains a list of 7 patches that I'd like us to get close to merging. The first sync up helped clarify a few things, and I've a couple of suggestions to help us improve on the progress made: - Try to review the next set of patches before the sync up start - Use the meeting time to agree what changes need to be made if any - Document changes that can be added as follow up work (although rebasing the set with invasive reworks can help you understand how everything is working and help you better understand the entire series, it's best if we can avoid doing that for the proposed patches for the sync up, and do any more extensive rework outside of them). - Then look to try and sub divide some patches to spread the load I've pushed up a rebased set of Wayne's patches yesterday and this morning, to make it easier to follow the changes added to the series by a patch I've added in. Hopefully everyone gets a chance to review in time, and I think things will get substantially easier now after this week, hopefully see more landed outside of the sync-ups from next week. Chat to you in #openstack-sprint on Friday. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
Took a bit of doing, but worked out what was needed to use #openstack-sprint Created an entry on the Virtual Sprint wiki page: https://wiki.openstack.org/wiki/VirtualSprints#JJB_V2.0_API_Sprint Chat to you all tomorrow. On 6 July 2016 at 21:01, Darragh Bailey <daragh.bai...@gmail.com> wrote: > Hi, > > On 6 Jul 2016 20:26, "Paul Belanger" <pabelan...@redhat.com> wrote: > > > > Snipped > > > > Also thinking that spinning up a temporary public dedicated IRC chat > room > > > would be helpful here, probably look to avoid using one of the existing > > > meeting rooms because I'm assuming that would conflict with other > teams, > > > unless someone tells me there is a simple solution to this. Only > negative I > > > can see is that the chats wouldn't be logged. > > > > > You may want to check if the #openstack-sprint channel is available on > freenode. > > We have logging enabled on it and seems like a natural fit for your > agenda. > > > > Fantastic, thanks for that suggestion. Even if it's not those time, > hopefully it will be for the next ones. > > -- > Darragh Bailey > "Nothing is foolproof to a sufficiently talented fool" - unknown > -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
Sounds like Friday is the clear winner. Do we want to stay this Friday? Or would everyone prefer a bit more notice to plan things with their other work? I'm not around Friday 15th, but it could start then anyway if others are happy (might need one more core to help land stuff that day)? Otherwise could wait until the 22nd, naturally can still do normal reviews in the mean time. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" - unknown On 6 Jul 2016 08:31, "Dong Ma" <winterma.d...@gmail.com> wrote: > Hey Darragh, > > For Mon/Fri, the current time slots both works for me. > > Thanks, > Dong > > > >>> On Mon, Jul 4, 2016 at 8:47 PM, Dong Ma <winterma.d...@gmail.com> wrote: >>> >>>> Hey Darragh, >>>> >>>> >>>> >>>> I agree with your thought and would like to participate to the reviews, >>>> although the time slots is a little late for me but it works. >>>> >>>> >>>> >>> >> Would moving the time slot an hour earlier on either Mon/Fri suit you >> better? >> >> -- >> Darragh Bailey >> "Nothing is foolproof to a sufficiently talented fool" >> > > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
It's starting to sound like Friday would suit people best, I'm happy enough with that day as well, may have to leave slightly before 17hr utc, but should be able to be present for most of the time. Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" - unknown On 5 Jul 2016 17:21, "Wayne Warren" <wa...@puppet.com> wrote: > > > On Tue, Jul 5, 2016 at 6:32 AM, Darragh Bailey <daragh.bai...@gmail.com> > wrote: > >> >> >> On 5 July 2016 at 05:44, Kien Ha <kienha9...@gmail.com> wrote: >> >>> HI Darragh, >>> >>> I would like to help where I can in the reviews, but Tuesdays and >>> Thursdays are my busiest days. I can only guarantee that I am free after >>> 19:00 UTC on Tuesdays and Thursdays. Monday and Fridays are days where I am >>> free from summer class obligations but I am not too sure if that will work >>> with everyone else. >>> >> >> I can do Monday, assuming Friday is a little more awkward for people in >> European timezones as they plan to wrap up for the day earlier. >> >> Woud 13:00-1700 UTC suit on those days? >> >> >>> >>> Regards, >>> Kien Ha >>> >>> On Mon, Jul 4, 2016 at 8:47 PM, Dong Ma <winterma.d...@gmail.com> wrote: >>> >>>> Hey Darragh, >>>> >>>> >>>> >>>> I agree with your thought and would like to participate to the reviews, >>>> although the time slots is a little late for me but it works. >>>> >>>> >>>> >>> >> Would moving the time slot an hour earlier on either Mon/Fri suit you >> better? >> > > For the proposed time slots so far, Friday would be better for me--I have > a team meeting I cannot miss between UTC 1530 and UTC 1630 (although it > usually runs a half hour short). > > However, if Monday 1300 - 1700 UTC ends up being the best time for > everyone else, I can attempt to split my attention between the JJB 2.0 > review session and that meeting, or set aside dedicated time outside of the > review session to respond to comments and iterate on the 2.0 patches > (really I will want to do this anyway). > > >> -- >> Darragh Bailey >> "Nothing is foolproof to a sufficiently talented fool" >> >> ___ >> OpenStack-Infra mailing list >> OpenStack-Infra@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> >> > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB V2.0 planning
On 5 July 2016 at 05:44, Kien Ha <kienha9...@gmail.com> wrote: > HI Darragh, > > I would like to help where I can in the reviews, but Tuesdays and > Thursdays are my busiest days. I can only guarantee that I am free after > 19:00 UTC on Tuesdays and Thursdays. Monday and Fridays are days where I am > free from summer class obligations but I am not too sure if that will work > with everyone else. > I can do Monday, assuming Friday is a little more awkward for people in European timezones as they plan to wrap up for the day earlier. Woud 13:00-1700 UTC suit on those days? > > Regards, > Kien Ha > > On Mon, Jul 4, 2016 at 8:47 PM, Dong Ma <winterma.d...@gmail.com> wrote: > >> Hey Darragh, >> >> >> >> I agree with your thought and would like to participate to the reviews, >> although the time slots is a little late for me but it works. >> >> >> > Would moving the time slot an hour earlier on either Mon/Fri suit you better? -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] JJB V2.0 planning
Hi, To try and minimise trashing of both core reviews and V2.0 patch author(s), I'd like to propose that we pick a time/day every second week for 3-4 iterations where those interested set aside a set block of time to collaborate in getting the main rework patches landed. Consider it a set of mini bug days focused on JJB 2.0 API changes. To get the ball rolling, I'm going to suggest one of the following 2 timezones (obviously these suit me best, but I'm available the other days as well): 14:00-1800 UTC Thurs (Staring 7th July - not available the 14th hence suggesting this Thurs) 14:00-1800 UTC Tues (Staring 12th July) I'm assuming that later in the day for me aligns better with others, but I could be very wrong. Also thinking that spinning up a temporary public dedicated IRC chat room would be helpful here, probably look to avoid using one of the existing meeting rooms because I'm assuming that would conflict with other teams, unless someone tells me there is a simple solution to this. Only negative I can see is that the chats wouldn't be logged. More info below on why suggest this: Having gone through a few cycles where patches get reviewed, reworked and then broken by other changes landing, reworked again, reviewed and broken again, it can be quite onerous on both author and reviewer getting a change that touches a number of places to land as the risk of another patch landing causing a merge failure increases dramatically the more places the patch touches. The set of V2 patches has to bring the existing code through some amount of interim steps to make it easy to review, unfortunately given the amount of rework to do, the odds of anything else triggering a conflict is pretty high and basically faced with the following choices: - Take a long time complete the cycle of rework -> review -> rework -> break -> rework ->. ... - Block landing any changes that touch any of the code impacted by V2 work until most V2 patches are landed. However we can get enough cores on around the same time and try for some synchronized collaboration, I think it's probably far easier to land a series of patches over a few meetings and get everything far enough along with much less workload placed on everyone involved that we can then revert back to the more async approach without the same issues around the remaining changes. Expect that this would only take 3-4 of these to get the major part of the rewrite in place. Thoughts? Does this work for enough other JJB reviewers? -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Jenkins job builder status report
Hi Kien, I missed this email, but spotted this appearing in recent commits. On 13 June 2016 at 03:28, Kien Ha <kienha9...@gmail.com> wrote: > > On Sat, Jun 11, 2016 at 11:12 PM, Dong Ma <winterma.d...@gmail.com> wrote: > >> Hello Kien Ha, >> >> Thanks for the contribution to the Jenkins job builder projects, have one >> comment here, how about add your proposal document link or create a new >> etherpad into the commit message of each patches as reference to keep >> tracking the process. >> >> Vincent >> > This kind goes outside of what is generally described as https://wiki.openstack.org/wiki/GitCommitMessages#Information_in_commit_messages The update to the mailing list is more than enough, let's keep the commits describing the functional reason for the change, or if there is a separate proposal spec/blueprint that covers changing to use the convert to xml function in JJB, changes relevant can include a reference, otherwise lets keep the commit messages focused on the change itself. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB's use of inspect plugin info requires administrator permissions
The 1.652.x series is an lts release, so fixes were backported to it that are not in subsequent dev releases. Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" - unknown On 14 Jun 2016 20:02, "Zaro" <zaro0...@gmail.com> wrote: > - [ snippet ] > > > > The behavior changed between 1.651.1 and 1.652.2. > > > > Specifically this was a security fix that came in with 1.652.2. See the > > security fixes [0] that came with the release notes. Search for > > SECURITY-250 or CVE-2016-3723. > > > > -Andy- > > > > [0] > > > https://wiki.jenkins-ci.org/display/SECURITY/Jenkins+Security+Advisory+2016-05-11 > > Hmm. I just tested with Jenkins ver 1.653 and was still able to > access plugin info using REST api as an anonymous user. > I enabled security with following settings: > * jenkins own db > * logged-in user can do anything > * prevent cross site request > > While not logged in I can get plugin info using > '/pluginManager/api/json?depth=1' > > Maybe this there's some setting you have enabled that's causing your > jenkins to require admin to access plugin info? > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Fwd: JJB 1.6.0 "Cannot create a file when that file already exists"
Doh! It's called not using windows enough. The os.rename() it appears throws an error on windows if the target file exists. We need to catch that and remove the file, then retry, or remove the old file first. The fun of cross platform. Sorry about that. Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" - unknown On 8 Jun 2016 21:15, "Sebastian Schuberth" <sschube...@gmail.com> wrote: > Hi, > > since upgrading to JJB 1.6.0 from 1.5.0 today I consistently get > "Cannot create a file when that file already exists" errors on update > / delete operations on Windows. The stack trace looks like > > $ jenkins-jobs delete github-checker.yaml > INFO:jenkins_jobs.builder:Removing jenkins job(s): github-checker.yaml > Traceback (most recent call last): > File "c:\python27\lib\runpy.py", line 162, in _run_module_as_main > "__main__", fname, loader, pkg_name) > File "c:\python27\lib\runpy.py", line 72, in _run_code > exec code in run_globals > File "C:\Python\Scripts\jenkins-jobs.exe\__main__.py", line 9, in > > File "c:\python27\lib\site-packages\jenkins_jobs\cmd.py", line 191, in > main > execute(options, config) > File "c:\python27\lib\site-packages\jenkins_jobs\cmd.py", line 357, in > execute > builder.delete_job(job, options.path) > File "c:\python27\lib\site-packages\jenkins_jobs\builder.py", line > 322, in delete_job > self.cache.save() > File "c:\python27\lib\site-packages\jenkins_jobs\builder.py", line > 113, in save > self._os.rename(tfile.name, self.cachefilename) > WindowsError: [Error 183] Cannot create a file when that file already > exists > ERROR:jenkins_jobs.builder:Failed to write to cache file > > 'C:\Users\name\.cache\jenkins_jobs\cache-host-jobs-https___hostname_com_.yml' > on exit: [Error 183] Cannot create a file when that file already > exists > > Is anyone else seeing this? > > -- > Sebastian Schuberth > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB's use of inspect plugin info requires administrator permissions
Hi Thanh, Comments inline. On 7 June 2016 at 21:35, Thanh Ha <thanh...@linuxfoundation.org> wrote: > Taking a look at the code, I realized the test command allowed spoofing of > the plugins_info. I thought I'd try and see what happens if we allowed > spoofing with the update command too and submitted this patch: > > https://review.openstack.org/326722 > > I'm wondering if this could be a possible solution to the Administrator > permissions issue assuming that providing the plugins_info yaml file causes > JJB to not query the live Jenkins system for the info. > Definitely worth looking at, though probably worth digging in first to understand why the information couldn't be retrieved in case some documentation is needed to help users. > On 7 June 2016 at 15:34, Thanh Ha <thanh...@linuxfoundation.org> wrote: > >> Hi Everyone, >> > > Unfortunately it's come to our attention that this feature in Jenkins >> requires the Administrator permission which can be problematic if you have >> an environment where you prefer not to give this permission out. I think >> the ideal solution is to build into Jenkins a separate permission for >> viewing plugin information. I'll try contacting Jenkins devs to see if this >> is something they can do inside Jenkins. >> > Curious to know what version of Jenkins you used? Is this a new security feature added by recent versions, or is it something depending on what other permissions have been enabled by default for various users? Because I can query a 1.565.3 installation of Jenkins for it's list of plugins as an anonyous user using the following URL: /pluginManager/api/json?depth=1 Perhaps it's a new behaviour for the 2.x series to prevent access to this part of the XML API by default? Definitely worth following up with Jenkins devs after looking closely at the permissions with your > Failing that maybe we can somehow make the plugin info optional in JJB? >> any thoughts around this topic? >> >> One of our use cases with this is that we have a sandbox instance of >> Jenkins deployed for our community to test jobs with however for obvious >> reasons we cannot give folks administrator access to this instance but >> unfortunately if someone is trying to use a plugin (such as the Slack >> plugin) that needs to inspect plugin versions jjb fails to push the job. >> > I'd like to know what's changed either way, if we do need additional privs to read this information over being a normal user, or if some privilege in something like the matrix security plugin or role based authentication plugin is needed, it would be important to be able to call this out in the documentation. That way if you don't want to make this information directly available to JJB, you could still allow an approved script (that runs with higher privileges) to be run as a build step to generate the plugins info for it to be read in by the user running JJB. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [logging] Announcing openstack-infra/logstash-filters
On 13 May 2016 at 18:12, Jonathan Harker <jesusau...@hpe.com> wrote: > The openstack-infra team has put a lot of effort into creating logstash > filters to parse openstack logs. These filters are primarily used to > parse service logs from devstack runs, but should work for production > deployments as well. Yesterday I worked with Clark Boylan to move these > filters out of puppet and into their own project called > openstack-infra/logstash-filters to make them easy to reconsume. This > project has three files in the filters/ directory: an example input > section, an example output section, and the filters section used to > index devstack service log data into logstash.openstack.org. Using > conf.d style logstash configs, you can easily drop these filters into > your own config while using custom input and output config sections. You > can see how this is done for logstash.openstack.org using puppet at [1]. > Is this intended as a general place for filters that might be of interest to the community to consume? Or just those that would be used by infra? Have some written for ansible and vagrant log parsing that might help people avoid rewriting, as well as rule that drops empty messages. Been wondering whether there was somewhere to contribute them. > [1] > > https://review.openstack.org/#/c/310052/6/modules/openstack_project/manifests/logstash_worker.pp > [2] > > http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/logstash/jenkins-log-client.yaml#n31 > > -- > Jonathan Harker > -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Austin Design Summit space needs
Hi Wayne, Unfortunately I won't be there to be able to catch up with you on the proposed changes, but I will try to be around on IRC more during the summit mornings, so if you happen to be covering anything involving specs or need some stuff reviewed/approved I'll try to make myself available. Course if we could manage it next week to set aside a 3-4 hours on one day to try and close out as many of the existing changes that would be applicable without your changes for 2.x it might make things easier for you if we could land them and cut a final stable 1.x release, with a view to focus on landing your proposed changes. On 23 March 2016 at 18:12, Wayne Warren <wa...@puppetlabs.com> wrote: > > > On Tue, Mar 22, 2016 at 3:23 PM, Elizabeth K. Joseph <l...@princessleia.com > > wrote: > >> On Fri, Mar 11, 2016 at 9:27 AM, Elizabeth K. Joseph >> <l...@princessleia.com> wrote: >> > https://etherpad.openstack.org/p/infra-newton-summit-planning >> >> I've also gone ahead and added a section at the bottom for "Other >> sessions of interest" for non-infra sessions that an infra presence >> would be particularly valuable at. >> > > Would this be a good place to propose JJB design discussion (both my > recent 2.x work and more general YamlParser features moving forward)? > > >> >> We've been pretty good at divide and conquer on the fly at summits, >> but with the team growing I thought it would be valuable to call out >> some of the key sessions ahead of time to make sure we have coverage. >> I know I could always use some infra backup at the translations >> sessions, which I've added a reference to seed this section. >> >> -- >> Elizabeth Krumbach Joseph || Lyz || pleia2 >> >> ___ >> OpenStack-Infra mailing list >> OpenStack-Infra@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> > > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > > -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] Fwd: JJB nested template variables
Sorry forgot use reply-all.
Also slight mistake below, ignore my comment about needing to look at
deep_format. I'm pretty certain that it's just the dimensions piece you
want to look at. Assuming I understand your intent correctly.
-- Forwarded message --
From: Darragh Bailey <daragh.bai...@gmail.com>
Date: 14 March 2016 at 14:51
Subject: Re: [OpenStack-Infra] JJB nested template variables
To: Thanh Ha <thanh...@linuxfoundation.org>
Hi Thanh,
On 9 Mar 2016 22:47, "Thanh Ha" <thanh...@linuxfoundation.org> wrote:
> Hi Everyone,
>
> I'm trying to nest template variables and discovered that JJB behaves in a
> way I didn't expect when a template variable is nested. For example:
>
> - project:
> name: test
> jobs:
> - '{name}-verify-{value}-{jdk}'
>
> value:
> - a:
> jdk:
> - openjdk8
>
> - job-template:
> name: '{name}-verify-{value}-{jdk}'
>
> When jdk is nested under the value a. It generates a job with the name:
>
> *'test-verify-a-['\''openjdk8'\'']'*
>
Possibly we should through an exception and complain that it's not
acceptable to have jdk resolve to a list instead of a string? Or is it you
want to expand this to be an additional dimension upon which to generate
more templates based on jdk being a list of possible values?
I noticed that if I switch the above to:
value:
- a:
jdk: openjdk8
That job is expanded as: test-verify-a-openjdk8
> You'll notice the name also includes extra single quotes at the beginning
> and end of the job name itself in addition to a python list being passed
> out in place of the jdk variable. However if you don't nest JDK and put it
> by itself you get the expected name of *test-verify-a-openjdk8 *without
> the extra single quotes and python list.
>
It's a little unclear what you formats you were referring to.
> I'd be interested in attempting to fix this since I have a use case for
> having job templates with nested variables. Would this be something that's
> easy to fix or would this have some larger affect on the code base?
>
> Can someone point me to the code files related to the template name
> scheme. I can have a poke at it and see if it's something that can be fixed.
>
I believe this involves looking at how the deep_format function works. The
various values for jdk get stored in the dimensions variable in the
jenkins_jobs/parser.py code see
https://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/jenkins_jobs/parser.py?id=ccf682934d14f4a2395c7b9f6d463b9667b938c8#n253
Assuming what you want to do is allow the template to be expanded to
different job names based on nested variables being a list, I think the
exact piece of code to be changed is the loop that constructs the
dimensions that is then iterated over to generate the jobs. See
https://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/jenkins_jobs/parser.py?id=ccf682934d14f4a2395c7b9f6d463b9667b938c8#n258
This snippet is where you need to focus:
for (k, v) in project.items():
tmpk = '{{{0}}}'.format(k)
if tmpk not in template_name:
continue
if type(v) == list:
dimensions.append(zip([k] * len(v), v))
> Thanks,
> Thanh
>
--
Darragh Bailey
"Nothing is foolproof to a sufficiently talented fool"
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB optional parameters usefulness
xisting behaviour via some setting. Expect we'll have to support existing use of optionals for at least one major release for the current modules whether we decide to switch the behaviour for the future or not. So the change identified will have to keep that in consideration. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] JJB 1.4.0 delete-all no longer deletes jobs
Hi Thanh, I suspect there's an issue with the groovy script that is used in place of the REST api point executing, see: https://github.com/openstack-infra/jenkins-job-builder/blob/master/jenkins_jobs/builder.py#L168-L173 Maybe you don't have sufficient privs to execute the script, but the code isn't checking the result of calling run_script() on Jenkins. Any chance you could try and capture the return value from those lines? On 19 January 2016 at 18:00, Wayne Warren <wa...@puppetlabs.com> wrote: > What command line flags did you pass? > > I don't generally use this or the 'delete' subcommand, preferring > instead to use python-jenkins directly. > > On Sun, Jan 17, 2016 at 5:03 PM, Thanh Ha <thanh...@linuxfoundation.org> > wrote: >> Hi Everyone, >> >> It seems to me that JJB 1.4.0's delete-all function has regressed and no >> longer performs the delete function. Instead it simply provides the >> following output and exits without performing any deletes. >> >> Sure you want to delete *ALL* jobs from Jenkins server? >> (including those not managed by Jenkins Job Builder) (Y/N): y >> INFO:root:Deleting all jobs >> INFO:jenkins_jobs.builder:Number of jobs to delete: 50 >> INFO:jenkins_jobs.builder:Cache saved >> >> >> Has anyone else noticed this issue as well? >> >> I'll try to find some time to investigate which patch introduced this issue >> unless someone else gets to it first but thought I'd inform the team. >> >> Regards, >> >> Thanh >> >> >> ___ >> OpenStack-Infra mailing list >> OpenStack-Infra@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Regression in JJB 1.4.0 maximum recursion depth exceeded
Btw, as a very dirty hack the following might work within the jenkins job: jenkins-jobs test -r builder/jjb 2>&1 | cat On 11 January 2016 at 17:49, Thanh Ha <thanh...@linuxfoundation.org> wrote: > On 11 January 2016 at 11:03, Thanh Ha <thanh...@linuxfoundation.org> wrote: >> >> On 9 January 2016 at 07:10, Darragh Bailey <daragh.bai...@gmail.com> >> wrote: >>> >>> On 8 Jan 2016 21:47, "Thanh Ha" <thanh...@linuxfoundation.org> wrote: >>> >>> > >>> > (I truncated some of the repetitive output below) >>> > >>> > I just tried with the -o argument and seems like it passes. I guess it >>> > only affects stdout output, good to know. >>> > >>> > We're using python 2.7.5 in production. I did some testing today with >>> > Python 3.4.3 and it seems the 3.4.x stream at least doesn't run into this >>> > issue so I'm guessing this only affects Python 2.7.x. >>> >>> I wonder if it's something that occurs with earlier 2.7 releases, I >>> thought I checked that behavior out on 2.7.9 >>> >>> I'll look to check with pyenv on Monday to go over a few python versions. >> >> >> Hi Darragh, >> >> The extremely strange thing about this one is I've only been able to >> reproduce it in Jenkins. So imagine having JJB installed on the same server >> as your Jenkins instance or slave. If you ssh to the machine and run JJB on >> the commandline it works. Create a job running on the same system and >> Jenkins will see the failure in console logs. >> >> In case it helps with reproducing. The git repo I've been using to >> reproduce this error is this one: >> >> https://git.opendaylight.org/gerrit/#/admin/projects/releng/builder >> >> After cloning this repo run "jenkins-jobs test -r jjb/". > > > > I just tested this with Python 2.7.9 and confirm the issue still persists > for me even with 2.7.9. You can see my test here: > > https://jenkins.opendaylight.org/sandbox/job/thanh-test/7/console > > Regards, > > Thanh > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Regression in JJB 1.4.0 maximum recursion depth exceeded
Hi Thanh, On 8 Jan 2016 21:47, "Thanh Ha" <thanh...@linuxfoundation.org> wrote: > > (I truncated some of the repetitive output below) > > I just tried with the -o argument and seems like it passes. I guess it only affects stdout output, good to know. > > We're using python 2.7.5 in production. I did some testing today with Python 3.4.3 and it seems the 3.4.x stream at least doesn't run into this issue so I'm guessing this only affects Python 2.7.x. I wonder if it's something that occurs with earlier 2.7 releases, I thought I checked that behavior out on 2.7.9 I'll look to check with pyenv on Monday to go over a few python versions. -- Darragh Bailey "Nothing is foolproof to a sufficiently talented fool" - unknown ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [jjb] What's the deal with {{?
Hi,
macros do not get substitution performed unless you provide a variable to
be substituted in.
Following would trigger the behaviour:
- job-template:
name: '{foo}-test'
builders:
- test_builder:
FOO_BAR: hello
- shell: |
echo ${{FOO_3}}
Definitely a bit confusing, essentially the macro is added in through a
lookup in the register.ModuleRegistry.dispatch() method, where only if the
component is a dict is a substitution performed:
https://github.com/openstack-infra/jenkins-job-builder/blob/4a8b93b8c2d517c8dc27d91437454890cc49a640/jenkins_jobs/registry.py#L149-L168
I wonder if jinja templating would avoid some of the quirks we run into
around using python's string formatting for substitution?
On 13 August 2015 at 07:05, Ian Wienand iwien...@redhat.com wrote:
Hi,
Just trying to get my head around this from [1]:
---
- builder:
name: test_builder
builders:
- shell: |
echo ${FOO_1}
echo ${{FOO_2}}
- job-template:
name: '{foo}-test'
builders:
- test_builder
- shell: |
echo ${{FOO_3}}
- project:
name: 'foo'
jobs:
- '{foo}-test':
foo: bar
---
that's going to output a job basically
---
echo ${FOO_1}
echo ${{FOO_2}}
echo ${FOO_3}
---
Why do I *not* get a FOO_1 parameter missing for test_builder? If I
do
---
- test_builder:
FOO_1: bar
---
it does actually come out with echo $bar as you might expect.
Or the same question in reverse: why *do* I get an error about a
missing parameter if I have just ${FOO_3} in the job-template?
I can't find a clear explanation for this, although there might be
one I'm missing. If I can find one, I'll add it to some sort of
documentation.
-i
[1] https://review.openstack.org/#/c/212246
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
--
Darragh Bailey
Nothing is foolproof to a sufficiently talented fool
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] enforce bug ids in commit message
I believe the its-* plugins for gerrit could handle this for you, recently someone mentioned on the repo mailing list that you can even exclude branches from the requirement. I'd suggest asking there directly if you haven't already. Darragh On 23 Mar 2015 22:17, Jeremy Stanley fu...@yuggoth.org wrote: On 2015-03-23 12:00:26 +0530 (+0530), Vinay Mahuli wrote: I need info on where to make this change in a similar way how Change-Id is checked/validated. Require Change-Id in commit message is a project-specific setting built into Gerrit by default. URL: https://gerrit-review.googlesource.com/Documentation/project-configuration.html#_require_change_id Something similar may be possible by writing a Gerrit plug-in or a hook script, but it will almost certainly use a different mechanism as it won't be implemented directly in the Gerrit codebase as a built-in feature. -- Jeremy Stanley ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [infra] Nominations for jenkins-job-builder core
On 20 June 2014 22:01, James E. Blair cor...@inaugust.com wrote: Hi, The Jenkins Job Builder project (part of the Infrastructure program) is quite popular even outside of OpenStack and has a group of specialist core reviewers supplemental to the rest of the Infrastructure program. To that group I would like to add Darragh Bailey: https://review.openstack.org/#/q/reviewer:%22Darragh+Bailey%22+project:openstack-infra/jenkins-job-builder,n,z -Jim Thanks for the nomination Jim, I'm enjoying working on JJB immensely and hope that I can continue to contribute in a way that others appreciate. -- Darragh Bailey Nothing is foolproof to a sufficiently talented fool ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] feedback request on Use yaml local tags to support including files
Hi, Looking to see if there is anything more needed for https://review.openstack.org/#/c/48783/, whether I can help answer any concerns or if more information is required in the commit message to explain? The intent behind this change is to be able to move all scripts out from being embedded in the yaml into separate files, which can then be included by the yaml loading in JJB in a transparent manner. This in turn makes it much easier to separate code (scripts) from configuration (yaml) and subsequently test your scripts directly. -- Darragh Bailey Nothing is foolproof to a sufficiently talented fool ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
