Re: Jenkins in new cloud - status update
On Fri, Jul 31, 2015 at 12:24:50AM +0200, Mikolaj Izdebski wrote: > On 07/30/2015 09:11 PM, Kevin Fenzi wrote: > >> Deployment details: There are: 1 master and 3 slaves (workers) running > >> in the new cloud. All instances are of type "m1.small" (1 CPU, 2 GB > >> RAM, 20 GB disk). > > > > In the old cloud some of the builders were bigger, should we bump > > these up some when we move to production? > > I will check instances on the old cloud, now I should be able to access > them. I wanted to avoid unnecessary resource consumption, given that > this is only dev instance. +1 for a larger disk size. We got bitten by this in the past that some projects are quite demanding space wise and keeping the results of the builds also increases the disk usage. > >> Only members of sysadmin-jenkins group have admin access through web > >> interface. Should sysadmin-main be added there too? > > > > We could, but I think all the sysadmin-main folks that are interested > > are already in sysadmin-jenkins and/or we could add them easily. > > Ok, I'll leave it as-is (sysadmin-jenkins only). > > >> Subversion plugin is currently disabled due to unresolved problem. > >> Projects using subversion SCM (if any) won't work until this is fixed. > > > > Do we have any SVN projects in the old cloud? > > I don't know, but there is subversion plugin installed. As far as I remember there are none, I added the plugin to be complete but I don't know of any project using it. Thanks for working on this! Pierre ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Jenkins in new cloud - status update
On 07/30/2015 09:11 PM, Kevin Fenzi wrote: >> Deployment details: There are: 1 master and 3 slaves (workers) running >> in the new cloud. All instances are of type "m1.small" (1 CPU, 2 GB >> RAM, 20 GB disk). > > In the old cloud some of the builders were bigger, should we bump > these up some when we move to production? I will check instances on the old cloud, now I should be able to access them. I wanted to avoid unnecessary resource consumption, given that this is only dev instance. >> Only members of sysadmin-jenkins group have admin access through web >> interface. Should sysadmin-main be added there too? > > We could, but I think all the sysadmin-main folks that are interested > are already in sysadmin-jenkins and/or we could add them easily. Ok, I'll leave it as-is (sysadmin-jenkins only). >> Subversion plugin is currently disabled due to unresolved problem. >> Projects using subversion SCM (if any) won't work until this is fixed. > > Do we have any SVN projects in the old cloud? I don't know, but there is subversion plugin installed. I'm sure there is room for improvement - removing unneeded plugins and slave packages, but I didn't manage to do that yet, give the short deadline. -- Mikolaj Izdebski Software Engineer, Red Hat IRC: mizdebsk ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Jenkins in new cloud - status update
On 07/30/2015 07:46 PM, Joshua hoblitt wrote: > On 07/30/2015 10:39 AM, Mikolaj Izdebski wrote: >> Jenknis runs as unprivileged user, so it can't listen on port 80 - it >> listens on port 8080. Old Jenkins was running httpd proxy, which >> forwarded incoming requests from port 80 to 8080. New Jenkins >> forwards ports using netfilter (aka iptables). Was there any other >> reason for running httpd proxy? > > At $day_job, I've been using nginx as a reverse proxy in front of the > jenkins master to handle TLS termination and inject HTTP headers (HSTS, > key pinning). In theory, it also allows us to show a help page instead > of timeout / negative http status codes from jenkins during a restart or > other problem. That's a good point. It may be worth to have similar setup for Fedora Jenkins. -- Mikolaj Izdebski Software Engineer, Red Hat IRC: mizdebsk ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Jenkins in new cloud - status update
On Thu, 30 Jul 2015 19:39:12 +0200 Mikolaj Izdebski wrote: > Overall status: most of things are configured and working. New > Jenkins should be ready for data migration from old instance. I've > migrated two projects for now and they seem to be working. Great news. Thanks so much for working on this. ;) > Deployment details: There are: 1 master and 3 slaves (workers) running > in the new cloud. All instances are of type "m1.small" (1 CPU, 2 GB > RAM, 20 GB disk). In the old cloud some of the builders were bigger, should we bump these up some when we move to production? > master: > jenkins.fedorainfracloud.org (Fedora 22) > slaves: > jenkins-slave-el6.fedorainfracloud.org (Centos 6.6) > jenkins-slave-el7.fedorainfracloud.org (RHEL 7.1) > jenkins-slave-f22.fedorainfracloud.org (Fedora 22) > > Some notes or questions, in random order: > > Slaves have public IP assigned. It's not needed by Jenkins itself, > but I think it is required to be able to manage them through ansible. Yeah, easiest to just leave this as is. We could proxy to them via the master, but it would be a hassle to get all working right. > Master is running official jenkins package from Fedora 22, with > several core plugins also from Fedora. Plugins which were not > available in Fedora are packaged in Copr repo and installed as RPMs > too. Packages in Copr are just wrappers around upstream binaries - > they are not built from source in Copr. > > Jenknis runs as unprivileged user, so it can't listen on port 80 - it > listens on port 8080. Old Jenkins was running httpd proxy, which > forwarded incoming requests from port 80 to 8080. New Jenkins > forwards ports using netfilter (aka iptables). Was there any other > reason for running httpd proxy? > > Only members of sysadmin-jenkins group have admin access through web > interface. Should sysadmin-main be added there too? We could, but I think all the sysadmin-main folks that are interested are already in sysadmin-jenkins and/or we could add them easily. > Subversion plugin is currently disabled due to unresolved problem. > Projects using subversion SCM (if any) won't work until this is fixed. Do we have any SVN projects in the old cloud? > Slaves have the same set of extra packages installed as in the old > cloud. Like in the old Jenkins instance, jenkins user is member of > mock group, which opens possibility for arbitrary code execution on > any of Jenkins hosts (on slaves as root, on master as jenkins user) > for anyone with write access to any project. I'm not saying it's > necessairly a problem, but I wanted to point it out clearly. Yep. :( > Current Jenkins master doesn't have persistent storage, but I would > like to add it when moving to production. Easy to do anytime. Thanks again for working on it. kevin pgpY4TCZLLTSO.pgp Description: OpenPGP digital signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Jenkins in new cloud - status update
On 07/30/2015 10:39 AM, Mikolaj Izdebski wrote: Jenknis runs as unprivileged user, so it can't listen on port 80 - it listens on port 8080. Old Jenkins was running httpd proxy, which forwarded incoming requests from port 80 to 8080. New Jenkins forwards ports using netfilter (aka iptables). Was there any other reason for running httpd proxy? At $day_job, I've been using nginx as a reverse proxy in front of the jenkins master to handle TLS termination and inject HTTP headers (HSTS, key pinning). In theory, it also allows us to show a help page instead of timeout / negative http status codes from jenkins during a restart or other problem. -Josh -- ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Jenkins in new cloud - status update
Overall status: most of things are configured and working. New Jenkins should be ready for data migration from old instance. I've migrated two projects for now and they seem to be working. Deployment details: There are: 1 master and 3 slaves (workers) running in the new cloud. All instances are of type "m1.small" (1 CPU, 2 GB RAM, 20 GB disk). master: jenkins.fedorainfracloud.org (Fedora 22) slaves: jenkins-slave-el6.fedorainfracloud.org (Centos 6.6) jenkins-slave-el7.fedorainfracloud.org (RHEL 7.1) jenkins-slave-f22.fedorainfracloud.org (Fedora 22) Some notes or questions, in random order: Slaves have public IP assigned. It's not needed by Jenkins itself, but I think it is required to be able to manage them through ansible. Master is running official jenkins package from Fedora 22, with several core plugins also from Fedora. Plugins which were not available in Fedora are packaged in Copr repo and installed as RPMs too. Packages in Copr are just wrappers around upstream binaries - they are not built from source in Copr. Jenknis runs as unprivileged user, so it can't listen on port 80 - it listens on port 8080. Old Jenkins was running httpd proxy, which forwarded incoming requests from port 80 to 8080. New Jenkins forwards ports using netfilter (aka iptables). Was there any other reason for running httpd proxy? Only members of sysadmin-jenkins group have admin access through web interface. Should sysadmin-main be added there too? Subversion plugin is currently disabled due to unresolved problem. Projects using subversion SCM (if any) won't work until this is fixed. Slaves have the same set of extra packages installed as in the old cloud. Like in the old Jenkins instance, jenkins user is member of mock group, which opens possibility for arbitrary code execution on any of Jenkins hosts (on slaves as root, on master as jenkins user) for anyone with write access to any project. I'm not saying it's necessairly a problem, but I wanted to point it out clearly. Current Jenkins master doesn't have persistent storage, but I would like to add it when moving to production. Any questions or comments? -- Mikolaj Izdebski Software Engineer, Red Hat IRC: mizdebsk ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
