The bare metal scenario: ---------------------------------- > now, if you explain more, i can help you more, but the technical > requirements (how) should came after the business requirements (what/why) > Forget about the outer virtualisation for a moment. In a bare metal world, vagrant+vb are used to build testing environments. It is used to run unit and functional tests.
Let's take a simple web project as an example: - the project has ansible playbooks tied to it - the playbooks define a web and a database server - to be able to run the tests inside the specified environment, jenkins needs to build the servers according to current specifications - all this is done to ensure the code runs in the currently defined systems (e.g. there may be php upgrades in the feature branch or any other nasty things) As far as I know, it is common practice to use vagrant for that scenario. Vagrant is also used for development. So the pretty much same environment is used for development, for tests and later also for deployment. Production and staging servers are not deployed using vagrant and they do not run inside virtual box, but the ansible-playbooks provision them the same way, as the virtual boxes in development or during tests in Jenkins are. Does that make sense so far? > if you will be using this as part of your pipeline, say ci/cd then i will > say looks more prod than dev, and i would ask " is vagrant the best box for > this ? " > I think vagrant is the best solution for that. This is why there are vagrant plugins for jenkins, I guess. Now we come to nested virtualisation: --------------------------------------------------- Our operations department would like us to run Jenkins inside a KVM domain, because they want to avoid to maintain a bare metal box. This is the business requirement and this is also why I tried to set up things inside KVM. I found everything is working inside the KVM domain, but virtual box performs weak. Although the performance of the web project would be good enough for development, it is too weak for unit tests. The tests should run as fast as possible. After some investigation, people tend to say virtual box does not like to be nested. Well, according to my findings, this may be true. This is what brings us to the cloud: ---------------------------------------------- If the bare metal scenario with vagrant+vb+jenkins is common practice, I wonder how people do this in a cloud environment. I do not want to move Jenkins to the cloud, nor do I care why other people do it. But I am sure there are people out there who run jenkins in the cloud. This is the only thing, that counts at the moment. And if they do it in the cloud, they would run VB inside a VM, as any cloud server is virtualised, isn't it? Is anybody able to follow my arguments? So one of the following is true: 1) People do not use VB to build their testing environments, neither bare metal nor in the cloud (what else do they do to make sure the test system is well defined?) 2) People use VB to build the testing environments and so they use it in the cloud. If 1) applies, I am eagerly interested in what they use instead of VB? When doing it in the cloud, do they set up new EC2 instances for any test run?!?! If 2) applies, it is an indicator that our local setup with KVM is wrong. This does not yet prove it is possible with KVM, but at least we know that virtual box is able to be run as a nested hypervisor somehow. I need some people to tell me about their experience running Jenkins+vagrant+vb in the cloud. People having this setup run in KVM are also welcome, but it is probably easier to find someone doing this in the cloud. Experiences of somebody having done (or having tried) this in in the cloud, would give me valuable hints. Where is the failure in the line? PS: Docker is not an option, as we are not able to deploy containers to our production servers. > Thanks > Alvaro > Thank you, too! Thomas > > On Thu, Feb 8, 2018 at 7:42 PM, Thomas Baumann <tho...@thomas-baumann.me> > wrote: > >> No no it is different. Jenkins runs inside a KVM guest. Jenkins uses >> vagrant + virtual box to build the testing environment in which the actual >> application tests run. The latter is common practice. >> >> In deed this setup does not perform very well. Virtual box instances can >> be set up, they do run with multiple cores, but performance is weak. >> >> Now let's assume virtual box does really not like to be run inside a >> guest, and given that using vagrant + virtual box to build the testing >> environments is common practice with Jenkins, I wonder how people run such >> a setup in cloud environments, then. They will not have the KVM part, but >> as soon as they use EC2 instances, they are actually running Jenkins inside >> a virtualized environment. Don't they? AFAIK amazon uses Xen, which is a >> type 1 hypervisor. KVM on the other hand is considered to be a type 2 >> hypervisor, >> >> So why is it my setup does not play along nicely? Is it the specific KVM >> and virtual box configurations, that might be wrong or does virtual box >> just not like to be virtualized? Does it just not like type 2 hypervisors >> or no hypervisors in general? If people really do not do it the >> vagrant+virtual box way in the cloud, what other strategy do they follow to >> build their virtualized testing environments? >> >> I think the configurations are correct. An answer to the other questions >> would help me to get a bigger picture on the issue and narrow down the >> problem. >> >> >> >> On Thursday, February 8, 2018 at 6:55:41 PM UTC+1, Antony Stone wrote: >>> >>> On Thursday 08 February 2018 at 18:21:43, Alvaro Miranda Aguilera wrote: >>> >>> > hello >>> > >>> > how do people run a setup of vagrant + virtual box + jenkins in the >>> cloud? >>> > >>> > they don't >>> >>> Is that because it doesn't work, or because it's just not a useful idea? >>> >>> > what do you want to achieve that vagrant + virtual box + jenkins in >>> the >>> > cloud is the answer ? >>> >>> I got the impression that the original question was simply "how to run >>> Jenkins >>> in Virtualbox on a KVM guest machine". The "cloud" aspect was simply >>> introduced as an example where people would have to be running >>> Virtualbox >>> inside an already-virtualised server. I don't think Thomas intends to >>> actually run things that way, he was just giving context to his >>> question. >>> >>> >>> Antony. >>> >>> > On Thu, Feb 8, 2018 at 12:15 PM, Thomas Baumann wrote: >>> > > Recently, I tried virtualising a jenkins instance in my company, >>> meaning >>> > > jenkins should go into some KVM domain. Well, jenkins itself ran >>> just >>> > > fine under these conditions, but virtual box did not perform very >>> well. >>> > > We made sure all nested virtualisation options were set correctly. >>> The >>> > > jenkins guest is able to use processor-flags vmx, ept and >>> > > vpid. /sys/module/kvm_intel/parameters/nested states nested paging >>> is >>> > > enabled with an 'Y'. >>> > > >>> > > I asked at the virtual box forums for experiences running virtual >>> box >>> > > inside a KVM domain, but they could not really give me a hint. >>> > > >>> > > Anyway, I meanwhile accept answers like "nested virtualisation does >>> not >>> > > perform very well". We just gave it a try. Unless anybody here tells >>> me >>> > > he has managed to run VB inside KVM with good performance results, >>> that >>> > > is fine, we accept that. >>> > > >>> > > The only thing I am still wondering about is, how do people run a >>> setup >>> > > of vagrant + virtual box + jenkins in the cloud? Cloud servers are >>> > > always virtualised, so we have nested virtualisation there, too. >>> > > >>> > > Can anybody confirm vagrant + virtual box + jenkins is running fine >>> in >>> > > the cloud? Let's say with an Amazon EC2 instance? The guys from the >>> > > virtual box forum sent me here, as they actually do not use virtual >>> box >>> > > in CI environments. So they cannot say. >>> > > >>> > > Any experiences? >>> > > >>> > > Thanks! >>> >>> -- >>> I conclude that there are two ways of constructing a software design: >>> One way >>> is to make it so simple that there are _obviously_ no deficiencies, and >>> the >>> other way is to make it so complicated that there are no _obvious_ >>> deficiencies. >>> >>> - C A R Hoare >>> >>> Please reply to the >>> list; >>> please *don't* >>> CC me. >>> >> -- >> This mailing list is governed under the HashiCorp Community Guidelines - >> https://www.hashicorp.com/community-guidelines.html. Behavior in >> violation of those guidelines may result in your removal from this mailing >> list. >> >> GitHub Issues: https://github.com/mitchellh/vagrant/issues >> IRC: #vagrant on Freenode >> --- >> You received this message because you are subscribed to the Google Groups >> "Vagrant" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to vagrant-up+unsubscr...@googlegroups.com. >> To view this discussion on the web visit https://groups.google.com/d/ms >> gid/vagrant-up/4f8083bd-ca46-46eb-afdb-693285222c33%40googlegroups.com >> <https://groups.google.com/d/msgid/vagrant-up/4f8083bd-ca46-46eb-afdb-693285222c33%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Alvaro > > -- > This mailing list is governed under the HashiCorp Community Guidelines - > https://www.hashicorp.com/community-guidelines.html. Behavior in > violation of those guidelines may result in your removal from this mailing > list. > > GitHub Issues: https://github.com/mitchellh/vagrant/issues > IRC: #vagrant on Freenode > --- > You received this message because you are subscribed to a topic in the > Google Groups "Vagrant" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/vagrant-up/dLPhInOh4Pc/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > vagrant-up+unsubscr...@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/vagrant-up/CAHqq0ewvBPph4hLSezpt7QtBrCSvG > Fz2mofHdMn2YHJUG9g19A%40mail.gmail.com > <https://groups.google.com/d/msgid/vagrant-up/CAHqq0ewvBPph4hLSezpt7QtBrCSvGFz2mofHdMn2YHJUG9g19A%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list. GitHub Issues: https://github.com/mitchellh/vagrant/issues IRC: #vagrant on Freenode --- You received this message because you are subscribed to the Google Groups "Vagrant" group. To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/CA%2BNbP4OJv5CdFcAP66nQBB-SxWhHohaPTidsRdJ4Ju%3DsFgS%3Dmg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.