Re: [vagrant-up] Any experience how vagrant + virtual box + jenkins perform in the cloud?

[Thomas Baumann]

The bare metal scenario:
--

> now, if you explain more, i can help you more, but the technical
> requirements (how) should came after the business requirements (what/why)
>
Forget about the outer virtualisation for a moment. In a bare metal world,
vagrant+vb are used to build testing environments. It is used to run unit
and functional tests.

Let's take a simple web project as an example:
- the project has ansible playbooks tied to it
- the playbooks define a web and a database server
- to be able to run the tests inside the specified environment, jenkins
needs to build the servers according to current specifications
- all this is done to ensure the code runs in the currently defined systems
(e.g. there may be php upgrades in the feature branch or any other nasty
things)

As far as I know, it is common practice to use vagrant for that scenario.
Vagrant is also used for development. So the pretty much same environment
is used for development, for tests and later also for deployment.
Production and staging servers are not deployed using vagrant and they do
not run inside virtual box, but the ansible-playbooks provision them the
same way, as the virtual boxes in development or during tests in Jenkins
are.

Does that make sense so far?


> if you will be using this as part of your pipeline, say ci/cd then i will
> say looks more prod than dev, and i would ask " is vagrant the best box for
> this ? "
>
I think vagrant is the best solution for that. This is why there are
vagrant plugins for jenkins, I guess.

Now we come to nested virtualisation:
---
Our operations department would like us to run Jenkins inside a KVM domain,
because they want to avoid to maintain a bare metal box. This is the
business requirement and this is also why I tried to set up things inside
KVM. I found everything is working inside the KVM domain, but virtual box
performs weak. Although the performance of the web project would be good
enough for development, it is too weak for unit tests. The tests should run
as fast as possible.

After some investigation, people tend to say virtual box does not like to
be nested. Well, according to my findings, this may be true.

This is what brings us to the cloud:
--
If the bare metal scenario with vagrant+vb+jenkins is common practice, I
wonder how people do this in a cloud environment. I do not want to move
Jenkins to the cloud, nor do I care why other people do it. But I am sure
there are people out there who run jenkins in the cloud. This is the only
thing, that counts at the moment.

And if they do it in the cloud, they would run VB inside a VM, as any cloud
server is virtualised, isn't it? Is anybody able to follow my arguments?

So one of the following is true:
1) People do not use VB to build their testing environments, neither bare
metal nor in the cloud (what else do they do to make sure the test system
is well defined?)
2) People use VB to build the testing environments and so they use it in
the cloud.

If 1) applies, I am eagerly interested in what they use instead of VB? When
doing it in the cloud, do they set up new EC2 instances for any test run?!?!
If 2) applies, it is an indicator that our local setup with KVM is wrong.
This does not yet prove it is possible with KVM, but at least we know that
virtual box is able to be run as a nested hypervisor somehow.

I need some people to tell me about their experience running
Jenkins+vagrant+vb in the cloud. People having this setup run in KVM are
also welcome, but it is probably easier to find someone doing this in the
cloud. Experiences of somebody having done (or having tried) this in in the
cloud, would give me valuable hints.

Where is the failure in the line?

PS: Docker is not an option, as we are not able to deploy containers to our
production servers.


> Thanks
> Alvaro
>
Thank you, too!
Thomas


>
> On Thu, Feb 8, 2018 at 7:42 PM, Thomas Baumann 
> wrote:
>
>> No no it is different. Jenkins runs inside a KVM guest. Jenkins uses
>> vagrant + virtual box to build the testing environment in which the actual
>> application tests run. The latter is common practice.
>>
>> In deed this setup does not perform very well. Virtual box instances can
>> be set up, they do run with multiple cores, but performance is weak.
>>
>> Now let's assume virtual box does really not like to be run inside a
>> guest, and given that using vagrant + virtual box to build the testing
>> environments is common practice with Jenkins, I wonder how people run such
>> a setup in cloud environments, then. They will not have the KVM part, but
>> as soon as they use EC2 instances, they are actually running Jenkins inside
>> a virtualized environment. Don't they? AFAIK amazon uses Xen, which is a
>> type 1 hypervisor. KVM on the other hand is considered to be a type 2
>> hypervisor,
>>
>> So why is it my setup does not play along nicely? Is it the sp

Re: [vagrant-up] Any experience how vagrant + virtual box + jenkins perform in the cloud?

[Thomas Baumann]

The bare metal scenario:
--

> now, if you explain more, i can help you more, but the technical 
> requirements (how) should came after the business requirements (what/why)
>
Forget about the outer virtualisation for a moment. In a bare metal world, 
vagrant+vb are used to build testing environments. It is used to run unit 
and functional tests.

Let's take a simple web project as an example:
- the project has ansible playbooks tied to it
- the playbooks define a web and a database server
- to be able to run the tests inside the specified environment, jenkins 
needs to build the servers according to current specifications
- all this is done to ensure the code runs in the currently defined systems 
(e.g. there may be php upgrades in the feature branch or any other nasty 
things)

As far as I know, it is common practice to use vagrant for that scenario. 
Vagrant is also used for development. So the pretty much same environment 
is used for development, for tests and later also for deployment. 
Production and staging servers are not deployed using vagrant and they do 
not run inside virtual box, but the ansible-playbooks provision them the 
same way, as the virtual boxes in development or during tests in Jenkins 
are.

Does that make sense so far?
 

> if you will be using this as part of your pipeline, say ci/cd then i will 
> say looks more prod than dev, and i would ask " is vagrant the best box for 
> this ? "
>
I think vagrant is the best solution for that. This is why there are 
vagrant plugins for jenkins, I guess.

Now we come to nested virtualisation:
---
Our operations department would like us to run Jenkins inside a KVM domain, 
because they want to avoid to maintain a bare metal box. This is the 
business requirement and this is also why I tried to set up things inside 
KVM. I found everything is working inside the KVM domain, but virtual box 
performs weak. Although the performance of the web project would be good 
enough for development, it is too weak for unit tests. The tests should run 
as fast as possible.

After some investigation, people tend to say virtual box does not like to 
be nested. Well, according to my findings, this may be true.

This is what brings us to the cloud:
--
If the bare metal scenario with vagrant+vb+jenkins is common practice, I 
wonder how people do this in a cloud environment. I do not want to move 
Jenkins to the cloud, nor do I care why other people do it. But I am sure 
there are people out there who run jenkins in the cloud. This is the only 
thing, that counts at the moment.

And if they do it in the cloud, they would run VB inside a VM, as any cloud 
server is virtualised, isn't it? Is anybody able to follow my arguments?

So one of the following is true:
1) People do not use VB to build their testing environments, neither bare 
metal nor in the cloud (what else do they do to make sure the test system 
is well defined?)
2) People use VB to build the testing environments and so they use it in 
the cloud.

If 1) applies, I am eagerly interested in what they use instead of VB? When 
doing it in the cloud, do they set up new EC2 instances for any test run?!?!
If 2) applies, it is an indicator that our local setup with KVM is wrong. 
This does not yet prove it is possible with KVM, but at least we know that 
virtual box is able to be run as a nested hypervisor somehow.

I need some people to tell me about their experience running 
Jenkins+vagrant+vb in the cloud. People having this setup run in KVM are 
also welcome, but it is probably easier to find someone doing this in the 
cloud. Experiences of somebody having done (or having tried) this in in the 
cloud, would give me valuable hints.

Where is the failure in the line?

PS: Docker is not an option, as we are not able to deploy containers to our 
production servers.
 

> Thanks
> Alvaro
>
Thank you, too!

 

> On Thu, Feb 8, 2018 at 7:42 PM, Thomas Baumann  > wrote:
>
>> No no it is different. Jenkins runs inside a KVM guest. Jenkins uses 
>> vagrant + virtual box to build the testing environment in which the actual 
>> application tests run. The latter is common practice.
>>
>> In deed this setup does not perform very well. Virtual box instances can 
>> be set up, they do run with multiple cores, but performance is weak.
>>
>> Now let's assume virtual box does really not like to be run inside a 
>> guest, and given that using vagrant + virtual box to build the testing 
>> environments is common practice with Jenkins, I wonder how people run such 
>> a setup in cloud environments, then. They will not have the KVM part, but 
>> as soon as they use EC2 instances, they are actually running Jenkins inside 
>> a virtualized environment. Don't they? AFAIK amazon uses Xen, which is a 
>> type 1 hypervisor. KVM on the other hand is considered to be a type 2 
>> hypervisor,
>>
>> So why is it my set

Re: [vagrant-up] Any experience how vagrant + virtual box + jenkins perform in the cloud?

[Alvaro Miranda Aguilera]

hello

there are same technical aspects chosen before the scope of the
requirement, like you want to use virtualbox nested.

If that is the case, then you should look for:
- bare metal
- ideally per minute or per hour

check terraform + packet.net if you want to build and use later


now, if you explain more, i can help you more, but the technical
requirements (how) should came after the business requirements (what/why)

if you will be using this as part of your pipeline, say ci/cd then i will
say looks more prod than dev, and i would ask " is vagrant the best box for
this ? "

then we can move into there are tools that build around this already and
its easy (vs simple) to implement, and you may have this in house and runs
fine, so you want to make it scale and move it to the cloud

valid points but guessing whats the backstory is not good, better have the
whole story first, so ppl can actually share how/what they do.


vagrant defaults to virtualbox, but with plugins you can use aws or digital
ocean or others, not sure will be easy to move from what you got to that

happy to comment and help, but need more context please.

Thanks
Alvaro

On Thu, Feb 8, 2018 at 7:42 PM, Thomas Baumann 
wrote:

> No no it is different. Jenkins runs inside a KVM guest. Jenkins uses
> vagrant + virtual box to build the testing environment in which the actual
> application tests run. The latter is common practice.
>
> In deed this setup does not perform very well. Virtual box instances can
> be set up, they do run with multiple cores, but performance is weak.
>
> Now let's assume virtual box does really not like to be run inside a
> guest, and given that using vagrant + virtual box to build the testing
> environments is common practice with Jenkins, I wonder how people run such
> a setup in cloud environments, then. They will not have the KVM part, but
> as soon as they use EC2 instances, they are actually running Jenkins inside
> a virtualized environment. Don't they? AFAIK amazon uses Xen, which is a
> type 1 hypervisor. KVM on the other hand is considered to be a type 2
> hypervisor,
>
> So why is it my setup does not play along nicely? Is it the specific KVM
> and virtual box configurations, that might be wrong or does virtual box
> just not like to be virtualized? Does it just not like type 2 hypervisors
> or no hypervisors in general? If people really do not do it the
> vagrant+virtual box way in the cloud, what other strategy do they follow to
> build their virtualized testing environments?
>
> I think the configurations are correct. An answer to the other questions
> would help me to get a bigger picture on the issue and narrow down the
> problem.
>
>
>
> On Thursday, February 8, 2018 at 6:55:41 PM UTC+1, Antony Stone wrote:
>>
>> On Thursday 08 February 2018 at 18:21:43, Alvaro Miranda Aguilera wrote:
>>
>> > hello
>> >
>> > how do people run a setup of vagrant + virtual box + jenkins in the
>> cloud?
>> >
>> > they don't
>>
>> Is that because it doesn't work, or because it's just not a useful idea?
>>
>> > what do you want to achieve that vagrant + virtual box + jenkins in the
>> > cloud is the answer ?
>>
>> I got the impression that the original question was simply "how to run
>> Jenkins
>> in Virtualbox on a KVM guest machine".  The "cloud" aspect was simply
>> introduced as an example where people would have to be running Virtualbox
>> inside an already-virtualised server.  I don't think Thomas intends to
>> actually run things that way, he was just giving context to his question.
>>
>>
>> Antony.
>>
>> > On Thu, Feb 8, 2018 at 12:15 PM, Thomas Baumann wrote:
>> > > Recently, I tried virtualising a jenkins instance in my company,
>> meaning
>> > > jenkins should go into some KVM domain. Well, jenkins itself ran just
>> > > fine under these conditions, but virtual box did not perform very
>> well.
>> > > We made sure all nested virtualisation options were set correctly.
>> The
>> > > jenkins guest is able to use processor-flags vmx, ept and
>> > > vpid. /sys/module/kvm_intel/parameters/nested states nested paging
>> is
>> > > enabled with an 'Y'.
>> > >
>> > > I asked at the virtual box forums for experiences running virtual box
>> > > inside a KVM domain, but they could not really give me a hint.
>> > >
>> > > Anyway, I meanwhile accept answers like "nested virtualisation does
>> not
>> > > perform very well". We just gave it a try. Unless anybody here tells
>> me
>> > > he has managed to run VB inside KVM with good performance results,
>> that
>> > > is fine, we accept that.
>> > >
>> > > The only thing I am still wondering about is, how do people run a
>> setup
>> > > of vagrant + virtual box + jenkins in the cloud? Cloud servers are
>> > > always virtualised, so we have nested virtualisation there, too.
>> > >
>> > > Can anybody confirm vagrant + virtual box + jenkins is running fine
>> in
>> > > the cloud? Let's say with an Amazon EC2 instance? The guys from the
>> > > virtual box forum sent me here, as they 

Re: [vagrant-up] Any experience how vagrant + virtual box + jenkins perform in the cloud?

[Thomas Baumann]

No no it is different. Jenkins runs inside a KVM guest. Jenkins uses 
vagrant + virtual box to build the testing environment in which the actual 
application tests run. The latter is common practice.

In deed this setup does not perform very well. Virtual box instances can be 
set up, they do run with multiple cores, but performance is weak.

Now let's assume virtual box does really not like to be run inside a guest, 
and given that using vagrant + virtual box to build the testing 
environments is common practice with Jenkins, I wonder how people run such 
a setup in cloud environments, then. They will not have the KVM part, but 
as soon as they use EC2 instances, they are actually running Jenkins inside 
a virtualized environment. Don't they? AFAIK amazon uses Xen, which is a 
type 1 hypervisor. KVM on the other hand is considered to be a type 2 
hypervisor,

So why is it my setup does not play along nicely? Is it the specific KVM 
and virtual box configurations, that might be wrong or does virtual box 
just not like to be virtualized? Does it just not like type 2 hypervisors 
or no hypervisors in general? If people really do not do it the 
vagrant+virtual box way in the cloud, what other strategy do they follow to 
build their virtualized testing environments?

I think the configurations are correct. An answer to the other questions 
would help me to get a bigger picture on the issue and narrow down the 
problem.


On Thursday, February 8, 2018 at 6:55:41 PM UTC+1, Antony Stone wrote:
>
> On Thursday 08 February 2018 at 18:21:43, Alvaro Miranda Aguilera wrote: 
>
> > hello 
> > 
> > how do people run a setup of vagrant + virtual box + jenkins in the 
> cloud? 
> > 
> > they don't 
>
> Is that because it doesn't work, or because it's just not a useful idea? 
>
> > what do you want to achieve that vagrant + virtual box + jenkins in the 
> > cloud is the answer ? 
>
> I got the impression that the original question was simply "how to run 
> Jenkins 
> in Virtualbox on a KVM guest machine".  The "cloud" aspect was simply 
> introduced as an example where people would have to be running Virtualbox 
> inside an already-virtualised server.  I don't think Thomas intends to 
> actually run things that way, he was just giving context to his question. 
>
>
> Antony. 
>
> > On Thu, Feb 8, 2018 at 12:15 PM, Thomas Baumann wrote: 
> > > Recently, I tried virtualising a jenkins instance in my company, 
> meaning 
> > > jenkins should go into some KVM domain. Well, jenkins itself ran just 
> > > fine under these conditions, but virtual box did not perform very 
> well. 
> > > We made sure all nested virtualisation options were set correctly. The 
> > > jenkins guest is able to use processor-flags vmx, ept and 
> > > vpid. /sys/module/kvm_intel/parameters/nested states nested paging is 
> > > enabled with an 'Y'. 
> > > 
> > > I asked at the virtual box forums for experiences running virtual box 
> > > inside a KVM domain, but they could not really give me a hint. 
> > > 
> > > Anyway, I meanwhile accept answers like "nested virtualisation does 
> not 
> > > perform very well". We just gave it a try. Unless anybody here tells 
> me 
> > > he has managed to run VB inside KVM with good performance results, 
> that 
> > > is fine, we accept that. 
> > > 
> > > The only thing I am still wondering about is, how do people run a 
> setup 
> > > of vagrant + virtual box + jenkins in the cloud? Cloud servers are 
> > > always virtualised, so we have nested virtualisation there, too. 
> > > 
> > > Can anybody confirm vagrant + virtual box + jenkins is running fine in 
> > > the cloud? Let's say with an Amazon EC2 instance? The guys from the 
> > > virtual box forum sent me here, as they actually do not use virtual 
> box 
> > > in CI environments. So they cannot say. 
> > > 
> > > Any experiences? 
> > > 
> > > Thanks! 
>
> -- 
> I conclude that there are two ways of constructing a software design: One 
> way 
> is to make it so simple that there are _obviously_ no deficiencies, and 
> the 
> other way is to make it so complicated that there are no _obvious_ 
> deficiencies. 
>
>  - C A R Hoare 
>
>Please reply to the 
> list; 
>  please *don't* CC 
> me. 
>

-- 
This mailing list is governed under the HashiCorp Community Guidelines - 
https://www.hashicorp.com/community-guidelines.html. Behavior in violation of 
those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
--- 
You received this message because you are subscribed to the Google Groups 
"Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vagrant-up+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/vagrant-up/4f8083bd-ca46-46eb-afdb-693285222c33%40googlegroups.com.
F

Re: [vagrant-up] Any experience how vagrant + virtual box + jenkins perform in the cloud?

[Antony Stone]

On Thursday 08 February 2018 at 18:21:43, Alvaro Miranda Aguilera wrote:

> hello
> 
> how do people run a setup of vagrant + virtual box + jenkins in the cloud?
> 
> they don't

Is that because it doesn't work, or because it's just not a useful idea?

> what do you want to achieve that vagrant + virtual box + jenkins in the
> cloud is the answer ?

I got the impression that the original question was simply "how to run Jenkins 
in Virtualbox on a KVM guest machine".  The "cloud" aspect was simply 
introduced as an example where people would have to be running Virtualbox 
inside an already-virtualised server.  I don't think Thomas intends to 
actually run things that way, he was just giving context to his question.


Antony.

> On Thu, Feb 8, 2018 at 12:15 PM, Thomas Baumann wrote:
> > Recently, I tried virtualising a jenkins instance in my company, meaning
> > jenkins should go into some KVM domain. Well, jenkins itself ran just
> > fine under these conditions, but virtual box did not perform very well.
> > We made sure all nested virtualisation options were set correctly. The
> > jenkins guest is able to use processor-flags vmx, ept and
> > vpid. /sys/module/kvm_intel/parameters/nested states nested paging is
> > enabled with an 'Y'.
> > 
> > I asked at the virtual box forums for experiences running virtual box
> > inside a KVM domain, but they could not really give me a hint.
> > 
> > Anyway, I meanwhile accept answers like "nested virtualisation does not
> > perform very well". We just gave it a try. Unless anybody here tells me
> > he has managed to run VB inside KVM with good performance results, that
> > is fine, we accept that.
> > 
> > The only thing I am still wondering about is, how do people run a setup
> > of vagrant + virtual box + jenkins in the cloud? Cloud servers are
> > always virtualised, so we have nested virtualisation there, too.
> > 
> > Can anybody confirm vagrant + virtual box + jenkins is running fine in
> > the cloud? Let's say with an Amazon EC2 instance? The guys from the
> > virtual box forum sent me here, as they actually do not use virtual box
> > in CI environments. So they cannot say.
> > 
> > Any experiences?
> > 
> > Thanks!

-- 
I conclude that there are two ways of constructing a software design: One way 
is to make it so simple that there are _obviously_ no deficiencies, and the 
other way is to make it so complicated that there are no _obvious_ 
deficiencies.

 - C A R Hoare

   Please reply to the list;
 please *don't* CC me.

Re: [vagrant-up] Any experience how vagrant + virtual box + jenkins perform in the cloud?

[Alvaro Miranda Aguilera]

hello

how do people run a setup of vagrant + virtual box + jenkins in the cloud?

they don't


what do you want to achieve that vagrant + virtual box + jenkins in the
cloud is the answer ?



On Thu, Feb 8, 2018 at 12:15 PM, Thomas Baumann 
wrote:

> Recently, I tried virtualising a jenkins instance in my company, meaning
> jenkins should go into some KVM domain. Well, jenkins itself ran just fine
> under these conditions, but virtual box did not perform very well. We made
> sure all nested virtualisation options were set correctly. The jenkins
> guest is able to use processor-flags vmx, ept and
> vpid. /sys/module/kvm_intel/parameters/nested states nested paging is
> enabled with an 'Y'.
>
> I asked at the virtual box forums for experiences running virtual box
> inside a KVM domain, but they could not really give me a hint.
>
> Anyway, I meanwhile accept answers like "nested virtualisation does not
> perform very well". We just gave it a try. Unless anybody here tells me he
> has managed to run VB inside KVM with good performance results, that is
> fine, we accept that.
>
> The only thing I am still wondering about is, how do people run a setup of
> vagrant + virtual box + jenkins in the cloud? Cloud servers are always
> virtualised, so we have nested virtualisation there, too.
>
> Can anybody confirm vagrant + virtual box + jenkins is running fine in the
> cloud? Let's say with an Amazon EC2 instance? The guys from the virtual box
> forum sent me here, as they actually do not use virtual box in CI
> environments. So they cannot say.
>
> Any experiences?
>
> Thanks!
>
> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in
> violation of those guidelines may result in your removal from this mailing
> list.
>
> GitHub Issues: https://github.com/mitchellh/vagrant/issues
> IRC: #vagrant on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Vagrant" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vagrant-up+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/vagrant-up/cd0e6653-6adf-4ba1-ad75-3588c121951e%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Alvaro

-- 
This mailing list is governed under the HashiCorp Community Guidelines - 
https://www.hashicorp.com/community-guidelines.html. Behavior in violation of 
those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
--- 
You received this message because you are subscribed to the Google Groups 
"Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to vagrant-up+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/vagrant-up/CAHqq0ew8PpEsfKTV5wGF_3JAerNcO12tA5_7isz60ryk8ZkN7g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.