Re: Regarding local environment based on Vagrant

[Juan P]

+1

On 4 April 2018 at 09:10, Javi Roman  wrote:

> I going to merge the PR [1] related with the Vagrant environment reworking.
>
> Please feel free to comment whatever enhance or inconvenient you think
> about the commit. I will merge he PR if there are no negative
> comments/votes.
>
>
> [1] https://github.com/apache/incubator-myriad/pull/107
> --
> Javi Roman
>
> Twitter: @javiromanrh
> GitHub: github.com/javiroman
> Linkedin: es.linkedin.com/in/javiroman
> Big Data Blog: dataintensive.info
>
>
> On Mon, Feb 5, 2018 at 2:36 PM, Javi Roman 
> wrote:
> > For sure, this is something to include in the vagrant testbed. Good
> point.
> > --
> > Javi Roman
> >
> > Twitter: @javiromanrh
> > GitHub: github.com/javiroman
> > Linkedin: es.linkedin.com/in/javiroman
> > Big Data Blog: dataintensive.info
> >
> >
> > On Mon, Feb 5, 2018 at 10:01 AM, Juan P  wrote:
> >> Hello Javi, I think it's a great idea.
> >>
> >> Is it possible to prepare the automatic environment to perform automatic
> >> end-to-end tests (acceptance) or with a minimum of configuration?
> >>
> >> In later versions, would it be possible to include all the known Yarn
> >> frameworks and prepare some automatic use cases that serve as an example
> >> for the community and regression / integration test?
> >>
> >> Thank you
> >>
> >> On 4 February 2018 at 10:09, Javi Roman 
> wrote:
> >>
> >>> I would like share with you the strategy that I'm following for a
> >>> powerful local development environment for Myriad.
> >>>
> >>> The idea is to create an easy-to-use environment for new contributors,
> >>> setting up a Mesos cluster, or DCOS cluster (with hadoop running which
> >>> is a tedious task). I guess automate this task is a key point for
> >>> accelerating the development and focusing in the Myriad hacking.
> >>>
> >>> In this first task [1] I am working on Vagrant libvirt support for a
> >>> plain Mesos cluster. However the roadmap for this environment would
> >>> be:
> >>>
> >>> 1. vagrant-libvirt based cluster for Mesos, provisioned with Ansible
> >>> (already working in my environment).
> >>> 2. vagrant-libvirt based cluster for DC/OS provisioned with Ansible:
> >>> Easier environment for testing Myriad with Marathon, Mesos-DNS, VIPS,
> >>>  and so on.
> >>> 3. vagrant-virtualbox based cluster (maintain the original environment
> >>> used right now).
> >>>
> >>> The layout would be something like this:
> >>>
> >>> vagrant/
> >>> ├── libvirt
> >>> │   ├── dcos
> >>> │   │   └── Vagrantfile
> >>> │   └── mesos
> >>> │   └── Vagrantfile
> >>> └── virtualbox
> >>> └── Vagrantfile
> >>>
> >>>
> >>> In a second stage, the final scenario would be, only one folder with
> >>> one "smart" Vagrantfile for supporting Mesos and DC/OS for Linux and
> >>> OSX, the tool will be able of selecting the suitable virtualization
> >>> technology by user configuration.
> >>>
> >>> This is the idea.
> >>>
> >>> [1] https://issues.apache.org/jira/browse/MYRIAD-257
> >>>
> >>> --
> >>> Javi Roman
> >>>
> >>> Twitter: @javiromanrh
> >>> GitHub: github.com/javiroman
> >>> Linkedin: es.linkedin.com/in/javiroman
> >>> Big Data Blog: dataintensive.info
> >>>
>

Re: Regarding local environment based on Vagrant

[Javi Roman]

I going to merge the PR [1] related with the Vagrant environment reworking.

Please feel free to comment whatever enhance or inconvenient you think
about the commit. I will merge he PR if there are no negative
comments/votes.


[1] https://github.com/apache/incubator-myriad/pull/107
--
Javi Roman

Twitter: @javiromanrh
GitHub: github.com/javiroman
Linkedin: es.linkedin.com/in/javiroman
Big Data Blog: dataintensive.info


On Mon, Feb 5, 2018 at 2:36 PM, Javi Roman  wrote:
> For sure, this is something to include in the vagrant testbed. Good point.
> --
> Javi Roman
>
> Twitter: @javiromanrh
> GitHub: github.com/javiroman
> Linkedin: es.linkedin.com/in/javiroman
> Big Data Blog: dataintensive.info
>
>
> On Mon, Feb 5, 2018 at 10:01 AM, Juan P  wrote:
>> Hello Javi, I think it's a great idea.
>>
>> Is it possible to prepare the automatic environment to perform automatic
>> end-to-end tests (acceptance) or with a minimum of configuration?
>>
>> In later versions, would it be possible to include all the known Yarn
>> frameworks and prepare some automatic use cases that serve as an example
>> for the community and regression / integration test?
>>
>> Thank you
>>
>> On 4 February 2018 at 10:09, Javi Roman  wrote:
>>
>>> I would like share with you the strategy that I'm following for a
>>> powerful local development environment for Myriad.
>>>
>>> The idea is to create an easy-to-use environment for new contributors,
>>> setting up a Mesos cluster, or DCOS cluster (with hadoop running which
>>> is a tedious task). I guess automate this task is a key point for
>>> accelerating the development and focusing in the Myriad hacking.
>>>
>>> In this first task [1] I am working on Vagrant libvirt support for a
>>> plain Mesos cluster. However the roadmap for this environment would
>>> be:
>>>
>>> 1. vagrant-libvirt based cluster for Mesos, provisioned with Ansible
>>> (already working in my environment).
>>> 2. vagrant-libvirt based cluster for DC/OS provisioned with Ansible:
>>> Easier environment for testing Myriad with Marathon, Mesos-DNS, VIPS,
>>>  and so on.
>>> 3. vagrant-virtualbox based cluster (maintain the original environment
>>> used right now).
>>>
>>> The layout would be something like this:
>>>
>>> vagrant/
>>> ├── libvirt
>>> │   ├── dcos
>>> │   │   └── Vagrantfile
>>> │   └── mesos
>>> │   └── Vagrantfile
>>> └── virtualbox
>>> └── Vagrantfile
>>>
>>>
>>> In a second stage, the final scenario would be, only one folder with
>>> one "smart" Vagrantfile for supporting Mesos and DC/OS for Linux and
>>> OSX, the tool will be able of selecting the suitable virtualization
>>> technology by user configuration.
>>>
>>> This is the idea.
>>>
>>> [1] https://issues.apache.org/jira/browse/MYRIAD-257
>>>
>>> --
>>> Javi Roman
>>>
>>> Twitter: @javiromanrh
>>> GitHub: github.com/javiroman
>>> Linkedin: es.linkedin.com/in/javiroman
>>> Big Data Blog: dataintensive.info
>>>

Re: Regarding local environment based on Vagrant

[Javi Roman]

For sure, this is something to include in the vagrant testbed. Good point.
--
Javi Roman

Twitter: @javiromanrh
GitHub: github.com/javiroman
Linkedin: es.linkedin.com/in/javiroman
Big Data Blog: dataintensive.info


On Mon, Feb 5, 2018 at 10:01 AM, Juan P  wrote:
> Hello Javi, I think it's a great idea.
>
> Is it possible to prepare the automatic environment to perform automatic
> end-to-end tests (acceptance) or with a minimum of configuration?
>
> In later versions, would it be possible to include all the known Yarn
> frameworks and prepare some automatic use cases that serve as an example
> for the community and regression / integration test?
>
> Thank you
>
> On 4 February 2018 at 10:09, Javi Roman  wrote:
>
>> I would like share with you the strategy that I'm following for a
>> powerful local development environment for Myriad.
>>
>> The idea is to create an easy-to-use environment for new contributors,
>> setting up a Mesos cluster, or DCOS cluster (with hadoop running which
>> is a tedious task). I guess automate this task is a key point for
>> accelerating the development and focusing in the Myriad hacking.
>>
>> In this first task [1] I am working on Vagrant libvirt support for a
>> plain Mesos cluster. However the roadmap for this environment would
>> be:
>>
>> 1. vagrant-libvirt based cluster for Mesos, provisioned with Ansible
>> (already working in my environment).
>> 2. vagrant-libvirt based cluster for DC/OS provisioned with Ansible:
>> Easier environment for testing Myriad with Marathon, Mesos-DNS, VIPS,
>>  and so on.
>> 3. vagrant-virtualbox based cluster (maintain the original environment
>> used right now).
>>
>> The layout would be something like this:
>>
>> vagrant/
>> ├── libvirt
>> │   ├── dcos
>> │   │   └── Vagrantfile
>> │   └── mesos
>> │   └── Vagrantfile
>> └── virtualbox
>> └── Vagrantfile
>>
>>
>> In a second stage, the final scenario would be, only one folder with
>> one "smart" Vagrantfile for supporting Mesos and DC/OS for Linux and
>> OSX, the tool will be able of selecting the suitable virtualization
>> technology by user configuration.
>>
>> This is the idea.
>>
>> [1] https://issues.apache.org/jira/browse/MYRIAD-257
>>
>> --
>> Javi Roman
>>
>> Twitter: @javiromanrh
>> GitHub: github.com/javiroman
>> Linkedin: es.linkedin.com/in/javiroman
>> Big Data Blog: dataintensive.info
>>

Re: Regarding local environment based on Vagrant

[Juan P]

Hello Javi, I think it's a great idea.

Is it possible to prepare the automatic environment to perform automatic
end-to-end tests (acceptance) or with a minimum of configuration?

In later versions, would it be possible to include all the known Yarn
frameworks and prepare some automatic use cases that serve as an example
for the community and regression / integration test?

Thank you

On 4 February 2018 at 10:09, Javi Roman  wrote:

> I would like share with you the strategy that I'm following for a
> powerful local development environment for Myriad.
>
> The idea is to create an easy-to-use environment for new contributors,
> setting up a Mesos cluster, or DCOS cluster (with hadoop running which
> is a tedious task). I guess automate this task is a key point for
> accelerating the development and focusing in the Myriad hacking.
>
> In this first task [1] I am working on Vagrant libvirt support for a
> plain Mesos cluster. However the roadmap for this environment would
> be:
>
> 1. vagrant-libvirt based cluster for Mesos, provisioned with Ansible
> (already working in my environment).
> 2. vagrant-libvirt based cluster for DC/OS provisioned with Ansible:
> Easier environment for testing Myriad with Marathon, Mesos-DNS, VIPS,
>  and so on.
> 3. vagrant-virtualbox based cluster (maintain the original environment
> used right now).
>
> The layout would be something like this:
>
> vagrant/
> ├── libvirt
> │   ├── dcos
> │   │   └── Vagrantfile
> │   └── mesos
> │   └── Vagrantfile
> └── virtualbox
> └── Vagrantfile
>
>
> In a second stage, the final scenario would be, only one folder with
> one "smart" Vagrantfile for supporting Mesos and DC/OS for Linux and
> OSX, the tool will be able of selecting the suitable virtualization
> technology by user configuration.
>
> This is the idea.
>
> [1] https://issues.apache.org/jira/browse/MYRIAD-257
>
> --
> Javi Roman
>
> Twitter: @javiromanrh
> GitHub: github.com/javiroman
> Linkedin: es.linkedin.com/in/javiroman
> Big Data Blog: dataintensive.info
>