GitHub user nickwallen opened a pull request:

    https://github.com/apache/metron/pull/903

    METRON-1370 Create Full Dev Equivalent for Ubuntu

    
    
    This PR adds a single node, VM based deployment mechanism based on Ubuntu 
Trusty.  This has feature parity with the existing CentOS single node VM.  This 
will help us continue to improve and support the Ubuntu-based DEB packages that 
were added in #868.
    
    ### Changes
    
    * The Metron, Elasticsearch, and Kibana MPacks were updated to support 
installation on Ubuntu.  This included making the following changes.
    
        * Adding repo definitions for Ubuntu
        * Changing the default location of the `/etc/sysconfig/elasticsearch` 
configuration to work across platforms
        * Adding an additional configuration in the MPack to allow 
Elasticsearch to perform a memory lock on Ubuntu
        * Support configuration of Elasticsearch on a platform running Systemd; 
like Ubuntu 
    
    * The Elasticsearch MPack was enhanced to correctly perform a service 
check.  Previously the hostname and port were hard coded and the service check 
was not functional in most environments.
    
    * The Ubuntu DEBs were enhanced to allow for the addition of "maintainer 
scripts".  This allows the Management UI and Alerts UI service scripts to be 
installed with the DEBs.  This was needed to reach feature parity with the RPMs.
    
    * The name `full-dev-platform` no longer makes sense IMHO.  We do not have 
a `quick-dev-platform` any longer to distinguish it from.  Now the main 
distinguisher is the underlying operating system.  I renamed our "Full Dev" 
development environment to `metron-on-centos` and `metron-on-ubuntu`.  I am 
completely open to community suggestions on what these should be named.
    
        ```
        metron-deployment/vagrant/
        ├── README.md
        ├── fastcapa
        ├── metron-on-centos
        └── metron-on-ubuntu
        ```
        
    * The organization of `metron-deployment` had to change so that assets 
could be reused across both the CentOS and Ubuntu deployments.  All shared 
Ansible assets were moved to `metron-deployment/ansible`.  This includes the 
Ansible modules, playbooks and roles that are all used across multiple 
environments.
    
        ```
        metron-deployment/ansible/
        ├── extra_modules
        ├── playbooks
        └── roles
        ```
    
    * The following Ansible roles had to be updated to support both CentOS and 
Ubuntu based on small differences in package names and conventions.
    
        * `ambari-common` 
        * `ambari-config`
        * `ambari-master`
        * `ambari-slave`
        * `libselinux-python`
        * `ntp`
    
    * A new role was added to enable swap space; `enable-swap`.  This is 
required for the Ubuntu deployment as the underlying Ubuntu Trusty image does 
not have swap space enabled by default.
    
    * The `metron-builder` role was changed to selectively build either the 
DEBs or the RPMs as needed.
    
    * The `metron-rpms` role was renamed to `metron-packages` and was also 
enhanced to create the local repository on a Ubuntu host.  This also 
selectively copies either RPMs or DEBs to the VM as needed.
    
    * Monit is no longer needed and so has been removed from all VM 
deployments.  Monit was added prior to the existence of our Mpack installer.  
It is no longer needed
    
    * Removed the `metron-streaming` role which is no longer applicable since 
these functions are now performed by the MPack.
    
    ### Testing
    
    I have performed the following testing based on these changes.
    
    - [ ] Run Metron on CentOS.  
    
        ```
        cd metron-deployment/vagrant/metron-on-centos
        vagrant up
        ```
    
        * Validate that alerts are visible within the Alerts UI.
        * Validate that the Metron Service Check completes successfully.
        * Validate that the Elasticsearch service check completes successfully.
    
        Be sure to shutdown and halt or destroy this VM before moving on to the 
next step.
    
    - [ ] Run Metron on Ubuntu.
    
        ```
        cd metron-deployment/vagrant/metron-on-centos
        vagrant up
        ```
    
        * Validate that alerts are visible within the Alerts UI.
        * Validate that the Metron Service Check completes successfully.
        * Validate that the Elasticsearch service check completes successfully.
    
        Be sure to shutdown and halt or destroy this VM before moving on to the 
next step.
    
    - [ ] Run one of the Fastcapa test environments.
    
        ```
        cd metron-deployment/vagrant/fastcapa/centos-7.1/
        vagrant up
        ```
    
        * If the process fails at the task "fastcapa : Restart for modified 
kernel params" simply run `vagrant provision` again.
        * Ensure that the deployment process reports success.  Automated 
validation is performed as part of the deployment process.
    
    - [ ] Deploy Metron to AWS EC2.
    
        ```
        cd metron-deployment/amazon-ec2
        ./run.sh
        ```
    
        * Validate that alerts are visible within the Alerts UI.
        * Validate that the Metron Service Check completes successfully.
        * Validate that the Elasticsearch service check completes successfully.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/nickwallen/metron METRON-1370-PR

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/metron/pull/903.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #903
    
----
commit 5cb44aadece20b185d552c9e869fec5e01eff77b
Author: Nick Allen <nick@...>
Date:   2018-01-03T17:54:04Z

    METRON-1370 Create Full Dev Equivalent for Ubuntu

----


---

Reply via email to