Hi Bryan, For automation, B23 is planning to contribute Ansible scripts to deploy the Metron stack. The playbooks use Ambari blueprints for the hadoop ecosystem. We also install Elasticsearch, configure the legacy OpenSOC UI (based on Kibana/nodejs), create directories in hdfs, populate a MySQL database for geotagging. We template the OpenSOC_Config files so that we can use variable injection for the different services: Zookeeper, Hbase, Elasticsearch, MySQL, etc. Everyone's deployment might be slightly different, but I think this will be a really good start.
I'm in the process of decoupling the scripts from our internal tooling - I should be able to make available the Ambari stuff later this week. Once we merge the disparate forks of the Cisco codebase, there will be some work to bring the playbooks up to date (i.e., use of Storm Flux), but not a ton. Mark On 12/9/15, 2:48 PM, "Bryan Taylor" <[email protected]> wrote: > >Seems like a good list. I'm probably not your UI guy, but I'll be happy to >help with any of the others. Test coverage is probably a good place for me >to start learning my way around. Do we have CI set up? I see a travis file >in the code. > >Regarding automation deployments, are we targeting Ambari or something >else? > >On the hadoop component compatibility, I see from the opens-streaming pom >that we are using Storm-0.9.2, Kafka 0.8.0, Hadoop 2.2.0, and HBase >0.98.0-hadoop2. These are all several iterations old. How aggressive do we >want to be, generally, with tracking new releases? Are Hive, Flume, and >Spark also going to up rev for us? > >Bryan > >On 12/9/15 12:22 PM, "James Sirota" <[email protected]> wrote: > >>Hi Brian, >> >>Welcome. Glad to have you contribute. There will be changes to the code >>base that the community will contribute back shortly. We are waiting for >>the Jira to be setup so the backlog can be created and voted on. I think >>the overall feeling is that we need to make the code base compatible with >>the latest version of HDP, automate deployments, increase test coverage, >>and start working on a new UI. There may be more significant >>architectural changes to the code base, but we need to get the essential >>items knocked out before we go there. >> >>Thanks, >>James >> >> >> >> >>On 12/9/15, 11:52 AM, "Bryan Taylor" <[email protected]> wrote: >> >>>Hi Folks, >>> >>>I just joined the list and thought I'd say "hi". I work at Rackspace and >>>will be joining Andrew Hartnett's team there and hacking on metron. This >>>is my first ASF project and I'm looking forward to being part of this >>>community. >>> >>>I'm curious what the development vision is for metron. What do people >>>like and not like about the codebase? I gather this code transitioned >>>from a Cisco internal project and is incubating now at the ASF. Are >>>there any code changes that need to be made to support incubation? >>> >>>Bryan >
