Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
n Wed, May 3, 2017 at 10:26 AM, Otto Fowler <ottobackwa...@gmail.com> > wrote: > > https://travis-ci.org/ottobackwards/incubator-metron/builds/228364894?utm_ > source=email_medium=notification > > On May 3, 2017 at 11:12:15, Justin Leet (justinjl...@gmail.com) wrote: &

[DISCUSS] platform_info.sh and PR template / Dev Guidelines

2017-05-03 Thread Otto Fowler
I think we should have documented in the template and guidelines something to the extent ( appropriate for the document ) of : “Metron provides a platform_info.sh script in metron-deployment/scripts that outputs information that may be important to troubleshooting build and deployment issues. It

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Otto Fowler
portMasterNodeAction.java#L148 > which suggests that a cluster action has been requested on a local > (loopback) address. This is not > surprising given what I’ve learned about the semantics of network.host > with wildcard address. > See next message, item C. Basically, while the wi

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-02 Thread Otto Fowler
community members and a suggestion in the old 1.x documentation for ES are correct. The 2.x documentation (we specify ES 2.3) doesn’t mention “0.0.0.0”, but I think it’s likely to still work, but it needs testing. Thanks, --Matt From: Otto Fowler <ottobackwa...@gmail.com> Date: Tuesday, May 2

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-02 Thread Otto Fowler
Thanks! On May 2, 2017 at 14:58:37, Nick Allen (n...@nickallen.org) wrote: I don't know how to uninstall, but you can reinstall by passing the --force flag On Tue, May 2, 2017 at 2:33 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Do you know how to uninstall an mpack fro

Re: introduction

2017-05-02 Thread Otto Fowler
Welcome! On May 2, 2017 at 07:55:58, Christian Tramnitz (tramn...@trasec.de) wrote: Hello Metron developers, I thought I’d first introduce myself before I ask any questions: I’m new to Metron, but we are running a fairly complex OpenSOC-based installation for a customer. Since Metron evolved

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
Do we have any async methods in the current api I could look at? On April 27, 2017 at 10:45:09, Otto Fowler (ottobackwa...@gmail.com) wrote: I thought as much. I have not worked with the framework before however. On April 27, 2017 at 10:20:20, Ryan Merriman (merrim...@gmail.com) wrote: I

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
(HDFS read/write, Zookeeper read/write). Asynchronous functions are also supported so no issues there. On Thu, Apr 27, 2017 at 9:09 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Also, if you want to help that would be great ;) > > > On April 27, 2017 at 09:30:59, Otto

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
-> configurations to ‘available area’ ( ES, ParserConfig, ) UI -> Create Parser Instance -> take available, clone, edit save/deploy for XXX On April 27, 2017 at 10:28:18, Otto Fowler (ottobackwa...@gmail.com) wrote: Maybe the templates can be installed somewhere, not ES but some store. The UI

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
Maybe the templates can be installed somewhere, not ES but some store. The UI can list the available templates, and you can edit and deploy from the UI? They can still be packaged with the parser, just not go to ES. Does that address your concerns? On April 27, 2017 at 10:25:37, Otto Fowler

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
to think about is whether this belongs in the metron-rest package (personally i think it probably does) or whether it is a separate micro-service. Anyway, happy to help in any way I can on this. Simon > On 27 Apr 2017, at 15:09, Otto Fowler <ottobackwa...@gmail.com> wrote: > > Al

Re: [DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
Also, if you want to help that would be great ;) On April 27, 2017 at 09:30:59, Otto Fowler (ottobackwa...@gmail.com) wrote: So, assuming ( I know I know ) that METRON-777 eventually lands, it will have lain the framework for the extension system for parsers out, including the capability

[DISCUSS] Metron Rest to Install Parser (METRON-258)

2017-04-27 Thread Otto Fowler
So, assuming ( I know I know ) that METRON-777 eventually lands, it will have lain the framework for the extension system for parsers out, including the capability to create parsers outside the metron tree. The next step is METRON-258, side loading of parsers. This will be the effort to

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
. On Wed, Apr 26, 2017 at 4:48 PM, Nick Allen <n...@nickallen.org> wrote: > Yes, Otto. I just experienced that myself. It is a bug that we should > create a JIRA for. > > I was able to work around it by just using "start" instead of "restart". > >

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
would file a Jira. On Wed, Apr 26, 2017 at 1:41 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Ok, I did ifconfig, i mean ip a and got the right values > that and the brackets and it looks like es is up. > > Now my next problem is prob. a bug. > > Ambari thinks it

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
file. network.host: ["_lo:ipv4_","_eth0:ipv4_"] On Wed, Apr 26, 2017 at 3:09 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > I think it is literally crashing trying to load the config file > > > > On April 26, 2017 at 15:00:25, Michael Miklavc

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
Actually never mind. I am not sure how to read the exception. Is it failing to read the configuration? On April 26, 2017 at 14:01:26, Otto Fowler (ottobackwa...@gmail.com) wrote: So the elasticsearch service is disabled. That is the issue. When ES calls get status ( service elasticsearch

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
So the elasticsearch service is disabled. That is the issue. When ES calls get status ( service elasticsearch status ) it fails. I don’t know systemd too well, but does that mean we didn’t do the equiv of chkconfig on? On April 26, 2017 at 13:54:10, Otto Fowler (ottobackwa...@gmail.com) wrote

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
Getting the same error in ES as before from what I can see On April 26, 2017 at 13:47:09, Otto Fowler (ottobackwa...@gmail.com) wrote: I still can’t keep ES running, it dies. Also, I’m getting exceptions from enrichment trying to restart when the topology wasn’t running. Is there some

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
...@gmail.com (zeo...@gmail.com) wrote: I'm also interested to know why that's important at such a small scale. Jon On Wed, Apr 26, 2017, 10:51 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > I am following > > https://community.hortonworks.com/articles/60805/deploying-a-fresh

Re: Reliable place to get latest release info for script

2017-04-26 Thread Otto Fowler
that passes data through parsers/enrichments classes and stores in Hive. So what you suggesting is to have internal repo with all our changes alone and to merge it with Apache Metron whenever we need to get full build? - Dima On 04/26/2017 09:09 PM, Otto Fowler wrote: Thanks! Parsers

Re: Quick Dev - Atlas Images

2017-04-26 Thread Otto Fowler
s? - How does the "time to start" compare for a "canned" release image as Dave suggested? On Tue, Apr 25, 2017 at 4:09 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > If it is docker, then it is just the Dockerfiles. > I think you can have vagrant use

Re: Reliable place to get latest release info for script

2017-04-26 Thread Otto Fowler
them to the decent level currently). We also merged our custom tools that extends Metron batching processing and store data in HDFS and Hive to be used by tableau, zeppelin, etc. Is that answers your question? Let me know if you want to know more. - Dima On 04/26/2017 08:29 PM, Otto Fowler wrote

Re: Reliable place to get latest release info for script

2017-04-26 Thread Otto Fowler
Can I ask Dima, as comfortable you are in describing, what areas do you make changes or additions to that require you to maintain a fork? Custom parsers? Enrichments? Other? I have done some work, and have more planned and lined up to try to eliminate the requirement to develop in the metron

Re: auto-install on bare metal

2017-04-26 Thread Otto Fowler
otally misunderstood what you're looking for. > Sorry. > > > > On Wed, Apr 26, 2017 at 9:59 AM, Otto Fowler <ottobackwa...@gmail.com> > > wrote: > > > >> Right, I think this : https://cwiki.apache.org/ > confluence/pages/viewpage. > >> action?pageId=6514

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
, 2017 at 7:02 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > File > "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line > 293, in _call > > raise ExecutionFailed(err_msg, code, out, err) > > ExecutionFailed: Execution of 'ser

Re: auto-install on bare metal

2017-04-26 Thread Otto Fowler
do that. I created this quite a while ago, but it shows you the structure and how you could manage multiple environments with this method. https://github.com/nickwallen/metron-environments On Tue, Apr 25, 2017 at 9:46 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > I failed at t

Re: Ambari Wizard: Repo Tab

2017-04-26 Thread Otto Fowler
Can you describe what ES issues you are working on? Fixing the repos got everything installed, but my ES components don’t stay running. I need to harvest the errors. On April 25, 2017 at 16:46:00, Otto Fowler (ottobackwa...@gmail.com) wrote: Nm. sorry. I fixed it. On April 25, 2017 at 16

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-26 Thread Otto Fowler
in the outbox Casey? > > Jon > > On Wed, Apr 26, 2017, 6:53 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > > > What other thread? > > > > > > On April 25, 2017 at 19:56:56, Casey Stella (ceste...@gmail.com) wrote: > > > > Ok, I spun up that disc

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
http://stackoverflow.com/questions/22150209/maven-changes-the-order-of-plugins-of-different-profiles https://issues.apache.org/jira/browse/MNG-2258 On April 25, 2017 at 22:33:58, Otto Fowler (ottobackwa...@gmail.com) wrote: So, the copy resources configuration is still there, and not governed

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
So, the copy resources configuration is still there, and not governed by the profile. Is it possible that since they both run in package, that they are out of order? On April 25, 2017 at 22:31:27, Otto Fowler (ottobackwa...@gmail.com) wrote: Having said that, i see the problem. I am very sorry

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
Having said that, i see the problem. I am very sorry How do I go about resubmitting if I can fix it? On April 25, 2017 at 22:29:56, Otto Fowler (ottobackwa...@gmail.com) wrote: OK, I’ll take a look. Sorry - I thought they read out of the RPMS dir and verified that they were produced both

Re: auto-install on bare metal

2017-04-25 Thread Otto Fowler
I failed at this today, but maybe it was the way I tried. An example would be great. On April 25, 2017 at 20:11:26, David Lyle (dlyle65...@gmail.com) wrote: Hi Dima, The same Ansible playbooks that work for EC2 and Vagrant will work for bare metal installations. The only difference is that

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
Nm. sorry. I fixed it. On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwa...@gmail.com) wrote: Ok, now I see the repos in the ‘pick version’ screen, but it is erring on the f://localrepo even though the folder exists, there is no repodata/repomd.xml. What is the command to create

Re: Quick Dev - Atlas Images

2017-04-25 Thread Otto Fowler
If it is docker, then it is just the Dockerfiles. I think you can have vagrant use docker as a back end too right? On April 25, 2017 at 14:34:14, Nick Allen (n...@nickallen.org) wrote: >> I hadn't really reasoned about the notion of a "released" Quick Dev image, but I can see a lot of value in

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
, Apr 25, 2017 at 3:44 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > 2.4.1 as a matter of fact > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65...@gmail.com) wrote: > > Any chance they're running Ambari < 2.4.2? > > -D... > > > On Tue,

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
kaging/ambari/metron-mpack/src/main/ > resources/addon-services/ELASTICSEARCH/2.3.3/repos/repoinfo.xml > > > On Tue, Apr 25, 2017 at 12:16 PM, Otto Fowler <ottobackwa...@gmail.com> > wrote: > > > stderr: > > > > Traceback (most recent call last): > >

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
17-04-25 14:13:03,429 <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Skipping installation of existing package curl 2017-04-25 14:13:03,430 <http://airmail.calendar/2017-04-25%2014:13:03%20EDT> - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
PR probably would have done so. Can you elaborate on "prior capability wrt centos 6.x work again" a bit? On Apr 25, 2017 10:59 AM, "Otto Fowler" <ottobackwa...@gmail.com> wrote: > Also, I can’t find any list discussion of the issue or other approaches. > > &g

Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
Hi, Should I see the repo tab in the ambari wizard for metron at this point? Asking for a friend who can’t get the wizard to work :)

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
Also, I can’t find any list discussion of the issue or other approaches. On April 25, 2017 at 12:48:46, Otto Fowler (ottobackwa...@gmail.com) wrote: What I am looking for is the prior capability wrt centos 6.x work again ( which I think my pr fixing the docker in docker issue would do

Re: metron_full_install playbook with small_cluster and multiple hosts

2017-04-25 Thread Otto Fowler
deployments, I only saw one build being executed. I don't know what you mean by "set to become", Become is set to false because I didn't want to sudo to execute the build. -D... On Tue, Apr 25, 2017 at 10:51 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Now that I

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
, or alternatively create some Ansible roles to make the REST calls that Ambari requires to stand up a cluster. -D... [1] https://cwiki.apache.org/confluence/display/AMBARI/Blueprints On Tue, Apr 25, 2017 at 11:35 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Let m

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
in my Metron cluster, I must run on RHEL 7. I assume that will be common across many enterprises. Semi-recentlyI took a stab at CentOS 7 support but it was a bit of a rough go and I dropped it down on my priority list. Jon On Tue, Apr 25, 2017 at 11:28 AM Otto Fowler <ottobackwa...@gmail.com>

Status of METRON-153

2017-04-25 Thread Otto Fowler
https://issues.apache.org/jira/browse/METRON-153 What is the other approach that is mentioned here? Was it implemented? The ansible build still does not support this and I was thinking of looking at it.

Re: Ansible Docker fails to build latest

2017-04-21 Thread Otto Fowler
OK, the issue is the old old maven that is in the ansible image. Changing it to 3.3.9 resolved. On April 21, 2017 at 17:21:02, Otto Fowler (ottobackwa...@gmail.com) wrote: Yes. I am following the readme in there and starting the image, then just running mvn package -DskipTests On April 21

Re: Reducing Warnings in Build

2017-04-21 Thread Otto Fowler
Until there is a use case supporting changing to other then the 80% plus case, this should not be changed. or a customer requirement etc. This would need some design and discussion time as well to map out all the implications for the complete pipeline. I think configurable charset should be

Re: [GitHub] incubator-metron issue #538: METRON-868 Fix documentation on building RPMs

2017-04-20 Thread Otto Fowler
Maybe it is the relative parent path? On April 20, 2017 at 20:05:31, mmiklavc (g...@git.apache.org) wrote: Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/538 Further, the reason that install worked is not because it needs the artifacts in the

Re: Failing build

2017-04-20 Thread Otto Fowler
Maybe it is a container thing? I don’t think my PR for extensions has the problem, or has had it in my travis - and it is a vm in travis. On April 20, 2017 at 09:06:21, Casey Stella (ceste...@gmail.com) wrote: oh man, yeah, it's pissing me off too. I'm still looking at ways to get around

<    1   2   3   4   5   6